How accurate is gender prediction from names? We tested it

Gender prediction from a first name is right about 97% of the time when it gives an answer, and it gives an answer for about 94% of names. That is what we measured on 4,610 labelled names from 25 countries: 4,198 were answered correctly, 91% of the whole set. The spread behind that average is wide. Latin-script names were answered correctly 96% of the time end to end; Greek 78%; Hebrew 64%; Thai, where we now answer 51% of the names, 49%.

Updated 7 October 2026 after Thai and Arabic given names from Wikidata were added: 88.6% end to end became 91.1%, Thai went from 2.5% to 49% and Arabic from 87% to 96.5%. The probability table was measured again on the same day.

Updated 28 September 2026 with a fresh run of the same benchmark, after corrected US counts, new national registers and global answers built from every counted register. The first version reported 88% answered and 84% end to end.

Those three numbers are the ones to ask any vendor for, including us:

A single "accuracy" figure without coverage next to it can mean almost anything. A service that answers only the easy half of a list can report 99%. How to compare gender APIs goes through this in detail.

Accuracy by writing system

The biggest single factor is the script a name is written in, because it decides whether a name can be matched directly, needs transliteration, or cannot be read at all.

Read the Greek row carefully: when it answers, it is right 96% of the time, but it answers only 81% of Greek names. That is a coverage problem, not an accuracy problem, and the fix is different. The script-level detail, including how Thai coverage changed, is in our coverage by writing system.

Is the probability field worth trusting?

Yes, in the sense that matters: the stated probability tracks how often the answer is right. We grouped every answered name in the latest run of the benchmark, on 7 October 2026, by the probability the API returned and checked how many were correct.

So a threshold works the way you would hope. If a use case cannot tolerate one wrong answer in eight, drop everything under 90 and treat it as unknown; you keep about 95% of the answers and the error rate falls to 2.1%. The guide to probability, confidence and sample_size explains how to combine the fields.

A probability of 100 is not a guarantee. It means the counted records for that name are overwhelmingly one gender, and 0.8% of those answers were still marked wrong in the test, partly because of the test itself (see below).

Where the errors come from

We looked at all 412 names that were not answered correctly.

Names with no data: 263 of the 412. Thai names that are not in our Wikidata-derived list (98), Hebrew spellings the Israeli register does not contain (68), and smaller gaps in Greek (39) and Vietnamese (22). These come back as unknown, which is the honest answer, but they still count as misses end to end.

Names that change gender by country. Jean is male in France and female in the United States; the answer depends on the country you pass. We counted 65 common names that do this in official birth records: the full list.

Genuinely ambiguous names. Robin is 68% female worldwide; Riley is 57% female in US records. No system can be certain about these, and one that returns them at 99% is overstating what it knows.

Transliteration misses. Arabic-script names are matched through a consonant skeleton, and a few come out wrong: هانى (Hani) was answered as female. These are real errors on our side.

Wrong labels in the test set. The labels come from public name lists in the Faker library, which we did not compile. They are independent, which is the point, but they are not perfect. Faker's US male list includes Dagmar and Skye; the US Social Security records list Dagmar as female in all 1,591 registrations and Skye as female in 89% of 26,495. Those rows count as errors for any vendor that agrees with the registration records, so every absolute score on this list is a little lower than the truth.

Names that are not in the dataset at all

The benchmark above measures names the way customers send them, and most real names are in the dataset in some form. A separate test measures the opposite case: we removed 2,000 names from the dataset entirely and asked for them anyway. The API answered only 9.5% of them, all by closest spelling, and 96.8% of those answers were correct. The other 1,810 came back as unknown.

That is the behaviour we want: when there is no evidence, say so. But it is also why that 96.8% figure should never be quoted on its own: it describes 190 answers out of 2,000 names, not 2,000 answers.

How to measure it on your own data

Averages across 25 countries tell you little about your list. The reliable test is small and takes an afternoon:

1. Take 200 to 500 rows from your own data where you already know the answer.
2. Send them with the country you would send in production.
3. Count coverage, accuracy when answered, and end to end separately, and slice by country.
4. Look at the name field of every wrong answer. If the name that was looked up is not the given name you expected, the problem is extraction, not classification.

If the result will feed a published study, the guidelines for gender inference in research cover thresholds, unknowns, bias checks and what to report in the methods section.

The numbers on this page come from php artisan lookup:benchmark (the 4,610-name benchmark, published at NameGender Research) and php artisan lookup:accuracy (the 2,000-name holdout test), with the method in how we measure. The figures are read from the reports those commands write: the benchmark was last run on 7 October 2026 and the holdout test on 28 September 2026. When the dataset changes, they are measured again and this page follows.