Dafont.com offers a large array of fonts, and includes a forum where users seek help identifying unknown fonts. A live benchmark was set up to assess LLMs in identifying fonts from images not previously identified by the community, alleviating issues with benchmark contamination. Two specific LLMs, gpt-4o-mini and gemini-2.5-flash-preview-05-20, were evaluated with image uploads along with thread titles and descriptions for improved identification accuracy. Results showed diminished performance levels for the LLMs within this task.
Dafont.com hosts a vast collection of fonts, featuring a forum for users seeking unidentified fonts. It surpasses Google Fonts in comprehensiveness and variety.
I set up a live benchmark to evaluate LLMs on their ability to identify fonts from images that the community hasn't identified yet, ensuring no prior exposure.
Evaluating only two LLMs, gpt-4o-mini and gemini-2.5-flash-preview-05-20, I provided images, thread titles, and descriptions to assist in accurate font identification.
Performance is assessed using a top-k accuracy metric, which checks if the correct font appears within the LLM's first k guesses, allowing for leniency.
Collection
[
|
...
]