Apple has updated its AI models for its Apple Intelligence features, yet benchmarks show these models underperform compared to rivals from OpenAI, Google, and Alibaba. Human testers rated the quality of Apple's 'On-Device' model, which functions offline, as comparable to rival models but inferior to OpenAI's GPT-4o. Surprising results also showed that Apple's image analysis capabilities are outperformed by Meta’s Llama 4 Scout. These developments underscore the ongoing challenges within Apple's AI research division, which has faced delays and customer lawsuits over unmet marketing promises.
According to Apple, human testers rated its newest 'Apple On-Device' model's text quality as 'comparably' good as Google and Alibaba's, but performance lags behind OpenAI's GPT-4o.
The recent benchmark results revealed that Apple's AI research division may be struggling to compete as customers raised concerns about unmet AI feature marketing commitments.
Despite improvements in tool use and efficiency, Apple's AI capabilities, including the delayed Siri upgrade, have underperformed compared to rival models from leading tech firms.
With both Apple On-Device and Apple Server having approximately 3 billion parameters, they facilitate features like summarization and text analysis, but lag behind competitors.
Collection
[
|
...
]