"The idea of automating tasks for desktop users is not entirely novel. Last year in October, Anthropic became the first LLM provider to showcase the possibility of controlling a computer or some parts of its operating system. That ability, which Anthropic had termed "computer use," enabled developers to instruct Claude 3.5 Sonnet, through the Anthropic API, to read and interpret what's on the display, type text, move the cursor, click buttons, and switch between windows or applications."
"That ability, which Anthropic had termed "computer use," enabled developers to instruct Claude 3.5 Sonnet, through the Anthropic API, to read and interpret what's on the display, type text, move the cursor, click buttons, and switch between windows or applications. It caught the attention of experts and enterprises as the ability was a major step up from more traditional automation practices, such as robotic process automation (RPA) tools, which required more time and labor to set up and yet would require constant maintenance."
Automating desktop tasks has been attempted before, but Anthropic demonstrated a new level of capability by enabling an LLM to control parts of an operating system. The feature, named "computer use," allowed developers to instruct Claude 3.5 Sonnet via the Anthropic API to read and interpret on-screen content, type text, move the cursor, click buttons, and switch between windows or applications. The capability attracted attention from experts and enterprises because it represented a meaningful advancement over traditional robotic process automation tools, which required extensive setup, more labor, and ongoing maintenance.
Read at Computerworld
Unable to calculate read time
Collection
[
|
...
]