"Back in November, there was, I think around Opus 4.5, pretty much all the developers started to realize that these models were actually getting pretty good and there's no longer, vibe coding was less of a joke and more like, you know, maybe this will work. And then by the time, you know, around Fe"
"So before we jump into what learned during these experiments and how effective local large language models actually are as coding assistants. Let's talk a bit about why we're having this discussion in the first place. And I understand that AI coding assistants are about to become way more expensive."
"This week on The Kettle, host Brandon Vigliarolo is joined by Mann and Claburn to discuss their work with locally-hosted LLMs, why we're revisiting the topic at all, how to do local LLMs safely, and whether there's orbital relief coming for the compute crunch."
"Welcome back to another episode of T he Register's Kettle podcast. I'm Reg reporter Brandon Vigliarolo and with me this week are systems editor Tobias Mann and senior reporter Tom Claburn to talk about some experiments they've been doing with AI coding assistants, but not just any AI coding assistant mind you, we're talking about local ones that live right on your own machine."
Locally hosted large language models can act as effective coding assistants, potentially reducing the compute load that drives up prices for cloud-hosted AI services. The work focuses on experiments with on-machine LLMs and revisiting the topic due to rising costs and changing capabilities of cloud models. Key concerns include how to run local LLMs safely, including practical considerations for deployment and risk management. The conversation also weighs whether local deployment can provide “orbital relief” from the compute crunch affecting AI providers and users. The goal is to understand performance, feasibility, and safety tradeoffs compared with cloud-based tools.
Read at theregister
Unable to calculate read time
Collection
[
|
...
]