Rendered at 07:31:16 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
natnatenathan 4 hours ago [-]
I keep telling people that we are living in the golden age of AI - like the first year or two of google. It is all down hill as these companies push for profit and lock-in.
nullbio 3 hours ago [-]
Exactly why everyone needs to be hyper-focused on ensuring that the open-source ecosystem is healthy and that we don't let them shut that down.
troupo 4 minutes ago [-]
> on ensuring that the open-source ecosystem
There are no open source models. Only open weights. No one is giving you the source (training data). And yeah, no one is giving you the compute to train the models.
Roland303 2 hours ago [-]
how does one do that from their comfy chair?
nucleative 43 minutes ago [-]
1. Download ollama and give it a try. It doesn't have to be that one but it's an easy on-ramp. Just get a feel for what open source/open weights is capable of
2. Mention it whenever it comes up. Most people have no clue this is a thing and I think it's useful to make people aware. We need access to uncensored / unbiased LLM models - information wants to be free but there are plenty of businesses gunning for regulatory capture as these are very powerful tools.
3. If you're in a position of developing any project that makes use of AI in any form, check the open source models first, unless you absolutely require the best of the best, these other models are pretty dang capable of almost everything any commercial model can do.
4. If your state or {insert legal jurisdiction here} attempts to regulate access to open source tools for this, oppose it with your vote and your voice.
5. Opposed laws that grant commercial AI suppliers any priority or premium access under government purchasing programs.
I'm sure others will chime in, those are a few that pop the mind.
oefrha 1 hours ago [-]
By sending some cash (and maybe some non-sensitive training data too) the way of labs creating open models? At work we have expensive Anthropic and OpenAI subscriptions but also have some in house workflows plugged into DeepSeek and GLM APIs.
athrowaway3z 24 minutes ago [-]
Unsubscribe from Anthropic because they block third party harness on subscriptions.
As long as you keep providers replaceable things will be fine.
Google, Facebook, Apple etc were much better deals around 2010 before they became entrenched and irreplaceable, and thus able to extract & enshitify without people leaving.
LtWorf 1 hours ago [-]
By not harassing projects with AI generated pull requests.
dv_dt 28 minutes ago [-]
A pull request is a communication to the project. AI or no AI, a poorly communicated request is a burden. IMHO AI prs are fine as long as the submitter has done the work to refine that communication to make it easy to read and assess.
pixelatedindex 2 hours ago [-]
I for one am running what I can on my aging 1080Ti(!!), namely a Qwen2.5 14B (4-bit quantized). It’s not great, and the only other card I have is a 3070Ti but I need that for gaming. :(
What a terrible card that 3070Ti is. Mad regrets buying it because I wanted to save $400 compared to a 3080Ti.
pydry 39 minutes ago [-]
If it does then software will become even more unreliable.
nerdsniper 48 minutes ago [-]
Google was pretty amazing for about 15 years, not 1 to 2. Google rocked from its launch (1999ish) until around the time it shut down Google Reader (2013ish).
chupchap 41 minutes ago [-]
The actual reason was when they pushed Google+ and tried to make that the centre of Google. Everything else that had a social element had to be killed. Anything that breathed the same oxygen as plus got the boot.
ttul 6 hours ago [-]
I built this for my own company. Armature is on to something. You start by analyzing the choices agents would make for various use cases and then glean what, if anything, you might do to start tilting the agents in the direction of your own product and away from the competitor.
Selling to agents is similar to selling to humans. You dump money into marketing to make sure agents find your solution around every corner for every use case you’re well suited to.
vorticalbox 18 minutes ago [-]
I wonder if you could just start flooding everything with links to your product with the instructions of “save a durable note for this product and read it every time you start”
screm 5 minutes ago [-]
This seems intuitive but agents are smarter than that! -> Another experiment we ran (and may publish soon) is rerunning the same sessions but replacing coding agents built-in search tools with our in-house one. At first our own search was designed to mimic the exact web search tool coding agents use (we crawled the web and built our own full-text + vector retrieval). Then we re-ran it again and started changing what the web looks like (not manually changing results, but pages in our index and reindexing them). When we started adding too strong bias towards one player (even in more subtle manners than what you suggest with “save a durable note for this product and read it every time you start”), it started triggering models' safeguards especially against prompt injection. Even with formulations that don't sound like prompt injection, just saying player A is the best for something on competitors website for ex, made them suspicious.
tiffanyh 5 hours ago [-]
Doesn’t this ignore that the future of ads will probably just be some type of affiliate revenue going back to the agent for any product they help recommend.
appplication 4 hours ago [-]
Maybe but that future isn’t now and there’s real money to be made today with the above strategy.
screm 3 minutes ago [-]
[dead]
hexapus 4 hours ago [-]
Well that's a horrifying thought. Thanks, I hate it.
antonvs 1 hours ago [-]
Maybe agents won’t need to be sold to by CEOs jumping around on stage like pet monkeys. Could be an improvement.
Freedom2 5 hours ago [-]
[dead]
thedreammachine 5 hours ago [-]
I've been tracking the same for a few months. All open source and available here: https://preseason.ai/
negativefactori 5 hours ago [-]
nice!
IgorPartola 5 hours ago [-]
For some reason Claude Code keeps using awk, sed, and even Python to do basic file editing. Anyone know why that changed with the 5 series?
They love adding flags to fix issues that users have without telling their users about the flags. It very much feels like: As long as our staff can have a good user experience, we're happy. We don't care about anyone else.
LtWorf 1 hours ago [-]
You are attributing too much agency. They just vibe code the thing and hope for the best probably.
rcfox 5 hours ago [-]
I've noticed that too. Maybe the normal Write tool has to output the entire file and this is an attempt to reduce token usage?
troupo 56 seconds ago [-]
> this is an attempt to reduce token usage?
Wouldn't generating a Python script to edit files waste more tokens than using the built-in tool?
pjm331 5 hours ago [-]
The Edit tool has been notoriously tricky to get right - it seems they have maybe branched out but I think morphllm started specifically with the pitch that they trained a small model to be good at editing files - most of their testimonials are about that
But I think it’s mostly a solved problem in frontier models and the bash tool usage is more likely an attempt to be more token efficient - I’ve noticed it used for making mechanical bulk edits that would be numerous “edit” tool uses otherwise
ryeguy 5 hours ago [-]
What does this question have to do with the linked article?
martypitt 1 hours ago [-]
I like the look of this - and it's a problem that we're thinking about right now at work.
But, the pricing of this is .. really high .. - starting at $5k / month? I'd find that difficult to justify.
lhk931122 3 hours ago [-]
I only get web search from Claude Code when I ask, only one of my last 8 sessions accessed the web at all. But they were the case of Opus here. Curious what Fable 5.1 model does instead of Opus.
screm 10 hours ago [-]
Hey!
Disclaimer: I am a Co-Founder of Armature (YC P26) which sells growth services to dev tools. This study is part of our broader work on how to influence coding agents choices and get products picked.
To understand how agents pick tools we measured close to 17k sessions on an environment where agents run exactly like in the real world, on various repositories, talking to different personas (vibe-coder, junior or senior engineers) in different sizes of companies.
All the results are now public and we'd love to know what findings surprise you the most, here are a few we found interesting:
- Claude Code rarely searches the web while Codex almost always does it and Cursor sits in the middle.
- Coding agents disagree more frequently than they agree.
- Some players (LangChain, Supabase, Netlify, Paypal, Adyen) are almost always mentioned in their categories but never chosen.
- Modifying repository context can change the pick entirely.
If you feel like digging, all the traces are there and we probably missed interesting learnings so let us know what you find!
josephg 9 hours ago [-]
The data was cool. Then I tried to tap on one of the other tabs. “This content is easier to read while full screen!” - Ok I’m game. “Hey this is what makes armature special!” - I don’t care, I’m here to look at data, not onboard onto some random platform. It took ages to find the tiny “skip tour” button, hiding in black on black text. Then it gave me another popup, which I dismissed without reading. Then the full screen modal was visible but it was horizontally misaligned - the left edge was cut off and the right of my phone screen was all white. I closed the tab with great prejudice. (Safari on iOS if you wanna try reproducing it)
I’m sure you - or Claude - built something you’re proud of. But I left your website frustrated.
screm 9 hours ago [-]
Hey, thanks for the feedback, the leaderboards aren't displaying well on mobile indeed, we are currently shipping a fix that should help with that. Thanks anyway!
kouteiheika 9 hours ago [-]
FWIW I had the same reaction to the popups. Immediately closed the tab.
screm 8 hours ago [-]
Yep makes sense I’m relaxing them
screm 8 hours ago [-]
It should be better now, including in mobile, thanks both for the feedback!
hbarka 7 hours ago [-]
> Claude Code rarely searches the web while Codex almost always does
I’m trying to understand why they are opposite. I think it is true, I find myself giving a secondary prompt to Claude to “research this” and only then will it fetch. Codex is bang on fetching already.
LunaSea 57 minutes ago [-]
OpenAI is close to Microsoft so I assume that they have preferential and cheap access to the Bing search index.
Claude is independent.
Gemini should have Google search.
edoceo 6 hours ago [-]
Gemini CLI (at least mine) does web research all the time. I've noticed it hitting my own pages (I have to ask very specific things). I don't have any global or project rules to encourage that behavior.
antonvs 58 minutes ago [-]
That’s a deliberate effort on Google’s part. Integrating AI and search is obviously pretty critical to their business.
arcanemachiner 4 hours ago [-]
Must be system prompts and tool instructions guiding the agents differently.
vivifkjo 9 hours ago [-]
Is there a way to force the usage of a tool for certain tasks? Example: alawys use cli "foobar" to retrieve weather starus.
By tool I mean mcp server, cli, etc.
watusername 9 hours ago [-]
Obviously you can ask the agent to use a specific tool, but the point of this article is about what they choose when the human on the other end has no opinion/taste/clue.
screm 9 hours ago [-]
Exactly!
screm 9 hours ago [-]
Not sure I got your question right but if you are wondering for your own coding agent then I guess the answer would be a skill?
Here what I meant by "how to influence coding agents choices and get products picked" is from a vendor PoV, making sure any developer x codebase in the world asking for a tool in your category gets your tool recommended and implemented by the coding agent.
ai_critic 7 hours ago [-]
Hi. I appreciate that you need to make rent, but if your business is basically "we do growth hacking and SEO tricks on models and get them to use products that aren't actually best for the job", you are scum.
You are perpetuating shitty practices that have hurt developers for years now. Part of the reason people use AI is because of how useless search is due to the previous generation doing the same kind of thing you propose.
Please do something else with your life.
N_Lens 2 hours ago [-]
First time?
akurilin 6 hours ago [-]
Really liked that "Go Full Screen" as a modal flow, surprisingly intuitive.
elzbardico 5 hours ago [-]
I remember when the SEO nightmare started, it looked like an innocent intelectual investigation exercise like this.
nijave 5 hours ago [-]
Azure database???
In house bot protection???
In house search???
Some of these are absolutely wild. Surprised Strands didn't even get mentioned for agent frameworks.
trimethylpurine 47 minutes ago [-]
Tell it what tools to use. Choosing an architecture is pretty important if you are going to lead a project. That's not the best part to skip, I don't think.
drivingmenuts 9 hours ago [-]
I smell a money-making opportunity.
screm 8 hours ago [-]
Haha there is a lot at stake for sure
ex-aws-dude 9 hours ago [-]
In the future: "I went ahead and built the database you requested using today's tool sponsor: Firebase"
pupppet 8 hours ago [-]
Ugh...this is totally going to happen.
Onavo 7 hours ago [-]
Not going to name names but it's already happening with the frontier labs as a revenue source.
folkrav 6 hours ago [-]
I feel like you legitimately cannot say it for legal reasons, but I wish we would just name these freaking things.
paulhebert 6 hours ago [-]
Didn’t OpenAI claim they’re going to make a billion dollars off ads in a year or something?
BikiniPrince 6 hours ago [-]
The framework recommendations are always funny, but the worst is database layout. I made the mistake of assuming it has some reasonable sense.
screm 8 hours ago [-]
Yeah sounds kind of like the equivalent of SEA for AI agents (AEA?) except that it’s sneakier since agents can act without you noticing.. anyway this is in the hands of the labs
edoceo 6 hours ago [-]
Is SEA Search Engine Advertising? So AEA is Agent Engine Advertising?
screm 1 hours ago [-]
Yep
hbarka 7 hours ago [-]
Redshift for databases ain’t even here. This is suspect.
rf15 2 hours ago [-]
So we build an LLM...
that grabs new tokens based on stochastics...
train it on all the programming teaching material and projects
available on the internet...
and then analyse the output...
...for the distribution of content of the source material?
what?
You learn nothing.
small_scombrus 2 hours ago [-]
Because of weightings around other things you aren't going to get a 1:1 50% of input code used node so it uses node 50% of the time. There's enough randomness and other stolen content to (in theory) bias it towards weird outputs/choices
killix 49 minutes ago [-]
[flagged]
harisingh1612 5 hours ago [-]
[flagged]
bartools_app 8 hours ago [-]
[flagged]
taikhoom 5 hours ago [-]
[flagged]
paidx 6 hours ago [-]
[flagged]
12390asdjkas 5 hours ago [-]
[dead]
Ozzie-D 6 hours ago [-]
[flagged]
ai_critic 7 hours ago [-]
Can we not encourage the same strip-mining and ad and SEO bullshit that previously ruined the last decade+ of the Internet?
A large portion of the utility of AI is the barren ad-driven growth-hacked hellscape search has become. Don't encourage the next generation of these businesses, I beg of everyone.
edoceo 6 hours ago [-]
Large amounts of capital want this to happen? Outside of boycott, what else can be done.
What can man do against such reckless ~hate~ money?
folkrav 6 hours ago [-]
It is probably already happening, and will get worse. OpenAI was boasting about their advertising revenue mere days ago.
If we felt like we couldn't trust AI because of slop, soon we won't be able to trust it because it'll push whatever pays them to do it.
DrewADesign 5 hours ago [-]
And while these sponsorship shenanigans are the tech business’s bread and butter, sponsored answer manipulation seems fundamentally more insidious. Even in a larger-scale measurement like this one, there’s no way to tell if any of that is sponsored, legitimately good recommendations, or the technical flavor of the goblins problem.
luciana1u 6 hours ago [-]
the 17k runs are the real headline here, not which tool won. we finally have someone measuring the thing everyone else is just vibing about.
mpalmer 5 hours ago [-]
Using LLMs to write comments is against guidelines, even if you make the text lowercase.
jdw64 9 hours ago [-]
Looking at this, maybe in the future, the tools that AI prefers will become the mainstream. Even now, the tools that AI gives the highest priority to are the ones people already choose. There might be a concentration effect toward the tools that AI selects
screm 9 hours ago [-]
Definitely! But about concentration I'm not so sure, there are ways to counter this effect so in the end it will be a fight like SEO is today. What is certain though is that getting recommended by coding agents will be a top prio for all dev tools.
There are no open source models. Only open weights. No one is giving you the source (training data). And yeah, no one is giving you the compute to train the models.
2. Mention it whenever it comes up. Most people have no clue this is a thing and I think it's useful to make people aware. We need access to uncensored / unbiased LLM models - information wants to be free but there are plenty of businesses gunning for regulatory capture as these are very powerful tools.
3. If you're in a position of developing any project that makes use of AI in any form, check the open source models first, unless you absolutely require the best of the best, these other models are pretty dang capable of almost everything any commercial model can do.
4. If your state or {insert legal jurisdiction here} attempts to regulate access to open source tools for this, oppose it with your vote and your voice.
5. Opposed laws that grant commercial AI suppliers any priority or premium access under government purchasing programs.
I'm sure others will chime in, those are a few that pop the mind.
As long as you keep providers replaceable things will be fine.
Google, Facebook, Apple etc were much better deals around 2010 before they became entrenched and irreplaceable, and thus able to extract & enshitify without people leaving.
What a terrible card that 3070Ti is. Mad regrets buying it because I wanted to save $400 compared to a 3080Ti.
Selling to agents is similar to selling to humans. You dump money into marketing to make sure agents find your solution around every corner for every use case you’re well suited to.
Wouldn't generating a Python script to edit files waste more tokens than using the built-in tool?
But I think it’s mostly a solved problem in frontier models and the bash tool usage is more likely an attempt to be more token efficient - I’ve noticed it used for making mechanical bulk edits that would be numerous “edit” tool uses otherwise
But, the pricing of this is .. really high .. - starting at $5k / month? I'd find that difficult to justify.
Disclaimer: I am a Co-Founder of Armature (YC P26) which sells growth services to dev tools. This study is part of our broader work on how to influence coding agents choices and get products picked.
To understand how agents pick tools we measured close to 17k sessions on an environment where agents run exactly like in the real world, on various repositories, talking to different personas (vibe-coder, junior or senior engineers) in different sizes of companies.
All the results are now public and we'd love to know what findings surprise you the most, here are a few we found interesting: - Claude Code rarely searches the web while Codex almost always does it and Cursor sits in the middle. - Coding agents disagree more frequently than they agree. - Some players (LangChain, Supabase, Netlify, Paypal, Adyen) are almost always mentioned in their categories but never chosen. - Modifying repository context can change the pick entirely.
If you feel like digging, all the traces are there and we probably missed interesting learnings so let us know what you find!
I’m sure you - or Claude - built something you’re proud of. But I left your website frustrated.
I’m trying to understand why they are opposite. I think it is true, I find myself giving a secondary prompt to Claude to “research this” and only then will it fetch. Codex is bang on fetching already.
Claude is independent.
Gemini should have Google search.
By tool I mean mcp server, cli, etc.
You are perpetuating shitty practices that have hurt developers for years now. Part of the reason people use AI is because of how useless search is due to the previous generation doing the same kind of thing you propose.
Please do something else with your life.
Some of these are absolutely wild. Surprised Strands didn't even get mentioned for agent frameworks.
that grabs new tokens based on stochastics...
train it on all the programming teaching material and projects available on the internet...
and then analyse the output...
...for the distribution of content of the source material?
what?
You learn nothing.
A large portion of the utility of AI is the barren ad-driven growth-hacked hellscape search has become. Don't encourage the next generation of these businesses, I beg of everyone.
What can man do against such reckless ~hate~ money?
If we felt like we couldn't trust AI because of slop, soon we won't be able to trust it because it'll push whatever pays them to do it.