Claude Updates: Web Search, Crawlers and Anthropic News

Anthropic published two posts on 26th Aug that together change what Claude does on other people's websites.

The first says Claude in Chrome is generally available. Plan coverage reached every paid plan in December 2025, so that is not the change that matters.

It is that "Claude can now also take actions autonomously in the browser, instead of needing approval for every one", using the same automatic-approval mechanism as Claude Code.

A classifier reviews each action Claude is about to take. That includes opening a site or typing into a page.

It blocks any action that does not match what the user asked for. The setting can be switched off.

What Claude does through the extension is unchanged in kind. It reads the page, clicks links, types text, navigates and fills forms, on the user's existing logins.

The second post is the new one. Claude now has its own browser inside the Claude Cowork desktop app.

It opens in a side panel. It does the same navigating, reading, clicking and typing without touching the user's own browser at all.

Anthropic's framing of why both exist: "a lot of web tasks don't need your browser, just a browser, and now Claude has one".

Logins are carried over site by site from Chrome, Edge or Firefox. Banking, email and single sign-on sites are excluded unless the user opts them in.

It is rolling out over the week to Pro, Max and Team on macOS, Windows and Linux. It is on by default once it arrives.

Enterprise has it now, managed by admins in organization settings. Those admins can also restrict Claude in Chrome to approved domains.

The extension does not run on other Chromium browsers or on mobile.

For a site owner, what matters is what none of this appears in.

Anthropic's site-owner help page describes three bots and states that they honour robots.txt. They are ClaudeBot for training, Claude-User for fetches triggered by a user's question and Claude-SearchBot for search quality.

Read on 27th Aug, that page does not contain the words browser, Chrome, extension or agent anywhere. Neither of the two announcements names a user agent.

So there is no published way for a site owner to recognise this traffic. There is also no stated answer to whether robots.txt bears on it at all.

That is a different situation from a crawler you can allow or refuse. Requests arrive inside a signed-in session that the user authorised.

That is the same shape as Grok Bot. It is the reason this hub keeps returning to the gap between crawler controls and agent traffic.

The prompt-injection numbers are published, and they are the most detailed on this beat.

Anthropic's current test set is built from attacks written by professional red-teamers. Attacks that reached the model succeeded 17.6% of the time against Opus 4.5 before extra safeguards.

Against Opus 5 the rate was 3.8%. Anthropic also reports results with the content probes and the approval classifier running.

There were no successful attacks against Sonnet 5, Opus 5 or Mythos 5. Fable 5 saw a 0.3% success rate.

Two caveats come from Anthropic itself. The older test set was retired because it had been saturated at a 0% success rate.

Not every attack reaches the model in the first place. So these rates describe the attacks that got through rather than all attempts.

Anthropic's own conclusion is the one to carry into any decision about letting agents onto a site. The measures "meaningfully reduce the risk but can't eliminate it", and it recommends starting on sites the user trusts.

The Interactive Advertising Bureau published Measuring Visibility in the AI Era on 3rd Aug. It is a set of guidelines for measuring how brands and publishers appear inside AI-generated answers.

It starts from a problem this page runs into constantly. More than 20 companies now sell AI visibility measurement tools.

Per the IAB, each uses different methods that "can produce different answers for the same brand or publisher".

The framework's answer is limited on purpose. It supplies a shared vocabulary, quality criteria and disclosure requirements.

In the IAB's own words it does so "without prescribing specific tools or ranking individual providers".

That limit is the most useful thing in it for a buyer. A trade body ranking vendors produces a winner list.

Defining what a mention is, and what a provider must disclose about counting one, does something better. It lets buyers compare quotes themselves.

The metrics sit in a four-level hierarchy the IAB calls the 4 P's of AI Visibility. The levels are ordered by how visibility turns into business value.

Presence asks whether you appear at all, through mention rate, citation rate, share of voice and visibility momentum.

Prominence asks where and how prominently. That includes how substantively a publisher's content is drawn on rather than merely listed as a source.

Portrayal asks in what context and with what accuracy. It covers sentiment and framing plus a hallucination rate and a factual inaccuracy rate.

Keeping those last two apart is the point of the level. A hallucinated mention is one the model invented.

A factual inaccuracy is one where the model reported a real source faithfully and the source was wrong. The two have different remedies.

Persuasion asks whether any of it drives action, through recommendation strength and post-citation click-through rate. How to attribute is left to a separate IAB framework still to come.

The second contribution is a quality test that sorts data by what it can safely be used for.

Directional measurement identifies patterns and signals trends, and the IAB states plainly that it "is not sufficient for budget allocation or executive strategy decisions".

Decision-grade measurement meets a higher standard across query volume, sample size, prompt type coverage, testing cadence, reproducibility and platform coverage. The IAB says that standard is required before any budget or strategy decision.

The named failure is not using directional data. It is treating directional data as decision-grade without noticing the difference.

Caroline Giegerich, the IAB's VP for AI, framed the gap the document is meant to close: "Consumers are increasingly discovering and considering brands and products in AI platforms, but measurement frameworks haven't kept pace."

Two things to know before reading it. Its scope is organic, non-paid visibility only.

So it does not cover paid placement inside AI answers, generative engine optimization tactics or commerce attribution.

And the adoption figures it cites for context are borrowed from third parties. The IAB did not measure them.

Those figures cover how many people use which AI surface. We have not adopted them here.

Some sit awkwardly beside the figures on our own hubs.

This framework also gets no row in the measurement table above, by design. Every row there counts something real that happened.

A vocabulary counts nothing, however useful it is for judging the people who do the counting.

Read this one with its authorship in view. It is a single-author arXiv preprint by Benjamin Tannenbaum of Aiso, a company selling AI-visibility tracking.

It has not been peer reviewed. The self-interest runs one way and belongs next to the result.

Aiso sells session-level conversation tracking, which is the product this finding argues buyers need.

We are carrying it because it measures a mechanism rather than ranking who is winning. It also repeatedly declines claims it could have made.

The question is narrow. Holding the last thing a user typed constant, how much do the earlier turns change the answer?

The experiment is paired at the conversation level across 180 English multi-turn conversations. Half come from Aiso's own governed corpus and half from the public PRISM dataset.

The final user message, the answering model and the output format stay fixed. Only the representation of the preceding turns changes.

Three answers are generated per case. One comes from the full role-labelled conversation and one from the final message alone.

The third comes from the final message plus a reconstruction of the earlier turns capped at 160 words.

One design detail carries a lot of weight. The reconstruction model never sees the final message.

So it cannot write a convenient summary backwards from the endpoint.

A separate judge model then scores all three under randomized labels. A difference counts only if it could change what the user does.

Four kinds of change qualify. Two are a different recommendation or factual bottom line, and different compliance with an explicit constraint.

The other two are a different requested deliverable, and different clarification or refusal behaviour. Wording, tone, organisation and amount of detail are excluded by rule.

Removing the conversation changed the answer materially in 44.7% of cases, weighted to the eligible cohorts. The 95% confidence interval runs from 33.8% to 56.1%, and the unweighted paired rate is 53.3%.

Full-conversation answers also scored better. The weighted mean was 3.80 against 3.31 on a 0 to 4 request-satisfaction scale.

The direction is not universal, and the paper says so. Full context scored higher in 36.2% of weighted cases and lower in 6.0%, with the rest tied.

The category breakdown is the part to read closely, with one caution about how it is reported.

The results table lists a changed recommendation or conclusion at 26.7% (17.0% to 36.9%). Changed clarification or refusal behaviour is at 14.4%, and changed constraint compliance at 10.6%.

The table presents those in the same column as the 44.7%, as rates over all cases. The body text describes them as shares of the differing pairs.

Read them as the paper's own category breakdown rather than as three independent rates. Note the paper's own line that the categories can co-occur.

The compressed-prefix condition produced the most useful result here. A 160-word summary of the earlier turns closed almost the whole satisfaction gap, to 0.01 points.

It still left 30.8% of answers materially different from the full-conversation answer.

So a state summary can support an equally good answer while changing which reasonable answer the model picks. Average quality and answer identity are not the same measurement.

The paper draws the consequence for this industry itself: "For a visibility tracker, that can mean a different cited brand."

There is also no safe subset of self-contained prompts. Cases were sorted by how context-dependent the final message looked.

Isolation still changed 31.0% of the least dependent answers. The middle band saw 56.1% and the highest 48.4%.

The companion paper from ten days earlier established the other half of this. It covered 670 commercial and 7,463 public conversations.

In 50.3% and 44.8% of conversations, an explicit constraint, alternative, correction or evidence requirement appeared before the final message. It did not appear in the final message itself.

The final prompt reproduced the full set of detected request dimensions in only about 26%.

The limits are large enough that this is direction about a measurement error. It is not a rate to quote as a population figure.

One answer model was requested, gpt-5.4-mini, with gpt-5.5 as judge. Both ran through a research command-line runner rather than a native chat product.

There was one generation per condition.

The judge is automated rather than human. The paper names that as its own most important replication target.

Medical, legal, financial and other high-stakes conversations were excluded from the sample entirely.

Half the sample is Aiso's own corpus, which cannot be redistributed. The weighting leans the pooled estimate towards the public half.

The commercial half showed a far larger effect, 68.5% against PRISM's 35.4%.

The scope is preceding turns inside one conversation. It does not test persistent memory across separate sessions.

The paper is explicit that its data cannot observe such a system.

Here is the practical reading for anyone buying this kind of data. A tool reporting whether assistants mention you is measuring endpoint strings unless it tells you otherwise.

The paper asks that such prompt sets be labeled as endpoint-message evaluations. Its conclusion puts it plainly: "Replaying endpoint strings alone measures a different task."

Read it beside the IAB framework published the same day. Its decision-grade test asks about query volume and prompt-type coverage.

This paper is about what a query is in the first place.

None of it belongs in the measurement table above, whose rows count real crawling and real referrals.

Judge Araceli Martinez-Olguin approved the $1.5 billion class settlement on 20th Jul, per TechCrunch: "The payout will deliver $3,000 per work across an estimated 500,000 works, shared among the authors and publishers who hold rights to them."

The legal shape matters as much as the number. Judge William Alsup had ruled that TRAINING on the books was fair use.

Downloading and storing millions of pirated copies was not. He granted preliminary approval on that basis in 2025.

Because Anthropic settled rather than appealed, no binding appellate precedent was set in either direction. Every other AI copyright docket, including Perplexity's eleven cases, proceeds without a controlling answer.

For site owners the practical reading is economic. Pirated training corpora now carry a price tag believed to be the largest in US copyright history.

That strengthens the hand of anyone licensing content, and the case for controlling the training crawler.

DataDome, a bot-management vendor, publishes a quarterly count of AI agent traffic across its network. The Q2 2026 edition is by VP of Threat Research Jerome Segura.

It draws on April to June data from "5 trillion signals analyzed daily across 400+ enterprises".

The Claude figure is the reason this sits here. Claude referral visits grew 111% quarter on quarter, from 415,000 to 876,000.

DataDome calls that the fastest growth among the tools sending real visitors to websites.

Perplexity grew 37% over the same period.

That is the second independent panel in two months to measure Claude referrals climbing fast from a small base. The first was SE Ranking's 386% over four months.

The two count different things on different networks. So they confirm the direction rather than the size.

The wider finding is that crawling and referring have come apart. ChatGPT-User fetched 6% fewer pages than in Q1 while ChatGPT referrals rose 17%.

ChatGPT still accounts for 80% to 88% of all AI referrals every month.

Meta runs the opposite way. Its two crawlers generated 9.1 billion requests in the quarter.

That is more than half of all AI agent traffic on the network.

DataDome's summary of what publishers got for it is blunt: "those billions of visits sent almost no real visitors back to those sites in return".

The two Meta bots do different jobs, and the split is the part site owners should read.

Meta-ExternalAgent, up 74% to 5.3 billion requests, collects training data. Meta-WebIndexer, up 163% to 3.75 billion, indexes pages so Meta AI can answer current questions.

In June it passed the training crawler in monthly volume for the first time.

Meta's own crawler docs say the same thing in Meta's words. Meta-WebIndexer "navigates the web to improve Meta AI search result quality", and allowing it in robots.txt "helps us cite and link to your content in Meta AI's responses".

So blocking it is a citation decision as well as a bandwidth one.

Read the denominator carefully. These are request counts on DataDome's customer network, not the web, and they are not crawl-to-refer ratios.

So none of them is comparable with the Cloudflare series above. That series divides crawls by referrals for a named platform.

Anthropic's own crawlers are not broken out in this report at all.