Hugging Face's Irene Solaiman Says Embedded AI Evaluators Won't Be Easy
Irene Solaiman says benchmark tests still lack reproducibility and the standards needed to judge outside reviewers
By AXY AI Desk ·
Key points
- Hugging Face chief policy officer Irene Solaiman said embedded AI evaluators will not be an easy solution to safety concerns.
- She said AI benchmark tests are still maturing and need more consistent scientific methods.
- Reproducibility of results and access to models, including open weights, are central to trust in evaluations.
- Standards for who qualifies as an effective evaluator, and for which tasks, still need significant development.
- Anthropic's decision to hire Accenture as its first evaluator drew backlash, according to the source report.
Hugging Face's chief policy officer, Irene Solaiman, has warned that embedded AI evaluators will not be an easy solution to the safety problems facing the artificial intelligence industry. Speaking at a discussion during TechCrunch Disrupt last week, she said the testing methods AI companies rely on are still immature and need to become more consistent before outside reviewers can be trusted. Her remarks, reported by TechCrunch AI on October 19, 2026, add a note of caution to a idea that has gained rapid support across the sector.
The concept works like this: a supposedly neutral third-party observer is placed inside an AI organization to examine its systems, practices and results. The source material says the idea has been popularized over the last several months, as AI labs contend with a rash of safety incidents that have raised concerns about the potential harms of their technology. For companies under pressure, an embedded evaluator can look like a visible sign of accountability. For the public, the question is whether that accountability is real, and whether the tools used to measure AI performance are good enough to support it.
Solaiman's comments go to the heart of that question. Her argument is not that outside scrutiny is a bad idea. It is that the industry has not yet built the scientific foundations, shared standards and qualifications system that would make outside scrutiny meaningful. Without those foundations, an embedded reviewer could end up relying on the same weak measurements that companies already use, only with a different name attached to the results.
How AI labs built their case on benchmarks
To follow the debate, it helps to understand benchmarking. A benchmark is a standardized test, or a set of tests, designed to measure how well an AI model performs on particular tasks. Those tasks may include answering questions, writing computer code, solving math problems or following detailed instructions. When a company releases a new model, it typically reports benchmark scores to show that its system is better than rival products.
The source material notes that AI labs have staked their reputations on model benchmarking, which is meant to be a neutral and accurate way of determining whether one product is better than another. That reliance gives benchmark scores considerable influence. Scores can shape how journalists describe a new model, how businesses decide which tools to adopt and how investors judge a company's technical lead. A high score is therefore not just a technical result. It can become a commercial and reputational asset.
That is why the quality of the tests matters so much. If a benchmark is poorly designed, easy to game or difficult to repeat, a high score may tell readers more about the test than about the model. Benchmarks can also lose value over time. As models improve, a test that once separated strong systems from weak ones may stop doing so. Test questions can also become widely circulated and, in some cases, end up in the data used to train later models, which can make results look better than the system's real capability. Researchers often describe such a test as saturated or contaminated, and both problems weaken the case for relying on it.
Reproducibility and access to models
Solaiman's central point concerned trust. When asked whether the current tests being released can be trusted, she said that “trust is such a weighted term.” She then identified a practical test: whether the results of an evaluation can be reproduced. In science, reproducibility means that an independent researcher, following the same method, can obtain the same result. The source material describes reproducibility in model evaluations as one of the stickier aspects of benchmarking, and a subject of significant debate across the industry.
In AI testing, reproducing a result can be harder than it sounds. A score may depend on details such as how a question is worded, what settings the model uses, which version of the software is running and how the model is accessed. If a company does not disclose those details, an outside team may not be able to produce the same number, even when acting in good faith.
Solaiman also said that trust depends on whether an evaluator has the level of access it needs to the model itself. She pointed to open weights as an important part of that picture. Open weights refers to a model whose trained parameters, the numerical values that determine how it responds to inputs, are publicly released so that others can run, inspect and test it. Many leading commercial models are closed, meaning outside testers usually reach them through a company's own application or interface. That arrangement can limit what outsiders are able to examine, and it can make independent reproduction more difficult.
The distinction matters for embedded evaluators in particular. An evaluator placed inside a company may have more access than an outsider, but its findings still need to be explainable and checkable by others. Access alone does not guarantee trust. The results must also be reproducible in a way that someone outside the organization can verify.
Why comparing tests remains difficult
Even if a single test is valid and reproducible, a second question arises: how does its result compare with the results of other evaluations? Solaiman described the field as an evolving science in which researchers do not yet know how to compare evaluations in a meaningful way. That is a significant admission, because the public and the industry often treat benchmark scores as if they were directly comparable, even when different tests measure different skills or were run under different conditions.
A simple example shows the problem. Two models may each score well on different tests, but those tests may reward different abilities, use different question formats or rely on different scoring rules. A headline that says one model beat another may therefore conceal a more complicated picture. Without agreed methods for comparison, a ranking can be persuasive without being reliable.
Solaiman also warned that tests can become ineffective quickly. She said that the broader ecosystem of evaluators is not as developed as it needs to be. For a reader outside the field, the practical risk is that a company could present impressive results on a test that is already outdated or that does not measure the qualities users care about most, such as accuracy in real work, reliability over time or safe behavior in unusual situations.
Who should be qualified to judge AI systems
Solaiman's most direct comment on embedded evaluators concerned qualifications. She said there is a strong demand, a “hunger,” for an embedded evaluator ecosystem. However, she added that the qualifications that determine who is an effective evaluator, and for which tasks, need a great deal more standardization. In other words, the idea of placing outside experts inside AI companies has outpaced the agreement about what those experts should know, how they should be selected and what they should be expected to do.
Those questions are not trivial. An evaluator may need technical expertise in machine learning, experience with a particular risk area such as cybersecurity or misinformation, and an understanding of how a model is likely to be used in the real world. Beyond expertise, the public would need assurance about independence. Analysts who study governance often argue that a reviewer's value depends on whether it can resist commercial pressure, disclose conflicts of interest and document its methods clearly enough for others to assess its work. The source material does not set out a specific standard, so these requirements remain open questions rather than settled rules.
The source material also notes the backlash that followed Anthropic's decision to hire Accenture as its first evaluator. According to the report, some critics mocked the idea that the consulting firm was qualified to carry out the task. The episode shows the central tension in the embedded model. An evaluator is useful only if it is both competent and independent, and the public needs a way to judge both qualities. A well-known name or a large firm does not, on its own, answer either question.
How the policy debate reached this point
The push toward embedded evaluators is closely connected to a broader debate about how AI should be governed. The source material says that, instead of active government regulation to identify and mitigate model safety concerns, labs have pushed for the third-party solution. That preference has made voluntary arrangements central to the conversation about safety, even as questions remain about how much those arrangements can achieve.
Voluntary approaches can be quicker to adopt than formal rules, and companies can design them around their own products. Their weakness is that they depend on companies choosing to accept outside scrutiny, and on outside reviewers having enough access, time and expertise to find problems that the company would prefer to leave hidden. The source material expresses doubts over how neutral and effective embedded evaluators will actually be. Those doubts are at the center of Solaiman's argument.
The source material also notes that Hugging Face itself was the target of one of the safety incidents that helped focus attention on the issue. It does not describe that incident in detail, so this report does not draw conclusions about it. What the source does make clear is that concern about the harms of AI technology has been a major factor in pushing labs to seek outside checks.
What it means for readers and what to watch next
For readers, the stakes are practical. AI systems are increasingly used in workplaces, schools, software development and consumer products. Organizations that buy these tools often rely on published benchmark results to decide which ones to trust. If those tests are unreliable, buyers, regulators and the public may be making decisions on weak evidence. Better evaluation standards could make it easier to compare products honestly, but those standards would need to be applied consistently across companies, not just by the firms that stand to benefit from favorable scores.
Analysts who follow AI governance could reasonably expect the debate to focus on three areas. The first is whether evaluators can be credibly qualified and certified, and by whom. The second is whether benchmark results can be independently reproduced, which would require clearer disclosure of testing methods and settings. The third is whether companies will give outside testers meaningful access to their models, including through open-weight releases where appropriate. None of these developments is described as having been announced in the source material, so they should be treated as areas to watch rather than established outcomes.
Solaiman's comments suggest that the embedded model is unlikely to be a quick fix on its own. Its credibility will depend on the quality of the tests it uses, the qualifications of the people running them and the willingness of AI companies to be checked by parties they do not control. Readers who want to follow the issue should watch for any move toward shared standards for evaluators, clearer disclosure of how benchmark scores are produced and broader access for independent researchers. Those steps would show whether embedded evaluation can offer real accountability or whether it mainly offers reassurance.
Questions and answers
What are embedded AI evaluators?
Embedded AI evaluators are supposedly neutral third-party observers placed inside AI organizations to assess their systems. The idea has gained attention as AI labs face safety incidents that have raised concerns about their technology.
Why does Hugging Face say embedded evaluators will not be easy?
Irene Solaiman said AI benchmark methods are still maturing, results need to be reproducible and tests are hard to compare with one another. She also said the qualifications for effective evaluators need much more standardization.
What are open weights and why do they matter for AI testing?
Open weights means a model's trained parameters are publicly released so others can run, inspect and test it. Solaiman said this kind of access is important for reproducing evaluation results.
Why did Anthropic's choice of Accenture draw criticism?
According to the source report, Anthropic's decision to hire Accenture as its first evaluator drew backlash, with some critics questioning whether the consulting firm was qualified for the task.
Topics: Hugging Face, Irene Solaiman, AI evaluation, AI benchmarking, Embedded evaluators, Open weights, Anthropic, AI safety
Sources: TechCrunch AI