Artefact’s earlier study on The Future of Work with AI concluded that repetitious and monotonous tasks will be augmented by Agentic AI and transformed into agentic supervision. This new report, “The Future of Agentic Supervision”, takes a deep dive into how organizations can prepare to oversee and manage the performance, safety, and strategic value of these new intelligent systems, and, eventually, reinvent work around Agentic AI supervision. “We found that although AI agents will replace humans on tedious and repetitive tasks, a new type of work will appear: agentic supervision,” says Florence Bénézit, Expert Partner Data & AI Governance at Artefact. “Supervision should not be an afterthought, it must be embedded early in the agent’s design and development,” remarks Hanan Ouazan, Managing Partner & Lead Generative AI at Artefact.
In the evolving landscape of enterprise AI, the rise of agentic systems marks a pivotal shift. AI agents are autonomous applications powered by large language models (LLMs) capable of reasoning, memory, and action. They are active decision-makers influencing business processes in real time. But with autonomy comes risk, and with risk comes the need for structured supervision. In this new paradigm, supervision of tech systems is no longer optional; it is foundational.
NB: Florence Bénézit and Hanan Ouazan, our Subject Matter Experts, co-authored this survey and interviewed 14 enterprises and five Artefact agentic product managers and engineers. We also contacted key agentic supervision providers, including major data & AI platforms with years of software supervision experience (such as Google and Microsoft) as well as specialized start-ups (WB, Giskard, RobustIntelligence…).
Key insights from the survey “The Future of Agentic Supervision”: from governance principles to operational readiness.
Key insights from the survey “The Future of Agentic Supervision”: from governance principles to operational readiness.
Agentic AI systems are not traditional software, they are probabilistic.
:
- Software production is driven by deterministic rules, constantly applying the same, invariant and correct logic, while AI agents are probabilistic, meaning their outputs, while strongly influenced by input context, do vary at every run.
- They integrate natural language capabilities with the power to act autonomously across internal tools, APIs, or databases to solve new problems. This flexibility enables impressive value creation across customer support, operations, HR, and procurement.
The central trade-off of agentic deployment: Value vs. risk.
:
- Where traditional code is tested once and deployed with confidence, agents must be deployed with risk in mind, constantly monitored, evaluated, and improved. The future of agentic governance is therefore about ongoing supervision at scale.
- This value vs.risk trade-off must be explicitly managed. Enterprises must define what “value” means in context (task success rate, user engagement, productivity gains) as well as which risks must be controlled: hallucination, latency, bias, reputational damage, or cost overruns.
“Supervision becomes the operational mechanism that tunes this balance at every step of the AI agent lifecycle: at design, development, deployment, and run times,” explains Florence.
Managing agentic supervision in three steps: Observe, Evaluate, Act.
To achieve this balance, companies need to build agentic supervision around three core capabilities:
- 1. Observation: Capture structured telemetry data: inputs, outputs, tool calls, errors, and human feedback. 2. Evaluation: Use quality metrics and risk indicators to assess performance against business-defined objectives and control thresholds. 3. Action: Escalate and manage incidents, retrain models, adjust guardrails, or roll back agent updates.
“This process, called “active supervision”, mirrors DevOps practices but must be adapted for the probabilistic, evolving nature of AI, and extended beyond the tech teams to encompass business processes and teams Customer Success, HR, Legal, Operations, etc.”
The overarching conclusion of this new study is that agentic supervision is the future of AI.
Florence Bénézit, co-author of Artefact’s The Future of Agentic Supervision, presented the study at our booth at VivaTech 2025.
Florence Bénézit, co-author of Artefact’s The Future of Agentic Supervision, presented the study at our booth at VivaTech 2025.
Florence explains that agentic governance begins well before deployment:
:
- From the exploration phase onward, business and technical teams must collaborate to define success criteria, identify risk types, and decide on the evaluation strategy. This co-design approach ensures that agents are technically robust and aligned with business priorities from the outset. Without this level of organization, it will be difficult to scale agentic trust.
- In the design phase, teams often need to build “ground truth” datasets that reflect desired agent behavior. These become essential for both training and evaluation. During development, teams must determine go/no-go release thresholds across multiple metrics.
“Our goal is to lay the foundation for the new profession of AI agent supervision, a hybrid role that spans business, tech, and governance – and one we believe represents the future of work with AI,” emphasizes Florence.
We offer a practical playbook with four recommendations to help companies move from theory to practice.
- Start with real projects, not prototypes: Focus governance efforts on high-value agents intended for production. Build real systems, not throwaway demos, to uncover operational realities early.
- Think developer-first in tooling: Choose observability tools that support engineering workflows. Business dashboards are helpful, but developer adoption is critical to collecting quality metadata.
- Clarify risk ownership: Define which functions own which risks. Who is accountable for security, privacy, bias, or UX degradation? Establish escalation paths and sign-off rules.
- Unify AgentOps with DataOps: Treat agents and data pipelines as two sides of the same coin. Jointly supervise data quality and agent behavior to diagnose root causes of incidents.
“Agent governance will evolve rapidly. But its foundation is timeless: clarity, collaboration, and continuous learning. Enterprises that embrace this discipline early will not only avoid costly mistakes, they’ll build a durable competitive advantage.”
Four Artefact offers to assist organizations as they initiate or accelerate their Gen AI & Agentic AI journeys.
Four Artefact offers to assist organizations as they initiate or accelerate their Gen AI & Agentic AI journeys.
Read Artefact’s chapter in the VivaTech’s Hub Institute Report
Our suite of generative AI and agentic AI services is customized for each client’s level of maturity, enabling them to start or scale quickly.
– Offer #1 | AI strategy and organization: The ideal offer for companies just beginning with AI or looking to scale. In 6–12 weeks, Artefact helps define a clear AI roadmap, operating model, and change plan to embark employees. – Offer #2 | AI adoption: For companies with AI tools already in place, this offer rapidly boosts usage and productivity gains, often by deploying secure internal GPTs. The aim: fast, measurable gains in just four months. – Offer #3 | AI / Agentic factory and platform: This offer allows AI leaders to move fast by designing and launching an acceleration program covering solution development, platform build, and adoption, with measurable victories at 3, 6 and 12 months. – Offer #4 | AI for function transformation: With a focus on specific business functions like client service or operations, this offer delivers major performance improvements with Agentic AI, such as >30% productivity gains and up to 80% lead-time reduction.
From medical writing assistants to customer care chatbots and synthetic personas, Artefact’s successful use cases demonstrate the breadth and business impact of Gen AI solutions across sectors.
Join us at Artefact’s International Adopt AI Summit in the iconic Grand Palais in Paris, on November 24, 25 & 26, 2025.
Join us at Artefact’s International Adopt AI Summit in the iconic Grand Palais in Paris, on November 24, 25 & 26, 2025.
Artefact has been at the forefront of organizing the largest AI summits across industries for over a decade. Adopt AI – Grand Palais is the next chapter in this journey. With 25,000 attendees, 500 speakers, and 250 exhibitors, Adopt AI is the place to be for the most innovative AI discussions. The CEO Stage, featuring 30+ cross-industry CEOs. We’re particularly honored to already announce that Olivier Laureau, CEO and President of Servier, and Dirk Hoke, President & CEO of Voith Group, will be speaking at Adopt AI.
Three Main Stages, where top executives will share their strategic visions of AI. Speakers already confirmed include:
:
- Dr. Mohammed Rahim, Group Chief Data Officer, Standard Chartered
- Pauline Thomson, Managing Director – Infrastructure Funds & Head of Data Science, Ardian
- Yves Tyrode, Chief Executive Officer: Digital & Payments, BPCE Group
- Emma Charles, Senior Vice President European Markets, Bristol Myers Squibb
- Diogo Rau, Executive Vice President and Chief Information and Digital Officer, Lilly
- Sébastien Arbola, EVP Data, Digital, IT, Strategy, R&I, Engie
- Alain Becoulet, Deputy Director General, ITER
- Philippe Keryer, SEVP Strategy, Research and Technology, Thales
A series of Masterclass Stages, for hands-on, technical, and sector-focused content. Don’t miss this unique opportunity to connect with global leaders and join in high-impact discussions across seven simultaneous stages dedicated to every major sector: Health, Finance, Industry, Luxury, Travel, Sport and Sustainability
Summer break notice 🌞
Summer break notice 🌞
As we’re taking a break in August, the next Data Digest will be published at the end of September. Stay tuned to our Artefact LinkedIn page over the next two weeks – we’ll be sharing a series of client testimonials and reports to recap key insights and case studies from the first half of the year.
Happy Summer Reading!
