Elevate support excellence with Quality Evaluation Agent in Dynamics 365

Based on the Microsoft Dynamics 365 blog post from October 30, 2025, here is the breakdown of the Quality Evaluation Agent.

Elevate support excellence with Quality Evaluation Agent

(An autonomous AI agent designed to automate the Quality Assurance (QA) process by evaluating 100% of customer interactions against defined business rubrics).

Real-time User Journey: From Manual Sampling to Total Oversight

This journey highlights how a Quality Manager shifts from “hunting for errors” to “strategic coaching”:

1. Continuous Analysis: Throughout the day, thousands of calls and chats occur. In the background, the Quality Evaluation Agent automatically analyzes every single transcript.
2. Rubric Application: The agent applies the company’s specific “Excellence Rubric,” checking for compliance, empathy, accurate product knowledge, and adherence to greeting protocols.
3. Anomaly Detection: The agent identifies a specific agent who is consistently providing high-quality service but struggling with a new “return policy” detail. It also flags a wider trend where customer sentiment drops whenever “shipping delays” are mentioned.
4. Proactive Reporting: Instead of the manager listening to random tapes, they open their Insights Dashboard. They see a prioritized list of interactions that require human attention (e.g., high-risk calls or those with low AI-generated quality scores).
5. Targeted Coaching: The manager clicks on a flagged call. The agent provides a Reasoning Summary: “Score: 65/100. Agent missed the mandatory privacy disclosure at 02:14.”
6. Loop Closure: The manager uses these precise insights to provide a 5-minute targeted coaching session. The agent continues to monitor the agent’s next calls to track improvement in real-time.

Step-by-Step: How to Enable This Feature

The Quality Evaluation Agent is part of the “Agentic” suite in Dynamics 365 Customer Service.

• Step 1: Access the Agent Hub

Sign in to the Customer Service admin center. Navigate to Operations > Agent Hub.

• Step 2: Activate the Quality Evaluation Agent

Locate the Quality Evaluation Agent card and select Manage. Toggle the status to Enabled.

• Step 3: Define Your Evaluation Rubric

Upload or define your scoring criteria. You can set specific parameters such as:

• Compliance: Did the agent verify the customer’s identity?
• Soft Skills: Was the tone professional and empathetic?
• Process: Did the agent follow the standard troubleshooting steps?
• Step 4: Configure Sampling (Optional)

While the agent can evaluate 100% of calls, you can configure it to prioritize specific high-value queues or new trainees for more frequent reporting.

• Step 5: Connect to Power BI Insights

Enable the Quality Insights Dashboard. This will pull the agent’s evaluations into visual reports showing trends across teams, time periods, and specific product categories.

• Step 6: Set Up Supervisor Alerts

Configure “Critical Failure” alerts. For example, if a “Compliance Score” drops below 50 on a single call, the supervisor receives an immediate notification to review the interaction.

Infographic: Traditional QA vs. Quality Evaluation Agent

Feature	Traditional Manual QA	Quality Evaluation Agent
Coverage	1–2% of interactions (Randomized).	100% of interactions (Total).
Speed	Retrospective (Days or weeks later).	Near Real-time (Minutes after call).
Objectivity	Subject to human bias and fatigue.	Consistent & Unbiased application of rules.
Insights	Anecdotal/Small sample size.	Data-driven trends across the whole org.
Supervisor Role	Listening to tapes (Manual labor).	Coaching & Strategy (High value).

References

Elevate support excellence with Quality Evaluation Agent