>>>>> gd2md-html alert: ERRORs: 0; WARNINGs: 0; ALERTS: 19.

See top comment block for details on ERRORs and WARNINGs.
In the converted Markdown or HTML, search for inline alerts that start with >>>>> gd2md-html alert: for specific instances that need correction.

Links to alert messages:

alert1 alert2 alert3 alert4 alert5 alert6 alert7 alert8 alert9 alert10 alert11 alert12 alert13 alert14 alert15 alert16 alert17 alert18 alert19

>>>>> PLEASE check and correct alert issues and delete this message and the inline alerts.

https://koreteam.atlassian.net/browse/STAT-16401

https://koreteam.atlassian.net/browse/STAT-20720

https://koreteam.atlassian.net/browse/STAT-21035

https://koreteam.atlassian.net/browse/STAT-13432

https://koreteam.atlassian.net/browse/STAT-21620

https://koreteam.atlassian.net/browse/STAT-21524

By Question¶

The Evaluation Metrics feature is a core component of the Quality AI module, allowing supervisors to define, tailor, and monitor performance indicators that assess the quality of interactions between agents and customers. This process is driven entirely by individual questions. Users can create and customize evaluation criteria using various measurement types, which are managed within the **Evaluation Forms **section.

Access Evaluation Metrics¶

Access Evaluation Metrics **by navigating to **Contact Center AI > Quality AI > Configure > Evaluation Metrics.

>>>>> gd2md-html alert: inline image link here (to images/image1.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

By Question¶

This configuration evaluates adherence to specific questions asked or answered during customer-agent interactions.

Configuration steps:

Select an **Evaluation Metrics Measurement Type **from the dropdown.

>>>>> gd2md-html alert: inline image link here (to images/image2.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

Enter a referenceable Name of the metric for future reference (for example, agent’s warm greeting).
Select a preferred **language **from the dropdown menu to evaluate the form and configure the required metric.

>>>>> gd2md-html alert: inline image link here (to images/image3.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

    **Note**:

* Only the added languages are available for selection in the **Settings**. [Learn more](https://docs.google.com/document/d/10hRrRjnBXyqPBs2ueJBZQhe_N6nMlDE9MGNTMj0YZ1U/edit?usp=sharing). 
* Once a primary language is selected, you can configure additional languages as needed.

Enter an evaluation **Question **to help supervisors assess agent adherence as a reference for the audits and interaction reviews.

>>>>> gd2md-html alert: inline image link here (to images/image4.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

Select the agent Adherence Type: Static **or **Dynamic.

>>>>> gd2md-html alert: inline image link here (to images/image5.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

    **Note**:

* For **Static**, you must configure at least one agent answer utterance for the Adherence Type.
* For **Dynamic**, you must configure at least one trigger and one agent answer utterance for the adherence type.

Adherence Type Configuration¶

    The following are the key trigger components that you can configure:

Customer Utterance: Evaluation initiated by customer statements
Agent Utterance: Evaluation triggered by agent responses
Multiple Triggers: Support for complex conditional scenarios
GenAI Suggestions: Automated variation generation

¶

    **Dynamic Adherence **


    Evaluates agent adherence only if the configured trigger is detected. Based on the evaluation trigger, the agent and customer option is selected. This allows the conditional check-in. It is a trigger-based detection, in which you set up a trigger either for a customer or an agent utterance and then configure the answers appropriate to that scenario.


    For example, Greetings and Etiquette use cases have a lower adherence similarity, which is close to 60% (Yellow). For Policy Privacy or Disclaimer, the adherence similarity must be close to 100% (Green), because this is critical for the user to follow, depending on the use cases.

>>>>> gd2md-html alert: inline image link here (to images/image6.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

    **Note**: For **Dynamic**, configure at least one **Trigger **and one Agent **Answer **utterance for the Adherence Type.

¶

**Trigger Configuration**


    Provides the following two options to select based on the trigger created by **Agent Utterance **or **Customer Utterance **for evaluation.

* **Trigger: **Provides the following options to select and evaluate responses based on triggers created from agent or customer utterances. 
    * You can add multiple trigger utterances for conditional checks. 
    * You can define a trigger for either a customer or agent utterance and configure the corresponding responses for that scenario.

>>>>> gd2md-html alert: inline image link here (to images/image7.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

    * **Customer Utterance**: Configure the Customer Utterance that triggers the adherence check. You can enter more than one utterance using Generative AI Assistants that are similar to utterances with the same meaning. 
    * **Agent Utterance**: Configure the Agent Utterance for triggers initiated by the agent. Enter the utterances using generative AI Assistants’ suggestions that have similar utterances with the same meaning. You can add or delete multiple utterances for the Customer and the Agent.

¶

    **Trigger Detection Method**

    * **Gen AI-based Adherence: **Enter a descriptive prompt to detect contextually if the agent’s reply matches the intended goal, without relying on predefined samples.

>>>>> gd2md-html alert: inline image link here (to images/image8.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

    * **Deterministic Adherence:** This relies on predefined sample utterances and detects adherence based on semantic similarity.

>>>>> gd2md-html alert: inline image link here (to images/image9.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

        * **Utterance**: Enter sample training utterances to detect adherence.

    **Note**: This **Gen AI-based Answer Adherence** **and Trigger Detection **is activated only when the **GenAI-based agent answer adherence and customer trigger detection **option is [Enabled](https://docs.kore.ai/xo/generative-ai-tools/genai-features/) and [Published](https://docs.kore.ai/xo/deploy/publishing-bot/?h=publis#publishing-components) from the **Manage **> **Generative AI **> **GenAI Features**.** **


    **Enablement**

>>>>> gd2md-html alert: inline image link here (to images/image10.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

¶

    **Static Adherence **


    Evaluates agent adherence across all conversations where no conditional check-in is required. To get the agent's answers, you set up different acceptable utterances for a particular queue, and then configure the extent of similarities expected against the set-up utterances.

¶

**Agent Answer Configuration**

Gen AI-Based Adherence: **Use AI and natural language understanding (LLM) to detect meaning, context, and intent. This evaluates whether the agent's answer fulfills the **intents, even if phrased differently..
Deterministic Adherence: This relies on an ML-based method using the semantic similarity.
Description: Enter a prompt Description to check the agent's intent accuracy, phrasing response, and adherence.

Gen AI-Based Adherence¶

    Uses Generative AI to automatically evaluate agent responses based on natural language understanding and set criteria.

Agent Answer:
1. Select an **Answer Detection Method **to evaluate whether agents respond according to the prompt’s intent. This uses a probabilistic LLM-based method and requires no model training.
2. Enable the Gen AI-Based Adherence. Use a **Large Language Model **(LLM) to detect trigger phrases and evaluate adherence using contextual understanding.
3. Enter a prompt **Description **explaining the metric’s intent or details behind the adherence metric. This applies to all selected languages.

>>>>> gd2md-html alert: inline image link here (to images/image11.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

    **Note**:

Before assessing Gen AI-based adherence responses, ensure that the supported model and features, including Gen AI-based agent answer adherence and customer trigger detection, are enabled for the respective products in the Gen AI features section. Learn more.
No example utterances or similarity thresholds are required; LLMs evaluate adherence contextually using zero-shot prompts.
For effective prompts and LLM-based adherence detection, Learn more.

¶

    **Deterministic Adherence**


    Evaluates agent responses based on similarity to predefined sample answers.

**Agent Answer: **
1. Select **Deterministic Adherence **to assess responses based on similarity to sample answers. Encoder-based mode lets you define expected replies.

>>>>> gd2md-html alert: inline image link here (to images/image12.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

5. Define an **Answer **as a set of acceptable utterances for each queue, using **Generative AI **to generate the following automated response variations.
        * Set a similarity threshold to determine how closely user input must match expected utterances to get agent answers. 
        * Add language-specific, prompt-based evaluation parameters. 
        * Helps you find relevant answers by suggesting different ways to ask the same question. 
        * Provides expected answers that match the meaning of your question, even if it is asked in different ways.
        * Delete AI-suggested answers that are not required.

Set the **Similarity **percentage for the metric based on the defined use case and attribute.
- You can set the Similarity percentage for the desired evaluation metrics. Whether it is Static **or **Dynamic, you can configure the expected Similarity threshold.
- You should set a lower adherence similarity threshold (for example, 60%) for greetings or etiquette, and a higher adherence similarity threshold (for example, 100%) for compliance-critical (Policy Privacy or Disclaimer) statements, because it is critical for the user to follow the adherence depending on the use cases.

>>>>> gd2md-html alert: inline image link here (to images/image13.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

    **Note**: The **Similarity **threshold option is available only when **Gen AI-based Adherence** is enabled. You can configure thresholds for both **Static** and **Dynamic** evaluations.

¶

**Count Type Configuration**


The following are the key count types that you can configure:

Entire Conversation: Full interaction evaluation
Time Bound: Specific time ranges or message counts
Flexible Parameters: First/last conversation portions with configurable limits
Choose a Count Type (Entire Conversation or Time Bound) based on the selected **Adherence Type **from the dropdown.

>>>>> gd2md-html alert: inline image link here (to images/image14.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

Entire Conversation: This allows you to check for adherence at different points of conversation. It does not matter where the agent wants to check adherence throughout the conversation.
Time Bound: This evaluates adherence within a specific time range or number of messages in the interaction. It can occur at the start or end of the conversation, either for a defined number of seconds or a set number of chat messages.

>>>>> gd2md-html alert: inline image link here (to images/image15.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

Parameter: **Choose between the **First Part of Conversation **or the **Last Part of Conversation, and configure the relevant criteria to evaluate the metric.
Voice: Enter the number of seconds from the start or end of the interaction to evaluate this metric.
**Chat: **Enter the number of messages from the start or end of the interaction to evaluate this metric.
Click Create to save and activate the By Question configured adherence metric.

Managing Evaluation Metrics¶

Edit Evaluation Metrics¶

Steps to edit or delete existing Evaluation Metrics:

Right-click to select any of the existing** Evaluation Metrics **(Name) to choose a type of evaluation metric measurement.

>>>>> gd2md-html alert: inline image link here (to images/image16.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

Click **Edit **to update the required **Edit Evaluation Metrics **dialog box fields.

>>>>> gd2md-html alert: inline image link here (to images/image17.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

Click **Delete **to remove the selected evaluation metric.
Click **Update **to save the changes.

¶

**Language Dependency Warnings**


This section outlines the limitations and dependencies associated with modifying language settings in evaluation metrics.

Modification Warnings¶

You cannot remove a language if any evaluation form or attribute currently uses it.
Remove the language from all associated evaluation forms and attributes before modifying their language settings.
You can safely remove languages that are not linked to any forms or metrics. \

>>>>> gd2md-html alert: inline image link here (to images/image18.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

Delete Evaluation Metrics¶

Steps to delete existing Evaluation Metrics:

Right-click to select any of the existing** Evaluation Metrics **(Name).

Deletion Warnings¶

This section describes the warnings and prerequisites you must address before deleting a metric.

If the metric is used in any evaluation form, the system displays a warning message.
Remove the metric from all associated evaluation forms before you delete it.
If any attributes are linked to the metric, assign a different metric to those attributes before proceeding with deletion.
The system allows you to delete the metric only after resolving all dependencies.

>>>>> gd2md-html alert: inline image link here (to images/image19.png). Store image on your image server and adjust path/filename/extension if necessary.
(Back to top)(Next alert)
>>>>>

Click **Delete **to remove the selected evaluation metric.

Send Feedback

By Question¶

Access Evaluation Metrics¶

By Question¶

Adherence Type Configuration¶

** Gen AI-Based Adherence**¶

Managing Evaluation Metrics¶

Edit Evaluation Metrics¶

Modification Warnings¶

Delete Evaluation Metrics¶

Deletion Warnings¶

Gen AI-Based Adherence¶