LLM and Generative AI Usage Logs¶
The LLM and Gen AI Usage Logs on the XO Platform provide detailed information about requests sent to LLMs and the corresponding responses. The logs include data on features accessing the LLMs, response generation time, payload details, tokens used, and more. It enables bot designers to track and compare usage across various LLM features and refine prompts and settings to boost performance and user experience.
The log analysis focuses on the following key areas:
- Request-response dynamics: Analysis of request-response dynamics between user prompts and model responses offers insights into prompt and model performance in specific scenarios.
- Payload details: Analyzing the payload data exchanged during interactions allows for effective monitoring and optimization of advanced AI functionalities.
To access the logs, Go to Analytics > Gen AI Analytics > Usage Logs. Click any record to view the log summary and payload details.
Field Description¶
You can sort the data by either Newest to Oldest or Oldest to Newest. Click the record to view the Summary and Payload Details.
Summary¶
| Fields | Description |
| Overview | |
| Description | Extra details about the node and task name linked to the feature. |
| Date & Time | The timestamp of the call made to the LLM. |
| Language | The language in which the conversation occurred.
If it is a multi-lingual VA, you can select specific languages to filter the conversations that occurred in those languages. The page shows the conversations that happen in all enabled languages by default. |
| Channel | The communication channel or platform used for the interaction with LLM. |
| Session ID | Identifier for the session. |
| Status | Status of a call made to the LLM. “Success” or “Failure”. |
| User Details | |
| User | Bot designer or end user who made a call to the LLM. |
| User ID | The distinct identifier of the end user engaged in the conversation.
You can view the metrics based on the Kore User id or Channel User Id. Channel-specific ids are shown only for the users who have interacted with the VA during the selected period. |
| Generative AI | |
| Feature | The XO Platform feature (Co-Pilot and Dynamic conversation features) is making calls to the LLM models. |
| Model | The Large Language Model to which the request was made. |
| Prompt Name | Prompt used with the model and running at the node/task level. The pre-built prompts are named ‘Default. |
| Request Tokens | Request tokens for a Language Model (LLM) are the individual parts of input text, like words or punctuation, given to the model to create a response. These tokens are the basis for the model’s understanding and its output generation. |
| Integration Type | Type of integration used (e.g. System/Custom). |
| Response Duration | Time taken by the LLM to generate the response. |
| Response Tokens | Response tokens for a Large Language Model (LLM) are the pieces of generated output, like words or punctuation, showing the model’s response. These tokens make up the structured parts of the LLM’s text, making it easier to understand and analyze. |
| Guardrails | |
| Configured Guardrails | Restrict Toxicity, Restrict Topics, Detect Prompt Injections, Filter Response. |
| Outcome | Indicates whether the guardrail was Detected or Not Detected or Not Applicable. |
| Risk Score | Calculated risk score on a scale of 0 to 1. If not detected, it will be 0. |
Note
The Guardrail details are displayed only if they are configured.
Payload Details¶
| Fields | Description |
| Request Payload | The request payload sent to a Large Language Model (LLM) is the user’s input or question, along with any extra details needed for the model to give a good response. |
| Response Payload | The Large Language Model (LLM) produces a response payload as its answer to the input it receives. It’s in the text format and contains additional information required to present the response. This payload helps developers use the model’s output effectively. | Tokens Used | Response tokens for a Large Language Model (LLM) are the pieces of generated output, like words or punctuation, showing the model’s response. These tokens make up the structured parts of the LLM’s text, making it easier to understand and analyze. | Stream | If the stream = true, then the response is delivered incrementally, token by token in real-time. |
Filter Criteria¶
The LLM and GenAI logs data can be viewed based on specific filter criteria that can be selected. Learn more.