LLM Topics

Freeform prompt design guide

Implementation Background

LLM Topics rely on receiving back structured output from the OpenAI gpt-4o-mini model via the function calling feature.

We define two parameters, reason and verdict. verdict is a boolean which is true if the LLM believes there is a match, false otherwise. reason is a Chain-of-Thought explanation of why the LLM has made the choice of verdict.

When a new message is analyzed, the LLM topic's prompt is sent as the system message. The model is then prompted with a user message containing the ingested conversation message. We send instructions to OpenAI via the tools parameter to always respond with a function call.

LLM Message Diagram

The parameters are recorded below for reference.

parameters: {
  type: "object",
  properties: {
    reason: {
      type: "string",
      description: "The reason for the verdict"
    },
    verdict: {
      type: "boolean",
      description: "Does the message fulfill the requirements of the given task"
    }
  },
  required: ["reason", "verdict"]
}