Installing LLM Engine

The LLM Engine is a microservice that enables LLM-based policy evaluation in the Witboost Computational Governance Platform. It connects to Azure OpenAI to evaluate resource descriptors against natural language policies.

Before proceeding, make sure to read the requirements.

Prerequisites

• An Azure OpenAI instance with a deployed model (GPT-5.2 or later is recommended).
• The API key for your Azure OpenAI instance, stored in your secret store.

Installation

In order to install the LLM Engine, your secret store needs to have the following variable filled:

AZURE_OPENAI_API_KEY=<azure_openai_api_key>

Additionally, you need to enable and configure the llm-engine module in the Witboost configuration file (values.yaml):

...
llm-engine:
  enabled: true
  configOverride: |
    provider: azure_openai
    endpoint: <azure_openai_endpoint>
    deployment: <azure_openai_deployment_name>
    api_version: "2024-12-01-preview"   # Optional — defaults to "2023-05-15"
  extraEnvVars:
    - name: AZURE_OPENAI_API_KEY
      valueFrom:
        secretKeyRef:
          name: <secret_name>
          key: <secret_key>
...

Parameter	Description
endpoint	The endpoint URL of your Azure OpenAI instance (e.g. https://my-instance.openai.azure.com/).
deployment	The name of the deployed model in Azure OpenAI (e.g. gpt-5.2-chat).
api_version	The Azure OpenAI API version (e.g. 2024-12-01-preview). Optional — defaults to "2023-05-15" if omitted.

Once deployed, you also need to enable the LLM engine option in the Witboost UI configuration. See the LLM Engine configuration section for details.

For more information on how LLM policies work, see LLM Engine.