For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Dynamically compute and set LLM request fields using CEL expressions.

Use LLM request transformations to dynamically compute and set fields in LLM requests using Common Expression Language (CEL)CEL (Common Expression Language)A simple expression language used throughout agentgateway to enable flexible configuration. CEL expressions can access request context, JWT claims, and other variables to make dynamic decisions. expressions. Transformations let you enforce policies such as capping token usage or conditionally modifying request parameters, without changing client code.

Use the following table to choose the field for what you want to change.

To learn more about CEL, see the following resources:

• CEL expression reference
• cel.dev tutorial

Before you begin

Configure LLM request transformations

1. Create a configuration file with your LLM transformation settings. The following example caps max_tokens to 10, regardless of what the client requests.

   cat <<'EOF' > config.yaml
   # yaml-language-server: $schema=https://agentgateway.dev/schema/config
   llm:
     models:
     - name: "*"
       provider: openAI
       params:
         apiKey: "$OPENAI_API_KEY"
       transformation:
         max_tokens: "min(llmRequest.max_tokens, 10)"
   EOF

   Setting	Description
   transformation	A map of LLM request field names to CEL expressions. Each key is the field to set; each value is a CEL expression evaluated against the original request. Use the llmRequest variable to access the original LLM request body.

   Note

   Transformations take priority over overrides for the same field. If an expression fails to evaluate, the field is silently removed from the request.

2. Start agentgateway.

   agentgateway -f config.yaml

3. Send a request with max_tokens set to a value greater than 1024. The transformation caps it to 10 before the request reaches the LLM provider.

   curl -s 'http://localhost:4000/v1/chat/completions' \
   --header 'Content-Type: application/json' \
   --data '{
     "model": "gpt-3.5-turbo",
     "max_tokens": 5000,
     "messages": [
       {
         "role": "user",
         "content": "Tell me a short story"
       }
     ]
   }' | jq .

   Example output:

   {"model":"gpt-3.5-turbo-0125","usage":
   {"prompt_tokens":12,"completion_tokens":10,
   "total_tokens":22,"completion_tokens_details":
   {"reasoning_tokens":0,"audio_tokens":0,
   "accepted_prediction_tokens":0,
   "rejected_prediction_tokens":0},"prompt_tokens_details":
   {"cached_tokens":0,"audio_tokens":0}},"choices":
   [{"message":{"content":"Once upon a time, in a quaint
   village nestled","role":"assistant","refusal":null,
   "annotations":[]},"index":0,"logprobs":null,
   "finish_reason":"length"}],
   "id":"chatcmpl-DHyGUsdgf2P5FidTbZIZFxdVGRfpq",
   "object":"chat.completion","created":1773175606,
   "service_tier":"default","system_fingerprint":null}%
   

   In the response, the completion_tokens value reflects a completion capped at 10 tokens.

Conditionally set fields based on headers

Use a CEL expression in the model-level transformation field to dynamically set max_tokens based on the caller’s identity from a request header. This example gives admin users a higher token limit than regular users.

cat <<'EOF' > config.yaml
# yaml-language-server: $schema=https://agentgateway.dev/schema/config

llm:
  models:
  - name: "*"
    provider: openAI
    params:
      apiKey: "$OPENAI_API_KEY"
    transformation:
      max_tokens: "request.headers['x-user-id'] == 'admin' ? 100 : 10"
EOF

If agentgateway is still running from the previous section, it reloads config.yaml automatically when you save the file. Otherwise, start agentgateway.

agentgateway -f config.yaml

Send a request as an admin user and verify the response uses the higher token limit.

curl -s http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-user-id: admin" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [{"role": "user", "content": "Tell me a story"}]
  }' | jq .

Send a request as a regular user and verify the response is capped at the lower token limit.

curl -s http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-user-id: alice" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [{"role": "user", "content": "Tell me a story"}]
  }' | jq .

In the responses, the admin user receives up to 100 completion tokens while the regular user is capped at 10.

Transform requests after provider conversion

The transformation field runs before agentgateway converts the request into the format that the provider expects. To write one, you must know how the conversion works, and you cannot change a field that the conversion itself adds.

The finalTransformation field runs after the conversion instead. You only need to know the shape of the target API.

The difference shows in the field names. When a client sends max_tokens to an OpenAI provider, the conversion renames the field to max_completion_tokens, so the provider receives the following request body.

{
  "model": "gpt-3.5-turbo",
  "max_completion_tokens": 5000,
  "messages": [{"role": "user", "content": "Tell me a short story"}]
}

A transformation entry must target max_tokens, the name that the client sent. A finalTransformation entry must target max_completion_tokens, the name that the provider receives.

1. Create a configuration file that caps the converted field and removes a field from the provider request. The min() expression caps max_completion_tokens at 10. The fail("remove") expression always fails to evaluate, which deletes reasoning_effort from the request.

   cat <<'EOF' > config.yaml
   # yaml-language-server: $schema=https://agentgateway.dev/schema/config
   llm:
     models:
     - name: "*"
       provider: openAI
       params:
         apiKey: "$OPENAI_API_KEY"
       finalTransformation:
         max_completion_tokens: "min(llmRequest.max_completion_tokens, 10)"
         reasoning_effort: 'fail("remove")'
   EOF

   Setting	Description
   finalTransformation	A map of provider request field names to CEL expressions. Each key is the field to set in the converted request; each value is a CEL expression. Entries take priority over overrides for the same field.

   Warning

   In a finalTransformation expression, llmRequest is the converted request body, not the request that the client sent. An expression that reads a field which the converted body does not have, such as llmRequest.max_tokens for an OpenAI provider, fails to evaluate. A failed expression removes the target field, so a mistyped field name silently deletes the field that you meant to set. For more information about the expression language, see the CEL reference.

2. If agentgateway is still running from a previous section, it reloads config.yaml automatically when you save the file. Otherwise, start agentgateway.

   agentgateway -f config.yaml

3. Send a request that sets max_tokens and reasoning_effort. The conversion renames max_tokens to max_completion_tokens, and the transformation then caps that field at 10 and drops reasoning_effort.

   curl -s 'http://localhost:4000/v1/chat/completions' \
   --header 'Content-Type: application/json' \
   --data '{
     "model": "gpt-3.5-turbo",
     "max_tokens": 5000,
     "reasoning_effort": "high",
     "messages": [
       {
         "role": "user",
         "content": "Tell me a short story"
       }
     ]
   }' | jq .

   Example output:

   {"model":"gpt-3.5-turbo-0125","usage":
   {"prompt_tokens":12,"completion_tokens":10,
   "total_tokens":22},"choices":
   [{"message":{"content":"Once upon a time, in a quaint
   village nestled","role":"assistant"},"index":0,
   "finish_reason":"length"}],
   "id":"chatcmpl-DHyGUsdgf2P5FidTbZIZFxdVGRfpq",
   "object":"chat.completion","created":1773175606}%
   

   The completion_tokens value reflects a completion capped at 10 tokens, which confirms that the transformation reached the converted request.

Modify the HTTP request to the provider

Use requestTransformation to change the HTTP request that agentgateway sends to the provider, such as its headers or metadata. For a header with a fixed value, use requestHeaders instead, which needs no CEL. If both set the same header, the requestHeaders value is sent, because requestHeaders runs after requestTransformation.

The requestTransformation expressions can read the HTTP request, such as request.headers. They cannot read the LLM request body, because the llmRequest CEL variable is not set for them. To read or change fields in the LLM request body, use transformation or finalTransformation. You can set either one together with requestTransformation on the same provider or model.

1. Create a configuration file that selects an OpenAI project from a client header. The provider default sets the openai-project header based on the x-team request header, and removes x-team so that it is not sent to OpenAI. The gpt-4o-batch model sets its own requestTransformation, which replaces the provider default for that model.

   Warning

   Because a model-level requestTransformation replaces the provider default completely, the model loses every setting in the default, not only the ones it overrides. In the following example, gpt-4o-batch repeats remove: [x-team]. Without that line, requests to gpt-4o-batch would send the x-team header to OpenAI.

   cat <<'EOF' > config.yaml
   # yaml-language-server: $schema=https://agentgateway.dev/schema/config
   llm:
     providers:
     - name: openai-prod
       provider: openAI
       params:
         apiKey: "$OPENAI_API_KEY"
       defaults:
         requestTransformation:
           set:
             openai-project: '"x-team" in request.headers && request.headers["x-team"] == "research" ? "proj_research" : "proj_default"'
           remove:
           - x-team
     models:
     - name: gpt-4o
       provider:
         reference: openai-prod
       params:
         model: gpt-4o
     - name: gpt-4o-batch
       provider:
         reference: openai-prod
       params:
         model: gpt-4o-mini
       requestTransformation:
         set:
           openai-project: '"proj_batch"'
         remove:
         - x-team
   EOF

   Setting	Description
   llm.providers[].defaults.requestTransformation	The HTTP request transformation for every model that references the provider and does not set its own requestTransformation.
   llm.models[].requestTransformation	The HTTP request transformation for one model. If set, it replaces the provider default completely. The two are not merged.
   set	Headers to set. Each value is a CEL expression. To set a fixed string, quote the string inside the expression, such as '"proj_batch"'.
   add	Headers to append. Each value is a CEL expression.
   remove	Header names to remove.
   replace	A CEL expression that returns the complete set of headers. It is applied before add, set, and remove.
   body	A CEL expression that returns a replacement request body. The replacement happens before agentgateway converts the request to the provider format, and before transformation and finalTransformation run, so the new body must be a valid request for the API that the client called.
   metadata	Metadata values to add. Each value is a CEL expression.

2. If agentgateway is still running from a previous section, it reloads config.yaml automatically when you save the file. Otherwise, start agentgateway.

   agentgateway -f config.yaml

Available CEL variables

You can use these variables in your CEL transformation expressions.

For a complete list of available variables and functions, see the CEL reference documentation.

Common transformation patterns

Cap token usage

Enforce a maximum token limit regardless of what the client requests.

llm:
  models:
  - name: "*"
    provider: openAI
    params:
      apiKey: "$OPENAI_API_KEY"
    transformation:
      max_tokens: "min(llmRequest.max_tokens, 1024)"

Set temperature based on headers

Allow callers to control creativity through a header while enforcing bounds.

llm:
  models:
  - name: "*"
    provider: openAI
    params:
      apiKey: "$OPENAI_API_KEY"
    transformation:
      temperature: "request.headers['x-creativity'] == 'high' ? 0.9 : 0.1"

Combine multiple transformations

Apply several field-level transformations in a single configuration.

llm:
  models:
  - name: "*"
    provider: openAI
    params:
      apiKey: "$OPENAI_API_KEY"
    transformation:
      max_tokens: "request.headers['x-user-tier'] == 'premium' ? 4096 : 256"
      temperature: "request.headers['x-user-tier'] == 'premium' ? 0.8 : 0.3"

Next steps

• Learn about CEL expressions for advanced expression logic.
• Set up authentication to use JWT claims in transformations.