Changing a Model
Changing a model in PR-Agent¶
See here for a list of supported models in PR-Agent.
The default model of PR-Agent is GPT-5.6 from OpenAI.
To use a different model than the default, you need to edit in the configuration file the fields:
To see which of these models actually handled a given PR, enable config.output_run_details (see Additional configurations).
For models and environments not from OpenAI, you might need to provide additional keys and other parameters. You can give parameters via a configuration file, or from environment variables.
Model-specific environment variables
See litellm documentation for the environment variables needed per model, as they may vary and change over time. Our documentation per-model may not always be up-to-date with the latest changes. Failing to set the needed keys of a specific model will usually result in litellm not identifying the model type, and failing to utilize it.
OpenAI like API¶
To use an OpenAI like API, set the following in your .secrets.toml file:
or use the environment variables (make sure to use double underscores __):
OpenAI Flex Processing¶
To reduce costs for non-urgent/background tasks, enable Flex Processing:
See OpenAI Flex Processing docs for details.
Azure¶
To use Azure, set in your .secrets.toml (working from CLI), or in the GitHub Settings > Secrets and variables (working from GitHub App or GitHub Action):
[openai]
key = "" # your azure api key
api_type = "azure"
api_version = '2023-05-15' # Check Azure documentation for the current API version
api_base = "" # The base URL for your Azure OpenAI resource. e.g. "https://<your resource name>.openai.azure.com"
deployment_id = "" # The deployment name you chose when you deployed the engine
and set in your configuration file:
To use Azure AD (Entra id) based authentication set in your .secrets.toml (working from CLI), or in the GitHub Settings > Secrets and variables (working from GitHub App or GitHub Action):
[azure_ad]
client_id = "" # Your Azure AD application client ID
client_secret = "" # Your Azure AD application client secret
tenant_id = "" # Your Azure AD tenant ID
api_base = "" # Your Azure OpenAI service base URL (e.g., https://openai.xyz.com/)
Passing custom headers to the underlying LLM Model API can be done by setting extra_headers parameter to litellm.
[litellm]
extra_headers='{"projectId": "<authorized projectId >", ...}') #The value of this setting should be a JSON string representing the desired headers, a ValueError is thrown otherwise.
This enables users to pass authorization tokens or API keys, when routing requests through an API management gateway.
Ollama¶
You can run models locally through either VLLM or Ollama
E.g. to use a new model locally via Ollama, set in .secrets.toml or in a configuration file:
[config]
model = "ollama/qwen2.5-coder:32b"
fallback_models=["ollama/qwen2.5-coder:32b"]
custom_model_max_tokens=128000 # set the maximal input tokens for the model
duplicate_examples=true # will duplicate the examples in the prompt, to help the model to generate structured output
[ollama]
api_base = "http://localhost:11434" # or whatever port you're running Ollama on
By default, Ollama uses a context window size of 2048 tokens. In most cases this is not enough to cover pr-agent prompt and pull-request diff. Context window size can be overridden with the OLLAMA_CONTEXT_LENGTH environment variable. For example, to set the default context length to 8K, use: OLLAMA_CONTEXT_LENGTH=8192 ollama serve. More information you can find on the official ollama faq.
Please note that the custom_model_max_tokens setting should be configured in accordance with the OLLAMA_CONTEXT_LENGTH. Failure to do so may result in unexpected model output.
Local models vs commercial models
PR-Agent is compatible with almost any AI model, but analyzing complex code repositories and pull requests requires a model specifically optimized for code analysis.
Commercial models such as GPT-5, Claude Sonnet, and Gemini have demonstrated robust capabilities in generating structured output for code analysis tasks with large input. In contrast, most open-source models currently available (as of January 2025) face challenges with these complex tasks.
Based on our testing, local open-source models are suitable for experimentation and learning purposes (mainly for the ask command), but they are not suitable for production-level code analysis tasks.
Hence, for production workflows and real-world usage, we recommend using commercial models.
Hugging Face¶
To use a new model with Hugging Face Inference Endpoints, for example, set:
[config] # in configuration.toml
model = "huggingface/meta-llama/Llama-2-7b-chat-hf"
fallback_models=["huggingface/meta-llama/Llama-2-7b-chat-hf"]
custom_model_max_tokens=... # set the maximal input tokens for the model
[huggingface] # in .secrets.toml
key = ... # your Hugging Face api key
api_base = ... # the base url for your Hugging Face inference endpoint
(you can obtain a Llama2 key from here)
Replicate¶
To use Llama2 model with Replicate, for example, set:
[config] # in configuration.toml
model = "replicate/llama-2-70b-chat:2c1608e18606fad2812020dc541930f2d0495ce32eee50074220b87300bc16e1"
fallback_models=["replicate/llama-2-70b-chat:2c1608e18606fad2812020dc541930f2d0495ce32eee50074220b87300bc16e1"]
[replicate] # in .secrets.toml
key = ...
(you can obtain a Llama2 key from here)
Also, review the .secrets_template.toml file for instructions on how to set keys for other models.
Groq¶
To use Llama3 model with Groq, for example, set:
[config] # in configuration.toml
model = "llama3-70b-8192"
fallback_models = ["groq/llama3-70b-8192"]
[groq] # in .secrets.toml
key = ... # your Groq api key
(you can obtain a Groq key from here)
SambaNova¶
To use MiniMax-M3 model with SambaNova, for example, set:
[config] # in configuration.toml
model = "sambanova/MiniMax-M3"
fallback_models = ["sambanova/MiniMax-M2.7"]
[sambanova] # in .secrets.toml
key = ... # your SambaNova api key
(you can obtain a SambaNova key from here)
xAI¶
To use xAI's models with PR-Agent, set:
[config] # in configuration.toml
model = "xai/grok-2-latest"
fallback_models = ["xai/grok-2-latest"] # or any other model as fallback
[xai] # in .secrets.toml
key = "..." # your xAI API key
You can obtain an xAI API key from xAI's console by creating an account and navigating to the developer settings page.
Vertex AI¶
To use Google's Vertex AI platform and its associated models (chat-bison/codechat-bison) set:
[config] # in configuration.toml
model = "vertex_ai/codechat-bison"
fallback_models="vertex_ai/codechat-bison"
[vertexai] # in .secrets.toml
vertex_project = "my-google-cloud-project"
vertex_location = ""
Your application default credentials will be used for authentication so there is no need to set explicit credentials in most environments.
If you do want to set explicit credentials, then you can use the GOOGLE_APPLICATION_CREDENTIALS environment variable set to a path to a json credentials file.
Google AI Studio¶
To use Google AI Studio models, set the relevant models in the configuration section of the configuration file:
[config] # in configuration.toml
model="gemini/gemini-1.5-flash"
fallback_models=["gemini/gemini-1.5-flash"]
[google_ai_studio] # in .secrets.toml
gemini_api_key = "..."
If you don't want to set the API key in the .secrets.toml file, you can set the GOOGLE_AI_STUDIO.GEMINI_API_KEY environment variable.
Anthropic¶
To use Anthropic models, set the relevant models in the configuration section of the configuration file:
[config]
model="anthropic/claude-3-opus-20240229"
fallback_models=["anthropic/claude-3-opus-20240229"]
And also set the api key in the .secrets.toml file:
See litellm documentation for more information about the environment variables required for Anthropic.
Amazon Bedrock¶
To use Amazon Bedrock and its foundational models, add the below configuration:
[config] # in configuration.toml
model="bedrock/anthropic.claude-3-5-sonnet-20240620-v1:0"
fallback_models=["bedrock/anthropic.claude-3-5-sonnet-20240620-v1:0"]
[aws]
AWS_ACCESS_KEY_ID="..."
AWS_SECRET_ACCESS_KEY="..."
AWS_REGION_NAME="..."
You can also use the new Meta Llama 4 models available on Amazon Bedrock:
[config] # in configuration.toml
model="bedrock/us.meta.llama4-scout-17b-instruct-v1:0"
fallback_models=["bedrock/us.meta.llama4-maverick-17b-instruct-v1:0"]
Grok 4.3 is available through Amazon Bedrock Mantle rather than the classic Bedrock runtime:
[config] # in configuration.toml
model="bedrock_mantle/xai.grok-4.3"
fallback_models=["bedrock_mantle/xai.grok-4.3"]
Bedrock Mantle uses the same AWS credential sources, but its IAM permissions differ from the classic runtime. See the AWS Mantle inference permissions.
Using IAM Role Credentials (Recommended on AWS Compute)¶
When running PR-Agent on AWS infrastructure (EC2, ECS/Fargate, EKS with IRSA, Lambda, or any self-hosted GitHub Actions runner on AWS), the instance or task already has an IAM role attached. You can use those ambient credentials directly instead of storing long-lived static keys.
Set AWS_USE_IMDS=true in the environment. PR-Agent will resolve credentials via boto3's standard provider chain, which handles all AWS compute contexts transparently:
| Compute context | Mechanism |
|---|---|
| EC2 instance with IAM role | IMDSv2 (169.254.169.254) |
| ECS / Fargate task role | Task metadata endpoint |
| EKS pod with IRSA | Web identity token + STS |
| Lambda function | Runtime-injected credentials |
Minimal GitHub Actions workflow (no AWS secret keys required):
- uses: the-pr-agent/pr-agent@main
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
AWS_USE_IMDS: "true"
# AWS_REGION_NAME: us-east-1 # optional if the instance metadata provides it
with:
command: review
The IAM role must have bedrock:InvokeModel permission on the target model ARN, for example:
{
"Effect": "Allow",
"Action": "bedrock:InvokeModel",
"Resource": "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20240620-v1:0"
}
If you also configure static keys in [aws], they serve as an automatic fallback: if the ambient credentials fail a Bedrock call (e.g., the role lacks bedrock:InvokeModel), PR-Agent retries with the static keys and logs a warning.
Custom Inference Profiles¶
To invoke an application inference profile with a classic bedrock/ model (for cost allocation tags and other configuration settings), set model_id to the profile ARN in your configuration:
[config] # in configuration.toml
model="bedrock/anthropic.claude-3-5-sonnet-20240620-v1:0"
fallback_models=["bedrock/anthropic.claude-3-5-sonnet-20240620-v1:0"]
[aws]
AWS_ACCESS_KEY_ID="..."
AWS_SECRET_ACCESS_KEY="..."
AWS_REGION_NAME="..."
[litellm]
model_id = "your-application-inference-profile-arn"
The litellm.model_id parameter applies only to classic bedrock/ calls made through the bedrock-runtime APIs. It does not apply to bedrock_mantle/; for cost allocation with the Mantle Chat Completions and Responses APIs, use Amazon Bedrock Projects.
Using a Custom VPC Endpoint (PrivateLink)¶
To route Bedrock traffic through a VPC interface endpoint instead of the public bedrock-runtime endpoint, set AWS_BEDROCK_RUNTIME_ENDPOINT either as an environment variable or in [aws]:
See litellm documentation for more information about the environment variables required for Amazon Bedrock.
DeepSeek¶
To use deepseek-v4 model with DeepSeek, for example, set:
[config] # in configuration.toml
model = "deepseek/deepseek-v4-pro"
fallback_models=["deepseek/deepseek-v4-flash"]
and fill up your key
(you can obtain a deepseek-v4 key from here)
GLM (Z.AI)¶
To use GLM models with Z.AI (Zhipu), for example, set:
and fill up your key
(you can obtain a Z.AI API key from here)
Kimi (Moonshot)¶
To use Kimi models with Moonshot, for example, set:
and fill up your key
(you can obtain a Moonshot API key from here)
If you are on the China endpoint instead, add api_base = "https://api.moonshot.cn/v1" under [moonshot].
Qwen (DashScope)¶
To use Qwen models with Alibaba DashScope, for example, set:
[config] # in configuration.toml
model = "dashscope/qwen3.8-max"
fallback_models=["dashscope/qwen3.8-max"]
and fill up your key
(you can obtain a DashScope API key from here)
Xiaomi MiMo¶
To use Xiaomi MiMo models, for example, set:
[config] # in configuration.toml
model = "xiaomi_mimo/mimo-v2.5"
fallback_models=["xiaomi_mimo/mimo-v2.5"]
and fill up your key
(you can obtain a Xiaomi MiMo API key from here)
DeepInfra¶
To use DeepSeek model with DeepInfra, for example, set:
[config] # in configuration.toml
model = "deepinfra/deepseek-ai/DeepSeek-R1-Distill-Llama-70B"
fallback_models = ["deepinfra/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B"]
[deepinfra] # in .secrets.toml
key = ... # your DeepInfra api key
(you can obtain a DeepInfra key from here)
Mistral¶
To use models like Mistral or Codestral with Mistral, for example, set:
[config] # in configuration.toml
model = "mistral/mistral-small-latest"
fallback_models = ["mistral/mistral-medium-latest"]
[mistral] # in .secrets.toml
key = "..." # your Mistral api key
(you can obtain a Mistral key from here)
Codestral¶
To use Codestral model with Codestral, for example, set:
[config] # in configuration.toml
model = "codestral/codestral-latest"
fallback_models = ["codestral/codestral-2405"]
[codestral] # in .secrets.toml
key = "..." # your Codestral api key
(you can obtain a Codestral key from here)
Databricks¶
To use a model hosted on Databricks (e.g. an Azure Databricks serving endpoint), set:
[config] # in configuration.toml
model = "databricks/databricks-claude-sonnet-4"
fallback_models = ["databricks/databricks-claude-sonnet-4"]
[databricks] # in .secrets.toml
api_key = "..." # your Databricks personal access token (PAT)
api_base = "https://adb-xxxx.azuredatabricks.net/serving-endpoints" # your workspace serving-endpoints URL
The model name after the databricks/ prefix is the name of your serving endpoint. See LiteLLM's Databricks provider docs for details.
Openrouter¶
To use model from Openrouter, for example, set:
[config] # in configuration.toml
model="openrouter/anthropic/claude-3.7-sonnet"
fallback_models=["openrouter/deepseek/deepseek-chat"]
custom_model_max_tokens=20000
[openrouter] # in .secrets.toml or passed an environment variable openrouter__key
key = "..." # your openrouter api key
(you can obtain an Openrouter API key from here)
Openrouter provider routing, reasoning and output cap¶
For openrouter/... models you can optionally restrict which upstream providers Openrouter uses, control reasoning, and cap the completion length. All keys live in the [openrouter] section of configuration.toml. Models listed in SUPPORT_REASONING_EFFORT_MODELS inherit config.reasoning_effort unless an Openrouter-specific effort or token budget is set.
[openrouter]
# Uncomment and adjust the keys you need; unset keys keep Openrouter's defaults.
# provider_only = ["z-ai"] # hard allowlist of upstream providers; empty = default routing
# provider_order = ["z-ai", "novita"] # preferred order instead of an allowlist; ignored when provider_only is set
# allow_fallbacks = true # when provider_order is set, allow routing beyond the list
# reasoning_effort = "low" # override global effort: "none", "minimal", "low", "medium", "high", "xhigh" or "max"
# reasoning_max_tokens = 2048 # explicit budget; takes precedence over effort unless effort is "none"
# max_tokens = 16000 # hard cap on completion tokens for the request
provider_only and reasoning_effort = "none" are useful to pin a specific provider and to bound the cost of reasoning models. Because Openrouter treats effort and token budgets as mutually exclusive, an explicit Openrouter-specific "none" keeps reasoning disabled; otherwise a positive reasoning_max_tokens value takes precedence over the global effort and other Openrouter-specific values. Invalid Openrouter-specific effort values are warned about and treated as unset, so registered reasoning models fall back to config.reasoning_effort. Openrouter normalizes "max" to "xhigh" in this path to match LiteLLM 1.98.0. Supported effort values vary by model, and models whose metadata marks reasoning as mandatory reject "none". For Anthropic models using a reasoning budget, set the effective output max_tokens higher than reasoning_max_tokens so the final answer has output headroom. See the Openrouter provider routing and reasoning tokens docs.
OrcaRouter¶
OrcaRouter is an OpenAI-compatible AI gateway. It needs no provider-specific code in PR-Agent: the openai/ prefix routes the request to OrcaRouter's base URL through litellm's OpenAI-compatible path, the same way the Neon AI Gateway is handled.
To use a model through OrcaRouter, set:
[config] # in configuration.toml
model = "openai/anthropic/claude-fable-5"
fallback_models = ["openai/auto"]
custom_model_max_tokens = 20000
[openai] # in .secrets.toml
api_base = "https://api.orcarouter.ai/v1"
key = "..." # your OrcaRouter api key
or use the environment variables (make sure to use double underscores __):
(you can obtain an OrcaRouter API key from here)
Keep the openai/ prefix on the model name, whatever OrcaRouter model ID you use (openai/anthropic/claude-fable-5, openai/auto, ...): the prefix routes the request through litellm's OpenAI-compatible path. A prefixed name is not in the MAX_TOKENS table here, so you also have to set custom_model_max_tokens. OrcaRouter governs routing and guardrails itself, but config.reasoning_effort still reaches it: PR-Agent matches the last segment of the model ID against SUPPORT_REASONING_EFFORT_MODELS, so an ID such as openai/google/gemini-2.5-pro or openai/o3 sends the configured effort (default "medium") even with nothing set. The example IDs above are not in that list and are unaffected.
Neon AI Gateway¶
Neon AI Gateway is an OpenAI-compatible inference gateway. Each Neon branch has its own gateway host, so the base URL points at a single branch and not at an account. Neon publishes that host alongside the credential as NEON_AI_GATEWAY_BASE_URL. The value has no path, so append /v1 to reach chat completions.
To use a model served by a Neon branch, set:
[config] # in configuration.toml
model = "openai/gpt-5-mini"
fallback_models = ["openai/gpt-5-mini"]
custom_model_max_tokens = 400000 # the context window Neon publishes for the model
[openai] # in .secrets.toml
api_base = "https://<your-neon-branch-host>/v1"
key = "..." # your Neon AI Gateway credential
or use the environment variables (make sure to use double underscores __):
Keep the openai/ prefix on the model name, whichever Neon model ID you use: the prefix routes the request through litellm's OpenAI-compatible path. A prefixed name is not in the MAX_TOKENS table here, so you also have to set custom_model_max_tokens. Take the value from Neon's model catalog.
Create the credential per branch in the Neon Console with the ai_gateway:invoke scope. The credential also works on branches descended from the one it was created on. The gateway is in beta and requires a paid Neon plan. It runs only in AWS US East (Ohio), aws-us-east-2.
Chat completions only
Some model IDs in Neon's catalog are served through the OpenAI Responses API, which Neon exposes under /openai/v1 instead of /v1. The configuration above points at the chat-completions endpoint, so it cannot reach those models. Neon also documents a few models that return message.content as an array of typed blocks rather than a string, and PR-Agent reads the reply as a string.
Custom models¶
If the relevant model doesn't appear here, you can still use it as a custom model:
- Set the model name in the configuration file:
- Set the maximal tokens for the model:
-
Go to litellm documentation, find the model you want to use, and set the relevant environment variables.
-
Most reasoning models do not support chat-style inputs (
systemandusermessages) or temperature settings. To bypass chat templates and temperature controls, setconfig.custom_reasoning_model = truein your configuration file.
Dedicated parameters¶
OpenAI models¶
With the OpenAI models that support reasoning effort (eg: gpt-5.6-terra), you can specify its reasoning effort via config section. The default value is medium. You can change it to any supported value based on your usage. Available values depend on the model and provider.
Anthropic models¶
[config]
enable_claude_extended_thinking = false # Set to true to enable extended thinking feature
extended_thinking_budget_tokens = 2048
extended_thinking_max_output_tokens = 4096
By default, PR-Agent applies the extended-thinking payload only to a built-in list of Claude models
(see CLAUDE_EXTENDED_THINKING_MODELS in pr_agent/algo/__init__.py). If you use a newer or custom
Claude model that is not in that list, you can override it:
When claude_extended_thinking_models_override is non-empty, it fully replaces the built-in list, so
include every model that should receive extended thinking. Leave it empty (the default) to use the
built-in defaults.
Only models that accept a thinking budget are supported
PR-Agent enables extended thinking through the manual
thinking={"type": "enabled", "budget_tokens": ...} request. Adaptive-only Claude models
(e.g. Opus 4.7/4.8, Sonnet 5, Fable 5) reject budget_tokens and will error if you add them to
the list — they are intentionally excluded from the built-in defaults.
Output token limit¶
By default PR-Agent does not send an output token limit (max_tokens) on model calls, so the
provider's own default applies. On some providers that default is low — for example, AWS Bedrock
(Converse API) can cap Claude reasoning models at 4096 output tokens, and since reasoning tokens
count against that budget, the visible answer can come back empty or truncated. Set
config.max_output_tokens to a positive value (e.g. 16000) to send it as max_tokens on every
completion call. When Claude extended thinking is enabled, extended_thinking_max_output_tokens
takes precedence.
For models with small context windows, keep in mind that prompt and completion tokens share the
model's context window: size config.max_model_tokens so the packed prompt leaves room for the
configured output limit.