AI-Models
The AI-Models page is where an administrator configures the chat models that msg.ZenTestAI can use. Multiple models can be configured per tenant — typically a cheap one for high-volume tasks (element identification, simple decisions), a vision-capable one for assertions, and a high-end reasoning model for the AI Assistant.
Models are configured per tenant (product). There is no global model registry.
Where to configure
Configure AI models in Administration → Select your Tenant → AI tab. The page lists every configured model and lets you add new ones with the + action.
After saving, configured models become selectable in:
- The default-model fields on Tenant Settings (assertions, element identification, step identification, agents).
- The per-step AI model override on the Steps tab.
Supported vendors
msg.ZenTestAI ships connectors for seven vendor platforms. Pick a vendor first; the form then adapts to show only the fields relevant to that vendor.
| Vendor | Use for |
|---|---|
| OpenAI | OpenAI's public API (api.openai.com or a compatible reverse proxy). |
| Azure OpenAI | OpenAI models hosted in your Azure subscription as deployments. |
| Anthropic | Anthropic's public API (api.anthropic.com or compatible relay). |
| AWS Bedrock | Anthropic Claude (and others) hosted via AWS Bedrock. |
| Google Gemini | Google's public Gemini API (generativelanguage.googleapis.com). |
| Google Vertex AI | Gemini hosted on Google Cloud Vertex AI. |
| Azure AI Studio | Llama-family and other models served from Azure AI Studio deployments. |
Supported models
Each vendor exposes a curated list of base models. The list below is the current set at the time of writing — the actual dropdown in your tenant reflects what the platform ships.
OpenAI / Azure OpenAI
| Base model | Notes |
|---|---|
gpt-5, gpt-5.2 | Latest GPT-5 generation. Reasoning-capable. |
gpt-5-mini, gpt-5-nano | Lower-cost GPT-5 variants. Reasoning-capable. |
gpt-4.1 | Mid-tier reasoning model. |
gpt-4.1-mini, gpt-4.1-nano | Lower-cost 4.1 variants. |
gpt-4o | General-purpose, vision-capable. |
gpt-4o-mini | Low-cost, vision-capable. |
gpt-o3, gpt-o3-mini, gpt-o4-mini | Dedicated reasoning models. |
openai-generic / openai-reasoning-generic | "BYO model name" — use this when you want to point at an OpenAI-compatible model that is not in the curated list. |
azure-openai-generic / azure-openai-reasoning-generic | Same idea, but for Azure-hosted deployments. |
Anthropic / AWS Bedrock
| Base model | Notes |
|---|---|
claude-4-6-opus | Current flagship Claude. Reasoning-capable. Recommended for the AI Assistant. |
claude-4-6-sonnet | Current high-quality Claude. Reasoning-capable. |
claude-4-5-sonnet | Previous-generation Sonnet. Reasoning-capable. |
claude-4-5-haiku | Current low-cost Claude. Reasoning-capable. |
claude-3-7-sonnet | Older Sonnet (still supported). |
claude-3-5-sonnet | Vision-capable, widely deployed. |
claude-3-haiku | Low-cost, no reasoning. |
bedrock-generic / bedrock-reasoning-generic | "BYO model id" — use this to point at any Bedrock-hosted model by its provider id. |
The same Claude models are reachable via Anthropic (direct API) and AWS Bedrock (managed).
Google Gemini / Vertex AI
| Base model | Notes |
|---|---|
gemini-3.1-pro | Current flagship Gemini. Reasoning-capable. |
gemini-3.1-flash-lite | Current low-cost Gemini. Reasoning-capable. |
gemini-3-pro, gemini-3-flash | Gemini 3.0 family. |
gemini-2.5-pro | High-quality, reasoning-capable. |
gemini-2.5-flash, gemini-2.5-flash-lite | Fast, low-cost variants. |
gemini-2.0-pro, gemini-2.0-flash | Earlier 2.x generation. |
gemini-1.5-pro | Legacy, still supported. |
vertex-generic / vertex-reasoning-generic | "BYO model id" for Vertex deployments not in the curated list. |
Azure AI Studio
| Base model | Notes |
|---|---|
llama-3-2-90b-vision-instructor | Vision-capable Llama for assertions. |
azure-ai-studio-generic / azure-ai-studio-reasoning-generic | "BYO model id" for any Azure AI Studio deployment. |
The *-generic and *-reasoning-generic entries are the supported way to use a model that isn't in the curated list — for instance a brand-new vendor release that has not yet been added by name. Pick the generic entry of the right vendor and put the actual provider model id into the Deployment Name field. Use the -reasoning- variant if the underlying model supports reasoning / extended thinking.
Model capabilities
Some capabilities are flagged on the model:
- Reasoning — the model supports extended thinking. Required for the AI Assistant; recommended for step identification on complex applications. All
gpt-5*,gpt-o*, Claude 4.x, Gemini 2.5+, and the*-reasoning-genericentries are reasoning-capable. - Vision — the model can interpret a screenshot. Required for the Default AI Model for Assertions in Tenant Settings. GPT-4o / 4o-mini, Claude 3.5+, and the Llama-3.2-90B-Vision-Instructor are vision-capable; all current Gemini models are vision-capable too.
- Low-cost — used to flag cheap models (currently
gpt-4o-mini,claude-3-haiku, the various Gemini -flash-lite variants). The platform doesn't enforce anything based on this flag; it's a hint for admins picking sensible defaults.
Configuration form
When you add or edit a model, the form is split into four sections.
Basic info
| Field | Description |
|---|---|
| ID | A unique name for this model entry inside msg.ZenTestAI. Used everywhere a model is selected; pick something descriptive (e.g. claude-4-6-opus-eu, gpt-5-cheap). Required. |
| AI-Model Vendor | The vendor connector to use. Required. Determines which fields appear below. |
| Base AI Model | The base model from the curated list (see above). Required. |
Once a model is saved, a Test the connection button is shown next to the ID. Pressing it calls the configured vendor with the stored credentials and reports success or the exact error returned by the vendor.
Authentication
The authentication section adapts to the vendor:
| Vendor | Field | Notes |
|---|---|---|
| OpenAI | API-Key | Your OpenAI API key. |
| Azure OpenAI | API-Key | The key of your Azure OpenAI resource. |
| Anthropic | API-Key | Your Anthropic API key. |
| Google Gemini | API-Key | A Google AI Studio API key. |
| Azure AI Studio | API-Key | The key from the Azure AI Studio deployment. |
| AWS Bedrock | Credentials | JSON containing AWS region and access keys for the IAM principal that may call Bedrock. |
| Google Vertex AI | Credentials | The JSON of a Google Cloud service account that has Vertex AI permissions. |
API keys and credentials are stored encrypted and never displayed back in plain text.
Configuration
The configuration section is the most vendor-specific part of the form:
| Field | Required for | What it is |
|---|---|---|
| Deployment Name | All vendors | The provider-side name of the deployment / model. For OpenAI this is typically the model name (gpt-5); for Azure it is the deployment name you created in your Azure portal; for AWS Bedrock the Bedrock model id (e.g. anthropic.claude-3-5-sonnet-20241022-v2:0). |
| API-Version | Azure OpenAI, Azure AI Studio (and optional for Vertex AI / Gemini) | The API version string of the vendor (e.g. 2024-08-01-preview for Azure OpenAI). |
| API-Instance-Name | Azure OpenAI | The Azure OpenAI resource name (the one that goes into the URL {instance}.openai.azure.com). |
| API-Base-Path | Optional for most vendors | Custom endpoint URL — useful when you front the vendor with a reverse proxy or use a regional endpoint. |
The form hides fields that don't apply to the selected vendor, so you only see what you need to fill in.
Advanced settings
The advanced section is collapsed by default. It contains:
- Fallback Model — pick another configured model to use when this one fails (connection error, rate limit, vendor outage). Useful when you have a low-cost primary model and a higher-quality backup. Leave empty if you don't want a fallback.
- Reasoning Effort — shown only when the selected base model is reasoning-capable. Lets you set how much reasoning the model spends per call:
none,low,medium,high, orauto. Higher effort improves quality on complex steps but costs more tokens and runs slower. - Input Cost Per Million Tokens / Output Cost Per Million Tokens — optional. Fill these in so that test executions and AI Assistant sessions can show a meaningful cost estimate. The numbers are in USD per million tokens, matching the format vendors publish on their pricing pages.
Lifecycle
Saving
Saving validates the form and stores the encrypted credentials. The vendor connection itself is not validated on save — use Test the connection after saving to verify that the credentials and configuration actually work end-to-end.
Deleting
Models can be deleted from the AI tab at any time. There is no safeguard preventing deletion of a model that is currently referenced (as a tenant default, as a per-step override, as the default model for agents). If you delete a model that's in use, the tests that reference it will fail at execution time with a "model not found" error. To be safe, swap references to another model first and then delete the entry.
Reordering
The list of models can be reordered via drag and drop. This only changes the display order — it does not change which model is the default for anything.
Picking the right model for each task
The platform uses several model defaults that are configured separately in Tenant Settings. The recommendations below are starting points:
| Task / setting | Recommended model class |
|---|---|
| Default model for element identification | A cheap model with a large context window (gpt-4o-mini, claude-3-haiku, a Gemini flash variant). |
| Default model for assertions | A vision-capable model. gpt-4o, claude-3-5-sonnet, or a high-quality Gemini. |
| Default model for step identification | A high-quality reasoning model — gpt-5, claude-4-6-sonnet, gemini-3-pro. |
| Default model for the AI Assistant | A top-tier reasoning model — Claude Opus or GPT-5+ ("Copilot"). See AI Assistant → Recommended models. |
If you set a low-cost model as the default for element identification, the runner will automatically escalate to a higher-quality model for steps that need it. You don't have to oversize the default just to handle the occasional difficult step.
Migration notes
- The platform retains older model entries (Claude 3.x, Gemini 1.5, GPT-4o) for backward compatibility. Existing test configurations continue to work, but for new tests prefer the current-generation models listed above.
- When a vendor introduces a new model that isn't in the curated dropdown yet, use the corresponding
*-genericentry and provide the new model id in Deployment Name. - If you have any other vendor or model not covered above, contact us at hello@zentest.ai.