Skip to content

Model providers

LLMJury + Azure OpenAI

Azure OpenAI is the same openai client class pointed at your Azure deployment, so it is the same wrap() call — there is no separate Azure code path, and none is needed.

⚡ Automatic capture with wrap()

The short answer: Supported on the interception path, because AzureOpenAI is the same client class as OpenAI with a different endpoint — not because we built anything Azure-specific.

How it works

The AzureOpenAI class ships in the same openai package and exposes the same chat.completions.create surface. wrap() intercepts the shape, not the vendor, so it works here for the same reason it works for OpenAI directly — and we would rather say that than imply an integration effort that never happened.

The interesting experiment on Azure is usually the deployment, not the model name: two deployments of the same model in different regions or at different capacity tiers behave differently under load. Put the deployment name in the variant’s variables and the latency percentiles will tell you which one your users actually experience.

Nothing about your Azure networking changes. Calls go from your process to your Azure endpoint; if that endpoint sits behind a private link or inside a VNet, it stays there. The only traffic we see is the event batch your SDK flushes to us, over TLS, after the fact.

What it looks like in your code

from llmjury import Client
from openai import AzureOpenAI

client = Client(experiments=["checkout-prompt"])
llm = client.wrap(AzureOpenAI(api_version="2024-10-21"), "checkout-prompt")   # same client class as OpenAI, so the same interception

with client.as_user(user_id):
    # the variant's prompt, from client memory — your literal is the fallback
    p = client.get_prompt("checkout-prompt", user_id,
                          default="You are a helpful assistant.")
    response = llm.chat.completions.create(
        model="my-gpt-4o-deployment",   # your Azure deployment name
        messages=[{"role": "system", "content": p.prompt},
                  {"role": "user", "content": user_input}],
    )

# the only metric you send yourself
client.track("business_event", {"experiment_id": "checkout-prompt", "user_id": user_id,
                                "variant": p.variant, "business_metric": "conversion", "value": 1})

The same path in all three SDKs is on the five-minute quickstart, and the per-method reference is in the documentation.

What gets measured

  • Latency per deployment, as percentiles — the number that moves when a region is saturated
  • Tokens and computed cost per variant
  • Errors and throttling responses, kept separate from successful calls
  • Judge-scored quality on a sampled subset (Pro and up)
  • Your business outcome, sent with track(…)

What this does not do

  • We have not built an Azure-specific code path and do not claim one. What works is the client shape; if Microsoft changes it, this stops being true and we will say so here.
  • We do not read your Azure resource configuration, quotas, or content-filter settings. What we know about a call is what the wrapper measured.
  • The interception path sends us prompt and response text. If your Azure deployment exists specifically to keep text inside a compliance boundary, use the key-only path instead — it sends none.

Which of the two paths you take decides whether your prompt and response text reaches us. The full breakdown is on the security page.

Point it at Azure OpenAI and see

Free plan, no credit card. The sample experiment is already in your account, so there is a real verdict to read before you have any traffic of your own.

Free plan · no credit card required

Using something we have not listed? Tell us — the key-only path already works with it, and which providers people ask about is how the interception list grows.

Other model providers