Skip to content

Frameworks & runtimes

LLMJury + Vercel AI SDK

Resolve the variant on the server, pass the prompt into generateText or streamText, and track the outcome. The publishable key is write-only, so assignment can happen at the edge.

⌘ Key-only — you send the metrics

The short answer: Works on the key-only path with the TypeScript SDK. The AI SDK’s provider abstraction is its own shape, so there is no interception — the prompt is an argument you pass.

How it works

The AI SDK deliberately hides which provider you are talking to behind its own model objects. That is good design and it means there is no client of a recognised shape to wrap, so this is the key-only path — resolve the variant, pass the prompt in, send the outcome.

getPrompt(experiment, userId, defaultPrompt) resolves from client memory and returns synchronously enough to sit inside a route handler without adding a round trip. await client.ready() on cold start is the one thing worth doing, so the first request after a deploy is not answered from an empty config cache.

The publishable key (llmj_pk_…) is write-only and rate-limited, which is why it is safe in an edge function or a client component. Secret keys are server-side only, and both are revocable from the dashboard.

Streaming is where the measurement question gets interesting. What users feel is time to first token, not total duration, and those two diverge badly under load — so if you are testing anything that touches perceived speed, send ttft_ms on the event and compare that.

What it looks like in your code

import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import { Client } from 'llmjury-sdk';

const client = new Client({ experiments: ['checkout-prompt'] });

export async function POST(req: Request) {
  const { userId, messages } = await req.json();

  // resolve the arm from client memory — no network call on the request path
  const p = client.getPrompt('checkout-prompt', userId, 'You are a helpful assistant.');

  const result = streamText({
    model: openai('gpt-4o-mini'),
    system: p.prompt,
    messages,
    onFinish: () =>
      client.track('business_event', {
        experiment_id: 'checkout-prompt',
        user_id: userId,
        variant: p.variant,
        business_metric: 'conversion',
        value: 1,
      }),
  });

  return result.toDataStreamResponse();
}

The same path in all three SDKs is on the five-minute quickstart, and the per-method reference is in the documentation.

What gets measured

  • Your business outcome, sent from onFinish or from wherever the outcome actually happens
  • Time to first token and total latency, if you send them — the two diverge, and only one is what users feel
  • Tokens and cost, from the usage the AI SDK reports back
  • Exposures per arm and the sample-ratio-mismatch gate

What this does not do

  • No interception, so no automatic capture. Every measurement on this path is one you send.
  • In a serverless or edge runtime, the SDK’s background flush has to finish before the function is frozen. Track before the response resolves, or use the platform’s wait-until primitive.
  • The browser build spills to IndexedDB when offline and replays for 24 hours; a frozen serverless process gets neither. Server-side tracking is the reliable placement.

Which of the two paths you take decides whether your prompt and response text reaches us. The full breakdown is on the security page.

Point it at Vercel AI SDK and see

Free plan, no credit card. The sample experiment is already in your account, so there is a real verdict to read before you have any traffic of your own.

Free plan · no credit card required

Using something we have not listed? Tell us — the key-only path already works with it, and which providers people ask about is how the interception list grows.

Other frameworks & runtimes