Terms of Service

This agreement is between you (or the organization you are signing up on behalf of, in which case you are confirming you may bind it) and PermuteLab LLC, which operates LLMJury. Creating an account or using the SDKs means you accept these terms. If you do not, do not use the service.

LLMJury is in open beta. The statistical engine and the SDKs are production-grade and tested, but the product is young: expect occasional rough edges, frequent releases, and features that change shape.

During beta we offer no uptime guarantee and no SLA. This is why every SDK carries an in-code default prompt: if LLMJury is unavailable, your application must keep serving from that fallback. Design your integration that way. We document the pattern in every SDK, and we consider an outage that takes your app down with it to be a bug in the integration guidance, not an acceptable outcome.

We will give at least 30 days’ notice before removing a feature you are actively using, and we will not raise your price mid-subscription.

You keep all rights to your prompts, your experiment configuration, your events, and your results. We claim no ownership of any of it. You grant us only the licence we need to run the service for you: to store your data, send sampled outputs to the judge model for grading, compute statistics, and show the results back to you.

We never train models on your data — ours or a vendor’s — and we never use one customer’s data to serve another. You can export your data and request deletion at any time. How we handle it in detail is in the Privacy Policy, which forms part of these terms.

You are responsible for having the right to send us what you send us — including any personal data inside a prompt or a graded output, and any consent your own users needed to give.

Do not use LLMJury to:

• break the law, or help someone else break it;
• grade or store content you have no right to — including personal data you did not collect lawfully;
• attack, overload, probe, or reverse-engineer the service, or work around your plan limits, rate limits, or judge budget;
• resell LLMJury or expose it as a competing experimentation service to third parties;
• share account credentials or API keys outside your organization, or use one account to serve several unrelated businesses.

You are responsible for what happens under your API keys. If a key leaks, rotate it from the dashboard and tell us. If we have to suspend an account for something on this list, we will tell you why and what would fix it.

Every plan starts free with no credit card. The Free plan stays free. Paid plans are Pro at $149/month, Business at $799/month, billed monthly in advance through our payment processor. Current limits and prices are always on the pricing page, and that page governs if it ever disagrees with this one.

Overage. If you exceed your monthly event allowance on a paid plan, the excess is billed in blocks at the rate shown on the pricing page (Pro: +$49 per extra 500k events; Business: +$99 per extra 2M events). On the Free plan there is no overage charge — the SDK simply falls back to your in-code default prompt until the allowance resets at the start of the next month, so nothing breaks and nothing is deleted.

Cancelling. Cancel any time from the billing page. Your plan runs to the end of the period you have paid for and then stops; we do not pro-rate partial months. Taxes, where they apply, are added at checkout.

LLMJury reports statistical estimates computed from the data you send: effect sizes, confidence intervals, and p-values, gated by a sample-ratio-mismatch check and corrected for false discovery across metrics. The methods are documented in full, and the raw numbers behind every verdict are one click away precisely so you can check them.

A verdict is evidence for a decision, not the decision itself, and it is only as good as the data and the rubric you gave it. You remain responsible for what you ship. NothingLLMJury outputs is a warranty that a variant will perform a particular way in your product.

LLMJury — the service, the dashboard, the statistical engine, the documentation, and the brand — belongs to PermuteLab LLC. These terms grant you a limited, non-exclusive, non-transferable right to use it while your account is in good standing, and nothing more.

The client SDKs are a deliberate exception: they are open source under their published licences, and those licences govern them, not this section.

Feedback you send us — a bug report, a feature idea, a complaint about a confusing screen — we may use freely and without obligation. We would rather you send it than hold it back over this sentence.

The service is provided “as is” and “as available”, without warranties of any kind, express or implied, including merchantability, fitness for a particular purpose, and non-infringement. We do not warrant that the service will be uninterrupted, error-free, or that it will produce any particular result. See section 2 — this is a beta.

To the maximum extent the law allows, neither party is liable for indirect, incidental, special, consequential, or punitive damages, or for lost profits, revenue, or data, however caused.

Our total liability for all claims arising out of or relating to these terms is capped at the amount you paid us in the twelve months before the claim arose — or, if you are on the Free plan, at one hundred US dollars. Nothing here limits liability that cannot be limited by law.

You may close your account at any time, for any reason. We may suspend or close an account that breaches section 4, that fails to pay, or that we are legally required to act on — with notice and an explanation wherever giving one is possible.

After closure we delete your data within 30 days, backups included. Export anything you want to keep before you close it, or ask us and we will help.

We may update these terms as the product changes. The date at the top always reflects the current version. For a material change we will email account owners at least 30 days before it takes effect; continuing to use LLMJury after that means you accept it, and if you do not, you can cancel before it applies.

These terms and the Privacy Policy are the whole agreement between us on this subject. They are governed by the laws of the State of Delaware, United States, without regard to its conflict-of-laws rules, and the state and federal courts sitting in Delaware have exclusive jurisdiction. If any provision is held unenforceable, the rest stays in force. Neither of us waives a right by not enforcing it once. You may not assign this agreement without our consent; we may assign it in a merger or sale of the business.

Questions about any of this go to [email protected] — a founder answers, within one business day.