Jev benchmarks
Accuracy of TypeSafe Jev 1.13 and eight other decision models on 15 text suites and 2 image suites, as published by DecisionNode. Preliminary results, October 2026.
Facts checked
The 15-suite mean
- DecisionNode MoE 0.816
- TypeSafe Jev 1.13 0.803
- Cloudflare Clef 27B 0.786
- DecisionNode Flash 0.784
- Cloudflare Clef-flash 9B 0.759
- pplx-decider-27b 0.746
- Cygnet (Gemma 4 12B) 0.717
- Kev-4B 0.717
- Strands Decider 2B 0.622
0 to 1, higher is better, full axis. Preliminary results, October 2026.
Jev against DecisionNode, suite by suite
| Suite | Items | DecisionNode MoE | TypeSafe Jev 1.13 | DecisionNode Flash | MoE against Jev |
|---|---|---|---|---|---|
| Held-out decisions Typed decisions on tasks kept out of development. | 636 | 0.940 0.921 to 0.959 | 0.948 0.929 to 0.965 | 0.918 0.896 to 0.940 | Within noise, −0.8 pts |
| Public decision set A small public set of choice, score and yes/no questions. | 117 | 0.880 0.817 to 0.939 | 0.855 0.784 to 0.916 | 0.821 0.743 to 0.884 | Within noise, +2.5 pts |
| Transfer tasks Decisions phrased the way other decision APIs phrase them. | 730 | 0.838 0.803 to 0.873 | 0.852 0.814 to 0.886 | 0.767 0.729 to 0.804 | Within noise, −1.4 pts |
| Web tasks Pick the right element or action on a real web page. | 1,106 | 0.765 0.735 to 0.794 | 0.698 0.660 to 0.734 | 0.753 0.723 to 0.784 | MoE leads, +6.7 pts |
| Knowledge (MMLU-Pro) Hard multiple-choice questions that need world knowledge. | 1,479 | 0.698 0.676 to 0.723 | 0.820 0.801 to 0.840 | 0.575 0.548 to 0.601 | Jev leads, −12.2 pts |
| Short classifications One-line inputs sorted into a few labels. | 178 | 0.921 0.882 to 0.955 | 0.904 0.860 to 0.944 | 0.882 0.826 to 0.933 | Within noise, +1.7 pts |
| Open decision tasks A broad public mix of typed decision tasks. | 5,011 | 0.795 0.784 to 0.806 | 0.798 0.787 to 0.809 | 0.770 0.758 to 0.781 | Within noise, −0.3 pts |
| Typed decisions Mixed choice, score and yes/no questions in one request. | 955 | 0.713 0.682 to 0.748 | 0.735 0.702 to 0.770 | 0.733 0.702 to 0.768 | Within noise, −2.2 pts |
| New task families Task families the model never saw while it was built. | 3,629 | 0.797 0.783 to 0.809 | 0.823 0.811 to 0.835 | 0.752 0.739 to 0.766 | Jev leads, −2.6 pts |
| Familiar task families New items from task families the model knows. | 4,100 | 0.796 0.784 to 0.809 | 0.774 0.763 to 0.787 | 0.766 0.752 to 0.778 | Within noise, +2.2 pts |
| Large option sets Choices with dozens of options to pick from. | 1,677 | 0.806 0.787 to 0.825 | 0.775 0.755 to 0.796 | 0.796 0.776 to 0.816 | Within noise, +3.1 pts |
| Long inputs Long threads and documents with the answer deep inside. | 1,147 | 0.835 0.810 to 0.858 | 0.853 0.832 to 0.874 | 0.827 0.801 to 0.851 | Within noise, −1.8 pts |
| Multilingual The same decisions asked in many languages. | 1,812 | 0.796 0.776 to 0.814 | 0.782 0.762 to 0.802 | 0.775 0.755 to 0.793 | Within noise, +1.4 pts |
| Rules and policies Apply a written policy to a case and decide. | 1,759 | 0.891 0.875 to 0.907 | 0.831 0.812 to 0.849 | 0.872 0.855 to 0.890 | MoE leads, +6.0 pts |
| Prompt injection Inputs that try to talk the model into the wrong answer. | 798 | 0.764 0.734 to 0.793 | 0.600 0.565 to 0.634 | 0.747 0.718 to 0.777 | MoE leads, +16.4 pts |
| Photos Choice and yes/no questions about real photos. | 1,668 | 0.918 0.906 to 0.930 | not supported | 0.900 0.887 to 0.914 | Image suite, Jev reads text only |
| Rendered scenes How likely a statement is, judged from a computer-rendered scene. | 450 | 0.687 0.643 to 0.732 | not supported | 0.620 0.575 to 0.665 | Image suite, Jev reads text only |
Preliminary results, October 2026. 15 text suites, 25,134 items, development splits. 95% intervals for the DecisionNode models and Jev come from a bootstrap that resamples whole inputs. Jev (jev-1.13.0) was called through its official API; open models ran as their authors serve them. DecisionNode models are release candidate 1.
“Leads” and “trails” are used only where the two 95% intervals do not overlap. Source: decisionnode.com/benchmarks. DecisionNode is not affiliated with TypeSafe AI, Cloudflare or the other model publishers. Their names identify the systems measured.
What these results mean when choosing between the two: Jev vs DecisionNode.
Every system, every suite
Accuracy only. Intervals are published for DecisionNode and Jev alone (in the table above). Blank cells are results that were not published. Each system is described on the Jev alternatives page.
| Suite | DecisionNode MoE | TypeSafe Jev 1.13 | Cloudflare Clef 27B | DecisionNode Flash | Cloudflare Clef-flash 9B | pplx-decider-27b | Cygnet (Gemma 4 12B) | Kev-4B | Strands Decider 2B |
|---|---|---|---|---|---|---|---|---|---|
| Held-out decisions text, 636 items | 0.940 | 0.948 | 0.933 | 0.918 | 0.959 | 0.924 | 0.892 | ||
| Public decision set text, 117 items | 0.880 | 0.855 | 0.855 | 0.821 | 0.850 | 0.868 | 0.744 | ||
| Transfer tasks text, 730 items | 0.838 | 0.852 | 0.810 | 0.767 | 0.829 | 0.786 | 0.762 | ||
| Web tasks text, 1,106 items | 0.765 | 0.698 | 0.697 | 0.753 | 0.672 | 0.632 | 0.612 | ||
| Knowledge (MMLU-Pro) text, 1,479 items | 0.698 | 0.820 | 0.634 | 0.575 | 0.627 | 0.544 | 0.494 | ||
| Short classifications text, 178 items | 0.921 | 0.904 | 0.921 | 0.882 | 0.941 | 0.913 | 0.792 | ||
| Open decision tasks text, 5,011 items | 0.795 | 0.798 | 0.834 | 0.770 | 0.790 | 0.759 | 0.811 | ||
| Typed decisions text, 955 items | 0.713 | 0.735 | 0.705 | 0.733 | 0.720 | 0.663 | 0.663 | ||
| New task families text, 3,629 items | 0.797 | 0.823 | 0.800 | 0.752 | 0.800 | 0.758 | 0.738 | ||
| Familiar task families text, 4,100 items | 0.796 | 0.774 | 0.772 | 0.766 | 0.777 | 0.727 | 0.707 | ||
| Large option sets text, 1,677 items | 0.806 | 0.775 | 0.842 | 0.796 | 0.775 | 0.694 | 0.701 | ||
| Long inputs text, 1,147 items | 0.835 | 0.853 | 0.845 | 0.827 | 0.492 | 0.634 | 0.758 | ||
| Multilingual text, 1,812 items | 0.796 | 0.782 | 0.760 | 0.775 | 0.800 | 0.756 | 0.727 | ||
| Rules and policies text, 1,759 items | 0.891 | 0.831 | 0.798 | 0.872 | 0.845 | 0.778 | 0.704 | ||
| Prompt injection text, 798 items | 0.764 | 0.600 | 0.582 | 0.747 | 0.318 | 0.327 | 0.652 | ||
| Photos image, 1,668 items | 0.918 | text only | 0.882 | 0.900 | 0.905 | 0.891 | |||
| Rendered scenes image, 450 items | 0.687 | text only | 0.783 | 0.620 | 0.742 | 0.809 |
How it was measured
15 text suites, 25,134 items, development splits. 95% intervals for the DecisionNode models and Jev come from a bootstrap that resamples whole inputs. Jev (jev-1.13.0) was called through its official API; open models ran as their authors serve them. DecisionNode models are release candidate 1.
- TypeSafe Jev 1.13: Reads text only, so it has no image results.
- Strands Decider 2B: Its published training data contains items of two of the test sets, so its score may be inflated.
Every figure on this page is copied from decisionnode.com/benchmarks, which DecisionNode publishes about its own models. DecisionNode is not affiliated with TypeSafe AI, Cloudflare or the other model publishers. Their names identify the systems measured.
What these numbers do not show
- Your task. Development splits are not your data. Test both models on a sample of your own requests.
- Calibration. How well each probability matches how often it is right is not published yet.
- Speed against Jev. No head-to-head timing is published. DecisionNode MoE's latency is not published either.
- Later versions. These are Jev 1.13 and DecisionNode's release candidate 1. Either can change with a release.
Run your own decisions on DecisionNode
Send text, JSON or images with typed questions and get probabilities back. Free tier: 50 Flash and 20 MoE requests a day.
Try DecisionNode