---
title: When a Small AI Model Beats Claude Opus 5 at JSON Extraction
description: See how a fine-tuned Qwen2.5-1.5B model compared with Claude Opus 5 for domain-specific JSON extraction, accuracy, cost, and data control.
image: https://social.jebra.io/hubfs/experiment5_jebra_wide.png
---

[Skip to content](https://social.jebra.io/blog/when-a-small-ai-model-beats-claude-opus-5-at-json-extraction#main-content)

[![jebra.io](https://social.jebra.io/hs-fs/hubfs/Jebra-Email-Sig-Logo-1.png?width=384&height=171&name=Jebra-Email-Sig-Logo-1.png)Homepage](https://www.jebra.io/)

[Request Demo](https://www.jebra.io/request-demo)

[Request Demo](https://www.jebra.io/request-demo)

![](https://social.jebra.io/hs-fs/hubfs/experiment5_jebra_wide.png?width=1774&height=887&name=experiment5_jebra_wide.png)

# When a Small AI Model Beats Claude Opus 5 at JSON Extraction

![Harsh S.](https://social.jebra.io/hs-fs/hubfs/TNZVBQYB1-U08A8VBMWF8-5ea1881e7324-512.jpg?width=48&height=48&name=TNZVBQYB1-U08A8VBMWF8-5ea1881e7324-512.jpg)

 Harsh S.

August 26, 2026

#### **Jebra AI Experiments #5**

In [**our previous blog**](https://dev.to/jebra/fine-tuning-with-qlora-for-json-extraction-f8k), we described how we successfully fine-tuned Qwen2.5-1.5B-Instruct using QLoRA to extract structured JSON from unstructured text for a domain-specific use case. The results were promising, with the model achieving 62% exact match accuracy and 97% field match accuracy.

However, a natural question arose: How does our fine-tuned small model compare against state-of-the-art frontier models like Claude Opus 5? Would a massive general-purpose model outperform our domain-specific fine-tuned model?

We decided to run the same evaluation pipeline on Claude Opus 5 to find out.

---

## Evaluation Setup

We used the identical test set and the same evaluation metrics to ensure a fair comparison.

**Exact Match:** A binary score per sample, 1 if every field matches exactly and 0 otherwise, aggregated across the test dataset.

**Field Match:** Partial credit per sample based on the number of correct fields divided by total fields, aggregated across the test dataset.

---

## Results

Despite being roughly 1,000 times smaller than Claude Opus 5, our fine-tuned Qwen2.5-1.5B achieved approximately 15 percentage points higher field-match accuracy: 96.72% versus 81.81%.

It also achieved 62% exact-match accuracy, compared with Claude Opus 5 at 0%. Beyond accuracy, the smaller fine-tuned model is much less expensive and provides much more control over your data.

![https___dev-to-uploads.s3.us-east-2.amazonaws.com_uploads_articles_lq6dipdsk1ra7088l9me](https://social.jebra.io/hs-fs/hubfs/https___dev-to-uploads.s3.us-east-2.amazonaws.com_uploads_articles_lq6dipdsk1ra7088l9me.webp?width=800&height=600&name=https___dev-to-uploads.s3.us-east-2.amazonaws.com_uploads_articles_lq6dipdsk1ra7088l9me.webp)

---

## Conclusion

This comparison demonstrates a powerful lesson in modern AI development: a well-tuned small model can outperform even the largest frontier models on domain-specific use cases.

[Original article on DEV Community](https://dev.to/jebra/fine-tuned-qwen25-15b-vs-claude-opus-5-for-json-extraction-5aco)

## Share this post

<https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fsocial.jebra.io%2Fblog%2Fwhen-a-small-ai-model-beats-claude-opus-5-at-json-extraction><https://twitter.com/intent/tweet?url=https%3A%2F%2Fsocial.jebra.io%2Fblog%2Fwhen-a-small-ai-model-beats-claude-opus-5-at-json-extraction><https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Fsocial.jebra.io%2Fblog%2Fwhen-a-small-ai-model-beats-claude-opus-5-at-json-extraction><https://pinterest.com/pin/create/button/?url=https%3A%2F%2Fsocial.jebra.io%2Fblog%2Fwhen-a-small-ai-model-beats-claude-opus-5-at-json-extraction>[mailto:https%3A%2F%2Fsocial.jebra.io%2Fblog%2Fwhen-a-small-ai-model-beats-claude-opus-5-at-json-extraction](mailto:https%3A%2F%2Fsocial.jebra.io%2Fblog%2Fwhen-a-small-ai-model-beats-claude-opus-5-at-json-extraction)

## Keep reading

### [![](https://social.jebra.io/hs-fs/hubfs/Hubspot_header_blog_grammar.jpg?width=1024&height=400&name=Hubspot_header_blog_grammar.jpg) Dicts and Docs: The Value of Grammar and Documentation for LLM-Based Automation](https://social.jebra.io/blog/grammar-documentation-llm-payroll-automation)

### [![](https://social.jebra.io/hs-fs/hubfs/blog_header_try6.png?width=1000&height=500&name=blog_header_try6.png) 10 Claude Prompts That Turn Payroll Data Into Better Studio Decisions](https://social.jebra.io/blog/claude-prompts-payroll-data-studio-decisions)

[![Jebra1_chatgptwhite-1](https://social.jebra.io/hs-fs/hubfs/Jebra1_chatgptwhite-1.png?width=300&height=100&name=Jebra1_chatgptwhite-1.png "Jebra1_chatgptwhite-1")](https://www.jebra.io/)

<https://www.linkedin.com/company/jebra><https://www.facebook.com/jebra.io><https://x.com/JebraPayroll><https://www.instagram.com/hello.jebra>

---

[Privacy Policy](https://www.jebra.io/privacy-policy) · [Legal](https://www.jebra.io/terms-of-services) · © 2025. All rights reserved.

```json
{
  "@context" : "https://schema.org",
  "@type" : "BlogPosting",
  "author" : {
    "@type" : "Person",
    "name" : "Harsh S.",
    "url" : "https://social.jebra.io/blog/author/harsh-s-2"
  },
  "dateModified" : "2026-08-26T23:42:27.217Z",
  "datePublished" : "2026-08-26T20:40:23.000Z",
  "headline" : "When a Small AI Model Beats Claude Opus 5 at JSON Extraction",
  "image" : [ "https://social.jebra.io/hubfs/experiment5_jebra_wide.png" ],
  "mainEntityOfPage" : {
    "@id" : "https://social.jebra.io/blog/when-a-small-ai-model-beats-claude-opus-5-at-json-extraction",
    "@type" : "WebPage"
  },
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject",
      "url" : "https://social.jebra.io/hubfs/Jebra-Email-Sig-Logo-1.png"
    }
  }
}
```