Invoice Data Extraction: deep learning vs machine learning vs LLM (2026 guide)

Five years ago, extracting structured data from invoices meant a deep learning pipeline: an OCR stage, a layout-aware model (LayoutLM, Donut) fine-tuned on thousands of hand-labeled documents, and a maintenance burden every time a vendor changed their template. In 2026 the entire pipeline collapses into one vision-LLM API call: image in, schema-validated JSON out — zero training data, zero hosted models, and new layouts handled zero-shot. This guide shows the full working pattern, what it costs per invoice, and how to keep accuracy production-grade.

Invoice extraction approaches compared: rules, machine learning, deep learning, LLM

Four generations of technique have been used to get structured data out of an invoice. They are worth laying side by side, because the tradeoff that decided between them for two decades — accuracy bought with labeled data — is the one that stopped applying.

The first three all share one property: accuracy is bought with labeled data. Every vendor template, every CRF feature, every LayoutLM fine-tune is an annotation project. A vision LLM has already paid that cost during pretraining, so the marginal cost of a new layout is zero. That is the whole shift — not that the older methods stopped working, but that the thing they spent their budget on became free.

Fine-tuned deep learning vs vision LLM, in detail

A fine-tuned extractor can still win on a single frozen high-volume layout, but for the long tail of real-world invoices — different vendors, languages, scan quality — the zero-shot vision LLM is more accurate in practice and radically cheaper to own. More patterns for this class of workload live in the data-extraction use case.

The complete pattern: image → schema-validated JSON

Everything below runs against Kunavo's OpenAI-compatible endpoint — point base_url at https://api.kunavo.com/v1 and the same code can hit Claude, Gemini or GPT by changing one string:

from openai import OpenAI
import base64, json

client = OpenAI(
    base_url="https://api.kunavo.com/v1",
    api_key="sk-kn-...",
)

SCHEMA = {
    "name": "invoice",
    "schema": {
        "type": "object",
        "properties": {
            "vendor_name":    {"type": "string"},
            "vendor_tax_id":  {"type": ["string", "null"]},
            "invoice_number": {"type": "string"},
            "invoice_date":   {"type": "string", "description": "ISO 8601"},
            "due_date":       {"type": ["string", "null"]},
            "currency":       {"type": "string", "description": "ISO 4217"},
            "line_items": {
                "type": "array",
                "items": {
                    "type": "object",
                    "properties": {
                        "description": {"type": "string"},
                        "quantity":    {"type": "number"},
                        "unit_price":  {"type": "number"},
                        "amount":      {"type": "number"},
                    },
                    "required": ["description", "amount"],
                },
            },
            "subtotal":  {"type": ["number", "null"]},
            "tax":       {"type": ["number", "null"]},
            "total":     {"type": "number"},
        },
        "required": ["vendor_name", "invoice_number", "invoice_date",
                     "currency", "line_items", "total"],
    },
}

def extract(path: str) -> dict:
    image_b64 = base64.b64encode(open(path, "rb").read()).decode()
    resp = client.chat.completions.create(
        model="claude-haiku-4-5",   # or gemini-2-5-flash for the cheapest run
        response_format={"type": "json_schema", "json_schema": SCHEMA},
        messages=[
            {"role": "system", "content":
                "Extract the invoice into the schema. Copy values exactly as "
                "printed; use null when a field is absent. Never invent data."},
            {"role": "user", "content": [
                {"type": "image_url",
                 "image_url": {"url": f"data:image/png;base64,{image_b64}"}},
            ]},
        ],
    )
    return json.loads(resp.choices[0].message.content)

inv = extract("invoice_0231.png")

# Cheap arithmetic guardrail: reject when the line items don't add up.
delta = abs(sum(li["amount"] for li in inv["line_items"])
            + (inv.get("tax") or 0) - inv["total"])
if delta > 0.02:
    raise ValueError(f"line items disagree with total by {delta:.2f}")

print(inv["vendor_name"], inv["total"], inv["currency"])

Three details do most of the work: the JSON schema in response_format (the model cannot emit anything else), the system-prompt rule “copy values exactly; null when absent; never invent” (kills hallucinated tax IDs), and the arithmetic guardrail — line items + tax must equal the total, or the document goes to a stronger model or a human. See the chat completions reference for the structured-output options.

The same schema, as a typed Pydantic model

A raw JSON-schema dict is fine for one endpoint and painful for ten. In production most teams define the schema once as a Pydantic model and derive both the constraint and the parsed object from it, so the extractor and the rest of the codebase can never disagree about the shape:

from pydantic import BaseModel, Field
from typing import Literal
from openai import OpenAI

client = OpenAI(base_url="https://api.kunavo.com/v1", api_key=KEY)

class LineItem(BaseModel):
    description: str
    quantity: float | None = None
    unit_price: float | None = None
    amount: float

class Invoice(BaseModel):
    vendor_name: str
    vendor_tax_id: str | None = None
    invoice_number: str
    invoice_date: str = Field(description="ISO 8601")
    currency: str = Field(description="ISO 4217")
    line_items: list[LineItem]
    tax: float | None = None
    total: float
    # A confidence field the model fills in itself is a cheap, surprisingly
    # reliable router: low self-reported confidence correlates well with the
    # documents a human should see.
    confidence: Literal["high", "medium", "low"]

resp = client.chat.completions.create(
    model="claude-haiku-4-5",
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "invoice", "schema": Invoice.model_json_schema()},
    },
    messages=[...],
)

invoice = Invoice.model_validate_json(resp.choices[0].message.content)

Two things are worth copying here. Fields typed as float | None rather than float give the model a legal way to say “absent” — without it, a missing tax line is a strong pull toward inventing a zero. And the self-reported confidence field costs three output tokens and routes surprisingly well; it is not calibrated, but “low” is a reliable signal that a human should look.

Two-tier routing: what production actually runs

A single model on every document is the wrong shape. Clean printed invoices — the overwhelming majority — are handled by the cheapest vision model available; the hard tail (handwriting, bad scans, unusual layouts) is where a stronger model earns its rate. Escalate on validation failure, not on a guess:

CHEAP, STRONG = "gemini-2-5-flash", "claude-sonnet-4-6"

def valid(inv: dict) -> bool:
    """Arithmetic is the guardrail no confidence score replaces."""
    items = sum(li["amount"] for li in inv["line_items"])
    return abs(items + (inv.get("tax") or 0) - inv["total"]) <= 0.02

def extract_routed(path: str) -> tuple[dict, str]:
    inv = extract(path, model=CHEAP)
    if valid(inv) and inv.get("confidence") != "low":
        return inv, CHEAP

    # Escalate: same prompt, same schema, stronger model. Only the failures
    # pay the higher rate, which is what keeps the blended cost near the
    # cheap tier's.
    inv = extract(path, model=STRONG)
    if not valid(inv):
        raise NeedsHumanReview(path, inv)
    return inv, STRONG

# Blended cost at a 90/10 split, per 10,000 invoices:
#   9,000 x $0.00042 + 1,000 x $0.00426
#   = $8.04

The arithmetic check is doing the real work: it is the one signal that is independent of the model's own opinion of itself. A document whose line items sum to the stated total is almost never wrong in a way that matters; one that fails is worth the stronger model or a human, whatever the confidence field claimed.

Cost per invoice, by model

A one-page invoice ≈ 1,800 input tokens (sent as an image) + ~350 output tokens of JSON:

Rates are live from the pricing page (about 60% under the providers' list prices), and failed requests are not billed. With two-tier routing — everything through Haiku, arithmetic-check failures escalated to Sonnet — 10,000 invoices/month typically lands under $17.04.

Accuracy: the checklist that gets you to production

1. Schema first. Mark truly-required fields as required, allow null everywhere else — forcing a value forces a hallucination.
2. Send the image, not OCR text. Layout carries meaning (columns, totals boxes); vision models read it directly. Only fall back to text for born-digital PDFs where you can extract a clean text layer.
3. Validate arithmetic in code. Line items + tax = total catches most extraction errors for free.
4. Escalate, don't retry blindly. Failures go to Sonnet with the validation error included in the prompt; persistent failures go to a human queue.
5. Few-shot the true edge cases. Two or three examples of your worst layouts (credit notes, multi-currency) in the system prompt beat any amount of instruction tuning.

Beyond invoices

The same schema-constrained pattern covers receipts, purchase orders, delivery notes, customs forms and bank statements — swap the schema, keep the guardrails. Browse the broader structured data extraction use case for RAG-adjacent patterns, or the cost optimization guide for routing strategies once volume grows. One Kunavo key covers every model tier used here — sign up free and create a key to run the snippet as-is.

FAQ

How does machine learning invoice extraction compare to using an LLM?

Classical machine-learning extractors (a CRF or SVM over OCR output plus hand-engineered position and font features) and their deep-learning successors (LayoutLM, Donut) both buy accuracy with labeled data — thousands of annotated invoices per deployment, re-labeled whenever layouts change. A vision LLM paid that cost during pretraining, so it reads an unseen vendor layout zero-shot and returns schema-constrained JSON from one call. Machine learning still wins on a single frozen, very-high-volume layout where inference cost dominates; for the long tail of real invoices the LLM is both more accurate in practice and far cheaper to own.

Can I still use LayoutLM or Donut for invoice extraction?

Yes — they remain reasonable when you process millions of documents in one stable layout and can amortize the labeling and GPU serving. What changed is the default. For a new project a vision LLM reaches production accuracy in an afternoon with no training set, and at $0.00042–$0.00426 per invoice the fine-tuning economics rarely pay back. A sensible middle path: ship the LLM first, and if volume later justifies a fine-tune, use its accepted outputs as the labeled corpus you never had to annotate by hand.

Can I extract data from PDF invoices, not just images?

Yes — render each PDF page to PNG (e.g. with pdf2image) and send it as above. For born-digital PDFs you can also extract the text layer and send it as plain text, which is cheaper; keep the image path for scans.

How accurate is LLM invoice extraction?

On clean printed invoices, schema-constrained vision extraction with the arithmetic guardrail routinely clears 98%+ field-level accuracy — and unlike a fine-tuned model, accuracy holds on layouts it has never seen. Measure on your own documents: 100 labeled invoices make a solid eval set.

Is my invoice data used for training?

No — requests through Kunavo are passed to the model providers under API-usage terms (not consumer-app terms) and are not used to train models. See the billing & data docs for retention details.