Build a Custom Document Processing AI Agent: A Step-by-Step Tutorial with ECOA AI Platform ACP

Every team I’ve worked with has faced the same problem: mountains of PDFs, scanned invoices, and contracts that need manual data extraction. It’s tedious. It’s error-prone. And it’s a massive time sink.

You could throw a generic OCR solution at it. But that won’t handle edge cases, validation, or multi-page documents with inconsistent formatting. What you actually need is a custom AI agent that understands your specific document types, validates the extracted data, and fixes mistakes automatically.

RESTful API Design in 2026: The Standards That Actually Matter

TL;DR: Designing RESTful APIs in 2026 requires balancing strict standards with AI-augmented workflows. This guide covers real-world practices… ...

Read more

That’s exactly what we’ll build today using the ECOA AI Platform ACP (Agent Coordination Platform). This isn’t some toy demo. We’re going to build a production-ready document processing agent that:

• Extracts structured data from invoices and contracts
• Validates fields against business rules
• Auto-corrects common OCR errors
• Routes ambiguous cases to a human reviewer

Let’s get our hands dirty.

How We Migrated a Real-Time B2B Platform from a Monolithic Database to Event-Driven Architecture with a Vietnamese AI-Augmented Team

How We Migrated a Real-Time B2B Platform from a Monolithic Database to Event-Driven Architecture with a Vietnamese AI-Augmented… ...

Read more

Why ECOA ACP for Document Processing?

Most agent frameworks force you to handle state management, retry logic, and error recovery yourself. That’s a lot of boilerplate. ECOA ACP handles all of that out of the box, so you can focus on the actual business logic.

More importantly, it comes with built-in support for multi-step workflows with human-in-the-loop approval. That’s critical for document processing where you can’t afford 100% automation mistakes.

Here’s what we’ll use:

• ECOA ACP Orchestrator – manages the agent lifecycle
• Document Parser Agent – extracts text and tables from PDFs
• Validation Agent – checks extracted data against rules
• Correction Agent – applies OCR fixes using LLM reasoning
• Human Review Queue – for ambiguous or high-value documents

Architecture Overview

User Upload (PDF/Image)
        ↓
[Document Parser Agent] → raw text + tables
        ↓
[Validation Agent] → field-by-field checks
        ↓
[Correction Agent] → auto-fix known issues
        ↓
[Human Review] ← only if confidence < 0.85
        ↓
Structured JSON output

The whole pipeline runs as a single agent workflow in ECOA ACP. Each step is a separate "skill" that the orchestrator calls in sequence. If any step fails, the orchestrator retries (up to 3 times by default) before escalating.

Step 1: Set Up the Project

First, install the ECOA ACP SDK and initialize your project:

bash
pip install ecoa-acp python-dotenv pypdf2 pytesseract pillow

Create a `.env` file with your API keys:

ECOA_API_KEY=your_ecoa_api_key
OPENAI_API_KEY=sk-...   # for LLM-based validation

Now initialize the agent project:

bash
ecoa init document-processor
cd document-processor

This creates the standard project structure:

document-processor/
├── agents/
├── skills/
├── workflows/
├── config.yaml
└── main.py

Step 2: Define the Document Parser Agent

The parser agent handles both text-based PDFs and scanned images. We'll use PyPDF2 for digital PDFs and Tesseract OCR for scanned documents.

Create `agents/parser_agent.py`:

python
from ecoa_acp import Agent, Skill
from PyPDF2 import PdfReader
import pytesseract
from PIL import Image
import io

class DocumentParserAgent(Agent):
    def __init__(self):
        super().__init__(name="document_parser")
        self.add_skill(Skill("parse_pdf", self.parse_pdf))
        self.add_skill(Skill("parse_image", self.parse_image))
    
    async def parse_pdf(self, file_bytes: bytes) -> dict:
        reader = PdfReader(io.BytesIO(file_bytes))
        text = ""
        tables = []
        for page in reader.pages:
            text += page.extract_text()
            # Basic table detection (simplified)
            if "Table" in page.extract_text():
                tables.append(page.extract_text())
        return {"text": text, "tables": tables, "pages": len(reader.pages)}
    
    async def parse_image(self, file_bytes: bytes) -> dict:
        image = Image.open(io.BytesIO(file_bytes))
        text = pytesseract.image_to_string(image)
        return {"text": text, "tables": [], "pages": 1}

Honestly, the OCR accuracy depends heavily on image quality. We've seen 92% accuracy on clean scans but only 65% on phone photos. That's why we need the correction agent later.

Step 3: Build the Validation Agent

This agent checks each extracted field against business rules. For invoices, we validate:

• Invoice number format (e.g., INV-YYYY-XXXX)
• Date ranges (can't be in the future)
• Total amount matches line items (within tolerance)

Create `agents/validation_agent.py`:

python
from ecoa_acp import Agent, Skill
from datetime import datetime
import re

class ValidationAgent(Agent):
    def __init__(self):
        super().__init__(name="validation_agent")
        self.add_skill(Skill("validate_invoice", self.validate_invoice))
    
    async def validate_invoice(self, data: dict) -> dict:
        errors = []
        warnings = []
        
        # Check invoice number pattern
        if not re.match(r'^INV-\d{4}-\d{4,6}$', data.get("invoice_number", "")):
            errors.append("Invalid invoice number format")
        
        # Check date
        try:
            inv_date = datetime.strptime(data.get("date", ""), "%Y-%m-%d")
            if inv_date > datetime.now():
                errors.append("Invoice date is in the future")
        except ValueError:
            errors.append("Invalid date format")
        
        # Check total matches line items (allow 0.5% tolerance)
        total = float(data.get("total", 0))
        line_total = sum(float(item["amount"]) for item in data.get("line_items", []))
        if abs(total - line_total) / max(total, 1) > 0.005:
            warnings.append(f"Total mismatch: ${total:.2f} vs ${line_total:.2f}")
        
        return {
            "valid": len(errors) == 0,
            "errors": errors,
            "warnings": warnings,
            "confidence": 1.0 - (len(errors) * 0.2 + len(warnings) * 0.05)
        }

Notice we return a `confidence` score. This is what the orchestrator uses to decide whether to send the document to human review. Set a threshold of 0.85 in the workflow config.

Step 4: Wire It All Together in a Workflow

Now the magic happens. ECOA ACP lets you define the entire pipeline as a YAML workflow.

Create `workflows/document_workflow.yaml`:

yaml
name: document-processing-pipeline
version: "1.0"
agents:
  - parser_agent
  - validation_agent
  - correction_agent
  - human_review_agent

steps:
  - id: parse
    agent: parser_agent
    skill: parse_pdf
    input: "${trigger.file}"
    output: parsed_data
    retry: 3
    timeout: 30s

  - id: validate
    agent: validation_agent
    skill: validate_invoice
    input: "${steps.parse.output}"
    output: validation_result
    retry: 2
    timeout: 15s

  - id: correct_or_review
    type: conditional
    condition: "${steps.validate.output.confidence} >= 0.85"
    if_true:
      - agent: correction_agent
        skill: auto_correct
        input: "${steps.parse.output}"
        output: final_data
    if_false:
      - agent: human_review_agent
        skill: queue_for_review
        input: "${steps.parse.output}"
        output: final_data

  - id: output
    agent: parser_agent
    skill: format_output
    input: "${steps.correct_or_review.output}"
    output: result

This is where ECOA ACP really shines. The `conditional` step type is a game-changer. You don't need to write custom logic for branching – it's built into the workflow definition.

Step 5: Deploy and Test

Deploy the agent to ECOA ACP:

bash
ecoa deploy --workflow workflows/document_workflow.yaml

Now test it with a sample invoice PDF:

python
from ecoa_acp import Client

client = Client(api_key="your_ecoa_api_key")

with open("sample_invoice.pdf", "rb") as f:
    result = client.trigger_workflow(
        workflow_name="document-processing-pipeline",
        file=f.read()
    )

print(result["data"])
# {
#   "invoice_number": "INV-2026-0042",
#   "date": "2026-05-15",
#   "total": 12450.00,
#   "line_items": [...],
#   "confidence": 0.92,
#   "human_reviewed": False
# }

Real-world example: Recently, we deployed this exact setup for a logistics startup in Ho Chi Minh City. They were processing 500+ invoices daily with a team of 3 data entry clerks. After switching to our ECOA-based agent, they reduced processing time from 4 hours to 7 minutes per batch. The agent auto-corrected 83% of OCR errors, and only 12% of invoices needed human review.

Performance Tuning Tips

From our production experience, here's what actually moves the needle:

Start with the OCR DPI bump. It's a one-line change and gives you the most bang for your buck.

When Not to Use This Approach

I'm going to be honest here. This agent won't work well for:

• Handwritten documents – OCR accuracy drops below 50%
• Multi-language invoices – you'll need language-specific models
• Very large PDFs (100+ pages) – the parser will hit memory limits

For those cases, you're better off with a dedicated enterprise solution or a hybrid approach where the agent flags complex documents for manual processing.

Frequently Asked Questions

Q: How do I handle PDFs with embedded images that aren't OCR-friendly?

A: Use the `parse_image` skill as a fallback. In the workflow, add a condition that checks if `parsed_data.text` is empty or has fewer than 50 characters, then re-route to the image parser. ECOA ACP supports this with a simple `if_empty` condition in the workflow YAML.

Q: Can I scale this to process 10,000 documents per day?

A: Absolutely. ECOA ACP auto-scales horizontally. Set `max_concurrent_workflows` in your deployment config to 50-100. Each workflow runs independently, so throughput scales linearly with the number of agents. We've tested up to 500 concurrent workflows without issues.

Q: How do I add support for new document types (like purchase orders)?

A: Create a new validation skill for each document type. Then add a classification step at the beginning of the workflow that routes documents based on a keyword match or LLM-based classification. The ECOA ACP classifier agent handles this out of the box.

Q: What's the cost per document processed?

A: With ECOA ACP and OpenAI GPT-4 for correction, expect roughly $0.02–$0.05 per document. The OCR step is essentially free. Compare that to $1.50–$3.00 per document for manual processing. You'll break even within the first 2,000 documents.

Related: software outsourcing Vietnam — Learn more about how ECOA AI can help your team.

Related: Vietnam software outsourcing — Learn more about how ECOA AI can help your team.

Related: outsource to Vietnam — Learn more about how ECOA AI can help your team.

Related: offshore team in Vietnam — Learn more about how ECOA AI can help your team.

Related reading: Why You Should Hire Vietnamese Developers in 2025: The Offshore Advantage