Independent architecture study
Documents processed with checks
Accelerated Intelligent Document Processing with Generative AI
A business receives documents in uneven batches and needs extracted information it can inspect. This independent architecture study follows AWS guidance to queue uploads, control processing concurrency, extract with AI and compare outputs with predefined baselines.
An independent architecture study based on public AWS guidance. A design exercise, not a client engagement.
- Workload
- PDF classification, extraction and document analysis
- Constraint
- Uneven arrivals, downstream processing limits and outputs that need checking
- Approach
- Queued workflows use a concurrency counter, document AI and baseline evaluation
What makes this difficult
Uploads arrive in bursts
A sudden batch of documents can exceed processing capacity unless intake is separated from the number of workflows allowed to run.
Extracted does not mean correct
AI can return plausible fields that differ from the expected answer, so the processing result needs evidence beyond successful execution.
People need progress
A document can wait, run or finish while its owner watches, making status tracking part of the workflow rather than decoration.
Follow an uploaded document
A PDF enters a queue, waits for processing capacity and produces extracted results with an evaluation report.
- 01
Detect the uploaded document
A client uploads a PDF through the web interface or directly to S3, and EventBridge invokes the sender function so processing can begin asynchronously.
Illustrative example Illustrative example: an uploaded invoice produces a document event.
- 02
Record it before processing
The sender records the document event through AppSync and places work on SQS, separating arrival from processing while making the document available for tracking.
Illustrative example Illustrative example: the invoice is tracked and waiting in the work queue.
- 03
Start work within capacity
The queue processor retrieves messages in batches and uses a DynamoDB counter to manage concurrency before starting Step Functions, keeping admission separate from document arrival.
Illustrative example Illustrative example: a waiting document starts when processing capacity is available.
- 04
Analyse the document
Step Functions coordinates Lambda functions that invoke Bedrock Data Automation and handle its output, turning the uploaded document into structured information for downstream evaluation.
Illustrative example Illustrative example: document analysis returns extracted invoice fields.
- 05
Compare against expected output
After processing, the evaluation function compares output with predefined baseline documents and saves JSON results and reports, making differences available for inspection rather than assuming correctness.
Illustrative example Illustrative example: extracted fields are saved alongside a baseline comparison report.
- 06
Keep the document owner informed
The Cognito-authenticated dashboard queries AppSync and receives status updates backed by DynamoDB, allowing users to follow processing progress through the CloudFront-delivered web interface.
Illustrative example Illustrative example: the dashboard shows that the document has completed processing.
- Request
- Data
- Later, queued
- Control
The hard part
A queue absorbs arrivals; a counter controls starts
Queuing a document does not itself limit how many workflows run. The queue processor also consults a DynamoDB counter before starting Step Functions. This separates the rate of incoming uploads from admitted processing. Counter updates, completion handling and recovery need careful implementation so retries do not admit excess work or leave capacity incorrectly marked as occupied.
Why this design
Use document AI for varied inputs
- Chosen
- Use Bedrock Data Automation through Lambda-orchestrated processing.
- Because
- The workload includes classification, extraction, summarisation and questions across varied documents.
- Cost
- AI output requires evaluation and introduces usage costs beyond a simple fixed-template extractor.
- Reconsider when
- Prefer a narrower extraction approach when document layouts and required fields are consistently predictable.
Control workflow admission
- Chosen
- Combine SQS with a queue processor and a DynamoDB concurrency counter.
- Because
- Document arrivals should not directly dictate concurrent demand on downstream processing services.
- Cost
- Counter consistency and recovery add coordination work to the pipeline.
- Reconsider when
- Simplify admission when another verified mechanism already enforces the required processing limit.
Keep evaluation in the workflow
- Chosen
- Compare processing output with predefined baselines and retain the evaluation report.
- Because
- A completed AI call alone does not establish that its extracted fields are correct.
- Cost
- Useful evaluation requires maintained baselines and attention to reported differences.
- Reconsider when
- Adapt the evaluation method when new document types no longer have representative baseline examples.
Cost and operating assumptions
Document analysis
Bedrock Data Automation usage and any foundation-model token usage depend on the selected processing capabilities and document workload.
Workflow execution
Step Functions transitions and Lambda requests and duration cover dispatch, processing, result handling and evaluation.
Queue and status tracking
SQS requests, DynamoDB operations and AppSync activity depend on document arrivals, concurrency coordination and status updates.
Files and dashboard delivery
S3 retains inputs, baselines, results and web assets, while CloudFront delivers the dashboard.
Assumptions behind the numbers
No monthly total is published for this guidance; pricing depends on usage, region and options, including document characteristics, AI processing, retention and dashboard activity.
How it stays reliable
Buffer incoming work
SQS separates document arrival from workflow execution; queue buffering still requires appropriate retry and failure handling in the implementation.
Respect processing capacity
The queue processor uses a DynamoDB counter to manage concurrent workflows, with counter recovery an implementation concern to validate.
Retain checking evidence
The evaluation stage stores JSON reports alongside final results, making baseline differences inspectable without treating them as a guarantee of accuracy.
Authenticate progress access
Cognito authenticates dashboard users, while AppSync and the DynamoDB tracking table maintain document status for real-time updates.
When this is a poor fit. A poor fit when a simple fixed-template extractor already meets the need or the team cannot maintain representative baselines for evaluating AI output.
Full technical detailThe numbered AWS flow, every service and the Well-Architected notes.
How the AWS guidance describes it
- Amazon S3 and Amazon EventBridge. Clients submit PDFs through the web interface or directly to the input bucket, and EventBridge detects uploads to invoke document processing.
- AWS Lambda, AWS AppSync and Amazon SQS. The Queue-Sender function records document events through AppSync and sends work to SQS.
- AWS Lambda, Amazon DynamoDB and AWS Step Functions. The Queue-Processor retrieves message batches, manages concurrency using a DynamoDB counter and starts Step Functions executions.
- AWS Step Functions and Amazon Bedrock Data Automation. The workflow coordinates Lambda functions, including a function that invokes Bedrock Data Automation and a Process-Results function that handles its output.
- AWS Lambda and Amazon S3. After processing completes, the Evaluation function compares outputs with predefined documents in the evaluation-baseline bucket and stores final results and evaluation reports as JSON in the output bucket.
- AWS AppSync and Amazon DynamoDB. AppSync maintains document status in the tracking table and enables real-time web updates, while the tracker function updates processing metrics and status.
- Amazon S3, Amazon CloudFront and Amazon Cognito. The web interface is hosted in the web-app bucket, distributed through CloudFront and authenticated with Cognito for monitoring document processing.
Well-Architected
- Operational Excellence
- Design reasoning: workflow orchestration and AppSync status tracking make document progress available for operational inspection.
- Security
- Design reasoning grounded in the flow: Cognito authenticates users of the CloudFront-delivered monitoring interface.
- Reliability
- Design reasoning: queue buffering, concurrency management and explicit workflow steps help isolate load and processing failures.
- Performance Efficiency
- Design reasoning grounded in the guidance: serverless components accommodate varying document volume without a fixed processing cluster.
- Cost Optimisation
- Design reasoning: usage-based processing links spend to document activity, with AI options and workflow complexity affecting the bill.
- Sustainability
- Design reasoning: event-driven serverless processing aligns compute activity with submitted documents.