While preparing for the AWS SAP-C02, many candidates get confused by when to use Lambda vs. EC2 for asynchronous processing. In the real world, this is fundamentally a decision about execution time limits, cost elasticity, and message broker compatibility. Let’s drill into a simulated scenario.
The Scenario #
GlobalMedia Studios operates a video transcoding platform where creative professionals upload raw footage via a web portal. Currently, uploaded files are stored on a Network Attached Storage (NAS) appliance, and a legacy message queue notifies on-premises processing servers to begin transcoding. Each video file requires up to 60 minutes of compute time to process.
The company observes a predictable usage pattern: upload and processing demand spikes sharply during business hours (8 AM–6 PM) and drops to near-zero overnight. The existing infrastructure is over-provisioned to handle peak load, resulting in 70% idle capacity during off-hours.
Leadership has mandated migration to AWS to improve cost efficiency and eliminate infrastructure overhead.
Key Requirements #
Design a cloud-native architecture that:
- Minimizes Total Cost of Ownership (TCO) by eliminating over-provisioning.
- Handles variable workload peaks without manual intervention.
- Supports processing tasks lasting up to 1 hour per file.
- Requires minimal changes to the existing web server’s message publishing logic.
The Options #
-
A) Use Amazon SQS to create a queue; configure the existing web server to publish messages to the new queue; trigger AWS Lambda functions to pull requests from the queue and process files when messages arrive; store processed files in an Amazon S3 bucket.
-
B) Use Amazon MQ to create a queue; configure the existing web server to publish messages to the new queue; create new Amazon EC2 instances to pull requests from the queue and process files when messages arrive; store processed files in Amazon EFS; terminate EC2 instances after task completion.
-
C) Use Amazon MQ to create a queue; configure the existing web server to publish messages to the new queue; trigger AWS Lambda functions to pull requests from the queue and process files when messages arrive; store processed files in Amazon EFS.
-
D) Use Amazon SQS to create a queue; configure the existing web server to publish messages to the new queue; use Amazon EC2 instances within an Auto Scaling group to pull requests from the queue and process files; scale EC2 instances based on SQS queue depth; store processed files in an Amazon S3 bucket.
Correct Answer #
D
💎 The Architect’s Deep Dive: Why Options Fail #
Correct Answer #
Option D
Step-by-Step Winning Logic #
Option D represents the optimal trade-off for this scenario because:
- SQS meets the queueing requirement with lower cost and operational overhead than Amazon MQ, and requires minimal refactoring of the existing web server’s message publishing logic.
- EC2 Auto Scaling provides elastic compute that can handle 1-hour processing tasks (far exceeding Lambda’s 15-minute limit) while automatically scaling based on SQS ApproximateNumberOfMessagesVisible metric.
- S3 storage is the most cost-effective and durable option for storing processed media files (vs. EFS, which is designed for shared file systems and incurs higher per-GB costs).
- FinOps Impact: Auto Scaling eliminates the 70% idle capacity waste by scaling down to zero instances during off-hours, aligning cost with actual demand.
The Traps (Distractor Analysis) #
-
Why not Option A?
Lambda has a maximum execution timeout of 15 minutes. Processing tasks that can take up to 1 hour will fail or require complex workarounds (e.g., Step Functions with chunking), introducing architectural complexity and potential data integrity issues. This is a fundamental incompatibility. -
Why not Option B?
Amazon MQ is a managed message broker for lift-and-shift migrations of applications using protocols like AMQP or MQTT. The scenario does not indicate the web server requires these protocols—using MQ here introduces unnecessary cost (~$30–$50/month baseline) and operational complexity when SQS would suffice. Additionally, manually creating and terminating EC2 instances is operationally inefficient compared to Auto Scaling, and EFS storage is more expensive than S3 for static processed files. -
Why not Option C?
This combines two critical flaws: Lambda’s 15-minute limit (same as Option A) and the use of Amazon MQ (same as Option B), plus EFS adds unnecessary cost for object storage use cases better served by S3.
The Architect Blueprint #
graph TD
User([Creative Professional]) -->|Upload via HTTPS| WebServer[Existing Web Server]
WebServer -->|Store Raw File| S3Raw[Amazon S3
Raw Media Bucket]
WebServer -->|Publish Message| SQS[Amazon SQS Queue]
SQS -->|ApproximateNumberOfMessages| CW[CloudWatch Metric]
CW -->|Trigger Scale-Out| ASG[EC2 Auto Scaling Group]
ASG -->|Poll Queue| EC2[EC2 Worker Instances
c5.xlarge]
EC2 -->|Read Raw File| S3Raw
EC2 -->|Transcode Video
Up to 60 min| Processing[FFmpeg/Custom Codec]
Processing -->|Write Processed File| S3Processed[Amazon S3
Processed Media Bucket]
EC2 -->|Delete Message| SQS
CW -->|Trigger Scale-In
Queue Empty| ASG
ASG -->|Terminate Instances| EC2
style SQS fill:#FF9900,stroke:#232F3E,stroke-width:2px,color:#fff
style S3Processed fill:#569A31,stroke:#232F3E,stroke-width:2px,color:#fff
style ASG fill:#FF4F00,stroke:#232F3E,stroke-width:2px,color:#fff
Diagram Note: Users upload to the web server, which stores raw files in S3 and publishes SQS messages. EC2 instances in an Auto Scaling group poll the queue, process files (up to 1 hour), write results to S3, and auto-terminate when the queue is empty—achieving zero cost during idle periods.
The Decision Matrix #
| Option | Est. Complexity | Est. Monthly Cost (Quantified) | Pros | Cons |
|---|---|---|---|---|
| A (SQS + Lambda + S3) | Low | Low ($80–$150) • SQS: ~$10 • Lambda: $50–$100 (assuming failures/retries) • S3: ~$20 |
• Serverless simplicity • No instance management • S3 storage cost-efficient |
• Lambda 15-min timeout = architectural blocker • Requires workarounds (Step Functions, chunking) • Higher retry costs due to failures |
| B (MQ + EC2 Manual + EFS) | High | High ($450–$650) • Amazon MQ: ~$40 • EC2 (manual, inefficient): $200–$400 • EFS: ~$200 (1TB @ $0.30/GB) • S3: N/A |
• Supports AMQP/MQTT (not needed here) • EFS shared file system (not needed here) |
• Amazon MQ unnecessary cost • Manual EC2 lifecycle = operational burden • EFS 10x more expensive than S3 for static files • No auto-scaling = waste |
| C (MQ + Lambda + EFS) | Medium | Medium ($250–$350) • Amazon MQ: ~$40 • Lambda: $50–$100 (with failures) • EFS: ~$200 |
• Managed broker (overkill) | • Lambda 15-min timeout (fatal flaw) • Amazon MQ unnecessary • EFS cost inefficiency |
| D (SQS + EC2 ASG + S3) ✅ | Medium | Low-Medium ($120–$250) • SQS: ~$10 • EC2 (ASG, scales to 0): $80–$200 (demand-based) • S3: ~$20 • CloudWatch: ~$10 |
• Supports 1-hour tasks • Auto Scaling = zero cost at night • SQS simple, low-cost • S3 optimal for media storage • Aligns cost with demand |
• Requires EC2 instance patching/management • Slightly more complex than serverless (if Lambda were viable) |
FinOps Winner: Option D delivers the lowest TCO when factoring in elastic scaling (scales to zero), no wasted capacity, and S3’s $0.023/GB storage cost vs. EFS’s $0.30/GB.
Real-World Practitioner Insight #
Exam Rule #
For the SAP-C02 exam, when you see:
- “Processing time up to 1 hour” → Eliminate Lambda (15-minute hard limit).
- “Variable workload” + “cost optimization” → Auto Scaling Group (not manual EC2).
- “Minimal changes to existing app” → SQS (not Amazon MQ unless AMQP/MQTT/JMS is explicitly required).
- “Store processed media files” → S3 (not EFS unless shared file access is needed).
Real World #
In production at GlobalMedia Studios, we would:
- Use SQS FIFO queues if processing order matters (e.g., episode sequences).
- Implement EC2 Spot Instances within the Auto Scaling group to reduce compute costs by 70% (acceptable for fault-tolerant transcoding).
- Add S3 Lifecycle Policies to transition processed files to S3 Glacier after 90 days, reducing storage costs by 85%.
- Use AWS Batch instead of raw EC2 Auto Scaling for better job orchestration, retry logic, and cost allocation tagging.
- Monitor SQS message age (not just depth) to prevent processing SLA violations during extreme spikes.
- Consider AWS Elemental MediaConvert for native, serverless video transcoding (though the scenario implies custom processing logic).
The exam tests the correct answer; the real world demands the optimal answer within budget, compliance, and team skill constraints.