Another late night on Sunday shipping features. This week I added a caching layer to ResumeMatch that drops duplicate analysis latency from 25 seconds to 19 milliseconds. Here's how I built it, what broke along the way, and why I chose DynamoDB over Redis.
Why Cache at All?
ResumeMatch runs a 4-pass LLM pipeline for every analysis: Textract OCR, keyword extraction, match scoring, and AI resume rewriting. The whole thing takes about 25 seconds and costs roughly $0.004 per run in Bedrock tokens. Not bad for a single analysis—but when someone submits the same resume against the same job description twice (tweaking, double-checking a score, or just re-uploading by accident), they're waiting 25 seconds and burning tokens for an identical result.
The fix is straightforward: hash the inputs, check if we've seen them before, and return the cached result if we have. The interesting part is doing it reliably in a serverless architecture where there's no persistent process to hold an in-memory cache.
Why DynamoDB, Not Redis?
The obvious choice for caching is Redis (ElastiCache). But ResumeMatch is 100% serverless: Lambda, API Gateway, S3, DynamoDB. Adding an ElastiCache cluster would mean paying for an always-on instance (minimum $13+/month), wrestling with VPC config so Lambda can reach it, and introducing a new failure mode.
DynamoDB gives me everything I need here: key-value lookups in single-digit milliseconds, built-in TTL for automatic expiry, pay-per-request pricing that costs nothing when idle, and it's already in my stack. The caching patterns are basically identical to Redis—deterministic keys, TTL expiry, hit/miss logic, fallback on failure. The only real tradeoff is sub-millisecond reads, but when your backend pipeline takes 25 seconds, 4ms vs 0.5ms doesn't matter.
Cache Key Design
The cache key has to be perfectly deterministic: the same resume text and job description must always hash to the same value, even with differences in whitespace, line breaks, or trailing spaces.
The approach: normalize each input separately (lowercase, collapse whitespace, trim), combine them into a JSON object with sorted keys, and SHA-256 hash the result. The key is namespaced with a version prefix for future-proofing.
def build_cache_key(resume_text, job_description):
def normalize(text):
return re.sub(r'\s+', ' ', text.strip().lower())
payload = json.dumps({
"resume": normalize(resume_text),
"jd": normalize(job_description)
}, sort_keys=True)
hash_hex = hashlib.sha256(payload.encode('utf-8')).hexdigest()
return f"v1#analysis#{hash_hex}"
A few details that actually matter: I normalize the inputs before combining them (concatenating first can mess with delimiters and break determinism). The sort_keys=True prevents JSON key ordering from affecting the hash. And the v1#analysis# prefix lets me share the same table for other cache types later—like v1#questions#<hash> for interview question banks.
Cache Read with Manual TTL Validation
DynamoDB's built-in TTL is handy, but there's a gotcha: TTL deletion can lag up to 48 hours. The service guarantees eventual deletion, but a GetItem can still return an item whose expiresAt timestamp is already in the past.
The fix is a manual TTL check after every read. If the item exists but is expired, treat it as a miss and let the pipeline run fresh.
expires_at = item.get('expiresAt', 0)
if int(time.time()) >= int(expires_at):
# Expired but not yet deleted by DynamoDB TTL
return None
I chose a 48-hour TTL for successful results. During active job searching, someone might re-run the same combo within a session, but after two days they've likely updated their resume. Failed results (model completed but output was poor) get a 10-minute TTL so they're retried quickly.
Compression for Large Payloads
DynamoDB has a 400KB item size limit. Most analysis results are well under that (mine clocked in at ~14KB), but the result includes the full resume text, rewritten text, keyword lists, and score breakdowns. A long resume with detailed suggestions could approach the limit.
The write path checks serialized size and decides:
- Under 200KB: store as a raw JSON map in a
resultattribute - 200KB to 350KB: gzip compress and store as binary in
resultCompressed - Over 350KB: skip caching entirely (too close to the 400KB limit after attribute overhead)
The two paths use mutually exclusive attributes (result vs resultCompressed) with a compressed boolean flag. On read, the flag tells the code which path to decode. This avoids any ambiguity about how to interpret the stored data.
Fallback Isolation
This was my biggest design priority: cache failures must never break the user experience. The cache is a nice-to-have optimization, not a hard dependency.
Cache read and cache write are wrapped in separate try/except blocks. If the cache read fails (DynamoDB timeout, permission error, malformed item), the pipeline runs normally. If the cache write fails after a successful pipeline run, the user still gets their result. The only evidence of failure is a structured log entry.
try:
cached = cache_read(cache_key)
except Exception as e:
log_warning("cache_read_error", error=str(e))
cached = None
source = 'fallback'
# Pipeline always runs if no cached result
try:
cache_write(cache_key, result)
except Exception as e:
log_warning("cache_write_error", error=str(e))
The pipeline itself is never wrapped in the cache error handling. This is deliberate: a cache failure and a pipeline failure are different categories of problems. A cache failure is an optimization miss. A pipeline failure is a user-facing error.
Three-State Source Tracking
Every completed request logs one of three source states: hit, miss, or fallback.
- hit: Valid cache result, pipeline skipped entirely
- miss: No cache result, pipeline ran, and we cached for next time
- fallback: Cache layer failed, pipeline ran anyway
This is more useful than a binary hit/miss boolean. The fallback state tells me the cache had an infrastructure problem (permissions, timeout, corruption) without me needing to grep through error logs. Both source and cacheLatencyMs are written to every analysis record in DynamoDB, so I can compute hit rates and latency comparisons from the same data the frontend already fetches.
Test Matrix
I ran 10 test cases covering the full range of cache behavior before shipping:
| # | Test Case | Result |
|---|---|---|
| 1 | Cold cache, first call | Miss, 24,879ms, wrote 14KB |
| 2 | Identical second call | Hit, 19ms (4ms DynamoDB read) |
| 3 | Malformed item (missing result field) | Treated as miss, pipeline reran |
| 4 | Expired TTL (expiresAt in the past) | Manual check caught it, miss |
| 5 | Whitespace-only JD difference | Same hash, cache hit at 44ms |
| 6 | DynamoDB GetItem denied | Fallback, pipeline ran, user unaffected |
| 7 | DynamoDB PutItem denied | Write failed silently, user got results |
| 8 | Compressed round-trip (200KB+ payload) | Gzip compress/decompress intact |
| 9 | result is None | Write skipped (code path verified) |
| 10 | Bedrock exception during pipeline | Pipeline error, cache untouched (code path verified) |
Tests 6 and 7 were done by temporarily removing IAM permissions from the Lambda role for the ResumeCache table, running an analysis, confirming the correct error was logged and the user still got their results, then restoring permissions. It's a little more effort than mocking, but it tests the real failure mode end-to-end.
The Numbers
First run (cold cache): 24,879ms total — full pipeline, 14KB written.
Second run (cache hit): 19ms total — 4ms DynamoDB read, zero Bedrock calls, zero cost.
That's a 99.9% latency drop on duplicates. The extra 15ms beyond the DynamoDB read is just Lambda overhead and the analysis table write.
Table Design Decisions
I chose DynamoDB Standard-IA (Infrequent Access) for the table class. Cache items are written once and read occasionally (only on duplicate submissions), with most items expiring via TTL without ever being read. That's a storage-dominated access pattern, which is exactly where Standard-IA saves money over Standard.
The table uses on-demand (PAY_PER_REQUEST) capacity since traffic is unpredictable. The table name is referenced via an environment variable (CACHE_TABLE_NAME) rather than hardcoded, so the same Lambda role can be extended to additional tables without code changes.
What I Chose Not to Build
ConsistentRead=True: Dropped it. Eventual consistency is usually sub-second, and for a 48-hour TTL cache, an extra pipeline run on a rare stale read is harmless. Consistent reads cost double anyway.
A full cache analytics dashboard: The data is there (every record has cacheSource and cacheLatencyMs), but building a frontend dashboard for hit rates and latency charts isn't worth the time right now. The structured CloudWatch logs give me everything I need for debugging. The dashboard can wait until there's a reason to look at cache metrics regularly.
Key Takeaways
DynamoDB makes a perfectly fine cache for serverless apps. Millisecond lookups, built-in TTL, pay-per-request, and zero extra ops. Skip ElastiCache unless you truly need sub-ms latency or fancy Redis structures.
Manual TTL checks are mandatory. DynamoDB's TTL deletion is eventually consistent and can lag up to 48 hours. Any cache read must validate expiresAt before trusting the result.
Cache failures are not pipeline failures. Wrapping cache operations in separate try/except blocks from the core pipeline ensures a DynamoDB hiccup never turns into a user-facing error. The three-state source tracking (hit, miss, fallback) makes it easy to monitor cache health without conflating it with pipeline errors.
Test the real failure modes. Temporarily revoking IAM permissions is more work than mocking, but it tests the actual error path end-to-end. My mock would have caught the AccessDeniedException, but it wouldn't have caught a hypothetical bug in how boto3 surfaces that error.
The caching layer is live at ResumeMatch. Try submitting the same resume and job description twice and watch the second result come back instantly.
Stay inspired.