How I Fixed False Negative Keyword Matching in My LLM-Powered Resume Analyzer

Happy Friday! Weekend's almost here, but before I sign off, I wanted to write up a fix I shipped this week for ResumeMatch. A user in the accounting field reported that keywords clearly on her resume were showing up as missing in the analysis results. Turns out the LLM was doing string matching when it should have been doing concept matching. Here's how I fixed it without adding any new infrastructure.

The Problem: Users Don't Trust Incorrect Results

ResumeMatch is a serverless app I built on AWS that scores how well a resume fits a job description. It uses a multi-stage LLM pipeline: Textract extracts text, Bedrock (Claude Haiku) extracts keywords from the JD, compares them to the resume, scores the match, and suggests improvements.

The pipeline worked well in most cases. Then a user in the accounting field filed a bug report that surfaced a fundamental flaw.

Her resume listed ONESOURCE under Technical Skills. The job description required Thomson Reuters OneSource Tax System. The LLM marked it as missing. Her resume listed Microsoft Office Suite. The JD required Microsoft Word and Microsoft PowerPoint. Both marked as missing.

The result: 72% match with 12 of 28 keywords matched — a score that didn't reflect her actual qualifications.

At the same time, I tested my own software engineering resume against an ad-tech role. The JD required Go. My resume lists Go in a comma-separated skills line: "Python, TypeScript, JavaScript, C, C++, C#, Java, Go, PHP...". The LLM missed it entirely.

Two distinct failure modes. Both erode user trust. Both needed a systematic fix, not a per-user patch.

Why Prompt Engineering Alone Isn't Enough

My first instinct was to fix the LLM prompt. I added explicit matching rules telling the model to treat abbreviations as equivalent, umbrella terms as covering sub-components, and to carefully read comma-separated skill lists.

This helped. Thomson Reuters OneSource Tax System moved from missing to matched. But Microsoft Word and Microsoft PowerPoint still showed as missing even though Microsoft Office Suite was on the resume. And the short keyword problem, Go, was inconsistent across runs.

The core issue is that LLMs are non-deterministic. The same prompt with the same input produces different results on different runs. Haiku might catch "Office Suite covers Word" on one analysis and miss it on the next. For a product that users rely on, this inconsistency is unacceptable.

Design principle: Use the LLM for what it's good at — semantic understanding, scoring, and reasoning — and use deterministic Python code for what needs to be reliable: string matching, abbreviation detection, and umbrella term coverage.

This is the same pattern I already use for years-of-experience detection: the LLM extracts the requirement, Python parses the dates programmatically. Now I applied it to keyword matching.

The Architecture: LLM Proposes, Python Validates

The validation layer sits between Pass 2, the LLM keyword comparison, and Pass 2.5, the top missing keyword ranking. It takes the LLM's missingKeywords list and checks each one against the resume text using five deterministic checks.

1. Pass 1: Extract keywords from the job description.
2. Pass 2: Compare the resume against those keywords with the LLM.
3. Python validation: Correct false negatives using deterministic checks.
4. Pass 2.5: Rank the top missing keywords.
5. Pass 3: Rewrite resume bullets using the corrected keyword list.

The key insight: the validation layer only moves keywords from missing to present, correcting false negatives. It never moves keywords from present to missing. This means it can only improve results, never make them worse.

The Five Checks

1. Umbrella Term Coverage

   If the resume says "Microsoft Office Suite", that covers Word, Excel, PowerPoint, and Outlook. A dictionary maps umbrella terms to their sub-components. This only fires when the child name shares zero words with the parent, which is rare outside of software suites.

2. Direct Word-Boundary Match

   The full keyword text appears as whole words in the resume. It uses \b regex boundaries so "java" doesn't match inside "javascript" and "sql" doesn't match inside "mysql".

3. Abbreviation / Product Name Detection

   If a significant word, six or more characters and not a common English word, from the resume appears inside the keyword, or vice versa, it's a match. "onesource" from the resume appears inside "thomson reuters onesource tax system" as a distinct word. This is the workhorse check that handles any industry without hardcoding.

4. Token Overlap for Multi-Word Keywords

   If 75% or more of a keyword's significant tokens appear in the resume, it's a near-match. "estimated tax payments" on the resume matches "quarterly estimated tax payments" in the JD: three out of four tokens, or 75%. This requires at least three overlapping tokens to avoid false positives.

5. Context-Aware Short Keyword Matching

   For ambiguous terms like "Go", "C", and "R" that are both programming languages and common English words, this check only matches if the term appears in a skills-list context: comma-separated, near technical indicator words like "programming", "backend", or "tools".

Hardening Against Edge Cases

The initial implementation worked, but it had critical false positive risks. Every check went through stress testing.

The "react" inside "reactive" problem

The abbreviation check originally used Python's in operator for substring matching. This meant "react", from React on the resume, would match inside "reactive programming" as a keyword. Same issue with "power" from Power BI matching "powerpoint", and "graph" from graph algorithms matching "graphql".

Fix: Switched to word-boundary regex matching and added a blocklist of roughly 200 common English words that should never trigger abbreviation matching.

Before — unsafe:

# "react" matches inside "reactive programming"
if token in kw_lower:
    matched = True

After — safe:

# "react" only matches the standalone word "react"
if (len(token) >= 6
        and token not in COMMON_ENGLISH_BLOCKLIST
        and word_boundary_match(token, kw_lower)):
    matched = True

Score staleness after validation

The LLM scores 72% based on 12 matched keywords. After validation corrects four more to present, the count is 16, but the score still says 72%. Users see a mismatch between the score circle and the keyword chips.

Fix: Added adjust_score_after_validation(), which proportionally bumps the score based on the correction delta.

OCR artifacts from Textract

Textract sometimes splits compound words: "C++" becomes "C + +", ".NET" becomes ". NET", and "Node.js" becomes "Node. js". These fail every string-matching check.

Fix: Added normalize_ocr_artifacts() with 20+ regex patterns for common Textract splitting issues. It runs before validation so the cleaned text feeds into all five checks.

LLM output structure inconsistency

Haiku occasionally nests the scoreSummary string inside the scoreBreakdown object instead of returning it as a sibling key. The code was doing int(v) on every value in the breakdown dictionary, which crashed when it hit a sentence of English text.

Fix: Added type checking to skip non-numeric values and a fallback lookup for scoreSummary in both possible locations.

# Before: crashes on unexpected string values
score_breakdown = {
    k: int(v) for k, v in pass2_result.get('scoreBreakdown', {}).items()
}

# After: handles both possible locations and types
score_breakdown = {
    k: int(v) for k, v in pass2_result.get('scoreBreakdown', {}).items()
    if isinstance(v, (int, float))
       or (isinstance(v, str) and v.strip().isdigit())
}
score_summary = (
    pass2_result.get('scoreSummary', '')
    or pass2_result.get('scoreBreakdown', {}).get('scoreSummary', '')
)

What's Hardcoded vs. Algorithmic

A natural concern: does this approach require maintaining per-industry dictionaries?

Check	Type	Maintenance
Check 1: Umbrella terms	Data-driven	Roughly 15 entries total, universal terms like Office Suite, AWS, and Adobe Creative Cloud. Rarely changes.
Check 2: Word boundary	Algorithmic	Zero. Works on any keyword in any industry.
Check 3: Abbreviation	Algorithmic	Zero. Detects shared words between any abbreviation and full name.
Check 4: Token overlap	Algorithmic	Zero. Percentage-based and industry-agnostic.
Check 5: Short keywords	Data-driven	Roughly 25 ambiguous terms. Mostly programming languages. Stable set.

Checks 2, 3, and 4 handle approximately 90% of corrections with zero configuration. ONESOURCE matching Thomson Reuters OneSource Tax System is Check 3. Epic matching Epic Systems EHR is Check 3. Bloomberg matching Bloomberg Terminal is Check 3. All algorithmic, all industry-agnostic.

What the Validation Layer Doesn't Catch

There are categories of matches that require genuine semantic understanding.

Scenario	LLM	Python	Together
ONESOURCE → Thomson Reuters OneSource	Sometimes	Always	Always
Office Suite → Microsoft Word	Sometimes	Always	Always
"Go" in a skills list	Sometimes	Always	Always
"client calls" → verbal communication	Usually	Never	Usually
"ASC 740 work" → GAAP knowledge	Usually	Never	Usually

The semantic leaps — "client calls" implies verbal communication, and "ASC 740 work" implies GAAP knowledge — remain in the LLM's domain. That's the right division of labor: deterministic checks for string-level matching, LLM reasoning for concept-level inference.

If enough semantic misses accumulate from user feedback, the next step would be an embedding-based similarity layer using Bedrock Titan Embeddings. But the current approach resolves the reported issues without adding infrastructure complexity or API cost.

Implementation

The validation function runs in the Lambda handler between Pass 2 and Pass 2.5. It adds zero API calls, only pure Python string operations, and negligible latency.

def validate_keyword_matches(missing, present, resume_text):
    resume_clean = normalize_ocr_artifacts(resume_text)
    resume_lower = resume_clean.lower()
    resume_tokens = set(re.findall(r'\b[\w#+.]+\b', resume_lower))

    # Pre-compute umbrella coverage
    umbrella_covers = set()
    for umbrella, components in UMBRELLA_TERMS.items():
        if umbrella in resume_lower:
            umbrella_covers.update(components)

    for keyword in missing:
        # Run 5 checks in order...
        # If any check matches, move to present
        # Otherwise, stays in missing

    return corrected_missing, list(dict.fromkeys(corrected_present))

Downstream consumers — Pass 2.5 ranking, Pass 3 rewrite, and the suggestions list — all receive the corrected keyword lists. The top-five missing keyword ranking doesn't waste slots on false negatives, the rewrite doesn't inject keywords the user already has, and suggestions are filtered to remove corrected items.

Results

Accounting resume — tax associate role

Version	Score	Matched keywords
Before	72%	12 of 28 keywords matched
After	81%	20 of 31 keywords matched

Keywords correctly moved from missing to matched: Thomson Reuters OneSource Tax System, Microsoft Word, Microsoft Excel, Microsoft PowerPoint, estimated tax payments, and tax filings. The remaining 11 missing keywords — GAAP, IFRS, and soft skills — are legitimately absent from the resume.

Software engineering resume — ad-tech backend role

Missing keywords: data structures, algorithms, distributed systems, high-performance systems, low-latency systems, header bidding, and leadership. All are legitimately not on the resume for this specialized role. No false negatives were detected. The system correctly assessed this as a moderate match for a domain-specific position.

Key Takeaways

1. Trust LLMs for reasoning, not consistency. The LLM is great at understanding that "ASC 740" implies GAAP knowledge. It's unreliable at consistently catching that ONESOURCE equals Thomson Reuters OneSource Tax System. Use each tool for what it's good at.

2. Post-processing validation is cheap insurance. The validation layer adds zero API calls and negligible compute. It's pure Python string operations that run in milliseconds. The ROI compared to the user trust it preserves is enormous.

3. Most keyword matching is algorithmically solvable. Abbreviation detection, umbrella term coverage, and token overlap handle roughly 90% of false negatives without any industry-specific configuration.

4. Always defend against LLM output variance. Any field the LLM returns can have unexpected types or nesting. Type-check everything, have fallbacks, and never assume JSON structure is stable across runs.

---

Explore the live app at ResumeMatch. If you're building LLM-powered tools, consider where deterministic validation can catch what your model misses.