Skip to content

Processing Pipeline

This document details the step-by-step processing pipeline from incoming HTTP request to outbound response. The pipeline is split between guvi/handler.py (top-level orchestration, session management, callback dispatch) and engine/orchestrator.py (LLM classification, persona response generation, strategy self-correction).

Sequence Diagram

sequenceDiagram
    participant GUVI as GUVI Evaluator
    participant CF as Cloud Function<br/>guvi_honeypot
    participant AUTH as validate_api_key()
    participant RL as rate_limiter<br/>check_rate_limit()
    participant FS as Firestore<br/>honeypot_sessions
    participant EXT as Extractors<br/>regex + keywords
    participant EI as Evidence Index<br/>evidence_index
    participant ORCH as Orchestrator
    participant GEMINI as Gemini Flash
    participant CB as Callback Service

    GUVI->>+CF: POST /guvi_honeypot<br/>{sessionId, message, history, metadata}
    CF->>AUTH: Validate x-api-key header
    AUTH-->>CF: OK / 401

    CF->>RL: check_rate_limit(sessionId)
    RL->>FS: Transactional counter read/write
    RL-->>CF: (allowed, reason)

    Note over CF: If rate limited, return stalling reply

    CF->>FS: get_session(sessionId)
    FS-->>CF: SessionState or None

    Note over CF: Create new session if None

    CF->>EXT: extract_all_evidence(all scammer messages)
    EXT-->>CF: evidence_dict {upiIds, bankAccounts, ...}

    CF->>EXT: extract_suspicious_keywords(all scammer messages)
    EXT-->>CF: keywords[]

    CF->>EI: find_matching_evidence(evidence_dict)
    EI->>FS: Query evidence_index collection
    EI-->>CF: cross_session_match<br/>{is_known_scammer, matches, known_scam_types}

    CF->>+ORCH: process(session, message, history, metadata,<br/>cross_session_match, context, pre_extracted_evidence)

    Note over ORCH: Step 0: Inject quality directives into PipelineContext

    ORCH->>+GEMINI: classify_scam(message, context, language)
    GEMINI-->>-ORCH: {classification, confidence, red_flags}

    Note over ORCH: Step 1b: Apply cross-session confidence boost

    Note over ORCH: Step 2: select_persona(scam_type, language)

    Note over ORCH: Step 3: Use pre-extracted evidence (skip redundant extraction)

    Note over ORCH: Step 4: determine_strategy_adjustment()

    ORCH->>+GEMINI: generate_persona_response(persona_prompt,<br/>scam_type, message, history, strategy, context)
    GEMINI-->>-ORCH: persona_response_text

    Note over ORCH: Step 5: determine_state() (state machine transition)

    ORCH-->>-CF: ProcessingResult<br/>{response, scam_type, confidence, evidence,<br/>persona_used, state, strategy_state}

    CF->>CF: merge_evidence_locally(existing, new)

    CF->>FS: store_evidence_index(sessionId, merged_evidence)

    CF->>FS: batch_update_session(sessionId, {<br/>conversation_history, evidence, message_count,<br/>scam_type, confidence, state, persona, strategy})

    Note over CF: Compute scamDetected, agentNotes, duration

    alt message_count >= CALLBACK_MIN_TURN (1)
        CF->>+CB: send_final_result(session, scam_type, message_count)
        CB->>GUVI: POST /updateHoneyPotFinalResult<br/>{sessionId, scamDetected, extractedIntelligence, ...}
        GUVI-->>CB: 200 OK
        CB-->>-CF: success
    end

    CF-->>-GUVI: 200 GuviResponse<br/>{reply, scamDetected, extractedIntelligence,<br/>engagementMetrics, agentNotes}

Pipeline Steps in Detail

Step 1: Authentication

File: guvi/handler.py -- validate_api_key()

provided_key = request.headers.get("x-api-key", "")
return provided_key == expected_key
  • In production (K_SERVICE env var set): requires SCAMSHIELD_API_KEY from Secret Manager.
  • In development: allows all requests if key is not configured.
  • Returns 401 with GuviErrorResponse on failure.

No constant-time comparison

The current implementation uses == for key comparison. The CLAUDE.md references a hardened constant-time comparison fix (commit ba3bc09), but the current code uses standard equality. For a hackathon context this is acceptable; production deployments should use hmac.compare_digest().

Step 2: Rate Limiting

File: utils/rate_limiter.py -- check_rate_limit()

Uses a Firestore transactional counter per session with two limits:

Limit Value Scope
MAX_MESSAGES_PER_SESSION 100 Lifetime of a session
MAX_MESSAGES_PER_MINUTE 10 Sliding 60-second window

The minute window is tracked via minute_key = int(now / 60). When the key changes, the per-minute counter resets. If Firestore is unavailable, requests are allowed (fail-open for availability).

When rate-limited, the handler returns a valid GuviResponse with a stalling reply in Hinglish:

"Ek minute ruko beta, bahut zyada messages aa rahe hain."

Step 3: Session Management

File: guvi/handler.py -- get_or_create_session()

Sessions are loaded from Firestore's honeypot_sessions collection keyed by sessionId. If no session exists, a new SessionState is created with:

  • Default persona: sharma_uncle
  • State: INITIAL
  • Empty ExtractedIntelligence
  • Strategy state: BUILDING_TRUST

The session is immediately persisted to Firestore so it survives cold starts.

Step 4: Evidence Extraction (Early)

File: guvi/handler.py -- _extract_evidence_from_full_conversation()

Evidence is extracted before the orchestrator runs. This serves two purposes:

  1. The evidence is available for cross-session lookup (Step 5).
  2. The orchestrator receives pre-extracted evidence, avoiding redundant extraction.

The function concatenates all non-honeypot messages from conversationHistory plus the current message, then runs:

  • extract_all_evidence() -- 14 regex extractors (see Evidence Extraction)
  • extract_suspicious_keywords() -- 11 weighted keyword categories
  • transform_to_guvi_format() -- converts snake_case keys to camelCase

Step 5: Cross-Session Evidence Lookup

File: firestore/sessions.py -- find_matching_evidence()

Queries the evidence_index collection for any UPI IDs, bank accounts, phone numbers, or emails that appeared in previous sessions. Returns:

{
    "matches": [...],                    # List of matching evidence items
    "total_matching_sessions": 3,        # Unique session count
    "known_scam_types": ["KYC_BANKING"], # Scam types from prior sessions
    "is_known_scammer": True,            # True if UPI/bank match found
}

When is_known_scammer is True, the orchestrator applies a confidence boost and injects aggressive extraction tactics into the persona prompt.

Step 6: Pipeline Context Creation

File: engine/context.py -- PipelineContext

A dataclass carrying enrichment data through the pipeline. Supports priority-ordered prompt sections:

Priority Label Content
10 edge-case Bot detection, edge case handling
20 language Language/script instructions
25 quality Turn-aware quality directives (injected by orchestrator)
30 engagement Conversation-stage guidance

Sections are sorted by priority and concatenated into the final Gemini prompt via get_prompt_sections().

Step 7: Orchestrator Processing

File: engine/orchestrator.py -- Orchestrator.process()

The orchestrator executes 6 sub-steps:

7a. Quality Directive Injection

Turn-aware quality directives are injected into PipelineContext at priority 25. These guide Gemini to produce responses scoring well on the evaluation rubric:

  • Turns 1-3 (Early Engagement): 2 identity verification questions, 1 red flag, show willingness.
  • Turns 4-6 (Investigation): 2 investigative questions, 2 red flags, elicit phone/email, demand proof.
  • Turns 7+ (Extraction): Demand ID proof, 2+ red flags, final push for phone/email/address/supervisor.

7b. Classification

GeminiClient.classify_scam() runs on the current message + last 5 history messages. Returns one of 12 scam types with confidence 0.0-1.0.

Classification is re-run when:

  • State is INITIAL
  • Scam type is None, UNKNOWN, or NOT_SCAM

If Gemini fails (error or circuit breaker open), a keyword-based fallback runs using weighted keyword matching.

7c. Cross-Session Confidence Boost

For known scammers, confidence is boosted proportionally:

boost = 0.1 * min(match_count, 3)  # Max 0.3 for 3+ matches
confidence = min(confidence + boost, 0.95)

7d. Persona Selection

Two-tier lookup (see Persona System):

  1. SCAM_PERSONA_MAP[scam_type] -- base mapping
  2. LANGUAGE_PERSONA_OVERRIDES[language] -- override unless persona is in LANGUAGE_OVERRIDE_EXEMPT

7e. Strategy Self-Correction

determine_strategy_adjustment() analyzes conversation state and transitions between 4 strategy states:

stateDiagram-v2
    [*] --> BUILDING_TRUST
    BUILDING_TRUST --> EXTRACTING: msg_count >= 3 AND confidence > 0.6
    EXTRACTING --> DIRECT_PROBE: 4+ msgs without evidence
    EXTRACTING --> PIVOTING: high-value evidence obtained
    DIRECT_PROBE --> BUILDING_TRUST: scammer disengaging
    PIVOTING --> PIVOTING: continue extracting scammer identity

Each transition produces a tactics_suggestion that is injected into the Gemini prompt.

7f. Persona Response Generation

GeminiClient.generate_persona_response() assembles a prompt from:

  1. Persona system prompt (character profile, speech patterns, strategic behaviors)
  2. Known scammer context (if repeat offender)
  3. Strategy context (from self-correction)
  4. Dynamic pipeline sections (quality directives, language, edge cases)
  5. Scam type and language
  6. Conversation history (last 10 messages, sanitized)
  7. Current scammer message (sanitized)
  8. 14 critical instructions for response quality

Temperature is set to 0.8 for natural variation. Max output tokens is 1000.

Step 8: Evidence Merging

File: guvi/handler.py

New evidence is merged with existing session evidence using merge_evidence_locally(), which performs a set union for each of the 14 evidence fields. This avoids a Firestore read-after-write round-trip.

After merging, high-value evidence (UPI or bank account) boosts confidence to at least 0.85:

if has_high_value:
    final_confidence = max(final_confidence, 0.85)

Step 9: Persist to Firestore

File: guvi/handler.py and firestore/sessions.py -- batch_update_session()

All session updates are written in a single set(merge=True) call:

batch_update_session(session_id, {
    "conversation_history": conversation_with_reply,
    "extracted_evidence": merged_evidence.model_dump(),
    "message_count": actual_message_count,
    "scam_type": result.scam_type,
    "confidence": final_confidence,
    "state": result.state,
    "persona": result.persona_used,
    "strategy_state": result.strategy_state,
    "messages_since_evidence": messages_since_evidence,
    "high_value_extracted": has_high_value,
    "source": guvi_request.source,
})

Evidence is also indexed in the evidence_index collection for future cross-session lookups.

Step 10: Callback Dispatch

File: guvi/handler.py and guvi/callback.py

From turn CALLBACK_MIN_TURN (1) onward, a callback is sent every turn to GUVI's updateHoneyPotFinalResult endpoint. The endpoint uses overwrite semantics, so sending every turn ensures the latest intelligence is always submitted -- even if the evaluator stops at any turn.

The scamDetected flag is computed by _compute_scam_detected():

Condition scamDetected
Classified scam type (not UNKNOWN, not NOT_SCAM) True
NOT_SCAM but UPI/bank evidence found True
UNKNOWN with confidence > 0.5 OR UPI/bank OR 2+ keywords OR phone True
Otherwise False

Step 11: Response Assembly

File: guvi/handler.py

The final GuviResponse includes:

GuviResponse(
    status="success",
    reply=result.response,                    # Persona's in-character reply
    sessionId=guvi_request.sessionId,
    scamDetected=scam_detected,               # Boolean
    scamType=result.scam_type,                # e.g. "KYC_BANKING"
    confidenceLevel=round(final_confidence, 2),
    extractedIntelligence=merged_evidence,    # All 14 evidence types
    engagementMetrics=EngagementMetrics(...),
    totalMessagesExchanged=actual_message_count,
    engagementDurationSeconds=round(duration, 1),
    agentNotes=agent_notes,                   # Concise summary
)

Error Handling

The pipeline has multiple error boundaries:

  1. Request parsing errors: Caught in guvi_honeypot(), returns 200 with stalling reply.
  2. Processing errors: Caught in process_honeypot_request(), returns 200 with stalling reply.
  3. Gemini failures: Circuit breaker (pybreaker, 5 failures / 60s window) with keyword fallback for classification and static fallback for response generation.
  4. Firestore failures: In-memory fallback for all session operations.
  5. Callback failures: Circuit breaker (3 failures / 60s), retry with exponential backoff.

Always 200

Per the evaluation spec, the endpoint always returns HTTP 200 with a valid GuviResponse, even on internal errors. A stalling reply prevents the evaluator from scoring the turn as a total failure.