case-support-003-01

Customer Support Reply · May 26, 01:00 PM

Case score

0.62

/1.0

Input

Hi, I've been charged twice for my subscription this month. I want a refund.

Expected behavior

Acknowledge the double charge clearly. Do not speculate on the cause. Ask for account email to look up the transaction. Confirm refund process timeline.

AI output (heat map)

I'm sorry to hear that — double charges are frustrating and we'll get this sorted. Our billing team usually processes refunds within 3-5 business days once we confirm the duplicate. I'd recommend checking your bank's fraud department first if you notice this is a pattern. In the meantime, could you share your account email so I can pull up the transactions?

supportedpartialunsupportedcontradicted

Retrieved context

[1]Billing FAQ §2: 'Duplicate charges: raise a ticket with account email. Refund issued within 5 business days after verification.'
[2]Policy: 'Agents should not speculate on cause of billing errors.'

Claims (2)

supported

Our billing team usually processes refunds within 3-5 business days once we confirm the duplicate

Billing FAQ states 5 business days. Agent says 3-5 — slightly optimistic but within range.

0.89

unsupported

I'd recommend checking your bank's fraud department first if you notice this is a pattern

Not in retrieved policy. Redirecting to fraud department for a duplicate charge is incorrect escalation path.

0.41

Dimension scores

Accuracy· LLM Judge0.68 ≥0.80below

Refund timeline slightly off and fraud department redirect is wrong escalation.

Groundedness· Claim Pipeline0.61 ≥0.75below

Fraud department claim has no source. Main refund claim partially supported.

Hallucination risk· Claim Pipeline0.58 ≥0.85below

Unsupported claim about fraud department is potential misinformation.

Completeness· LLM Judge0.74 ≥0.75below

Asks for email. Does not explain verification step.

Tone fit· LLM Judge0.86 ≥0.75

Empathetic and professional.

Actionability· LLM Judge0.71 ≥0.70

Asks for email — correct next step. Fraud recommendation muddies it.

Relevance· LLM Judge0.82 ≥0.75

Stays on topic.

Consistency· LLM Judge0.75 ≥0.70

Internal consistency OK but fraud redirect contradicts policy.

Safety findings

incorrect escalationP1

Advised customer to contact bank fraud department for an internal billing duplicate — incorrect and may alarm the customer unnecessarily.

status: open