Fraud Analysis: Machine Learning vs The Cartel
How to detect stolen credit cards, proxy IPs, and organized retail crime rings using Shopify Shield and custom ML models.
A $5,000 order comes in. Shipping Address: Miami (Freight Forwarder). Billing Address: Nigeria. IP Address: Kansas (VPN). Is it fraud? Probably. But what if it’s a legitimate Nigerian diplomat listing a Miami reshipper? If you cancel it, you lose $5k. If you ship it, you get a Chargeback and pay $5k + $15 fee. Fraud Analysis is the art of probability.
The Signals
Shopify provides fraud_analysis on every order.
avs_code: Address Verification System (Does Zip match Bank?).cvv_code: Card Verification Value.ip_distance: Distance between IP and Billing Address.
Automated Workflows (Shopify Flow)
We don’t manually review every order. We set up Shopify Flow rules.
- Red Flag: If Risk Level = High -> Auto Cancel.
- Yellow Flag: If Risk Level = Medium AND Value > $500 -> Hold for Review. Send Slack alert.
- Green Flag: Auto Capture Payment.
3D Secure (3DS)
The layout. If you enable 3D Secure (Verified by Visa), the liability shifts to the Bank. Even if the card is stolen, you keep the money. The Bank eats the loss. However, 3DS lowers conversion by ~5%. Strategy: Only trigger 3DS for high-risk orders. (See Mobile Payments for Apple Pay which is effectively 3DS).
Device Fingerprinting
Professional fraudsters clear cookies. We use Device Fingerprinting (e.g., Sardine, Sift). It looks at:
- Battery Level.
- Screen Resolution.
- Installed Fonts. If we see 50 orders from different names but the exact same device hash, it is a bot ring. (See Bot Mitigation).
Friendly Fraud
“I didn’t order this.” (Lying customer). This is hard to beat. Defense:
- Signature on Delivery (FedEx).
- Photo Proof of Delivery.
- Dispute Evidence Generator: Automate the PDF export of the Access Logs showing the user’s IP and email login at time of purchase.
6. Velocity Checks (The Speed Trap)
Humans buy slowly. Bots buy fast. A user cannot place 10 orders in 1 minute. Velocity Checks block based on time.
- “Max 1 order per 5 minutes per IP”.
- “Max 3 different credit cards per hour per Session”. This prevents “Card Testing” attacks, where a thief buys $1 items to check if stolen cards work. (See Rate Limiting).
7. Refund Fraud (The Silent Killer)
“I returned the box.” (It was empty). “I never got the refund.” (They got it twice). This is harder to detect because it happens post-purchase. Defense:
- Weight Check: Verify return package weight matches outbound weight.
- Refund Adjudication: Do not auto-refund high-value items. Require human inspection.
- Blacklisting: If a user refunds > 50% of orders, fire the customer.
9. The Human Review Dashboard
You cannot automate everything. Sometimes, the model says “50% Fraud”. You need a human to look at it. We build custom Retool Dashboards for Risk Teams. They see:
- Map of IP vs Billing Address.
- Social Media Lookup (automated via Clearbit).
- Past Order History globally. One click: “Approve” (Captures Stripe) or “Reject” (Voids Auth). This empowers your CS team to make data-driven decisions.
10. Machine Learning: Chargeback vs Sale
Standard rules behave like: “If > $500, then block.” ML behaves like: “If > $500 AND Email contains numbers AND IP is VPN, then probability is 92%.” We train custom models on your historical data. We feed “Chargebacks” as positive labels and “Successful Orders” as negative labels. The model learns patterns you didn’t know existed. “Orders for Size 11 shoes from Brooklyn at 2 AM are 80% fraud.”
11. The Chargeback Triangle
When a chargeback happens, three parties fight.
- The Bank: Wants to protect the cardholder (their customer).
- The Merchant (You): Wants to keep the money.
- The Card Scheme (Visa): Determines the rules. To win, you must provide “Compelling Evidence” (Visa Compelling Evidence 3.0). This involves linking previous transaction history. “Yes, user X claimed fraud, but here are 10 previous orders from the same IP and Device ID.” Automating this evidence submission via Stripe API increases win rates from 20% to 60%.
12. Refund Adjudication 2.0
Not all refunds are equal. A VIP returning a size M to buy a size L is “Good Churn”. A new user returning a PS5 is “Bad Churn”. We use Risk-Based Refund Logic.
- VIP: “Instant Refund” (Money in bank in 5 mins).
- New User: “Refund upon Inspection” (Money in bank in 7 days). This friction discourages “Wardrobing” (buying for an event and returning).
13. Synthetic Identity Fraud
This is the new frontier. Fraudsters combine real data (SSN from a child) with fake data (Address, Phone). It creates a “Frankenstein Identity”. Traditional checks pass because the SSN is valid. Defense: Behavioral Biometrics. How does the user type? Do they copy-paste the phone number? (Real people type it). Do they hesitate on the “Last Name” field? (Real people don’t). We integrate tools like Forter or Sardine that analyze the motion data of the mouse/touchscreen to detect these synthetic profiles.
Why Maison Code Discusses This
At Maison Code, we have defended multi-million dollar drops. We know the panic of a “Card Testing” attack at 3 AM. We don’t rely on default Shopify settings (which are too loose). We build Custom Risk Engines. We layer Cloudflare (Network), Shopify Flow (Rules), and 3rd Party ML (Sardine) to create an “Iron Dome” for your payments. We ensure that you keep your revenue, and your chargeback rate stays below 0.5%.
14. The Anatomy of a Card Tester
“Card Testing” is when a bot tries 10,000 credit cards to see which ones work. They don’t buy your $500 jacket. They try to buy a $1 sticker. If it works, they sell the card on the dark web as “Live”. Impact on You:
- Stripe Fees: You pay $0.30 per failed auth. 10,000 attempts = $3,000 fee.
- Reputation: Visa flags your merchant account as “High Risk”.
Defense: Captcha on Checkout.
If IP places > 3 failed orders, require hCaptcha.
This kills the bot’s efficiency (
ROI < 0).
15. The Manual Review Protocol
Sometimes, AI isn’t enough. You need a human protocol. The checklist for a suspicious order:
- Call the Customer: “Hi, just verifying the size.” (If number is dead -> Fraud).
- Email Domain:
john.doe@gmail.com(Normal) vsaf3289@tempmail.com(Fraud). - Social Proof: Does “John Doe” exist on LinkedIn in “Miami”?
- Street View: Is the shipping address a house or a warehouse in an industrial park? A 2-minute investigation saves $2,000.
16. The Anti-Fraud Checklist (Pre-Flight)
Before you scale to $10M GMV, ensure you have these 20 checks:
- CVV Match Enforced: Reject if CVV doesn’t match.
- AVS Match Enforced: Reject if Zip doesn’t match (Soft relax for international).
- Velocity Check 1: Max 3 cards per session.
- Velocity Check 2: Max 5 orders per IP per day.
- Velocity Check 3: Max $2000 per guest checkout.
- Email Age: Domain must be > 30 days old (via API).
- IP Proxy: Block Tor/VPN IPs on checkout.
- Distance: Alert if IP is > 500 miles from Billing.
- High Value: Manual review for orders > $1000.
- SKU Velocity: Alert if 50 units of high-resale item sold in 10 mins.
- 3D Secure: Enable on High Risk.
- Phone Validation: Send SMS OTP for suspicious orders.
- Address Normalization: Use Google Places API to verify address exists.
- Blacklist: Shared database of known fraudsters.
- Refund Cap: Auto-block user after 30% return rate.
- Chargeback Alert: Use Ethoca/Verifi to get alerts before the chargeback hits.
- Employee Training: CS team knows what “Card Testing” looks like.
- Webhook Monitoring: Monitor
order.createdfor anomalies. - Ghosting: Silent CAPTCHA for bot-like behavior.
- Legal: Clear “Terms of Service” regarding fraud investigation.
17. Conclusion
Fraud is a cost of doing business. If your fraud rate is 0%, your fraud filters are too tight (You are rejecting good customers). Aim for 0.5%. But never let the “Cartel” win. Automate the defense. Sleep soundly.
Too many chargebacks?
We implement automated risk decision engines.