How Shopify's Built-In Fraud Analysis Works (And Where It Stops)

Most merchants either over-trust or completely ignore Shopify's built-in fraud analysis. Both are wrong. The native system is a genuinely useful first-pass filter — especially for stores under a few thousand orders per month. It also has specific, predictable blind spots that get more expensive as you scale.

This guide walks through what Shopify actually does under the hood when it labels an order "High risk," the signals it uses, and the three structural reasons it eventually plateaus.

What Shopify scores on

Every order placed through Shopify Payments — or through processors that pass risk signals back — gets scored on three categories: payment risk, identity risk, behavioral risk.

Payment risk

The card and the transaction itself.

• AVS (Address Verification System) match — does the billing address match what the cardholder's bank has?
• CVV match — does the security code check out?
• Card-decline history — multiple recent declines on this card?
• IP-to-billing-country consistency — does the IP geolocate to roughly where the card is registered?

Cleanest signals because they come straight from the payment network. AVS or CVV failing is significantly informative; both passing is weakly informative (a sophisticated fraudster will pass these).

Identity risk

The person placing the order.

• Email reputation — appears in known data breaches? Used recently across other Shopify stores?
• Email-name consistency — does [email protected] match the name "John Doe" on the order?
• Phone-country prefix consistency — does the phone match the shipping country?
• Prior order history on your store

Behavioral risk

How the order was placed.

• Logged-in customer vs. guest checkout
• Checkout completion speed (too fast = bot; too slow = nothing)
• IP type — residential, proxy, VPN, TOR exit, datacenter
• Velocity — multiple orders just came through with similar characteristics?

For each indicator, Shopify shows a green check, yellow caution, or red warning on the order detail page. A single red signal doesn't mean fraud — it means one input didn't match expectations.

The Low / Medium / High thresholds

Shopify aggregates the indicators into three buckets:

A "High risk" label means Shopify's model estimates meaningfully elevated probability of fraud — not certainty. The actual fraud rate of orders Shopify labels High varies wildly by store: we've seen stores where 35% of High-risk orders are confirmed fraud, and stores where less than 8% are.

Same score, same system, totally different outcomes — because Shopify's model is calibrated against its entire merchant base, not against your specific traffic mix.

The three signals Shopify uses really well

Where Shopify's native filter genuinely shines:

1. Cross-merchant card-network data

Shopify operates a fraud-signal network across the platform. If a card fired a confirmed-fraud chargeback on one Shopify store, that signal is available when the same card hits another. One of the most underrated assets in the platform — no single merchant could build this alone.

2. AVS / CVV payment-network signals

These come from the issuing bank, not Shopify. They're high-fidelity and free. Stores not using Shopify Payments lose some of this fidelity depending on the connector.

3. Basic anonymization detection

Shopify maintains lists of known VPNs, proxies, TOR exits, and obvious datacenter IPs. Flags orders from them. Depth of the list varies — sophisticated VPN services often slip through — but the obvious cases get caught.

For a store doing under 1,000 orders/month, these three signal categories cover most of the obvious fraud. The fraud Shopify catches at this level is the fraud that didn't take much effort to commit.

Where Shopify's filter plateaus

Three structural limits start to bite as you scale.

Limit 1: It runs post-checkout, not pre-checkout

Shopify's analysis fires after the order is created. By then:

• Inventory is committed
• The customer received a confirmation email
• The order appears in your dashboard
• Your fulfillment pipeline may have started

You can cancel — but you're cancelling something the customer thinks happened. That's a friction event with three downstream costs: support tickets, refund operational time, and brand-experience damage.

Pre-checkout filtering — stopping the bad actor before they reach the checkout form — is structurally different. It happens at the page-request level, ideally at the edge (CDN/WAF), where the cost of blocking is just a 403 response. For categories like card testing and bot scalping, pre-checkout is the only effective layer — by the time you're scoring an order, the attack already worked.

Limit 2: It doesn't correlate across orders

Shopify scores each order in isolation. It doesn't:

• Connect five orders from different emails placed on the same device
• Notice that the same shipping address has been receiving COD orders that never get accepted
• Recognise an affiliate's "customers" all share device characteristics
• Spot a velocity spike on a single promo code from coordinated accounts

These are cross-order patterns — where modern fraud rings operate. A single order in a ring usually looks fine; the entire point is to make each pass individual scrutiny. The pattern only becomes visible when you correlate device fingerprints, IPs, and behavioral signatures across many orders.

Shopify's filter wasn't designed for this and doesn't expose the data you'd need to build it yourself. This is the most common reason merchants graduate to a dedicated fraud app once they cross a few thousand orders/month.

Limit 3: It never acts

This is the biggest one. Shopify flags. It never blocks, cancels, or hides anything.

Every High-risk order sits in your queue waiting for you to decide. At 50 orders/week that's fine. At 500 orders/week it's a bottleneck. At 5,000/week the review queue becomes a full-time job for someone, and most stores either:

• Auto-fulfil High-risk anyway (and take the chargebacks)
• Auto-cancel everything High-risk (and lose 60-90% of orders that would have been legitimate)

Neither is right. The right answer is automated action calibrated to your specific store's false-positive rate — which requires a tool that scores AND acts.

How to use Shopify's filter well (the right way)

Don't auto-action on the score. Use it as a queue prioritizer:

1. Low risk → auto-fulfil (Shopify's default)
2. Medium risk → tag with fraud-review, hold fulfillment, alert your team
3. High risk → either auto-cancel (if your false-positive rate < 15%) or escalate to senior reviewer

Read the indicators, not the score. A High-risk order with clean AVS, clean CVV, but mismatched IP-country is usually a traveler. A Medium-risk order with AVS mismatch + no order history + freight-forwarder ZIP is more concerning than the score suggests.

Track your false-positive rate for at least 60 days. Log every cancelled High-risk order and check whether the customer disputed, contacted you, or just walked away. If your false-positive rate is above 50%, you're losing more to over-cancellation than to fraud — and the answer is layered controls, not stricter blocking.

When to layer beyond Shopify's filter

The clearest signal you've outgrown native fraud analysis alone:

• You spend more than 30 minutes/day reviewing flagged orders
• You're seeing cross-order patterns (repeat shipping addresses, device clusters, promo-code velocity spikes)
• Your chargeback ratio is creeping toward 0.6%
• You can see fraud trends but can't isolate root causes

At that point, the conversation shifts from "should we install a fraud app" to "which capabilities give the best ROI for our specific loss pattern."

How Shieldy extends Shopify's filter

Shieldy Fraud Filter doesn't replace Shopify's native analysis — it layers on top of it. Specifically:

• Pre-checkout filtering that Shopify doesn't do: country, state, city, ISP, IP range, VPN, proxy, TOR, headless browser, spy-extension detection
• Cross-order correlation Shopify doesn't surface: device fingerprinting, IP-history tracking, repeat shipping-address detection
• Automated action: auto-cancel high-risk orders (configurable threshold), auto-tag for review, auto-block IP after chargeback
• Checkout-level blocking by email, phone, name, address, ZIP
• Payment-method hiding via Shopify Functions (hide COD for high-risk profiles, etc.)
• Shopify Flow triggers for custom downstream workflows

The combination — Shopify's native filter + Shieldy's layers — typically cuts fraud cost 50-80% within 30 days for stores moving past the "Shopify alone is enough" stage.

The takeaway

Shopify's native fraud analysis is genuinely useful and free. Keep it on. Use the indicators as a queue prioritizer, not as a verdict. Track your false-positive rate honestly.

When the queue starts becoming a bottleneck — or the cross-order patterns become visible — layer on a dedicated fraud app. Don't replace Shopify's filter; complement it.

The merchants who handle fraud well don't fight Shopify's filter. They use it for what it does, and add layers for what it doesn't.