How to Read Fraud-App Visitor Logs and Risk Reports on Shopify
Fraud-app dashboards surface lots of data. Knowing which fields actually matter — and how to interpret them — is the difference between insight and noise.
Modern fraud apps surface a wealth of data: visitor logs, risk scores, geographic distributions, payment-signal breakdowns, behavioral analytics. The dashboards look impressive. Most of it is unused.
The reason isn't lack of effort — it's that the data isn't always organized around the decisions merchants actually need to make.
This guide walks through what a fraud-app visitor log and risk report typically contain, which fields are actually useful, and how to read the dashboards in a way that drives decisions rather than just informs curiosity.
What's in a typical fraud-app report
Most fraud apps surface some combination of:
- Visitor logs. Per-visit details: timestamp, IP, geolocation, browser/device, behavior pattern, risk classification.
- Order risk reports. Per-order risk scores, indicator breakdown, action taken.
- Aggregate dashboards. Top countries, top IPs, top fraud patterns, top blocked entities.
- Trend reports. Time-series of fraud-related metrics — chargebacks, blocked sessions, flagged orders.
- Specific event logs. Specific rule fires — when each rule activated, on which order, with what outcome.
Volume of data is significant. Challenge is knowing which slices matter.
The fields that actually matter
For visitor and order logs, a handful of fields carry most of the decision-relevant information:
IP address and geolocation
The visitor's IP and where it geolocates. Strongest individual signal for geographic fraud patterns. Useful for:
- Identifying coordinated attacks from specific infrastructure
- Cross-referencing with billing/shipping addresses
- Adding to blocklists when patterns emerge
Geolocation accuracy varies, so treat country attribution as informative but not authoritative.
Risk classification
The headline number — usually low/medium/high or 0-100 score. The aggregate signal that triggers most automated workflows. Useful for prioritising review queues.
Important caveat: the score's meaning depends on calibration. A "high risk" classification has different actual fraud probability across different stores and over time. Trust scores for triage, not for verdicts.
Risk indicators
The specific signals that contributed to the classification — AVS match, CVV match, IP-country consistency, prior history, etc. The indicators carry more information than the aggregate score.
A "high risk" order with clean AVS, clean CVV, and only an IP-country mismatch is usually a traveler. A "medium risk" order with AVS mismatch and a fresh email is often more concerning than the score suggests.
Train your eyes on the indicators. Score is for filtering; indicators are for deciding.
Customer history
Whether this customer (by email, account ID, or other identifier) has prior orders, prior disputes, prior interactions. The single strongest contextual signal for whether a current flagged order is genuinely concerning or just noise.
A first-time customer flagged as high-risk warrants more scrutiny than a 20-order veteran flagged with similar signals. Customer-history field is usually the difference.
Device fingerprint
The visitor's device signature — browser, version, screen resolution, plugins, etc. Stays constant across IP changes, which makes it valuable for detecting:
- Repeat fraudsters using new emails
- Fraud rings using rotating identities on a small set of devices
- Bot traffic with uniform device characteristics
Not all fraud apps capture device fingerprints; those that do often surface it in the visitor log.
Behavioral pattern
Time-on-site, navigation pattern, mouse movement entropy, checkout completion speed. Behavioral signals distinguish humans from bots and well-rehearsed fraudsters from natural shoppers.
A visitor who arrived 12 seconds ago, navigated straight to a $400 product, and is now at checkout doesn't have a human shopping pattern. Behavioral signals are increasingly the most reliable indicators.
Payment indicators (for orders)
AVS result, CVV result, BIN reputation, decline history. Strongest single-source signals on payment risk. AVS and CVV passing is meaningful; AVS or CVV failing is significantly meaningful.
The fields that look impressive but rarely matter
Several fields appear prominently on dashboards without providing much decision value:
Total session count. Big number that says "we tracked X sessions." Useful for marketing reports; not useful for fraud decisions.
Average risk score across all traffic. Aggregates many signals into one number that's hard to act on. Per-order or per-segment scoring is more useful.
Geographic heatmaps. Visually appealing; rarely actionable in detail. Most stores already know roughly where fraud comes from.
Browser distribution. "Most fraud comes from Chrome." Almost universally true because most legitimate traffic also comes from Chrome. The signal is small.
Operating system distribution. Same as browser — almost always biased toward the dominant platform regardless of fraud pattern.
These aren't bad to track, but they shouldn't drive decisions.
How to actually use the visitor log
A working pattern for reviewing visitor logs and order reports:
For an individual flagged order
When you need to make a decision on a specific order:
- Read the risk indicators (not just the score)
- Check customer history
- Look at the device fingerprint — has this device been seen before?
- Compare IP geolocation, billing country, shipping country — consistent or anomalous?
- Look at the behavioral pattern — does it look human or scripted?
The combined picture is usually clearer than any single field.
For investigating a fraud pattern
When you're trying to understand what's hitting your store:
- Group blocked or flagged events by some dimension (IP, country, ISP, time window)
- Look for concentrations — patterns where many events share characteristics
- Within concentrations, look at specific identifiers (IPs, emails, addresses) involved
- Build a hypothesis about what's happening, then verify by examining a few specific cases
Pattern-detection-then-case-verification cycle is faster than reading every event in detail.
For tuning fraud rules
When you want to know whether a specific rule is working:
- Look at events the rule fired on over a 30-90 day window
- Track outcomes — which led to chargebacks, which were released and didn't dispute, which were cancelled with subsequent customer contact
- Calculate the rule's precision (true positive rate) and recall (fraction of fraud caught)
- Decide whether to tighten, loosen, or remove the rule based on data
This is the analytical layer that separates fraud operations that improve over time from ones that stagnate.
What "good" rule performance looks like
When tuning, you need a sense of what acceptable rule performance looks like:
| Metric | Benchmark |
|---|---|
| True positive rate for high-risk classifications | 15-40% typical. Below 15% suggests over-flagging; above 50% rare. |
| Hit rate for IP blocks | >70% — proxy measures: how often the IP was flagged for other reasons, how many other merchants had it blocked |
| Bot-detection precision | >95% — bot signals usually high-confidence. Below 90% suggests catching legitimate users. |
Rough benchmarks; your specific numbers depend on your traffic mix and rule configuration. Useful as targets when tuning.
Common dashboard misreadings
A few patterns appear consistently:
Treating high-volume countries as high-fraud countries. Big countries (US, UK, Germany) appear prominently in your fraud reports because they're prominent in your traffic. **Fraud *rate* is the right metric, not count.**
Assuming all VPN traffic is fraud. The dashboard's "VPN detected" segment includes legitimate privacy-conscious customers. Treating it as monolithic fraud overstates the actual fraud rate from VPN users.
Conflating "high risk score" with "fraud." As discussed throughout this series, high-risk classifications are predictions, not verdicts. A 35% true-positive rate on high-risk doesn't mean the score is broken — it means high-risk warrants review, not auto-action.
Reading short windows. Daily and weekly snapshots have a lot of noise. Most fraud patterns are visible over 30+ days. Don't make rule changes based on weekly trends.
Comparing to industry averages. Different verticals, markets, customer bases have different baselines. Your numbers are meaningful relative to your historical numbers, not generic benchmarks.
How Shieldy structures its reports
Shieldy Fraud Filter organizes the dashboard around the decision categories above:
- Visitor log — per-visit detail with IP, geo, device fingerprint, behavioral signals
- Order risk report — per-order risk + indicator breakdown + customer history sidebar
- Pattern investigation — group events by IP, ASN, country, time window; identify concentrations
- Rule performance — per-rule precision, recall, false-positive rate over 30/60/90 day windows
- Aggregate dashboard — fraud-loss trend, chargeback ratio, blocked traffic, revenue protected vs conversion loss
Designed around decision-flow rather than visual impact.
A practical close
A useful fraud dashboard is one where the most-used fields are most prominent. Most fraud-app dashboards are designed to demonstrate the product's capability — lots of charts, lots of data, lots of visual interest — rather than to optimize for the decisions merchants need to make.
The practical step: define what you need to see, configure your view accordingly. For most stores:
- Daily: queue depth, oldest unresolved flag, today's chargebacks
- Weekly: chargeback rate trend, flagged-order volume trend
- Monthly: per-rule precision, false-positive rate, revenue-protected-vs-conversion-loss
- Quarterly: aggregate fraud-loss summary, strategic decisions
The dashboard you build is much simpler than the default fraud-app dashboard, and far more useful.
Shieldy's dashboard surfaces the decision-relevant fields prominently. Custom views let you build the simplified working report for your team.
Protect your Shopify store today
Install Shieldy free — block fraud, bots, and VPNs in under 5 minutes.
Install on Shopify — Free
