How to Block User Agents on Shopify (Bot User-Agent Guide)

The User-Agent HTTP header is the simplest fingerprint a bot leaves behind. Most automated tools — knowingly or by default — identify themselves in this string. Blocking by user agent is a cheap, fast way to filter out 60-80 % of unsophisticated scrapers.

This guide explains how user-agent blocking works, which patterns to block, and which to never block.

What is a user agent?

Every browser and HTTP client sends a User-Agent header on each request:

Mozilla/5.0 (Macintosh; Intel Mac OS X 14_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.5 Safari/605.1.15

The string describes the browser, OS, and version. Real browsers send detailed strings. Bots often send simplified or default UAs that betray them.

How to block user agents on Shopify

There is no native UA blocking in Shopify. With Shieldy:

1. Install Shieldy.
2. Open Block rules → New rule.
3. Rule type: Block.
4. Criteria: User agent → Contains / Starts with / Equals.
5. Enter the pattern.
6. Save.

UA-based rules are in the Premium plan ($4.99/mo) — the free plan focuses on IP/country.

Bot user agents to block (default list)

Shieldy ships with these enabled by default in the Bot Killer module:

python-requests/2.x
Scrapy/2.x
node-fetch
Apache-HttpClient
PostmanRuntime
Wget
curl/7.x
Python-urllib
Java/1.x
Go-http-client
ruby
HeadlessChrome
HeadlessFirefox
PhantomJS
Selenium
Puppeteer

These cover most low-effort scraping tools.

Spy-extension user agents

Many spy browser extensions modify the user agent or send a secondary header. Patterns Shieldy detects:

AliHunter-Extension
PPSpy
Minea
Koala-Inspector
Shophunter

Plus fingerprinting via JavaScript-injected behaviour (not visible in UA alone).

Patterns to NEVER block

Always allow these — blocking will hurt SEO or social integrations:

Googlebot
Bingbot
Applebot
DuckDuckBot
YandexBot
Baiduspider
Slackbot
LinkedInBot
Twitterbot
facebookexternalhit
WhatsApp
TelegramBot
Pinterest

Shieldy auto-whitelists verified versions of these by IP-confirmation (not just UA — since UA can be spoofed by bots pretending to be Googlebot).

Custom patterns for your niche

If you have niche-specific scrapers, add custom UA rules. Examples:

For a clothing store:

fashionhunter
trend-scraper
boutique-finder

For a food / supplements store:

nutritionspy
keto-scanner
supplement-tracker

For a digital products store:

content-curator
download-bot

Watch your Visitor Analytics for patterns specific to your industry.

Wildcard and regex matching

Shieldy supports three matchers:

• Equals — exact match (curl/7.88.1)
• Starts with — prefix (python-requests)
• Contains — substring (Scrapy)

For more complex patterns, use multiple rules — Shieldy combines them with OR logic.

What about spoofed user agents?

Sophisticated scrapers spoof real browser UAs. UA-only blocking does not catch them. For those:

• Combine UA blocking with IP/proxy detection (Bot Killer → Auto-block VPN/Proxy)
• Combine with behavioural fingerprinting (mouse patterns, request timing)
• Combine with datacenter IP blocking (most spoofed-UA bots run on cloud)

A scraper that spoofs Chrome UA, runs on AWS, and hits 50 pages in 10s gets blocked by all three layers.

How to find UA patterns in your logs

Open Shieldy's Visitor Analytics:

1. Filter by Risk score > 0.5.
2. Sort by Time on site ascending (bots usually have <3s).
3. Look at the User Agent column.

Common findings:

• Many requests from Mozilla/5.0 with no version detail → bot with minimal spoofing
• Mozilla/5.0 (Windows NT 10.0; Win64; x64) without browser detail → headless Chrome
• Repeated identical UAs from different IPs → coordinated scraper

Performance impact

UA matching is the fastest filter type. It runs in <1ms per request at Shopify's edge. No measurable impact on page speed.

Real-world example

A merchant we worked with had ~30 % of their traffic flagged as bots. Top patterns:

After enabling all five blocks: real human traffic dropped only 0.3 % (false positives), total request volume dropped 28 % (saved server load).

Frequently asked questions

Will blocking user agents break my checkout?

No, as long as you do not block Shopify's own webhooks or your installed apps. Shieldy whitelists all installed app callbacks automatically.

Can a bot bypass UA blocking?

Yes, by spoofing a real browser UA. That is why UA blocking is one layer among many — combine with IP, proxy, and behavioural rules.

Will UA blocking affect SEO?

Only if you block legitimate crawlers. Always verify Googlebot via IP (Shieldy does this automatically).

Are there too many UAs to maintain?

The default list covers the common ones. For niche-specific scrapers, monitor Visitor Analytics monthly and add patterns as needed.

What about mobile app user agents?

Mobile apps use OS-specific UAs (Shopify mobile, e-commerce apps). These are allowed by default.

Wrapping up

User-agent blocking is the cheapest, fastest scraper filter you can deploy. It will not stop sophisticated attackers — but it eliminates 60-80 % of low-effort scrapers in one config. Worth turning on day one.

Install Shieldy free → · See pricing →