Facebook crawler receiving 403 from Zeabur edge via Meta IP range

Resolved

?領先時代官方帳號FREE5/17/2026

Problem Summary

Requests from the Facebook crawler's real IP range (Meta AS32934) to a Next.js site deployed on Zeabur are receiving a 403 error from the Zeabur edge. Requests from general user IPs worldwide (TW / JP / US / BR / SG) using the same facebookexternalhit User-Agent to the same URL consistently return 200. This prevents link previews from generating for any articles posted to Facebook, Messenger, or Threads.

Impact

Facebook Sharing Debugger returns 403 for all Zeabur-hosted URLs with the message: "This response code could be due to a robots.txt block."
Posts fail to generate og:title / og:description / og:image previews, resulting in zero social sharing traffic.
The issue occurs on both the default *.zeabur.app domain and custom domains (even when proxied through Cloudflare), confirming the block occurs at the Zeabur edge proxy, after Cloudflare but before Next.js.

Affected Service

Project ID: 6a0973851985c8ed80e2a779 (leading-ai-lab)
Service ID: 6a09738e1985c8ed80e2a780
Default domain: https://leading-ai-lab.zeabur.app/
Custom domain: https://ai-lab.leadingmrk.com/
Region: Linode-Tokyo (server-6960eea5c73b56288f5791ef)
Framework: Next.js (Prebuilt V2)

Reproduction Steps

Open https://developers.facebook.com/tools/debug/
Enter https://leading-ai-lab.zeabur.app/ or https://ai-lab.leadingmrk.com/posts/paperclip-24h
Click "Scrape Again"
The response code shows 403, og:title cannot be fetched, and Facebook displays the misleading message "possibly due to robots.txt block."

Evidence (Why it's the Zeabur edge, not the application layer)

Direct curl tests from multiple locations using the facebookexternalhit/1.1 UA all return 200:

curl -A "facebookexternalhit/1.1" https://leading-ai-lab.zeabur.app/
→ HTTP/2 200, x-nextjs-cache: HIT, x-powered-by: Next.js

robots.txt explicitly Allows facebookexternalhit / Facebot / meta-externalagent / meta-externalfetcher, and the application has correct og:* meta tags.
Direct access to og:image returns 200 (content-type: image/png, including cache-control: public, max-age=31536000).
Cloudflare logs (on the custom domain layer) show no challenge/block events for the FB UA.
Only requests originating from Meta's real IP range (AS32934) (i.e., FB Sharing Debugger) receive a 403.

→ Zeabur edge is directly rejecting the Meta ASN after Cloudflare and before Next.js.

Excluded Possibilities

✗ robots.txt block — Content verified as Allow.
✗ Next.js application block — x-nextjs-cache: HIT proves the application is still returning 200.
✗ Cloudflare (custom domain layer) block — No records in Cloudflare WAF event log, and the default *.zeabur.app (without client Cloudflare) also returns 403.
✗ Browser Integrity Check / Bot Fight Mode (client Cloudflare) — Both disabled.
✗ Rate limit on user end — Facebook Debugger returns 403 even on a single manual scrape.

Root Cause Hypothesis

Zeabur edge proxy (located after Cloudflare and before the Next.js container) is applying some form of IP reputation / DDoS protection / rate-limit rule to Meta AS32934 / specific Facebook crawler IP ranges, causing a 403 before the request reaches the application layer.

Current Workaround

Added a Cache Rule on the client-side Cloudflare (cache_level=cache_everything, edge_cache_ttl=7200) to force-cache HTML responses for /posts/*, allowing the Facebook crawler to retrieve a 200 response directly from the Cloudflare edge cache without hitting Zeabur.

Problem: The default *.zeabur.app domain cannot be controlled via client Cloudflare and remains broken; this is a temporary bypass, not a permanent fix.

Requested Actions

We request the Zeabur engineering team to:

Confirm if there are platform-level deny rules on the Zeabur edge for Meta AS32934 / Facebook crawler IP ranges.
If so, please whitelist verified social crawler UAs such as facebookexternalhit / Facebot / meta-externalagent / meta-externalfetcher, or provide a "Trusted Bots" toggle in the dashboard for customer management.
Additionally, we suggest that when the FB crawler is blocked, the system should return 503 + Retry-After instead of a vague 403, so FB knows to retry.

Thank you.

5 Replies

BCBohan ChenEMPLOYEE5/19/2026

Hello, thank you for the very detailed troubleshooting report.

We have checked, and the DNS for *.zeabur.app is resolved directly to your Linode server IP via DNSPod without passing through the Cloudflare proxy, so the issue is not at the Cloudflare edge layer. Your Linode server does not have any Linode firewall mounted either, so it is not a network layer block.

The most likely cause at the moment is that the ingress-controller on your server is returning a 403 for requests from the Meta AS32934 IP range. To confirm the specific reason, we need to check the ingress configuration on the server. Could you please run the following commands in the Dashboard Terminal or via SSH and paste the results back here?

sudo kubectl get configmap -n default ingress-controller -o yaml
sudo kubectl logs -n default -l app=ingress-controller --tail=50 | grep -i "403\|block\|deny\|forbidden"

You can continue using the Cache Rule workaround you currently have set up for your custom domain on Cloudflare.

?領先時代官方帳號FREE5/19/2026

The results are as shown in the image.

BCBohan ChenEMPLOYEE5/19/2026

We SSH'd directly into your server for a deeper investigation (searching through 15,000 lines of ingress logs):

There are no 403s in the entire log — the ingress controller is not blocking any requests.
Requests with the facebookexternalhit UA are all 200 — crawls from 147.92.179.x (LINE crawler) to ai-lab.leadingmrk.com were all successful.
However, there are no request records from the Meta AS32934 IP range — Meta's crawler requests are not reaching your server at all.

This means the 403 is not being blocked by the ingress controller on your server. Meta's requests are being intercepted before they reach your server, likely due to a DNS layer issue (DNSPod returning different results to Meta's resolver) or a network layer issue.

Could you please go to the Facebook Sharing Debugger, scrape https://leading-ai-lab.zeabur.app/ again, and post a screenshot of the results? We want to confirm if this issue is still occurring.

?領先時代官方帳號FREE5/19/2026

Facebook scraping is working successfully now! Both the default Zeabur domain and custom domains work.

Thanks, the issue is resolved.

CCanEMPLOYEE5/21/2026

That's great! It would be perfect if both domains work. 🎉 Thank you for the very comprehensive troubleshooting report you provided at the beginning, which helped us quickly pinpoint the issue—and thank you for your hard work in testing and confirming it.

If you encounter any further issues with the OG preview, feel free to reach out to us again. Best of luck!

This issue has been marked as resolved

New replies are disabled for resolved issues.

Forum