For website owners

ForkleafBot

ForkleafBot is operated by Arcem Digital Solutions. It reads publicly posted restaurant menus so the Forkleaf app can show vegan options.

User agent
ForkleafBot/1.0 (+https://forkleaf.app/crawler; developer@arcemdigitalsolutions.com)
robots.txt token
ForkleafBot
Earlier pilot
Earlier small pilot crawls in the Needham area identified as ForkleafCrawler/0.1.
Contact
developer@arcemdigitalsolutions.com. We handle requests within one business day.

How it behaves

  • It honors robots.txt. Rules for ForkleafBot take priority over rules for *.
  • If robots.txt returns a server error, it treats the site as disallowed for the day.
  • It caches robots.txt for up to 24 hours.
  • It makes at most one request every 3 seconds per site, and honors a longer Crawl-delay.
  • It backs off on 429 and 5xx responses and honors Retry-After.
  • It stops trying a site after repeated failures.
  • It does not log in, solve captchas or use proxies.
  • It does not fetch online-ordering platforms or delivery aggregators.
  • It collects only public menu text (dish names, descriptions and prices, if listed) and the restaurant’s public website and contact information. It collects no personal data.

Opt out

To stop ForkleafBot from crawling your site, add this to your robots.txt:

User-agent: ForkleafBot
Disallow: /

Or email developer@arcemdigitalsolutions.com with your domain. We will stop crawling it and remove its menus from Forkleaf in our next data update.

Restaurant corrections

If Forkleaf shows something wrong about your restaurant or menu, email developer@arcemdigitalsolutions.com with the details.