About CreaseDealsBot
You have most likely arrived here from our crawler's User-Agent string in your access logs. This page is what that string points to: what the crawler does, what we keep, and how to make it stop.
CreaseDealsBot/0.1 (+https://creasedeals.com/bot; price tracking for lacrosse gear)
Who we are
CreaseDeals collects publicly listed lacrosse gear prices, records how they change over time, and shows shoppers which discounts are real. We are not a retailer, we sell nothing, and every listing links back to the store it came from. There is more about why the site exists on our about page.
What the crawler does
-
Requests only your public, unauthenticated
/products.jsonendpoint — the same data any visitor's browser can request. We do not log in, and we do not attempt to reach anything that requires authentication. -
Reads your
robots.txtbefore a retailer is ever added, and re-checks it periodically. If it disallows the path, we do not fetch it. - Runs once per day, one request at a time, with a one-second pause between requests. There is no concurrency. A full pass over a large catalog is a few dozen requests.
- Identifies itself honestly on every request. We do not disguise the crawler as a browser, and we will not do so to get around a block.
- Stops on its own if something is wrong. Three failed runs in a row and the retailer is disabled automatically and no longer fetched, until a human looks at why.
What we store, and what we don't
We keep the facts needed to show a price and link back to you:
- Product title, price, and compare-at price
- The URL of your product page
- The URL of your product image
- Whether the item is in stock
We deliberately do not:
-
Copy your product descriptions. Your
body_htmlis never stored. - Rehost your images. They are linked directly from your CDN, so you keep the traffic and stay in control of them.
- Reproduce your pages. Every listing links back to you; the click goes to your site.
If you would prefer we did not fetch your catalog, say so and we will stop. We would rather remove a retailer than be an unwelcome guest. No explanation required, and there is nothing to escalate — one message is enough.
Contact
Email hello@creasedeals.com. Removal requests are actioned first and discussed afterwards, if at all.