README
Few Guidelines:
- The system will only charge for the HTTP 200 and 404 responses.
- Each request has 30 seconds to return results successfully. It will retry several times until it reaches 30 seconds. If a request reaches 30 seconds, the scraper box will return error 500.
- Each request has limit of 2mb max per request. In case you want to scrape a file, make sure you keep this in mind.
- If you exceed your current limit, the system will automatically charge on a per-API-call basis.