I'm scraping a website that has a sequential ID, but everytime that I change the ID in the URL I'm redirected to the canonical url
Scrapeninja is not following the 301 code, is there anyway a set an argument to follow these redirects?
Thank you
아래에 의견을 추가하고 토론에 참여하세요.
Hi! I am glad to announce that new setting for
/scrape
endpoint was implemented. PassfollowRedirects: 1
to the payload to make ScrapeNinja follow redirects. The final destination url will be available in .info.finalUrl JSON property.Hi!
For the /scrape endpoint of Scrapeninja, there is no way to follow redirects now, though I am probably going to implement this in future.
For /scrape-js, all the redirects are followed automatically.
Thanks!