ScrapeNinja

FREEMIUM
От Anthony | Обновлено 5 days ago | Data
Популярность

9.9 / 10

Задержка

2,898ms

Уровень обслуживания

98%

Health Check

N/A

Назад ко всем обсуждениям

302 status code, empty body

Rapid account: Even Lopez
EvenLopez
2 years ago

I get a 302 status code response but with its body empty

I’m using scrapy so this is not the whole working code, I have another .py to instantiate this class and run .ninjaresponse() method. Anyway you can see the 302 status code in response.text, which doesn’t return requested html

class ScrapeNinja():
ninja_url = ‘https://scrapeninja.p.rapidapi.com/scrape

# get your subscription key at https://rapidapi.com/restyler/api/scrapeninja from "Code generator",
# copy&paste it to 'x-rapidapi-key' header below
def __init__(self,url):
    self.payload['url'] = url[0]
    self.payload['headers'][0] = f"referer: {url[0]}"

headers = {
    "Content-Type": "application/json",
    "x-rapidapi-host": "scrapeninja.p.rapidapi.com",
    "x-rapidapi-key": "xxxxx"
}

payload = {
    "url": "https://www.alcampo.es/compra-online/",
    "headers": [
        "authority: www.alcampo.es",
        "accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
        "accept-language: es-ES,es;q=0.9",
        "cache-control: no-cache",
        "cookie: JSESSIONID2=1A0A02AA8CFE4A2AF33C7712B9D5816D.hybris5; cookieLanguageHybris=es; deliveryMode=PICKUP_DELIVERY_MODE; precode=\"\"; fastDelivery=false; localizedAs=anonymous; cookie_consent_user_accepted=true; cp=28041; shopId=015; selectionMode=selectedByUser; _gcl_au=1.1.400785008.1651759856; _fbp=fb.1.1651759856852.417292830; uid=e44ecd1602fb9d6bb0835119f5de97e2; sto__vuid=e44ecd1602fb9d6bb0835119f5de97e2; www._km_lead_collection=false; www._km_id=UMx05BiguxwLGXdh6EmW4QNsZiZLa028; www._km_user_name=Poised Otter; COOKIE_SUPPORT=true; GUEST_LANGUAGE_ID=es_ES; YOMe1hXPsDqscWomlSmem06yAMon28t8Vfv7k7W8mw__=v1LIaGSQ__RYl; __atuvc=1%7C18; __atssc=google%3B1; alcampoSpainStoreUid-cart=f0fbfbe2-7274-4018-9b45-89f27d4591c0; __utmzz=utmcsr=google|utmcmd=cpc|utmccn=(not set)|utmgclid=CjwKCAjwjtOTBhAvEiwASG4bCFVVnqN6G0brhb5LvH82wujqZ3cHxJMBUrfu31s--nRsJj9RACG1vxoCJxQQAvD_BwE; __utmzzses=1; __utmzze=utmcsr%3Dgoogle%7Cutmcmd%3Dcpc%7Cutmccn%3D(not%20set)%7Cutmgclid%3DCjwKCAjwjtOTBhAvEiwASG4bCFVVnqN6G0brhb5LvH82wujqZ3cHxJMBUrfu31s--nRsJj9RACG1vxoCJxQQAvD_BwE; client_ID=undefined; user_ID=undefined; chatbot_id=1A0A02AA8CFE4A2AF33C7712B9D5816D; cookie_consent_level=%7B%22strictly-necessary%22%3Atrue%2C%22functionality%22%3Atrue%2C%22tracking%22%3Atrue%2C%22targeting%22%3Atrue%7D; __cf_bm=TkT1xwvbRj.DLTHqUSctteXdVjEe6vCS1jLs6ymHFcY-1652254121-0-AaikEUiilcLEdjkm4VAWs0hezJdkTpU0zyU6H1NW7l95h+DCvfnPNjVYtP+FtZ6xpvUf00VvNWJre4eOXjfF9Xx/l6NGZccjAHan8VqfG1ypEOA5uGpG3/jtQVrepRKqN9qcBw+m9tcB50gbyG2JBqEKFGgjQOORW/AwQivXsGmd; sto__session=1652254123302; sto__count=0; cto_bundle=5b-hSl9YSmlmZmI1SXNKYklnUHh1b0htZzJWQ1c3OXFCTWFDRDByS2dyWkY3QVdQcXI5R0xqNDFCWHdnSm15OFZTcGN0Ym5QbzJFbEtPc0ZxdDFzUjNKWE5Gb1F2R2EzZWdWUmJiRnlHUUtWNzRXNCUyQkx1amdSc1RBY1FrdnNQWDM2QjRZZG53ZWlpN3FTcUpoemZJWHlpNUoydyUzRCUzRA; sid=c433ef73e2646bbfb57caa972662058f; _clck=po0ql7|1|f1d|0; _clsk=1ba9xg0|1652254124095|1|1|h.clarity.ms/collect",
        "pragma: no-cache",
        "sec-ch-ua: \" Not A;Brand\";v=\"99\", \"Chromium\";v=\"101\", \"Google Chrome\";v=\"101\"",
        "sec-ch-ua-mobile: ?0",
        "sec-ch-ua-platform: \"Windows\"",
        "sec-fetch-dest: document",
        "sec-fetch-mode: navigate",
        "sec-fetch-site: none",
        "sec-fetch-user: ?1",
        "upgrade-insecure-requests: 1",
        "user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36"
    ]
}

def ninjaresponse(self):

    response = requests.request("POST", self.ninja_url, json=self.payload, headers=self.headers)
    test = 1
    return response
Rapid account: Even Lopez
EvenLopez Commented 2 years ago

I just tried running this code alone, apart from classes, and only then I get the right html. Only under classes I get this type of response.content (while response.status_code is 200):

b’{“info”:{“version”:“2”,“statusCode”:302,“statusMessage”:"",“headers”:{“date”:“Wed, 11 May 2022 13:57:00 GMT”,“content-length”:“0”,“location”:"/compra-online/",“cf-ray”:“709b6bd6481c1809-EWR”,“strict-transport-security”:“max-age=2592000”,“cf-cache-status”:“BYPASS”,“cf-cache-debug”:"{\“requestCacheable\”:false,\“responseCacheable\”:false,\“deviceType\”:null,\“resourceConfig\”:{\“regex\”:{},\“type\”:\“OTHER\”,\“ttl\”:3600},\“isBot\”:false,\“cacheKey\”:\“https://www.alcampo.es/compra-online–015false\",\“segment\”:\“015false\”}",“cf-cache-debug-cookies-diff”:"{}",“cf-cache-debug-request”:"{\“allowHitOnFirstAccess\”:true,\“dpc\”:false,\“session\”:true,\“bypass\”:true,\“segment\”:true,\“origin\”:true,\“method\”:true}",“cf-cache-debug-response”:"{\“edge\”:true,\“status\”:false,\“cookie\”:true}",“content-security-policy”:"frame-ancestors http://.alcampo.es https://.alcampo.es *.publitas.com”,“expect-ct”:“max-age=604800, report-uri=\“https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\””,“x-content-type-options”:“nosniff”,“x-frame-options”:“SAMEORIGIN”,“set-cookie”:["__cf_bm=dzI5yT.NIl3TDs4_HrWagcgtCp7aMNnr2tlpyRQb8zU-1652277420-0-AelXbmF37Q5Rn+x1FvMb5HdIMDDD3g0CxHlXMtddlnS1ZLSKfCNcp+4f42XMou29LZfZaNzq3ra4n+e5B8D6KAE=; path=/; expires=Wed, 11-May-22 14:27:00 GMT; domain=.alcampo.es; HttpOnly; Secure; SameSite=None"],“vary”:“Accept-Encoding”,“server”:“cloudflare”,“alt-svc”:“h3=\”:443\"; ma=86400, h3-29=\":443\"; ma=86400"}},“body”:""}’

Don’t know what this exactly means but it is strongly bonded to classes

Rapid account: Restyler
restyler Commented 2 years ago

I am getting 200 page with html of the website, I have tested it several times, seems to work fine…

Присоединяйтесь к обсуждению – добавьте комментарий ниже:

Войдите / Зарегистрируйтесь, чтобы публиковать новые комментарии