If you do not want to whitelist our crawling IP-addresses one by one, you can use our USER_AGENT script instead:
javascriptUSER_AGENT = ( "Mozilla/5.0 (Windows NT 10.0; Win64; x64) " "AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" )
By setting this User-Agent, you make it appear to the website as though a regular person using an up-to-date version of Google Chrome on Windows were accessing it. In many cases, this can prevent the request from being blocked (HTTP 403 Forbidden).