Reset scraping/ripping rate limit
set_scrape_delay(delay) set_rip_delay(delay)
delay | Delay between subsequent requests. Default for package is 5 sec. It can be set lower only under the condition of specifying a custom user-agent string. |
---|
Updates rate-limit property of scrape
and rip
functions, respectively.
#> <polite session> https://www.cheese.com #> User-agent: polite R package - https://github.com/dmi3kno/polite #> robots.txt: 0 rules are defined for 1 bots #> Crawl delay: 5 sec #> The path is scrapable for this user-agent# }