[Information] Scraping BotD Link
Scraping BotD Link Posted on: 04/22/2018 1:15am
Quote Post

Just a quick question: how does KHUT get the Badge of the Day link? I've been scraping it from the homepage, but that isn't exactly a "lite" method.

RE: Scraping BotD Link Posted on: 04/22/2018 3:52am
Quote Post
Ubi Maior Minor Cessat

I would get https://www.kongregate.com/badges and then get the botd data from the next sibling of 
<!-- BoTD Start -->

RE: Scraping BotD Link Posted on: 04/22/2018 7:20pm
Quote Post

Alright, so I ran a few tests. The badges page is about 90KiB, and the home page is about 170KiB. Quite a bit smaller. I was hoping for a direct REST query or something similar, but this will still work. Thanks for the suggestion.