Hi Nikhil.
Thanks for such a detailed email. I apologize for the delay in replying to
it.
I think you raise an important point around bandwidth consumption for the
entity hosting the data. And I appreciate the comprehensive list of
solutions. Projects like Pushshift
Hi Tarunima
I've not delved too deep into this stuff, but want to share some inputs
about one problem that might be one root cause for these changes. Apologies
in advance if this is under-informed:
Problem
There's a major cost factor in terms of bandwidth consumption for any kind
of situation
Hi All,
There have been a number of recent changes globally and nationally, in part
triggered by Gen AI chat bots, that could restrict scraping. Listing the
salient ones here:
- India coming up with the Digital India Act which would bear on data
access
- EU releasing the Data Services
