Yes definitely - have a look for articles about web scraping. I do that kind of stuff with Python full time.
Probably you would make a command line script to crawl and scrape that data into a structured form and then insert it into your web2py app database. On Sep 12, 7:09 am, bally boy <[email protected]> wrote: > I was just contemplating on this idea to develop some sort of app which > could crawl through the various social networking sites, look at the > comments ppl have posted on various things rangin from music to songs to > products to movies , and then categorize them into individual sections and > grade each product based on certain criterias etc. > For eg: there might be many ppl who might have said something on a certain > movie ' X' or a certain gadget 'Y', on sites like twitter or facebook.Could > a tool be made to crawl across and fish those comments or tweets and try to > make something intelligent out of that. I am sure there should be a lot of > intelligence hidden up there in those social networking sites which can be > utilized in making something comprehensible. > > Would love it if someone could please give a thought to it, and yeah would > such a application be feasible using web2py.(although I have not yet even > laid out the path ahead)
