Hi, I'm interested in "Scrapy benchmarking suite" idea in the ideas list for GSoC '17. Please help with what are the prerequisites for the same.
Thanks. On Saturday, 11 February 2017 21:20:19 UTC+5:30, Parth Verma wrote: > > Hi, > > I am Parth Verma, a second year undergraduate pursuing MSc. in Mathematics > and Computing at IIT Kharagpur, India. > I have been doing open-source programming for a year. My github profile is > https://github.com/Parth-Vader. > My programming knowledge includes Python (Intermediate) , C (Intermediate) > , C++(Intermediate), HTML/CSS (basic) and Bash. I use Ubuntu 16.04 as my > main operating system and Windows 8 for gaming. > I have been doing Data Analytics, and for that, I need to collect data > from various online sources and that's why I used Scrapy. > > I am interested in Scrapy benchmarking suite, since I have prior knowledge > of various algorithms and I want to learn memory management in CPUs. What > should be my next steps? > > Furthermore, I would like to suggest an idea. > > A new section in the official documentation could be added where people > could share their configuration files that they used to successfully scrape > data from a specific website (by successful, I mean not getting banned and > getting a good speed.) This way, I believe , it would be easier for people > without any prior knowledge of HTML, Python or Shell, could easily use > scrapy to get data from those specific sites. > In addition, we could create benchmarking for those sites as well. > > Thanks. > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users+unsubscr...@googlegroups.com. To post to this group, send email to scrapy-users@googlegroups.com. Visit this group at https://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
