indeed, thanks! —— Dr. Luca Fiaschi Director of Advertising Technology & Automation
Rocket Internet SE Charlottenstraße 4, 10969 Berlin | Germany Mobile: +49 174 1839 522 <tel:%2B49%20162%202904708> | Skype: fiaschi.job | www.rocket-internet.com <http://www.rocket-internet.com/> Management Board: Oliver Samwer (CEO), Peter Kimpel, Alexander Kudlich Chairman of the Supervisory Board: Prof. Dr. Marcus Englert Registration Court: Amtsgericht Charlottenburg, Registration No.: HRB 165662 B > On 24 Oct 2016, at 02:57, Rolando Espinoza <dark...@gmail.com> wrote: > > This error may happen when you have a pipeline that swallow the item, that > is, it doesn't return the item. > > Rolando > > On Sun, Oct 23, 2016 at 7:51 PM, Luca Fiaschi > <luca.fias...@rocket-internet.com <mailto:luca.fias...@rocket-internet.com>> > wrote: > I have a scraper which uses the Item pipeline to save items in a database. > > When running my scraper on scrapinghub I get that all the requests are > registered as failed and the following errors are reported. > > > Traceback (most recent call last): > File "/app/python/lib/python2.7/site-packages/twisted/internet/defer.py", > line 149, in maybeDeferred > result = f(*args, **kw) > File "/usr/local/lib/python2.7/site-packages/pydispatch/robustapply.py", > line 55, in robustApply > return receiver(*arguments, **named) > File "/usr/local/lib/python2.7/site-packages/sh_scrapy/extension.py", line > 39, in item_scraped > item = self.exporter.export_item(item) > File "/app/python/lib/python2.7/site-packages/scrapy/exporters.py", line > 304, in export_item > result = dict(self._get_serialized_fields(item)) > File "/app/python/lib/python2.7/site-packages/scrapy/exporters.py", line > 65, in _get_serialized_fields > field_iter = six.iterkeys(item) > File "/usr/local/lib/python2.7/site-packages/six.py", line 593, in iterkeys > return d.iterkeys(**kw) > AttributeError: 'NoneType' object has no attribute 'iterkeys' > > From the scraping hub dashboard I can see that all items appears as failed (0 > processed), however the scraper seems to work fine and serielizes the correct > data in the database. > > Anybody can explain this? > > > > Does it maybe depend on the fact that I am not retorning items as jsons, but > only serializing to database? > > > CONFIDENTIALITY NOTICE: This message (including any attachments) is > confidential and may be privileged. It may be read, copied and used only by > the intended recipient. If you have received it in error please contact the > sender (by return e-mail) immediately and delete this message. Any > unauthorized use or dissemination of this message in whole or in parts is > strictly prohibited. > > DISCLAIMER: The information contained in this email, attachment, document > and/or presentation have been researched by us with due care and all and any > evaluations or assessments stated herein represent our personal opinions. We > advise you that some of the available information has been independently > verified and may be based on statements by third persons, but no > representation or warranty, expressed or implied, is made as to, and no > reliance should be placed on, the fairness, accuracy, completeness or > correctness of this information or opinions contained herein. Certain > statements contained herein may be statements of future expectations and > other forward-looking statements that are based on our current personal views > and assumptions and involve known and unknown risks and uncertainties that > cause actual results, performance or events to differ materially from those > expressed or implied in such statements. None of our team shall assume any > liability whatsoever (in negligence or otherwise) for any loss howsoever > arising from any use of this email, attachment and/or presentation or the > statements contained herein as to unverified third person statements, any > statements of future expectations and other forward-looking statements, or > the fairness, accuracy, completeness or correctness of statements contained > herein, or otherwise arising in connection with this email, attachment, > document and/or presentation. > > This email, attachment, document and/or presentation does not constitute an > offer or invitation to purchase or subscribe for any investment and neither > it nor any part of it shall form the basis of or be relied upon in connection > with any commitment or contract whatsoever, and is solely intended to > communicate a favourable impression of our team members and their business > track-records. No one should base their investment decisions on this email, > attachment, document and/or presentation, which principally constitutes an > advertising message, as their main source of information or reliance but > should form their personal opinions solely on the basis of further available > information and evaluations and assumptions. The legal documentation held > ready for any prospective investor shall be authorised as the sole source of > relevant information and neither it nor any part of it shall constitute any > representation, warranty or other undertaking, expressed or implied, as to > the economic success of the offered investments. > > -- > You received this message because you are subscribed to the Google Groups > "scrapy-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to scrapy-users+unsubscr...@googlegroups.com > <mailto:scrapy-users+unsubscr...@googlegroups.com>. > To post to this group, send email to scrapy-users@googlegroups.com > <mailto:scrapy-users@googlegroups.com>. > Visit this group at https://groups.google.com/group/scrapy-users > <https://groups.google.com/group/scrapy-users>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. > > > -- > You received this message because you are subscribed to a topic in the Google > Groups "scrapy-users" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/scrapy-users/NNqSw0hBmu8/unsubscribe > <https://groups.google.com/d/topic/scrapy-users/NNqSw0hBmu8/unsubscribe>. > To unsubscribe from this group and all its topics, send an email to > scrapy-users+unsubscr...@googlegroups.com > <mailto:scrapy-users+unsubscr...@googlegroups.com>. > To post to this group, send email to scrapy-users@googlegroups.com > <mailto:scrapy-users@googlegroups.com>. > Visit this group at https://groups.google.com/group/scrapy-users > <https://groups.google.com/group/scrapy-users>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. -- CONFIDENTIALITY NOTICE: This message (including any attachments) is confidential and may be privileged. It may be read, copied and used only by the intended recipient. If you have received it in error please contact the sender (by return e-mail) immediately and delete this message. Any unauthorized use or dissemination of this message in whole or in parts is strictly prohibited. DISCLAIMER: The information contained in this email, attachment, document and/or presentation have been researched by us with due care and all and any evaluations or assessments stated herein represent our personal opinions. We advise you that some of the available information has been independently verified and may be based on statements by third persons, but no representation or warranty, expressed or implied, is made as to, and no reliance should be placed on, the fairness, accuracy, completeness or correctness of this information or opinions contained herein. Certain statements contained herein may be statements of future expectations and other forward-looking statements that are based on our current personal views and assumptions and involve known and unknown risks and uncertainties that cause actual results, performance or events to differ materially from those expressed or implied in such statements. None of our team shall assume any liability whatsoever (in negligence or otherwise) for any loss howsoever arising from any use of this email, attachment and/or presentation or the statements contained herein as to unverified third person statements, any statements of future expectations and other forward-looking statements, or the fairness, accuracy, completeness or correctness of statements contained herein, or otherwise arising in connection with this email, attachment, document and/or presentation. This email, attachment, document and/or presentation does not constitute an offer or invitation to purchase or subscribe for any investment and neither it nor any part of it shall form the basis of or be relied upon in connection with any commitment or contract whatsoever, and is solely intended to communicate a favourable impression of our team members and their business track-records. No one should base their investment decisions on this email, attachment, document and/or presentation, which principally constitutes an advertising message, as their main source of information or reliance but should form their personal opinions solely on the basis of further available information and evaluations and assumptions. The legal documentation held ready for any prospective investor shall be authorised as the sole source of relevant information and neither it nor any part of it shall constitute any representation, warranty or other undertaking, expressed or implied, as to the economic success of the offered investments. -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users+unsubscr...@googlegroups.com. To post to this group, send email to scrapy-users@googlegroups.com. Visit this group at https://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
