This error may happen when you have a pipeline that swallow the item, that is, it doesn't return the item.
Rolando On Sun, Oct 23, 2016 at 7:51 PM, Luca Fiaschi < luca.fias...@rocket-internet.com> wrote: > I have a scraper which uses the Item pipeline to save items in a database. > > When running my scraper on scrapinghub I get that all the requests are > registered as failed and the following errors are reported. > > Traceback (most recent call last): File "/app/python/lib/python2.7/ > site-packages/twisted/internet/defer.py", line 149, in maybeDeferred > result = f(*args, **kw) File "/usr/local/lib/python2.7/ > site-packages/pydispatch/robustapply.py", line 55, in robustApply return > receiver(*arguments, **named) File "/usr/local/lib/python2.7/ > site-packages/sh_scrapy/extension.py", line 39, in item_scraped item = > self.exporter.export_item(item) File "/app/python/lib/python2.7/ > site-packages/scrapy/exporters.py", line 304, in export_item result = > dict(self._get_serialized_fields(item)) File "/app/python/lib/python2.7/ > site-packages/scrapy/exporters.py", line 65, in _get_serialized_fields > field_iter = six.iterkeys(item) File > "/usr/local/lib/python2.7/site-packages/six.py", > line 593, in iterkeys return d.iterkeys(**kw) AttributeError: 'NoneType' > object has no attribute 'iterkeys' > > From the scraping hub dashboard I can see that all items appears as failed > (0 processed), however the scraper seems to work fine and serielizes the > correct data in the database. > > Anybody can explain this? > > > Does it maybe depend on the fact that I am not retorning items as jsons, > but only serializing to database? > > CONFIDENTIALITY NOTICE: This message (including any attachments) > is confidential and may be privileged. It may be read, copied and used only > by the intended recipient. If you have received it in error please contact > the sender (by return e-mail) immediately and delete this message. Any > unauthorized use or dissemination of this message in whole or in parts is > strictly prohibited. > > DISCLAIMER: The information contained in this email, attachment, > document and/or presentation have been researched by us with due care and > all and any evaluations or assessments stated herein represent our > personal opinions. We advise you that some of the available information has > been independently verified and may be based on statements by third > persons, but no representation or warranty, expressed or implied, is made > as to, and no reliance should be placed on, the fairness, accuracy, > completeness or correctness of this information or opinions > contained herein. Certain statements contained herein may be statements of > future expectations and other forward-looking statements that are based on > our current personal views and assumptions and involve known and unknown > risks and uncertainties that cause actual results, performance or events to > differ materially from those expressed or implied in such statements. None > of our team shall assume any liability whatsoever (in negligence or > otherwise) for any loss howsoever arising from any use of this email, > attachment and/or presentation or the statements contained herein as > to unverified third person statements, any statements of future > expectations and other forward-looking statements, or the fairness, > accuracy, completeness or correctness of statements contained herein, or > otherwise arising in connection with this email, attachment, > document and/or presentation. > > This email, attachment, document and/or presentation does not constitute > an offer or invitation to purchase or subscribe for any investment and > neither it nor any part of it shall form the basis of or be relied upon in > connection with any commitment or contract whatsoever, and is > solely intended to communicate a favourable impression of our team members > and their business track-records. No one should base their investment > decisions on this email, attachment, document and/or presentation, which > principally constitutes an advertising message, as their main source of > information or reliance but should form their personal opinions solely on > the basis of further available information and evaluations and assumptions. > The legal documentation held ready for any prospective investor shall be > authorised as the sole source of relevant information and neither it nor > any part of it shall constitute any representation, warranty or > other undertaking, expressed or implied, as to the economic success of the > offered investments. > > -- > You received this message because you are subscribed to the Google Groups > "scrapy-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to scrapy-users+unsubscr...@googlegroups.com. > To post to this group, send email to scrapy-users@googlegroups.com. > Visit this group at https://groups.google.com/group/scrapy-users. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users+unsubscr...@googlegroups.com. To post to this group, send email to scrapy-users@googlegroups.com. Visit this group at https://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
