Am Donnerstag, 7. Juli 2016 16:56:48 UTC+2 schrieb Peter Otten: > Thomas Kaufmann wrote: > > > Am Donnerstag, 7. Juli 2016 16:05:14 UTC+2 schrieb Thomas Kaufmann: > >> Hi, > >> > >> I have a little scrapy-script: scrapy gets the links from a webpage; this > >> works fine. Then I want to write these links in a sqlite3-table. There is > >> no error-note. But in the database are not any records. What is the > >> problem here? My code: > >> > >> # -*- coding: utf-8 -*- > >> 2 > >> 3 import scrapy > >> 4 import sqlite3 > >> 5 > >> 6 class MySpider(scrapy.Spider): > >> 7 > >> 8 name = "frisch2" > >> 9 allowed_domains = ["frischblau.de"] > >> 10 start_urls = ["http://www.frischblau.de/";,] > >> 11 > >> 12 def parse(self, response): > >> 13 > >> 14 for href in response.xpath("//ul/li/a/@href"): > >> 15 > >> 16 url = response.urljoin(href.extract()) > >> 17 yield scrapy.Request(url, > >> callback=self.parse_dir_contents) 18 > >> 19 def parse_dir_contents(self, response): > >> 20 > >> 21 sql = u' ' > >> 22 conn = sqlite3.connect('frisch.db') > >> 23 cur = conn.cursor() > >> 24 counter = 3; > >> 25 > >> 26 try: > >> 27 cur.execute("SELECT SQLITE_VERSION()") > >> 28 print "-> SQLite version: %s" % cur.fetchone() > >> 29 > >> 30 item = response.xpath('//a/@href').extract() > >> 31 for el in item: > >> 32 > >> 33 #print("----------> ", type(el)) > >> 34 sql = "INSERT INTO Frisch VALUES(" + unicode(counter) > >> + ", " + "'" + el.strip() + "');" 35 > >> 36 print (sql) > >> 37 cur.execute(sql) > >> 38 conn.commit > >> 39 counter = int(counter) > >> 40 counter += 1 > >> 41 > >> 42 except sqlite3.Error as e: > >> 43 print "Error %s:" % e.args[0] > >> 44 conn.close() > >> 45 > >> > >> > >> > >> > >> > >> Thanks for help. > >> > >> o-o > >> > >> Thomas > > > > Hi Chris, > > > > I changed my code in the way you suggested. > > There is no effect in the db. > > Did you replace the line > > conn.commit > > with > > con.commit() > > ? If not, can you provide the updated code? > > > > I start this script on the commandline with: > > > > tk@Hamlet:~/myscrapy/tutorial/tutorial/spiders$ scrapy crawl webbot > > > > When I call the sql-statement in the db it works. > > > > sqlite> INSERT INTO Frisch VALUES(4, '//:Üs.chi/hhh\n'); > > > > sqlite> SELECT * FROM Frisch; > > 1|Üschi/hhh > > 2|Üschi/hhh\n > > 3|//:Üs.chi/hhh\n > > 4|//:Üs.chi/hhh\n > > > > > > > > > > > > > >
Thanx a lot Peter. IT WORKS!!! -- https://mail.python.org/mailman/listinfo/python-list
