David Reyes Samblas Martinez wrote: > Well spanish one give me the same error before but now it works, Any idea what solved it? Or is it just random and will go away if I try it again? :)
> I'm parsing the de wikipedia right now (Count: 173000) lets see whats > happens :) I would definitely be interessted in the results... > Note:Parsing the 2009-Nov-11 > http://download.wikipedia.org/dewiki/latest/dewiki-latest-pages-articles.xml.bz2 > > Regards > > David Reyes Samblas Martinez > http://www.tuxbrain.com > Open ultraportable & embedded solutions > Openmoko, Openpandora, Arduino > Hey, watch out!!! There's a linux in your pocket!!! > > > > > 2009/11/20 Tilman Baumann <[email protected]>: >> Can you reproduce this with a neutral locale? >> export LC_ALL=C >> >> I'm at the moment trying the same. I had a lot of hickups, caused by >> many >> things. Among them missing tools and not enough memory. >> >> This is currently where I'm stuck with the German wikipedia. >> >> Count: 823000 >> Count: 824000 >> Count: 825000 >> Count: 826000 >> Count: 827000 >> Count: 828000 >> Count: 829000 >> Count: 830000 >> Count: 831000 >> Count: 832000 >> Count: 833000 >> Traceback (most recent call last): >> File "./ArticleParser.py", line 203, in <module> >> main() >> File "./ArticleParser.py", line 168, in main >> process_article_text(title.encode('utf-8'), f.read(length), newf) >> File "./ArticleParser.py", line 197, in process_article_text >> newf.write(text + '\n') >> IOError: [Errno 32] Broken pipe >> make[1]: *** [parse] Error 1 >> make[1]: Leaving directory >> `/home/tilli/wikireader/host-tools/offline-renderer' >> make: *** [parse] Error 2 >> >> I suppose it failed somewhere in PARSER_COMMAND >> >> >> Before that, the following steps went through without fail. >> make >> make DESTDIR=image WORKDIR=work >> XML_FILES=dewiki-20091028-pages-articles.xml index >> >> >> David Reyes Samblas Martinez wrote: >>> After the "success" of the spanish wikipedia pending to resolve the >>> indexing part, I was starting to work on the german wikipedia >>> http://download.wikipedia.org/dewiki/latest/dewiki-latest-pages-meta-current.xml.bz2 >>> >>> but it fails at first step with the following error >>> >>> #make DESTDIR=image WORKDIR=work >>> XML_FILES=dewiki-latest-pages-meta-current.xml index parse render >>> combine >>> >>> awk: línea ord.:1: fatal: no se puede abrir el fichero >>> `work/counts.text' para lectura (No existe el fichero ó directorio) >>> cd host-tools/offline-renderer && make index \ >>> >>> XML_FILES="/OE/Proyectos/tuxbrain/productos/wikireader/wikireader/dewiki-latest-pages-meta-current.xml" >>> RENDER_BLOCK="0" \ >>> >>> WORKDIR="/OE/Proyectos/tuxbrain/productos/wikireader/wikireader/work" >>> DESTDIR="/OE/Proyectos/tuxbrain/productos/wikireader/wikireader/image" >>> make[1]: se ingresa al directorio >>> `/OE/Proyectos/tuxbrain/productos/wikireader/wikireader/host-tools/offline-renderer' >>> ./ArticleIndex.py \ >>> >>> --article-index="/OE/Proyectos/tuxbrain/productos/wikireader/wikireader/work/articles.db" >>> \ >>> >>> --article-offsets="/OE/Proyectos/tuxbrain/productos/wikireader/wikireader/work/offsets.db" >>> \ >>> >>> --article-counts="/OE/Proyectos/tuxbrain/productos/wikireader/wikireader/work/counts.text" >>> \ >>> >>> --prefix="/OE/Proyectos/tuxbrain/productos/wikireader/wikireader/image/pedia" >>> /OE/Proyectos/tuxbrain/productos/wikireader/wikireader/dewiki-latest-pages-meta-current.xml >>> Traceback (most recent call last): >>> File "./ArticleIndex.py", line 611, in <module> >>> main() >>> File "./ArticleIndex.py", line 172, in main >>> limit = processor.process(f, limit) >>> File >>> "/OE/Proyectos/tuxbrain/productos/wikireader/wikireader/host-tools/offline-renderer/FileScanner.py", >>> line 141, in process >>> if '#' == body[0] and 'redirect' == body[1:9].lower(): >>> IndexError: string index out of range >>> Flushing databases >>> Writing: files >>> Time: 0s >>> Writing: articles >>> Time: 0s >>> Writing: offsets >>> Time: 0s >>> Loading: articles >>> Time: 0s >>> Loading: offsets and files >>> Time: 0s >>> make[1]: *** [index] Error 1 >>> make[1]: se sale del directorio >>> `/OE/Proyectos/tuxbrain/productos/wikireader/wikireader/host-tools/offline-renderer' >>> make: *** [index] Error 2 >>> >>> Regards >>> >>> David Reyes Samblas Martinez >>> http://www.tuxbrain.com >>> Open ultraportable & embedded solutions >>> Openmoko, Openpandora, Arduino >>> Hey, watch out!!! There's a linux in your pocket!!! >>> >>> _______________________________________________ >>> Openmoko community mailing list >>> [email protected] >>> http://lists.openmoko.org/mailman/listinfo/community >>> >> >> >> -- >> >> >> >> _______________________________________________ >> Openmoko community mailing list >> [email protected] >> http://lists.openmoko.org/mailman/listinfo/community >> > > _______________________________________________ > Openmoko community mailing list > [email protected] > http://lists.openmoko.org/mailman/listinfo/community > -- _______________________________________________ Openmoko community mailing list [email protected] http://lists.openmoko.org/mailman/listinfo/community
