After you run generate-settings, the file
core/src/main/scala/org/dbpedia/extraction/wikiparser/impl/wikipedia/Namespaces.scala
should contain the new 710 namespace. Could you post the contents of that
file?
When you run ../clean-install-run extraction , that file should be compiled
and the extraction should know about the new namespace. Unfortunately,
there's currently no easy way to trigger the bug quickly.
Maybe Maven somehow uses an old version of the Namespaces.scala class. You
could delete ~/.m2/org/dbpedia/* and try again.
Oh, I just noticed that Dimitris updated Namespaces.scala a few days ago.
If you update your local copy (hg pull, hg update), things should work.
HTH,
Christopher
http://dbpedia.hg.sourceforge.net/hgweb/dbpedia/extraction_framework/file/dump/core/src/main/scala/org/dbpedia/extraction/wikiparser/impl/wikipedia/Namespaces.scala
On Nov 25, 2012 5:15 PM, "Omri Oren" <[email protected]> wrote:
> Hi,
> Thanks, I tried that. It didn't help.
> I also tried running "../clean-install-run extraction" (instead of
> "../run extraction") like Jona suggested, and no luck there either.
> Does the extraction work in the last couple of days for any of you?
> Batica? Anyone...? :-(
> Also, how can I run a check that will not run for ~70 minutes before I see
> that the exception is raised again?
>
>
>
> On Sun, Nov 25, 2012 at 4:05 PM, Dimitris Kontokostas
> <[email protected]>wrote:
>
>> Hi Omni,
>>
>> Did you recompile / install the core module before you run the Dump
>> extraction?
>> try running
>> $mvn clean install from the root directory before the
>> $../run extraction
>>
>> Cheers,
>> Dimitris
>>
>>
>> On Sun, Nov 25, 2012 at 11:24 AM, Omri Oren <[email protected]> wrote:
>>
>>> Hi Jona, Batica,
>>>
>>> I ran into the same "*java.util.NoSuchElementException: key not found:
>>> 710*" problem.
>>> As Jona suggested, I tried running "../clean-install-run
>>> generate-settings" from ".../core" but the 710 still raises an exception
>>> when running "../run extraction"
>>> (I'm running on the English wikipedia dump, with
>>> "extractors=RedirectExtractor,InterLanguageLinksExtractor,MappingExtractor")
>>> BTW, the 710 exception raises after 70 minutes of processing! (i.e. it's
>>> not something that happens in the first stages of extraction)
>>>
>>> Have any of you managed to solve it yet?
>>> Any idea what to try next?
>>>
>>> Thanks,
>>> Omri
>>>
>>>
>>>
>>> p.s. this is the output:
>>>
>>> [...]
>>> Nov 22, 2012 7:24:50 PM
>>> org.dbpedia.extraction.mappings.Redirects$RedirectFinder apply
>>> WARNING: wrong redirect. page: [title=John Courthope
>>> (disambiguation);ns=0/Main/;language:wiki=en,locale=en].
>>> found by dbpedia: [title=John
>>> Courthope;ns=0/Main/;language:wiki=en,locale=en].
>>> found by wikipedia: [null]
>>> java.lang.reflect.InvocationTargetException
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:601)
>>> at
>>> org_scala_tools_maven_executions.MainHelper.runMain(MainHelper.java:161)
>>> at
>>> org_scala_tools_maven_executions.MainWithArgsInFile.main(MainWithArgsInFile.java:26)
>>> Caused by: java.util.NoSuchElementException: key not found: 710
>>> at scala.collection.MapLike$class.default(MapLike.scala:225)
>>> at scala.collection.immutable.HashMap.default(HashMap.scala:38)
>>> at scala.collection.MapLike$class.apply(MapLike.scala:135)
>>> at scala.collection.immutable.HashMap.apply(HashMap.scala:38)
>>> at
>>> org.dbpedia.extraction.sources.WikipediaDumpParser.readPage(WikipediaDumpParser.java:224)
>>> at
>>> org.dbpedia.extraction.sources.WikipediaDumpParser.readPages(WikipediaDumpParser.java:185)
>>> at
>>> org.dbpedia.extraction.sources.WikipediaDumpParser.readDump(WikipediaDumpParser.java:143)
>>> at
>>> org.dbpedia.extraction.sources.WikipediaDumpParser.run(WikipediaDumpParser.java:114)
>>> at
>>> org.dbpedia.extraction.sources.XMLReaderSource.foreach(XMLSource.scala:64)
>>> at
>>> scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:239)
>>> at
>>> org.dbpedia.extraction.sources.XMLReaderSource.flatMap(XMLSource.scala:60)
>>> at
>>> org.dbpedia.extraction.mappings.Redirects$.loadFromSource(Redirects.scala:165)
>>> at
>>> org.dbpedia.extraction.mappings.Redirects$.load(Redirects.scala:116)
>>> at
>>> org.dbpedia.extraction.dump.extract.ConfigLoader$$anon$1.<init>(ConfigLoader.scala:96)
>>> at org.dbpedia.extraction.dump.extract.ConfigLoader.org
>>> $dbpedia$extraction$dump$extract$ConfigLoader$$createExtractionJob(ConfigLoader.scala:51)
>>> at
>>> org.dbpedia.extraction.dump.extract.ConfigLoader$$anonfun$getExtractionJobs$1.apply(ConfigLoader.scala:36)
>>> at
>>> org.dbpedia.extraction.dump.extract.ConfigLoader$$anonfun$getExtractionJobs$1.apply(ConfigLoader.scala:36)
>>> at scala.collection.Iterator$$anon$19.next(Iterator.scala:401)
>>> at scala.collection.Iterator$class.foreach(Iterator.scala:772)
>>> at scala.collection.Iterator$$anon$19.foreach(Iterator.scala:399)
>>> at
>>> scala.collection.IterableViewLike$Transformed$class.foreach(IterableViewLike.scala:41)
>>> at
>>> scala.collection.IterableViewLike$$anon$3.foreach(IterableViewLike.scala:80)
>>> at
>>> org.dbpedia.extraction.dump.extract.Extraction$.main(Extraction.scala:29)
>>> at
>>> org.dbpedia.extraction.dump.extract.Extraction.main(Extraction.scala)
>>> ... 6 more
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [ERROR] BUILD ERROR
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] wrap: org.apache.commons.exec.ExecuteException: Process exited
>>> with an error: 240(Exit value: 240)
>>>
>>>
>>>
>>>
>>> On Thu, Nov 22, 2012 at 11:00 AM, Jona Christopher Sahnwaldt <
>>> [email protected]> wrote:
>>>
>>>> Hi,
>>>>
>>>> the problem is not the XML namespace.
>>>>
>>>> > Caused by: java.util.NoSuchElementException: key not found: 710
>>>> > at scala.collection.MapLike$class.default(MapLike.scala:225)
>>>> > at
>>>> scala.collection.immutable.HashMap.default(HashMap.scala:38)
>>>> > at scala.collection.MapLike$class.apply(MapLike.scala:135)
>>>> > at scala.collection.immutable.HashMap.apply(HashMap.scala:38)
>>>> > at
>>>> org.dbpedia.extraction.sources.WikipediaDumpParser.readPage(WikipediaDumpParser.java:224)
>>>>
>>>> The problem is that en.wiki recently added MediaWiki namespace code
>>>> 710 [1] that's not included in our configuration [2] yet. In line 224
>>>> in WikipediaDumpParser.java [3] we look up the namespace number that
>>>> we found in the Wikipedia XML dump in our configuration. That lookup
>>>> crashes if the namespace code is unknown.
>>>>
>>>> To update the configuration, please cd to core/ and run
>>>>
>>>> ../clean-install-run generate-settings
>>>>
>>>> This will download the current MediaWiki namespace settings from all
>>>> Wikipedia editions. May take a few minutes. Then run the extraction
>>>> again.
>>>>
>>>> Hope this helps!
>>>>
>>>> I'll try to update the configuration in the Mercurial repository soon.
>>>>
>>>> Cheers,
>>>> Christopher
>>>>
>>>> [1]
>>>> http://en.wikipedia.org/w/api.php?action=query&meta=siteinfo&siprop=namespaces
>>>> [2]
>>>> http://dbpedia.hg.sourceforge.net/hgweb/dbpedia/extraction_framework/file/331f8aba5f7d/core/src/main/scala/org/dbpedia/extraction/wikiparser/impl/wikipedia/Namespaces.scala
>>>> [3]
>>>> http://dbpedia.hg.sourceforge.net/hgweb/dbpedia/extraction_framework/file/331f8aba5f7d/core/src/main/java/org/dbpedia/extraction/sources/WikipediaDumpParser.java#l224
>>>>
>>>> On Mon, Nov 19, 2012 at 3:50 PM, Batica Dzonic <[email protected]>
>>>> wrote:
>>>> >
>>>> > Wikipedia again changed XML scheme from
>>>> > http://www.mediawiki.org/xml/export-0.7/ to 0.8...
>>>> > I had the following problem with the dump extraction Loading Redirect
>>>> From
>>>> > source (en)
>>>> >
>>>> > found by dbpedia: [title=Girls' Generation II ~Girls &
>>>> > Peace~;ns=0/Main/;language:wiki=en,locale=en].
>>>> > found by wikipedia: [title=Girls' Generation II: Girls &
>>>> > Peace;ns=0/Main/;language:wiki=en,locale=en]
>>>> > Nov 19, 2012 2:14:46 PM
>>>> > org.dbpedia.extraction.mappings.Redirects$RedirectFinder apply
>>>> > WARNING: wrong redirect. page:
>>>> > [title=萬琲臻;ns=0/Main/;language:wiki=en,locale=en].
>>>> > found by dbpedia: [title=Sharon
>>>> > Wan;ns=0/Main/;language:wiki=en,locale=en].
>>>> > found by wikipedia: [null]
>>>> > Nov 19, 2012 2:14:46 PM
>>>> > org.dbpedia.extraction.mappings.Redirects$RedirectFinder apply
>>>> > WARNING: wrong redirect. page: [title=John Courthope
>>>> > (disambiguation);ns=0/Main/;language:wiki=en,locale=en].
>>>> > found by dbpedia: [title=John
>>>> > Courthope;ns=0/Main/;language:wiki=en,locale=en].
>>>> > found by wikipedia: [null]
>>>> > java.lang.reflect.InvocationTargetException
>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> > at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>> > at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> > at java.lang.reflect.Method.invoke(Method.java:601)
>>>> > at
>>>> >
>>>> org_scala_tools_maven_executions.MainHelper.runMain(MainHelper.java:161)
>>>> > at
>>>> >
>>>> org_scala_tools_maven_executions.MainWithArgsInFile.main(MainWithArgsInFile.java:26)
>>>> > Caused by: java.util.NoSuchElementException: key not found: 710
>>>> > at scala.collection.MapLike$class.default(MapLike.scala:225)
>>>> > at
>>>> scala.collection.immutable.HashMap.default(HashMap.scala:38)
>>>> > at scala.collection.MapLike$class.apply(MapLike.scala:135)
>>>> > at scala.collection.immutable.HashMap.apply(HashMap.scala:38)
>>>> > at
>>>> >
>>>> org.dbpedia.extraction.sources.WikipediaDumpParser.readPage(WikipediaDumpParser.java:224)
>>>> > at
>>>> >
>>>> org.dbpedia.extraction.sources.WikipediaDumpParser.readPages(WikipediaDumpParser.java:185)
>>>> > at
>>>> >
>>>> org.dbpedia.extraction.sources.WikipediaDumpParser.readDump(WikipediaDumpParser.java:143)
>>>> > at
>>>> >
>>>> org.dbpedia.extraction.sources.WikipediaDumpParser.run(WikipediaDumpParser.java:114)
>>>> > at
>>>> >
>>>> org.dbpedia.extraction.sources.XMLReaderSource.foreach(XMLSource.scala:57)
>>>> > at
>>>> >
>>>> scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:239)
>>>> > at
>>>> >
>>>> org.dbpedia.extraction.sources.XMLReaderSource.flatMap(XMLSource.scala:53)
>>>> > at
>>>> >
>>>> org.dbpedia.extraction.mappings.Redirects$.loadFromSource(Redirects.scala:165)
>>>> > at
>>>> > org.dbpedia.extraction.mappings.Redirects$.load(Redirects.scala:116)
>>>> > at
>>>> >
>>>> org.dbpedia.extraction.dump.extract.ConfigLoader$$anon$1.<init>(ConfigLoader.scala:100)
>>>> > at
>>>> > org.dbpedia.extraction.dump.extract.ConfigLoader.org
>>>> $dbpedia$extraction$dump$extract$ConfigLoader$$createExtractionJob(ConfigLoader.scala:55)
>>>> > at
>>>> >
>>>> org.dbpedia.extraction.dump.extract.ConfigLoader$$anonfun$getExtractionJobs$1.apply(ConfigLoader.scala:40)
>>>> > at
>>>> >
>>>> org.dbpedia.extraction.dump.extract.ConfigLoader$$anonfun$getExtractionJobs$1.apply(ConfigLoader.scala:40)
>>>> > at scala.collection.Iterator$$anon$19.next(Iterator.scala:401)
>>>> > at scala.collection.Iterator$class.foreach(Iterator.scala:772)
>>>> > at
>>>> scala.collection.Iterator$$anon$19.foreach(Iterator.scala:399)
>>>> > at
>>>> >
>>>> scala.collection.IterableViewLike$Transformed$class.foreach(IterableViewLike.scala:41)
>>>> > at
>>>> >
>>>> scala.collection.IterableViewLike$$anon$3.foreach(IterableViewLike.scala:80)
>>>> > at
>>>> >
>>>> org.dbpedia.extraction.dump.extract.Extraction$.main(Extraction.scala:27)
>>>> > at
>>>> > org.dbpedia.extraction.dump.extract.Extraction.main(Extraction.scala)
>>>> > ... 6 more
>>>> > [INFO]
>>>> >
>>>> ------------------------------------------------------------------------
>>>> > [ERROR] BUILD ERROR
>>>> > [INFO]
>>>> >
>>>> ------------------------------------------------------------------------
>>>> > [INFO] wrap: org.apache.commons.exec.ExecuteException: Process exited
>>>> with
>>>> > an error: 240(Exit value: 240)
>>>> >
>>>> > [INFO]
>>>> >
>>>> ------------------------------------------------------------------------
>>>> > [INFO] Trace
>>>> > org.apache.maven.lifecycle.LifecycleExecutionException: wrap:
>>>> > org.apache.commons.exec.ExecuteException: Process exited with an
>>>> error:
>>>> > 240(Exit value: 240)
>>>> > at
>>>> >
>>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:719)
>>>> > at
>>>> >
>>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeStandaloneGoal(DefaultLifecycleExecutor.java:569)
>>>> > at
>>>> >
>>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:539)
>>>> > at
>>>> >
>>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
>>>> > at
>>>> >
>>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
>>>> > at
>>>> >
>>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
>>>> > at
>>>> org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
>>>> > at
>>>> org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
>>>> > at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
>>>> > at
>>>> >
>>>> org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> > at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>> > at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> > at java.lang.reflect.Method.invoke(Method.java:601)
>>>> > at
>>>> > org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
>>>> > at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
>>>> > at
>>>> > org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
>>>> > at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
>>>> > Caused by: org.apache.maven.plugin.MojoExecutionException: wrap:
>>>> > org.apache.commons.exec.ExecuteException: Process exited with an
>>>> error:
>>>> > 240(Exit value: 240)
>>>> > at
>>>> >
>>>> org_scala_tools_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:350)
>>>> > at
>>>> >
>>>> org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
>>>> > at
>>>> >
>>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
>>>> > ... 17 more
>>>> > Caused by: org.apache.commons.exec.ExecuteException: Process exited
>>>> with
>>>> > an error: 240(Exit value: 240)
>>>> > at
>>>> >
>>>> org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:346)
>>>> > at
>>>> >
>>>> org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:149)
>>>> > at
>>>> >
>>>> org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:136)
>>>> > at
>>>> >
>>>> org_scala_tools_maven_executions.JavaMainCallerByFork.run(JavaMainCallerByFork.java:80)
>>>> > at
>>>> >
>>>> org_scala_tools_maven_executions.JavaMainCallerSupport.run(JavaMainCallerSupport.java:96)
>>>> > at
>>>> > org_scala_tools_maven.ScalaRunMojo.doExecute(ScalaRunMojo.java:105)
>>>> > at
>>>> >
>>>> org_scala_tools_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:342)
>>>> > ... 19 more
>>>> > [INFO]
>>>> >
>>>> ------------------------------------------------------------------------
>>>> > [INFO] Total time: 37 minutes 58 seconds
>>>> > [INFO] Finished at: Mon Nov 19 14:14:46 UTC 2012
>>>> > [INFO] Final Memory: 19M/46M
>>>> > [INFO]
>>>> >
>>>> ------------------------------------------------------------------------
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > Has anyone encountered this problem or know how to skip this?
>>>> >
>>>> >
>>>> >
>>>> ------------------------------------------------------------------------------
>>>> > Monitor your physical, virtual and cloud infrastructure from a single
>>>> > web console. Get in-depth insight into apps, servers, databases,
>>>> vmware,
>>>> > SAP, cloud infrastructure, etc. Download 30-day Free Trial.
>>>> > Pricing starts from $795 for 25 servers or applications!
>>>> > http://p.sf.net/sfu/zoho_dev2dev_nov
>>>> > _______________________________________________
>>>> > Dbpedia-discussion mailing list
>>>> > [email protected]
>>>> > https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
>>>> >
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Monitor your physical, virtual and cloud infrastructure from a single
>>>> web console. Get in-depth insight into apps, servers, databases, vmware,
>>>> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
>>>> Pricing starts from $795 for 25 servers or applications!
>>>> http://p.sf.net/sfu/zoho_dev2dev_nov
>>>> _______________________________________________
>>>> Dbpedia-discussion mailing list
>>>> [email protected]
>>>> https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
>>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Monitor your physical, virtual and cloud infrastructure from a single
>>> web console. Get in-depth insight into apps, servers, databases, vmware,
>>> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
>>> Pricing starts from $795 for 25 servers or applications!
>>> http://p.sf.net/sfu/zoho_dev2dev_nov
>>> _______________________________________________
>>> Dbpedia-discussion mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
>>>
>>>
>>
>>
>> --
>> Kontokostas Dimitris
>>
>
>
------------------------------------------------------------------------------
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
_______________________________________________
Dbpedia-discussion mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
