Hi Joachim,
a first run has created the language model for german with 7 gb.
the machine is becoming very slow and may be swapping...
but i have used an outdated index_db.sh and an outdated(?) gittree.
(after it has been installed i could no longer build pignlproc, it was
complaining of missing classes)
git clone --depth 1 https://github.com/jodaiber/dbpedia-spotlight.git
changed the index/pom.xml there and called the launcher because i have
thought there is a memory issue when launching etc...
<launcher>
<id>CreateSpotlightModel</id>
<mainClass>org.dbpedia.spotlight.db.CreateSpotlightModel</mainClass>
<jvmArgs>
<jvmArg>-Xmx7g</jvmArg>
</jvmArgs>
<!--args>
<arg>../conf/indexing-3.8.properties</arg>
</args-->
</launcher>
i have had 1.3-1.4 mio resources to index.
i have now synced to the officially dbpedia spotlight trunk and i will
redo all steps.
the script index_db.sh has changed a lot compared to the old one i have
used.
i think i have got it from
https://raw.github.com/jodaiber/dbpedia-spotlight/master/bin/index_db.sh
which is linked here
https://github.com/dbpedia-spotlight/dbpedia-spotlight/wiki/Internationalization-%28DB-backed-core%29
in the chapter "The detailed way"
/This section describes the detailed steps for creating the Spotlight
model. These steps are all performed by index_db.sh
<https://raw.github.com/jodaiber/dbpedia-spotlight/master/bin/index_db.sh>.
/best regards
reinhard
Am 19.05.2013 12:03, schrieb Joachim Daiber:
> Hey Reinhard,
>
> I have usually run English with 24G, so German should probably fit in
> 12-14G. However, with 8G you will probably not have much chance to
> run CreateSpotlightModel for anything but the smallest languages.
>
> Best,
> Jo
>
>
> On Sun, May 19, 2013 at 10:21 AM, reinhard schwab
> <[email protected] <mailto:[email protected]>> wrote:
>
> I have just discovered a maven setting at the begin of index_db.sh
>
>
> https://github.com/dbpedia-spotlight/dbpedia-spotlight/blob/master/bin/index_db.sh
>
> export MVN_OPTS="-Xmx26G"
>
> 26G is needed?
> my notebook only has 8 G.
>
> I am using another index_db.sh, not the one in the master git tree,
> there is no such setting.
>
> regards
> reinhard
>
> Am 19.05.2013 09:52, schrieb reinhard schwab:
> > hi,
> >
> > i try to build the a language model using index_db.sh and i am
> at the
> > last step.
> >
> > mvn -pl index exec:java
> > -Dexec.mainClass=org.dbpedia.spotlight.db.CreateSpotlightModel
> > -Dexec.args="$2 $WDIR $TARGET_DIR $opennlp $STOPWORDS $4Stemmer";
> >
> > it has failed with an OutOfMemoryError.
> > how much java heap is needed or what are the memory setting you have
> > used to build the language model for german?
> >
> > best regards
> > reinhard
> >
> >
>
> ------------------------------------------------------------------------------
> > AlienVault Unified Security Management (USM) platform delivers
> complete
> > security visibility with the essential security capabilities.
> Easily and
> > efficiently configure, manage, and operate all of your security
> controls
> > from a single console and one unified framework. Download a free
> trial.
> > http://p.sf.net/sfu/alienvault_d2d
> > _______________________________________________
> > Dbp-spotlight-users mailing list
> > [email protected]
> <mailto:[email protected]>
> > https://lists.sourceforge.net/lists/listinfo/dbp-spotlight-users
> >
> >
>
>
>
> ------------------------------------------------------------------------------
> AlienVault Unified Security Management (USM) platform delivers
> complete
> security visibility with the essential security capabilities.
> Easily and
> efficiently configure, manage, and operate all of your security
> controls
> from a single console and one unified framework. Download a free
> trial.
> http://p.sf.net/sfu/alienvault_d2d
> _______________________________________________
> Dbp-spotlight-users mailing list
> [email protected]
> <mailto:[email protected]>
> https://lists.sourceforge.net/lists/listinfo/dbp-spotlight-users
>
>
------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d
_______________________________________________
Dbp-spotlight-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dbp-spotlight-users
