Hi Kaspar,
Thank you for your interest.
All the cached data is stored in the $KIM_HOME/context/default/populated
folder. It's enough to clear its contents to get rid of the cache. This
will affect both Sesame (the semantic repository) and Lucene's (the
document repository) stored data.
The idea behind the context folder of KIM is that all the data that may
be changed in order to tune KIM to the specific domain will be stored in
a centralized location. This location is the context folder. It contains
three parts. The 'kb' folder is the place for the ontology schema and
the knowledge base (KB). If you decide to extend the default KIM
ontology, this is a good place to add the files describing it. The
'resources' folder contains the resources which the NERC pipeline uses
for the annotation process. Here is the pipeline description itself, the
Jape grammars, the gazetteer lists etc. And of course the 'populated'
folder which contains KIM's cached data.
The reason the jape didn't work is that the transducer in the default
pipeline you are using reads its rules from another location . The jape
file used is $KIM_HOME/context/default/resources/grammar/main/main.jape
. This is where you have to include the new grammar. The GATE pipeline
configuration file is described in $KIM_HOME/nerc.properties . In the
default distribution KIM uses IE-no-keywords.gapp . This is where all
the processing resources are configured. Here you can see all the
transducers and the japes they use.
I will need more information about the exception. Please change the
debug level of Lucene in the log4j configuration file:
$KIM_HOME/config/log.prop . Change the line :
log4j.category.LUCENE = , aLUCENE
to :
log4j.category.LUCENE = DEBUG, aLUCENE
Then you should be able to see more details in the LUCENE log file in
$KIM_HOME/log .
All the best.
Philip
Kaspar Fischer wrote:
Hi Philip,
Thanks for your response and the JAPE rules you attached. I have done
the following:
1. I removed the directory "context/default" and replaced it with the
original one. With this, I (hope to have) cleared the repository. Is
this correct, or is there an easier way to clear the document and
semantic repository?
2. In directory "plugins/ANNIE/resources/NE/" I added your file
long_name.jape.
3. I inserted a line "long_name" into file
"plugins/ANNIE/resources/NE/main.jape" after the line "name".
4. I added the documents from the previous mail using
"./bin/tools/toolPopulate.sh".
5. The Web UI still shows that only "Fischer" is indexed. ("Nicole
Meier" is not indexed.)
I noticed however a few things, and maybe these observations help to
tackle down the problem:
- My main.jape (see attached) looks very different from the one you
sent. Could I be using a very old version of KIM? Mine is called
kim-platform-2.3-RC1.
- I first tried with your main.jape (which contains many phases for
which there are no JAPE files in the directory, I think). I did not
get any errors relating to JAPE rules, neither in KIM's All.txt nor in
the Populator's output. Will JAPE/GATE errors be reported?
- I always get an error when I try to index a document "Wealth.txt"
which contains the single line "Martin Fischer lives in Switzerland
with his mother Elisabeth Fischer. He owns buildings worth $280,30
million.". The error is this:
INFO 8049[Thread-3] - /Users/hbf/downloads/kim-docs/Wealth.txt
INFO 8061[Thread-3] - Creating ...
INFO 8076[Thread-3] - Creating GATE document ...
INFO 8161[Thread-3] - Parsing features ...
INFO 8184[Thread-3] - Annotating ...
INFO 8384[Thread-3] - Storing ...
ERROR 8434[Thread-3] - Failed storing the document!
com.ontotext.kim.client.documentrepository.DocumentRepositoryException
at
com.ontotext.kim.lucene.LuceneDocumentRepositoryImpl.addDocument(LuceneDocumentRepositoryImpl.java:341)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
org.openrdf.util.rmirouting.ChannelIfaceImpl.invoke(ChannelIfaceImpl.java:514)
at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:294)
at sun.rmi.transport.Transport$1.run(Transport.java:153)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:149)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:466)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:707)
at java.lang.Thread.run(Thread.java:613)
INFO 8446[Thread-3] -
Completed: 25% ( 1 of 4 files processed )
[Remaining documents indexed without problems...]
I guess the last item is not related to the entity recognition
problem; I just mention it to be on the safe side.
Thank you very much for your help!
Kaspar
--
/*
* main.jape
*
* Copyright (c) 1998-2004, The University of Sheffield.
*
* This file is part of GATE (see http://gate.ac.uk/), and is free
* software, licenced under the GNU Library General Public License,
* Version 2, June 1991 (in the distribution as file licence.html,
* and also available at http://gate.ac.uk/gate/licence.html).
*
* Diana Maynard, 02 Aug 2001
*
* $Id: main.jape 9233 2007-11-23 13:01:52Z dgmaynard $
*/
MultiPhase: TestTheGrammars
Phases:
first
firstname
name
name_post
date_pre
date
reldate
number
address
url_pre
url
email
identifier
jobtitle
final
unknown
name_context
org_context
loc_context
clean
_______________________________________________
Kim-discussion mailing list
[email protected]
http://ontotext.com/mailman/listinfo/kim-discussion