Hi Tobin,

If you have these tickets as XML files it is better to simply load/create 
them directly using the method from CorporaAPI
    createDocument(java.net.URL url, java.lang.String encoding)
instead of creating them through the method
    createDocument(java.lang.String content, java.lang.String encoding)
as the first tries to guess the format and perfomes some clever parsing 
underneath and the second is assumed to take clear unformatted and untagged 
text and causes problem when this is not so.

If you insist on creating documents as strings, it becomes more complex, you 
should use some "low-level" code (and you should be familiar with GATE) :

    FeatureMap fm = gate.Factory.newFeatureMap();
    fm.put(gate.Document.DOCUMENT_STRING_CONTENT_PARAMETER_NAME,content);
    fm.put(gate.Document.DOCUMENT_ENCODING_PARAMETER_NAME, "UTF-8");
    fm.put(gate.Document.DOCUMENT_MARKUP_AWARE_PARAMETER_NAME, 
Boolean.FALSE);
    DocumentImpl gateDoc = (DocumentImpl) 
Factory.createResource("gate.corpora.DocumentImpl", fm);
    KIMDocumentImpl doc = corpApi.createDocument(gateDoc);

Here you create a KIMDocument through a GATE document. And you create the 
GATE document using the GATE factory explicitly stating that you want the 
document to be assumed "plain", i.e. to forget about markup and handle it as 
a part of the text when creating a document.

However, if your goal is to index in Lucene the tags themselves this depends 
on the use case. Since on annotation step these tags will be transformed 
into annotations in a separate AnnotationSet called OriginalMarkup and will 
be removed from the text, when storage step comes the document is already 
changed. If you are not annotating the documents but just creating them and 
storing them this may work.

Tell me what you want to achieve and try one of the above option.
Regards,
Ilian

----- Original Message ----- 
From: "Tobin Hupsel" <[EMAIL PROTECTED]>
To: "Ilian Kitchukov" <[EMAIL PROTECTED]>
Sent: 01 February 2007, Thursday 14:44
Subject: RE: [KIM-discussion] KIM problem


Hi Ilian,

Still don't know why netbeans doesn't work, but I have another problem.
If I try to index and store a helpdesk log text (as a string) with xml
tags in it, it gives an error:
java.rmi.RemoteException: ; nested exception is:
gate.creole.ResourceInstantiationException: Couldn't unpack markup in
document!

It seems KIM is trying to read the xml code and not just indexing it as
a string. Is there a way to let KIM just index the String? Lucene can
handle this string fine.
I've got 15.000 helpdesk-tickets, so checking if xml tags (as text) are
in the documents and removing them, would take a very long time.

Example of string is:

8/12/2005 9:20:45 AM bultmanj
gegevens ontvangen.

8/12/2005 1:45:07 PM bultmanj
Heren,

Nu spoedig actie gewenst.
Het probleem wordt van kwaad tot erger.
We kunnen nu ineens niet meer bestellen.
Gaarne deze log file bekijken en aangeven waarom er niet besteld wordt.
(See attached file: xml)

Groet,

Hans

8/12/2005 2:08:52 PM bultmanj
louis heeft ingebeld. xml wordt niet volledig aangemaakt.

Microsoft   Windows Script Host Version 5.6
Copyright   Microsoft Corporation 1996-2001. All rights reserved.

Status: 200
----------- XML CONTENT -----------

----------- XML CONTENT -----------
----------- XML RESPONSE -----------
<?xml version="1.0" encoding="ISO-8859-1"?>   <----- here it goes wrong
<ISP version="1.0" tpTransactionId="error"
timestamp="2005-08-12T12:17:44">
      <ZTIErrorCode>1</ZTIErrorCode>
      <ZTIError



content is leeg.


als jet het met de hand doet in dos dan gaat het goed.
we hebben gezien dat de Filter API RPC Timeout (seconds) op 0 staat.
default is 60. 0 is minimaal.
Voicemail van hans ingesproken om mij terug te bellen. wil namelijk deze
weer op 60 zetten. wellicht dat het hier aan licht.
8/12/2005 2:33:39 PM bultmanj
hans heeft time out aangepast. na 5en wordt server gestopt en gestart.


Regards,

Tobin


-----Oorspronkelijk bericht-----
Van: Ilian Kitchukov [mailto:[EMAIL PROTECTED]
Verzonden: woensdag 31 januari 2007 12:45
Aan: [EMAIL PROTECTED]
Onderwerp: Re: [KIM-discussion] KIM problem

Hi Tobin,

I'm glad to hear that you've found the problem and a way to workaround
it.
Please, if you find out further details about it, send them to us, so we
can
provide help to other people who may fall in a situation similar to
yours.
And don't hesitate to contact us again if another problem arises.

Best,
Ilian

----- Original Message ----- 
From: <[EMAIL PROTECTED]>
To: "Ilian Kitchukov" <[EMAIL PROTECTED]>
Sent: 31 January 2007, Wednesday 13:20
Subject: Re: [KIM-discussion] KIM problem


> Hi Ilian,
>
> Thanks for your help. Well, i use netbeans 5.0 and it has a build-in
> Tomcat.
> So i tried to deploy the code on a standalone version of tomcat and
guess
> what, it works! The first time i run the code i got a:
> java.io.IOException:
> Lock obtain timed out: [EMAIL 
> PROTECTED]:\DOCUME~1\hupselt\LOCALS~1\Temp\lucene-
> bb3757ddb0fb202f73ab6d4c570dd19-write.loc error. Something to do with
the
> index being locked, but after a restart it works fine.
> There must be some error with the enviroment of netbeans. It's a shame

> because
> i like the "deploy" button in netbeans, saves time :)
> Maybe i figure out the problem later. In the meantime again thanks for

> your
> help.
>
> regards,
>
> Tobin
>
> Quoting Ilian Kitchukov <[EMAIL PROTECTED]>:
>
>> Hi again Tobin,
>>
>> I'm sorry to say that but I have some good news that unfortunately
may
>> not
>> be so good for you. I tried your both scenarios - using directly JSP
code
>> and using a Java handler - and the BOTH worked fine for me, no
exceptions
>> and document really stored by IndexAndPersistAPI . Here, I am
attaching
>> the
>> files that I used, but actually they aren't much different from what
>> you've
>> sent me.
>>
>> =====================================
>> (1) JSP version - test-kim.jsp
>>
>> page output:
>>
>> KIM- semantic search
>>
>> Connected to Host Server version: KIM Platform: [1.7.13.8] KIM
Server:
>> [1.7.13.8] KB Version: [The Current KIM KB does not have a Version
>> specified.]
>>
>> CorporaAPI and IndexAndPersistAPI obtained successfully.
>> All documents in store: 3
>> LastPersistenceID=3
>> Created document: Test
>>
>> This is a Test
>>
>> Document indexed. ID=Test
>>
>> =====================================
>> (2) servlet version - handler.java, kim.java, test.jsp
>>
>> page output:
>>
>> Kim Search
>> Document indexed. naam=GATE document_0001C
>>
>> =====================================
>> Yes, I used a version which is slightly newer than yours, but the
>> difference
>>
>> between them is not relevant to this code, WebUI or JSP. This makes
me
>> think
>>
>> there is some difference in the environment - some Tomcat settings,
>> permissions, classpaths, etc. But I cannot think of a specific
reason.
>> What
>> we have as facts are:
>> - your JSP obviously compiles - which means you have all references
and
>> classpaths to KIM libs OK
>> - the JSP succeeds in connecting to the KIM Server and obtaining APIs
and
>> getting document counts - which means the connection is OK
>> - the JSP succeeds in creating doc but fails in storing it - this is
the
>> strange thing I cannot tell means what ...
>> I just cannot reproduce your problem, everything was running at once,
and
>> I
>> guess I'm not of much help. Really sorry about that :(
>>
>> May be if you tell me more about the environment you're running KIM
and
>> your
>>
>> application in I could be more helpfull.
>> - What is your OS? If Linux - what users do you run KIM and Tomcat
with?
>> - Do you run Tomcat from start/stop scripts or as a service(Windows
>> only)?
>> - Did you check all the possible problems form
>> http://www.ontotext.com/kim/doc/sys-doc/faq-howto-bugs/howto-
> procedures.html#q8?
>>
>> I am really out of ideas :(
>> We're now going on a blind search :(
>>
>> Ilian
>>
>> ----- Original Message ----- 
>> From: "Tobin Hupsel" <[EMAIL PROTECTED]>
>> To: "Ilian Kitchukov" <[EMAIL PROTECTED]>
>> Cc: <[EMAIL PROTECTED]>
>> Sent: 29 January 2007, Monday 13:13
>> Subject: RE: [KIM-discussion] KIM problem
>>
>>
>> Hi Ilian,
>>
>> I rewrote the code in a servlet and Java class. When I run the java
>> class it works fine. But when I call the java class method handleDoc
in
>> the servlet it gives the same error.
>>
>> Java class is:
>>
>> package com.ontotext.kim.servlet;
>>
>> import com.ontotext.kim.client.*;
>> import com.ontotext.kim.client.corpora.*;
>> import com.ontotext.kim.client.index.*;
>>
>> import gate.util.*;
>> import java.net.*;
>> import java.util.*;
>>
>> public class handler {
>>
>>   public handler() {
>>
>>   }
>>
>>   public static void main(String[] args) {
>>         try{
>>         System.out.println("Kim Search");
>>
>>         String name = handleDoc();
>>         System.out.println("Document indexed. naam="+ name +"<br>");
>>
>>         }catch (Exception e){
>>             System.out.println(e);
>>         }
>>   }
>>
>> public static String handleDoc() throws Exception{
>>
>> String INDEX_AND_PERSIST_URL =
>>
"lucene:/d:/kim-platform-1.7.12.15/context/default/populated/index-and-s
>> tore";
>>
>> String content ="This is again a test";
>> KIMService serviceKim = GetService.from();
>> CorporaAPI apiCorpora = serviceKim.getCorporaAPI();
>> IndexAndPersistAPI apiIndexAndPersist =
>> serviceKim.getIndexAndPersistAPI(INDEX_AND_PERSIST_URL);
>>
>> System.out.println("LastPersistenceID=" +
>> apiIndexAndPersist.getLastPersistenceID()+"<br>");
>>
>> KIMDocument kdoc = apiCorpora.createDocument(content,"UTF-8");
//create
>> doc    KIMDocument newDoc =apiIndexAndPersist.storeDoc(kdoc);
>> //store doc
>>
>> System.out.println("LastPersistenceID=" +
>> apiIndexAndPersist.getLastPersistenceID()+"<br>");
>>
>> return newDoc.getName();
>> }
>>
>> }
>>
>> output:
>>
>> Kim Search
>> LastPersistenceID=19<br>
>> LastPersistenceID=20<br>
>> Document indexed. naam=GATE document_00044<br>
>>
>> The java servlet code:
>>
>> package com.ontotext.kim.servlet;
>>
>> import java.io.*;
>> import java.net.*;
>>
>> import javax.servlet.*;
>> import javax.servlet.http.*;
>>
>> import com.ontotext.kim.client.*;
>> import com.ontotext.kim.client.corpora.*;
>> import com.ontotext.kim.client.index.*;
>> import com.ontotext.kim.servlet.*;
>>
>> import gate.util.*;
>>
>> public class kim extends HttpServlet {
>>
>>
>>     protected void processRequest(HttpServletRequest request,
>> HttpServletResponse response)
>>     throws ServletException, IOException {
>>         response.setContentType("text/html;charset=UTF-8");
>>         PrintWriter out = response.getWriter();
>>
>>         try{
>>         out.println("<html>");
>>         out.println("<head>");
>>         out.println("<title>Servlet kim</title>");
>>         out.println("</head>");
>>         out.println("<body>");
>>         out.println("<h1> Kim Search</h1>");
>>
>>         String name =handler.handleDoc();
>>         out.println("Document indexed. naam="+ name +"<br>");
>>
>>         }catch (Exception e){
>>             out.println(e);
>>         }
>>
>>         out.println("</body>");
>>         out.println("</html>");
>>         out.close();
>>     }
>>
>>     protected void doGet(HttpServletRequest request,
HttpServletResponse
>> response)
>>     throws ServletException, IOException {
>>         processRequest(request, response);
>>     }
>>
>>     protected void doPost(HttpServletRequest request,
>> HttpServletResponse response)
>>     throws ServletException, IOException {
>>         processRequest(request, response);
>>     }
>>
>>     public String getServletInfo() {
>>         return "Short description";
>>     }
>> }
>>
>> output:
>>
>> Kim Search
>>
>> java.lang.reflect.UndeclaredThrowableException
>>
>>
>> I can't figure out the problem here. I just call the method and get a
>> string back, no complex object. Could you run the same code and look
if
>> you get the same error? This would help me out a lot! Thanks
>>
>> Regards,
>>
>> Tobin Hupsel
>>
>> -----Oorspronkelijk bericht-----
>> Van: [EMAIL PROTECTED]
>> [mailto:[EMAIL PROTECTED] Namens Ilian Kitchukov
>> Verzonden: dinsdag 23 januari 2007 10:50
>> Aan: [EMAIL PROTECTED]
>> CC: [email protected]
>> Onderwerp: Re: [KIM-discussion] KIM problem
>>
>> Hi Tobin,
>>
>> This result makes me think the call doesn't reach the KIM Server at
all.
>> Our
>> KIM WebUI is also JSP based, but we actually never call KIM's APIs
>> directly
>> from the JSPs and rather use Java classes as handlers underneath.
This
>> at
>> least provides some sensible exceptions if something goes wrong. So I
>> cannot
>> tell it is the JSP-KIM communication to blame but it definitely
prevents
>> us
>> from seeing everything.
>>
>> I suppose you're running everything on the same machine? And your
Tomcat
>> and
>> KIM use the same JDK (but run in separate JVMs of course)? You say
the
>> exmaple is running OK and you obviously succeed in connecting to your
>> locally running KIM Server via JSP. So my guess is that the problem
>> arises
>> only when you call a method that passes or returns a complex object
>> (e.g.
>> KIMDocument). And the reason for this could be one of the described
on
>> this
>> page (you may also refer your local copy of the sys-doc) :
>>
http://www.ontotext.com/kim/doc/sys-doc/faq-howto-bugs/howto-procedures.
>> html#q8
>>
>> If this doesn't help you, as a last resort, I could try to reproduce
the
>>
>> problem at my side and see if some sensible exception, explanation
and
>> solution will appear. I just have quite a bunch of tasks to finish
and I
>>
>> cannot promise you I'll deal with it immediately.
>>
>> Regards,
>> Ilian
>>
>> ----- Original Message ----- 
>> From: <[EMAIL PROTECTED]>
>> To: "Ilian Kitchukov" <[EMAIL PROTECTED]>
>> Cc: <[email protected]>
>> Sent: 23 January 2007, Tuesday 10:42
>> Subject: Re: [KIM-discussion] KIM problem
>>
>>
>>
>>
>> __________ NOD32 2009 (20070126) Information __________
>>
>> This message was checked by NOD32 antivirus system.
>> http://www.eset.com
>>
>>
>
>
>
> __________ NOD32 2009 (20070126) Information __________
>
> This message was checked by NOD32 antivirus system.
> http://www.eset.com
>
>



__________ NOD32 2025 (20070201) Information __________

This message was checked by NOD32 antivirus system.
http://www.eset.com



_______________________________________________
NOTE: Please REPLY TO ALL to ensure that your reply reaches all members of this 
mailing list.

KIM-discussion mailing list
[email protected]
http://ontotext.com/mailman/listinfo/kim-discussion_ontotext.com

Reply via email to