Hi Iain,
congratulations on having the patience to go that far!
b

Iain wrote:
>
> I’ve finally managed to load the Reuters Corpus into KIM (though I 
> have not yet tried to access it – wait for another mail on that).
>
> Whatever the problem I was having with the features XML files, it went 
> away of its own accord, which was nice.
>
> I ran this under Windows 2003 Server x64 with 64 bit Java, allocating 
> 12 GB (yes GB) to KIM and 8GB to SESAME. The KIM console reports it 
> used a tad over 3.1 GB of memory, but task manager reports about 7.5 
> GB of VM used.
>
> The trick seems to be to split the files out into directories – I put 
> 1000 articles in each of 800 odd directories. Without this there seems 
> to be some issue with the Document Store tool initialising. Also, 
> directories with 1.5 million files in are just a tad unwieldy!
>
> For those interested, it took 3 days and 2 hours to complete the task, 
> processing a little over 3 per second.
>
> The statistics showed (times in seconds)
>
> Doc count 806791
>
> Doc size 1,168,948K
>
> Create time 23,928
>
> Parse Features 197,012
>
> Annotation time 146,348
>
> Store time 43,028
>
> Index time 0
>
> We appear to have 1.36 million ‘new instances’ with new-tem-triples 
> being 1,751,486 KB and 14 MB of trusted entities in the cache. I don’t 
> know what that means, but it certainly sounds impressive.
>
> Finally, the Index and store files are around 45GB.
>
> Now to get Tomcat 5.5 working so I can SEE what was in there!
>
> Iain
>
> ---------------
>
> Iain Downs (Microsoft MVP)
>
> Commercial Software Therapist
>
> E: [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> T:+44 (0) 1423 872988
>
> W: www.idcl.co.uk <http://www.idcl.co.uk>
>
> http://mvp.support.microsoft.com
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> NOTE: Please REPLY TO ALL to ensure that your reply reaches all members of 
> this mailing list.
>
> KIM-discussion mailing list
> [email protected]
> http://ontotext.com/mailman/listinfo/kim-discussion_ontotext.com
>   
> ------------------------------------------------------------------------
>
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.1.394 / Virus Database: 268.10.3/395 - Release Date: 7/21/2006
>   

_______________________________________________
NOTE: Please REPLY TO ALL to ensure that your reply reaches all members of this 
mailing list.

KIM-discussion mailing list
[email protected]
http://ontotext.com/mailman/listinfo/kim-discussion_ontotext.com

Reply via email to