Hi Iain, congratulations on having the patience to go that far! b Iain wrote: > > I’ve finally managed to load the Reuters Corpus into KIM (though I > have not yet tried to access it – wait for another mail on that). > > Whatever the problem I was having with the features XML files, it went > away of its own accord, which was nice. > > I ran this under Windows 2003 Server x64 with 64 bit Java, allocating > 12 GB (yes GB) to KIM and 8GB to SESAME. The KIM console reports it > used a tad over 3.1 GB of memory, but task manager reports about 7.5 > GB of VM used. > > The trick seems to be to split the files out into directories – I put > 1000 articles in each of 800 odd directories. Without this there seems > to be some issue with the Document Store tool initialising. Also, > directories with 1.5 million files in are just a tad unwieldy! > > For those interested, it took 3 days and 2 hours to complete the task, > processing a little over 3 per second. > > The statistics showed (times in seconds) > > Doc count 806791 > > Doc size 1,168,948K > > Create time 23,928 > > Parse Features 197,012 > > Annotation time 146,348 > > Store time 43,028 > > Index time 0 > > We appear to have 1.36 million ‘new instances’ with new-tem-triples > being 1,751,486 KB and 14 MB of trusted entities in the cache. I don’t > know what that means, but it certainly sounds impressive. > > Finally, the Index and store files are around 45GB. > > Now to get Tomcat 5.5 working so I can SEE what was in there! > > Iain > > --------------- > > Iain Downs (Microsoft MVP) > > Commercial Software Therapist > > E: [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> T:+44 (0) 1423 872988 > > W: www.idcl.co.uk <http://www.idcl.co.uk> > > http://mvp.support.microsoft.com > > ------------------------------------------------------------------------ > > _______________________________________________ > NOTE: Please REPLY TO ALL to ensure that your reply reaches all members of > this mailing list. > > KIM-discussion mailing list > [email protected] > http://ontotext.com/mailman/listinfo/kim-discussion_ontotext.com > > ------------------------------------------------------------------------ > > No virus found in this incoming message. > Checked by AVG Free Edition. > Version: 7.1.394 / Virus Database: 268.10.3/395 - Release Date: 7/21/2006 >
_______________________________________________ NOTE: Please REPLY TO ALL to ensure that your reply reaches all members of this mailing list. KIM-discussion mailing list [email protected] http://ontotext.com/mailman/listinfo/kim-discussion_ontotext.com
