Andy, thanks for the tips.
I decided to modify the producer to make n-triples. It works fine but
initial tests do not run as fast as I had hoped. When making a new TDB,
it averaged 3000 triples/sec, but I was seeing higher rates reading from
ttl files. Still need to do more testing on a higher-powered machine.
Regards,
--Paul
On Thu, 2013-05-16 at 12:50 +0100, Andy Seaborne wrote:
> On 16/05/13 04:21, Paul Tyson wrote:
> > Hi,
> >
> > I'm trying to use the TDBLoader api to stream turtle to the bulk loader
> > to create a new TDB repository.
> >
> > I suspect none of the TDBLoader.load*() methods accept turtle input. I'm
> > using version 2.10.1.
>
> When the URL is given, they should pick up the language. If not, please
> can we have a complete minimal example.
>
> >
> > This sort of code produces an immediate RIOT exception:
> >
> > InputStream is = ...;
> > Dataset ds = TDBFactory.createDataset("/tdb");
> > TDBLoader ldr = new TDBLoader();
> > ldr.loadGraph((GraphTDB) ds.getDefaultModel().getGraph(),is);
>
> When reading from a stream, it does not know the syntax and, yes,
> N-Triples expected.
>
> > (I am working from memory so I might not have the syntax quite right.)
> >
> > It throws a RIOT exception on the first triple it sees, which is like:
> >
> > <iri1> a <foo>; <p1> "str";...
> >
> > Says it expected IRI instead of keyword "a".
> >
> > Any pointers on how to stream turtle to TDBLoader? I do have control
> > over the RDF production, so could produce straight triples. But until
> > now I've been writing the RDF to ttl files, then loading with the
> > tdbloader command line.
> >
> > Thanks,
> > --Paul
>
> When streaming, you can feed through the turtle script to convert Turtle
> to N-Triples
>
> ... files ... | turtle | tdbloader -- -
>
> At scale, the extra cross-processes costs don't show any effect.
> Parsing is not (usually) the dominate cost at scale.
>
>
> If you are incrementally loading graphs into an existing store,
> TDBLoader does not have any advantages. It's magic is when the store is
> empty.
>
> RDFDataMgr.read(in, ds.getDefaultModel(), Lang.TURTLE) ;
>
> should be fine.
>
> Andy
>
>
>
>
>
