Hi group,

I had some discussions with Andy before outside the list. I have a
pipeline that creates a bunch of new triples via CONSTRUCT or INSERT
queries. The patterns are in my opinion pretty easy to stream, if I do
not do an insert but just a count, the query executes fine in TDB. As
soon as I do constructs on top of that it runs forever, takes CPU like
crazy and eventually runs out of memory.

Now my guess is that it tries to create everything in-memory which is
probably simply too much. From some googling I found:

https://issues.apache.org/jira/browse/JENA-329

https://issues.apache.org/jira/browse/JENA-205

if I get that correctly there is streaming for this kind of queries but
I can't figure out if I can use that in tdb cli interfaces like tdbupdate.

Is there a way to use that from cli? My pipelines are run in batches so
I don't really care about speed, I just care that the queries finish.

regards

Adrian

Reply via email to