Re: 10GB file loading on Fuseki

[Osma Suominen]

Hi Reihan,

I hope you already made some progress with help from Stian and others on 
this list.


These files together constitute the TDB database, i.e. the whole 
directory is the TDB. Then you just need to configure Fuseki to use that 
database. You don't need to load them anymore, as they are already in a 
form that Fuseki can use directly. When you load RDF files into Fuseki, 
it will create a TDB itself, but that is not needed when you have done 
the loading externally using tdbloader[2].


The simplest way is to use the --loc command line argument. For example, 
if your TDB directory is named "tdb", you can run Fuseki like this:


fuseki-server --loc tdb /ds

Then Fuseki will serve data from that TDB.

-Osma

20.01.2017, 15:54, Reihaneh Amini kirjoitti:

Hi Osma,

Could you please help me more?
I load the data into an directory by TDB modeler and now I have 20-30 files 
with .dat and .idn prefix.
Could you please help me out ? how can I load them into fuseki now?




--
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 26 (Kaikukatu 4)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.suomi...@helsinki.fi
http://www.nationallibrary.fi

Re: Jena 3.2.0-SNAPSHOT Node.ANY serialization causes StackOverFlow when called from Kryo JavaSerialization.

[Dick Murray]

Hi. Thanks for the detail. I think the problem is in Kryo as it seems to be
using FieldSerializaer no matter what you tell it. Thus Node_URI is
probably working by accident!

I'll get to the bottom of it and update for prosperity...

On 22 January 2017 at 11:02, Andy Seaborne  wrote:

> Hi,
>
> Test TestSerializable.serialize_node_04 serializes and deserializes a
> Node.ANY.
>
> All serialization is handled via Node.writeReplace() indirecting to SNode
> and SNode.readResolve() to deserialize.
>
> The provided serializer is Jena's RDF Thrift.  ThriftConvert.toThrift
> handles the conversion for Thrift.
>
> There is nothing in Node.ANY.  Node.writeReplace points to
> oaj.system.SerializerRDF.
>
> Does JavaSerializer handle writeReplace?  Default serialization of
> Node_URI may work by accident because it has a field.
>
> Initialization is in org.apache.jena.system.SerializerRDF, which is
> called from InitRIOT which is called by by system initialization based on
> ServiceLoader.
>
> Andy
>
>
> On 20/01/17 18:35, Dick Murray wrote:
>
>> Whilst this issue is reported and possibly caused by Kryo I think it's my
>> understanding of how Jena is or is not serializing...
>>
>> I'm using Jena 3.2.0-SNAPSHOT and Kryo(Net) to serialize Jena nodes but
>> Kryo baulks when asked to handle a (the) Node_ANY;
>>
>> Exception in thread "Server" java.lang.StackOverflowError
>> at
>> com.esotericsoftware.kryo.io.ByteBufferOutput.writeVarInt(By
>> teBufferOutput.java:323)
>> at
>> com.esotericsoftware.kryo.util.DefaultClassResolver.writeCla
>> ss(DefaultClassResolver.java:102)
>> at com.esotericsoftware.kryo.Kryo.writeClass(Kryo.java:517)
>> at
>> com.esotericsoftware.kryo.serializers.ObjectField.write(Obje
>> ctField.java:76)
>> at
>> com.esotericsoftware.kryo.serializers.FieldSerializer.write(
>> FieldSerializer.java:518)
>> at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552)
>> at
>> com.esotericsoftware.kryo.serializers.ObjectField.write(Obje
>> ctField.java:80)
>> at
>> com.esotericsoftware.kryo.serializers.FieldSerializer.write(
>> FieldSerializer.java:518)
>> at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552)
>> at
>> com.esotericsoftware.kryo.serializers.ObjectField.write(Obje
>> ctField.java:80)
>> at
>> com.esotericsoftware.kryo.serializers.FieldSerializer.write(
>> FieldSerializer.java:518)
>>
>> Basically the server side dies horribly!
>>
>> I've looked at SerializerRDF and Node and Node.ANY (Node_Fluid) and I
>> cannot see where it goes wrong.
>>
>> Node_URI works without issue and a breakpoint on protected Object
>> writeReplace() throws ObjectStreamException {...} will break. However
>> Node_ANY never triggers a break...
>>
>> Does Node_ANY have the code to actually serialize itself? Should I be
>> overriding something and returning the Node_ANY?
>>
>> NB: I need to register the Node_URI class a JavaSerialization because Kryo
>> was trying to FieldSerialize and that was not working!
>>
>> Just the shared Network class as follows;
>>
>> package org.twocows.jena.mosaic.kryo;
>>
>> import java.util.Iterator;
>>
>> NB: I need to register the Node_URI class a JavaSerialization because Kryo
>> was trying to FieldSerialize and that was not working!
>>
>> import org.apache.jena.graph.Node;
>> import org.apache.jena.graph.Node_ANY;
>> import org.apache.jena.graph.Node_URI;
>> import org.twocows.jena.mosaic.MosaicDatasetGraph;
>>
>> import com.esotericsoftware.kryo.Kryo;
>> import com.esotericsoftware.kryo.Serializer;
>> import com.esotericsoftware.kryo.io.Input;
>> import com.esotericsoftware.kryo.io.Output;
>> import com.esotericsoftware.kryo.serializers.JavaSerializer;
>> import com.esotericsoftware.kryonet.EndPoint;
>> import com.esotericsoftware.kryonet.rmi.ObjectSpace;
>> import com.github.jsonldjava.core.RDFDataset.Quad;
>>
>> public class Network {
>> public static final int port = 1972;
>>
>> public static final int MOSAIC_DATASET_GRAPH = 1;
>> // This registers objects that are going to be sent over the network.
>> static public void register (final EndPoint endPoint) {
>> Kryo kryo = endPoint.getKryo();
>> // This must be called in order to use ObjectSpaces.
>> ObjectSpace.registerClasses(kryo);
>> // The interfaces that will be used as remote objects must be registered.
>> kryo.register(MosaicDatasetGraph.class);
>> // The classes of all method parameters and return values
>> // for remote objects must also be registered.
>> kryo.register(ExceptionInInitializerError.class);
>> kryo.register(UnsupportedOperationException.class);
>> kryo.register(String.class);
>> kryo.register(Node.class, new JavaSerializer());
>> kryo.register(Node_URI.class, new JavaSerializer());
>> kryo.register(Node_ANY.class, new JavaSerializer());
>> kryo.register(Quad.class);
>> kryo.register(Iterator.class);
>> }
>> }
>>
>>