42 million quads is nothing like so many that either TDB version should have 
any problem doing normal indexing (assuming very little in the way of 
hardware-- I ingest datasets like that on my laptop all the time).

Do you have some extraordinary hardware limitations?

Adam

> On Apr 14, 2018, at 11:42 AM, Andy Seaborne <a...@apache.org> wrote:
> 
> Hi Samita,
> 
> Firstly - as Adam points out - if theer are no indexes then access to the 
> data will be very slow.  For a GSPO index,  that means squeries must be 
> "GRAPH <uri> { ... }" and probably "GRAPH <uri> { <fixedSubject>.. }".
> 
> GSPO means lookup by G then S within those G and the same for P then O.
> 
> I looked at the data and it seems to be able 42 million quads.
> 
> Using TDB1 (the loader is faster at this scale currently) is likely to be a 
> better choice.
> 
> Looking at StoreParams in TDB2:
> 
> The code below creates the database at TDB2Factory.connectDataset so any 
> StoreParams after that do not affect indexing.
> 
> I tried to make it work in the release but the code ignores provided 
> StoreParams - sorry.  Even if it did work, it hits a test to make sure there 
> are basic indexing (Adam's point).
> 
>    Andy
> 
> 
> On 13/04/18 13:42, Samita Bai  / PhD CS Scholar @ City Campus wrote:
>> I wrote the following code to build only one type of triple and quad index 
>> but it is still creating all indexes 😞
>> package ldbqPack;
>> import org.apache.jena.query.Dataset;
>> import org.apache.jena.tdb2.TDB2Factory;
>> import org.apache.jena.tdb2.setup.StoreParams;
>> import org.apache.jena.tdb2.sys.DatabaseConnection;
>> import org.apache.jena.dboe.base.block.FileMode;
>> import org.apache.jena.dboe.base.file.Location;
>> import org.apache.jena.tdb2.setup.StoreParamsFactory;
>> public class StrPrms {
>> static String[] tindexes= {"SPO"};
>> static String[] qindexes= {"GSPO"};
>> static String[] pindexes= {"GPU"};
>> static final StoreParams pApp = StoreParams.builder()
>>        .blockSize(12)              // Not dynamic
>>        .nodeMissCacheSize(12)      // Dynamic
>>        .build();
>>    static final StoreParams pLoc = StoreParams.builder()
>>        .blockSize(0)
>>        .nodeMissCacheSize(0).build();
>>    static final StoreParams pDft = StoreParams.builder()
>>     .fileMode(FileMode.mapped)
>>     .blockSize(8192)
>>     .blockReadCacheSize(5000)
>>     .blockWriteCacheSize(1000)
>>     .node2NodeIdCacheSize(200000)
>>     .nodeId2NodeCacheSize(750000)
>>     .nodeMissCacheSize(1000)
>>     .nodeTableBaseName("nodes")
>>     .primaryIndexTriples("SPO")
>>     .tripleIndexes(tindexes)
>>     .primaryIndexQuads("GSPO")
>>     .quadIndexes(qindexes)
>>     .prefixTableBaseName("prefixes")
>>     .primaryIndexPrefix("GPU")
>>     .prefixIndexes(pindexes)
>>     .build();
>> public static void main(String[] args) {
>> // TODO Auto-generated method stub
>> final String DATASET_DIR_NAME = "DyLDO100";
>>         Dataset dataset = TDB2Factory.connectDataset ( DATASET_DIR_NAME );
>>         Location location = Location.create(DATASET_DIR_NAME);
>>         StoreParams custom_params = 
>> StoreParamsFactory.decideStoreParams(location, true, pApp, pLoc,  pDft);
>>        DatabaseConnection.connectCreate(location, custom_params);
>>        StoreParams params = StoreParams.getSmallStoreParams();
>>         System.out.println(params);
>> }
>> }
>> Please help.
>> Regards,
>> Samita Bai
>> ________________________________
>> P : Please consider the environment before printing this e-mail
>> ________________________________
>> CONFIDENTIALITY / DISCLAIMER NOTICE: This e-mail and any attachments may 
>> contain confidential and privileged information. If you are not the intended 
>> recipient, please notify the sender immediately by return e-mail, delete 
>> this e-mail and destroy any copies. Any dissemination or use of this 
>> information by a person other than the intended recipient is unauthorized 
>> and may be illegal.
>> ________________________________

Reply via email to