SQL query existing cache data

2018-02-25 Thread VT 
Hi all, 

Can I do a SQL query over existing cache data? The cache was created and
data was loaded ahead of time, without using java specific annotations or
QueryEntity configuration. I notice that "create table" DDL will create a
new cache, which is undesirable for my use case. What I want is, given a
cache with "structured" data, given a way to specify/define SQL schema
dynamically, I can use SQL to query data. 

However, it seems that currently cache data has to be "declared" to use SQL
AHEAD of time BEFORE data is loaded in order to use SQL. Is this correct?
Can this restriction be circumvented?

Many thanks!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: SSD writes get slower and slower

2018-02-25 Thread VT 
Hi Stan, 

The setting is very simple and straightforward, as follows.

DataStorageConfiguration dsCfg = new DataStorageConfiguration();
dsCfg.setWalMode(WALMode.DEFAULT);
dsCfg.setPageSize(4 * 1024);
dsCfg.setWriteThrottlingEnabled(true);

..

DataRegionConfiguration regionCfg1 = new DataRegionConfiguration();
regionCfg1.setName("region_1");
regionCfg1.setInitialSize(2L * 1024 * 1024 * 1024);
regionCfg1.setMaxSize(10L * 1024 * 1024 * 1024);
regionCfg1.setCheckpointPageBufferSize(2L * 1024 * 1024 * 1024);
regionCfg1.setPageEvictionMode(DataPageEvictionMode.RANDOM_2_LRU);
regionCfg1.setPersistenceEnabled(true);
regionCfg1.setSwapPath("/data/ignite2/swap/"); //SSD

...

cacheCfg.setDataRegionName("region_1");
cacheCfg.setName(CacheName);
cacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_ASYNC);
cacheCfg.setCacheMode(CacheMode.PARTITIONED);
cacheCfg.setCopyOnRead(false);

...

I used DataStreammer very simply, like the following.

IgniteDataStreamer stmr.addData(key, value);

I have tried multiple settings such as perNodeBufferSize,
perNodeParallelOperations. Still very slow. Please help. Thanks!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

SSD writes get slower and slower

2018-02-23 Thread VT 
Hello, 

I am trying to test use SSD. But as more data are written, the write
performance drops significantly. Is there something I can do to solve this
problem? Or are there any known issues to this problem that might be fixed
in the upcoming versions? The test was done on a single node, locally. The
total data size is small, not enough to fill half of SSD.

Thanks!

For example, 

Loaded 1 keys 239ms.
Loaded 2 keys 122ms.
Loaded 3 keys 141ms.
Loaded 4 keys 110ms.

...

Loaded 842 keys 8068ms.
Loaded 843 keys 201ms.
Loaded 844 keys 187ms.
Loaded 845 keys 162ms.
Loaded 846 keys 182ms.
Loaded 847 keys 164ms.
Loaded 848 keys 171ms.
Loaded 849 keys 8475ms.

...

Loaded 8437 keys 5684ms.
Loaded 8438 keys 4329ms.
Loaded 8439 keys 5191ms.
Loaded 8440 keys 7352ms.
Loaded 8441 keys 11171ms.

...

Loaded 9994 keys 13352ms.
Loaded 9995 keys 28771ms.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/