Re: If I want to know how long it takes for data reading from the server side, where should I add logs?

[aealexsandrov]

Hi,

Will be better if you shared your code. However, I see that you used
persistence. So here are some reasons why you can see these drops:

1)You data region is less than full loaded data size stored on disk. In case
if read operation can't find the value in RAM then it will read it from the
disk. In the case of RANDOM_2_LRU it means that this value will be stored in
RAM and some other value will be evicted from the memory.

So when you will try to read this value you could see some performance drop.

2)Another problem could be related to initial loading of the values to
memory from the disk. I think that before starting of the benchmark you
should to warm up the Ignite by reading of the stored values for some time.

Could you please check can these moments be applicable to you?

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: If I want to know how long it takes for data reading from the server side, where should I add logs?

[Justin Ji]

Andrei - 

Thank for your reply, sorry for missing a sentence in this post, it should
be: there are almost 0.013% read requests take more than 20ms, and 0.0336%
more than 200ms. 
When the traffic is very large, the number of requests over 200ms will be
very large. So I want to know where the time is spent, client or server.

I think GC is not the root cause since most of GC time is less than 10ms and
GC is not frequently.

My benchmark code is simple, read and write cache with 8 threads, about 3300
getAll operations and 3300 putAll operations every second, and every
getAll/putAll contains three entries.

The ignite cluster has three nodes(8core 16g, 3g JVM heap).

*Here is my server configuration:*
































































*Here is my cache configuration:*

private static CacheConfiguration
getCacheConfiguration(IgniteConfiguration cfg) {
CacheConfiguration cacheCfg =
DefaultIgniteConfiguration.getCacheConfiguration(IgniteCacheKey.DATA_POINT.getCode());
cacheCfg.setBackups(1);
cacheCfg.setDataRegionName(Constants.THREE_GB_REGION);
   
cacheCfg.setCacheStoreFactory(FactoryBuilder.factoryOf(DataPointCacheStore.class));
cacheCfg.setWriteThrough(true);
cacheCfg.setWriteBehindEnabled(true);
cacheCfg.setWriteBehindFlushThreadCount(2);
//every 15 seconds
cacheCfg.setWriteBehindFlushFrequency(15 * 1000);
cacheCfg.setWriteBehindFlushSize(409600);
cacheCfg.setWriteBehindBatchSize(512);
cacheCfg.setStoreKeepBinary(true);
cacheCfg.setQueryParallelism(4);
//2M
cacheCfg.setRebalanceBatchSize(2 * 1024 * 1024);

List entities = getQueryEntities();
cacheCfg.setQueryEntities(entities);

cfg.setCacheConfiguration(cacheCfg);
return cacheCfg;
}

private static List getQueryEntities() {
List entities = Lists.newArrayList();

QueryEntity entity = new QueryEntity(String.class.getName(),
DpCache.class.getName());
entity.setTableName(IgniteTableKey.T_DATA_POINT.getCode());

LinkedHashMap map = new LinkedHashMap<>();
map.put("id", "java.lang.String");
map.put("gmtCreate", "java.lang.Long");
map.put("gmtModified", "java.lang.Long");
map.put("devId", "java.lang.String");
map.put("dpId", "java.lang.Integer");
map.put("code", "java.lang.String");
map.put("name", "java.lang.String");
map.put("customName", "java.lang.String");
map.put("mode", "java.lang.String");
map.put("type", "java.lang.String");
map.put("value", "java.lang.String");
map.put("rawValue", byte[].class.getName());
map.put("time", "java.lang.Long");
map.put("status", "java.lang.Boolean");
map.put("uuid", "java.lang.String");

entity.setFields(map);

QueryIndex devIdIdx = new QueryIndex("devId");
devIdIdx.setName("idx_devId");
QueryIndex dpIdIdx = new QueryIndex("dpId");
dpIdIdx.setName("idx_dpId");
List indexes = Lists.newArrayList(devIdIdx, dpIdIdx);
entity.setIndexes(indexes);

entities.add(entity);

return entities;
}



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: If I want to know how long it takes for data reading from the server side, where should I add logs?

[aealexsandrov]

Hi,

Can you share your benchmark and explain what results you are expecting? Are
you sure that there is no GC pauses at that moment?

Ignite already have its own benchmarks that used Yardstick:

https://apacheignite.readme.io/docs/perfomance-benchmarking

Are you use them? If not possible you can take a look.

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

If I want to know how long it takes for data reading from the server side, where should I add logs?

[Justin Ji]

I found there are almost 0.013% read requests take more than 20ms when I test
the performance of ignite,  so I want to know where the time takes, client
or server? I do not know where to add logs clearly, can anyone tell me?

Is
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManager#read(org.apache.ignite.internal.processors.cache.GridCacheContext,
org.apache.ignite.internal.processors.cache.KeyCacheObject)
a good place?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/