Activating Cluster taking too long

2017-08-10 Thread iostream
Hi! I am experimenting with v2.1 persistence store enabled. 1. Created 8 caches and pumped data into them. 2. Restarted the ignite cluster. 3. Waited for all server nodes to join the cluster. 4. called Ignite.active(true); I observed the cluster activation time is more than 1 hour with the

Re: Persistent store and eviction policy.

2017-08-10 Thread Alexey Kukushkin
Hi, 1. Persistence and Data Eviction are alternative options to handle out-of-memory scenarios.Persistence makes Ignite fill all available RAM and move the oldest page to the “disk” part of the cache when there is not enough memory. Data eviction policy makes Ignite to completely remove some

Re: Ignite fails to allocate more memory then initially allocated when maxSize property provided in MemoryPolicyConfiguration

2017-08-10 Thread afedotov
Hi, Provided configuration works as expected on Ignite version 2.1. On my side, in this configuration, I got 3 memory segment allocations. Have you tried requesting cache.size() after populating the cache? Please provide the full configuration files so as I could check them. Kind regards, Alex.

Re: EOFException in Hadoop Datanode when writing to Ignite

2017-08-10 Thread Mikhail
Hi Rodrigo, I'm not sure how Flink works, but to write to IGFS you need to use special implantation of HDFS: fs.igfs.impl org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem fs.AbstractFileSystem.igfs.impl org.apache.ignite.hadoop.fs.v2.IgniteHadoopFileSystem

Re: EOFException in Hadoop Datanode when writing to Ignite

2017-08-10 Thread Mikhail
Hi Rodrigo, I'm not sure how Flink works, but to write to IGFS you need to use special implantation of HDFS: fs.igfs.impl org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem fs.AbstractFileSystem.igfs.impl org.apache.ignite.hadoop.fs.v2.IgniteHadoopFileSystem

Visor: cache counts are twice of what is expected

2017-08-10 Thread Roger Fischer (CW)
Hello, the cache counts that are shown in Visor seem to be twice the number that is expected. I am using ver. 2.1.0#20170720-sha1:a6ca5c8a, with native persistence. For a replicated cache, with 363 objects loaded (select count(*) returns 363): Nodes for: FabricCache(@c0)

Re: Setting custom Log location - log4j

2017-08-10 Thread vkulichenko
Path for log files is ${IGNITE_HOME}/work/log/, as specified in the log4j file. If log4j logger is used and if configuration was not changed, then most likely the change to IGNITE_HOME property variable you made was not picked by the process. You can check it in the log - Ignite prints out

Re: Visor: cache counts are twice of what is expected

2017-08-10 Thread vkulichenko
Hi Roger, It's a known issue and fixed in master. You can try to build from there or check the latest nightly build: https://builds.apache.org/view/H-L/view/Ignite/job/Ignite-nightly/lastSuccessfulBuild/ -Val -- View this message in context:

Re: Apache Tez support for Ignite

2017-08-10 Thread vkulichenko
Ravi, Have you seen the Hadoop Accelerator? https://apacheignite-fs.readme.io/docs/hadoop-accelerator It also provides custom implementation of MR engine. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Apache-Tez-support-for-Ignite-tp16086p16114.html

Re: Running Spark SQL on Spark Thrift Server with Ignite

2017-08-10 Thread vkulichenko
Ravi, If you need to speed up SQL, you should make sure Ignite uses indexes to execute queries. I think you can do the following: - Create Hive RDD and map it to RDD of key value pairs. - Create new IgniteRDD on top of a cache and use IgniteRDD#savePairs method to load data from Hive to Ignite. -

Re: Activating Cluster taking too long

2017-08-10 Thread ezhuravlev
Hi, Please share full logs from all nodes so I can help in investigating of your problem. Evgenii -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Activating-Cluster-taking-too-long-tp16093p16099.html Sent from the Apache Ignite Users mailing list archive at

Re: Persistent store and eviction policy.

2017-08-10 Thread userx
Thank you. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Persistent-store-and-eviction-policy-tp16092p16107.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Setting custom Log location - log4j

2017-08-10 Thread userx
Hi all, I intend to use log4j for Apache Ignite logging. The steps I followed are 1) add maven dependency of org.apache.ignite ignite-log4j 2.0.0 2) set the following in IgniteConfiguration file

Re: Persistent store and eviction policy.

2017-08-10 Thread Denis Magda
Folks, I’ve updated the documentation avoiding any misunderstanding - " If Ignite Persistence is enabled then the page-based evictions have no effect because the oldest pages will be evicted from RAM automatically if there is not enough space available.”

Re: Apache Tez support for Ignite

2017-08-10 Thread ravi
Hi Thanks for the details. I have sent the email to subscribe now and got confirmation back. The reason i have asked the question about Tez and LLAP is apache hive community has deprecated the MR as execution engine and moving towards Tez and LLAP, Will Ignite have equivalent In Memory Tez/LLAP

Re: Setting custom Log location - log4j

2017-08-10 Thread userx
Hi Val, Can you help me out with the configuration change for log4j ? I have provided the value of LOG_HOME in environment variables in eclipse. Here is what I have I am having similar problem setting

Re: Streaming test

2017-08-10 Thread Jessie Lin
Val, please see thread print attached. This is take after a server is run by "bin\ignite.bat config\ignite-writebehind.xml" and the service initialization didn't complete. Thank you very much for helping out! "srvc-deploy-#33%null%" #59 prio=5 os_prio=0 tid=0x577b8000 nid=0x1ef8 waiting

Re: Streaming test

2017-08-10 Thread Jessie Lin
Val, thanks for pointing it out. Now I call AtomicLong Function from service#execute() and it's working. Thank you very much! Jessie On Thu, Aug 10, 2017 at 3:08 PM, vkulichenko wrote: > Jessie, > > You still call atomicLong() method from Service#init(). As I