Can Ignite code throw any RuntimeExceptions ?

2018-12-26 Thread userx
Hi Let say we have a java program which starts Ignite in a client mode. Let us say there is a RuntimeException which happens in the Ignite code (on the client side only), would it be wrapped as an IgniteException and passed to the Java client or would it be thrown as a RuntimeException to the

Data Replication Between Ignite Clusters Through Kafka

2018-12-26 Thread Prashant Murthy
Hi, I am referring the below link to create data replication between two clusters using kafka. https://dzone.com/articles/real-time-data-replication-between-ignite-clusters I started two clusters and started kafka/zeekeeper service and created a topic - test2 But once I start to connect, I

Re: If I want to know how long it takes for data reading from the server side, where should I add logs?

2018-12-26 Thread Justin Ji
Andrei - Thank for your reply, sorry for missing a sentence in this post, it should be: there are almost 0.013% read requests take more than 20ms, and 0.0336% more than 200ms. When the traffic is very large, the number of requests over 200ms will be very large. So I want to know where the time

Re: ignite stops expiring caches after a period of time

2018-12-26 Thread Scott Feldstein
hi Ilya, Why wouldn't many people use Expiry with 2.x? Is there another feature that I should be using that accomplishes the same thing? I've reproduced the issue for 2.6.0, but with 2.7.0 it looks like it is working. Here is the repro if anyone is interested -

Re: Query Execution very slow

2018-12-26 Thread Prasad Bhalerao
How to push the ids to temp table, can you please give any example? Is it a in memory temp table created by ignite? Can you please explain how enforceJoinOrder will help in this case? Thanks , Prasad On Wed 26 Dec, 2018, 9:37 PM Ilya Kasnacheev Hello! > > Can you try pushing temp table to 1st

Re: error in ignite-spark

2018-12-26 Thread Ilya Kasnacheev
Hello! With -Pscala command or via netbeans dialogs. Regards, -- Ilya Kasnacheev ср, 26 дек. 2018 г. в 19:54, Mehdi Seydali : > How could i enable scala profile?? > > On Wed, 26 Dec 2018, 13:44 Ilya Kasnacheev, > wrote: > >> Hello! >> >> You have to enable 'scala' profile in Maven to be

Re: error in ignite-spark

2018-12-26 Thread Mehdi Seydali
How could i enable scala profile?? On Wed, 26 Dec 2018, 13:44 Ilya Kasnacheev, wrote: > Hello! > > You have to enable 'scala' profile in Maven to be able to build Spark > examples. However, even after I rebuild with scala profile, netbeans will > not show them propetly. You will still be able

Re: Query Execution very slow

2018-12-26 Thread Ilya Kasnacheev
Hello! Can you try pushing temp table to 1st position and setting enforceJoinOrder=true? SELECT ipv4agd.id, ipv4agd.assetGroupId, ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE (assetGroupId bigint = ? ) temp JOIN IpV4AssetGroupData ipv4agd ON ipv4agd.assetGroupId = temp.assetGroupId

Index inline size

2018-12-26 Thread Prasad Bhalerao
Hi, I have set sqlIndexMaxInline size in cache configuration level as follows. cacheCfg.setSqlIndexMaxInlineSize(100); But still I am getting following warning message in log. WARN o.a.i.i.p.q.h2.database.H2TreeIndex - Indexed columns of a row cannot be fully inlined into index what may

Re: [E] Re: Updating Apache Ignite WebSession attributes

2018-12-26 Thread Ilya Kasnacheev
Hello! You can add *WEB_SES_KEEP_BINARY_PARAM* as true or false as an init parameter to WebSessionFilter. Regards, -- Ilya Kasnacheev ср, 26 дек. 2018 г. в 17:30, Rout, Biswajeet : > Hi Ilya, > > - No, I have not used explicitly setting the setAttribute() method. I will > definitely try

Re: [E] Re: Updating Apache Ignite WebSession attributes

2018-12-26 Thread Rout, Biswajeet
Hi Ilya, - No, I have not used explicitly setting the setAttribute() method. I will definitely try that. - What is binary or POJO session store? is it a feature in Ignite? how to use this? Where to to add *WEB_SES_KEEP_BINARY_PARAM = true|false* *- *Yes I have enabled DEBUG logging for Ignite, I

Re: Write Synchronization Mode per client

2018-12-26 Thread aealexsandrov
Hi, Cache in Ignite is a cluster wide storage for data. This data will be available around all clients. And it should work similarly for each of them. That's why the configuration can't be changed during runtime on the already started cache. I don't think that all your clients will use the same

Re: [E] Re: Updating Apache Ignite WebSession attributes

2018-12-26 Thread Ilya Kasnacheev
Hello! - Have you tried to explicitly do setAttribute(key, changedObject) during request handling? - Do you have binary or POJO session store? This is controlled by org.apache.ignite.cache.websession.WebSessionFilter#WEB_SES_KEEP_BINARY_PARAM = true|false. Can you try both? - Can you enable DEBUG

Re: continous query

2018-12-26 Thread Ilya Kasnacheev
Hello! Have you tried? Got any relevant errors? Can you share a sample project? Regards, -- Ilya Kasnacheev ср, 26 дек. 2018 г. в 12:39, Som Som <2av10...@gmail.com>: > hello. > > i would like to use labda expression for initial and remote filters of > continousquery. simething like this: >

Can we put a timeout on Ignite.close ?

2018-12-26 Thread userx
Hi Is there a way we can put a timeout on close operation of Ignite. For an example I have a java program which does some computation and then starts the Ignite in client mode so that the computed numbers can be written to cache on Ignite Servers in persistent mode. Say if there is an execution

Re: Write Synchronization Mode per client

2018-12-26 Thread rom_we
Hi Andrei, Thanks for the response. I'm trying to understand how can i use ignite as a *global cross applications* cache for our system entities. This cache could be used by different parties with different needs. Some care more about data consistency and some requires low latency. I just wanted

Re: Use of GridFutureAdapter$Node objects ?

2018-12-26 Thread userx
Thanks Andrei, will try what you have suggested. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Loading data from Spark Cluster to Ignite Cache to perform Ignite ML

2018-12-26 Thread zaleslaw
Hi, Igniters! I am looking for a possibility to load data from Spark RDD or DataFrame to Ignite cache with next declaration IgniteCache dataCache to perform Ignite ML algorithms. As I understand the current mechanism of Ignite-Spark integration helps to store RDD/DF from Spark in Ignite to

Re: If I want to know how long it takes for data reading from the server side, where should I add logs?

2018-12-26 Thread aealexsandrov
Hi, Can you share your benchmark and explain what results you are expecting? Are you sure that there is no GC pauses at that moment? Ignite already have its own benchmarks that used Yardstick: https://apacheignite.readme.io/docs/perfomance-benchmarking Are you use them? If not possible you can

Re: Write Synchronization Mode per client

2018-12-26 Thread aealexsandrov
Hi, Synchronization mode is a part of the cache configuration. It will be set up at the cache starting. After that, the cache configuration can't be changed in runtime. The only case when cache configuration could be changed is when you destroy old cache and create a new one with the new

Re: Use of GridFutureAdapter$Node objects ?

2018-12-26 Thread aealexsandrov
Hi, Looks like it's a part of the current implementation. I will try to investigate it. However, as a workaround you can use the next code to decrease the possible memory allocation in case if you are going to load a lot of data: int chunk_size = 1000; try (IgniteDataStreamer

Re: [E] Re: Updating Apache Ignite WebSession attributes

2018-12-26 Thread Ilya Kasnacheev
Hello! When you do getAttribute/setAttribute/removeAttribute on web session, it should be updated at the end of requedt. Do you have reproducer for this kind of behavior? Regards, -- Ilya Kasnacheev ср, 26 дек. 2018 г. в 13:46, Rout, Biswajeet : > Hi, > I did not get your question. > > I

Re: error in ignite-spark

2018-12-26 Thread Ilya Kasnacheev
Hello! No you don't. You can use the following maven commands from examples directory to run example: mvn clean install -Pscala -DskipTests mvn -Pscala exec:java -Dexec.mainClass=org.apache.ignite.examples.spark.SharedRDDExample Regards, -- Ilya Kasnacheev ср, 26 дек. 2018 г. в 13:42, Mehdi

Re: diferrences between IgniteRdd and SparkRdd

2018-12-26 Thread aealexsandrov
Hi, The main difference between native Spark RDD and IgniteRDD is that Ignite RDD provides a shared in-memory view on data across different Spark jobs, workers, or applications, while native Spark RDD cannot be seen by other Spark jobs or applications. You can see the attached image that

Re: error in importing ignite 2.7 to netbeans

2018-12-26 Thread aealexsandrov
Please check that you set the scale profile because of without it the spark dependencies will not be added: ... scala src/main/spark org.apache.ignite

Re: Failed to send partition supply message to node: 5423e6b5-c9be-4eb8-8f68-e643357ec2b3 class org.apache.ignite.IgniteCheckedException: Could not find start pointer for partition

2018-12-26 Thread Pavel Kovalenko
This sounds strange. There definetely should be a cause of such behaviour. Rebalancing is happened only after an topology change (node join/leave, deactivation/activation). Could you please share logs from node with exception you mentioned in message, node with id

Re: error in importing ignite 2.7 to netbeans

2018-12-26 Thread Mehdi Seydali
this is pom file in folder of example. http://maven.apache.org/POM/4.0.0; xmlns:xsi=" http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> 4.0.0 org.apache.ignite

Re: error in importing ignite 2.7 to netbeans

2018-12-26 Thread aealexsandrov
Hi, Depends on how exactly you start the SharedRDDExample? 1)In case if you use the new maven project then possible you forgot to add dependencies for Ignite spark: org.apache.ignite ignite-core 2.7.0 org.apache.ignite

Re: Failed to send partition supply message to node: 5423e6b5-c9be-4eb8-8f68-e643357ec2b3 class org.apache.ignite.IgniteCheckedException: Could not find start pointer for partition

2018-12-26 Thread aMark
Thanks Pavel for prompt response. I could confirm that node "5423e6b5-c9be-4eb8-8f68-e643357ec2b3" (and no other node in the cluster) did not go down, not sure how did stale data cropped up on few nodes. And this type of exception is coming from every server node in the cluster. What happens if

Re: error in ignite-spark

2018-12-26 Thread Mehdi Seydali
spark example have been written with java not scala. i think this error relate to spark version. i have another question. for running this example is it necessary to intsall spark before running?. On Wed, Dec 26, 2018 at 1:44 PM Ilya Kasnacheev wrote: > Hello! > > You have to enable 'scala'

Re: [E] Re: Updating Apache Ignite WebSession attributes

2018-12-26 Thread Ilya Kasnacheev
Hello! Are you sure that your servers actually form an Ignite topology? Can you provide logs? Regards, -- Ilya Kasnacheev ср, 26 дек. 2018 г. в 10:07, Rout, Biswajeet : > ++ Ilya > > On Wed, Dec 26, 2018 at 12:35 PM Rout, Biswajeet < > biswajeet.r...@verizon.com> wrote: > >> Hi Ilya, >> >>

Re: Threads and their purposes

2018-12-26 Thread Ilya Kasnacheev
Hello! Here is description of most of thread pools: https://apacheignite.readme.io/docs/thread-pools tcp-disco is discovery: https://apacheignite.readme.io/docs/tcpip-discovery tcp-comm is communication: https://apacheignite.readme.io/docs/network-config#tcpcommunicationspi upd-ver-checker

Threads and their purposes

2018-12-26 Thread ashishb888
I have an application, which uses data streamers (4 streamers) to load data in caches (4 caches). I can see different threads and their count: 1. tcp-comm-worker 1 2. pub 16 3. sys 16 4. sys-stripe 17 5. utility 16 6. upd-ver-checker 1 7. ttl-cleanup-worker 1 8. tcp-disco-srvr 1 9.

Re: error in ignite-spark

2018-12-26 Thread Ilya Kasnacheev
Hello! You have to enable 'scala' profile in Maven to be able to build Spark examples. However, even after I rebuild with scala profile, netbeans will not show them propetly. You will still be able to run them all right, I think. Regards, -- Ilya Kasnacheev ср, 26 дек. 2018 г. в 12:27, mehdi

continous query

2018-12-26 Thread Som Som
hello. i would like to use labda expression for initial and remote filters of continousquery. simething like this: [Serializable] public class ScanQueryFilter3 :ICacheEntryFilter { private readonly Expression, bool>> _pdct; public ScanQueryFilter3(Expression, bool>>

error in ignite-spark

2018-12-26 Thread mehdi sey
hi. i want to execute a RDD example in spark from example folder of ignite 2.7, but i have encounter and error just like an attached picture. in import section you see an underlined line. i have added dependency but still is remained why?

Re: Failed to send partition supply message to node: 5423e6b5-c9be-4eb8-8f68-e643357ec2b3 class org.apache.ignite.IgniteCheckedException: Could not find start pointer for partition

2018-12-26 Thread Pavel Kovalenko
Hello, It means that node with id "5423e6b5-c9be-4eb8-8f68-e643357ec2b3" has outdated data (possibly due to restart) and started to rebalance missed updates from a node with up-to-date data (where you have exception) using WAL. WAL rebalance is used when the number of entries in some partition

Failed to send partition supply message to node: 5423e6b5-c9be-4eb8-8f68-e643357ec2b3 class org.apache.ignite.IgniteCheckedException: Could not find start pointer for partition

2018-12-26 Thread aMark
Hi, We are using Ignite 2.6 as persistent store in Partitioned Mode having 12 server node running in cluster, each node is running on different machine. There are around 48 client JVM as well which connect to cluster to fetch the data. Recently we have started getting following exception on

Re: Full GC in client after cluster become unstable with "Unable to await partitions release latch within timeout"

2018-12-26 Thread aMark
I will send you the logs soon. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Limiting the number of files in WalArchive folder

2018-12-26 Thread aMark
Using Ignite-2.7 solved the issue. Thanks a lot. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/