Multithreading SQL queries in Apache Ignite

In my case I have data present on 1 server node and 25 clients connected to this server, concurrently firing sql queries. So, does Ignite by default parallelizes these queries or do we have to do some settings? Can we apply some kind of multithreading on server side to handle these queries for

Re: Remote Server Thread Not exit when Job finished,Cause out of memory

Hi vdpyatkov, I tried like this , problem still exists. The client code just as below: ClusterGroup rmts = getIgnite().cluster().forRemotes(); IgniteCompute compute = getIgnite().compute(rmts).withAsync(); compute.apply(new IgniteClosure() {

How can I detect DB writing abnormal in case of write-behind?

Hello: I am trying to put some data to a cache configured with write-through and write-behind. In this process, something wrong in the DB, then data can not be written into it. I want to know how can I detect the database error as soon as possible? Can I detect it by the metrics of the

Re: Remote Server Thread Not exit when Job finished,Cause out of memory

Hi Alex, You should do like this: IgniteCompute compute = ignite.compute().withAsync(); compute.apply(...) IgniteFuture future = compute.future(); ... future.cancel(); And handle Thread.interrupted() flag into closure. On Mon, Nov 14, 2016 at 2:35 PM, alex wrote: >

C++ API can't build successfully in my linux environment

Hi, all I build C++ api in linux environment, linux is Center OS, and g++ version is 4.4.6?? I get a lot build error, and I have to modify the code, include : 1??ignite_error.h : ~IgniteError(); ~IgniteError() throw(); 2??ignite_error.cpp :

Re: Multithreading SQL queries in Apache Ignite

>From my point of view, it depends. Only performance measurements can give answer. On Mon, Nov 14, 2016 at 5:06 PM, rishi007bansod wrote: > I have set it to default value i.e. double the number of cores. But will it > improve performance if I increase it further? > > >

Re: C++ API can't build successfully in my linux environment

Hi, Which version do you use? Best Regards, Igor On Mon, Nov 14, 2016 at 4:59 PM, smile wrote: > Hi, all > I build C++ api in linux environment, linux is Center OS, and g++ > version is 4.4.6， I get a lot build error, and I have to modify the code, > include : > >

Re: Multithreading SQL queries in Apache Ignite

I have set it to default value i.e. double the number of cores. But will it improve performance if I increase it further? -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Multithreading-SQL-queries-in-Apache-Ignite-tp8944p8949.html Sent from the Apache Ignite

Re: Apache Spark & Ignite Integration

Ok. Is there a jira task that I can track for the dataframes and datasets support? I do have a couple of follow up questions to understand the memory representation of the shared RDD support that ignite brings with the spark integration. 1. Could you detail on how are shared RDD's implemented

Re: Problem with v for listening updates

Have you used configuration from ignite-examples module ? - "examples/config/example-ignite.xml" I asked my colleagues to run that code and they've got same result as me. Also, I've found if I disable peer class loading (peerClassLoadingEnabled in example-default.xml) it works as expected for

Re: Multithreading SQL queries in Apache Ignite

Hi, Of course all requests will be processed concurrently by server. At this moment queries use Ignite system pool for execution. You can adjust size of this pool if needed using IgniteConfiguration.setSystemThreadPoolSize() method. On Mon, Nov 14, 2016 at 4:10 PM, rishi007bansod

Re: Partitioning on a non-uniform cluster

Hi, In order to implement resources aware affinity function you can use node attributes (available via ClusterNode interface). But node attributes should be initialized before Ignite node started (except of default node attributes that can be found in IgniteNodeAttributes class). Also you can

Re: start C++ server in linux， it report "Failed to initialize JVM" error

Hi, Ignite fully compatible with JDK 7. I think, you try to run Ignite into lower version than 7. 51.0 matches to JDK 1.7 Please, make sure, that Ignite runs in particular JDK (check env). -- View this message in context:

Re: java.lang.ClassNotFoundException: Failed to peer load class

Hi Alsex, Can you please provide this class com.testlab.api.inf.dao.RepositoryDao? It can has serialization issue for particular class. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/java-lang-ClassNotFoundException-Failed-to-peer-load-class-tp8778p8953.html

Partitioning on a non-uniform cluster

Hello, Judging by the documentation and some discussions on this list, can you confirm that Ignite cache does not take into account different memory settings, i.e. if we have various nodes with 16GB and 32GB allocated for cache, there would be no two times more partitions assigned to larger

Re: Remote Server Thread Not exit when Job finished,Cause out of memory

Hi Alex, I think, these threads are executing into pools of threads, and number of threads always restricted by pool size[1]. You can configure sizes manually: [1]: https://apacheignite.readme.io/v1.7/docs/performance-tips#configure-thread-pools -- View this message in context:

Remote Server Thread Not exit when Job finished,Cause out of memory

Hi, When I start a remote compute job , call() Or affinityCall(). Remote server will create 6 threads, and these thread never exit. Just like the VisualVM shows below:

Re: Ignite Cache setIndexedTypes Question

Hi, You can use Igniteconfiguration.setIndexedTypes(Kei.class, Val.class) and annotations or (not both) Query Entity and configure indexes in it. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cache-setIndexedTypes-Question-tp8895p8937.html Sent from

Re: Remote Server Thread Not exit when Job finished,Cause out of memory

Thanks vdpyatkov. Currently, the number of threads is not the problem. The problem is that when client finished, how to finish threads on server node which created by this client. For example, client code is: String cacheKey = "jobIds"; String cname = "myCacheName"; ClusterGroup rmts

Re: Multiple servers in a Ignite Cluster

Tracy, You can limit the set of nodes where the cache is deployed via CacheConfiguration.setNodeFilter() config property. Generally, all your nodes should be in the same cluster, but you can create multiple roles and logical cluster groups. -Val -- View this message in context:

Re: Checkingpointing

Yes, checkpoints are the feature of the Compute Grid. What kind of checkpoints do you want to create? Can you give more details on the use case? -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Checkingpointing-tp8933p8977.html Sent from the Apache Ignite

Re: Cluster hung after a node killed

Do you want Ignite to be running in DEBUG? or System.out should be enough from all 3 nodes? -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Cluster-hung-after-a-node-killed-tp8965p8978.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: How can I detect DB writing abnormal in case of write-behind?

I am not understand what you mean: "In this process, something wrong in the DB, then data can not be written into it." -->For one case, some field setting was wrong in DB, such as the field length of DB is less than that in the Pojo. For another case, when process write-behind, the DB down.

Re: rest-http can't get data if key is Integer or others

Only strings are currently supported in HTTP REST. This will be improved when we have full JSON support (probably next year). In the meantime I would recommend to use Ignite API directly if you need support for different data types. -Val -- View this message in context:

Re: Cluster hung after a node killed

Hi Sam, Please attach full logs and full thread dumps if you want someone to take a look. There is not enough information in your message to understand the reason of the issue. -Val -- View this message in context:

Re: C++ API can't build successfully in my linux environment

Here is the link: https://builds.apache.org/view/H-L/view/Ignite/job/Ignite-nightly/lastSuccessfulBuild/ Best Regards, Igor On Mon, Nov 14, 2016 at 6:41 PM, Igor Sapego wrote: > Or you can try nightly build if you need binaries: > > Best Regards, > Igor > > On Mon, Nov

Re: C++ API can't build successfully in my linux environment

Or you can try nightly build if you need binaries: Best Regards, Igor On Mon, Nov 14, 2016 at 6:28 PM, Igor Sapego wrote: > Can you try master? I'm pretty sure I was fixing these issues. > > Best Regards, > Igor > > On Mon, Nov 14, 2016 at 6:19 PM, smile

Re: How can I detect DB writing abnormal in case of write-behind?

Hi, I am not understand what you mean: "In this process, something wrong in the DB, then data can not be written into it." How did you detect this? If "write behind" flag was set as true, a data will inserted into DB asynchronously (in dedicated thread). You should to wait when the data will be

?????? C++ API can't build successfully in my linux environment

Thank you for your answer, and I download ignite from this link: http://ignite.apache.org/download.cgi#sources and download the ignite1.7.0, which realse on 2016-08-05; And I will try the link that you gived to

Re: C++ API can't build successfully in my linux environment

Can you try master? I'm pretty sure I was fixing these issues. Best Regards, Igor On Mon, Nov 14, 2016 at 6:19 PM, smile wrote: > I used ignite1.7.0 > -- 原始邮件 -- > *发件人:* "Igor Sapego" > *发送时间:* 2016年11月14日(星期一) 晚上10:09

CacheInterceptor with client nodes

Hi, all!. I've tried to add a CacheInterceptor to Apache Ignite cache. It works fine on server nodes, but when i am trying to access cache("get" operation) from a client node interceptor is invoked localy (on a client node). Is there any way to make it work only on server node, before sending

Re: Cache Memory Behavior \ GridDhtLocalPartition

Hi, On remove operation entry should be removed from primary node and backup nodes as well. Can you reproduce the issue? Can you check if entry was removed only from primary node and exists on backup, e.g. using constant affinity function? I think its possible that the backup is not being

Re: Cache Memory Behavior \ GridDhtLocalPartition

Hi My Cache key class is java.util.UUID I am not using any collocation affinity, could you please elaborate how can i use constant affinity to check if cache entry still exists on backup? Thanks On Mon, Nov 14, 2016 at 4:50 PM, Andrey Mashenkov wrote: > Hi, > > On

?????? C++ API can't build successfully in my linux environment

I used ignite1.7.0-- -- ??: "Igor Sapego" : 2016??11??14??(??) 10:09 ??: "user"; : Re: C++ API can't build successfully in my linux environment Hi, Which version do you use? Best

Re: Does Ignite support SQL statement of "INSERT"?

Hi, No, Ignite does not support SQL DELETE for now. You may watch this issue: https://issues.apache.org/jira/browse/IGNITE-2294 I hope this will be available in upcoming ignite 1.8 release. Meanwhile you could try smth. like this: 1) select _key from Tbl where _your_condition. 2) put all keys to

Re: Does Ignite support SQL statement of "INSERT"?

Does Ignite SQL statement support "DELETE" operation now? If not, What is the best way when I want to remove some data from cache in the specified condition? -- View this message in context:

Re: Remote Server Thread Not exit when Job finished,Cause out of memory

You may want to implement ComputeJobMasterLeaveAware [1] interface for your compute jobs. An interface implementation will be called on servers side for every job that was spawned by client node that has been shut down. Besides, you can refer to this example [2] that demonstrates how to use the

Re: Apache Spark & Ignite Integration

Hi, Here is the ticket https://issues.apache.org/jira/browse/IGNITE-3084 Feel free to paste your questions there as well so that the implementer takes them into account. — Denis > On Nov 14, 2016, at 6:14 AM, pragmaticbigdata

Cluster hung after a node killed

Hi, I have configured cache as off-heap partitioned cache. Running 3 nodes on separate machine. Loaded some data into cache using my application's normal operations. Used "/kill -9 /" to kill node 3. Node 2 shows below Warning on console after every 10 seconds - /11:03:03,320 WARNING

Question about backups

Hello I am dumping the cache configuration for my defined caches. I am seeing this as the backup number memMode=OFFHEAP_TIERED cacheMode=REPLICATED, atomicityMode=TRANSACTIONAL, atomicWriteOrderMode=null, backups=2147483647 I am not setting the backups property in any of my configurations so this

Re: Question about backups

Hi, Replicated cache has backups amount that equal to cluster nodes amount minus one because this one is primary. Amount of nodes in cluster can be changed during time so we use Integer.MAX_VALUE magic number for backups in case of replicated cache. Amount of backups doesn't depend on cache

Re: CacheInterceptor with client nodes

Hi, What exactly are you trying to do? What's the use case? -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/CacheInterceptor-with-client-nodes-tp8964p8970.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Multithreading SQL queries in Apache Ignite

Hi, Single query is currently not parallelized on per-node level. I.e., a query will be be split across nodes, but within each node there will be a single thread processing this query. However, if you issue multiple concurrent queries, they will be executed concurrently in the system thread pool.

Re: Problem with v for listening updates

Can you create a small standalone project that will reproduce the issue? It looks like we're missing somethin here. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Problem-with-Continuous-query-for-listening-updates-tp8709p8972.html Sent from the Apache

Re: How can I obtain a list of executing jobs on an ignite node

How do you identify a job in a bad state? What does this exactly mean? If you can detect this state within the job, you can simply throw an exception from it and will be automatically failed over to another node. -Val -- View this message in context: