Re: Optimistic Serializable SQL Transactions

2019-10-18 Thread Ivan Pavlukhin
Hi Justin, Thank you for sharing a details about your use case. Quite interesting. It seems that in Ignite something like that is possible with optimistic cache transactions (key-value API). Technically it seems to be achievable when transaction protocol accumulates transaction read-write set in

[DISCUSS] Proposal for Ignite Extensions as a separate Bahir module or Incubator project

2019-10-18 Thread Saikat Maitra
Hello, We wanted to discuss on a proposal to move and support the Apache Ignite integrations as separate Ignite Extensions as discussed here http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Pub-Sub-Streamer-Implementation-td43944.html . The reason we wanted to move our Apache Ignite

Re: Ingite node in SpringBoot App

2019-10-18 Thread niamin
Looks like it is something with the using ARInvoiceId as key. All the other combinations of key/value(String/String, String/ARInovice) can be retrieved from thin client. I also serialized the ARInvoiceId object to a file from client project and was able to to deserialize successfully from the

Re: Ingite node in SpringBoot App

2019-10-18 Thread niamin
Looks like it is something with the using ARInvoiceId as key. All the other combinations of key/value(String/String, String/ARInovice) can be retrieved from thin client. I also serialized the ARInvoiceId object to a file from client project and was able to to deserialize successfully from the

Re: Striim support for Ignite

2019-10-18 Thread niamin
So my use case is to be able use SQL/Query API on my existing system to retrieve from Ignite in-memory Grid. I don't expect a lot of write-through from Ignite. I DO expect most of my writes to happen on my RDBMS. I don't see any option rather than writing custom code on RDBMS(may be triggers) to

Re: Ingite node in SpringBoot App

2019-10-18 Thread niamin
Thanks! That worked. Now when I try to fetch object using thin client from a seperate process I get no object. However, when I fetch object using the same server process I get object. Difference on fetching object is the usage of Ignite or IgniteClient. Attached is the client driver.

Re: Unresponsive cluster after "Checkpoint read lock acquisition has been timed out" Error

2019-10-18 Thread ihalilaltun
Hi Ilya, Sorry for the late response. We don't use lock mechanism in our environment. We have a lot of put, get operaitons, as far as i remember these operations does not hold the locks. In addition to these operations, in many update/put operations we use CacheEntryProcessor which also does not

Re: nodes in the baseline topology is going to OFFLINE state

2019-10-18 Thread Ilya Kasnacheev
Hello! There is no supported way to gracefully restart cluster currently. You will have to stop all nodes, start them again and then activate (or auto-activate). Regards, -- Ilya Kasnacheev пт, 18 окт. 2019 г. в 16:42, Shiva Kumar : > Hi Ilya, > My goal is to deactivate the cluster and not

Re: Striim support for Ignite

2019-10-18 Thread Ilya Kasnacheev
Hello! Ignite can fetch data from RDBMS on the fly and even write it back when it changes, it is called Cache Store. https://apacheignite.readme.io/docs/3rd-party-store Regards, -- Ilya Kasnacheev пт, 18 окт. 2019 г. в 17:02, niamin : > There isn't anything that I can find. For synching

Re: Striim support for Ignite

2019-10-18 Thread Evgeniy Rudenko
Ignite has support for Kafka. You can add it to the middle between your RDBMS and Ignite. On Fri, Oct 18, 2019 at 5:02 PM niamin wrote: > There isn't anything that I can find. For synching data i.e. CDC between > RDBMS and Ignite what would you recommend? > > > > -- > Sent from:

Re: Apache Spark + Ignite Connection Issue

2019-10-18 Thread sri hari kali charan Tummala
so spark + jdbc thin client is the only way to go forward? Reading:- val df = spark.read .format("jdbc") .option("url", "jdbc:ignite:thin://18.206.247.40") .option("fetchsize",100) //.option("driver", "org.apache.ignite.IgniteJdbcDriver") .option("dbtable", "Person2").load()

Re: Apache Spark + Ignite Connection Issue

2019-10-18 Thread sri hari kali charan Tummala
Hi Stephen/All, got it working somewhat using below, but have an issue table and data which is created using thin client is failing to read using spark but table created by spark can be read using a thin client, that means table created in Ignite using spark are the only ones read using spark in

Re: Striim support for Ignite

2019-10-18 Thread niamin
There isn't anything that I can find. For synching data i.e. CDC between RDBMS and Ignite what would you recommend? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ingite node in SpringBoot App

2019-10-18 Thread Evgeniy Rudenko
This happens because you create your node inside try () statement. So it is being stopped as soon as try section ends. If you want to keep it active change code to try { Ignite ignite = Ignition.start("example-ignite.xml"); On Fri, Oct 18, 2019 at 4:35 PM niamin wrote: > springbeans.zip

Re: nodes in the baseline topology is going to OFFLINE state

2019-10-18 Thread Shiva Kumar
Hi Ilya, My goal is to deactivate the cluster and not restart !! There is an issue in deactivating the cluster in my deployment so I am going with restart. I have the ignite deployment on kubernetes and during deactivation entire cluster and even request to deactivate (rest or control.sh) hangs

Re: Ingite node in SpringBoot App

2019-10-18 Thread niamin
springbeans.zip Here you go. I've tried with @PostConstruct and EventListener but in both occasions Ignite engine stops immediately after application bootstrap with the following: [09:32:03] Ignite node stopped OK

Re: Apache Spark + Ignite Connection Issue

2019-10-18 Thread sri hari kali charan Tummala
do you mean communication ports 47100-47200 as mentioned ( https://www.gridgain.com/docs/8.7.6//installation-guide/manual-install-on-ec2) ? bare in mind I am running my spark job outside ignite ec2 box (MAC PC). which option is right in my default.xml file? Option 1:-

Re: Apache Spark + Ignite Connection Issue

2019-10-18 Thread sri hari kali charan Tummala
Hi Stephen , do you mean 3.88.248.113: 47500..47700 something like this? or just public ip 3.88.248.113 I tried all the possibilities none of them are getting connected. Thanks Sri On Fri, Oct 18, 2019 at 6:02 AM Stephen Darlington <

Re: nodes in the baseline topology is going to OFFLINE state

2019-10-18 Thread Ilya Kasnacheev
Hello! If cluster is persistent, you can deactivate it and then restart. Regards, -- Ilya Kasnacheev пт, 18 окт. 2019 г. в 09:51, shivakumar : > Hi Ilya Kasnacheev, > Is there any other way of gracefully shutting down/restart the entire > cluster? > > regards, > shiva > > > > -- > Sent from:

Re: Starvation in striped pool

2019-10-18 Thread Ilya Kasnacheev
Hello! We, we had IGNITE_ENABLE_FORCIBLE_NODE_KILL, but the best solution is, in my opinion, to avoid adding anything unstable to the cluster. Regards, -- Ilya Kasnacheev пт, 18 окт. 2019 г. в 08:35, ihalilaltun : > Hi Ilya, > > From time to time, we have faced exactly the same problem. Is

Re: Node stopped.

2019-10-18 Thread Denis Mekhanikov
The following documentation page has some useful points on deployment in a virtualised environment: https://apacheignite.readme.io/docs/vmware-deployment Denis On 17 Oct 2019, 17:41 +0300, John Smith , wrote: > Ok I have metribeat running on the VM hopefully I will see something... > > > On Thu,

Re: Apache Spark + Ignite Connection Issue

2019-10-18 Thread Stephen Darlington
You’re trying to connect a thick client (the Spark integration) to the thin client port (10800). Your example-default.xml file needs to have the same configuration as your server node(s). Regards, Stephen > On 17 Oct 2019, at 18:12, sri hari kali charan Tummala > wrote: > > Hi Community, >

Re: Ingite node in SpringBoot App

2019-10-18 Thread Evgeniy Rudenko
Can you share code of your Bean to check? On Thu, Oct 17, 2019 at 10:11 PM niamin wrote: > I've created a SpringBoot App that includes code to start an Ignite > standalone node and populates a cache. The logic is wrapped in a > PostConstruct method on a Bean. However Ignite server stops as soon

Re: Getting data[parquet,json,...] from S3 buckets to apache ignite

2019-10-18 Thread viktor
dmagda wrote > Hi Viktor, > > Could you please clarify a bit, do you need just to load data > once/periodically or do you want Ignite to write-back to S3 on updates? If > the loading is all you need then create a custom Java app/class that pulls > data from S3 and streams into Ignite

Re: nodes in the baseline topology is going to OFFLINE state

2019-10-18 Thread shivakumar
Hi Ilya Kasnacheev, Is there any other way of gracefully shutting down/restart the entire cluster? regards, shiva -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/