Ignite AWS library to support spark 3.0.1 and scala 2.12.12 version

Hi All, Can anyone please let me know which apache or grid-gain ignite library for AWS EC2 instance deployment will support AWS EMR spark application with spark-3.0.1 and scala-2.12.12 version to access the ignite cache launched in AWS EC2 instances from EMR spark job? Currently, I am using

Re: Getting ERROR IgniteKernal: Failed to start manager: GridManagerAdapter, Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to deserialize object with given class loader: sun.misc.L

Hi, any help or suggestion for this? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Getting ERROR IgniteKernal: Failed to start manager: GridManagerAdapter, Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to deserialize object with given class loader: sun.misc.Launc

Hi, I am using "gridgain-community-8.8.1" package to bring up ignite in one AWS EC2 instance and access that from AWS EMR Spark-3.0.1 job. While trying to get the ignite instance from spark job using "Ignition.getOrStart(igniteConf)", its throwing exceptions. I am using same java openJDK8 and

Re: Getting "Failed to deserialize object" exception while trying print value from Cache.Entry oblect

Thank you Alex. I have resolved the issue already. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Getting "Failed to deserialize object" exception while trying print value from Cache.Entry oblect

Thank you Alex Plehanov for the quick response. Could you please provide some details to make it work. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Getting "Failed to deserialize object" exception while trying print value from Cache.Entry oblect

Hi, I am working on implementing ignite in AWS as EC2 cluster and I am following documentation from "https://www.gridgain.com/docs/latest/installation-guide/aws/manual-install-on-ec2; for my POC. I am using three EC2 instances to form a cluster with gridgain-community-8.8.1 package

Ignite yarn job's resources (number of executors and cores) increase automatically after running for 7 days

Hi, I had submitted a spark job for ignite-yarn application on last 12th March, 2020 with 40 executors and 2 cores (running containers 41 and allocated CPU 81 on spark web UI page) and after running it for 7 days, Today on 19th March I have seen on spark web UI page that resources were went up by

Yarnclient killApplication method throwing java.nio.channels.ClosedByInterruptException

While I am trying to kill my ignite yarn application, killApplication method is throwing the below exception and I am not able to kill my yarn job from the application. java.io.IOException: Failed on local exception: java.nio.channels.ClosedByInterruptException; Host Details : local host is:

Re: Ignite yarn resources keep on increasing

The problem is I am not able to kill my ignite yarn application even though exception has been thrown while starting the ignite and the resources of the ignite-yarn spark job keeps piling up. Please find attached image of spark web-ui. From my "StartStandalone" spark application I use to submit

Re: Ignite yarn resources keep on increasing

Hi Ilya, Yes, I start the ignite in client mode and my other applications access this ignite cluster. I use to start the ignite cluster with 40-executors -and 2-cores. Usually, in success scenario my starting ignite and data loading finish within 5/6 minutes. The logs are for negative scenario,

Re: Ignite yarn resources keep on increasing

Hi Ilya, Yes, I start the ignite in client mode and my other applications access this ignite cluster. I use to start the ignite cluster with 40-executors -and 2-cores. Usually, in success scenario my starting ignite and data loading finish within 5/6 minutes. The logs are for negative scenario,

Re: Ignite yarn resources keep on increasing

Hi Ilya, When it doesn't fine existing ignite cluster, I use to create a new one. If you see the logs, you will get >>> Starting Ignite Client with cluster name: titan-resourceUP You can also see ignite initial start logs as well: >>>__ >>> / _/ ___/ |/ /

Re: Ignite yarn resources keep on increasing

Hi Ilya, >From my spark application, I am trying to do two things: First, start Ignite yarn cluster using ignite-yarn.jar Second, load data to ignite cluster. In the first step, I have my own method to submit another spark job with YarnConfiguration to start the ignite. In second step, I use to

Re: Ignite yarn resources keep on increasing

Hi Ilya, My submitted Ignite-yarn application works fine for best case scenarios. The issue is with worst case scenarios. E.g., below is my correct ignite_path property and my application works fine with it: IGNITE_PATH=/project/ecpdevbermuda/ignite/apache-ignite-2.7.0-bin.zip But, if I give

Re: Ignite yarn resources keep on increasing

Hi Andrei, I am using below configurations: IGNITE_NODE_COUNT=40 IGNITE_RUN_CPU_PER_NODE=5 IGNITE_MEMORY_PER_NODE=15000 IGNITE_PATH=/project/ecpdevbermuda/ignite/apache-ignite-2.7.0-bin.zip IGNITE_VERSION=2.7.0 IGNITE_WORKING_DIR=/project/ecpdevbermuda/ignite/

Ignite yarn resources keep on increasing

I am using ignite 2.7 version for ignite yarn deployment. I have my own spark application that start ignite yarn cluster and load data to ignite. It works fine in positive scenarios, but whenever there is an exception from the ignite-yarn.jar side like giving wrong path for some properties