Ignite 2.8.1: Database Closed error

Hi, Any pointers on what the below error means. There seems to be an out of memory on discoverySPI. what can cause this? ^-- TxLog region [used=0MB, free=100%, comm=100MB] ^-- Ignite persistence [used=50071MB] ^-- sysMemPlc region [used=0MB] ^-- default region

Re: WAL and WAL Archive volume size recommendation

Dennis "The WAL archive is used to store WAL segments that may be needed to recover the node after a crash. The number of segments kept in the archive is such that the total size of all segments does not exceed the specified size of the WAL archive" Given the above in documentation, if we

Re: ReadFromBackup, Primary_SYNC, Backups

Hi, Can you please give some feedback on the below? From: Mahesh Renduchintala Sent: Tuesday, November 3, 2020 8:20 AM To: user@ignite.apache.org Subject: ReadFromBackup, Primary_SYNC, Backups Hi I have a large SQL table (12 million records) in cacheMode

New Node - Rebalancing

Hi, As soon as we add a new server node into the cluster, rebalancing starts this is clear. is there a way to know when the rebalancing successfully ends on the new server node? Caches in the cluster are both replicated and partitioned. regards Mahesh

Re: Failed to Resolve NodeTopology - ignite 2.8.1

you please file a ticket about this issue against Apache Ignite JIRA? https://issues.apache.org/jira/projects/IGNITE Regards, -- Ilya Kasnacheev вт, 3 нояб. 2020 г. в 05:59, Mahesh Renduchintala mailto:mahesh.renduchint...@aline-consulting.com>>: Hi, We saw all ignite nodes crash today m

Failed to Resolve NodeTopology - ignite 2.8.1

Hi, We saw all ignite nodes crash today morning. Below are the error logs? why would "Failed to Resolve Node Topology" occur? what would cause this? If there is a network disturbance, should I not get some sort of segmentation error?

ReadFromBackup, Primary_SYNC, Backups

Hi I have a large SQL table (12 million records) in cacheMode partitioned. This table is distributed over two server nodes -1- When running a large SELECT from a thick client node, could data be fetched from the backup instead of primary partitions? Below is the configuration. />

2.9.0 migration - Thick Client error

Hi, Migrating from 2.8.1 to 2.9.0. We are seeing the below error with thick client Please suggest how to work around it regards mahesh WARNING: Failed to resolve default logging config file: config/java.util.logging.properties [12:33:47]__ [12:33:47] / _/

Ignite Thin Client - IDLE TIMEOUT

Hi, is there a way one can set IdleTImeOut for a thin client? I did not see such configuration in ClientConfiguration. regards Mahesh

Ignite Thin Client - Compute Jobs

Hi, WIth a thin client handle, is it possible to launch tasks on the compute grid? regards mahesh

Re: Ignite 2.81. - NULL pointer exception

I sent the logs again. There is no specific activity. We have a cluster - 2 servers and about 15 thick clients Just happened without much info. I can say it is likely a new node joined in and it may have triggered this crash.

Re: Ignite 2.81. - NULL pointer exception

We received this null pointer exception again 05:57:14,810][INFO][exchange-worker-#81][time] Finished exchange init [topVer=AffinityTopologyVersion [topVer=1139, minorTopVer=0], crd=true] [05:57:15,553][INFO][exchange-worker-#81][GridCachePartitionExchangeManager] Skipping rebalancing (nothing

Ignite 2.81. - NULL pointer exception

Hi, we have a null pointer exception in one of our servers. No major activity was happening when the server crashed Please check the logs and see if there is any workaround we can use. We are in a production environment. regards mahesh [05:42:42,194][SEVERE][sys-stripe-19-#20][] Critical

ignite 2.8.1: Failed to resolve node topology

Hi, We have a crash in our environment with the below error. Any insight into what might have gone wrong? regards Mahesh ^-- Heap [used=17211MB, free=64.98%, comm=49152MB] ^-- Off-heap [used=48448MB, free=26.41%, comm=65736MB] ^-- sysMemPlc region [used=0MB, free=99.98%, comm=100MB]

Atomic Sequence/Auto Increment via IgniteClient - Thin Client - Ignite 2.8.1

Hello, Is there a way to access ignite Atomic Sequence via thin client apis? We have a situation where we have thick clients (java micro-services) and thin clients (Spark) and both need to generate IDs for inserting data into tables. regards Mahesh

Re: Ignite 2.8.1: Spark 2.4.4. No class found error

this issue is solved. I was using 2.4.4 cluster and using 2.3 binaries

Re: Ignite Spark - Error - Did you call Ignition.start(..)?

I am seeing this problem even on Ignite 2.8.1, spark 2.4.4 Does spark-worker and ignite be on the same server? val CONFIG = "resources/node-config-spark.xml" val TABLE_NAME = "table_access_master" //Creating Ignite-specific implementation of Spark session. val igniteSession =

Ignite 2.8.1: Spark 2.4.4. No class found error

Hi, I am running a program that connects to a remote spark cluster. I get the following error. Any insight into the error? Spark 2.4.4 Scala 2.11.12 Ignite 2.8.1 Exception in thread "main" java.lang.NoSuchMethodError:

Ignite Spark - Error - Did you call Ignition.start(..)?

Hi, I have very simple code to try out Ignite Spark. public class SparkTestJava { private static final String CONFIG = "examples/config/example-ignite.xml"; private static final String CACHE_NAME = "testCache"; private static final String TableName = "table_access_master"; /**

Re: UserVersion for Remote Deployment

typos - corrected a) is it a correct understanding that as long as UserVersion of the client matches that of the remote node for that class, the class will NOT be redeployed from any other client nodes?

UserVersion for Remote Deployment

Hi, We have a class (SqlQuery) that migrates from the client node to the remote node for execution. is the below way of fixing UserVersion for this class on the client node side correct? If not, what is the right way? see below. ignite.xml http://www.springframework.org/schema/beans;

Re: Ignite 2.7.6: Memory Leak with Direct buffers in TCPCommunication SPI

Please see the attached jhist. In this condition one of the node consumed about 18 GB. jhist Description: jhist

Cache ExpirationPolicy

Hi I have a cache template marked as below in the defaul_config.xml. I was expecting that table created as such would automatically get deleted from off-heap and backup. This is because of the expiration policy set as below However, I observe that these tables are not getting deleted? What am

Ignite 2.7.6: Memory Leak with Direct buffers in TCPCommunication SPI

Hi, We have been searching for a cause of memory leak in ignite server nodes for many weeks now. The memory leak exists and below is the scenario. Scenario * Scenario * Our ignite servers have about 50GB of data. Two servers were baselined * There are about 10 client nodes

Ignite 2.7.6 : Discovery Threads reporting OOPS

Hi, We have Ignite data nodes reporting out of memory after many days. What could be possible reasons? [16:45:04,018][SEVERE][tcp-disco-client-message-worker-#2404][TcpDiscoverySpi] Runtime error caught during grid runnable execution: GridWorker [name=tcp-disco-client-message-worker,

Re: NearCache for SQL tables

Dennis, We use ignite for supply chain management usecases and almost all are SQL schemas In replicated mode, for large tables and for a single query, the data reads are happening from ram and disk of a single node. We measured a 4-time increase in query latencies (reads) in replicated mode

NearCache for SQL tables

Hi, is there a way to create near cache on CLIENT nodes for SQL tables? regards Mahesh

Fetching Server DataStorageMetric

Hi, I need to fetch the DataStorageMetric of server nodes. Tried the following API. It seems to give the dataStorageMetric of the local node (a thick client) only. DataStorageMetrics pm = ignite.dataStorageMetrics() How do I programmatically fetch DataStorageMetric of the server node(s)?

Re: GridCachePartitionExchangeManager Null pointer exception

Pavel, I don't have the logs for the client node. It happened 2 times in our cluster till now in 45 days. Difficult to reproduce. But the logs show a null point exception on server nodes... 1st one server node (192.168.1.6) went down and then the other. In 12255, it is noted that an assertion

Re: nodes are restarting when i try to drop a table created with persistence enabled

https://issues.apache.org/jira/browse/IGNITE-12255 Upon reviewing 12255, the description of this issue shows an exception occurring on the thick client side. However, the logs, that I attached show a null pointer exception on the ALL the server nodes, leading to a complete cluster crash. isnt

Re: nodes are restarting when i try to drop a table created with persistence enabled

Shivakumar's system configuration and mine could be different. But I feel, we are seeing the same issue here. Deleting tables via a single thick client causes other thick clients to go out of memory. This OOM issue was reported below here.

Re: GridCachePartitionExchangeManager Null pointer exception

Hello Pavel, OK. I am a little bit not clear on the workaround you suggested on your previous comment As a workaround, I can suggest to not explicitly declare caches in the client configuration. During joining to cluster process, the client node will receive all configured caches

Re: GridCachePartitionExchangeManager Null pointer exception

Pavel, Thanks for your analysis. The two logs, that I attached, are those of two server data nodes (none are configured in thick client mode). The logs did show a server data node, losing connection and try to connect back to the other node (192.168.1.6)... On second thoughts, the below still

Re: Changing password of USER "ignite"

So what we found out is, we are able to change the password with DBeaver after connecting to the DB using username: ignite password: ignite However, the same does not work via Grid gain. Grid gain gives the following error. Gridgain pops up Error: Operation not allowed: authorized context is

Re: GridCachePartitionExchangeManager Null pointer exception

This seems to be a new bug, and unrelated to IGNITE-10010. Both the nodes were fully operational when the null pointer exception happened. The logs show that and both the nodes crashed Can you give some insights into this, possible scenarios this could have led this? Is there any potential

Re: Changing password of USER "ignite"

-1- Step a) Login to GridGain enter user name "ignite" and password "ignite" to connect to your cluster. Step b) Go to the notebook tab, execute the SQL The SQL used was in note book was - ALTER USER "ignite" WITH PASSWORD 'abc123' Gridgain pops up Error: Operation not allowed:

Re: Ignite 2.7.0: Ignite client:: memory leak

We observed the same memory leak issue on 2.7.6 as well. Deleting tables continuously from a thick client causes out of memory exceptions in other thick clients. Can you please recheck? We badly need a workaround for this issue. This is probably related to another issue that is discussed on

Re: Changing password of USER "ignite"

We followed all that, ignite user name and ignite password for the DB work fine Now we want change the password to something else. This is when we get Error: Operation not allowed: authorized context is empty. The SQL used was ALTER USER "ignite" WITH PASSWORD 'abc123'

Changing password of USER "ignite"

Hi, we are looking to start adding security with some basic authentication. To begin with change the USER "ignite" password to something else. When using the SQL - ALTER USER "ignite" WITH PASSWORD 'abc123'; We get the below error. Error: Operation not allowed: authorized context is empty.

Re: nodes are restarting when i try to drop a table created with persistence enabled

We noted the same on 2.7.6 as well. Deleting tables continuously from a thick client causes out of memory exceptions in other thick clients. The fix, regarding grid partition message exchanges, that went in 2.7.6 does not seem to work.

Ignition Start - Timeout if connection is unsuccessful

Hello We are currently using Ignition.Start to get the handle for thick client. >> ignite = Ignition.start(cfg); As I understand, this API is a blocking API, unless the connection is successfully established. However, in some scenarios, where the thick client is unable to connect properly,

Re: Ignite 2.7.0: Ignite client:: memory leak

Dennis, Thanks for the update. Will wait for the release.

Re: Ignite 2.7.0: Ignite client:: memory leak

The clients we use have memory ranging from 4GB to 8GB. OOM was produced on all these clientssome sooner, some little later, bit always was seen. The workaround is still stable for more than 48 hours now.

Re: Ignite 2.7.0: Ignite client:: memory leak

Denis, Thanks. Meanwhile, we made some progress...the via workaround seems to use these two flags in the thick client config -DIGNITE_SKIP_PARTITION_SIZE_VALIDATION -DIGNITE_EXCHANGE_HISTORY_SIZE=8 So far, we haven't seen clients going OOM for about 24 hours (still watching) Based on the logs,

Re: Ignite 2.7.0: unresponsive: Found long running cache future -

Denis, It is quite difficult to capture all clients logs from our production environment. I gave you the server log. I can devise a better test if you can say explain this error? What does "long running cache futures" mean? regards mahesh

Re: Ignite 2.7.0: Ignite client:: memory leak

Infact, in the logs you can see that whenever the below print comes up, memory jumps up by 100-200MB >>Full map updating for 873 groups performed in 16 ms Metrics for local node (to disable set 'metricsLogFrequency' to 0) ^-- Node [id=4c8b23b4, uptime=00:19:13.959] ^-- H/N/C [hosts=8,

Re: Ignite 2.7.0 : server node: null pointer exception

Will try the configuration at server and report back. Slava, Denis, Can you also take a look at the below thread. I certainly believe this again a problem with ignite discovery spi or some such. I have attached all logs and configuration.

Re: Ignite 2.7.0 : server node: null pointer exception

IGNITE_DISCOVERY_HISTORY_SIZE=700 Does this go on the server side or the thick client side ?

Re: Ignite 2.7.0 : server node: null pointer exception

The clients come in and get disconnected from the cluster for many reasons - some intentionally and some due to poor network. Cant have Ignite nodes crashing with null pointer exception.

Re: Ignite 2.7.0 : thick client has all system threads busy indefinitely

please see client and server configs. <>

Ignite node crash: IndexOutOfBoundsException

Hi We have an IndexOutOfBoundsException and ignite JVM stopped. Can you please check if it is a known bug? regards mahesh [12:42:43,456][SEVERE][exchange-worker-#63][CacheAffinitySharedManager] Failed to initialize cache. Will try to rollback cache start routine. [cacheName=SQL_PUBLIC_FSDZ]

Re: ignite cluster lock up

We are now testing by increasing failureDetectionTimeout values Even if full GC is running, why are ignite system threads blocked? why aren't ignite system threads free to accept new connections? Why exactly would rebooting a few of previously connected nodes, reset everything. There could

Re: ignite cluster lock up

The long JVM pauses are probably due to long time taken by GC... The -XMX parameter is 64GB for me. should I be using more aggressive parameters to free up runtime heap quicker on the server node? I am using the recommended JVM options on ignite website.

Re: ignite cluster lock up

attached are the config files of the server and the client. From: Mahesh Renduchintala Sent: Friday, July 5, 2019 12:37 AM To: user@ignite.apache.org Subject: ignite cluster lock up Hi, we have 10 clients (thick) connected to a ignite cluster (2 node, 16