[jira] [Commented] (IGNITE-8728) "IllegalStateException: Duplicate Key" on node join

2020-07-08 Thread Abhay (Jira) 


[ 
https://issues.apache.org/jira/browse/IGNITE-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153489#comment-17153489
 ] 

Abhay commented on IGNITE-8728:
---

Moreover I saw that if we use a create table command and use primary key 
against any column name then idxname is __KEY _PK and is this not duplicate 
KEY_ 

> "IllegalStateException: Duplicate Key" on node join
> ---
>
> Key: IGNITE-8728
> URL: https://issues.apache.org/jira/browse/IGNITE-8728
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.7
>Reporter: Mahesh Renduchintala
>Priority: Critical
> Fix For: 2.10
>
> Attachments: NS1_ignite-9676df15.0.log, NS2_ignite-7cfc8008.0.log, 
> ignite.txt, node-config.xml
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> I have two nodes on which we have 3 tables which are partitioned.  Index are 
> also built on these tables. 
> For 24 hours caches work fine.  The tables are definitely distributed across 
> both the nodes
> Node 2 reboots due to some issue - goes out of the baseline - comes back and 
> joins the baseline.  Other baseline nodes crash and in the logs we see 
> duplicate Key error
> [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> accepted incoming connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> spawning a new thread for connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] Started 
> serving remote node connection [rmtAddr=/192.168.1.7:45102, rmtPort=45102]
>  [10:38:35,451][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] 
> Finished serving remote node connection [rmtAddr=/192.168.1.7:45102, 
> rmtPort=45102
>  [10:38:35,457][SEVERE][tcp-disco-msg-worker-#3|#3][TcpDiscoverySpi] 
> TcpDiscoverSpi's message worker thread failed abnormally. Stopping the node 
> in order to prevent cluster wide instability.
>  *java.lang.IllegalStateException: Duplicate key*
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>  [10:38:35,459][SEVERE][tcp-disco-msg-worker-#3|#3][] Critical system error 
> detected. Will be handled accordingly to configured handler [hnd=class 
> o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext 
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.IllegalStateException: 
> Duplicate key]]
> java.lang.IllegalStateException: Duplicate key
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at

[jira] [Commented] (IGNITE-8728) "IllegalStateException: Duplicate Key" on node join

2020-07-08 Thread Abhay (Jira) 


[ 
https://issues.apache.org/jira/browse/IGNITE-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153485#comment-17153485
 ] 

Abhay commented on IGNITE-8728:
---

I have a test server where I have only one node on version 2.8.1 and it still 
crashes with same Duplicate key error and the only option that remains is to 
remove the work folder and create the complete tables again . 

The logs are attached [^ignite.txt]

> "IllegalStateException: Duplicate Key" on node join
> ---
>
> Key: IGNITE-8728
> URL: https://issues.apache.org/jira/browse/IGNITE-8728
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.7
>Reporter: Mahesh Renduchintala
>Priority: Critical
> Fix For: 2.10
>
> Attachments: NS1_ignite-9676df15.0.log, NS2_ignite-7cfc8008.0.log, 
> ignite.txt, node-config.xml
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> I have two nodes on which we have 3 tables which are partitioned.  Index are 
> also built on these tables. 
> For 24 hours caches work fine.  The tables are definitely distributed across 
> both the nodes
> Node 2 reboots due to some issue - goes out of the baseline - comes back and 
> joins the baseline.  Other baseline nodes crash and in the logs we see 
> duplicate Key error
> [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> accepted incoming connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> spawning a new thread for connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] Started 
> serving remote node connection [rmtAddr=/192.168.1.7:45102, rmtPort=45102]
>  [10:38:35,451][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] 
> Finished serving remote node connection [rmtAddr=/192.168.1.7:45102, 
> rmtPort=45102
>  [10:38:35,457][SEVERE][tcp-disco-msg-worker-#3|#3][TcpDiscoverySpi] 
> TcpDiscoverSpi's message worker thread failed abnormally. Stopping the node 
> in order to prevent cluster wide instability.
>  *java.lang.IllegalStateException: Duplicate key*
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>  [10:38:35,459][SEVERE][tcp-disco-msg-worker-#3|#3][] Critical system error 
> detected. Will be handled accordingly to configured handler [hnd=class 
> o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext 
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.IllegalStateException: 
> Duplicate key]]
> java.lang.IllegalStateException: Duplicate key
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
>

[jira] [Commented] (IGNITE-8728) "IllegalStateException: Duplicate Key" on node join

2019-10-08 Thread Maxim Muzafarov (Jira) 


[ 
https://issues.apache.org/jira/browse/IGNITE-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16946855#comment-16946855
 ] 

Maxim Muzafarov commented on IGNITE-8728:
-

Moved to 2.9 due to inactivity. Please, feel free to move it back if you will 
be able to complete the ticket by 2.8 code freeze date, December 2, 2019.

> "IllegalStateException: Duplicate Key" on node join
> ---
>
> Key: IGNITE-8728
> URL: https://issues.apache.org/jira/browse/IGNITE-8728
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.7
>Reporter: Mahesh Renduchintala
>Assignee: Stanislav Lukyanov
>Priority: Critical
> Fix For: 2.8
>
> Attachments: NS1_ignite-9676df15.0.log, NS2_ignite-7cfc8008.0.log, 
> node-config.xml
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> I have two nodes on which we have 3 tables which are partitioned.  Index are 
> also built on these tables. 
> For 24 hours caches work fine.  The tables are definitely distributed across 
> both the nodes
> Node 2 reboots due to some issue - goes out of the baseline - comes back and 
> joins the baseline.  Other baseline nodes crash and in the logs we see 
> duplicate Key error
> [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> accepted incoming connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> spawning a new thread for connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] Started 
> serving remote node connection [rmtAddr=/192.168.1.7:45102, rmtPort=45102]
>  [10:38:35,451][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] 
> Finished serving remote node connection [rmtAddr=/192.168.1.7:45102, 
> rmtPort=45102
>  [10:38:35,457][SEVERE][tcp-disco-msg-worker-#3|#3][TcpDiscoverySpi] 
> TcpDiscoverSpi's message worker thread failed abnormally. Stopping the node 
> in order to prevent cluster wide instability.
>  *java.lang.IllegalStateException: Duplicate key*
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>  [10:38:35,459][SEVERE][tcp-disco-msg-worker-#3|#3][] Critical system error 
> detected. Will be handled accordingly to configured handler [hnd=class 
> o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext 
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.IllegalStateException: 
> Duplicate key]]
> java.lang.IllegalStateException: Duplicate key
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at

[jira] [Commented] (IGNITE-8728) "IllegalStateException: Duplicate Key" on node join

2019-02-03 Thread Stanislav Lukyanov (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759392#comment-16759392
 ] 

Stanislav Lukyanov commented on IGNITE-8728:


Some thoughts and rationale.

The question of adding a UUID as a suffix is a question of how we want the 
following code to work
{code}
CREATE INDEX ON FOO (A)
CREATE INDEX ON FOO (A)
{code}
Should the second statement fail, or should it create another index (that does 
nothing good)?

Another example is
{code}
CREATE INDEX ON FOO (A)
CREATE SPATIAL INDEX ON FOO (A)
{code}
Even if the first example fails, should this one fail too?

Things to consider:
- ANSI doesn't care about this - indexes are not part of the standard
- Most databases (e.g. Oracle, MS SQL, MySQL) require to specify index names
- Some databases that allow unspecified index name (e.g. PostgreSQL, H2) 
actually create two indexes in that case
- `CREATE INDEX IF NOT EXISTS ON FOO (A)` is not a valid statement - `IF NOT 
EXISTS` must always be followed by a name; this is the same for PostgreSQL, H2 
and Ignite

All in all, it seems to me that creating two identical indexes shouldn't be 
possible - that's an obvious shot-in-the-foot type of thing that we can easily 
protect users from. Yes, PostgreSQL and H2 chose a different behavior - but we 
don't have to follow.
On the other hand, when the index type differs it seems reasonable to allow 
creating two indexes with unspecified name. We can handled that without a UUID 
though - just add an index type to the generated name.

> "IllegalStateException: Duplicate Key" on node join
> ---
>
> Key: IGNITE-8728
> URL: https://issues.apache.org/jira/browse/IGNITE-8728
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.7
>Reporter: Mahesh Renduchintala
>Assignee: Stanislav Lukyanov
>Priority: Critical
> Fix For: 2.8
>
> Attachments: NS1_ignite-9676df15.0.log, NS2_ignite-7cfc8008.0.log, 
> node-config.xml
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> I have two nodes on which we have 3 tables which are partitioned.  Index are 
> also built on these tables. 
> For 24 hours caches work fine.  The tables are definitely distributed across 
> both the nodes
> Node 2 reboots due to some issue - goes out of the baseline - comes back and 
> joins the baseline.  Other baseline nodes crash and in the logs we see 
> duplicate Key error
> [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> accepted incoming connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> spawning a new thread for connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] Started 
> serving remote node connection [rmtAddr=/192.168.1.7:45102, rmtPort=45102]
>  [10:38:35,451][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] 
> Finished serving remote node connection [rmtAddr=/192.168.1.7:45102, 
> rmtPort=45102
>  [10:38:35,457][SEVERE][tcp-disco-msg-worker-#3|#3][TcpDiscoverySpi] 
> TcpDiscoverSpi's message worker thread failed abnormally. Stopping the node 
> in order to prevent cluster wide instability.
>  *java.lang.IllegalStateException: Duplicate key*
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>  [10:38:35,459][SEVERE][tcp-disco-msg-worker-#3|#3][] Critical system error 
> detected. Will be handled accordingly to configured handler [hnd=class 
> o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext 
> [type=SYSTEM_WORKER_TERMINATION,

[jira] [Commented] (IGNITE-8728) "IllegalStateException: Duplicate Key" on node join

2019-02-01 Thread Vladimir Ozerov (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758422#comment-16758422
 ] 

Vladimir Ozerov commented on IGNITE-8728:
-

[~slukyanov], 
As discussed personally, let's do the following:
# Do not transform anything for {{QueryEntity}} in configuration
# For DDL generate index name approximately as follows: 
{code}
String idxName = (tableName + "_" + UUID.randomUUID().replace("-", 
"_")).toUpper();
{code}

Decided to remove column names from such and index, as we have 255 char limit, 
which might be exceeded should column names are appended

> "IllegalStateException: Duplicate Key" on node join
> ---
>
> Key: IGNITE-8728
> URL: https://issues.apache.org/jira/browse/IGNITE-8728
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.7
>Reporter: Mahesh Renduchintala
>Assignee: Stanislav Lukyanov
>Priority: Critical
> Attachments: NS1_ignite-9676df15.0.log, NS2_ignite-7cfc8008.0.log, 
> node-config.xml
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> I have two nodes on which we have 3 tables which are partitioned.  Index are 
> also built on these tables. 
> For 24 hours caches work fine.  The tables are definitely distributed across 
> both the nodes
> Node 2 reboots due to some issue - goes out of the baseline - comes back and 
> joins the baseline.  Other baseline nodes crash and in the logs we see 
> duplicate Key error
> [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> accepted incoming connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> spawning a new thread for connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] Started 
> serving remote node connection [rmtAddr=/192.168.1.7:45102, rmtPort=45102]
>  [10:38:35,451][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] 
> Finished serving remote node connection [rmtAddr=/192.168.1.7:45102, 
> rmtPort=45102
>  [10:38:35,457][SEVERE][tcp-disco-msg-worker-#3|#3][TcpDiscoverySpi] 
> TcpDiscoverSpi's message worker thread failed abnormally. Stopping the node 
> in order to prevent cluster wide instability.
>  *java.lang.IllegalStateException: Duplicate key*
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>  [10:38:35,459][SEVERE][tcp-disco-msg-worker-#3|#3][] Critical system error 
> detected. Will be handled accordingly to configured handler [hnd=class 
> o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext 
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.IllegalStateException: 
> Duplicate key]]
> java.lang.IllegalStateException: Duplicate key
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
>

[jira] [Commented] (IGNITE-8728) "IllegalStateException: Duplicate Key" on node join

2019-02-01 Thread Stanislav Lukyanov (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758276#comment-16758276
 ] 

Stanislav Lukyanov commented on IGNITE-8728:


[~vozerov] Thanks for the comments!

1. Two simultaneous `CREATE INDEX ON FOO (A)` will currently create the same 
index as well. The only difference is that without the fix the name in the 
`QueryIndex` will be `null` and with the fix it will be the name that Ignite 
generates. So the current behavior is not changed. I can agree that it is 
somewhat awkward (not only because of the lack of UUID, but also because, for 
example, IGNITE-11146), but let's deal with that separately.
2. The old exception wasn't very helpful - seems like it wasn't expected to 
ever be thrown, so I've added more info to that. The exception type also wasn't 
a good fit - AFAIK `IllegalStateException` is supposed to be an API exception 
(like "you're calling this API in a wrong state"), not signal that the system 
has went into an unexpected state because of a bug. In other words, "illegal 
state" in the name of the exception means "a normal state that is illegal for 
this operation", not "an erroneous state which system shouldn't be in". There 
isn't a standard exception type for this (except for `AssertionError` I guess, 
but `Error` is tricky to throw), so I just chose our generic `IgniteException`.

> "IllegalStateException: Duplicate Key" on node join
> ---
>
> Key: IGNITE-8728
> URL: https://issues.apache.org/jira/browse/IGNITE-8728
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.7
>Reporter: Mahesh Renduchintala
>Assignee: Stanislav Lukyanov
>Priority: Critical
> Attachments: NS1_ignite-9676df15.0.log, NS2_ignite-7cfc8008.0.log, 
> node-config.xml
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> I have two nodes on which we have 3 tables which are partitioned.  Index are 
> also built on these tables. 
> For 24 hours caches work fine.  The tables are definitely distributed across 
> both the nodes
> Node 2 reboots due to some issue - goes out of the baseline - comes back and 
> joins the baseline.  Other baseline nodes crash and in the logs we see 
> duplicate Key error
> [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> accepted incoming connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> spawning a new thread for connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] Started 
> serving remote node connection [rmtAddr=/192.168.1.7:45102, rmtPort=45102]
>  [10:38:35,451][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] 
> Finished serving remote node connection [rmtAddr=/192.168.1.7:45102, 
> rmtPort=45102
>  [10:38:35,457][SEVERE][tcp-disco-msg-worker-#3|#3][TcpDiscoverySpi] 
> TcpDiscoverSpi's message worker thread failed abnormally. Stopping the node 
> in order to prevent cluster wide instability.
>  *java.lang.IllegalStateException: Duplicate key*
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>  [10:38:35,459][SEVERE][tcp-disco-msg-worker-#3|#3][] Critical system error 
> detected. Will be handled accordingly to configured handler [hnd=class 
> o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext 
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.IllegalStateException: 
> Duplicate key]]
> java.lang.IllegalStateException: Duplicate key
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at

[jira] [Commented] (IGNITE-8728) "IllegalStateException: Duplicate Key" on node join

2019-01-29 Thread Ignite TC Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754894#comment-16754894
 ] 

Ignite TC Bot commented on IGNITE-8728:
---

{panel:title=-- Run :: All: No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=2929323buildTypeId=IgniteTests24Java8_RunAll]

> "IllegalStateException: Duplicate Key" on node join
> ---
>
> Key: IGNITE-8728
> URL: https://issues.apache.org/jira/browse/IGNITE-8728
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.7
>Reporter: Mahesh Renduchintala
>Assignee: Stanislav Lukyanov
>Priority: Critical
> Attachments: NS1_ignite-9676df15.0.log, NS2_ignite-7cfc8008.0.log, 
> node-config.xml
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> I have two nodes on which we have 3 tables which are partitioned.  Index are 
> also built on these tables. 
> For 24 hours caches work fine.  The tables are definitely distributed across 
> both the nodes
> Node 2 reboots due to some issue - goes out of the baseline - comes back and 
> joins the baseline.  Other baseline nodes crash and in the logs we see 
> duplicate Key error
> [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> accepted incoming connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> spawning a new thread for connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] Started 
> serving remote node connection [rmtAddr=/192.168.1.7:45102, rmtPort=45102]
>  [10:38:35,451][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] 
> Finished serving remote node connection [rmtAddr=/192.168.1.7:45102, 
> rmtPort=45102
>  [10:38:35,457][SEVERE][tcp-disco-msg-worker-#3|#3][TcpDiscoverySpi] 
> TcpDiscoverSpi's message worker thread failed abnormally. Stopping the node 
> in order to prevent cluster wide instability.
>  *java.lang.IllegalStateException: Duplicate key*
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>  [10:38:35,459][SEVERE][tcp-disco-msg-worker-#3|#3][] Critical system error 
> detected. Will be handled accordingly to configured handler [hnd=class 
> o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext 
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.IllegalStateException: 
> Duplicate key]]
> java.lang.IllegalStateException: Duplicate key
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
>

[jira] [Commented] (IGNITE-8728) "IllegalStateException: Duplicate Key" on node join

2019-01-29 Thread Stanislav Lukyanov (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754896#comment-16754896
 ] 

Stanislav Lukyanov commented on IGNITE-8728:


[~amashenkov], Done. Can you take another look?

> "IllegalStateException: Duplicate Key" on node join
> ---
>
> Key: IGNITE-8728
> URL: https://issues.apache.org/jira/browse/IGNITE-8728
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.7
>Reporter: Mahesh Renduchintala
>Assignee: Stanislav Lukyanov
>Priority: Critical
> Attachments: NS1_ignite-9676df15.0.log, NS2_ignite-7cfc8008.0.log, 
> node-config.xml
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> I have two nodes on which we have 3 tables which are partitioned.  Index are 
> also built on these tables. 
> For 24 hours caches work fine.  The tables are definitely distributed across 
> both the nodes
> Node 2 reboots due to some issue - goes out of the baseline - comes back and 
> joins the baseline.  Other baseline nodes crash and in the logs we see 
> duplicate Key error
> [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> accepted incoming connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> spawning a new thread for connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] Started 
> serving remote node connection [rmtAddr=/192.168.1.7:45102, rmtPort=45102]
>  [10:38:35,451][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] 
> Finished serving remote node connection [rmtAddr=/192.168.1.7:45102, 
> rmtPort=45102
>  [10:38:35,457][SEVERE][tcp-disco-msg-worker-#3|#3][TcpDiscoverySpi] 
> TcpDiscoverSpi's message worker thread failed abnormally. Stopping the node 
> in order to prevent cluster wide instability.
>  *java.lang.IllegalStateException: Duplicate key*
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>  [10:38:35,459][SEVERE][tcp-disco-msg-worker-#3|#3][] Critical system error 
> detected. Will be handled accordingly to configured handler [hnd=class 
> o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext 
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.IllegalStateException: 
> Duplicate key]]
> java.lang.IllegalStateException: Duplicate key
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>  [10:38:35,460][SEVERE][tcp-disco-msg-worker-#3|#3][] JVM will

[jira] [Commented] (IGNITE-8728) "IllegalStateException: Duplicate Key" on node join

2019-01-25 Thread Stanislav Lukyanov (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752247#comment-16752247
 ] 

Stanislav Lukyanov commented on IGNITE-8728:


[~amashenkov] Thanks! I didn't know we have that. Added the check for both JDBC 
and Java.
I guess the change is local to the test, so no need to restart TC. I've checked 
the new code locally and it passes.

Can you take a look again?

> "IllegalStateException: Duplicate Key" on node join
> ---
>
> Key: IGNITE-8728
> URL: https://issues.apache.org/jira/browse/IGNITE-8728
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.7
>Reporter: Mahesh Renduchintala
>Assignee: Stanislav Lukyanov
>Priority: Critical
> Attachments: NS1_ignite-9676df15.0.log, NS2_ignite-7cfc8008.0.log, 
> node-config.xml
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> I have two nodes on which we have 3 tables which are partitioned.  Index are 
> also built on these tables. 
> For 24 hours caches work fine.  The tables are definitely distributed across 
> both the nodes
> Node 2 reboots due to some issue - goes out of the baseline - comes back and 
> joins the baseline.  Other baseline nodes crash and in the logs we see 
> duplicate Key error
> [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> accepted incoming connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> spawning a new thread for connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] Started 
> serving remote node connection [rmtAddr=/192.168.1.7:45102, rmtPort=45102]
>  [10:38:35,451][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] 
> Finished serving remote node connection [rmtAddr=/192.168.1.7:45102, 
> rmtPort=45102
>  [10:38:35,457][SEVERE][tcp-disco-msg-worker-#3|#3][TcpDiscoverySpi] 
> TcpDiscoverSpi's message worker thread failed abnormally. Stopping the node 
> in order to prevent cluster wide instability.
>  *java.lang.IllegalStateException: Duplicate key*
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>  [10:38:35,459][SEVERE][tcp-disco-msg-worker-#3|#3][] Critical system error 
> detected. Will be handled accordingly to configured handler [hnd=class 
> o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext 
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.IllegalStateException: 
> Duplicate key]]
> java.lang.IllegalStateException: Duplicate key
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
>

[jira] [Commented] (IGNITE-8728) "IllegalStateException: Duplicate Key" on node join

2019-01-25 Thread Andrew Mashenkov (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752249#comment-16752249
 ] 

Andrew Mashenkov commented on IGNITE-8728:
--

Let's make full TC run as bot visa is quite outdated.
There are number of changes in QueryProcessor and other internal classes that 
were not tested on TC.

> "IllegalStateException: Duplicate Key" on node join
> ---
>
> Key: IGNITE-8728
> URL: https://issues.apache.org/jira/browse/IGNITE-8728
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.7
>Reporter: Mahesh Renduchintala
>Assignee: Stanislav Lukyanov
>Priority: Critical
> Attachments: NS1_ignite-9676df15.0.log, NS2_ignite-7cfc8008.0.log, 
> node-config.xml
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> I have two nodes on which we have 3 tables which are partitioned.  Index are 
> also built on these tables. 
> For 24 hours caches work fine.  The tables are definitely distributed across 
> both the nodes
> Node 2 reboots due to some issue - goes out of the baseline - comes back and 
> joins the baseline.  Other baseline nodes crash and in the logs we see 
> duplicate Key error
> [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> accepted incoming connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> spawning a new thread for connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] Started 
> serving remote node connection [rmtAddr=/192.168.1.7:45102, rmtPort=45102]
>  [10:38:35,451][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] 
> Finished serving remote node connection [rmtAddr=/192.168.1.7:45102, 
> rmtPort=45102
>  [10:38:35,457][SEVERE][tcp-disco-msg-worker-#3|#3][TcpDiscoverySpi] 
> TcpDiscoverSpi's message worker thread failed abnormally. Stopping the node 
> in order to prevent cluster wide instability.
>  *java.lang.IllegalStateException: Duplicate key*
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>  [10:38:35,459][SEVERE][tcp-disco-msg-worker-#3|#3][] Critical system error 
> detected. Will be handled accordingly to configured handler [hnd=class 
> o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext 
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.IllegalStateException: 
> Duplicate key]]
> java.lang.IllegalStateException: Duplicate key
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at

[jira] [Commented] (IGNITE-8728) "IllegalStateException: Duplicate Key" on node join

2019-01-25 Thread Andrew Mashenkov (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752233#comment-16752233
 ] 

Andrew Mashenkov commented on IGNITE-8728:
--

[~slukyanov], I've left a comment to the PR.

I'd think SQL exception should return some meaningful error code. E.g. 
INDEX_ALREADY_EXISTS. 
What do you think?

> "IllegalStateException: Duplicate Key" on node join
> ---
>
> Key: IGNITE-8728
> URL: https://issues.apache.org/jira/browse/IGNITE-8728
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.7
>Reporter: Mahesh Renduchintala
>Assignee: Stanislav Lukyanov
>Priority: Critical
> Attachments: NS1_ignite-9676df15.0.log, NS2_ignite-7cfc8008.0.log, 
> node-config.xml
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> I have two nodes on which we have 3 tables which are partitioned.  Index are 
> also built on these tables. 
> For 24 hours caches work fine.  The tables are definitely distributed across 
> both the nodes
> Node 2 reboots due to some issue - goes out of the baseline - comes back and 
> joins the baseline.  Other baseline nodes crash and in the logs we see 
> duplicate Key error
> [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> accepted incoming connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> spawning a new thread for connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] Started 
> serving remote node connection [rmtAddr=/192.168.1.7:45102, rmtPort=45102]
>  [10:38:35,451][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] 
> Finished serving remote node connection [rmtAddr=/192.168.1.7:45102, 
> rmtPort=45102
>  [10:38:35,457][SEVERE][tcp-disco-msg-worker-#3|#3][TcpDiscoverySpi] 
> TcpDiscoverSpi's message worker thread failed abnormally. Stopping the node 
> in order to prevent cluster wide instability.
>  *java.lang.IllegalStateException: Duplicate key*
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>  [10:38:35,459][SEVERE][tcp-disco-msg-worker-#3|#3][] Critical system error 
> detected. Will be handled accordingly to configured handler [hnd=class 
> o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext 
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.IllegalStateException: 
> Duplicate key]]
> java.lang.IllegalStateException: Duplicate key
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at

[jira] [Commented] (IGNITE-8728) "IllegalStateException: Duplicate Key" on node join

2019-01-25 Thread Ignite TC Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752127#comment-16752127
 ] 

Ignite TC Bot commented on IGNITE-8728:
---

{panel:title=-- Run :: All: No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=2885542buildTypeId=IgniteTests24Java8_RunAll]

> "IllegalStateException: Duplicate Key" on node join
> ---
>
> Key: IGNITE-8728
> URL: https://issues.apache.org/jira/browse/IGNITE-8728
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.7
>Reporter: Mahesh Renduchintala
>Assignee: Stanislav Lukyanov
>Priority: Critical
> Attachments: NS1_ignite-9676df15.0.log, NS2_ignite-7cfc8008.0.log, 
> node-config.xml
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I have two nodes on which we have 3 tables which are partitioned.  Index are 
> also built on these tables. 
> For 24 hours caches work fine.  The tables are definitely distributed across 
> both the nodes
> Node 2 reboots due to some issue - goes out of the baseline - comes back and 
> joins the baseline.  Other baseline nodes crash and in the logs we see 
> duplicate Key error
> [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> accepted incoming connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> spawning a new thread for connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] Started 
> serving remote node connection [rmtAddr=/192.168.1.7:45102, rmtPort=45102]
>  [10:38:35,451][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] 
> Finished serving remote node connection [rmtAddr=/192.168.1.7:45102, 
> rmtPort=45102
>  [10:38:35,457][SEVERE][tcp-disco-msg-worker-#3|#3][TcpDiscoverySpi] 
> TcpDiscoverSpi's message worker thread failed abnormally. Stopping the node 
> in order to prevent cluster wide instability.
>  *java.lang.IllegalStateException: Duplicate key*
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>  [10:38:35,459][SEVERE][tcp-disco-msg-worker-#3|#3][] Critical system error 
> detected. Will be handled accordingly to configured handler [hnd=class 
> o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext 
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.IllegalStateException: 
> Duplicate key]]
> java.lang.IllegalStateException: Duplicate key
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
>

[jira] [Commented] (IGNITE-8728) "IllegalStateException: Duplicate Key" on node join

2019-01-24 Thread Mahesh Renduchintala (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751308#comment-16751308
 ] 

Mahesh Renduchintala commented on IGNITE-8728:
--

Stan,  The work around is acceptable.  The fix seems to be coming in 2.8, which 
is great. You may close the ticket, if you wish.

 

> "IllegalStateException: Duplicate Key" on node join
> ---
>
> Key: IGNITE-8728
> URL: https://issues.apache.org/jira/browse/IGNITE-8728
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.7
>Reporter: Mahesh Renduchintala
>Assignee: Stanislav Lukyanov
>Priority: Critical
> Attachments: NS1_ignite-9676df15.0.log, NS2_ignite-7cfc8008.0.log, 
> node-config.xml
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I have two nodes on which we have 3 tables which are partitioned.  Index are 
> also built on these tables. 
> For 24 hours caches work fine.  The tables are definitely distributed across 
> both the nodes
> Node 2 reboots due to some issue - goes out of the baseline - comes back and 
> joins the baseline.  Other baseline nodes crash and in the logs we see 
> duplicate Key error
> [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> accepted incoming connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> spawning a new thread for connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] Started 
> serving remote node connection [rmtAddr=/192.168.1.7:45102, rmtPort=45102]
>  [10:38:35,451][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] 
> Finished serving remote node connection [rmtAddr=/192.168.1.7:45102, 
> rmtPort=45102
>  [10:38:35,457][SEVERE][tcp-disco-msg-worker-#3|#3][TcpDiscoverySpi] 
> TcpDiscoverSpi's message worker thread failed abnormally. Stopping the node 
> in order to prevent cluster wide instability.
>  *java.lang.IllegalStateException: Duplicate key*
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>  [10:38:35,459][SEVERE][tcp-disco-msg-worker-#3|#3][] Critical system error 
> detected. Will be handled accordingly to configured handler [hnd=class 
> o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext 
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.IllegalStateException: 
> Duplicate key]]
> java.lang.IllegalStateException: Duplicate key
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>

[jira] [Commented] (IGNITE-8728) "IllegalStateException: Duplicate Key" on node join

2019-01-23 Thread Stanislav Lukyanov (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750261#comment-16750261
 ] 

Stanislav Lukyanov commented on IGNITE-8728:


Reproduced this and have a fix.
The scenario is:
- Start two nodes, persistence enabled
- Create two unnamed indexes on the same table
- Restart one node

When the restarting node joins the other node goes over its own indexes, 
notices that there are two indexes with the same name (null) and fails.

The fix is to set the auto-generated index name in the QueryIndex instead of 
leaving it null.

> "IllegalStateException: Duplicate Key" on node join
> ---
>
> Key: IGNITE-8728
> URL: https://issues.apache.org/jira/browse/IGNITE-8728
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.7
>Reporter: Mahesh Renduchintala
>Assignee: Stanislav Lukyanov
>Priority: Critical
> Attachments: NS1_ignite-9676df15.0.log, NS2_ignite-7cfc8008.0.log, 
> node-config.xml
>
>
> I have two nodes on which we have 3 tables which are partitioned.  Index are 
> also built on these tables. 
> For 24 hours caches work fine.  The tables are definitely distributed across 
> both the nodes
> Node 2 reboots due to some issue - goes out of the baseline - comes back and 
> joins the baseline.  Other baseline nodes crash and in the logs we see 
> duplicate Key error
> [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> accepted incoming connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> spawning a new thread for connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] Started 
> serving remote node connection [rmtAddr=/192.168.1.7:45102, rmtPort=45102]
>  [10:38:35,451][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] 
> Finished serving remote node connection [rmtAddr=/192.168.1.7:45102, 
> rmtPort=45102
>  [10:38:35,457][SEVERE][tcp-disco-msg-worker-#3|#3][TcpDiscoverySpi] 
> TcpDiscoverSpi's message worker thread failed abnormally. Stopping the node 
> in order to prevent cluster wide instability.
>  *java.lang.IllegalStateException: Duplicate key*
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>  [10:38:35,459][SEVERE][tcp-disco-msg-worker-#3|#3][] Critical system error 
> detected. Will be handled accordingly to configured handler [hnd=class 
> o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext 
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.IllegalStateException: 
> Duplicate key]]
> java.lang.IllegalStateException: Duplicate key
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
>

[jira] [Commented] (IGNITE-8728) "IllegalStateException: Duplicate Key" on node join

2019-01-16 Thread Mahesh Renduchintala (JIRA) 


[ 
https://issues.apache.org/jira/browse/IGNITE-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743794#comment-16743794
 ] 

Mahesh Renduchintala commented on IGNITE-8728:
--

Stan, thanks for the visibility. 
 # Over the last year, we move from various versions of ignite 2.4, 2.5 to 2.7. 
I always keep work folder in tact. 
 # Over a period of development, we might have tried to create index a second 
or many times on the same column on which an index already existed. Now, could 
that cause a confusion at ignite level, especially in a multi-node scenario? 
Was something out of sync? Was a check missing?
 # Over a period of time, we dropped the table several times and recreated the 
table several times and indexes. Was something stable left out in work folder. 
 # We always used 2 or more nodes. 
 # Over a period of time, we saw issues with index creation as well. My 
colleague posted another strange behaviour with index creation. See the issue 
here, 
[http://apache-ignite-users.70518.x6.nabble.com/Failing-to-create-index-on-Ignite-table-column-td26252.html#a26258]
 # Summary is if we don't give index names the ignite gives exceptions.         
                        

 

Something seems to be wrong with Ignite index handling in multi-node 
environment. 

Regarding your point 2, absolutely, makes sense not to crash the node. We have 
about 100GB data (tables) on ignite and the only work around right now seems to 
be 

Boot node 1. Keep its work folder. 

Boot node 2 after removing its work folder

This scenario though does not produce an issue, gives the cluster a down-time 
of about 1-2 hours. 

 

 

> "IllegalStateException: Duplicate Key" on node join
> ---
>
> Key: IGNITE-8728
> URL: https://issues.apache.org/jira/browse/IGNITE-8728
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.7
>Reporter: Mahesh Renduchintala
>Priority: Critical
> Attachments: NS1_ignite-9676df15.0.log, NS2_ignite-7cfc8008.0.log, 
> node-config.xml
>
>
> I have two nodes on which we have 3 tables which are partitioned.  Index are 
> also built on these tables. 
> For 24 hours caches work fine.  The tables are definitely distributed across 
> both the nodes
> Node 2 reboots due to some issue - goes out of the baseline - comes back and 
> joins the baseline.  Other baseline nodes crash and in the logs we see 
> duplicate Key error
> [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> accepted incoming connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-srvr-#2|#2][TcpDiscoverySpi] TCP discovery 
> spawning a new thread for connection [rmtAddr=/192.168.1.7, rmtPort=45102]
>  [10:38:35,437][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] Started 
> serving remote node connection [rmtAddr=/192.168.1.7:45102, rmtPort=45102]
>  [10:38:35,451][INFO][tcp-disco-sock-reader-#12|#12][TcpDiscoverySpi] 
> Finished serving remote node connection [rmtAddr=/192.168.1.7:45102, 
> rmtPort=45102
>  [10:38:35,457][SEVERE][tcp-disco-msg-worker-#3|#3][TcpDiscoverySpi] 
> TcpDiscoverSpi's message worker thread failed abnormally. Stopping the node 
> in order to prevent cluster wide instability.
>  *java.lang.IllegalStateException: Duplicate key*
> at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:223)
> at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:174)
> at 
> org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)
> at 
> org.apache.ignite.internal.processors.cache.DynamicCacheDescriptor.makeSchemaPatch(DynamicCacheDescriptor.java:360)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.validateNode(GridCacheProcessor.java:2536)
> at 
> org.apache.ignite.internal.managers.GridManagerAdapter$1.validateNode(GridManagerAdapter.java:566)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processJoinRequestMessage(ServerImpl.java:3629)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2736)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at 
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>  [10:38:35,459][SEVERE][tcp-disco-msg-worker-#3|#3][] Critical system error 
> detected. Will be handled accordingly to configured handler [hnd=class 
> o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext 
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.IllegalStateException: 
> Duplicate key]]
>