[jira] [Created] (IGNITE-14504) ClusterGroupEmptyException: Cluster group is empty error after client reconnect

[Andrey Aleksandrov (Jira)]

Andrey Aleksandrov created IGNITE-14504:
---

 Summary: ClusterGroupEmptyException: Cluster group is empty error 
after client reconnect
 Key: IGNITE-14504
 URL: https://issues.apache.org/jira/browse/IGNITE-14504
 Project: Ignite
  Issue Type: Bug
  Components: messaging
Affects Versions: 2.10
Reporter: Andrey Aleksandrov
Assignee: Andrey Aleksandrov
 Fix For: 2.11
 Attachments: SendMessageAfterClientReconnect.java

Please run the attached test.

It will produce the following exception:

Exception in thread "main" class 
org.apache.ignite.cluster.ClusterGroupEmptyException: Cluster group is empty.
at org.apache.ignite.internal.util.IgniteUtils$6.apply(IgniteUtils.java:927)
at org.apache.ignite.internal.util.IgniteUtils$6.apply(IgniteUtils.java:925)
at 
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1083)
at 
org.apache.ignite.internal.IgniteMessagingImpl.send0(IgniteMessagingImpl.java:105)
at 
org.apache.ignite.internal.IgniteMessagingImpl.send(IgniteMessagingImpl.java:81)
at npe.IgnitePartitioningTest.main(IgnitePartitioningTest.java:110)
Caused by: class 
org.apache.ignite.internal.cluster.ClusterGroupEmptyCheckedException: Cluster 
group is empty.
at 
org.apache.ignite.internal.util.IgniteUtils.emptyTopologyException(IgniteUtils.java:5106)
at 
org.apache.ignite.internal.IgniteMessagingImpl.send0(IgniteMessagingImpl.java:100)
... 2 more

Fix:

change

return new ClusterGroupAdapter(ctx, null, 
Collections.singleton(cfg.getNodeId()));



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-14499) TcpDiscoveryVM IP Finder: thick client does not try to resolve the FQDN again during reconnection

[Andrey Aleksandrov (Jira)]

Andrey Aleksandrov created IGNITE-14499:
---

 Summary: TcpDiscoveryVM IP Finder: thick client does not try to 
resolve the FQDN again during reconnection
 Key: IGNITE-14499
 URL: https://issues.apache.org/jira/browse/IGNITE-14499
 Project: Ignite
  Issue Type: Bug
  Components: messaging
Affects Versions: 2.10
Reporter: Andrey Aleksandrov


Client reconnect logic has for the following scenarios:

1)You have 3 servers. FQDN name can be resolved to one of them (but only to 
reachable server node).
2)You have 3 servers. FQDN name will be resolved to the list of the reachable 
server IPs.

Results:

Thick clients do not try to resolve FQDN once again upon connection failure. 
Reconnect doesn't work.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13381) ClusterGroupEmptyException: Cluster group is empty error after client reconnect

[Andrey Aleksandrov (Jira)]

Andrey Aleksandrov created IGNITE-13381:
---

 Summary: ClusterGroupEmptyException: Cluster group is empty error 
after client reconnect
 Key: IGNITE-13381
 URL: https://issues.apache.org/jira/browse/IGNITE-13381
 Project: Ignite
  Issue Type: Bug
  Components: networking
Affects Versions: 2.8.1
Reporter: Andrey Aleksandrov
Assignee: Andrey Aleksandrov
 Fix For: 2.9
 Attachments: SendMessageAfterClientReconnect.java

Please run the attached test.

It will produce the following exception:

Exception in thread "main" class 
org.apache.ignite.cluster.ClusterGroupEmptyException: Cluster group is empty.
at org.apache.ignite.internal.util.IgniteUtils$6.apply(IgniteUtils.java:927)
at org.apache.ignite.internal.util.IgniteUtils$6.apply(IgniteUtils.java:925)
at 
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1083)
at 
org.apache.ignite.internal.IgniteMessagingImpl.send0(IgniteMessagingImpl.java:105)
at 
org.apache.ignite.internal.IgniteMessagingImpl.send(IgniteMessagingImpl.java:81)
at npe.IgnitePartitioningTest.main(IgnitePartitioningTest.java:110)
Caused by: class 
org.apache.ignite.internal.cluster.ClusterGroupEmptyCheckedException: Cluster 
group is empty.
at 
org.apache.ignite.internal.util.IgniteUtils.emptyTopologyException(IgniteUtils.java:5106)
at 
org.apache.ignite.internal.IgniteMessagingImpl.send0(IgniteMessagingImpl.java:100)
... 2 more

Fix:

change

return new ClusterGroupAdapter(ctx, null, 
Collections.singleton(cfg.getNodeId()));

on

return new ClusterGroupAdapter(ctx, null, 
Collections.singleton(ctx.discovery().localNode().id()));



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-12637) IgniteSparkSession doesn't start the clients on really distributed cluster

[Andrey Aleksandrov (Jira)]

Andrey Aleksandrov created IGNITE-12637:
---

 Summary: IgniteSparkSession doesn't start the clients on really 
distributed cluster
 Key: IGNITE-12637
 URL: https://issues.apache.org/jira/browse/IGNITE-12637
 Project: Ignite
  Issue Type: Bug
  Components: spark
Affects Versions: 2.7.6
Reporter: Andrey Aleksandrov
 Fix For: 2.8


Next code:

IgniteSparkSession igniteSession = IgniteSparkSession.builder()
   .appName("Spark Ignite example")
   .igniteConfig(configPath)
   .getOrCreate();

Throws:

class org.apache.ignite.IgniteIllegalStateException: Ignite instance with 
provided name doesn't exist. Did you call Ignition.start(..) to start an Ignite 
instance? [name=grid]

Client config was located in all Spark nodes at the same place. 

When I ran these tests on the same host with several local executors then it 
worked. But if executors were located on different hosts then it didn't.

DataFrame API works fine with the same config.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-12170) DOC: Advanced security should be better documented

[Andrey Aleksandrov (Jira)]

Andrey Aleksandrov created IGNITE-12170:
---

 Summary: DOC: Advanced security should be better documented
 Key: IGNITE-12170
 URL: https://issues.apache.org/jira/browse/IGNITE-12170
 Project: Ignite
  Issue Type: Bug
  Components: documentation
Affects Versions: 2.7.5
Reporter: Andrey Aleksandrov
 Fix For: 2.8


[https://apacheignite.readme.io/docs/advanced-security#section-enable-authentication]

After applying the example, in the server log you can see:

[12:44:11] Security status [authentication=off, tls/ssl=off]

After that, I was able to connect the client without any username/password.

So looks like we should add into documentation the information about how it 
works. 

I checked that it worked with thin clients and web console.  But for user it 
can be not clear that authentication between nodes doesn' work out of the boxes.

 

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (IGNITE-12159) Ignite spark doesn't support Alter Column syntax

[Andrey Aleksandrov (Jira)]

Andrey Aleksandrov created IGNITE-12159:
---

 Summary: Ignite spark doesn't support Alter Column syntax
 Key: IGNITE-12159
 URL: https://issues.apache.org/jira/browse/IGNITE-12159
 Project: Ignite
  Issue Type: Bug
  Components: spark
Affects Versions: 2.7.5
Reporter: Andrey Aleksandrov
 Fix For: 2.8


Steps:

1)Start the server
2)Run next SQL commands

CREATE TABLE person (id LONG, name VARCHAR(64), age LONG, city_id DOUBLE, 
zip_code LONG, PRIMARY KEY (name)) WITH "backups=1"
ALTER TABLE person ADD COLUMN (first_name VARCHAR(64), last_name VARCHAR(64))

3)After that run next spark code:

       String configPath = "client.xml";
      
   SparkConf sparkConf = new SparkConf()
   .setMaster("local")
   .setAppName("Example"); 

      IgniteSparkSession.builder()
   .appName("Spark Ignite catalog example")
   .master("local")
   .config("ignite.disableSparkSQLOptimization", true)
   .igniteConfig(configPath)
   .getOrCreate();
 
      Dataset df2 = igniteSession.sql("select * from person1");   
      df2.show();

The result will contain only 5 columns from CREATE TABLE call.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (IGNITE-12026) DOC: JavaDoc for MemoryEventStorageSpi contains incorrect example

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-12026:
---

 Summary: DOC: JavaDoc for MemoryEventStorageSpi contains incorrect 
example
 Key: IGNITE-12026
 URL: https://issues.apache.org/jira/browse/IGNITE-12026
 Project: Ignite
  Issue Type: Bug
  Components: documentation
Affects Versions: 2.7.5
Reporter: Andrey Aleksandrov
 Fix For: 2.8


h2. Spring Example

MemoryEventStorageSpi can be configured from Spring XML configuration file:
 
 ...
 
 
 
 
 
 ...
 
 

But MemoryEventStorageSpi can't be applied to discoverySpi. It should be 
"eventStorageSpi"




 

 



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Created] (IGNITE-11966) Using AdaptiveLoadBalancingSpi without IgniteConfiguration.setIncludeEventTypes(EventType.EVT_TASK_FINISHED, EventType.EVT_TASK_FAILED) leads to memory leak

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-11966:
---

 Summary: Using AdaptiveLoadBalancingSpi without 
IgniteConfiguration.setIncludeEventTypes(EventType.EVT_TASK_FINISHED,  
EventType.EVT_TASK_FAILED) leads to memory leak
 Key: IGNITE-11966
 URL: https://issues.apache.org/jira/browse/IGNITE-11966
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.7.5
Reporter: Andrey Aleksandrov
 Fix For: 2.8
 Attachments: Top_Consumers.zip

You can use in debug mode any of tests related to AdaptiveLoadBalancingSpi like 
GridAdaptiveLoadBalancingSpiSelfTest.

In case if you don't subscribe on EventType.EVT_TASK_FINISHED,  
EventType.EVT_TASK_FAILED then next code will be never called:



case EVT_TASK_FINISHED:
case EVT_TASK_FAILED: {
 TaskEvent taskEvt = (TaskEvent)evt;

 taskTops.remove(taskEvt.taskSessionId()); //this

As a result, you will see the growth of memory usage.

[^Top_Consumers.zip]


 if (log.isDebugEnabled())
 log.debug("Removed task topology from topology cache for session: " +
 taskEvt.taskSessionId());

 break;
}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-11965) Pyton client: Expiration policies are missed

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-11965:
---

 Summary: Pyton client: Expiration policies are missed
 Key: IGNITE-11965
 URL: https://issues.apache.org/jira/browse/IGNITE-11965
 Project: Ignite
  Issue Type: Bug
  Components: thin client
Affects Versions: 2.7.5, 2.7
Reporter: Andrey Aleksandrov
 Fix For: 2.8


[https://apacheignite.readme.io/docs/expiry-policies] are missed but 
[https://apacheignite.readme.io/docs/expiry-policies#section-eager-ttl] can be 
set.

Should be added.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-11724) IgniteSpark integration forget to close the IgniteContext and stops the client node in case if error during PairFunction logic

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-11724:
---

 Summary: IgniteSpark integration forget to close the IgniteContext 
and stops the client node in case if error during PairFunction logic 
 Key: IGNITE-11724
 URL: https://issues.apache.org/jira/browse/IGNITE-11724
 Project: Ignite
  Issue Type: Bug
  Components: spark
Affects Versions: 2.7
Reporter: Andrey Aleksandrov
 Fix For: 2.8


Next code could hang in case if PairFunction logic will throw the exception:



JavaPairRDD rdd_records = records.mapToPair(new MapFunction());

JavaIgniteContext igniteContext = new JavaIgniteContext<>(sparkCtx, 
configUrl);

JavaIgniteRDD igniteRdd = igniteContext.fromCache(cacheName);

igniteRdd.savePairs(rdd_records);

Looks like next internal code (saveValues method)should also close the 
IgniteContext in case of an unexpected exception, not only data streamer:

 try {
    it.foreach(value ⇒ {
         val key = affinityKeyFunc(value, node.orNull)

          streamer.addData(key, value)
       })
    }
    finally {
        streamer.close()
    }
 })
}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-11723) IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame)

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-11723:
---

 Summary: IgniteSpark integration should support skipStore option 
for internal dataStreamer (IgniteRdd and Ignite DataFrame)
 Key: IGNITE-11723
 URL: https://issues.apache.org/jira/browse/IGNITE-11723
 Project: Ignite
  Issue Type: Improvement
  Components: spark
Affects Versions: 2.7
Reporter: Andrey Aleksandrov
Assignee: Andrey Aleksandrov
 Fix For: 2.8


At the moment this option can't be set. But this integrations could be used for 
initial data loading also for the caches with cache stores implementation. 

With skipStore option, we could avoid write-through behavior during this 
initial data loading.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-11576) Web console: Documentation for webconsole upgraging from version to vertion should be created (including mongoDB specific)

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-11576:
---

 Summary: Web console: Documentation for webconsole upgraging from 
version to vertion should be created (including mongoDB specific)
 Key: IGNITE-11576
 URL: https://issues.apache.org/jira/browse/IGNITE-11576
 Project: Ignite
  Issue Type: Task
  Components: UI
Affects Versions: 2.7
Reporter: Andrey Aleksandrov
 Fix For: 2.8


It's important to get the documentation about all steps and limitation related 
to upgrading of the web console.

Also in case if upgrading requires changing of the mongoDB version then the 
instructions for backuping and restoring of the data should be added to our 
documentation too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-11084) Change copyrights to 2019

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-11084:
---

 Summary: Change copyrights to 2019
 Key: IGNITE-11084
 URL: https://issues.apache.org/jira/browse/IGNITE-11084
 Project: Ignite
  Issue Type: Task
Reporter: Andrey Aleksandrov
Assignee: Andrey Aleksandrov
 Fix For: 2.8


Change copyrights to 2019



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-10944) Plugin discovery data from PluginProvider.provideDiscoveryData should be available in PluginProvider.validateNewNode

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-10944:
---

 Summary: Plugin discovery data from 
PluginProvider.provideDiscoveryData should be available in 
PluginProvider.validateNewNode
 Key: IGNITE-10944
 URL: https://issues.apache.org/jira/browse/IGNITE-10944
 Project: Ignite
  Issue Type: Improvement
Reporter: Andrey Aleksandrov


*Current  sequence:*
 
1)Server starts (coordinator)
2)Client starts
3)Client sends to the coordinator its discovery data using 
*provideDiscoveryData* method
4)Coordinator start validation of the node using *validateNewNode*
5)The client joins the cluster; otherwise fails
6)Coordinator provides the discovery data to plugin provider in 
*receiveDiscoveryData.* After that this data could be passed to the plugin.

But in case if we require to validate new node using the discovery data from 
*receiveDiscoveryData* method. We should add a possibility to get this data in 
*validateNewNode* method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-10716) Rest clients should call ctx.security().onSessionExpired(subjid) on disconnect to avoid the possible leaking of the memory

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-10716:
---

 Summary: Rest clients should call 
ctx.security().onSessionExpired(subjid) on disconnect to avoid the possible 
leaking of the memory
 Key: IGNITE-10716
 URL: https://issues.apache.org/jira/browse/IGNITE-10716
 Project: Ignite
  Issue Type: Bug
  Components: clients
Affects Versions: 2.7
Reporter: Andrey Aleksandrov
Assignee: Andrey Aleksandrov
 Fix For: 2.8


At the moment ctx.security().onSessionExpired(subjid) didn't called on rest 
session expiration.

It provides the leaking of the memory related to resources that were generated 
during authentification.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-10657) Thin clients (ODBC, JDBC, Java) should call ctx.security().onSessionExpired(subjid) on disconnect to avoid the possible leaking of the memory

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-10657:
---

 Summary: Thin clients (ODBC, JDBC, Java) should call 
ctx.security().onSessionExpired(subjid) on disconnect to avoid the possible 
leaking of the memory
 Key: IGNITE-10657
 URL: https://issues.apache.org/jira/browse/IGNITE-10657
 Project: Ignite
  Issue Type: Bug
  Components: thin client
Affects Versions: 2.7
Reporter: Andrey Aleksandrov
Assignee: Andrey Aleksandrov
 Fix For: 2.8


At the moment all classes that implement 
ClientListenerAbstractConnectionContext doesn't call the 
ctx.security().onSessionExpired(subjid) in disconect.

It provides the leaking of the memory related to resources that were generated 
during authentification.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-10655) .NET client should add peerClassLoading property in IgniteConfiguration class

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-10655:
---

 Summary: .NET client should add peerClassLoading property in 
IgniteConfiguration class
 Key: IGNITE-10655
 URL: https://issues.apache.org/jira/browse/IGNITE-10655
 Project: Ignite
  Issue Type: Improvement
  Components: platforms
Affects Versions: 2.7
Reporter: Andrey Aleksandrov
 Fix For: 2.8


In case if java server nodes in a cluster were set up with peerClassLoading set 
to true then there is no way to start node via .NET client using 
IgniteConfiguration (because the default value is false).

The only way to do it is using of the XML configuration.

Possibility to set this property from .Net client programmatically should be 
added.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-10471) SQL: Alter table to add column sytax doesn't support the user types but drop column can drop such fields

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-10471:
---

 Summary: SQL: Alter table to add column sytax doesn't support the 
user types but drop column can drop such fields
 Key: IGNITE-10471
 URL: https://issues.apache.org/jira/browse/IGNITE-10471
 Project: Ignite
  Issue Type: Improvement
  Components: sql
Affects Versions: 2.6
Reporter: Andrey Aleksandrov
 Fix For: 2.8


Looks like ALTER COLUMN commands could be used only on a subset of supported 
types (that could be used in CREATE TABLE syntax):

[https://apacheignite-sql.readme.io/docs/data-types]

But using query entities we can provide to use a field that has user types.

This fields can be used in 

ALTER TABLE CACHE_NAME DROP COLUMN COLUMN_NAME

But it can't be used as next:

ALTER TABLE CACHE_NAME  ADD COLUMN COLUMN_NAME 'type'

Where type is a user's one.

Possible we should think about supporting POJO and enums.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-10061) Improvement: warning or debug message is required in case if some data in SQL join requests isn't collocated.

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-10061:
---

 Summary: Improvement: warning or debug message is required in case 
if some data in SQL join requests isn't collocated.
 Key: IGNITE-10061
 URL: https://issues.apache.org/jira/browse/IGNITE-10061
 Project: Ignite
  Issue Type: Improvement
  Components: sql
Affects Versions: 2.6
Reporter: Andrey Aleksandrov
 Fix For: 2.8


When data isn't collocated then during executing of the join request will be 
great to get for some warning like "your data isn't collocated"



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9984) Documentation for EVT_MANAGEMENT_TASK_STARTED will be required.

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-9984:
--

 Summary: Documentation for EVT_MANAGEMENT_TASK_STARTED will be 
required.
 Key: IGNITE-9984
 URL: https://issues.apache.org/jira/browse/IGNITE-9984
 Project: Ignite
  Issue Type: Task
  Components: documentation
Affects Versions: 2.8
Reporter: Andrey Aleksandrov
 Fix For: 2.8


New EVT_MANAGEMENT_TASK_STARTED will be added in future release. Documentation 
for it should be added too.

Next information should be added to web console and visor documentation:

EVT_MANAGEMENT_TASK_STARTED provide the possibility to track next tasks that 
could be started by the user from web console and visor during some operations:

+Baseline:+

VisorBaselineTask - Task that will collect information about baseline topology 
and can change its state

+Binaries:+

VisorBinaryMetadataCollectorTask - Task that collects binary metadata.

+Services:+

VisorCancelServiceTask - Task for cancel services with the specified name.

+Metrics:+

VisorComputeResetMetricsTask - Task for cancel services with specified name.

+Caches:+

VisorCacheLostPartitionsTask - Collect list of lost partitions.
VisorCacheResetLostPartitionsTask - Reset lost partitions for caches.
VisorCacheStartTask - Task that start cache or near cache with specified 
configuration.
VisorCacheStopTask - Task that stop specified caches on specified node.
VisorCacheAffinityNodeTask - Task that will find affinity node for key.
VisorCacheModifyTask - Task that modify value in specified cache.
VisorCacheRebalanceTask - Pre-loads caches. Made callable just to conform 
common pattern.
VisorCacheLoadTask - Task to loads caches.
VisorCacheClearTask - Task that clears specified caches on specified node.

+Queries+:

VisorQueryResetMetricsTask - Reset compute grid query metrics.
VisorQueryTask - Task for executing SQL fields query and get the first page of 
results.
VisorQueryCancelTask - Task to cancel queries.

+Computes:+

VisorComputeResetMetricsTask - Reset compute grid metrics.
VisorComputeCancelSessionsTask - Cancels given tasks sessions.

+DEBUG:+

VisorThreadDumpTask - Creates a thread dump.

+IGFS:+

VisorIgfsFormatTask - Format IGFS instance.
VisorIgfsProfilerClearTask - Remove all IGFS profiler logs.
VisorIgfsResetMetricsTask - Resets IGFS metrics.

+LOGS:+

VisorLogSearchTask - Search text matching in logs

+CLUSTER:+

VisorChangeGridActiveStateTask - Task for changing grid active state.
VisorNodeGcTask - Task to run gc on nodes.
VisorNodeRestartTask - Restarts nodes.
VisorNodeStopTask - Stops nodes.

 

{color:#33}

{color}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9975) Possible data loss during starting of the nodes with empty pds

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-9975:
--

 Summary: Possible data loss during starting of the nodes with 
empty pds
 Key: IGNITE-9975
 URL: https://issues.apache.org/jira/browse/IGNITE-9975
 Project: Ignite
  Issue Type: Bug
  Components: persistence
Affects Versions: 2.4
Reporter: Andrey Aleksandrov
Assignee: Alexey Stelmak
 Fix For: 2.7
 Attachments: reproducer.java

Case: 

1)Start 3 data nodes and activate the cluster with cache with 1 backup and 
PartitionLossPolicy.READ_ONLY_SAFE.
2)Start client and add the data to your cache. Stop the client
3)Stop DN2 and clear it pds and val
4)Start DN2. Rebalance will start.
5)During rebalance stop DN3.
6)Start DN3.

At this moment some partitions from DN2 marked as LOST and cache size will be 
less than expected.

7) Run resetLostPartitions(caches).

Now all partitions on DN2 marked as OWNING but cache size is still less than 
expected.

Workaround:

after step 6 do:

7)force rebalance using deactivate/activate methods.
8)wait for completion of rebalance

Now cache size is expected but some partitions from DN2 marked as LOST

9)Run resetLostPartitions(caches).

Now cache size is OK and all partitions from DN2 marked as OWNING.

However, looks like without force rebalance we have data loss here.




 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9914) SubjectId for TaskEvents is incorrect for remote clients

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-9914:
--

 Summary: SubjectId for TaskEvents is incorrect for remote clients
 Key: IGNITE-9914
 URL: https://issues.apache.org/jira/browse/IGNITE-9914
 Project: Ignite
  Issue Type: Bug
  Components: clients
Affects Versions: 2.6
Reporter: Andrey Aleksandrov
Assignee: Andrey Aleksandrov
 Fix For: 2.8


In case if the task was initialized from the remote client as web console then 
taskEvent.subjectId() will return the incorrect value. 

It will contain the subject id for the node where this task was deployed.

Ignite already have the thread local store for subject id but looks like we 
have a problem in GridTaskProcessor:

UUID subjId = getThreadContext(TC_SUBJ_ID);

But it always is null because of the correct value for TC_SUBJ_ID stored in 

Map map = thCtx.get();

So it should be changed to 

UUID subjId = (UUID)map.get(TC_SUBJ_ID);



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9881) Management events for webconsole and visor

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-9881:
--

 Summary: Management events for webconsole and visor
 Key: IGNITE-9881
 URL: https://issues.apache.org/jira/browse/IGNITE-9881
 Project: Ignite
  Issue Type: New Feature
Affects Versions: 2.6
Reporter: Andrey Aleksandrov
Assignee: Andrey Aleksandrov
 Fix For: 2.8


New events for the handling of the tasks that were started on web console or 
visor could be helpful for tracking the user activities.

At the moment all events will be handled as data node events. So it could be 
difficult to understand why they were fired in case if it was done on web 
console or visor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9840) Possible deadlock on transactional future on client node in case of network problems or long GC pauses

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-9840:
--

 Summary: Possible deadlock on transactional future on client node 
in case of network problems or long GC pauses
 Key: IGNITE-9840
 URL: https://issues.apache.org/jira/browse/IGNITE-9840
 Project: Ignite
  Issue Type: Bug
  Components: clients
Affects Versions: 2.6
Reporter: Andrey Aleksandrov
 Fix For: 2.8


Steps to reproduce:

1)Start the server node with next timeouts. DefaultTxTimeout should be greater 
than other:

 
{code:java}







    
        
    






{code}
On the server side you should create a cache with next parameters:

 

 
{code:java}

    
    
    
    
    
    {code}
2)After that start the client with the next code:
{code:java}
IgniteCache cache = dr1Hub.getOrCreateCache("LITARGETINFO");

try (Transaction tx = dr1Hub.transactions().txStart()) {

cache.put("Key", new Object());

System.out.println("Stop me");

//here we will get long GC pause on server side

Thread.sleep(1);

// Commit the transaction.
tx.commitAsync().get();
}

{code}
 

On step "Stop me" you should suspend all the thread on the server side to 
emulate the networking problem or long GC pause on the server side.

Finally, you will face in client node next:
{code:java}
[2018-10-10 16:46:10,157][ERROR][nio-acceptor-tcp-comm-#28%GRIDC1%][root] 
Critical system error detected. Will be handled accordingly to configured 
handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, 
super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet 
[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext 
[type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker 
[name=grid-timeout-worker, igniteInstanceName=GRIDC1, finished=false, 
heartbeatTs=1539179057570]]]
{code}
Also, the similar issue could be reproduced in 2.4. In both cases looks like we 
have a deadlock during trying to display the TxEntryValueHolder. Looks like 
this values are already used by the transaction with long DefaultTxTimeout .
{code:java}
java.lang.Thread.State: WAITING
at sun.misc.Unsafe.park(Unsafe.java:-1)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
at 
org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.metadata0(CacheObjectBinaryProcessorImpl.java:526)
at 
org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.metadata(CacheObjectBinaryProcessorImpl.java:510)
at 
org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl$2.metadata(CacheObjectBinaryProcessorImpl.java:193)
at 
org.apache.ignite.internal.binary.BinaryContext.metadata(BinaryContext.java:1265)
at org.apache.ignite.internal.binary.BinaryUtils.type(BinaryUtils.java:2407)
at 
org.apache.ignite.internal.binary.BinaryObjectImpl.rawType(BinaryObjectImpl.java:302)
at 
org.apache.ignite.internal.binary.BinaryObjectExImpl.toString(BinaryObjectExImpl.java:205)
at 
org.apache.ignite.internal.binary.BinaryObjectExImpl.toString(BinaryObjectExImpl.java:186)
at 
org.apache.ignite.internal.binary.BinaryObjectImpl.toString(BinaryObjectImpl.java:919)
at java.lang.String.valueOf(String.java:2994)
at java.lang.StringBuilder.append(StringBuilder.java:131)
at 
org.apache.ignite.internal.processors.cache.transactions.TxEntryValueHolder.toString(TxEntryValueHolder.java:161)
...{code}
On the client side, it could be looked like a hanging transaction because we 
waiting on:
{code:java}
tx.commitAsync().get();{code}
 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9773) GridAffinityProcessorMemoryLeakTest should be fixed

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-9773:
--

 Summary: GridAffinityProcessorMemoryLeakTest should be fixed
 Key: IGNITE-9773
 URL: https://issues.apache.org/jira/browse/IGNITE-9773
 Project: Ignite
  Issue Type: Bug
Reporter: Andrey Aleksandrov


/** Max value for affinity history size name. Should be the same as in 
GridAffinityAssignmentCache.MAX_HIST_SIZE */
private final int MAX_HIST_SIZE = getInteger(IGNITE_AFFINITY_HISTORY_SIZE, 100);

The default value should be 500. (not 100)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9699) Web console: Mutual authentication on Web Console isn't supported

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-9699:
--

 Summary: Web console: Mutual authentication on Web Console isn't 
supported
 Key: IGNITE-9699
 URL: https://issues.apache.org/jira/browse/IGNITE-9699
 Project: Ignite
  Issue Type: New Feature
  Components: UI
Affects Versions: 2.6
Reporter: Andrey Aleksandrov
 Fix For: 2.8


Ignite has a possibility to configure SSL connection with Certificate 
Authority. But in the web console, we are able to use only one certificate used 
in web console agent. It could be a problem when a lot of users will use the 
same web console and have different certificates for validation.

We require to allow the user to pick his own certificate for SSL connection to 
the grid.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8987) Ignite hangs during getting of atomic structure after autoactivation

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-8987:
--

 Summary: Ignite hangs during getting of atomic structure after 
autoactivation
 Key: IGNITE-8987
 URL: https://issues.apache.org/jira/browse/IGNITE-8987
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.4
Reporter: Andrey Aleksandrov
 Fix For: 2.7
 Attachments: reproducer.java

I investigate the use cases with autoactivation and creating of the 
IgniteAtomicSequence. It hangs on awaitInitialization() method in case if it 
called after the last node from BLT was started.

Steps to reproduce:

First iteration:
 
Do next in one thread:

1)Start server 1
2)Start server 2
3)Activate the cluster 
4)Create the IgniteAtomicSequence using next code:


IgniteAtomicSequence igniteAtomicSequence = ignite.atomicSequence(
 "TestName",
 atomicConfiguration,
 10,
 true);

Second iteration:

1)Start server 1
2)Start server 2 (Autoactivation will be started)
3)Get the IgniteAtomicSequence using next code:


IgniteAtomicSequence igniteAtomicSequence = ignite.atomicSequence(
 "TestName",
 10,
 true); //could be false because TestName was already created in iteration 1

In this case, we hang in awaitInitialization() method in 
DataStructureProcessor.getAtomic() method.

In case if I added some sleep timeout between step 2 and 3 in the second 
iteration then everything was ok. Looks like we have some race here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8888) Possible data loss durring restaring of the nodes with empty pds

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-:
--

 Summary: Possible data loss durring restaring of the nodes with 
empty pds
 Key: IGNITE-
 URL: https://issues.apache.org/jira/browse/IGNITE-
 Project: Ignite
  Issue Type: Bug
  Components: persistence
Affects Versions: 2.4
Reporter: Andrey Aleksandrov
 Fix For: 2.7
 Attachments: reproducer.java

Case: 

1)Start 3 data nodes and activate the cluster with cache with 1 backup and 
PartitionLossPolicy.READ_ONLY_SAFE.
2)Start client and add the data to your cache. Stop the client
3)Stop DN2 and clear it pds and val
4)Start DN2. Rebalance will start.
5)During rebalance stop DN3.
6)Start DN3.

At this moment some partitions from DN2 marked as LOST and cache size will be 
less than expected.

7) Run resetLostPartitions(caches).

Now all partitions on DN2 marked as OWNING but cache size is still less than 
expected.

Workaround:

after step 6 do:

7)force rebalance using deactivate/activate methods.
8)wait for completion of rebalance

Now cache size is expected but some partitions from DN2 marked as LOST

9)Run resetLostPartitions(caches).

Now cache size is OK and all partitions from DN2 marked as OWNING.

However, looks like without force rebalance we have data loss here.




 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8833) IgniteCache.isLocalLocked method has unexpected behivior in case of several nodes started in one JVM in different threads

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-8833:
--

 Summary: IgniteCache.isLocalLocked method has unexpected behivior 
in case of several nodes started in one JVM in different threads
 Key: IGNITE-8833
 URL: https://issues.apache.org/jira/browse/IGNITE-8833
 Project: Ignite
  Issue Type: Bug
  Components: cache, documentation
Affects Versions: 2.5
Reporter: Andrey Aleksandrov
 Fix For: 2.6
 Attachments: ThreadLockedTest.java

According to specification:

[https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteCache.html#isLocalLocked-K-boolean-]


Checks if specified key is locked.This is a local in-VM operation and does not 
involve any network trips or access to persistent storage in any way.

Parameters:

{{key}} - Key to check.

{{byCurrThread}} - If {{true}} method will check that current thread owns a 
lock on this key, other vise will check that any thread on any node owns a lock 
on this key.

Returns:{{True}} if lock is owned by some node.

In the attached test we start one node in the main thread and another node from 
the second thread. In second node we take a lock but in main thread 
isLocalLocked shows that no thread held the lock.

However tryLock works ok. So the behavior of the isLocalLocked method should be 
described in this case or fixed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8718) Documentation about using of the C++ BinaryWriter/BinaryReader should be updated

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-8718:
--

 Summary: Documentation about using of the C++ 
BinaryWriter/BinaryReader should be updated
 Key: IGNITE-8718
 URL: https://issues.apache.org/jira/browse/IGNITE-8718
 Project: Ignite
  Issue Type: Improvement
  Components: documentation
Affects Versions: 2.5
Reporter: Andrey Aleksandrov
 Fix For: 2.6


The usage that should be documented:


1)In case if you get some writer from BinaryWriter then you started writing 
session. Until method close will not be called for this writer you can't get 
another writer. 
 
For example, next code isn't correct:


{code:java}
BinaryMapWriter field1Writer = writer.WriteMap("field1", MapType::HASH_MAP); //here you start writing session
BinaryMapWriter field2Writer = writer.WriteMap("field2", MapType::HASH_MAP); //here you start another writing session 
- error

{code}

Should be:
 
{code:java}
BinaryMapWriter field1Writer = writer.WriteMap("field1", MapType::HASH_MAP); //here you start writing session

//do something

field1Writer.Close() //here you end writing session

BinaryMapWriter field2Writer = writer.WriteMap("field2", MapType::HASH_MAP); //here you start another writing session

//do something

field2Writer.Close() //here you end another writing session
{code}
 
2) In case if you get some reader from BinaryWriter then you started reading 
session. Until something will not be read from this reader you can't get 
another reader. 
 
For example, next code isn't correct:
 
{code:java}
BinaryMapReader field1Reader = reader.ReadMap("field1"); //start reading session
BinaryMapReader field2Reader = reader.ReadMap("field2"); //start another read session - error

{code}
Should be for example:


{code:java}
BinaryMapReader field1Reader = reader.ReadMap("field1"); //start reading session

...
field1Reader.GetNext(key, val);  //reading done
...

BinaryMapReader field2Reader = reader.ReadMap("field2"); //start another read session

...
field2Reader.GetNext(key, val);  //reading done
...{code}
 
 
 
In the case of the writer, it could be it looks like expected. In case of the 
reader, it looks a little bit confusing.
 
These two behaviors should be described in the documentation as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8676) Possible data loss after stoping/starting several nodes at the same time

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-8676:
--

 Summary: Possible data loss after stoping/starting several nodes 
at the same time
 Key: IGNITE-8676
 URL: https://issues.apache.org/jira/browse/IGNITE-8676
 Project: Ignite
  Issue Type: Bug
  Components: persistence
Affects Versions: 2.4
Reporter: Andrey Aleksandrov
 Fix For: 2.6
 Attachments: DataLossTest.zip, image-2018-06-01-12-34-54-320.png, 
image-2018-06-01-13-12-47-218.png, image-2018-06-01-13-15-17-437.png

Steps to reproduce:

1)Start 3 data (DN1, DN2, DN3) nodes with the configuration that contains the 
cache with node filter for only these three nodes and 1 backup. (see 
configuration from attachment)
2)Activate the cluster. Now you should have 3 nodes in BLT
3)Start new server node (SN). Now you should have 3 nodes in BLT and 1 node not 
in the baseline.
4)Using some node load about 1 (or more) entities into the cache.
5)Start visor and check that number of primary partitions equals to backup 
partitions.

!image-2018-06-01-12-34-54-320.png!
6)Now stop DN3 and SN. After that start them at the same time.
7)When DN3 and SN will be online, check that number of primary partitions (PN) 
equals to backup partitions (BN).

7.1)In a case if PN == BN => go to step 6)
7.2)In a case if PN != BN => go to step 8)

 

!image-2018-06-01-13-12-47-218.png!

8)Deactivate the cluster with control.sh.
9)Activate the cluster with control.sh.

Not you should see the data loss.

!image-2018-06-01-13-15-17-437.png!

Notes:
1)Stops/Starts should be done at the same time
2)Consistent Ids for nodes should be constant.



Not you should see the data loss.

Also, I provide the reproducer that often possible to reproduce this issue (not 
always).  Free the working directory and restart reproducer in case if there is 
no data loss in this iteration.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8631) IgniteInstanceResource doesn't inject the Ignite instance to Ignite Predicate during deploying cache with nodeFilter

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-8631:
--

 Summary: IgniteInstanceResource doesn't inject the Ignite instance 
to Ignite Predicate during deploying cache with nodeFilter
 Key: IGNITE-8631
 URL: https://issues.apache.org/jira/browse/IGNITE-8631
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.4
Reporter: Andrey Aleksandrov
 Fix For: 2.6


Next code:
{code:java}
Ignite ignite = IgnitionEx.start("examples/config/example-ignite.xml", 
"ignite-1");
Ignite ignite2 = IgnitionEx.start("examples/config/example-ignite.xml", 
"ignite-2");


ignite2.createCache(new CacheConfiguration()
.setName("my-cache")
.setNodeFilter(new IgnitePredicate() {
@IgniteInstanceResource Ignite filterIgnite;

@Override public boolean apply(ClusterNode node) {
System.out.println("Is local node: " + node.isLocal());
System.out.println("ignite: " + (isNull(filterIgnite) ? null : 
filterIgnite.name()));
return true;
}
})
);

{code}
Produces next output:
{code:java}
Is local node: true
ignite: null
Is local node: true
ignite: null
Is local node: false
ignite: null
Is local node: false
ignite: null
Is local node: false
ignite: null
Is local node: false
ignite: null
Is local node: true
ignite: null
Is local node: true
ignite: null
Is local node: true
ignite: null
Is local node: true
ignite: null
Is local node: true
ignite: null
{code}
Looks like @IgniteInstanceResource doesn't work in this case.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8535) WebConsole configuration over HTTPs guide should be documented

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-8535:
--

 Summary: WebConsole configuration over HTTPs guide should be 
documented 
 Key: IGNITE-8535
 URL: https://issues.apache.org/jira/browse/IGNITE-8535
 Project: Ignite
  Issue Type: Task
Affects Versions: 2.4
Reporter: Andrey Aleksandrov


Documentation provides the information related to HTTPs connection for web 
console.

[https://apacheignite-tools.readme.io/v2.4/docs/getting-started#section-https]

But there is no guide how it could be configured. I believe it should be added.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8409) Ignite gets stuck on IgniteDataStreamer.addData when using Object with AffinityKeyMapped

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-8409:
--

 Summary: Ignite gets stuck on IgniteDataStreamer.addData when 
using Object with AffinityKeyMapped
 Key: IGNITE-8409
 URL: https://issues.apache.org/jira/browse/IGNITE-8409
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.3
Reporter: Andrey Aleksandrov
 Fix For: 2.6
 Attachments: ContextCpty.java, TradeKey.java, TradeKeyNew.java

This problem reproduces from time to time when we are streaming the data 
(TradeKey.java) to Ignite sql cache. As AffinityKeyMapped we used the object 
type (ContextCpty.java)

When we change AffinityKeyMapped type from object to long type 
(TradeKeyNew.java) then problem disappears.

Investigation help to understand that we hang in BPlusTree.java class in next 
method:

private Result putDown(final Put p, final long pageId, final long fwdId, final 
int lvl)

In this method:

res = p.tryReplaceInner(pageId, page, fwdId, lvl);

if (res != RETRY) // Go down recursively.
res = putDown(p, p.pageId, p.fwdId, lvl - 1);

if (res == RETRY_ROOT || p.isFinished())
return res;

if (res == RETRY)
checkInterrupted(); //WE ALWAYS GO TO THIS PLACE



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8243) Possible memory leak at ExchangeLatchManager during dynamic creating/removing of the local caches

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-8243:
--

 Summary: Possible memory leak at ExchangeLatchManager during 
dynamic creating/removing of the local caches
 Key: IGNITE-8243
 URL: https://issues.apache.org/jira/browse/IGNITE-8243
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.5
Reporter: Andrey Aleksandrov
 Attachments: image.png, reproducer.java

Reproducer was attached. Memory analizer report was attached too. 

Looks like that next collection never removes its items in case of dynamic 
creating/removing of the local caches:



/** Server latches collection. */
private final ConcurrentMap, ServerLatch> 
serverLatches = new ConcurrentHashMap<>();

To see it you can modify source code a little:



private Latch createServerLatch(String id, AffinityTopologyVersion topVer, 
Collection participants) {
 final T2 latchId = new T2<>(id, topVer);

 if (serverLatches.containsKey(latchId))
 return serverLatches.get(latchId);

 ServerLatch latch = new ServerLatch(id, topVer, participants);

 serverLatches.put(latchId, latch);

 if (log.isDebugEnabled())
 log.debug("Server latch is created [latch=" + latchId + ", participantsSize=" 
+ participants.size() + "]");

 log.error("Server latch is created [size=" + serverLatches.size() +
 ", latchId = " + latchId + "]");

And add some breakpoints in places where removing can be done.

Log should be like that:

[2018-04-13 09:55:44,911][ERROR][exchange-worker-#42][ExchangeLatchManager] 
Server latch is created [size=1990, latchId = IgniteBiTuple [val1=exchange, 
val2=AffinityTopologyVersion [topVer=1, minorTopVer=1989]]]
[2018-04-13 09:55:44,911][ERROR][exchange-worker-#42][ExchangeLatchManager] 
Server latch is created [size=1991, latchId = IgniteBiTuple [val1=exchange, 
val2=AffinityTopologyVersion [topVer=1, minorTopVer=1990]]]
[2018-04-13 09:55:44,911][ERROR][exchange-worker-#42][ExchangeLatchManager] 
Server latch is created [size=1992, latchId = IgniteBiTuple [val1=exchange, 
val2=AffinityTopologyVersion [topVer=1, minorTopVer=1991]]]
[2018-04-13 09:55:44,926][ERROR][exchange-worker-#42][ExchangeLatchManager] 
Server latch is created [size=1993, latchId = IgniteBiTuple [val1=exchange, 
val2=AffinityTopologyVersion [topVer=1, minorTopVer=1992]]]
[2018-04-13 09:55:44,926][ERROR][exchange-worker-#42][ExchangeLatchManager] 
Server latch is created [size=1994, latchId = IgniteBiTuple [val1=exchange, 
val2=AffinityTopologyVersion [topVer=1, minorTopVer=1993]]]
[2018-04-13 09:55:44,926][ERROR][exchange-worker-#42][ExchangeLatchManager] 
Server latch is created [size=1995, latchId = IgniteBiTuple [val1=exchange, 
val2=AffinityTopologyVersion [topVer=1, minorTopVer=1994]]]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8173) ignite.getOrCreateCache(cacheConfig).iterator() method works incorrect for replicated cache in case if some data node isn't in baseline

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-8173:
--

 Summary: ignite.getOrCreateCache(cacheConfig).iterator() method 
works incorrect for replicated cache in case if some data node isn't in baseline
 Key: IGNITE-8173
 URL: https://issues.apache.org/jira/browse/IGNITE-8173
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.5
Reporter: Andrey Aleksandrov
 Attachments: StartClientNode.java, StartOneServerAndActivate.java, 
StartSecondServerNode.java

How to reproduce:

1)Create new server node and activate the cluster (run the 
StartOneServerAndActivate.java example)
2)Create another server node. (run the StartSecondServerNode.java example after 
step 1)
3)Create client node that will create new replicated cache, put some values and 
try to get the cache iterator. (run the StartClientNode.java example)

In this case you should see the log like that:

* Elements in cache: 0
**
**
**
**
* Elements in cache: 1000
**
**
**
**
* Elements in cache: 1000
**
**
**
**
* Elements in cache: 0
**
**
**
**
* Elements in cache: 0
**
**



It means that sometime we see the cache values and sometime we do not. It 
happens because second node isn't in the baseline topology. In this case all 
data was stored to first server node and it's ok.

The problem here is that when we ask for the iterator then we choose the node 
that contains the required cache and send to it scan query. At the moment we 
can choose the node in baseline topology (with data) and empty node. Looks like 
this behavior should be changed.

Incorrect logic located at GridCacheQueryAdapter.java at "private 
Collection nodes() throws IgniteCheckedException" method:


{noformat}
case REPLICATED:
if (prj != null || part != null)
return nodes(cctx, prj, part);

if (cctx.affinityNode())
return Collections.singletonList(cctx.localNode());

Collection affNodes = nodes(cctx, null, null); //HERE WE HAVE 
BOTH NODES AT affNodes

return affNodes.isEmpty() ? affNodes : 
Collections.singletonList(F.rand(affNodes)) {noformat}
 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8098) Getting affinity for topology version earlier than affinity is calculated because of data race

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-8098:
--

 Summary: Getting affinity for topology version earlier than 
affinity is calculated because of data race
 Key: IGNITE-8098
 URL: https://issues.apache.org/jira/browse/IGNITE-8098
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.3
Reporter: Andrey Aleksandrov


>From time to time the Ignite cluster with services throws next exception 
>during restarting of  some nodes:

java.lang.IllegalStateException: Getting affinity for topology version earlier 
than affinity is calculated [locNode=TcpDiscoveryNode 
[id=c770dbcf-2908-442d-8aa0-bf26a2aecfef, addrs=[10.44.162.169, 127.0.0.1], 
sockAddrs=[clrv041279.ic.ing.net/10.44.162.169:56500, /127.0.0.1:56500], 
discPort=56500, order=11, intOrder=8, lastExchangeTime=1520931375337, loc=true, 
ver=2.3.3#20180213-sha1:f446df34, isClient=false], grp=ignite-sys-cache, 
topVer=AffinityTopologyVersion [topVer=13, minorTopVer=0], 
head=AffinityTopologyVersion [topVer=15, minorTopVer=0], 
history=[AffinityTopologyVersion [topVer=11, minorTopVer=0], 
AffinityTopologyVersion [topVer=11, minorTopVer=1], AffinityTopologyVersion 
[topVer=12, minorTopVer=0], AffinityTopologyVersion [topVer=15, minorTopVer=0]]]

Looks like the reason of this issue is the data race in GridServiceProcessor 
class.


How to reproduce:

1)To simulate data race you should update next place in source code:



Class: GridServiceProcessor
Method: @Override public void onEvent(final DiscoveryEvent evt, final 
DiscoCache discoCache) {
Place:



try {
 svcName.set(dep.configuration().getName());

 ctx.cache().internalCache(UTILITY_CACHE_NAME).context().affinity().
 affinityReadyFuture(topVer).get();

//HERE (between GET and REASSIGN) you should add Thread.sleep(100) for example.

//try {
//Thread.sleep(100);
//}
//catch (InterruptedException e1) {
//e1.printStackTrace();
//}
 
 reassign(dep, topVer);
}
catch (IgniteCheckedException ex) {
 if (!(e instanceof ClusterTopologyCheckedException))
 LT.error(log, ex, "Failed to do service reassignment (will retry): " +
 dep.configuration().getName());

 retries.add(dep);
}

...

2)After that you should imitate start/shutdown iterations. For reproducing I 
used GridServiceProcessorBatchDeploySelfTest (but timeout on future.get should 
be increased to avoid timeout error)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8065) Some of PDS indexing tests became red after IGNITE-7902

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-8065:
--

 Summary: Some of PDS indexing tests became red after IGNITE-7902
 Key: IGNITE-8065
 URL: https://issues.apache.org/jira/browse/IGNITE-8065
 Project: Ignite
  Issue Type: Bug
Reporter: Andrey Aleksandrov
Assignee: Andrey Aleksandrov


https://ci.ignite.apache.org/viewLog.html?buildId=1153730=buildResultsDiv=IgniteTests24Java8_RunAll



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8060) Sqline creating tables on client nodes works incorrect in case if node shutdown

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-8060:
--

 Summary: Sqline creating tables on client nodes works incorrect in 
case if node shutdown
 Key: IGNITE-8060
 URL: https://issues.apache.org/jira/browse/IGNITE-8060
 Project: Ignite
  Issue Type: Bug
  Components: jdbc
Affects Versions: 2.4
Reporter: Andrey Aleksandrov


For reproducing:

You should start one local server and one local client nodes and follow the 
instructions:

1)Connect to client node:

sqlline.bat --color=true --verbose=true -u jdbc:ignite:thin://127.0.0.1:10801

2)Create new table on client node:

CREATE TABLE City (id LONG PRIMARY KEY, name VARCHAR)WITH "template=replicated";

3)Check that table exists from server node:

!tables

On this step table should be shown in the response.

4)Drop the client node

5)Create new client node

6)Connect to new client node:

sqlline.bat --color=true --verbose=true -u jdbc:ignite:thin://127.0.0.1:10801

7)Check that table exists from server node:

!tables

*On this step there is no "city" table in the list.*

8)Try to drop the table:
DROP TABLE City;
java.sql.SQLException: Table doesn't exist: CITY
 at 
org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.sendRequest(JdbcThinConnection.java:671)
 at 
org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute0(JdbcThinStatement.java:130)
 at 
org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute(JdbcThinStatement.java:299)
 at sqlline.Commands.execute(Commands.java:823)
 at sqlline.Commands.sql(Commands.java:733)
 at sqlline.SqlLine.dispatch(SqlLine.java:795)
 at sqlline.SqlLine.begin(SqlLine.java:668)
 at sqlline.SqlLine.start(SqlLine.java:373)
 at sqlline.SqlLine.main(SqlLine.java:265)



9)Try to create new table:
CREATE TABLE City (id LONG PRIMARY KEY, name VARCHAR)WITH "template=replicated";

java.sql.SQLException: Table already exists: CITY
 at 
org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.sendRequest(JdbcThinConnection.java:671)
 at 
org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute0(JdbcThinStatement.java:130)
 at 
org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute(JdbcThinStatement.java:299)
 at sqlline.Commands.execute(Commands.java:823)
 at sqlline.Commands.sql(Commands.java:733)
 at sqlline.SqlLine.dispatch(SqlLine.java:795)
 at sqlline.SqlLine.begin(SqlLine.java:668)
 at sqlline.SqlLine.start(SqlLine.java:373)
 at sqlline.SqlLine.main(SqlLine.java:265)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8055) Sqline command !tables works incorrect for client node

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-8055:
--

 Summary: Sqline command !tables works incorrect for client node
 Key: IGNITE-8055
 URL: https://issues.apache.org/jira/browse/IGNITE-8055
 Project: Ignite
  Issue Type: Bug
  Components: jdbc
Affects Versions: 2.4
Reporter: Andrey Aleksandrov


For reproducing:

You should start one local server and one local client nodes and follow the 
instructions:


1)Connect to server node:

sqlline.bat --color=true --verbose=true -u jdbc:ignite:thin://127.0.0.1:10800

2)Create new table on server node:

CREATE TABLE City (id LONG PRIMARY KEY, name VARCHAR)WITH "template=replicated";

3)Check that table exists from server node:

!tables

On this step table should be shown in the response.

4)Connect to client node:

sqlline.bat --color=true --verbose=true -u jdbc:ignite:thin://127.0.0.1:10801

5)Check that table exists from server node:

!tables

*On this step there is no "city" table in the list.*

Next commands work from client node as well:
SELECT * FROM City
DROP TABLE City
 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)