[jira] [Commented] (IGNITE-13174) C++: Add Windows support to CMake build system

2020-08-24 Thread Igor Sapego (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183660#comment-17183660
 ] 

Igor Sapego commented on IGNITE-13174:
--

[~ivandasch] I was on vacation. No help is needed, the task is almost done, I'm 
running latest tests.

> C++: Add Windows support to CMake build system
> --
>
> Key: IGNITE-13174
> URL: https://issues.apache.org/jira/browse/IGNITE-13174
> Project: Ignite
>  Issue Type: Bug
>  Components: platforms
>Affects Versions: 2.8.1
>Reporter: Igor Sapego
>Assignee: Igor Sapego
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Ticket IGNITE-13078 adds CMake build system support, but only for Linux. Need 
> make sure it works on Windows and create TC job for it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-10563) Allow manual checkpoint and fsync for WAL

2020-08-24 Thread Stanislav Lukyanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-10563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stanislav Lukyanov updated IGNITE-10563:

Summary: Allow manual checkpoint and fsync for WAL  (was: Allow manual 
fsync for WAL)

> Allow manual checkpoint and fsync for WAL
> -
>
> Key: IGNITE-10563
> URL: https://issues.apache.org/jira/browse/IGNITE-10563
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Stanislav Lukyanov
>Priority: Major
>
> When walMode is set to LOG_ONLY or BACKGROUND there is a gap between 
> successful return of cache write operations and actual of the data to the 
> persistent memory. This gap is, while usually low, generally can be of any 
> length and depends on low-level system parameters (e.g. sysctl memory and IO 
> settings on Linux).
> On the other hand, there are situations when a user may want to ensure that 
> at certain points all writes have been propagated to the disk.
> For example, say a large batch of data is being inserted into Ignite from an 
> upstream system. After finishing the batch we want to ensure that all of the 
> data is secure, and after that we remove the batch from the upstream storage. 
> In other words, we want
> {code}
> map = upstream.getData();
> cache.putAll(map);
> cache.fsync(); // <---
> upstream.removeAll(map);
> {code}
> Currently there is no easy way to ensure that certain write (or all writes 
> until a point in time) has been flushed to a device. We can only rely on 
> settings like WAL sync interval, checkpoint timeout, etc.
> It would be nice to have a way to manually call fsync() for WAL to have a 
> strong guarantee that all previous updates have been fully written on disk.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13308) C++: Thin client transactions

2020-08-24 Thread Igor Sapego (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183595#comment-17183595
 ] 

Igor Sapego commented on IGNITE-13308:
--

[~zstan] I'm looking...

> C++: Thin client transactions
> -
>
> Key: IGNITE-13308
> URL: https://issues.apache.org/jira/browse/IGNITE-13308
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms
>Reporter: Ivan Daschinskiy
>Assignee: Stanilovsky Evgeny
>Priority: Major
>  Labels: C++, iep-34
> Fix For: 2.10
>
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-13382) DurableBackgroundTask can abandon incomplete task

2020-08-24 Thread Maria Makedonskaya (Jira)
Maria Makedonskaya created IGNITE-13382:
---

 Summary: DurableBackgroundTask can abandon incomplete task
 Key: IGNITE-13382
 URL: https://issues.apache.org/jira/browse/IGNITE-13382
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.7
Reporter: Maria Makedonskaya
Assignee: Maria Makedonskaya


DurableBackgroundTasks are tracked using metastorage, there's a specific marker 
for every task, and it is removed right after the task is complete.

But there's a race between checkpointer and metastorage. End-marker removal is 
a logical operation, while task itself is mostly physical (at least the 
existing one). So, following scenario is possible:
* Checkpoint occurs in the middle of the task;
* Task is completed before the next checkpoint;
* Metastorage record is deleted, this fact if written to WAL and synced to the 
storage;
* Node failed;
* Recovery process applies deletion from metastorage, this means that 
DurableBackgroundTasks info is lost;
* But part of the index is still present in the storage.

I think that we should remove markers from metastorage only after the next 
checkpoint, this will 100% save us from such situation.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-13072) Synchronization problems when different classloaders are used for deployment of same class

2020-08-24 Thread Vladislav Pyatkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladislav Pyatkov updated IGNITE-13072:
---
Release Note: Fixed issue with concurrent deployment on local p2p.

> Synchronization problems when different classloaders are used for deployment 
> of same class
> --
>
> Key: IGNITE-13072
> URL: https://issues.apache.org/jira/browse/IGNITE-13072
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladislav Pyatkov
>Assignee: Vladislav Pyatkov
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> If you concurrently deploy one class using different classloaders you can get 
> error:
> {noformat}
> 2020-04-28 
> 14:36:42.523[ERROR][sys-stripe-45-#46%GRID%GridNodeName%][o.a.i.i.m.d.GridDeploymentLocalStore]
>  Found more than one active deployment for the same resource [cls=class 
> org.some.class.old.InvokeIndexRemover, depMode=SHARED, dep=GridDeployment 
> [ts=1588067100125, depMode=SHARED, 
> clsLdr=org.some.class.factory.NodeClassLoader@14035d21, 
> clsLdrId=85ab310c171-a9fad11c-9f8c-4d2a-8146-6c87254303e7, userVer=0, 
> loc=true, sampleClsName=org.some.class.predicates.CompositePredicate, 
> pendingUndeploy=false, undeployed=false, usage=0]]
>  
> 2020-04-28 
> 14:36:42.544[ERROR][sys-stripe-45-#46%GRID%GridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
>  Failed to process message [senderId=f104e069-9d80-4202-b50a-b3dc1804ac89, 
> msg=GridNearAtomicSingleUpdateRequest [key=KeyCacheObject [hasValBytes=true], 
> super=GridNearAtomicSingleUpdateRequest [key=KeyCacheObject 
> [hasValBytes=true], parent=GridNearAtomicAbstractSingleUpdateRequest 
> [nodeId=null, futId=1376257, topVer=AffinityTopologyVersion [topVer=35, 
> minorTopVer=0], parent=GridNearAtomicAbstractUpdateRequest [res=null, 
> flags=]
> java.lang.AssertionError: null
>  at 
> org.apache.ignite.internal.managers.deployment.GridDeploymentLocalStore.getDeployment(GridDeploymentLocalStore.java:203)
>  at 
> org.apache.ignite.internal.managers.deployment.GridDeploymentManager.getLocalDeployment(GridDeploymentManager.java:383)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CacheClassLoader.findClass(GridCacheDeploymentManager.java:802)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CacheClassLoader.loadClass(GridCacheDeploymentManager.java:794)
>  at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8561)
>  at 
> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
>  at 
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:700)
>  at 
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1757)
>  at 
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>  at 
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>  at 
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:99)
>  at 
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>  at 
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9959)
>  at 
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10017)
>  at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateInvokeRequest.finishUnmarshal(GridNearAtomicSingleUpdateInvokeRequest.java:200)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1560)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:582)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:386)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:312)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:102)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:301)
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125)
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091)
>  at 
> 

[jira] [Updated] (IGNITE-13072) Synchronization problems when different classloaders are used for deployment of same class

2020-08-24 Thread Sergey Chugunov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Chugunov updated IGNITE-13072:
-
Ignite Flags: Release Notes Required  (was: Docs Required,Release Notes 
Required)

> Synchronization problems when different classloaders are used for deployment 
> of same class
> --
>
> Key: IGNITE-13072
> URL: https://issues.apache.org/jira/browse/IGNITE-13072
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladislav Pyatkov
>Assignee: Vladislav Pyatkov
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> If you concurrently deploy one class using different classloaders you can get 
> error:
> {noformat}
> 2020-04-28 
> 14:36:42.523[ERROR][sys-stripe-45-#46%GRID%GridNodeName%][o.a.i.i.m.d.GridDeploymentLocalStore]
>  Found more than one active deployment for the same resource [cls=class 
> org.some.class.old.InvokeIndexRemover, depMode=SHARED, dep=GridDeployment 
> [ts=1588067100125, depMode=SHARED, 
> clsLdr=org.some.class.factory.NodeClassLoader@14035d21, 
> clsLdrId=85ab310c171-a9fad11c-9f8c-4d2a-8146-6c87254303e7, userVer=0, 
> loc=true, sampleClsName=org.some.class.predicates.CompositePredicate, 
> pendingUndeploy=false, undeployed=false, usage=0]]
>  
> 2020-04-28 
> 14:36:42.544[ERROR][sys-stripe-45-#46%GRID%GridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
>  Failed to process message [senderId=f104e069-9d80-4202-b50a-b3dc1804ac89, 
> msg=GridNearAtomicSingleUpdateRequest [key=KeyCacheObject [hasValBytes=true], 
> super=GridNearAtomicSingleUpdateRequest [key=KeyCacheObject 
> [hasValBytes=true], parent=GridNearAtomicAbstractSingleUpdateRequest 
> [nodeId=null, futId=1376257, topVer=AffinityTopologyVersion [topVer=35, 
> minorTopVer=0], parent=GridNearAtomicAbstractUpdateRequest [res=null, 
> flags=]
> java.lang.AssertionError: null
>  at 
> org.apache.ignite.internal.managers.deployment.GridDeploymentLocalStore.getDeployment(GridDeploymentLocalStore.java:203)
>  at 
> org.apache.ignite.internal.managers.deployment.GridDeploymentManager.getLocalDeployment(GridDeploymentManager.java:383)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CacheClassLoader.findClass(GridCacheDeploymentManager.java:802)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CacheClassLoader.loadClass(GridCacheDeploymentManager.java:794)
>  at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8561)
>  at 
> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
>  at 
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:700)
>  at 
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1757)
>  at 
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>  at 
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>  at 
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:99)
>  at 
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>  at 
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9959)
>  at 
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10017)
>  at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateInvokeRequest.finishUnmarshal(GridNearAtomicSingleUpdateInvokeRequest.java:200)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1560)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:582)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:386)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:312)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:102)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:301)
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125)
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091)
>  at 
> 

[jira] [Updated] (IGNITE-13072) Synchronization problems when different classloaders are used for deployment of same class

2020-08-24 Thread Sergey Chugunov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Chugunov updated IGNITE-13072:
-
Ignite Flags:   (was: Release Notes Required)

> Synchronization problems when different classloaders are used for deployment 
> of same class
> --
>
> Key: IGNITE-13072
> URL: https://issues.apache.org/jira/browse/IGNITE-13072
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladislav Pyatkov
>Assignee: Vladislav Pyatkov
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> If you concurrently deploy one class using different classloaders you can get 
> error:
> {noformat}
> 2020-04-28 
> 14:36:42.523[ERROR][sys-stripe-45-#46%GRID%GridNodeName%][o.a.i.i.m.d.GridDeploymentLocalStore]
>  Found more than one active deployment for the same resource [cls=class 
> org.some.class.old.InvokeIndexRemover, depMode=SHARED, dep=GridDeployment 
> [ts=1588067100125, depMode=SHARED, 
> clsLdr=org.some.class.factory.NodeClassLoader@14035d21, 
> clsLdrId=85ab310c171-a9fad11c-9f8c-4d2a-8146-6c87254303e7, userVer=0, 
> loc=true, sampleClsName=org.some.class.predicates.CompositePredicate, 
> pendingUndeploy=false, undeployed=false, usage=0]]
>  
> 2020-04-28 
> 14:36:42.544[ERROR][sys-stripe-45-#46%GRID%GridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
>  Failed to process message [senderId=f104e069-9d80-4202-b50a-b3dc1804ac89, 
> msg=GridNearAtomicSingleUpdateRequest [key=KeyCacheObject [hasValBytes=true], 
> super=GridNearAtomicSingleUpdateRequest [key=KeyCacheObject 
> [hasValBytes=true], parent=GridNearAtomicAbstractSingleUpdateRequest 
> [nodeId=null, futId=1376257, topVer=AffinityTopologyVersion [topVer=35, 
> minorTopVer=0], parent=GridNearAtomicAbstractUpdateRequest [res=null, 
> flags=]
> java.lang.AssertionError: null
>  at 
> org.apache.ignite.internal.managers.deployment.GridDeploymentLocalStore.getDeployment(GridDeploymentLocalStore.java:203)
>  at 
> org.apache.ignite.internal.managers.deployment.GridDeploymentManager.getLocalDeployment(GridDeploymentManager.java:383)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CacheClassLoader.findClass(GridCacheDeploymentManager.java:802)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CacheClassLoader.loadClass(GridCacheDeploymentManager.java:794)
>  at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8561)
>  at 
> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
>  at 
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:700)
>  at 
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1757)
>  at 
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>  at 
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>  at 
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:99)
>  at 
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>  at 
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9959)
>  at 
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10017)
>  at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateInvokeRequest.finishUnmarshal(GridNearAtomicSingleUpdateInvokeRequest.java:200)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1560)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:582)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:386)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:312)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:102)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:301)
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125)
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091)
>  at 
> 

[jira] [Updated] (IGNITE-13072) Synchronization problems when different classloaders are used for deployment of same class

2020-08-24 Thread Sergey Chugunov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Chugunov updated IGNITE-13072:
-
Fix Version/s: 2.10

> Synchronization problems when different classloaders are used for deployment 
> of same class
> --
>
> Key: IGNITE-13072
> URL: https://issues.apache.org/jira/browse/IGNITE-13072
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladislav Pyatkov
>Assignee: Vladislav Pyatkov
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> If you concurrently deploy one class using different classloaders you can get 
> error:
> {noformat}
> 2020-04-28 
> 14:36:42.523[ERROR][sys-stripe-45-#46%GRID%GridNodeName%][o.a.i.i.m.d.GridDeploymentLocalStore]
>  Found more than one active deployment for the same resource [cls=class 
> org.some.class.old.InvokeIndexRemover, depMode=SHARED, dep=GridDeployment 
> [ts=1588067100125, depMode=SHARED, 
> clsLdr=org.some.class.factory.NodeClassLoader@14035d21, 
> clsLdrId=85ab310c171-a9fad11c-9f8c-4d2a-8146-6c87254303e7, userVer=0, 
> loc=true, sampleClsName=org.some.class.predicates.CompositePredicate, 
> pendingUndeploy=false, undeployed=false, usage=0]]
>  
> 2020-04-28 
> 14:36:42.544[ERROR][sys-stripe-45-#46%GRID%GridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
>  Failed to process message [senderId=f104e069-9d80-4202-b50a-b3dc1804ac89, 
> msg=GridNearAtomicSingleUpdateRequest [key=KeyCacheObject [hasValBytes=true], 
> super=GridNearAtomicSingleUpdateRequest [key=KeyCacheObject 
> [hasValBytes=true], parent=GridNearAtomicAbstractSingleUpdateRequest 
> [nodeId=null, futId=1376257, topVer=AffinityTopologyVersion [topVer=35, 
> minorTopVer=0], parent=GridNearAtomicAbstractUpdateRequest [res=null, 
> flags=]
> java.lang.AssertionError: null
>  at 
> org.apache.ignite.internal.managers.deployment.GridDeploymentLocalStore.getDeployment(GridDeploymentLocalStore.java:203)
>  at 
> org.apache.ignite.internal.managers.deployment.GridDeploymentManager.getLocalDeployment(GridDeploymentManager.java:383)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CacheClassLoader.findClass(GridCacheDeploymentManager.java:802)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CacheClassLoader.loadClass(GridCacheDeploymentManager.java:794)
>  at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8561)
>  at 
> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
>  at 
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:700)
>  at 
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1757)
>  at 
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>  at 
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>  at 
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:99)
>  at 
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>  at 
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9959)
>  at 
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10017)
>  at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateInvokeRequest.finishUnmarshal(GridNearAtomicSingleUpdateInvokeRequest.java:200)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1560)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:582)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:386)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:312)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:102)
>  at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:301)
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125)
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091)
>  at 
> org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:546)
>  at 

[jira] [Commented] (IGNITE-13328) Control.sh bash script swallow return code of CommandHandler and always return 0

2020-08-24 Thread Aleksey Plekhanov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183341#comment-17183341
 ] 

Aleksey Plekhanov commented on IGNITE-13328:


[~ivandasch], currently control.sh and ignite.sh use a loop to run main 
process, so it's not correct to fail on any error since there is some logic to 
analyze error code and rerun process. But for control.sh this loop is redundant 
and can be removed, control utility can't return error code 130, moreover, it 
doesn't work somehow with RESTART_SUCCESS_FILE, so this part of control.sh is 
also redundant and there are two identical branches of code for {{$INTERACTIVE 
== "1"}} condition.

I think current patch is not correct, control.sh still uses a loop, but loop 
logic is broken since we will never get error code other than 0 in the script. 
Most simple solution to the current problem without breaking the loop logic: 
just add {{exit $ERRORCODE}} at the end of the file. Or loop can be removed and 
in this case, {{-o errexit}} not needed at all.

Also, I'm not sure about other options, perhaps {{-o nounset}} is helpful, but, 
for example, {{-o pipefail}} is redundant for control.sh since pipes don't used 
inside this script.

What about other scripts, {{-o errexit}} definitely should not be set for 
ignite.sh. In sqlline.sh main process run is the last command in the script, so 
{{-o errexit}} is redundant here. ignitevisorcmd.sh should return tty settings 
after execution, so it's better to store error code after main process run and 
do {{exit $ERRORCODE}} at the end instead of adding {{-o errexit}}.

> Control.sh bash script swallow return code of CommandHandler and always 
> return 0
> 
>
> Key: IGNITE-13328
> URL: https://issues.apache.org/jira/browse/IGNITE-13328
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.8, 2.8.1
>Reporter: Ivan Daschinskiy
>Assignee: Ivan Daschinskiy
>Priority: Blocker
> Fix For: 2.9
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> After merging 
> [IGNITE-12367|https://issues.apache.org/jira/browse/IGNITE-12367],
> control.sh always return 0, despite the fact that CommandHandler returns 
> correct code.
> For example:
> Ignite 2.8.1
> {code}
> Failed to execute baseline command='collect'
> Latest topology update failed.
> Connection to cluster failed. Latest topology update failed.
> Command [BASELINE] finished with code: 2
> Control utility has completed execution at: 2020-08-05T15:01:34.123
> Execution time: 26627 ms
> >>> echo $?
> 0
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13345) Warming up strategy

2020-08-24 Thread Anton Kalashnikov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183251#comment-17183251
 ] 

Anton Kalashnikov commented on IGNITE-13345:


[~sergey-chugunov] can you help with the merge, please.

> Warming up strategy
> ---
>
> Key: IGNITE-13345
> URL: https://issues.apache.org/jira/browse/IGNITE-13345
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Kirill Tkalenko
>Assignee: Kirill Tkalenko
>Priority: Major
>  Labels: IEP-40
> Fix For: 2.10
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Summary of 
> [Dev-list|http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Cache-warmup-td48582.html]
> # Adding a marker interface 
> *org.apache.ignite.configuration.WarmUpConfiguration*;
> # Adding a configuration to
> ## 
> *org.apache.ignite.configuration.DataRegionConfiguration#setWarmUpConfiguration*
> ## 
> *org.apache.ignite.configuration.DataStorageConfiguration#setDefaultWarmUpConfiguration*
> # Add an internal warm-up interface that will start in [1] after [2] (after 
> recovery);
> {code:java}
> package org.apache.ignite.internal.processors.cache.warmup;
> import org.apache.ignite.IgniteCheckedException;
> import org.apache.ignite.configuration.WarmUpConfiguration;
> import org.apache.ignite.internal.GridKernalContext;
> import org.apache.ignite.internal.processors.cache.persistence.DataRegion;
> /**
>  * Interface for warming up.
>  */
> public interface WarmUpStrategy {
> /**
>  * Returns configuration class for mapping to strategy.
>  *
>  * @return Configuration class.
>  */
> Class configClass();
> /**
>  * Warm up.
>  *
>  * @param kernalCtx Kernal context.
>  * @param cfg   Warm-up configuration.
>  * @param regionData region.
>  * @throws IgniteCheckedException if faild.
>  */
> void warmUp(GridKernalContext kernalCtx, T cfg, DataRegion region) throws 
> IgniteCheckedException;
> /**
>  * Closing warm up.
>  *
>  * @throws IgniteCheckedException if faild.
>  */
> void close() throws IgniteCheckedException;
> }
> {code}
> # Adding an internal plugin extension for add own strategies;
> {code:java}
> package org.apache.ignite.internal.processors.cache.warmup;
>  
> import java.util.Collection;
> import org.apache.ignite.plugin.Extension;
>  
> /**
>  * Interface for getting warm-up strategies from plugins.
>  */
> public interface WarmUpStrategySupplier extends Extension {
> /**
>  * Getting warm-up strategies.
>  *
>  * @return Warm-up strategies.
>  */
> Collection strategies();
> }
> {code}
> # Adding strategies:
> ## Without implementation, for the possibility of disabling the warm-up: NoOP
> ## Loading everything while there is RAM with priority to indexes: LoadAll
> # Add a command to "control.sh", to stop current warm-up and cancel all 
> others: --warm-up stop in IGNITE-13362
> [1] - 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.CacheRecoveryLifecycle#afterLogicalUpdatesApplied
> [2] - 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.CacheRecoveryLifecycle#restorePartitionStates



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13345) Warming up strategy

2020-08-24 Thread Anton Kalashnikov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183243#comment-17183243
 ] 

Anton Kalashnikov commented on IGNITE-13345:


[~ktkale...@gridgain.com] LGTM.

> Warming up strategy
> ---
>
> Key: IGNITE-13345
> URL: https://issues.apache.org/jira/browse/IGNITE-13345
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Kirill Tkalenko
>Assignee: Kirill Tkalenko
>Priority: Major
>  Labels: IEP-40
> Fix For: 2.10
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Summary of 
> [Dev-list|http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Cache-warmup-td48582.html]
> # Adding a marker interface 
> *org.apache.ignite.configuration.WarmUpConfiguration*;
> # Adding a configuration to
> ## 
> *org.apache.ignite.configuration.DataRegionConfiguration#setWarmUpConfiguration*
> ## 
> *org.apache.ignite.configuration.DataStorageConfiguration#setDefaultWarmUpConfiguration*
> # Add an internal warm-up interface that will start in [1] after [2] (after 
> recovery);
> {code:java}
> package org.apache.ignite.internal.processors.cache.warmup;
> import org.apache.ignite.IgniteCheckedException;
> import org.apache.ignite.configuration.WarmUpConfiguration;
> import org.apache.ignite.internal.GridKernalContext;
> import org.apache.ignite.internal.processors.cache.persistence.DataRegion;
> /**
>  * Interface for warming up.
>  */
> public interface WarmUpStrategy {
> /**
>  * Returns configuration class for mapping to strategy.
>  *
>  * @return Configuration class.
>  */
> Class configClass();
> /**
>  * Warm up.
>  *
>  * @param kernalCtx Kernal context.
>  * @param cfg   Warm-up configuration.
>  * @param regionData region.
>  * @throws IgniteCheckedException if faild.
>  */
> void warmUp(GridKernalContext kernalCtx, T cfg, DataRegion region) throws 
> IgniteCheckedException;
> /**
>  * Closing warm up.
>  *
>  * @throws IgniteCheckedException if faild.
>  */
> void close() throws IgniteCheckedException;
> }
> {code}
> # Adding an internal plugin extension for add own strategies;
> {code:java}
> package org.apache.ignite.internal.processors.cache.warmup;
>  
> import java.util.Collection;
> import org.apache.ignite.plugin.Extension;
>  
> /**
>  * Interface for getting warm-up strategies from plugins.
>  */
> public interface WarmUpStrategySupplier extends Extension {
> /**
>  * Getting warm-up strategies.
>  *
>  * @return Warm-up strategies.
>  */
> Collection strategies();
> }
> {code}
> # Adding strategies:
> ## Without implementation, for the possibility of disabling the warm-up: NoOP
> ## Loading everything while there is RAM with priority to indexes: LoadAll
> # Add a command to "control.sh", to stop current warm-up and cancel all 
> others: --warm-up stop in IGNITE-13362
> [1] - 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.CacheRecoveryLifecycle#afterLogicalUpdatesApplied
> [2] - 
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.CacheRecoveryLifecycle#restorePartitionStates



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-13381) ClusterGroupEmptyException: Cluster group is empty error after client reconnect

2020-08-24 Thread Andrey Aleksandrov (Jira)
Andrey Aleksandrov created IGNITE-13381:
---

 Summary: ClusterGroupEmptyException: Cluster group is empty error 
after client reconnect
 Key: IGNITE-13381
 URL: https://issues.apache.org/jira/browse/IGNITE-13381
 Project: Ignite
  Issue Type: Bug
  Components: networking
Affects Versions: 2.8.1
Reporter: Andrey Aleksandrov
Assignee: Andrey Aleksandrov
 Fix For: 2.9
 Attachments: SendMessageAfterClientReconnect.java

Please run the attached test.

It will produce the following exception:

Exception in thread "main" class 
org.apache.ignite.cluster.ClusterGroupEmptyException: Cluster group is empty.
at org.apache.ignite.internal.util.IgniteUtils$6.apply(IgniteUtils.java:927)
at org.apache.ignite.internal.util.IgniteUtils$6.apply(IgniteUtils.java:925)
at 
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1083)
at 
org.apache.ignite.internal.IgniteMessagingImpl.send0(IgniteMessagingImpl.java:105)
at 
org.apache.ignite.internal.IgniteMessagingImpl.send(IgniteMessagingImpl.java:81)
at npe.IgnitePartitioningTest.main(IgnitePartitioningTest.java:110)
Caused by: class 
org.apache.ignite.internal.cluster.ClusterGroupEmptyCheckedException: Cluster 
group is empty.
at 
org.apache.ignite.internal.util.IgniteUtils.emptyTopologyException(IgniteUtils.java:5106)
at 
org.apache.ignite.internal.IgniteMessagingImpl.send0(IgniteMessagingImpl.java:100)
... 2 more

Fix:

change

return new ClusterGroupAdapter(ctx, null, 
Collections.singleton(cfg.getNodeId()));

on

return new ClusterGroupAdapter(ctx, null, 
Collections.singleton(ctx.discovery().localNode().id()));



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13377) WalModeChangeAdvancedSelfTest.testServerRestartNonCoordinator is flaky

2020-08-24 Thread Alexey Scherbakov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183170#comment-17183170
 ] 

Alexey Scherbakov commented on IGNITE-13377:


[~v.pyatkov]

I've left a comment in PR.

> WalModeChangeAdvancedSelfTest.testServerRestartNonCoordinator is flaky 
> ---
>
> Key: IGNITE-13377
> URL: https://issues.apache.org/jira/browse/IGNITE-13377
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vladislav Pyatkov
>Assignee: Vladislav Pyatkov
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-11863) .NET: Add cluster read-only mode API (status, run-time change)

2020-08-24 Thread Stanilovsky Evgeny (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-11863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183167#comment-17183167
 ] 

Stanilovsky Evgeny commented on IGNITE-11863:
-

[~ptupitsyn] plz take into account.

> .NET: Add cluster read-only mode API (status, run-time change)
> --
>
> Key: IGNITE-11863
> URL: https://issues.apache.org/jira/browse/IGNITE-11863
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms
>Reporter: Sergey Antonov
>Priority: Major
>  Labels: .NET, read-only-mode
>
> We would introduce at IGNITE-11256 new methods in IgniteCluster.
> We need their support in .Net side.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13265) Historical iterator for atomic group should transfer few more rows than required

2020-08-24 Thread Alexey Scherbakov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183166#comment-17183166
 ] 

Alexey Scherbakov commented on IGNITE-13265:


[~v.pyatkov]

Added a new comment.
1. What will happen if you try to rebalance from non reserved segment ? It can 
be deleted by archiver I assume causing node failure.
2. I don't understand second part.

> Historical iterator for atomic group should transfer few more rows than 
> required
> 
>
> Key: IGNITE-13265
> URL: https://issues.apache.org/jira/browse/IGNITE-13265
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vladislav Pyatkov
>Assignee: Vladislav Pyatkov
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> On a historical rebalance some updates move from one node to another wherein 
> this update may have various order in nodes. Reordering can happen in smell 
> interval, but it cannot avoid at all in current implementation atomic 
> protocol.
> This mean we will reduce a probably of loosing update if we make a margin 
> from initial counter for the historical iterator on atomic cache.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Comment Edited] (IGNITE-13265) Historical iterator for atomic group should transfer few more rows than required

2020-08-24 Thread Alexey Scherbakov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183166#comment-17183166
 ] 

Alexey Scherbakov edited comment on IGNITE-13265 at 8/24/20, 11:11 AM:
---

[~v.pyatkov]

Added a new comment.
1. What will happen if you try to rebalance from non reserved segment ? It can 
be deleted by archiver I assume causing node failure.
2. I don't understand second sentence.


was (Author: ascherbakov):
[~v.pyatkov]

Added a new comment.
1. What will happen if you try to rebalance from non reserved segment ? It can 
be deleted by archiver I assume causing node failure.
2. I don't understand second part.

> Historical iterator for atomic group should transfer few more rows than 
> required
> 
>
> Key: IGNITE-13265
> URL: https://issues.apache.org/jira/browse/IGNITE-13265
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vladislav Pyatkov
>Assignee: Vladislav Pyatkov
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> On a historical rebalance some updates move from one node to another wherein 
> this update may have various order in nodes. Reordering can happen in smell 
> interval, but it cannot avoid at all in current implementation atomic 
> protocol.
> This mean we will reduce a probably of loosing update if we make a margin 
> from initial counter for the historical iterator on atomic cache.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13308) C++: Thin client transactions

2020-08-24 Thread Stanilovsky Evgeny (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183154#comment-17183154
 ] 

Stanilovsky Evgeny commented on IGNITE-13308:
-

[~isapego] [~Shadowru] [~ptupitsyn] guys plz check it ?

> C++: Thin client transactions
> -
>
> Key: IGNITE-13308
> URL: https://issues.apache.org/jira/browse/IGNITE-13308
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms
>Reporter: Ivan Daschinskiy
>Assignee: Stanilovsky Evgeny
>Priority: Major
>  Labels: C++, iep-34
> Fix For: 2.10
>
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13265) Historical iterator for atomic group should transfer few more rows than required

2020-08-24 Thread Vladislav Pyatkov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183143#comment-17183143
 ] 

Vladislav Pyatkov commented on IGNITE-13265:


[~ascherbakov] Would you look this again according to my comments?

> Historical iterator for atomic group should transfer few more rows than 
> required
> 
>
> Key: IGNITE-13265
> URL: https://issues.apache.org/jira/browse/IGNITE-13265
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vladislav Pyatkov
>Assignee: Vladislav Pyatkov
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> On a historical rebalance some updates move from one node to another wherein 
> this update may have various order in nodes. Reordering can happen in smell 
> interval, but it cannot avoid at all in current implementation atomic 
> protocol.
> This mean we will reduce a probably of loosing update if we make a margin 
> from initial counter for the historical iterator on atomic cache.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13265) Historical iterator for atomic group should transfer few more rows than required

2020-08-24 Thread Vladislav Pyatkov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183138#comment-17183138
 ] 

Vladislav Pyatkov commented on IGNITE-13265:


[~ascherbakov] I answered you in PR.

More over I try to explain comments here:
1. It guarantees nothing, but makes a skip of record less probably during 
historical rebalance.
2. I think this behavior was applying earlier and goes out of scope this 
ticket. But if I correct realize you want to go through over whole WAL history, 
this cannot be more effective.



> Historical iterator for atomic group should transfer few more rows than 
> required
> 
>
> Key: IGNITE-13265
> URL: https://issues.apache.org/jira/browse/IGNITE-13265
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vladislav Pyatkov
>Assignee: Vladislav Pyatkov
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> On a historical rebalance some updates move from one node to another wherein 
> this update may have various order in nodes. Reordering can happen in smell 
> interval, but it cannot avoid at all in current implementation atomic 
> protocol.
> This mean we will reduce a probably of loosing update if we make a margin 
> from initial counter for the historical iterator on atomic cache.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (IGNITE-13218) System view walker generator wrong sorts imports

2020-08-24 Thread Aleksey Kurinov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Kurinov reassigned IGNITE-13218:


Assignee: Aleksey Kurinov

> System view walker generator wrong sorts imports
> 
>
> Key: IGNITE-13218
> URL: https://issues.apache.org/jira/browse/IGNITE-13218
> Project: Ignite
>  Issue Type: Bug
>Reporter: Amelchev Nikita
>Assignee: Aleksey Kurinov
>Priority: Trivial
>  Labels: newbie
>
> {{SystemViewRowAttributeWalkerGenerator}} wrong sorts imports.
> 1. Run SystemViewRowAttributeWalkerGenerator
> 2. Run check codestyle to obtain the error:
> {noformat}
> ComputeJobViewWalker.java:23: Wrong lexicographical order for 
> 'org.apache.ignite.spi.systemview.view.ComputeJobView' import. Should be 
> before 
> 'org.apache.ignite.spi.systemview.view.ComputeJobView.ComputeJobState'. 
> [CustomImportOrder]
> Audit done.
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13265) Historical iterator for atomic group should transfer few more rows than required

2020-08-24 Thread Alexey Scherbakov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183100#comment-17183100
 ] 

Alexey Scherbakov commented on IGNITE-13265:


[~v.pyatkov]

I've left several comments in PR.

Additionaly,

1. What gurantees us (_cntr - margin_)  location in a reserved segment ?
2. Why don't we use CheckpointHistory#earliestCp to decide if a search for a 
minPtr is more effcient from the start of a history ?

> Historical iterator for atomic group should transfer few more rows than 
> required
> 
>
> Key: IGNITE-13265
> URL: https://issues.apache.org/jira/browse/IGNITE-13265
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vladislav Pyatkov
>Assignee: Vladislav Pyatkov
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> On a historical rebalance some updates move from one node to another wherein 
> this update may have various order in nodes. Reordering can happen in smell 
> interval, but it cannot avoid at all in current implementation atomic 
> protocol.
> This mean we will reduce a probably of loosing update if we make a margin 
> from initial counter for the historical iterator on atomic cache.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Comment Edited] (IGNITE-13380) Output IgniteSystemProperties via ignite.sh

2020-08-24 Thread Nikolay Izhikov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183096#comment-17183096
 ] 

Nikolay Izhikov edited comment on IGNITE-13380 at 8/24/20, 9:18 AM:


Hello.

We need to output it with the following things:

* name
* description
* status (deprecated)
* type - string, integer, boolean, etc.

The output should be formatted and sorted alphabetically by the property name.


was (Author: nizhikov):
Hello.

We need to output it with the following things:

* name
* description
* status (deprecated)
* type - string, integer, boolean, etc.

Output should be formatted.

> Output IgniteSystemProperties via ignite.sh
> ---
>
> Key: IGNITE-13380
> URL: https://issues.apache.org/jira/browse/IGNITE-13380
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
>
> Provide the ability output of all available Ignite properties 
> ({{IgniteSystemProperties}}) with its descriptions in the {{ignite.sh}} 
> command. For example, 
> {noformat}
> ignite.sh -systemProps
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13380) Output IgniteSystemProperties via ignite.sh

2020-08-24 Thread Nikolay Izhikov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183096#comment-17183096
 ] 

Nikolay Izhikov commented on IGNITE-13380:
--

Hello.

We need to output it with the following things:

* name
* description
* status (deprecated)
* type - string, integer, boolean, etc.

Output should be formatted.

> Output IgniteSystemProperties via ignite.sh
> ---
>
> Key: IGNITE-13380
> URL: https://issues.apache.org/jira/browse/IGNITE-13380
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
>
> Provide the ability output of all available Ignite properties 
> ({{IgniteSystemProperties}}) with its descriptions in the {{ignite.sh}} 
> command. For example, 
> {noformat}
> ignite.sh -systemProps
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-13380) Output IgniteSystemProperties via ignite.sh

2020-08-24 Thread Amelchev Nikita (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amelchev Nikita updated IGNITE-13380:
-
Description: 
Provide the ability output of all available Ignite properties 
({{IgniteSystemProperties}}) with its descriptions in the {{ignite.sh}} 
command. For example, 
{noformat}
ignite.sh -systemProps
{noformat}

  was:Provide the ability output of all available Ignite properties 
({{IgniteSystemProperties}}) with its descriptions in the {{ignite.sh}} 
command. For example, {{ignite.sh -systemProps}}.


> Output IgniteSystemProperties via ignite.sh
> ---
>
> Key: IGNITE-13380
> URL: https://issues.apache.org/jira/browse/IGNITE-13380
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
>
> Provide the ability output of all available Ignite properties 
> ({{IgniteSystemProperties}}) with its descriptions in the {{ignite.sh}} 
> command. For example, 
> {noformat}
> ignite.sh -systemProps
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-13380) Output IgniteSystemProperties via ignite.sh

2020-08-24 Thread Amelchev Nikita (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amelchev Nikita updated IGNITE-13380:
-
Description: Provide the ability output of all available Ignite properties 
({{IgniteSystemProperties}}) with its descriptions in the {{ignite.sh}} 
command. For example, {{ignite.sh -systemProps}}.  (was: Provide the ability 
output of all available Ignite properties (`IgniteSystemProperties`) with its 
descriptions in the `ignite.sh` command. For example, `ignite.sh -systemProps`.)

> Output IgniteSystemProperties via ignite.sh
> ---
>
> Key: IGNITE-13380
> URL: https://issues.apache.org/jira/browse/IGNITE-13380
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
>
> Provide the ability output of all available Ignite properties 
> ({{IgniteSystemProperties}}) with its descriptions in the {{ignite.sh}} 
> command. For example, {{ignite.sh -systemProps}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-11719) Document cluster read-only mode

2020-08-24 Thread Stanilovsky Evgeny (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-11719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183066#comment-17183066
 ] 

Stanilovsky Evgeny commented on IGNITE-11719:
-

[~Artem Budnikov] thanks, looks like important to describe which operations are 
possible in certain mode, for example : rebalance in 
ClusterState.ACTIVE_READ_ONLY and so on.

> Document cluster read-only mode
> ---
>
> Key: IGNITE-11719
> URL: https://issues.apache.org/jira/browse/IGNITE-11719
> Project: Ignite
>  Issue Type: New Feature
>  Components: documentation
>Reporter: Sergey Antonov
>Assignee: Artem Budnikov
>Priority: Critical
>  Labels: important
> Fix For: 2.9
>
>
> We should document cluster wide read-only mode.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13270) Massive CPU burn in 2.8.0 and 2.8.1 after upgrading from 2.7.5

2020-08-24 Thread Taras Ledkov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183055#comment-17183055
 ] 

Taras Ledkov commented on IGNITE-13270:
---

[~zstan]
About reproducer with invalid results:
The root cause is using the index on VARCHAR column {{idx_B_COL2}} and lookup 
the integer value at this index.
We have a problem with lookup value with type conversion at the index.
The reproducer may be shorter:
{code}
sql("CREATE TABLE TEST (ID INTEGER PRIMARY KEY, VAL VARCHAR(100), VAL1 
int)");
sql("CREATE INDEX IDX_VAL ON TEST (VAL)");

   // Fail
res = sql("SELECT * FROM TEST WHERE VAL=?", i).getAll();

   // OK
res = sql("SELECT * FROM TEST WHERE VAL=?", 
Integer.toString(i)).getAll();
{code}

> Massive CPU burn in 2.8.0 and 2.8.1 after upgrading from 2.7.5
> --
>
> Key: IGNITE-13270
> URL: https://issues.apache.org/jira/browse/IGNITE-13270
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.8, 2.8.1
>Reporter: Tanmay Ambre
>Priority: Major
> Attachments: IDX13270.java, IgniteSqlDistributedJoinSelfTest.java
>
>
> After upgrading to ignite 2.8.0 from 2.7.5 the following query is causing 
> massive CPU burn. We have a 8 node server cluster for Ignite (with 32 CPUs on 
> each node). with 2.8.0, 2 nodes are maxed out on CPU. with 2.8.1 all nodes 
> are above 80%. On 2.7.5 the same query was performing very nicely. 
> The query is a join with 3 tables having same affinity key. however, we dont 
> know the affinity key when we are querying and there this results in a merge 
> scan. 
> Query:
> select A.A_ID, A.AFFINITYKEY, B.B_ID from A, B, C
> where
> A.A_ID = B.A_ID AND
> B.B_ID = C.B_ID AND
> A.AFFINITYKEY = B.AFFINITYKEY AND
> B.AFFINITYKEY = C.AFFINITYKEY AND
> B.B_ID = ? AND
> C.C_ID = ?
>  
> Performance:
> 2.7.5 ~ 2 to 5 ms
> 2.8.0 ~ 6 to 12 seconds
> 2.8.1 ~ within 3 seconds
>  
> Volume (we have a EDA based system where messages come in burst. One burst 
> has approx. 150K events - each event correspond to one query). 
>  
> throughput of our java based processor (that fires this query):
> 2.7.5 ~ 750 transactions per second
> 2.8.0 ~ 3 transactions per second
> 2.8.1 ~ 6 transactions per second
>  
> CPU burn
> 2.7.5 ~ 10 to 15% on each node
> 2.8.0 ~ 100% on 2 nodes other nodes are idling
> 2.8.1 ~ 80 to 90% on each node
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (IGNITE-12438) Extend communication protocol to establish client-server connection

2020-08-24 Thread Artem Budnikov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Artem Budnikov reassigned IGNITE-12438:
---

Assignee: Artem Budnikov

> Extend communication protocol to establish client-server connection
> ---
>
> Key: IGNITE-12438
> URL: https://issues.apache.org/jira/browse/IGNITE-12438
> Project: Ignite
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Alexey Goncharuk
>Assignee: Artem Budnikov
>Priority: Major
>  Labels: important
> Fix For: 2.9
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Recently there was quite a lot of questions related to thick clients 
> connectivity issues when the clients are deployed in a k8s pod [1]. The 
> general issue here is clients reporting network address which are not 
> reachable from server nodes. At the same time, the clients can connect to 
> server nodes.
> An idea of how to fix this is as follows:
>  * Make sure that think clients discovery SPI always maintains a connection 
> to a server node (this should be already implemented)
>  * (Optionally) detect when a client has only one-way connectivity with the 
> server nodes. This part should be investigated. We need this to avoid server 
> nodes attempt to connect to a client and send communication request to the 
> client node faster
>  * When a server attempts to establish a connection with a client, check if 
> client is unreachable or the previous connection attempt failed. If so, send 
> a discovery message to the client to force a client-server connection. In 
> this case, server will be able to send the original message via the newly 
> established connection.
> [1] 
> https://stackoverflow.com/questions/59192075/ignite-communicationspi-questions-in-paas-environment/59232504



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-13377) WalModeChangeAdvancedSelfTest.testServerRestartNonCoordinator is flaky

2020-08-24 Thread Alexey Scherbakov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Scherbakov updated IGNITE-13377:
---
Reviewer: Alexey Scherbakov

> WalModeChangeAdvancedSelfTest.testServerRestartNonCoordinator is flaky 
> ---
>
> Key: IGNITE-13377
> URL: https://issues.apache.org/jira/browse/IGNITE-13377
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vladislav Pyatkov
>Assignee: Vladislav Pyatkov
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-11719) Document cluster read-only mode

2020-08-24 Thread Artem Budnikov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-11719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183027#comment-17183027
 ] 

Artem Budnikov commented on IGNITE-11719:
-

First version of the documentation: 
https://github.com/apache/ignite/blob/IGNITE-7595/docs/_docs/cluster-states.adoc

 

> Document cluster read-only mode
> ---
>
> Key: IGNITE-11719
> URL: https://issues.apache.org/jira/browse/IGNITE-11719
> Project: Ignite
>  Issue Type: New Feature
>  Components: documentation
>Reporter: Sergey Antonov
>Assignee: Artem Budnikov
>Priority: Critical
>  Labels: important
> Fix For: 2.9
>
>
> We should document cluster wide read-only mode.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13334) Document initial tracing implementation

2020-08-24 Thread Artem Budnikov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183026#comment-17183026
 ] 

Artem Budnikov commented on IGNITE-13334:
-

First version of the page: 
https://github.com/apache/ignite/blob/IGNITE-7595/docs/_docs/monitoring-metrics/tracing.adoc

 

> Document initial tracing implementation
> ---
>
> Key: IGNITE-13334
> URL: https://issues.apache.org/jira/browse/IGNITE-13334
> Project: Ignite
>  Issue Type: Task
>  Components: documentation
>Reporter: Aleksey Plekhanov
>Assignee: Artem Budnikov
>Priority: Major
>  Labels: important
> Fix For: 2.9
>
>
> Document initial tracing implementation



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-13380) Output IgniteSystemProperties via ignite.sh

2020-08-24 Thread Amelchev Nikita (Jira)
Amelchev Nikita created IGNITE-13380:


 Summary: Output IgniteSystemProperties via ignite.sh
 Key: IGNITE-13380
 URL: https://issues.apache.org/jira/browse/IGNITE-13380
 Project: Ignite
  Issue Type: New Feature
Reporter: Amelchev Nikita
Assignee: Amelchev Nikita


Provide the ability output of all available Ignite properties 
(`IgniteSystemProperties`) with its descriptions in the `ignite.sh` command. 
For example, `ignite.sh -systemProps`.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)