[jira] [Updated] (IGNITE-4293) Deserialized value is cached if queries are enabled

[Alexandr Kuramshin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-4293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexandr Kuramshin updated IGNITE-4293:
---
Assignee: Semen Boikov  (was: Alexandr Kuramshin)

> Deserialized value is cached if queries are enabled
> ---
>
> Key: IGNITE-4293
> URL: https://issues.apache.org/jira/browse/IGNITE-4293
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 1.7, 1.8
>Reporter: Valentin Kulichenko
>Assignee: Semen Boikov
>Priority: Critical
>
> Here is the problematic piece of code in {{IgniteCacheObjectProcessorImpl}}:
> {code}
> boolean storeVal = ctx.config().isPeerClassLoadingEnabled() ||
> GridQueryProcessor.isEnabled(ccfg) ||
> !ccfg.isCopyOnRead();
> {code}
> This flag is set to true if queries are enabled even when binary marshaller 
> is used (this condition makes sense to other marshallers though). It is then 
> use in {{BinaryObjectImpl.deserializeValue}}:
> {code}
> if (coCtx != null && coCtx.storeValue())
> obj = obj0;
> {code}
> As a result, memory consumption doubles.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (IGNITE-4468) RoundRobinLoadBalancingSpi‘s documentation and Javadoc description incorrectly

[Yujue Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-4468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yujue Li resolved IGNITE-4468.
--
Resolution: Done

I will commit a pr.

> RoundRobinLoadBalancingSpi‘s documentation and Javadoc description incorrectly
> --
>
> Key: IGNITE-4468
> URL: https://issues.apache.org/jira/browse/IGNITE-4468
> Project: Ignite
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: 1.8
>Reporter: Yujue Li
>Priority: Minor
>  Labels: documentation
> Fix For: 1.9
>
>
> Class RoundRobinLoadBalancingSpi
> Description of the perTask attribute in this class is confusing.
> Some places describe the default value of true, some places describe the 
> default value of false, and from the source code analysis, the default value 
> should be false.
> I hope the community can unify the adjustment of Javadoc and related 
> documents.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (IGNITE-3537) Add tests checking that data structures work inside user transactions

[Vadim (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-3537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vadim reassigned IGNITE-3537:
-

Assignee: Vadim  (was: Milap Wadhwa)

> Add tests checking that data structures work inside user transactions
> -
>
> Key: IGNITE-3537
> URL: https://issues.apache.org/jira/browse/IGNITE-3537
> Project: Ignite
>  Issue Type: Task
>  Components: data structures
>Reporter: Semen Boikov
>Assignee: Vadim
>Priority: Minor
>  Labels: newbie
>
> Need create tests to check that Ignite data structures work inside user 
> transaction. Now data structures should use independent transactions, test 
> should jus check that there are no exceptions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (IGNITE-4230) Create documentation for the integration with Tableau

[Prachi Garg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prachi Garg closed IGNITE-4230.
---

> Create documentation for the integration with Tableau
> -
>
> Key: IGNITE-4230
> URL: https://issues.apache.org/jira/browse/IGNITE-4230
> Project: Ignite
>  Issue Type: Task
>  Components: documentation
>Reporter: Denis Magda
>Assignee: Prachi Garg
> Fix For: 2.0
>
>
> Let's create a simple documentation that will show how to connect from 
> Tableau to Ignite and what steps have to be performed to fulfill this.
> The documentation has to be placed under this section:
> https://apacheignite-mix.readme.io/docs/overview-1
> Refer to this discussion for more details:
> http://apache-ignite-developers.2346864.n4.nabble.com/Connecting-to-Ignite-from-Tableau-td12065.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4230) Create documentation for the integration with Tableau

[Prachi Garg (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781245#comment-15781245
 ] 

Prachi Garg commented on IGNITE-4230:
-

Fixed minor issues.

> Create documentation for the integration with Tableau
> -
>
> Key: IGNITE-4230
> URL: https://issues.apache.org/jira/browse/IGNITE-4230
> Project: Ignite
>  Issue Type: Task
>  Components: documentation
>Reporter: Denis Magda
>Assignee: Prachi Garg
> Fix For: 2.0
>
>
> Let's create a simple documentation that will show how to connect from 
> Tableau to Ignite and what steps have to be performed to fulfill this.
> The documentation has to be placed under this section:
> https://apacheignite-mix.readme.io/docs/overview-1
> Refer to this discussion for more details:
> http://apache-ignite-developers.2346864.n4.nabble.com/Connecting-to-Ignite-from-Tableau-td12065.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (IGNITE-4348) Documentation: RDBMS Integration using Web Console

[Denis Magda (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denis Magda closed IGNITE-4348.
---

> Documentation: RDBMS Integration using Web Console
> --
>
> Key: IGNITE-4348
> URL: https://issues.apache.org/jira/browse/IGNITE-4348
> Project: Ignite
>  Issue Type: Task
>Affects Versions: 1.8
>Reporter: Denis Magda
>Assignee: Denis Magda
> Fix For: 2.0
>
> Attachments: SchemaImport-2.png, WP_20161220_11_33_55_Pro.jpg
>
>
> Ignite has a documentation which describes how to set up "automatic 
> persistence" relying on the schema-import tool.
> https://apacheignite-mix.readme.io/docs/automatic-persistence
> Let's create a similar one but basing it on Web Console capabilities briefly 
> listed on this page
> https://ignite.apache.org/features/rdbmsintegration.html
> The schema-import based document has to be removed after that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (IGNITE-4045) .NET: Support DML API

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15778622#comment-15778622
 ] 

Pavel Tupitsyn edited comment on IGNITE-4045 at 12/27/16 4:23 PM:
--

1) BinaryObject.Equals fixed.
2) We don't need this. Identity resolver in .NET is needed only to write 
correct hash code to a stream. Enums are written as [typeId, value]. There is 
no hash code. In Java there is special handing for BinaryEnum, but I suspect 
that this code is never called. Let's see what should be done in a separate 
task: IGNITE-4500
3) There is already a ticket: IGNITE-4397. In current ticket FieldComparer is 
hidden from public API, please ignore it.
4) I agree. This is done intentionally because we are on a very hot path here 
and want to avoid any allocations or overhead associated with {{ReadByte}} and 
{{ReadByteArray}}. I've refactored this to a generic {{IBinaryStream.Apply}} 
method which allows operating on byte pointer directly without allocations.
5) IGNITE-4495

* Implemented public methods of {{IEqualityComparer}}
* Fixed hash code handling for wrappers ({{DateTimeHolder}}, 
{{SerializableObjectHolder}})
* Fixed array handling in BinaryObject.Equals
* Added a lot more tests


was (Author: ptupitsyn):
1) BinaryObject.Equals fixed.
2) We don't need this. Identity resolver in .NET is needed only to write 
correct hash code to a stream. Enums are written as [typeId, value]. There is 
no hash code. In Java there is special handing for BinaryEnum, but I suspect 
that this code is never called. Let's see what should be done in a separate 
task: IGNITE-4500
3) There is already a ticket: IGNITE-4397. In current ticket FieldComparer is 
hidden from public API, please ignore it.
4) I agree. This is done intentionally because we are on a very hot path here 
and want to avoid any allocations or overhead associated with {{ReadByte}} and 
{{ReadByteArray}}. I've refactored this to a generic {{IBinaryStream.Apply}} 
method which allows operating on byte pointer directly without allocations.
5) IGNITE-4495

Implemented public methods of {{IEqualityComparer}}, fixed hash code handling 
for wrappers ({{DateTimeHolder}}, {{SerializableObjectHolder}}), added a lot 
more tests.

> .NET: Support DML API
> -
>
> Key: IGNITE-4045
> URL: https://issues.apache.org/jira/browse/IGNITE-4045
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Reporter: Denis Magda
>Assignee: Pavel Tupitsyn
>  Labels: roadmap
> Fix For: 2.0
>
>
> Ignite's Java component will provide support for DML soon (IGNITE-2294). At 
> she same time DML will be supported at the level of ODBC and JDBC drivers.
> As the next step we should include the similar functionality into Ignite.NET 
> by doing the following:
> - Implement DML API;
> - Enhance {{QueryExample.cs}} by doing INSERTs instead of cache.puts and 
> adding UPDATE and DELETE operation examples.
> - Add documentation to Ignite.NET readme.io covering the feature. Most like 
> most of the content can be take from the general documentation when this 
> ticket IGNITE-4018 is ready 
> (https://apacheignite.readme.io/docs/distributed-dml).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (IGNITE-4045) .NET: Support DML API

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15778622#comment-15778622
 ] 

Pavel Tupitsyn edited comment on IGNITE-4045 at 12/27/16 4:21 PM:
--

1) BinaryObject.Equals fixed.
2) We don't need this. Identity resolver in .NET is needed only to write 
correct hash code to a stream. Enums are written as [typeId, value]. There is 
no hash code. In Java there is special handing for BinaryEnum, but I suspect 
that this code is never called. Let's see what should be done in a separate 
task: IGNITE-4500
3) There is already a ticket: IGNITE-4397. In current ticket FieldComparer is 
hidden from public API, please ignore it.
4) I agree. This is done intentionally because we are on a very hot path here 
and want to avoid any allocations or overhead associated with {{ReadByte}} and 
{{ReadByteArray}}. I've refactored this to a generic {{IBinaryStream.Apply}} 
method which allows operating on byte pointer directly without allocations.
5) IGNITE-4495

Implemented public methods of {{IEqualityComparer}}, fixed hash code handling 
for wrappers ({{DateTimeHolder}}, {{SerializableObjectHolder}}), added a lot 
more tests.


was (Author: ptupitsyn):
2) We don't need this. Identity resolver in .NET is needed only to write 
correct hash code to a stream. Enums are written as [typeId, value]. There is 
no hash code. In Java there is special handing for BinaryEnum, but I suspect 
that this code is never called. Let's see what should be done in a separate 
task: IGNITE-4500
3) There is already a ticket: IGNITE-4397. In current ticket FieldComparer is 
hidden from public API, please ignore it.
4) I agree. This is done intentionally because we are on a very hot path here 
and want to avoid any allocations or overhead associated with {{ReadByte}} and 
{{ReadByteArray}}. I've refactored this to a generic {{IBinaryStream.Apply}} 
method which allows operating on byte pointer directly without allocations.
5) IGNITE-4495

> .NET: Support DML API
> -
>
> Key: IGNITE-4045
> URL: https://issues.apache.org/jira/browse/IGNITE-4045
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Reporter: Denis Magda
>Assignee: Pavel Tupitsyn
>  Labels: roadmap
> Fix For: 2.0
>
>
> Ignite's Java component will provide support for DML soon (IGNITE-2294). At 
> she same time DML will be supported at the level of ODBC and JDBC drivers.
> As the next step we should include the similar functionality into Ignite.NET 
> by doing the following:
> - Implement DML API;
> - Enhance {{QueryExample.cs}} by doing INSERTs instead of cache.puts and 
> adding UPDATE and DELETE operation examples.
> - Add documentation to Ignite.NET readme.io covering the feature. Most like 
> most of the content can be take from the general documentation when this 
> ticket IGNITE-4018 is ready 
> (https://apacheignite.readme.io/docs/distributed-dml).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4157) Use discovery custom messages instead of marshaller cache

[Sergey Chugunov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780669#comment-15780669
 ] 

Sergey Chugunov commented on IGNITE-4157:
-

Finished with moving functionality to two distinct classes: *DiscoveryDataBag* 
from public API and *DiscoveryDataPacket* in internal implementation.
If all tests pass all comments to initial PR will be addressed.

> Use discovery custom messages instead of marshaller cache
> -
>
> Key: IGNITE-4157
> URL: https://issues.apache.org/jira/browse/IGNITE-4157
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Reporter: Alexey Goncharuk
>Assignee: Sergey Chugunov
> Fix For: 2.0
>
>
> Currently we use system caches for keeping classname to class ID mapping and 
> for storing binary metadata
> This has several serious disadvantages:
> 1) We need to introduce at least two additional thread pools for each of 
> these caches
> 2) Since cache operations require stable topology, registering a class ID or 
> updating metadata inside a transaction or another cache operation is tricky 
> and deadlock-prone.
> 3) It may be beneficial in some cases to have nodes with no caches at all, 
> currently this is impossible because system caches are always present.
> 4) Reading binary metadata leads to huge local contention, caching metadata 
> values in a local map doubles memory consumption
> I suggest we use discovery custom events for these purposes. Each node will 
> have a corresponding local map (state) which will be updated inside custom 
> event handler. From the first point of view, this should remove all the 
> disadvantages above.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4003) Slow or faulty client can stall the whole cluster.

[Semen Boikov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780621#comment-15780621
 ] 

Semen Boikov commented on IGNITE-4003:
--

Hi Andrey,

I started review, so far found two issues (left TODOs in branch), please take a 
look:
- 'onTimeout' method is called from dedicated thread (one thread per node), it 
is not safe to retry connect from this callback
- when connect is cancelled by ConnectionTimeoutObject you call 
'recoveryDesc.release()' from comm worker. Actually at this moment session can 
be already created and used, so call 'release' can break something. Need move 
descriptor release to NIO thread

Thanks!

> Slow or faulty client can stall the whole cluster.
> --
>
> Key: IGNITE-4003
> URL: https://issues.apache.org/jira/browse/IGNITE-4003
> Project: Ignite
>  Issue Type: Bug
>  Components: cache, general
>Affects Versions: 1.7
>Reporter: Vladimir Ozerov
>Assignee: Andrey Gura
>Priority: Critical
> Fix For: 2.0
>
>
> Steps to reproduce:
> 1) Start two server nodes and some data to cache.
> 2) Start a client from Docker subnet, which is not visible from the outside. 
> Client will join the cluster.
> 3) Try to put something to cache or start another node to force rabalance.
> Cluster is stuck at this moment. Root cause - servers are constantly trying 
> to establish outgoing connection to the client, but fail as Docker subnet is 
> not visible from the outside. It may stop virtually all cluster operations.
> Typical thread dump:
> {code}
> org.apache.ignite.IgniteCheckedException: Failed to send message (node may 
> have left the grid or TCP connection cannot be established due to firewall 
> issues) [node=TcpDiscoveryNode [id=a15d74c2-1ec2-4349-9640-aeacd70d8714, 
> addrs=[127.0.0.1, 172.17.0.6], sockAddrs=[/127.0.0.1:0, /127.0.0.1:0, 
> /172.17.0.6:0], discPort=0, order=7241, intOrder=3707, 
> lastExchangeTime=1474096941045, loc=false, ver=1.5.23#20160526-sha1:259146da, 
> isClient=true], topic=T4 [topic=TOPIC_CACHE, 
> id1=949732fd-1360-3a58-8d9e-0ff6ea6182cc, 
> id2=a15d74c2-1ec2-4349-9640-aeacd70d8714, id3=2], msg=GridContinuousMessage 
> [type=MSG_EVT_NOTIFICATION, routineId=7e13c48e-6933-48b2-9f15-8d92007930db, 
> data=null, futId=null], policy=2]
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1129)
>  [ignite-core-1.5.23.jar:1.5.23]
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager.sendOrderedMessage(GridIoManager.java:1347)
>  [ignite-core-1.5.23.jar:1.5.23]
>   at 
> org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.sendWithRetries(GridContinuousProcessor.java:1227)
>  ~[ignite-core-1.5.23.jar:1.5.23]
>   at 
> org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.sendWithRetries(GridContinuousProcessor.java:1198)
>  ~[ignite-core-1.5.23.jar:1.5.23]
>   at 
> org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.sendWithRetries(GridContinuousProcessor.java:1180)
>  ~[ignite-core-1.5.23.jar:1.5.23]
>   at 
> org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.sendNotification(GridContinuousProcessor.java:841)
>  ~[ignite-core-1.5.23.jar:1.5.23]
>   at 
> org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.addNotification(GridContinuousProcessor.java:800)
>  ~[ignite-core-1.5.23.jar:1.5.23]
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.onEntryUpdate(CacheContinuousQueryHandler.java:787)
>  [ignite-core-1.5.23.jar:1.5.23]
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.access$700(CacheContinuousQueryHandler.java:91)
>  [ignite-core-1.5.23.jar:1.5.23]
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler$1.onEntryUpdated(CacheContinuousQueryHandler.java:412)
>  [ignite-core-1.5.23.jar:1.5.23]
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.onEntryUpdated(CacheContinuousQueryManager.java:343)
>  [ignite-core-1.5.23.jar:1.5.23]
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.onEntryUpdated(CacheContinuousQueryManager.java:250)
>  [ignite-core-1.5.23.jar:1.5.23]
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.initialValue(GridCacheMapEntry.java:3476)
>  [ignite-core-1.5.23.jar:1.5.23]
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture$MiniFuture.onResult(GridDhtForceKeysFuture.java:548)
>  [ignite-core-1.5.23.jar:1.5.23]
>   at 
> 

[jira] [Commented] (IGNITE-4372) Set up code coverage reports

[Ksenia Rybakova (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780606#comment-15780606
 ] 

Ksenia Rybakova commented on IGNITE-4372:
-

- Trying solution with a new module to aggregate all reports. 

> Set up code coverage reports
> 
>
> Key: IGNITE-4372
> URL: https://issues.apache.org/jira/browse/IGNITE-4372
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Ksenia Rybakova
>Assignee: Ksenia Rybakova
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (IGNITE-4489) Maintain correct MERGE semantic in DML

[Alexander Paschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-4489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Paschenko updated IGNITE-4489:

Fix Version/s: (was: 1.8)
   2.0

> Maintain correct MERGE semantic in DML
> --
>
> Key: IGNITE-4489
> URL: https://issues.apache.org/jira/browse/IGNITE-4489
> Project: Ignite
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 1.8
>Reporter: Alexander Paschenko
>Assignee: Alexander Paschenko
> Fix For: 2.0
>
>
> Currently it's impossible to MERGE object in UPDATE style - i.e. when key is 
> present in cache, unaffected field values should be retained, and instead of 
> building new object we should base it on previous one for given key.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (IGNITE-4490) Optimize DML for fast INSERT and MERGE

[Alexander Paschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-4490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Paschenko updated IGNITE-4490:

Fix Version/s: (was: 1.8)
   2.0

> Optimize DML for fast INSERT and MERGE
> --
>
> Key: IGNITE-4490
> URL: https://issues.apache.org/jira/browse/IGNITE-4490
> Project: Ignite
>  Issue Type: Improvement
>  Components: SQL
>Affects Versions: 1.8
>Reporter: Alexander Paschenko
>Assignee: Vladimir Ozerov
> Fix For: 2.0
>
>
> It's possible to avoid any SQL querying and map some INSERT and MERGE 
> statements to cache operations in a way similar to that of UPDATE and DELETE 
> - i.e. don't make queries when there are no expressions to evaluate in the 
> query and enhance update plans to perform direct cache operations when INSERT 
> and MERGE affect columns {{_key}} and {{_val}} only.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4499) TcpDiscoverySpi is not reliable in some network split scenarios.

[Vyacheslav Daradur (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780538#comment-15780538
 ] 

Vyacheslav Daradur commented on IGNITE-4499:


I think it will decide in IGNITE-4501

> TcpDiscoverySpi is not reliable in some network split scenarios.
> 
>
> Key: IGNITE-4499
> URL: https://issues.apache.org/jira/browse/IGNITE-4499
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: 1.6
>Reporter: Alexei Scherbakov
> Fix For: 2.0
>
>
> Where is a possible caveat in current discovery implementation using ring of 
> nodes.
> Imagine grid consisting of nodes A B C D
> Let them form the ring:
> A-B-C-D-A
> If network connectivity issues will arise between nodes A-C and B-D
> discovery spi will never know it and will continue to assume the topology is 
> valid. 
> On other side, TcpCommunicationSpi will try to run transaction on this 
> topology and never will succeed.
> We must drop nodes from topology on communication spi errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4363) Inner properties mutation broken in SQL UPDATE

[Alexander Paschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780519#comment-15780519
 ] 

Alexander Paschenko commented on IGNITE-4363:
-

[~vozerov] TC looks fine to me, all tests that have failed appear to be flaky 
ones.

> Inner properties mutation broken in SQL UPDATE
> --
>
> Key: IGNITE-4363
> URL: https://issues.apache.org/jira/browse/IGNITE-4363
> Project: Ignite
>  Issue Type: Bug
>  Components: binary, SQL
>Affects Versions: 1.8
>Reporter: Alexander Paschenko
>Assignee: Alexander Paschenko
> Fix For: 2.0
>
>
> Discovered in course of working on IGNITE-4340.
> Say, we have following type for cache values
> {code:java}
> static final class AllTypes implements Serializable {
> /**
>  * Data Long.
>  */
> @QuerySqlField
> Long longCol;
> /**
>  * Inner type object.
>  */
> @QuerySqlField
> InnerType innerTypeCol;
> /** */
> static final class InnerType implements Serializable {
> /** */
> @QuerySqlField
> Long innerLongCol;
> /** */
> @QuerySqlField
> String innerStrCol;
>}
> }
> {code}
> Queries like this fail for both optimized and binary marshaller:
> {code:sql}
> UPDATE AllTypes set innerLongCol = ?
> {code}
> For optimized, current DML implementation mutates existing inner property 
> thus confusing DML statements re-run logic (query re-runs because engine sees 
> value as concurrently modified, though the only change is its own, and 
> ultimately fails). The solution is to clone inner objects and set new 
> property values on them because we need to have old value pristine.
> For binary, current DML implementation does not honor properties hierarchy 
> and, for above example, just sets {{innerLongCol}} field on {{AllTypes}} 
> binary object and not its child {{InnerType}} object. Thus, index will be 
> updated and SELECTs for that column will return correct value for that field, 
> but inner state of target property {{innerTypeCol}} does not change, and 
> {{AllTypes}} object gets its own odd field {{innerLongCol}} which it does not 
> know how to do with (no metadata about it in type descriptor).
> The patch for both problems is ready and lying on my shelf waiting for 1.8 
> release to happen to be applied. Then this will ultimately be fixed with the 
> rest of known problems/improvements (Jira issues for them will follow).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4459) Hadoop: IgniteHadoopWeightedMapReducePlanner must be default planner

[Taras Ledkov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780497#comment-15780497
 ] 

Taras Ledkov commented on IGNITE-4459:
--

[Tests 
result|http://195.239.208.174/project.html?projectId=IgniteTests=projectOverview_IgniteTests=pull%2F1391%2Fhead]

> Hadoop: IgniteHadoopWeightedMapReducePlanner must be default planner
> 
>
> Key: IGNITE-4459
> URL: https://issues.apache.org/jira/browse/IGNITE-4459
> Project: Ignite
>  Issue Type: Task
>  Components: hadoop
>Affects Versions: 1.8
>Reporter: Vladimir Ozerov
>Assignee: Taras Ledkov
> Fix For: 2.0
>
>
> It offers much better job distribution as current default implementation. 
> Let's make it default and remove the previous one.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (IGNITE-4459) Hadoop: IgniteHadoopWeightedMapReducePlanner must be default planner

[Taras Ledkov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780497#comment-15780497
 ] 

Taras Ledkov edited comment on IGNITE-4459 at 12/27/16 2:26 PM:


[Tests 
result|http://195.239.208.174/project.html?projectId=IgniteTests=projectOverview_IgniteTests=pull%2F1391%2Fhead]
Code review: 
[IGNT-CR-62|http://reviews.ignite.apache.org/ignite/review/IGNT-CR-62]


was (Author: tledkov-gridgain):
[Tests 
result|http://195.239.208.174/project.html?projectId=IgniteTests=projectOverview_IgniteTests=pull%2F1391%2Fhead]

> Hadoop: IgniteHadoopWeightedMapReducePlanner must be default planner
> 
>
> Key: IGNITE-4459
> URL: https://issues.apache.org/jira/browse/IGNITE-4459
> Project: Ignite
>  Issue Type: Task
>  Components: hadoop
>Affects Versions: 1.8
>Reporter: Vladimir Ozerov
>Assignee: Taras Ledkov
> Fix For: 2.0
>
>
> It offers much better job distribution as current default implementation. 
> Let's make it default and remove the previous one.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4459) Hadoop: IgniteHadoopWeightedMapReducePlanner must be default planner

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780491#comment-15780491
 ] 

ASF GitHub Bot commented on IGNITE-4459:


GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/1391

IGNITE-4459  Hadoop: IgniteHadoopWeightedMapReducePlanner must be default 
planner



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4459

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1391.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1391


commit c893da70a9757b16b0799adc8eaa29fa1b03d06e
Author: tledkov-gridgain 
Date:   2016-12-21T11:54:33Z

IGNITE-4399: IGFS: Merged IgfsSecondaryFileSystem and 
IgfsSecondaryFileSystemV2 interfaces. This closes #1346.

commit c5882a85f4e3a1f61723ac54fd92f087684df6da
Author: devozerov 
Date:   2016-12-26T11:15:42Z

Merge branch 'master' into ignite-2.0

commit 6c7d982327b74ec0b8e4d80b3392404c5f0b7727
Author: tledkov-gridgain 
Date:   2016-12-27T14:14:24Z

IGNITE-4459: remove IgniteHadoopMapReducePlanner, use Hadoop: 
IgniteHadoopWeightedMapReducePlanner by default




> Hadoop: IgniteHadoopWeightedMapReducePlanner must be default planner
> 
>
> Key: IGNITE-4459
> URL: https://issues.apache.org/jira/browse/IGNITE-4459
> Project: Ignite
>  Issue Type: Task
>  Components: hadoop
>Affects Versions: 1.8
>Reporter: Vladimir Ozerov
>Assignee: Taras Ledkov
> Fix For: 2.0
>
>
> It offers much better job distribution as current default implementation. 
> Let's make it default and remove the previous one.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4501) Improvement of connection in a cluster of new node

[Vyacheslav Daradur (JIRA)]

Vyacheslav Daradur created IGNITE-4501:
--

 Summary: Improvement of connection in a cluster of new node
 Key: IGNITE-4501
 URL: https://issues.apache.org/jira/browse/IGNITE-4501
 Project: Ignite
  Issue Type: Improvement
Reporter: Vyacheslav Daradur


h3. Main description:
Cluster nodes connect a ring.
For example: we have 6 nodes: A, B, C, D, E, F. 
They can connect a ring in any possible way: A-B-C-D-E-F-A, or A-F-B-E-C-D-A, 
etc.
If some node leaves topology, adjacent nodes must reconnect. 
If nodes A, B, C are in same physical place, nodes D, E, F are in other place, 
and places lost connect each other, we will have many ways of reconnections.
At best case, if we had a ring: A-B-CxD-E-FxA ('x' means disconnect) -- then we 
have only one reconnect (C
will be connected to A or F will be connected to D -- depends on what part of 
the cluster was alive.
Also, if we had a not ring: AxFxBxExCxDxA -- then we have a lot of 
reconnections (A to B, B to C, C to A -- in general n/2 reconnections, where n 
-- number of nodes). 
h3. Approach:
It is necessary to develop approach of node insertion to the correct place for 
creation of the correct ring-topology.
h3. Solutions:
Main idea is a sorting according to latency.
* group nodes in arcs on an ARC_ID. (manualy?)
* implement NodeComparator (nodes on the same host : nodes on the same subnet : 
other nodes). We will use it when we connect a new node.
* [dev list 
thread|http://mail-archives.apache.org/mod_mbox/ignite-dev/201612.mbox/%3CCAN+WSNyWYXSXEBpGErVt72zTgi2pTQzUWLv8JY=ke83-5-r...@mail.gmail.com%3E]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4212) Ignite Benchmarking Simplification and Automation

[Oleg Ostanin (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780407#comment-15780407
 ] 

Oleg Ostanin commented on IGNITE-4212:
--

[~dmagda]
I've sent a message to the dev list about removing some configuration files 
from compiled module. Please look and tell me what do you think about this 
subject.

> Ignite Benchmarking Simplification and Automation
> -
>
> Key: IGNITE-4212
> URL: https://issues.apache.org/jira/browse/IGNITE-4212
> Project: Ignite
>  Issue Type: Task
>Reporter: Denis Magda
>Assignee: Oleg Ostanin
> Fix For: 2.0
>
>
> There is a plenty of useful Yardstick benchmarks located in ignite-yardstick 
> module that is used by the community to check Ignite performance across 
> deployments of different configurations.
> However, it's not easy to start with the benchmarking process because the 
> user needs to download Ignite, build and set up benchmarks and only after 
> that run them.
> The goal of this task is to simplify the process in the following way:
> 1) ignite-yardstick benchmarks have to be pre-compiled and available with 
> every Ignite deliverable.
> 2) every deliverable must contain an executable (bat or sh file) with a clear 
> instruction on how to start a specific benchmark from the console.
> 3) the executable has to use some default yardstick configuration. The first 
> configuration should be intented for local execution (multicast IP finder) 
> and the second needs to be AWS oriented.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4458) Hadoop: striped shuffle mode must be enabled by default

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780351#comment-15780351
 ] 

ASF GitHub Bot commented on IGNITE-4458:


GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/1390

IGNITE-4458  Hadoop: striped shuffle mode must be enabled by default



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4458

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1390.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1390


commit c893da70a9757b16b0799adc8eaa29fa1b03d06e
Author: tledkov-gridgain 
Date:   2016-12-21T11:54:33Z

IGNITE-4399: IGFS: Merged IgfsSecondaryFileSystem and 
IgfsSecondaryFileSystemV2 interfaces. This closes #1346.

commit c5882a85f4e3a1f61723ac54fd92f087684df6da
Author: devozerov 
Date:   2016-12-26T11:15:42Z

Merge branch 'master' into ignite-2.0

commit d2b3989b2c6b71e12de4fd59c1108408355c4fc0
Author: tledkov-gridgain 
Date:   2016-12-27T12:29:53Z

IGNITE-4458: sets default value of the 
HadoopJobProperty.SHUFFLE_MAPPER_STRIPED_OUTPUT to 'true'




> Hadoop: striped shuffle mode must be enabled by default
> ---
>
> Key: IGNITE-4458
> URL: https://issues.apache.org/jira/browse/IGNITE-4458
> Project: Ignite
>  Issue Type: Task
>  Components: hadoop
>Affects Versions: 1.8
>Reporter: Vladimir Ozerov
>Assignee: Taras Ledkov
> Fix For: 2.0
>
>
> Striped mode is enabled with 
> {{HadoopJobProperty.SHUFFLE_MAPPER_STRIPED_OUTPUT}}. It shows consistently 
> better results comparing to original shuffling mode.
> Possible solutions:
> 1) Make this property enabled by default;
> 2) Or totally remove previous mode (do not think it is a good idea)..



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (IGNITE-1680) CPP: Implement basic API for user entry point lookup

[Igor Sapego (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15715245#comment-15715245
 ] 

Igor Sapego edited comment on IGNITE-1680 at 12/27/16 12:31 PM:


Ok, here is concept that I have implemented and checked:
{code}
// If user has any jobs, processors or other classes, that
// may be invoked by the remote side, they should be registered
// in the following function. Every module should have no more
// than one function with the following signature. This function
// called by Ignite when module is loaded.
IGNITE_EXPORTED_CALL void IgniteModuleInit(ignite::InvokeManager& im)
{
im.RegisterCacheEntryProcessor();
im.RegisterCacheEntryProcessor();

im.RegisterJob();
im.RegisterJob();

...
}

// User classes should implement corresponding Ignite classes and 
// have static methods, that return class id. Something like:
class UserProcessor1 : public CacheEntryProcessor<...>
{
// User implements CacheEntryProcessor interface here...

// Function, that return some unique ID for the class.
static int64_t GetIgniteId()
{
...
}
}
{code}

What do you guys think?


was (Author: isapego):
Ok, here is concept that I have implemented and checked:
{code}
// If user has any jobs, processors or other classes, that
// may be invoked by the remote side, they should be registered
// in the following function. Every module should have no more
// than one function with the following signature. This function
// called by Ignite when module is loaded.
IGNITE_EXPORTED_CALL void IgniteModuleInit(ignite::InvokeManager& im)
{
im.RegisterCacheEntryProcessor();
im.RegisterCacheEntryProcessor();

im.RegisterJob();
im.RegisterJob();

...
}

// User classes should implement corresponding Ignite classes and 
// have static methods, that return class id. Something like:
class UserProcessor1 : public CacheEntryProcessor<...>
{
// User implements CacheEntryProcessor interface here...

// Function, that return some unique ID for the class.
static int64_t GetIgniteId()
{
...
}
}
{code}

What do you guys think?

> CPP: Implement basic API for user entry point lookup
> 
>
> Key: IGNITE-1680
> URL: https://issues.apache.org/jira/browse/IGNITE-1680
> Project: Ignite
>  Issue Type: Task
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Igor Sapego
>Assignee: Igor Sapego
>  Labels: cpp, roadmap
> Fix For: 2.0
>
>
> Need to implement IgniteCompute class for C++ with one basic method 
> IgniteCompute::Call(...) which will provide basic remote job execution API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (IGNITE-1680) CPP: Implement basic API for user entry point lookup

[Igor Sapego (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15715245#comment-15715245
 ] 

Igor Sapego edited comment on IGNITE-1680 at 12/27/16 12:30 PM:


Ok, here is concept that I have implemented and checked:
{code}
// If user has any jobs, processors or other classes, that
// may be invoked by the remote side, they should be registered
// in the following function. Every module should have no more
// than one function with the following signature. This function
// called by Ignite when module is loaded.
IGNITE_EXPORTED_CALL void IgniteModuleInit(ignite::InvokeManager& im)
{
im.RegisterCacheEntryProcessor();
im.RegisterCacheEntryProcessor();

im.RegisterJob();
im.RegisterJob();

...
}

// User classes should implement corresponding Ignite classes and 
// have static methods, that return class id. Something like:
class UserProcessor1 : public CacheEntryProcessor<...>
{
// User implements CacheEntryProcessor interface here...

// Function, that return some unique ID for the class.
static int64_t GetIgniteId()
{
...
}
}
{code}

What do you guys think?


was (Author: isapego):
Ok, here is concept that I have implemented and checked:
{code}
// If user has any jobs, processors or other classes, that
// may be invoked by the remote side, they should be registered
// in the following function. Every module should have no more
// than one function with the following signature. This function
// called by Ignite when module is loaded.
IGNITE_EXPORTED_CALL void IgniteModuleInit(ignite::InvokeManager& im)
{
im.RegisterCacheEntryProcessor();
im.RegisterCacheEntryProcessor();

im.RegisterJob();
im.RegisterJob();

...
}

// User classes should implement corresponding Ignite classes and 
// have static methods, that return class id. Something like:
class UserProcessor1 : public CacheEntryProcessor<...>
{
// User implements CacheEntryProcessor interface here...

// Function, that return some unique ID for the class.
static int64_t GetIgniteId()
{
...
}
}
{code}

What do you guys think?

> CPP: Implement basic API for user entry point lookup
> 
>
> Key: IGNITE-1680
> URL: https://issues.apache.org/jira/browse/IGNITE-1680
> Project: Ignite
>  Issue Type: Task
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Igor Sapego
>Assignee: Igor Sapego
>  Labels: cpp, roadmap
> Fix For: 2.0
>
>
> Need to implement IgniteCompute class for C++ with one basic method 
> IgniteCompute::Call(...) which will provide basic remote job execution API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (IGNITE-1072) Need to automatically support LocalDateTime class in indexing

[Dmitry Karachentsev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Karachentsev reassigned IGNITE-1072:
---

Assignee: Dmitry Karachentsev

> Need to automatically support LocalDateTime class in indexing
> -
>
> Key: IGNITE-1072
> URL: https://issues.apache.org/jira/browse/IGNITE-1072
> Project: Ignite
>  Issue Type: Bug
>  Components: cache, SQL
>Affects Versions: 1.1.4
>Reporter: Valentin Kulichenko
>Assignee: Dmitry Karachentsev
>Priority: Critical
>  Labels: Usability
> Fix For: 2.0
>
>
> Currently this query doesn't work if {{localDate}} is an instance of 
> {{LocalDateTime}}:
> {code}
> SqlFieldsQuery qry = new SqlFieldsQuery(
> "SELECT localDate from MyObject " +
> "WHERE localDate = '2013-09-12T11:00'");
> {code}
> It's not supported because {{LocalDateTime}} is JDK8 class. But probably we 
> can use reflection to solve this.
> Also need to go through all classes in {{java.time}} package and see if any 
> of them need to be supported as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (IGNITE-4045) .NET: Support DML API

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15778622#comment-15778622
 ] 

Pavel Tupitsyn edited comment on IGNITE-4045 at 12/27/16 12:04 PM:
---

2) We don't need this. Identity resolver in .NET is needed only to write 
correct hash code to a stream. Enums are written as [typeId, value]. There is 
no hash code. In Java there is special handing for BinaryEnum, but I suspect 
that this code is never called. Let's see what should be done in a separate 
task: IGNITE-4500
3) There is already a ticket: IGNITE-4397. In current ticket FieldComparer is 
hidden from public API, please ignore it.
4) I agree. This is done intentionally because we are on a very hot path here 
and want to avoid any allocations or overhead associated with {{ReadByte}} and 
{{ReadByteArray}}. I've refactored this to a generic {{IBinaryStream.Apply}} 
method which allows operating on byte pointer directly without allocations.
5) IGNITE-4495


was (Author: ptupitsyn):
2) We don't need this. Identity resolver in .NET is needed only to write 
correct hash code to a stream. Enums are written as [typeId, value]. There is 
no hash code.
3) There is already a ticket: IGNITE-4397. In current ticket FieldComparer is 
hidden from public API, please ignore it.
4) I agree. This is done intentionally because we are on a very hot path here 
and want to avoid any allocations or overhead associated with {{ReadByte}} and 
{{ReadByteArray}}. I've refactored this to a generic {{IBinaryStream.Apply}} 
method which allows operating on byte pointer directly without allocations.
5) IGNITE-4495

> .NET: Support DML API
> -
>
> Key: IGNITE-4045
> URL: https://issues.apache.org/jira/browse/IGNITE-4045
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Reporter: Denis Magda
>Assignee: Pavel Tupitsyn
>  Labels: roadmap
> Fix For: 2.0
>
>
> Ignite's Java component will provide support for DML soon (IGNITE-2294). At 
> she same time DML will be supported at the level of ODBC and JDBC drivers.
> As the next step we should include the similar functionality into Ignite.NET 
> by doing the following:
> - Implement DML API;
> - Enhance {{QueryExample.cs}} by doing INSERTs instead of cache.puts and 
> adding UPDATE and DELETE operation examples.
> - Add documentation to Ignite.NET readme.io covering the feature. Most like 
> most of the content can be take from the general documentation when this 
> ticket IGNITE-4018 is ready 
> (https://apacheignite.readme.io/docs/distributed-dml).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4500) .NET: Support identity resolver for binary enums

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-4500:
--

 Summary: .NET: Support identity resolver for binary enums
 Key: IGNITE-4500
 URL: https://issues.apache.org/jira/browse/IGNITE-4500
 Project: Ignite
  Issue Type: Improvement
  Components: platforms
Reporter: Pavel Tupitsyn
 Fix For: 2.0


Enums are a special case when it comes to binary objects. They are written as 
[int typeId, int value] and do not include hash code. However, if user sets a 
custom identity resolver vi {{BinaryTypeConfiguration.EqualityComparer}}, he 
may expect it to be used when {{GetHashCode}} and {{Equals}} are called in 
binary form.

Investigate whether this is a right approach, add enum support to predefined 
identity resolvers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4428) Hadoop: move HadoopMapReducePlanner and dependent classes to public space.

[Taras Ledkov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780186#comment-15780186
 ] 

Taras Ledkov commented on IGNITE-4428:
--

Code revire: 
[IGNT-CR-58|http://reviews.ignite.apache.org/ignite/review/IGNT-CR-58]

> Hadoop: move HadoopMapReducePlanner and dependent classes to public space.
> --
>
> Key: IGNITE-4428
> URL: https://issues.apache.org/jira/browse/IGNITE-4428
> Project: Ignite
>  Issue Type: Task
>  Components: hadoop
>Affects Versions: 1.8
>Reporter: Vladimir Ozerov
>Assignee: Taras Ledkov
> Fix For: 2.0
>
>
> Currently these classes are located in private package. Need to move the to 
> public.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4428) Hadoop: move HadoopMapReducePlanner and dependent classes to public space.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780182#comment-15780182
 ] 

ASF GitHub Bot commented on IGNITE-4428:


GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/1389

IGNITE-4428 Hadoop: move HadoopMapReducePlanner and dependent classes to 
public space.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4428

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1389.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1389


commit c893da70a9757b16b0799adc8eaa29fa1b03d06e
Author: tledkov-gridgain 
Date:   2016-12-21T11:54:33Z

IGNITE-4399: IGFS: Merged IgfsSecondaryFileSystem and 
IgfsSecondaryFileSystemV2 interfaces. This closes #1346.

commit c5882a85f4e3a1f61723ac54fd92f087684df6da
Author: devozerov 
Date:   2016-12-26T11:15:42Z

Merge branch 'master' into ignite-2.0

commit 9a76cc294509613a71e3c76742800df049c8cfa6
Author: tledkov-gridgain 
Date:   2016-12-27T11:04:12Z

IGNITE-4428: Hadoop: move HadoopMapReducePlanner and dependent classes to 
public space




> Hadoop: move HadoopMapReducePlanner and dependent classes to public space.
> --
>
> Key: IGNITE-4428
> URL: https://issues.apache.org/jira/browse/IGNITE-4428
> Project: Ignite
>  Issue Type: Task
>  Components: hadoop
>Affects Versions: 1.8
>Reporter: Vladimir Ozerov
>Assignee: Taras Ledkov
> Fix For: 2.0
>
>
> Currently these classes are located in private package. Need to move the to 
> public.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4395) Implement communication backpressure per policy - SYSTEM or PUBLIC

[Dmitry Karachentsev (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780178#comment-15780178
 ] 

Dmitry Karachentsev commented on IGNITE-4395:
-

Should be implemented after this improvement 
[IGNITE-3220|https://issues.apache.org/jira/browse/IGNITE-3220]

> Implement communication backpressure per policy - SYSTEM or PUBLIC
> --
>
> Key: IGNITE-4395
> URL: https://issues.apache.org/jira/browse/IGNITE-4395
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, compute
>Affects Versions: 1.7
>Reporter: Dmitry Karachentsev
>Assignee: Dmitry Karachentsev
> Fix For: 2.0
>
>
> 1) Start two data nodes with some cache.
> 2) From one node in async mode post some big number of jobs to another. That 
> jobs do some cache operations.
> 3) Grid hangs almost immediately and all threads are sleeping except public 
> ones, they are waiting for response.
> This happens because all cache and job messages are queued on communication 
> and limited with default number (1024). It looks like jobs are waiting for 
> cache responses that could not be received due to this limit.
> Proper solution here is to have communication backpressure per policy -
> SYSTEM or PUBLIC, but not single point as it is now. It could be achieved
> with having two queues per communication session or (which looks a bit
> easier to implement) to have separate connections.
> [PR#1331|https://github.com/apache/ignite/pull/1331] with test that leads to 
> grid hang.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4499) TcpDiscoverySpi is not reliable in some network split scenarios.

[Alexei Scherbakov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780169#comment-15780169
 ] 

Alexei Scherbakov commented on IGNITE-4499:
---

Sadly I have no reproducer at the moment.

> TcpDiscoverySpi is not reliable in some network split scenarios.
> 
>
> Key: IGNITE-4499
> URL: https://issues.apache.org/jira/browse/IGNITE-4499
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: 1.6
>Reporter: Alexei Scherbakov
> Fix For: 2.0
>
>
> Where is a possible caveat in current discovery implementation using ring of 
> nodes.
> Imagine grid consisting of nodes A B C D
> Let them form the ring:
> A-B-C-D-A
> If network connectivity issues will arise between nodes A-C and B-D
> discovery spi will never know it and will continue to assume the topology is 
> valid. 
> On other side, TcpCommunicationSpi will try to run transaction on this 
> topology and never will succeed.
> We must drop nodes from topology on communication spi errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4499) TcpDiscoverySpi is not reliable in some network split scenarios.

[Alexei Scherbakov (JIRA)]

Alexei Scherbakov created IGNITE-4499:
-

 Summary: TcpDiscoverySpi is not reliable in some network split 
scenarios.
 Key: IGNITE-4499
 URL: https://issues.apache.org/jira/browse/IGNITE-4499
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 1.6
Reporter: Alexei Scherbakov
 Fix For: 2.0


Where is a possible caveat in current discovery implementation using ring of 
nodes.

Imagine grid consisting of nodes A B C D

Let them form the ring:
A-B-C-D-A

If network connectivity issues will arise between nodes A-C and B-D
discovery spi will never know it and will continue to assume the topology is 
valid. 

On other side, TcpCommunicationSpi will try to run transaction on this topology 
and never will succeed.

We must drop nodes from topology on communication spi errors.






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (IGNITE-4395) Implement communication backpressure per policy - SYSTEM or PUBLIC

[Dmitry Karachentsev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Karachentsev reassigned IGNITE-4395:
---

Assignee: Dmitry Karachentsev

> Implement communication backpressure per policy - SYSTEM or PUBLIC
> --
>
> Key: IGNITE-4395
> URL: https://issues.apache.org/jira/browse/IGNITE-4395
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, compute
>Affects Versions: 1.7
>Reporter: Dmitry Karachentsev
>Assignee: Dmitry Karachentsev
> Fix For: 2.0
>
>
> 1) Start two data nodes with some cache.
> 2) From one node in async mode post some big number of jobs to another. That 
> jobs do some cache operations.
> 3) Grid hangs almost immediately and all threads are sleeping except public 
> ones, they are waiting for response.
> This happens because all cache and job messages are queued on communication 
> and limited with default number (1024). It looks like jobs are waiting for 
> cache responses that could not be received due to this limit.
> Proper solution here is to have communication backpressure per policy -
> SYSTEM or PUBLIC, but not single point as it is now. It could be achieved
> with having two queues per communication session or (which looks a bit
> easier to implement) to have separate connections.
> [PR#1331|https://github.com/apache/ignite/pull/1331] with test that leads to 
> grid hang.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4487) NPE on query execution

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780133#comment-15780133
 ] 

ASF GitHub Bot commented on IGNITE-4487:


GitHub user SharplEr opened a pull request:

https://github.com/apache/ignite/pull/1388

IGNITE-4487: Fixed

Fix NPE in inner class 
'GridCacheQueryAdapter.ScanQueryFallbackClosableIterator' in constructor. 
Method ScanQueryFallbackClosableIterator#init() cannot be called with an empty 
field 'nodes'. Looks like that was the problem in IGNITE-4487.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/SharplEr/ignite ignite-4487

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1388.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1388


commit da84c2331abac3b049a295275603b3340c137602
Author: Alexander Menshikov 
Date:   2016-12-27T10:30:05Z

IGNITE-4487: Fixed




> NPE on query execution
> --
>
> Key: IGNITE-4487
> URL: https://issues.apache.org/jira/browse/IGNITE-4487
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 1.8
>Reporter: Dmitry Karachentsev
> Fix For: 2.0
>
> Attachments: IgniteThread.java, Main.java
>
>
> NPE may be thrown when called destroyCache() and started querying.
> Attached example reproduces this case when 
> GridDiscoveryManager#removeCacheFilter called but cache state haven't been 
> changed to STOPPED 
> org.apache.ignite.internal.processors.cache.GridCacheGateway#onStopped.
> {code:none}
> javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: 
> null
> at 
> org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:740)
> at 
> com.intellica.evam.engine.event.future.FutureEventWorker.processFutureEvents(FutureEventWorker.java:117)
> at 
> com.intellica.evam.engine.event.future.FutureEventWorker.run(FutureEventWorker.java:66)
> Caused by: class org.apache.ignite.IgniteCheckedException: null
> at 
> org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:1693)
> at 
> org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:494)
> at 
> org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:732)
> ... 2 more
> Caused by: java.lang.NullPointerException
> at 
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter$ScanQueryFallbackClosableIterator.init(GridCacheQueryAdapter.java:712)
> at 
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter$ScanQueryFallbackClosableIterator.(GridCacheQueryAdapter.java:677)
> at 
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter$ScanQueryFallbackClosableIterator.(GridCacheQueryAdapter.java:628)
> at 
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter.executeScanQuery(GridCacheQueryAdapter.java:548)
> at 
> org.apache.ignite.internal.processors.cache.IgniteCacheProxy$2.applyx(IgniteCacheProxy.java:497)
> at 
> org.apache.ignite.internal.processors.cache.IgniteCacheProxy$2.applyx(IgniteCacheProxy.java:495)
> at 
> org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
> at 
> org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:1670)
> ... 4 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (IGNITE-1943) ignitevisorcmd: wrong behavior for "mclear" command

[Andrey Novikov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780121#comment-15780121
 ] 

Andrey Novikov edited comment on IGNITE-1943 at 12/27/16 10:28 AM:
---

Hi [~samaitra],
# I think this command should be removed as part of this PR
# Command examples:
** Cache name variable: cache -reset -c=@c0
** Alert id variable: alert -u -id=@a0
** Host ip variable: deploy -h=@h0 -u= -p= -s=/local/path 
-d=/remote/path
# Unused variables clear automatically on node leave, this create gap in 
variable names. We should take all variables by namespace -> order it -> remove 
from mem and add using visor.setVar


was (Author: anovikov):
Hi [~samaitra],
* I think this command should be removed as part of this PR
* Command examples:
** Cache name variable: cache -reset -c=@c0
** Alert id variable: alert -u -id=@a0
** Host ip variable: deploy -h=@h0 -u= -p= -s=/local/path 
-d=/remote/path
* Unused variables clear automatically on node leave, this create gap in 
variable names. We should take all variables by namespace -> order it -> remove 
from mem and add using visor.setVar

> ignitevisorcmd: wrong behavior for "mclear" command
> ---
>
> Key: IGNITE-1943
> URL: https://issues.apache.org/jira/browse/IGNITE-1943
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: 1.5.0.final
>Reporter: Vasilisa  Sidorova
>Assignee: Saikat Maitra
> Fix For: 2.0
>
>
> -
> DESCRIPTION
> -
> "mclear" should clears all Visor console variables before they will be 
> updated during next run of the corresponding command. It OK for all type of 
> shortcut (@cX, @tX, @aX, @eX) exept shortcut for node-id variables. After 
> "mclear" @nX are disappear from any statistics 
> -
> STEPS FOR REPRODUCE
> -
> # Start cluster
> # Start visorcmd ([IGNITE_HOME]/bin/ignitevisorcmd.sh)
> # Connect to the started cluster (visor> open)
> # Execute any tasks in it (for example, run any example from EventsExample, 
> DeploymentExample, ComputeContinuousMapperExample, ComputeTaskMapExample, 
> ComputeTaskSplitExample) 
> # Create several alerts (visor> alert)
> # Run visor "mclear" command
> # Run visor "node" command
> -
> ACTUAL RESULT
> -
> There isn't @nX values in the first table's column:
> {noformat}
> visor> node
> Select node from:
> +==+
> | # |   Node ID8(@), IP   | Up Time  | CPUs | CPU Load | Free Heap |
> +==+
> | 0 | 6B3E4033, 127.0.0.1 | 00:38:41 | 8| 0.20 %   | 97.00 %   |
> | 1 | 0A0D6989, 127.0.0.1 | 00:38:22 | 8| 0.17 %   | 97.00 %   |
> | 2 | 0941E33F, 127.0.0.1 | 00:35:46 | 8| 0.23 %   | 97.00 %   |
> +--+
> {noformat}
> -
> EXPECTED RESULT
> -
> There is @nX values in the first table's column:
> {noformat}
> visor> node
> Select node from:
> +===+
> | # | Node ID8(@), IP  | Up Time  | CPUs | CPU Load | Free Heap |
> +===+
> | 0 | 6B3E4033(@n0), 127.0.0.1 | 00:38:13 | 8| 0.23 %   | 97.00 %   |
> | 1 | 0A0D6989(@n1), 127.0.0.1 | 00:37:54 | 8| 0.27 %   | 97.00 %   |
> | 2 | 0941E33F(@n2), 127.0.0.1 | 00:35:18 | 8| 0.20 %   | 97.00 %   |
> +---+
> {noformat}
> "mclear" should get effect only for "mget @nX" command (with result "Missing 
> variable with name: '@nX'") and @nX variables  should get new values after 
> "node" (or "tasks") command execution. Like in case for all others @ 
> variables (@cX, @tX, @aX, @eX) 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (IGNITE-1943) ignitevisorcmd: wrong behavior for "mclear" command

[Andrey Novikov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780121#comment-15780121
 ] 

Andrey Novikov edited comment on IGNITE-1943 at 12/27/16 10:28 AM:
---

Hi [~samaitra],
* I think this command should be removed as part of this PR
* Command examples:
** Cache name variable: cache -reset -c=@c0
** Alert id variable: alert -u -id=@a0
** Host ip variable: deploy -h=@h0 -u= -p= -s=/local/path 
-d=/remote/path
* Unused variables clear automatically on node leave, this create gap in 
variable names. We should take all variables by namespace -> order it -> remove 
from mem and add using visor.setVar


was (Author: anovikov):
* I think this command should be removed as part of this PR
* Command examples:
** Cache name variable: cache -reset -c=@c0
** Alert id variable: alert -u -id=@a0
** Host ip variable: deploy -h=@h0 -u= -p= -s=/local/path 
-d=/remote/path
* Unused variables clear automatically on node leave, this create gap in 
variable names. We should take all variables by namespace -> order it -> remove 
from mem and add using visor.setVar

> ignitevisorcmd: wrong behavior for "mclear" command
> ---
>
> Key: IGNITE-1943
> URL: https://issues.apache.org/jira/browse/IGNITE-1943
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: 1.5.0.final
>Reporter: Vasilisa  Sidorova
>Assignee: Saikat Maitra
> Fix For: 2.0
>
>
> -
> DESCRIPTION
> -
> "mclear" should clears all Visor console variables before they will be 
> updated during next run of the corresponding command. It OK for all type of 
> shortcut (@cX, @tX, @aX, @eX) exept shortcut for node-id variables. After 
> "mclear" @nX are disappear from any statistics 
> -
> STEPS FOR REPRODUCE
> -
> # Start cluster
> # Start visorcmd ([IGNITE_HOME]/bin/ignitevisorcmd.sh)
> # Connect to the started cluster (visor> open)
> # Execute any tasks in it (for example, run any example from EventsExample, 
> DeploymentExample, ComputeContinuousMapperExample, ComputeTaskMapExample, 
> ComputeTaskSplitExample) 
> # Create several alerts (visor> alert)
> # Run visor "mclear" command
> # Run visor "node" command
> -
> ACTUAL RESULT
> -
> There isn't @nX values in the first table's column:
> {noformat}
> visor> node
> Select node from:
> +==+
> | # |   Node ID8(@), IP   | Up Time  | CPUs | CPU Load | Free Heap |
> +==+
> | 0 | 6B3E4033, 127.0.0.1 | 00:38:41 | 8| 0.20 %   | 97.00 %   |
> | 1 | 0A0D6989, 127.0.0.1 | 00:38:22 | 8| 0.17 %   | 97.00 %   |
> | 2 | 0941E33F, 127.0.0.1 | 00:35:46 | 8| 0.23 %   | 97.00 %   |
> +--+
> {noformat}
> -
> EXPECTED RESULT
> -
> There is @nX values in the first table's column:
> {noformat}
> visor> node
> Select node from:
> +===+
> | # | Node ID8(@), IP  | Up Time  | CPUs | CPU Load | Free Heap |
> +===+
> | 0 | 6B3E4033(@n0), 127.0.0.1 | 00:38:13 | 8| 0.23 %   | 97.00 %   |
> | 1 | 0A0D6989(@n1), 127.0.0.1 | 00:37:54 | 8| 0.27 %   | 97.00 %   |
> | 2 | 0941E33F(@n2), 127.0.0.1 | 00:35:18 | 8| 0.20 %   | 97.00 %   |
> +---+
> {noformat}
> "mclear" should get effect only for "mget @nX" command (with result "Missing 
> variable with name: '@nX'") and @nX variables  should get new values after 
> "node" (or "tasks") command execution. Like in case for all others @ 
> variables (@cX, @tX, @aX, @eX) 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-1943) ignitevisorcmd: wrong behavior for "mclear" command

[Andrey Novikov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780121#comment-15780121
 ] 

Andrey Novikov commented on IGNITE-1943:


* I think this command should be removed as part of this PR
* Command examples:
** Cache name variable: cache -reset -c=@c0
** Alert id variable: alert -u -id=@a0
** Host ip variable: deploy -h=@h0 -u= -p= -s=/local/path 
-d=/remote/path
* Unused variables clear automatically on node leave, this create gap in 
variable names. We should take all variables by namespace -> order it -> remove 
from mem and add using visor.setVar

> ignitevisorcmd: wrong behavior for "mclear" command
> ---
>
> Key: IGNITE-1943
> URL: https://issues.apache.org/jira/browse/IGNITE-1943
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: 1.5.0.final
>Reporter: Vasilisa  Sidorova
>Assignee: Saikat Maitra
> Fix For: 2.0
>
>
> -
> DESCRIPTION
> -
> "mclear" should clears all Visor console variables before they will be 
> updated during next run of the corresponding command. It OK for all type of 
> shortcut (@cX, @tX, @aX, @eX) exept shortcut for node-id variables. After 
> "mclear" @nX are disappear from any statistics 
> -
> STEPS FOR REPRODUCE
> -
> # Start cluster
> # Start visorcmd ([IGNITE_HOME]/bin/ignitevisorcmd.sh)
> # Connect to the started cluster (visor> open)
> # Execute any tasks in it (for example, run any example from EventsExample, 
> DeploymentExample, ComputeContinuousMapperExample, ComputeTaskMapExample, 
> ComputeTaskSplitExample) 
> # Create several alerts (visor> alert)
> # Run visor "mclear" command
> # Run visor "node" command
> -
> ACTUAL RESULT
> -
> There isn't @nX values in the first table's column:
> {noformat}
> visor> node
> Select node from:
> +==+
> | # |   Node ID8(@), IP   | Up Time  | CPUs | CPU Load | Free Heap |
> +==+
> | 0 | 6B3E4033, 127.0.0.1 | 00:38:41 | 8| 0.20 %   | 97.00 %   |
> | 1 | 0A0D6989, 127.0.0.1 | 00:38:22 | 8| 0.17 %   | 97.00 %   |
> | 2 | 0941E33F, 127.0.0.1 | 00:35:46 | 8| 0.23 %   | 97.00 %   |
> +--+
> {noformat}
> -
> EXPECTED RESULT
> -
> There is @nX values in the first table's column:
> {noformat}
> visor> node
> Select node from:
> +===+
> | # | Node ID8(@), IP  | Up Time  | CPUs | CPU Load | Free Heap |
> +===+
> | 0 | 6B3E4033(@n0), 127.0.0.1 | 00:38:13 | 8| 0.23 %   | 97.00 %   |
> | 1 | 0A0D6989(@n1), 127.0.0.1 | 00:37:54 | 8| 0.27 %   | 97.00 %   |
> | 2 | 0941E33F(@n2), 127.0.0.1 | 00:35:18 | 8| 0.20 %   | 97.00 %   |
> +---+
> {noformat}
> "mclear" should get effect only for "mget @nX" command (with result "Missing 
> variable with name: '@nX'") and @nX variables  should get new values after 
> "node" (or "tasks") command execution. Like in case for all others @ 
> variables (@cX, @tX, @aX, @eX) 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (IGNITE-4395) Implement communication backpressure per policy - SYSTEM or PUBLIC

[Dmitry Karachentsev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Karachentsev updated IGNITE-4395:

Fix Version/s: 2.0

> Implement communication backpressure per policy - SYSTEM or PUBLIC
> --
>
> Key: IGNITE-4395
> URL: https://issues.apache.org/jira/browse/IGNITE-4395
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, compute
>Affects Versions: 1.7
>Reporter: Dmitry Karachentsev
> Fix For: 2.0
>
>
> 1) Start two data nodes with some cache.
> 2) From one node in async mode post some big number of jobs to another. That 
> jobs do some cache operations.
> 3) Grid hangs almost immediately and all threads are sleeping except public 
> ones, they are waiting for response.
> This happens because all cache and job messages are queued on communication 
> and limited with default number (1024). It looks like jobs are waiting for 
> cache responses that could not be received due to this limit.
> Proper solution here is to have communication backpressure per policy -
> SYSTEM or PUBLIC, but not single point as it is now. It could be achieved
> with having two queues per communication session or (which looks a bit
> easier to implement) to have separate connections.
> [PR#1331|https://github.com/apache/ignite/pull/1331] with test that leads to 
> grid hang.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-2793) ODBC: Add support for Arrays.

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780087#comment-15780087
 ] 

Pavel Tupitsyn commented on IGNITE-2793:


Looks good to me. Merged to master.

> ODBC: Add support for Arrays.
> -
>
> Key: IGNITE-2793
> URL: https://issues.apache.org/jira/browse/IGNITE-2793
> Project: Ignite
>  Issue Type: Task
>  Components: odbc
>Affects Versions: 1.5.0.final
>Reporter: Igor Sapego
>Assignee: Pavel Tupitsyn
>  Labels: roadmap
> Fix For: 2.0
>
>
> Support for arrays should be added. We need to at least support byte arrays 
> to match {{SQL_C_BINARY}} type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-2793) ODBC: Add support for Arrays.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780088#comment-15780088
 ] 

ASF GitHub Bot commented on IGNITE-2793:


Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1371


> ODBC: Add support for Arrays.
> -
>
> Key: IGNITE-2793
> URL: https://issues.apache.org/jira/browse/IGNITE-2793
> Project: Ignite
>  Issue Type: Task
>  Components: odbc
>Affects Versions: 1.5.0.final
>Reporter: Igor Sapego
>Assignee: Pavel Tupitsyn
>  Labels: roadmap
> Fix For: 2.0
>
>
> Support for arrays should be added. We need to at least support byte arrays 
> to match {{SQL_C_BINARY}} type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (IGNITE-2793) ODBC: Add support for Arrays.

[Igor Sapego (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Igor Sapego updated IGNITE-2793:

Assignee: Pavel Tupitsyn

> ODBC: Add support for Arrays.
> -
>
> Key: IGNITE-2793
> URL: https://issues.apache.org/jira/browse/IGNITE-2793
> Project: Ignite
>  Issue Type: Task
>  Components: odbc
>Affects Versions: 1.5.0.final
>Reporter: Igor Sapego
>Assignee: Pavel Tupitsyn
>  Labels: roadmap
> Fix For: 2.0
>
>
> Support for arrays should be added. We need to at least support byte arrays 
> to match {{SQL_C_BINARY}} type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (IGNITE-2793) ODBC: Add support for Arrays.

[Igor Sapego (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Igor Sapego updated IGNITE-2793:

Assignee: (was: Igor Sapego)

> ODBC: Add support for Arrays.
> -
>
> Key: IGNITE-2793
> URL: https://issues.apache.org/jira/browse/IGNITE-2793
> Project: Ignite
>  Issue Type: Task
>  Components: odbc
>Affects Versions: 1.5.0.final
>Reporter: Igor Sapego
>  Labels: roadmap
> Fix For: 2.0
>
>
> Support for arrays should be added. We need to at least support byte arrays 
> to match {{SQL_C_BINARY}} type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (IGNITE-4491) Commutation loss between two nodes leads to hang whole cluster

[Vladislav Pyatkov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-4491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladislav Pyatkov updated IGNITE-4491:
--
Description: 
Reproduction steps:
1) Start nodes:

{noformat}
DC1   DC2

1 (10.116.172.1)  8 (10.116.64.11)
2 (10.116.172.2)  7 (10.116.64.12)
3 (10.116.172.3)  6 (10.116.64.13)
4 (10.116.172.4)  5 (10.116.64.14)
{noformat}

each node have client which run in same host with server (look source in 
attachment).

2) Drop connection

Between 1-8,
{noformat}
1 (10.116.172.1)  8 (10.116.64.11)
{noformat}

Drop all input and output traffic
Invoke from 10.116.172.1
{code}
iptables -A INPUT -s 10.116.64.11 -j DROP
iptables -A OUTPUT -d 10.116.64.11 -j DROP
{code}

Between  4-5

{noformat}
4 (10.116.172.4)  5 (10.116.64.14)
{noformat}

Invoke from 10.116.172.4
{code}
iptables -A INPUT -s 10.116.64.14 -j DROP
iptables -A OUTPUT -d 10.116.64.14 -j DROP
{code}

3) Stop the grid, after several seconds

If you are looking into logs, you can find which nodes 1(10.116.172.1) and 
5(10.116.64.14) were segmented and stopped according to policy (pay attention, 
which clients did not segmented), after drop traffic:
{noformat}
[12:04:33,914][INFO][disco-event-worker-#211%null%][GridDiscoveryManager] 
Topology snapshot [ver=18, servers=6, clients=8, CPUs=456, heap=68.0GB]
{noformat}

And all operations stopped at the same time.

  was:
Reproduction steps:
1) Start nodes:

{noformat}
DC1   DC2

1 (10.116.172.1)  8 (10.116.64.11)
2 (10.116.172.2)  7 (10.116.64.12)
3 (10.116.172.3)  6 (10.116.64.13)
4 (10.116.172.4)  5 (10.116.64.14)
{noformat}

each node have client which run in same host with server (look source in 
attachment).

2) Drop connection

Between 1-8,
{noformat}
1 (10.116.172.1)  8 (10.116.64.11)
{noformat}

Drop all input and output traffic
Invoke from 10.116.172.1
{code}
iptables -A INPUT -s 10.116.64.11 -j DROP
iptables -A OUTPUT -d 10.116.64.11 -j DROP
{code}

Between  4-5

{noformat}
4 (10.116.172.4)  5 (10.116.64.14)
{noformat}

Invoke from 10.116.172.4
{code}
iptables -A INPUT -s 10.116.64.14 -j DROP
iptables -A OUTPUT -d 10.116.64.14 -j DROP
{code}

3) Stop the grid, after several seconds

If you are looking into logs, you can find which nodes 1(10.116.172.1) and 
5(10.116.64.14) were segmented (pay attention, which clients did not 
segmented), after drop traffic:
{noformat}
[12:04:33,914][INFO][disco-event-worker-#211%null%][GridDiscoveryManager] 
Topology snapshot [ver=18, servers=6, clients=8, CPUs=456, heap=68.0GB]
{noformat}

And all operations stopped at the same time.


> Commutation loss between two nodes leads to hang whole cluster
> --
>
> Key: IGNITE-4491
> URL: https://issues.apache.org/jira/browse/IGNITE-4491
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 1.8
>Reporter: Vladislav Pyatkov
>Priority: Critical
> Attachments: Segmentation.7z
>
>
> Reproduction steps:
> 1) Start nodes:
> {noformat}
> DC1   DC2
> 1 (10.116.172.1)  8 (10.116.64.11)
> 2 (10.116.172.2)  7 (10.116.64.12)
> 3 (10.116.172.3)  6 (10.116.64.13)
> 4 (10.116.172.4)  5 (10.116.64.14)
> {noformat}
> each node have client which run in same host with server (look source in 
> attachment).
> 2) Drop connection
> Between 1-8,
> {noformat}
> 1 (10.116.172.1)  8 (10.116.64.11)
> {noformat}
> Drop all input and output traffic
> Invoke from 10.116.172.1
> {code}
> iptables -A INPUT -s 10.116.64.11 -j DROP
> iptables -A OUTPUT -d 10.116.64.11 -j DROP
> {code}
> Between  4-5
> {noformat}
> 4 (10.116.172.4)  5 (10.116.64.14)
> {noformat}
> Invoke from 10.116.172.4
> {code}
> iptables -A INPUT -s 10.116.64.14 -j DROP
> iptables -A OUTPUT -d 10.116.64.14 -j DROP
> {code}
> 3) Stop the grid, after several seconds
> If you are looking into logs, you can find which nodes 1(10.116.172.1) and 
> 5(10.116.64.14) were segmented and stopped according to policy (pay 
> attention, which clients did not segmented), after drop traffic:
> {noformat}
> [12:04:33,914][INFO][disco-event-worker-#211%null%][GridDiscoveryManager] 
> Topology snapshot [ver=18, servers=6, clients=8, CPUs=456, heap=68.0GB]
> {noformat}
> And all operations stopped at the same time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (IGNITE-4491) Commutation loss between two nodes leads to hang whole cluster

[Vladislav Pyatkov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-4491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladislav Pyatkov updated IGNITE-4491:
--
Description: 
Reproduction steps:
1) Start nodes:

{noformat}
DC1   DC2

1 (10.116.172.1)  8 (10.116.64.11)
2 (10.116.172.2)  7 (10.116.64.12)
3 (10.116.172.3)  6 (10.116.64.13)
4 (10.116.172.4)  5 (10.116.64.14)
{noformat}

each node have client which run in same host with server (look source in 
attachment).

2) Drop connection

Between 1-8,
{noformat}
1 (10.116.172.1)  8 (10.116.64.11)
{noformat}

Drop all input and output traffic
Invoke from 10.116.172.1
{code}
iptables -A INPUT -s 10.116.64.11 -j DROP
iptables -A OUTPUT -d 10.116.64.11 -j DROP
{code}

Between  4-5

{noformat}
4 (10.116.172.4)  5 (10.116.64.14)
{noformat}

Invoke from 10.116.172.4
{code}
iptables -A INPUT -s 10.116.64.14 -j DROP
iptables -A OUTPUT -d 10.116.64.14 -j DROP
{code}

3) Stop the grid, after several seconds

If you are looking into logs, you can find which nodes 1(10.116.172.1) and 
5(10.116.64.14) were segmented (pay attention, which clients did not 
segmented), after drop traffic:
{noformat}
[12:04:33,914][INFO][disco-event-worker-#211%null%][GridDiscoveryManager] 
Topology snapshot [ver=18, servers=6, clients=8, CPUs=456, heap=68.0GB]
{noformat}

And all operations stopped at the same time.

  was:
Reproduction steps:
1) Start nodes:

{noformat}
DC1   DC2

1 (10.116.172.1)  8 (10.116.64.11)
2 (10.116.172.2)  7 (10.116.64.12)
3 (10.116.172.3)  6 (10.116.64.13)
4 (10.116.172.4)  5 (10.116.64.14)
{noformat}

each node have client which run in same host with server (look source in 
attachment).

2) Drop connection

Between 1-8,
{noformat}
1 (10.116.172.1)  8 (10.116.64.11)
{noformat}

Drop all input and output traffic
Invoke from 10.116.172.1
{code}
iptables -A INPUT -s 10.116.64.11 -j DROP
iptables -A OUTPUT -d 10.116.64.11 -j DROP
{code}

Between  4-5

{noformat}
4 (10.116.172.4)  5 (10.116.64.14)
{noformat}

Invoke from 10.116.172.4
{code}
iptables -A INPUT -s 10.116.64.14 -j DROP
iptables -A OUTPUT -d 10.116.64.14 -j DROP
{code}

3) Stop the grid, after several seconds

If you are looking into logs, you can find which node was segmented (pay 
attention, which clients did not segmented), after drop traffic:
{noformat}
[12:04:33,914][INFO][disco-event-worker-#211%null%][GridDiscoveryManager] 
Topology snapshot [ver=18, servers=6, clients=8, CPUs=456, heap=68.0GB]
{noformat}

And all operations stopped at the same time.


> Commutation loss between two nodes leads to hang whole cluster
> --
>
> Key: IGNITE-4491
> URL: https://issues.apache.org/jira/browse/IGNITE-4491
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 1.8
>Reporter: Vladislav Pyatkov
>Priority: Critical
> Attachments: Segmentation.7z
>
>
> Reproduction steps:
> 1) Start nodes:
> {noformat}
> DC1   DC2
> 1 (10.116.172.1)  8 (10.116.64.11)
> 2 (10.116.172.2)  7 (10.116.64.12)
> 3 (10.116.172.3)  6 (10.116.64.13)
> 4 (10.116.172.4)  5 (10.116.64.14)
> {noformat}
> each node have client which run in same host with server (look source in 
> attachment).
> 2) Drop connection
> Between 1-8,
> {noformat}
> 1 (10.116.172.1)  8 (10.116.64.11)
> {noformat}
> Drop all input and output traffic
> Invoke from 10.116.172.1
> {code}
> iptables -A INPUT -s 10.116.64.11 -j DROP
> iptables -A OUTPUT -d 10.116.64.11 -j DROP
> {code}
> Between  4-5
> {noformat}
> 4 (10.116.172.4)  5 (10.116.64.14)
> {noformat}
> Invoke from 10.116.172.4
> {code}
> iptables -A INPUT -s 10.116.64.14 -j DROP
> iptables -A OUTPUT -d 10.116.64.14 -j DROP
> {code}
> 3) Stop the grid, after several seconds
> If you are looking into logs, you can find which nodes 1(10.116.172.1) and 
> 5(10.116.64.14) were segmented (pay attention, which clients did not 
> segmented), after drop traffic:
> {noformat}
> [12:04:33,914][INFO][disco-event-worker-#211%null%][GridDiscoveryManager] 
> Topology snapshot [ver=18, servers=6, clients=8, CPUs=456, heap=68.0GB]
> {noformat}
> And all operations stopped at the same time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4442) Web console: Implement configuration of affinity function in cache configuration.

[Vasiliy Sisko (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15779962#comment-15779962
 ] 

Vasiliy Sisko commented on IGNITE-4442:
---

Implemented section for configuration of affinity function and affinity mapper.

> Web console: Implement configuration of affinity function in cache 
> configuration.
> -
>
> Key: IGNITE-4442
> URL: https://issues.apache.org/jira/browse/IGNITE-4442
> Project: Ignite
>  Issue Type: Bug
>  Components: wizards
>Affects Versions: 1.9
>Reporter: Vasiliy Sisko
>Assignee: Vasiliy Sisko
> Fix For: 1.9
>
>
> Also implement custom configuration of cache store.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4498) .NET: Remove legacy BinaryObject.Equals

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-4498:
--

 Summary: .NET: Remove legacy BinaryObject.Equals
 Key: IGNITE-4498
 URL: https://issues.apache.org/jira/browse/IGNITE-4498
 Project: Ignite
  Issue Type: Task
  Components: platforms
Reporter: Pavel Tupitsyn
 Fix For: 2.0


Remove legacy field-by-field comparison from {{BinaryObject.Equals}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-3961) IGFS: Support direct PROXY mode invocation in method: affinity

[Taras Ledkov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15779926#comment-15779926
 ] 

Taras Ledkov commented on IGNITE-3961:
--

Review issues are fixed
[Tests 
result|http://195.239.208.174/project.html?projectId=IgniteTests=projectOverview_IgniteTests=pull%2F1252%2Fhead]


> IGFS: Support direct PROXY mode invocation in method: affinity
> --
>
> Key: IGNITE-3961
> URL: https://issues.apache.org/jira/browse/IGNITE-3961
> Project: Ignite
>  Issue Type: Sub-task
>  Components: IGFS
>Affects Versions: 1.7
>Reporter: Vladimir Ozerov
>Assignee: Taras Ledkov
> Fix For: 2.0
>
>
> See 
> {{org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem.getFileBlockLocations}}
>  for reference.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4497) CacheDefaultBinaryAffinityKeyMapper class has a wrong duplicate

[Vasiliy Sisko (JIRA)]

Vasiliy Sisko created IGNITE-4497:
-

 Summary: CacheDefaultBinaryAffinityKeyMapper class has a wrong 
duplicate
 Key: IGNITE-4497
 URL: https://issues.apache.org/jira/browse/IGNITE-4497
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 1.9
Reporter: Vasiliy Sisko
 Fix For: 1.9


Two version of CacheDefaultBinaryAffinityKeyMapper exist.
org.apache.ignite.internal.processors.cache.binary.CacheDefaultBinaryAffinityKeyMapper
 and 
org.apache.ignite.internal.processors.cache.CacheDefaultBinaryAffinityKeyMapper

Version from binary project is not used in project.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)