Re: [VOTE] Apache Ignite 2.5.0 RC1

2018-05-24 Thread Denis Magda
+1 (binding)

On Thu, May 24, 2018 at 10:59 AM, Andrey Gura  wrote:

> Igniters,
>
> We've uploaded a 2.5.0 release candidate to
> https://dist.apache.org/repos/dist/dev/ignite/2.5.0-rc1/
>
> Git tag name is
> 2.5.0-rc1
>
> This release includes the following changes:
>
> Ignite:
> * Implemented Zookeeper discovery SPI.
> * Added Java thin client.
> * Added continuous queries with transformers.
> * Implemented general approach for critical failures handling.
> * Added the new method IgniteMXBean.getCurrentCoordinatorFormatted()
> which displays formatted information about current coordiantor.
> * Added partition update counters verification on partition map exchange.
> * Added ability to disable WAL during initial data rebalancing.
> * Added additional data storage metrics
> * Added additional transactional metrics.
> * Added DEB package and improved RPM package.
> * Added information about fields serialization/deserialization errors
> in BinaryMarshaller.
> * Added configurable automatic rollback timeout for pending
> transactions preventing partition map exchange to complete.
> * Added preventing infinite iterations of corrupted B+Tree. New
> IGNITE_BPLUS_TREE_LOCK_RETRIES property was added, default value is
> Integer.MAX_VALUE.
> * Implemented handling of broken segment in WAL compaction.
> * Fixed affinity to always choose primary node from one of the owners
> when persistence is enabled.
> * Partition eviction now does not block exchange.
> * Fixed the issue that prevented rebalancing on a new baseline node join.
> * Fixed potential deadlock during preloading.
> * Fixed exception handling in IsolatedUpdater of DataStreamer.
> * Fixed the issue when ComputeTaskFuture.get() threw a trimmed
> exception chain: now full exception chain is propagated to the caller;
> the exception thrown from IgniteCompute methods may now differ from
> the previous behavior.
> * Fixed null value being passed to entry processor when such entry
> actually exists.
> * Fixed problem when cluster could not survive after
> IgniteOutOfMemoryException.
> * Fixed NullPointerException on batch atomic cache operations with cache
> store.
> * Fixed bug with ScanQuery transformer which applies to the first
> result page only.
> * Fixed hanging of node stopping in case when node is invalidated and
> persistence is enabled.
> * BinaryMetadata exchange is triggered only if there are actual
> changes in BinaryMetadata.
> * Fixed contention while executing parallel atomic single get operations.
> * Fixed cache metadata corruption during concurrent checkpoint and
> cache destroy.
> * Implemented additional synchronization for correct partition
> counters update on partition map exchange.
> * Optimized checkpoint lock acquisition for demander thread.
> * Destroyed caches with enabled persistence now will not survive grid
> restart.
> * Nodes with incompatible SQL metadata will not be able to join grid.
> This fixes known case for index corruption.
> * Fixed fillFactor memory metric value.
> * Fixed bug with checkpoint buffer overflow with enabled throttling.
> * Fixed crash recovery problem when index tree could reference to
> removed data pages.
> * Fixed error during cluster initialization with partially acceptable
> WAL archive.
> * Fixed issue with local partitions size changing inside checkpoint write
> lock.
> * Fixed LOG_ONLY and BACKGROUND modes to be power-crash resistant.
> * Fixed incorrect finishing of implicit transactions on unstable topology.
> * Fixed incorrect mapping of smallint Cassandra type.
> * Introduced log markers support to IgniteLogger and standard DEV_ONLY
> marker.
> * Changed Log4J and Log4J2 adapters to update active logger
> configuration when configuration source file is modified.
> * Fixed sqlline.sh script to use Java from JAVA_HOME.
> * Improved Docker image build.
> * Default and recommended JVM options are updated.
>
> Ignite .Net:
> * Added SSL configuration for node-to-node links.
> * Added dynamic service proxies.
> * Propagated query metrics to ICache.GetQueryMetrics() and
> ICache.ResetQueryMetrics().
> * DML update via LINQ.
> * Fixed hang caused by mishandled exception during custom cache store
> deserialization.
> * Fixed GroupBy and Where issues in LINQ.
> * .Net thin client SSL support.
>
> Ignite C++:
> * Fixed bug with LocalSize cache method.
>
> SQL:
> * Implemented COPY command for fast data load.
> * Added support of authorization for CREATE TABLE and DROP TABLE commands.
> * Improved deadlock handling in DML.
> * Added IGNITE_SQL_FORCE_LAZY_RESULT_SET option to force all SQL
> queries on node using lazy result set.
> * Fixed SQL query hang if cluster contains nodes not in baseline.
>
> JDBC:
> * Added SSL support.
> * Added authentication.
> * Added streaming mode to thin driver.
> * Semicolon now can be used as delimiter for properties in URL string.
> * Changed default socket buffer size to 64Kb in thin driver.
> * Added FQDN to thin client in order to support multiple 

[GitHub] ignite pull request #4067: graceful shutdown test

2018-05-24 Thread BiryukovVA
GitHub user BiryukovVA opened a pull request:

https://github.com/apache/ignite/pull/4067

graceful shutdown test



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/BiryukovVA/ignite graceful_shutdown_test

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4067.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4067


commit c236caf256ab49c0b7ae3de06702c6f41f3b7ef6
Author: Vitaliy Biryukov 
Date:   2018-05-24T22:47:23Z

test




---


Re: Node.js Thin Client @ npmjs

2018-05-24 Thread Pavel Petroshenko
As Denis said, there is no need to download the entire Ignite repo to
install the client. Once published the client is going to be installed by
users with a command:

npm install -g apache-ignite-client

The sources are going to be distributed as a part of the ignite repository,
yes. But in general, release and installation process for the client and
the Ignite technically are completely independent.

And moreover: if there is a bug, especially critical, found in the client
we shouldn't wait for the next Ignite to be released to get it fixed. We
should be flexible enough to push the fixes and release the clients'
updates independently at any point in time.

Having an independent release/versioning scheme would allow the clients to
get bug-fixes (minor version update) or nonbreaking feature-adds or
improvements (medium version update) between major Ignite releases
(potentially breaking changes and thus - the major version update). But the
client and the Ignite versions mapping might be tricky and should be
clearly documented.

So there are pros and cons.

But I believe the release policy should be consistent across all the Thin
clients (I'm not talking about the "native" or Thick ones, which heavily
depend on the Ignite internals and are a different story).

p.


On Thu, May 24, 2018 at 1:44 PM, Denis Magda  wrote:

> Once the client is built it will be uploaded to the npmjs repository,
> right? So, a JS developer can download the client from there without
> touching the whole Ignite binary release.
>
> However, those who download the whole Ignite binary distribution will find
> node.js there (as well as .NET, C++, JDBC and ODBC).
>
> --
> Denis
>
> On Thu, May 24, 2018 at 1:06 PM, Dmitriy Setrakyan 
> wrote:
>
> > On Thu, May 24, 2018 at 12:36 PM, Pavel Petroshenko <
> pa...@petroshenko.com
> > >
> > wrote:
> >
> > > Fair enough. Consistency with the other clients is a good argument.
> > >
> > >
> > Pavel, I would discuss it a bit more. Does it really make sense for a
> > node.js user to download the whole Ignite distribution just to get a
> > node.js client?
> >
>


[GitHub] ignite pull request #4066: IGNITE-8612 Debugging NPE in GridCacheTtlManager:...

2018-05-24 Thread andrewmed
GitHub user andrewmed opened a pull request:

https://github.com/apache/ignite/pull/4066

IGNITE-8612 Debugging NPE in GridCacheTtlManager: add assert



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/andrewmed/ignite ignite-8612

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4066.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4066


commit 7554b6fc667954b37cc52feb3ccc23b663defdd2
Author: AMedvedev 
Date:   2018-05-24T20:50:54Z

IGNITE-8612 Debugging NPE: add assert




---


Re: Node.js Thin Client @ npmjs

2018-05-24 Thread Denis Magda
Once the client is built it will be uploaded to the npmjs repository,
right? So, a JS developer can download the client from there without
touching the whole Ignite binary release.

However, those who download the whole Ignite binary distribution will find
node.js there (as well as .NET, C++, JDBC and ODBC).

--
Denis

On Thu, May 24, 2018 at 1:06 PM, Dmitriy Setrakyan 
wrote:

> On Thu, May 24, 2018 at 12:36 PM, Pavel Petroshenko  >
> wrote:
>
> > Fair enough. Consistency with the other clients is a good argument.
> >
> >
> Pavel, I would discuss it a bit more. Does it really make sense for a
> node.js user to download the whole Ignite distribution just to get a
> node.js client?
>


[jira] [Created] (IGNITE-8612) NPE in GridCacheTtlManager#expire on commit() or close()

2018-05-24 Thread Andrew Medvedev (JIRA)
Andrew Medvedev created IGNITE-8612:
---

 Summary: NPE in GridCacheTtlManager#expire on commit() or close()
 Key: IGNITE-8612
 URL: https://issues.apache.org/jira/browse/IGNITE-8612
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Andrew Medvedev


We have got NPE in 
org.apache.ignite.internal.processors.cache.GridCacheTtlManager#expire(int) 
several times during 4 minutes from tx.close() and tx.commit() here  
[https://github.com/apache/ignite/blob/40845c67750c300b5568d157ab0ffeaf320802a8/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheTtlManager.java#L203]
 

{{Caused by: java.lang.NullPointerException}}
{{at 
org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:197)}}
{{at 
org.apache.ignite.internal.processors.cache.GridCacheUtils.unwindEvicts(GridCacheUtils.java:834)}}
{{at 
org.apache.ignite.internal.processors.cache.GridCacheUtils.unwindEvicts(GridCacheUtils.java:844)}}
{{at 
org.apache.ignite.internal.processors.cache.transactions.TransactionProxyImpl.leave(TransactionProxyImpl.java:136)}}
{{at 
org.apache.ignite.internal.processors.cache.transactions.TransactionProxyImpl.close(TransactionProxyImpl.java:326)}}

This could  have been IgniteCacheOffheapManager == null, cctx.offheap() 
returning null, but I  could not reproduce it. To debug this further, a PR with 
assert added will be submitted



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-8611) Binary marshaller documentation should cover how data classes can or can't be changed

2018-05-24 Thread Stanislav Lukyanov (JIRA)
Stanislav Lukyanov created IGNITE-8611:
--

 Summary: Binary marshaller documentation should cover how data 
classes can or can't be changed
 Key: IGNITE-8611
 URL: https://issues.apache.org/jira/browse/IGNITE-8611
 Project: Ignite
  Issue Type: Improvement
  Components: documentation
Reporter: Stanislav Lukyanov


Binary marshaller docs (https://apacheignite.readme.io/docs/binary-marshaller) 
give an idea that a class structure may be changed (fields can be added or 
removed) and the marshaller will handle such change.

However, not all changes are supported.
One corner case is when an enum value is stored in the cache: if the order of 
the enum constants is changed, or if a new constant is added at the start or at 
the middle of the constants list, it will lead to an error. This is because the 
enums are stored as ordinals (integers), and the ordinals of an enum depend on 
the order of values in the code.

The task is to update the documentation with the description of data class 
changes that are incompatible from binary marshallers point of view. At least 
the enum case should be covered. If more cases are discovered, they should be 
documented as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Node.js Thin Client @ npmjs

2018-05-24 Thread Dmitriy Setrakyan
On Thu, May 24, 2018 at 12:36 PM, Pavel Petroshenko 
wrote:

> Fair enough. Consistency with the other clients is a good argument.
>
>
Pavel, I would discuss it a bit more. Does it really make sense for a
node.js user to download the whole Ignite distribution just to get a
node.js client?


[GitHub] ignite pull request #4065: IGNITE-8459 Do first checkpoint after all partiti...

2018-05-24 Thread Jokser
GitHub user Jokser opened a pull request:

https://github.com/apache/ignite/pull/4065

IGNITE-8459 Do first checkpoint after all partitions have been initialized



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8459

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4065.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4065


commit df3fb2a24d4d43771094e56fea6680f23e3790e6
Author: Pavel Kovalenko 
Date:   2018-05-18T09:37:13Z

IGNITE-8459 WIP

commit 54e88d22bfd4c0b61593b4a8f0d5c319288593b7
Author: Pavel Kovalenko 
Date:   2018-05-18T16:22:01Z

IGNITE-8459 WIP

commit e8aeeea9d6c30df2ee03bfada3a8a399cfef7b6b
Author: Pavel Kovalenko 
Date:   2018-05-21T12:42:33Z

IGNITE-8459 WIP

commit 7d58eb9e3c3fa2358c7937ce3f73715d850a33f1
Author: Pavel Kovalenko 
Date:   2018-05-24T19:46:24Z

IGNITE-8459 Rework.

commit bd8ef85034bbf7c03d6c8b40b36fa4398a3b23ca
Author: Pavel Kovalenko 
Date:   2018-05-24T19:50:11Z

IGNITE-8459 Remove trash.




---


[jira] [Created] (IGNITE-8610) Searching checkpoint / WAL history for rebalancing is not properly working in case of local/global WAL disabling

2018-05-24 Thread Pavel Kovalenko (JIRA)
Pavel Kovalenko created IGNITE-8610:
---

 Summary: Searching checkpoint / WAL history for rebalancing is not 
properly working in case of local/global WAL disabling
 Key: IGNITE-8610
 URL: https://issues.apache.org/jira/browse/IGNITE-8610
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.5
Reporter: Pavel Kovalenko
Assignee: Pavel Kovalenko
 Fix For: 2.6


After implementation IGNITE-6411 and IGNITE-8087 we can face with situation 
when after some checkpoint, WAL was temporarily disabled and enabled again. In 
this case we can't treat such checkpoint as start point to rebalance, because 
WAL history after such checkpoint may contain gaps.

We should rework our checkpoint / wal history searching mechanism and ignore 
such checkpoints.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: NodeJS thin client: full API

2018-05-24 Thread Pavel Petroshenko
Hi Denis,

That's a good point, thanks. This should be a part of the "Usage" section.
I'll follow up in JIRA.

p.

On Thu, May 24, 2018 at 10:49 AM, Denis Magda  wrote:

> Pavel,
>
> Recalled that we've not described how to authenticate and set up SSL from
> the client side. Please consider this for the doc. Left some notes in the
> JIRA.
>
> --
> Denis
>
> On Wed, May 23, 2018 at 12:25 PM, Denis Magda  wrote:
>
> > Alexey, Pavel,
> >
> > I've done a preliminary review of the doc and moved it to the readme.io
> > page:
> > https://apacheignite.readme.io/v2.4/docs/nodejs-thin-client
> >
> > The page is hidden. I'll grant you access to readme so that you can
> update
> > the doc taking my suggestions into account:
> > https://issues.apache.org/jira/browse/IGNITE-8589
> >
> > --
> > Denis
> >
> >
> > On Mon, May 21, 2018 at 6:39 PM, Alexey Kuznetsov  >
> > wrote:
> >
> >> Hi,
> >>
> >> FYI, HZ also has NodeJs client: https://github.com/
> >> hazelcast/hazelcast-nodejs-client
> >> May be it is worth to take a look?
> >>
> >> --
> >> Alexey Kuznetsov
> >>
> >
> >
>


Re: Node.js Thin Client @ npmjs

2018-05-24 Thread Pavel Petroshenko
Fair enough. Consistency with the other clients is a good argument.

p.

On Thu, May 24, 2018 at 11:44 AM, Dmitriy Setrakyan 
wrote:

> On Thu, May 24, 2018 at 11:42 AM, Igor Sapego  wrote:
>
> > Well, all other clients have the same version as Ignite.
> > Are there any reasons to make a Node.js client some
> > kind of a special case?
> >
> >
> I may have spoken too soon. Do we plan to include the node.js client into
> Ignite build? If yes, then it will be built anew with every Ignite release,
> so the same versioning makes sense in this case.
>
> D.
>


Re: [VOTE] Apache Ignite 2.5.0 RC1

2018-05-24 Thread Pavel Tupitsyn
+1 (binding)

Checked .NET side of things: build from sources, run examples.

On Thu, May 24, 2018 at 8:59 PM, Andrey Gura  wrote:

> Igniters,
>
> We've uploaded a 2.5.0 release candidate to
> https://dist.apache.org/repos/dist/dev/ignite/2.5.0-rc1/
>
> Git tag name is
> 2.5.0-rc1
>
> This release includes the following changes:
>
> Ignite:
> * Implemented Zookeeper discovery SPI.
> * Added Java thin client.
> * Added continuous queries with transformers.
> * Implemented general approach for critical failures handling.
> * Added the new method IgniteMXBean.getCurrentCoordinatorFormatted()
> which displays formatted information about current coordiantor.
> * Added partition update counters verification on partition map exchange.
> * Added ability to disable WAL during initial data rebalancing.
> * Added additional data storage metrics
> * Added additional transactional metrics.
> * Added DEB package and improved RPM package.
> * Added information about fields serialization/deserialization errors
> in BinaryMarshaller.
> * Added configurable automatic rollback timeout for pending
> transactions preventing partition map exchange to complete.
> * Added preventing infinite iterations of corrupted B+Tree. New
> IGNITE_BPLUS_TREE_LOCK_RETRIES property was added, default value is
> Integer.MAX_VALUE.
> * Implemented handling of broken segment in WAL compaction.
> * Fixed affinity to always choose primary node from one of the owners
> when persistence is enabled.
> * Partition eviction now does not block exchange.
> * Fixed the issue that prevented rebalancing on a new baseline node join.
> * Fixed potential deadlock during preloading.
> * Fixed exception handling in IsolatedUpdater of DataStreamer.
> * Fixed the issue when ComputeTaskFuture.get() threw a trimmed
> exception chain: now full exception chain is propagated to the caller;
> the exception thrown from IgniteCompute methods may now differ from
> the previous behavior.
> * Fixed null value being passed to entry processor when such entry
> actually exists.
> * Fixed problem when cluster could not survive after
> IgniteOutOfMemoryException.
> * Fixed NullPointerException on batch atomic cache operations with cache
> store.
> * Fixed bug with ScanQuery transformer which applies to the first
> result page only.
> * Fixed hanging of node stopping in case when node is invalidated and
> persistence is enabled.
> * BinaryMetadata exchange is triggered only if there are actual
> changes in BinaryMetadata.
> * Fixed contention while executing parallel atomic single get operations.
> * Fixed cache metadata corruption during concurrent checkpoint and
> cache destroy.
> * Implemented additional synchronization for correct partition
> counters update on partition map exchange.
> * Optimized checkpoint lock acquisition for demander thread.
> * Destroyed caches with enabled persistence now will not survive grid
> restart.
> * Nodes with incompatible SQL metadata will not be able to join grid.
> This fixes known case for index corruption.
> * Fixed fillFactor memory metric value.
> * Fixed bug with checkpoint buffer overflow with enabled throttling.
> * Fixed crash recovery problem when index tree could reference to
> removed data pages.
> * Fixed error during cluster initialization with partially acceptable
> WAL archive.
> * Fixed issue with local partitions size changing inside checkpoint write
> lock.
> * Fixed LOG_ONLY and BACKGROUND modes to be power-crash resistant.
> * Fixed incorrect finishing of implicit transactions on unstable topology.
> * Fixed incorrect mapping of smallint Cassandra type.
> * Introduced log markers support to IgniteLogger and standard DEV_ONLY
> marker.
> * Changed Log4J and Log4J2 adapters to update active logger
> configuration when configuration source file is modified.
> * Fixed sqlline.sh script to use Java from JAVA_HOME.
> * Improved Docker image build.
> * Default and recommended JVM options are updated.
>
> Ignite .Net:
> * Added SSL configuration for node-to-node links.
> * Added dynamic service proxies.
> * Propagated query metrics to ICache.GetQueryMetrics() and
> ICache.ResetQueryMetrics().
> * DML update via LINQ.
> * Fixed hang caused by mishandled exception during custom cache store
> deserialization.
> * Fixed GroupBy and Where issues in LINQ.
> * .Net thin client SSL support.
>
> Ignite C++:
> * Fixed bug with LocalSize cache method.
>
> SQL:
> * Implemented COPY command for fast data load.
> * Added support of authorization for CREATE TABLE and DROP TABLE commands.
> * Improved deadlock handling in DML.
> * Added IGNITE_SQL_FORCE_LAZY_RESULT_SET option to force all SQL
> queries on node using lazy result set.
> * Fixed SQL query hang if cluster contains nodes not in baseline.
>
> JDBC:
> * Added SSL support.
> * Added authentication.
> * Added streaming mode to thin driver.
> * Semicolon now can be used as delimiter for properties in URL string.
> * Changed default socket buffer size to 64Kb in thin 

[jira] [Created] (IGNITE-8609) SQL: Get rid of syncronization in mvcc processor.

2018-05-24 Thread Pavel Kuznetsov (JIRA)
Pavel Kuznetsov created IGNITE-8609:
---

 Summary: SQL: Get rid of syncronization in mvcc processor.
 Key: IGNITE-8609
 URL: https://issues.apache.org/jira/browse/IGNITE-8609
 Project: Ignite
  Issue Type: Task
  Components: sql
Reporter: Pavel Kuznetsov
Assignee: Igor Seliverstov


Currently we have to do synchronized actions in 
org/apache/ignite/internal/managers/communication/GridIoManager.java:1124 
(org.apache.ignite.internal.managers.communication.GridIoManager#processRegularMessage)
 due to fails on nio thread on load.

We are able to optimize this code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Node.js Thin Client @ npmjs

2018-05-24 Thread Dmitriy Setrakyan
On Thu, May 24, 2018 at 11:42 AM, Igor Sapego  wrote:

> Well, all other clients have the same version as Ignite.
> Are there any reasons to make a Node.js client some
> kind of a special case?
>
>
I may have spoken too soon. Do we plan to include the node.js client into
Ignite build? If yes, then it will be built anew with every Ignite release,
so the same versioning makes sense in this case.

D.


Re: Node.js Thin Client @ npmjs

2018-05-24 Thread Igor Sapego
Well, all other clients have the same version as Ignite.
Are there any reasons to make a Node.js client some
kind of a special case?

Best Regards,
Igor

On Thu, May 24, 2018 at 8:21 PM, Dmitriy Setrakyan 
wrote:

> On Thu, May 24, 2018 at 10:03 AM, Pavel Petroshenko  >
> wrote:
>
> > Igor,
> >
> > Are you proposing to update the Thin Client versions on every Ignite
> > release regardless of the changes made to them?
> >
> > I tend to think that an independent versioning scheme for the Thin
> Clients
> > might be more flexible/meaningful.
> >
>
> Pavel, I agree, the thin clients should have independent versioning.
>


Re: IGNITE-8583 - review needed

2018-05-24 Thread Dmitry Pavlov
Reasonable proposal and I hope most Igniters will follow it.

In the same time I've just clicked to the link
https://issues.apache.org/jira/browse/IGNITE-8583 provided by Dmitriy to
check ticket name, and then I've clicked to review link
https://reviews.ignite.apache.org/ignite/review/IGNT-CR-623 to find out
review status. So I'm OK if contributor will provide just link to ticket.

чт, 24 мая 2018 г. в 20:47, Dmitriy Setrakyan :

> I continuously ask not to provide naked ticket numbers in the subject.
> Please also add titles or descriptions. I doubt anyone in the community can
> tell what this is about just by looking at the number.
>
> D.
>
> On Thu, May 24, 2018 at 4:09 AM, Dmitry Pavlov 
> wrote:
>
> > Hi Dmitriy,
> >
> > According to upsource
> > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-623 review was
> > already started by Alexey G. and  Alexey K.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> >
> >
> > чт, 24 мая 2018 г. в 14:04, Dmitriy Govorukhin <
> > dmitriy.govoruk...@gmail.com
> > >:
> >
> > > Igniters,
> > >
> > > I fixed DataStorageMetricsMXBean.getOffHeapSize, please review my
> > changes.
> > >
> > > https://issues.apache.org/jira/browse/IGNITE-8583
> > >
> >
>


[GitHub] ignite pull request #4064: TcpDiscoveryMulticastIpFinder changed to TcpDisco...

2018-05-24 Thread antkr
GitHub user antkr opened a pull request:

https://github.com/apache/ignite/pull/4064

TcpDiscoveryMulticastIpFinder changed to TcpDiscoveryVmIpFinder in xm…

PR for TC purposes.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6826

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4064.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4064


commit c413c236863202ee902c2f070bc6a2130ddb5201
Author: Anton Kurbanov 
Date:   2018-05-24T18:05:03Z

TcpDiscoveryMulticastIpFinder changed to TcpDiscoveryVmIpFinder in xml 
configurations.




---


[VOTE] Apache Ignite 2.5.0 RC1

2018-05-24 Thread Andrey Gura
Igniters,

We've uploaded a 2.5.0 release candidate to
https://dist.apache.org/repos/dist/dev/ignite/2.5.0-rc1/

Git tag name is
2.5.0-rc1

This release includes the following changes:

Ignite:
* Implemented Zookeeper discovery SPI.
* Added Java thin client.
* Added continuous queries with transformers.
* Implemented general approach for critical failures handling.
* Added the new method IgniteMXBean.getCurrentCoordinatorFormatted()
which displays formatted information about current coordiantor.
* Added partition update counters verification on partition map exchange.
* Added ability to disable WAL during initial data rebalancing.
* Added additional data storage metrics
* Added additional transactional metrics.
* Added DEB package and improved RPM package.
* Added information about fields serialization/deserialization errors
in BinaryMarshaller.
* Added configurable automatic rollback timeout for pending
transactions preventing partition map exchange to complete.
* Added preventing infinite iterations of corrupted B+Tree. New
IGNITE_BPLUS_TREE_LOCK_RETRIES property was added, default value is
Integer.MAX_VALUE.
* Implemented handling of broken segment in WAL compaction.
* Fixed affinity to always choose primary node from one of the owners
when persistence is enabled.
* Partition eviction now does not block exchange.
* Fixed the issue that prevented rebalancing on a new baseline node join.
* Fixed potential deadlock during preloading.
* Fixed exception handling in IsolatedUpdater of DataStreamer.
* Fixed the issue when ComputeTaskFuture.get() threw a trimmed
exception chain: now full exception chain is propagated to the caller;
the exception thrown from IgniteCompute methods may now differ from
the previous behavior.
* Fixed null value being passed to entry processor when such entry
actually exists.
* Fixed problem when cluster could not survive after IgniteOutOfMemoryException.
* Fixed NullPointerException on batch atomic cache operations with cache store.
* Fixed bug with ScanQuery transformer which applies to the first
result page only.
* Fixed hanging of node stopping in case when node is invalidated and
persistence is enabled.
* BinaryMetadata exchange is triggered only if there are actual
changes in BinaryMetadata.
* Fixed contention while executing parallel atomic single get operations.
* Fixed cache metadata corruption during concurrent checkpoint and
cache destroy.
* Implemented additional synchronization for correct partition
counters update on partition map exchange.
* Optimized checkpoint lock acquisition for demander thread.
* Destroyed caches with enabled persistence now will not survive grid restart.
* Nodes with incompatible SQL metadata will not be able to join grid.
This fixes known case for index corruption.
* Fixed fillFactor memory metric value.
* Fixed bug with checkpoint buffer overflow with enabled throttling.
* Fixed crash recovery problem when index tree could reference to
removed data pages.
* Fixed error during cluster initialization with partially acceptable
WAL archive.
* Fixed issue with local partitions size changing inside checkpoint write lock.
* Fixed LOG_ONLY and BACKGROUND modes to be power-crash resistant.
* Fixed incorrect finishing of implicit transactions on unstable topology.
* Fixed incorrect mapping of smallint Cassandra type.
* Introduced log markers support to IgniteLogger and standard DEV_ONLY marker.
* Changed Log4J and Log4J2 adapters to update active logger
configuration when configuration source file is modified.
* Fixed sqlline.sh script to use Java from JAVA_HOME.
* Improved Docker image build.
* Default and recommended JVM options are updated.

Ignite .Net:
* Added SSL configuration for node-to-node links.
* Added dynamic service proxies.
* Propagated query metrics to ICache.GetQueryMetrics() and
ICache.ResetQueryMetrics().
* DML update via LINQ.
* Fixed hang caused by mishandled exception during custom cache store
deserialization.
* Fixed GroupBy and Where issues in LINQ.
* .Net thin client SSL support.

Ignite C++:
* Fixed bug with LocalSize cache method.

SQL:
* Implemented COPY command for fast data load.
* Added support of authorization for CREATE TABLE and DROP TABLE commands.
* Improved deadlock handling in DML.
* Added IGNITE_SQL_FORCE_LAZY_RESULT_SET option to force all SQL
queries on node using lazy result set.
* Fixed SQL query hang if cluster contains nodes not in baseline.

JDBC:
* Added SSL support.
* Added authentication.
* Added streaming mode to thin driver.
* Semicolon now can be used as delimiter for properties in URL string.
* Changed default socket buffer size to 64Kb in thin driver.
* Added FQDN to thin client in order to support multiple IPs for
connection establishment.

ODBC:
* Added SSL support.
* Implemented username/password authentication.
* Added support of multiple addresses in connection string.
* Added support for SQL_ATTR_LOGIN_TIMEOUT.

Web Console:
* Added support for persistence in demo mode.
* Configuration screen 

Re: NodeJS thin client: full API

2018-05-24 Thread Denis Magda
Pavel,

Recalled that we've not described how to authenticate and set up SSL from
the client side. Please consider this for the doc. Left some notes in the
JIRA.

--
Denis

On Wed, May 23, 2018 at 12:25 PM, Denis Magda  wrote:

> Alexey, Pavel,
>
> I've done a preliminary review of the doc and moved it to the readme.io
> page:
> https://apacheignite.readme.io/v2.4/docs/nodejs-thin-client
>
> The page is hidden. I'll grant you access to readme so that you can update
> the doc taking my suggestions into account:
> https://issues.apache.org/jira/browse/IGNITE-8589
>
> --
> Denis
>
>
> On Mon, May 21, 2018 at 6:39 PM, Alexey Kuznetsov 
> wrote:
>
>> Hi,
>>
>> FYI, HZ also has NodeJs client: https://github.com/
>> hazelcast/hazelcast-nodejs-client
>> May be it is worth to take a look?
>>
>> --
>> Alexey Kuznetsov
>>
>
>


Re: IGNITE-8583 - review needed

2018-05-24 Thread Dmitriy Setrakyan
I continuously ask not to provide naked ticket numbers in the subject.
Please also add titles or descriptions. I doubt anyone in the community can
tell what this is about just by looking at the number.

D.

On Thu, May 24, 2018 at 4:09 AM, Dmitry Pavlov 
wrote:

> Hi Dmitriy,
>
> According to upsource
> https://reviews.ignite.apache.org/ignite/review/IGNT-CR-623 review was
> already started by Alexey G. and  Alexey K.
>
> Sincerely,
> Dmitriy Pavlov
>
>
>
> чт, 24 мая 2018 г. в 14:04, Dmitriy Govorukhin <
> dmitriy.govoruk...@gmail.com
> >:
>
> > Igniters,
> >
> > I fixed DataStorageMetricsMXBean.getOffHeapSize, please review my
> changes.
> >
> > https://issues.apache.org/jira/browse/IGNITE-8583
> >
>


Re: MTCGA & 6th mass run-all

2018-05-24 Thread Denis Magda
Great outcomes! Can't wait seeing how many tests were fixed since the
previous mass run.

--
Denis

On Thu, May 24, 2018 at 7:06 AM, Dmitry Pavlov 
wrote:

> Hi,
>
> A lot of Igniters made their contributions to Make TC Green during 6 last
> weeks. It is Aleksey Plekhanov, Maxim Muzafarov, Alexey Goncharuk, Ilya
> Lantukh, Ivan Rakov, Alexey Kuznetsov, Taras Ledkov, Andrey Kuznetsov,
> Alexei Scherbakov, Ivan Fedotov, Alexander Menshikov, Andrew Medvedev,
> Andrew Mashenkov, Nikolay Izhikov, Stanilovsky Evgeny, Vyacheslav Daradur,
> Alexey Kuznetsov, Pavel Kovalenko, Vitaliy Biryukov, Alexey Kukushkin,
> Pavel Pereslegin, Amelchev Nikita.
>
> Folks, let me say thank you for your contributions. This effort will
> definitely move us towards day with 0 fails.
>
> It’s time to check progress, so I’m going to schedule mass run-all's during
> weekend. I plan to trigger these builds Fri May 25 at 15:00 GMT, 18:00 MSK.
> Please reply if you have objections or need TC at this time. Feel free to
> move builds to top if you have needed fixes.
>
> Any finished patch review process for MTCGA PA issues is also appreciated.
>
> Sincerely,
> Dmitriy Pavlov
>


Re: Node.js Thin Client @ npmjs

2018-05-24 Thread Dmitriy Setrakyan
On Thu, May 24, 2018 at 10:03 AM, Pavel Petroshenko 
wrote:

> Igor,
>
> Are you proposing to update the Thin Client versions on every Ignite
> release regardless of the changes made to them?
>
> I tend to think that an independent versioning scheme for the Thin Clients
> might be more flexible/meaningful.
>

Pavel, I agree, the thin clients should have independent versioning.


[jira] [Created] (IGNITE-8608) .NET: Sign release NuGet packages

2018-05-24 Thread Pavel Tupitsyn (JIRA)
Pavel Tupitsyn created IGNITE-8608:
--

 Summary: .NET: Sign release NuGet packages
 Key: IGNITE-8608
 URL: https://issues.apache.org/jira/browse/IGNITE-8608
 Project: Ignite
  Issue Type: Improvement
  Components: platforms
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 2.6


NuGet signed package submissions has been introduced recently:
https://blog.nuget.org/20180522/Introducing-signed-package-submissions.html

Sign Ignite.NET packages when releasing them.





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Node.js Thin Client @ npmjs

2018-05-24 Thread Pavel Petroshenko
Igor,

Are you proposing to update the Thin Client versions on every Ignite
release regardless of the changes made to them?

I tend to think that an independent versioning scheme for the Thin Clients
might be more flexible/meaningful.

p.

On Thu, May 24, 2018 at 2:04 AM, Igor Sapego  wrote:

> 1. Sounds OK for me.
> 2. Since Node.js client is going to be included in Ignite releases,
> I guess it make sense to align its version with the version of
> Ignite, i.e. 2.6, etc...
>
> Best Regards,
> Igor
>
> On Thu, May 24, 2018 at 9:02 AM, Pavel Petroshenko 
> wrote:
>
> > Igniters,
> >
> > I'm finalizing the docs (with regards to installation instructions) for
> the
> > Node.js Thin Client at readme.io. And I've got a couple of questions.
> >
> > 1. Does the name "apache-ignite-client" for the Node.js Thin Client
> > npm module sound good to everyone? This name will be used to install the
> > client library, for example:
> >
> > npm i -g apache-ignite-client
> >
> > or include it from the source code:
> >
> > const IgniteClient = require('apache-ignite-client');
> >
> > 2. Which version should the npm module get: 1.0 or does it need to be
> > aligned with the Ignite release version when it's published or compatible
> > with, i.e. 2.4, 2.5 or 2.6?
> >
> > thanks,
> > p.
> >
>


[GitHub] ignite pull request #4063: IGNITE-8547 Use JVM serialization for enum values...

2018-05-24 Thread alamar
GitHub user alamar opened a pull request:

https://github.com/apache/ignite/pull/4063

IGNITE-8547 Use JVM serialization for enum values with OptimizedMarsh…

…aller, avoid deadlock.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8547

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4063.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4063


commit 4ddb8bd85c7000356badb7bd8daf233491aaa3bf
Author: Ilya Kasnacheev 
Date:   2018-05-22T11:32:12Z

IGNITE-8547 Use JVM serialization for enum values with OptimizedMarshaller, 
avoid deadlock.




---


[GitHub] ignite pull request #4042: IGNITE-8547 Use JVM serialization for enum values...

2018-05-24 Thread alamar
Github user alamar closed the pull request at:

https://github.com/apache/ignite/pull/4042


---


[GitHub] ignite pull request #4051: IGNITE-8560 checkFirst and checkThrough parameter...

2018-05-24 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4051


---


[GitHub] ignite pull request #4062: IGNITE-8567: Imputer and Binarizer

2018-05-24 Thread zaleslaw
GitHub user zaleslaw opened a pull request:

https://github.com/apache/ignite/pull/4062

IGNITE-8567: Imputer and Binarizer



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8567

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4062.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4062


commit 098d2832b3f4af23c72bc25f43e4ab8a95f2f416
Author: Zinoviev Alexey 
Date:   2018-04-11T18:40:27Z

IGNITE-7829: Added example

commit 78f9ea687bff77ec9f6bef87126569cb92cbe745
Author: Zinoviev Alexey 
Date:   2018-04-13T15:44:26Z

Merge branch 'master' of https://github.com/apache/ignite

commit 199e17d19ccbde9f15aba5375d834c3930b3a989
Author: Zinoviev Alexey 
Date:   2018-04-27T10:12:47Z

Merge branch 'master' of https://github.com/apache/ignite

commit aca9833df4d3cc4a641dd9109daaf628bc85acdf
Author: Zinoviev Alexey 
Date:   2018-05-08T05:29:49Z

Merge branch 'master' of https://github.com/apache/ignite

commit bb244de762b89d0a1e5606aa282e34d92752595b
Author: Zinoviev Alexey 
Date:   2018-05-16T11:42:06Z

Merge branch 'master' of https://github.com/apache/ignite

commit a0f6f0fcc54f8a6da36d4b61836719e19a6bbb4a
Author: Zinoviev Alexey 
Date:   2018-05-16T13:21:47Z

IGNITE-8511: Added MultiClass Log Regression

commit b4cb1a42d35a0da9f8b762207011a46c6f542a20
Author: Zinoviev Alexey 
Date:   2018-05-16T16:17:39Z

Merge branch 'master' of https://github.com/apache/ignite

commit eb6178cd8e4a61fa12d4558b4377d53f94029c86
Author: Zinoviev Alexey 
Date:   2018-05-16T16:18:17Z

Merge branch 'master' into ignite-8511

commit 21cb4433728618c06aa8e16fe681772ea25c9b63
Author: Zinoviev Alexey 
Date:   2018-05-21T07:47:08Z

IGNITE-8511: Added MultiClass Log Regression

commit 64d96c8c29e290d7ff98c8018056609c3bab83ed
Author: Zinoviev Alexey 
Date:   2018-05-21T07:56:28Z

IGNITE-8511: Added MultiClass Log Regression

commit e10fb00ed15ac348f097d0eb91abfa4661769025
Author: Zinoviev Alexey 
Date:   2018-05-23T09:51:13Z

IGNITE-8567: Added examples and 2 preprocessors

commit a3cf8b5c35b961fd772dabf350f937425822c6ba
Author: zaleslaw 
Date:   2018-05-24T15:05:21Z

ignite-8567: Added tests for Imputer/Binarizer preprocessors




---


Re: [jira] [Created] (IGNITE-8595) SQL: Ability to cancel DDL operations

2018-05-24 Thread Ray
Thanks for the reply, Vladimir.

Is there a ticket to track the cancel feature described by you?



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/


[GitHub] ignite pull request #4054: IGNITE-8583 fix checkpoint buffer size metric

2018-05-24 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4054


---


[jira] [Created] (IGNITE-8607) [.NET] Support metrics changes in DataStorageMetricsMXBean

2018-05-24 Thread Dmitriy Govorukhin (JIRA)
Dmitriy Govorukhin created IGNITE-8607:
--

 Summary: [.NET] Support metrics changes in DataStorageMetricsMXBean
 Key: IGNITE-8607
 URL: https://issues.apache.org/jira/browse/IGNITE-8607
 Project: Ignite
  Issue Type: Bug
Reporter: Dmitriy Govorukhin






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-8606) Node hangs on next exchange, when no access to marshaller's folder

2018-05-24 Thread Vladislav Pyatkov (JIRA)
Vladislav Pyatkov created IGNITE-8606:
-

 Summary: Node hangs on next exchange, when no access to 
marshaller's folder 
 Key: IGNITE-8606
 URL: https://issues.apache.org/jira/browse/IGNITE-8606
 Project: Ignite
  Issue Type: Bug
Reporter: Vladislav Pyatkov


{noformat}
2018-05-18 11:12:57.572 
[ERROR][tcp-disco-msg-worker-#3%DPL_GRID%DplGridNodeName%][o.a.i.i.MarshallerMappingFileStore]
 Failed to write class name to file [platformId=0id=1713316383, 
clsName=com.sbt.dpl.gridgain.affinity.DPLIndexAffinityPrimaryFilter, 
file=/u01/pprb/work/marshaller/1713316383.classname0]
java.io.FileNotFoundException: /u01/pprb/work/marshaller/1713316383.classname0 
(No such file or directory)
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(FileOutputStream.java:270)
at java.io.FileOutputStream.(FileOutputStream.java:213)
at java.io.FileOutputStream.(FileOutputStream.java:162)
at 
org.apache.ignite.internal.MarshallerMappingFileStore.writeMapping(MarshallerMappingFileStore.java:94)
at 
org.apache.ignite.internal.MarshallerMappingFileStore.mergeAndWriteMapping(MarshallerMappingFileStore.java:207)
at 
org.apache.ignite.internal.MarshallerContextImpl.onMappingDataReceived(MarshallerContextImpl.java:201)
at 
org.apache.ignite.internal.processors.marshaller.GridMarshallerMappingProcessor.processIncomingMappings(GridMarshallerMappingProcessor.java:356)
at 
org.apache.ignite.internal.processors.marshaller.GridMarshallerMappingProcessor.onJoiningNodeDataReceived(GridMarshallerMappingProcessor.java:336)
at 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$5.onExchange(GridDiscoveryManager.java:908)
at 
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.onExchange(TcpDiscoverySpi.java:1939)
at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processNodeAddedMessage(ServerImpl.java:4220)
at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2744)
at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] ignite pull request #4061: IGNITE-8605 Specify jline minor version to fix nc...

2018-05-24 Thread alamar
GitHub user alamar opened a pull request:

https://github.com/apache/ignite/pull/4061

IGNITE-8605 Specify jline minor version to fix ncurses issue.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8605

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4061.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4061


commit 4b7251df22be488b5931f98720ee1d4df9dd9d3d
Author: Ilya Kasnacheev 
Date:   2018-05-24T14:55:43Z

IGNITE-8605 Specify jline minor version to fix ncurses issue.




---


[jira] [Created] (IGNITE-8605) sqlline can't work with terminal on newer ncurses

2018-05-24 Thread Ilya Kasnacheev (JIRA)
Ilya Kasnacheev created IGNITE-8605:
---

 Summary: sqlline can't work with terminal on newer ncurses
 Key: IGNITE-8605
 URL: https://issues.apache.org/jira/browse/IGNITE-8605
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 2.5
Reporter: Ilya Kasnacheev


On Ubuntu 18.04:
{code}
~/w/incubator-ignite/modules/sqlline% IGNITE_HOME=~/w/incubator-ignite . 
bin/sqlline.sh
[ERROR] Failed to construct terminal; falling back to unsupported
java.lang.NumberFormatException: For input string: "0x100"
at 
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:580)
at java.lang.Integer.valueOf(Integer.java:766)
at jline.internal.InfoCmp.parseInfoCmp(InfoCmp.java:59)
at jline.UnixTerminal.parseInfoCmp(UnixTerminal.java:242)
at jline.UnixTerminal.(UnixTerminal.java:65)
at jline.UnixTerminal.(UnixTerminal.java:50)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at jline.TerminalFactory.getFlavor(TerminalFactory.java:211)
at jline.TerminalFactory.create(TerminalFactory.java:102)
at jline.TerminalFactory.get(TerminalFactory.java:186)
at jline.TerminalFactory.get(TerminalFactory.java:192)
at sqlline.SqlLineOpts.(SqlLineOpts.java:45)
at sqlline.SqlLine.(SqlLine.java:54)
at sqlline.SqlLine.start(SqlLine.java:372)
at sqlline.SqlLine.main(SqlLine.java:265)

[ERROR] Failed to construct terminal; falling back to unsupported
java.lang.NumberFormatException: For input string: "0x100"
at 
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:580)
at java.lang.Integer.valueOf(Integer.java:766)
at jline.internal.InfoCmp.parseInfoCmp(InfoCmp.java:59)
at jline.UnixTerminal.parseInfoCmp(UnixTerminal.java:242)
at jline.UnixTerminal.(UnixTerminal.java:65)
at jline.UnixTerminal.(UnixTerminal.java:50)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at jline.TerminalFactory.getFlavor(TerminalFactory.java:211)
at jline.TerminalFactory.create(TerminalFactory.java:102)
at jline.TerminalFactory.create(TerminalFactory.java:51)
at sqlline.SqlLine.getConsoleReader(SqlLine.java:705)
at sqlline.SqlLine.begin(SqlLine.java:639)
at sqlline.SqlLine.start(SqlLine.java:373)
at sqlline.SqlLine.main(SqlLine.java:265)

sqlline version 1.3.0
sqlline>
... and then history and command editing won't work
{code}

See also https://github.com/jline/jline2/issues/281

I think we should manually peg jline verison to 2.14.4



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] ignite pull request #4060: IGN-10640 [IGNITE-8603] Add JMX-metric to cluster...

2018-05-24 Thread dgladkikh
GitHub user dgladkikh opened a pull request:

https://github.com/apache/ignite/pull/4060

IGN-10640 [IGNITE-8603] Add JMX-metric to cluster: baseline nodes



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8603

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4060.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4060


commit 9ce4e57ddafa828d58f692bd4f8a1645d6e61f74
Author: dgladkikh 
Date:   2018-05-24T14:44:07Z

IGN-10640 [IGNITE-8603] Add JMX-metric to cluster: baseline nodes




---


[jira] [Created] (IGNITE-8604) .NET test failures probably after IGNITE-5789 merge

2018-05-24 Thread Dmitriy Pavlov (JIRA)
Dmitriy Pavlov created IGNITE-8604:
--

 Summary: .NET test failures probably after IGNITE-5789 merge
 Key: IGNITE-8604
 URL: https://issues.apache.org/jira/browse/IGNITE-8604
 Project: Ignite
  Issue Type: Improvement
Reporter: Dmitriy Pavlov
Assignee: Ilya Kasnacheev


62 new failed tests in .NET
{noformat}
Store.CacheStoreTest    
Current failure:refs/heads/master   #254    Changes (8) 
24 May 18 02:16
First failure:  refs/heads/master   #1  No changes  26 Apr 
18 10:36
TearDown method failed. HandleRegistry is not empty in grid '' (expected 4, 
actual 5):
 '[2, Apache.Ignite.Core.Impl.Cache.Store.CacheStore]
[3, Apache.Ignite.Core.Impl.Cache.Store.CacheStore]
[4, Apache.Ignite.Core.Impl.Cache.Store.CacheStore]
[5, Apache.Ignite.Core.Impl.Cache.Store.CacheStore]'
   at NUnit.Framework.Assert.Fail(String message, Object[] args)
   at Apache.Ignite.Core.Tests.TestUtils.AssertHandleRegistryHasItems(IIgnite 
grid, Int32 expectedCount, Int32 timeout) in 
c:\BuildAgent\work\c182b70f2dfa6507\modules\platforms\dotnet\Apache.Ignite.Core.Tests\TestUtils.Common.cs:line
 339
   at Apache.Ignite.Core.Tests.Cache.Store.CacheStoreTest.AfterTests
{noformat}

First time these test failed
https://ci.ignite.apache.org/viewLog.html?buildId=1323959=buildResultsDiv=IgniteTests24Java8_PlatformNet

Probably it is caused by merge:
https://issues.apache.org/jira/browse/IGNITE-5789

I've tried to revert commit in ignite-5789-1 branch and results:
https://ci.ignite.apache.org/viewLog.html?buildId=1326520=buildResultsDiv=IgniteTests24Java8_PlatformNet

Other test failed, but current tests were passed.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-8603) Add JMX-metric to cluster: baseline nodes

2018-05-24 Thread Dmitriy Gladkikh (JIRA)
Dmitriy Gladkikh created IGNITE-8603:


 Summary: Add JMX-metric to cluster: baseline nodes
 Key: IGNITE-8603
 URL: https://issues.apache.org/jira/browse/IGNITE-8603
 Project: Ignite
  Issue Type: Improvement
Reporter: Dmitriy Gladkikh
 Fix For: 2.6


Need to add a baseline nodes on JMX:


{code:java}
int org.apache.ignite.mxbean.ClusterMetricsMXBean#getBaselineNodes
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


MTCGA & 6th mass run-all

2018-05-24 Thread Dmitry Pavlov
Hi,

A lot of Igniters made their contributions to Make TC Green during 6 last
weeks. It is Aleksey Plekhanov, Maxim Muzafarov, Alexey Goncharuk, Ilya
Lantukh, Ivan Rakov, Alexey Kuznetsov, Taras Ledkov, Andrey Kuznetsov,
Alexei Scherbakov, Ivan Fedotov, Alexander Menshikov, Andrew Medvedev,
Andrew Mashenkov, Nikolay Izhikov, Stanilovsky Evgeny, Vyacheslav Daradur,
Alexey Kuznetsov, Pavel Kovalenko, Vitaliy Biryukov, Alexey Kukushkin,
Pavel Pereslegin, Amelchev Nikita.

Folks, let me say thank you for your contributions. This effort will
definitely move us towards day with 0 fails.

It’s time to check progress, so I’m going to schedule mass run-all's during
weekend. I plan to trigger these builds Fri May 25 at 15:00 GMT, 18:00 MSK.
Please reply if you have objections or need TC at this time. Feel free to
move builds to top if you have needed fixes.

Any finished patch review process for MTCGA PA issues is also appreciated.

Sincerely,
Dmitriy Pavlov


Re: Ability to check and completely fill transactions on creation

2018-05-24 Thread Dmitriy Setrakyan
Anton, why do you need to *alter* event sub-system to introduce a new
event? Yakov's issue was that you propagated private interface to public
API, which is bad of course. Come up with a clean design and it will be
accepted.

My problem with TransactionValidator is that it only solves a small problem
for transactions. If we do that, then we will have to add cache validators,
compute validators, etc, etc, etc. That is why we either should use the
existing event subsystem or come up with a holistic design that will work
across the whole project.

D.

On Thu, May 24, 2018 at 1:38 AM, Anton Vinogradov  wrote:

> Dmitriy,
>
> Yakov is against the solution based on event sub-system
> >> I think that we should think about some other solution instead of
> altering
> >> event sub-system.
>
> Also, I checked is there any chances to fix all the issues found by Yakov
> and see that solution becomes overcomplicated in that case.
> That's why I'm proposing this lightweight solution.
>
> As for me it's a good idea to have such validator since that's a common
> problem at huge deployments when more than one team have access to Ignite
> cluster and there is no other way to setup tx cretion rules.
>
> Yakov,
>
> Could you please share your thoughts on that?
>
>
> чт, 24 мая 2018 г. в 8:58, Dmitriy Setrakyan :
>
> > On Wed, May 23, 2018 at 4:08 AM, Anton Vinogradov  wrote:
> >
> > > Dmitriy, Yakov
> > >
> > > Are there any objections to updated design taking into account the
> > comments
> > > I provided?
> > >
> >
> > Anton, I do not like an additional validator. I think you can accomplish
> > the same with a transaction event. You just need to design it more
> cleanly,
> > incorporating the feedback from Yakov.
> >
>


Re: Cache 6 Suite Execution timeouts

2018-05-24 Thread Alexei Scherbakov
Hi, guys.

Thanks for pointing this out.

I'll look at this then I'll have available time.

Possibly next week.

Meanwhile I suggest not to use async rollbacks for transactions on near
caches.

чт, 24 мая 2018 г. в 14:09, Dmitry Pavlov :

> Alexey? Could I hope you will pick up
> https://issues.apache.org/jira/browse/IGNITE-8509?
>
> чт, 24 мая 2018 г. в 14:00, Maxim Muzafarov :
>
>> Hello,
>>
>> Just bump this topic up.
>>
>> 6/10 execution timeouts for the last 10 build in master!
>>
>> Will anyone help?
>>
>> [1]
>>
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Cache6=1=buildTypeHistoryList_IgniteTests24Java8=%3Cdefault%3E
>>
>> пн, 21 мая 2018 г. в 12:58, Dmitry Pavlov :
>>
>> > Hi Maxim,
>> >
>> > I observe the same, Cache 6 critical recent fails: 18,0% [9 fails / 50
>> > runs]
>> >
>> > I hope Aleksei Scherbakov can help here. Alexey, please step in.
>> >
>> > Sincerely,
>> > Dmitriy Pavlov
>> >
>> > вс, 20 мая 2018 г. в 20:37, Maxim Muzafarov :
>> >
>> > > Hello Igniters,
>> > >
>> > > Have anyone noticed a lot of execution timeouts for Cache 6 Suite?
>> > > Since recently it fails too much on TxRollbackOnTimeoutNearCacheTest
>> with
>> > > "Found long running transaction".
>> > >
>> > > Please refer to TC [1]. It has 4/10 execution timeouts for the last 10
>> > > builds in master.
>> > >
>> > > I've created IGNITE-8509 [2]. Will someone have time to look at? Or
>> I'll
>> > > try when I'm get free.
>> > >
>> > > [1]
>> > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Cache6_IgniteTests24Java8=%3Cdefault%3E=buildTypeStatusDiv
>> > > [2] https://issues.apache.org/jira/browse/IGNITE-8509
>> > >
>> >
>>
>

-- 

Best regards,
Alexei Scherbakov


[jira] [Created] (IGNITE-8602) Add support filter label=null for control.sh tx utility

2018-05-24 Thread Dmitry Sherstobitov (JIRA)
Dmitry Sherstobitov created IGNITE-8602:
---

 Summary: Add support filter label=null for control.sh tx utility
 Key: IGNITE-8602
 URL: https://issues.apache.org/jira/browse/IGNITE-8602
 Project: Ignite
  Issue Type: Improvement
Affects Versions: 2.5
Reporter: Dmitry Sherstobitov






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: async operation is not fair async

2018-05-24 Thread Yakov Zhdanov
Alexey Goncharuk, I remember we started working on async connection
establishment. This should fix latency issue related to network which I
believe gives the most contribution to overall latency. Mapping logic and
other stuff can be ignored as it can very rarely be an issue at least on
stable tolopogies. What is the status with async connections? That would
really be a huge improvement!

Also please remember that uncontrolled async operations may lead to OOME,
therefore at some point when there are too many uncompleted async
operations newly invoked async operations should become synchronous, i.e.
we should return completed future, ignoring the fact that user expected us
to be async.

I would like to have very strong reasons to start reapproaching this.

--Yakov


[jira] [Created] (IGNITE-8601) Add to control.sh tx utility information about transaction start time

2018-05-24 Thread Dmitry Sherstobitov (JIRA)
Dmitry Sherstobitov created IGNITE-8601:
---

 Summary: Add to control.sh tx utility information about 
transaction start time
 Key: IGNITE-8601
 URL: https://issues.apache.org/jira/browse/IGNITE-8601
 Project: Ignite
  Issue Type: Improvement
Affects Versions: 2.5
Reporter: Dmitry Sherstobitov


This information will be useful



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Ticket review checklist

2018-05-24 Thread Dmitry Pavlov
Petr, good point. It is more intuitive, we should mark test we can ignore
by mute.

So Vladimir, you or other Ignite veteran can mute test, if can say it is
not important.

чт, 24 мая 2018 г. в 15:07, Petr Ivanov :

> Why cannot we mute (and file corresponding tickets) all test failures
> (including flaky) to some date and start initiative Green TC?
>
>
>
> > On 24 May 2018, at 15:04, Vladimir Ozerov  wrote:
> >
> > Dmitry,
> >
> > We cannot add this requirements, because we do have failures on TC. This
> > requirement implies that all development would stop until TC is green.
> > We never had old requirement work, neither we need to enforce it now.
> >
> > On Thu, May 24, 2018 at 2:59 PM, Dmitry Pavlov 
> > wrote:
> >
> >> 3.c
> >>
> >>   1. All test suites *MUST* be run on TeamCity [3] before merge to
> master,
> >>   there *MUST NOT* be any test failures
> >>
> >>
> >> 'New' word should be removed because we cant separate `new` and `non
> new`
> >> failures.
> >>
> >> Let's imagine example, we have 50 green runs in master. And PR Run-All
> >> contains this test failed. Is it new or not new? Actually we don't know.
> >>
> >> Existing requirement is about all TC must be green, so let's keep it as
> is.
> >>
> >> ср, 23 мая 2018 г. в 17:02, Vladimir Ozerov :
> >>
> >>> Igniters,
> >>>
> >>> I created review checklist on WIKI [1] and also fixed related pages
> (e.g.
> >>> "How To Contribute"). Please let me know if you have any comments
> before
> >> I
> >>> go with public announce.
> >>>
> >>> Vladimir.
> >>>
> >>> [1]
> https://cwiki.apache.org/confluence/display/IGNITE/Review+Checklist
> >>>
> >>> On Thu, May 10, 2018 at 5:10 PM, Vladimir Ozerov  >
> >>> wrote:
> >>>
>  Ilya,
> 
>  We define that exception messages *SHOULD* have clear explanation on
> >> what
>  is wrong. *SHOULD* mean that the rule should be followed unless there
> >> is
> >>> a
>  reason not to follow. In your case you refer to some unexpected
> >> behavior.
>  I.e. an exceptional situation developer is not aware of. In this case
> >> for
>  sure we cannot force contributor to explain what is wrong, because,
> >> well,
>  we don't know. This is why we relaxed the rule from *MUST* to
> *SHOULD*.
> 
>  On Thu, May 10, 2018 at 3:50 PM, Ilya Kasnacheev <
>  ilya.kasnach...@gmail.com> wrote:
> 
> > I don't think I quite understand how exception explanations should
> >> work.
> >
> > Imagine we have the following exception:
> >
> > // At least RuntimeException can be thrown by the code above when
> > GridCacheContext is cleaned and there is
> > // an attempt to use cleaned resources.
> > U.error(log, "Unexpected exception during cache update", e);
> >
> > I mean, we genuinely don't know what happened here.
> >
> > Under new rules, what kind of "workaround" would that exception
> >> suggest?
> > "Try turning it off and then back on"?
> > What explanation how to resolve this exception can we offer? "Please
> >>> write
> > to d...@apache.ignite.org or to Apache JIRA, and then wait for a
> >> release
> > with fix?"
> >
> > I'm really confused how we can implement 1.6 and 1.7 when dealing
> with
> > messy real-world code.
> >
> > Regards,
> >
> >
> > --
> > Ilya Kasnacheev
> >
> > 2018-05-10 11:39 GMT+03:00 Vladimir Ozerov :
> >
> >> Andrey, Anton, Alex
> >>
> >> Agree, *SHOULD* is more appropriate here.
> >>
> >> Please see latest version below. Does anyone want to add or change
> >> something? Let's wait for several days for more feedback and then
> > publish
> >> and announce this list. Note that it would not be carved in stone
> >> and
> >>> we
> >> will be able to change it at any time if needed.
> >>
> >> 1) API
> >> 1.1) API compatibility *MUST* be maintained between minor releases.
> >> Do
> > not
> >> remove existing methods or change their signatures, deprecate them
> > instead
> >> 1.2) Default behaviour "SHOULD NOT* be changed between minor
> >> releases,
> >> unless absolutely needed. If change is made, it *MUST* be described
> >> in
> >> "Migration Guide"
> >> 1.3) New operation *MUST* be well-documented in code (javadoc,
> > dotnetdoc):
> >> documentation must contain method's purpose, description of
> >> parameters
> > and
> >> how their values affect the outcome, description of return value and
> > it's
> >> default, behavior in negative cases, interaction with other
> >> operations
> > and
> >> components
> >> 1.4) API parity between Java and .NET platforms *SHOULD* be
> >> maintained
> > when
> >> operation makes sense on both platforms. If method cannot be
> > implemented in
> >> a platform immediately, new JIRA 

Re: Ticket review checklist

2018-05-24 Thread Dmitry Pavlov
We cannot change this requirement to be softer because we need to come to
sutuation of 0-failed test.

If we allow commit with test failures, there will be a lot of mistakes new
failures will be considered as existing.

All contributors will check only new/not new failures. But actually all
failures should be checked.

чт, 24 мая 2018 г. в 15:04, Vladimir Ozerov :

> Dmitry,
>
> We cannot add this requirements, because we do have failures on TC. This
> requirement implies that all development would stop until TC is green.
> We never had old requirement work, neither we need to enforce it now.
>
> On Thu, May 24, 2018 at 2:59 PM, Dmitry Pavlov 
> wrote:
>
> > 3.c
> >
> >1. All test suites *MUST* be run on TeamCity [3] before merge to
> master,
> >there *MUST NOT* be any test failures
> >
> >
> > 'New' word should be removed because we cant separate `new` and `non new`
> > failures.
> >
> > Let's imagine example, we have 50 green runs in master. And PR Run-All
> > contains this test failed. Is it new or not new? Actually we don't know.
> >
> > Existing requirement is about all TC must be green, so let's keep it as
> is.
> >
> > ср, 23 мая 2018 г. в 17:02, Vladimir Ozerov :
> >
> > > Igniters,
> > >
> > > I created review checklist on WIKI [1] and also fixed related pages
> (e.g.
> > > "How To Contribute"). Please let me know if you have any comments
> before
> > I
> > > go with public announce.
> > >
> > > Vladimir.
> > >
> > > [1]
> https://cwiki.apache.org/confluence/display/IGNITE/Review+Checklist
> > >
> > > On Thu, May 10, 2018 at 5:10 PM, Vladimir Ozerov  >
> > > wrote:
> > >
> > > > Ilya,
> > > >
> > > > We define that exception messages *SHOULD* have clear explanation on
> > what
> > > > is wrong. *SHOULD* mean that the rule should be followed unless there
> > is
> > > a
> > > > reason not to follow. In your case you refer to some unexpected
> > behavior.
> > > > I.e. an exceptional situation developer is not aware of. In this case
> > for
> > > > sure we cannot force contributor to explain what is wrong, because,
> > well,
> > > > we don't know. This is why we relaxed the rule from *MUST* to
> *SHOULD*.
> > > >
> > > > On Thu, May 10, 2018 at 3:50 PM, Ilya Kasnacheev <
> > > > ilya.kasnach...@gmail.com> wrote:
> > > >
> > > >> I don't think I quite understand how exception explanations should
> > work.
> > > >>
> > > >> Imagine we have the following exception:
> > > >>
> > > >> // At least RuntimeException can be thrown by the code above when
> > > >> GridCacheContext is cleaned and there is
> > > >> // an attempt to use cleaned resources.
> > > >> U.error(log, "Unexpected exception during cache update", e);
> > > >>
> > > >> I mean, we genuinely don't know what happened here.
> > > >>
> > > >> Under new rules, what kind of "workaround" would that exception
> > suggest?
> > > >> "Try turning it off and then back on"?
> > > >> What explanation how to resolve this exception can we offer? "Please
> > > write
> > > >> to d...@apache.ignite.org or to Apache JIRA, and then wait for a
> > release
> > > >> with fix?"
> > > >>
> > > >> I'm really confused how we can implement 1.6 and 1.7 when dealing
> with
> > > >> messy real-world code.
> > > >>
> > > >> Regards,
> > > >>
> > > >>
> > > >> --
> > > >> Ilya Kasnacheev
> > > >>
> > > >> 2018-05-10 11:39 GMT+03:00 Vladimir Ozerov :
> > > >>
> > > >> > Andrey, Anton, Alex
> > > >> >
> > > >> > Agree, *SHOULD* is more appropriate here.
> > > >> >
> > > >> > Please see latest version below. Does anyone want to add or change
> > > >> > something? Let's wait for several days for more feedback and then
> > > >> publish
> > > >> > and announce this list. Note that it would not be carved in stone
> > and
> > > we
> > > >> > will be able to change it at any time if needed.
> > > >> >
> > > >> > 1) API
> > > >> > 1.1) API compatibility *MUST* be maintained between minor
> releases.
> > Do
> > > >> not
> > > >> > remove existing methods or change their signatures, deprecate them
> > > >> instead
> > > >> > 1.2) Default behaviour "SHOULD NOT* be changed between minor
> > releases,
> > > >> > unless absolutely needed. If change is made, it *MUST* be
> described
> > in
> > > >> > "Migration Guide"
> > > >> > 1.3) New operation *MUST* be well-documented in code (javadoc,
> > > >> dotnetdoc):
> > > >> > documentation must contain method's purpose, description of
> > parameters
> > > >> and
> > > >> > how their values affect the outcome, description of return value
> and
> > > >> it's
> > > >> > default, behavior in negative cases, interaction with other
> > operations
> > > >> and
> > > >> > components
> > > >> > 1.4) API parity between Java and .NET platforms *SHOULD* be
> > maintained
> > > >> when
> > > >> > operation makes sense on both platforms. If method cannot be
> > > >> implemented in
> > > >> > a platform immediately, new JIRA ticket *MUST* be 

Re: Ticket review checklist

2018-05-24 Thread Petr Ivanov
Why cannot we mute (and file corresponding tickets) all test failures 
(including flaky) to some date and start initiative Green TC?



> On 24 May 2018, at 15:04, Vladimir Ozerov  wrote:
> 
> Dmitry,
> 
> We cannot add this requirements, because we do have failures on TC. This
> requirement implies that all development would stop until TC is green.
> We never had old requirement work, neither we need to enforce it now.
> 
> On Thu, May 24, 2018 at 2:59 PM, Dmitry Pavlov 
> wrote:
> 
>> 3.c
>> 
>>   1. All test suites *MUST* be run on TeamCity [3] before merge to master,
>>   there *MUST NOT* be any test failures
>> 
>> 
>> 'New' word should be removed because we cant separate `new` and `non new`
>> failures.
>> 
>> Let's imagine example, we have 50 green runs in master. And PR Run-All
>> contains this test failed. Is it new or not new? Actually we don't know.
>> 
>> Existing requirement is about all TC must be green, so let's keep it as is.
>> 
>> ср, 23 мая 2018 г. в 17:02, Vladimir Ozerov :
>> 
>>> Igniters,
>>> 
>>> I created review checklist on WIKI [1] and also fixed related pages (e.g.
>>> "How To Contribute"). Please let me know if you have any comments before
>> I
>>> go with public announce.
>>> 
>>> Vladimir.
>>> 
>>> [1] https://cwiki.apache.org/confluence/display/IGNITE/Review+Checklist
>>> 
>>> On Thu, May 10, 2018 at 5:10 PM, Vladimir Ozerov 
>>> wrote:
>>> 
 Ilya,
 
 We define that exception messages *SHOULD* have clear explanation on
>> what
 is wrong. *SHOULD* mean that the rule should be followed unless there
>> is
>>> a
 reason not to follow. In your case you refer to some unexpected
>> behavior.
 I.e. an exceptional situation developer is not aware of. In this case
>> for
 sure we cannot force contributor to explain what is wrong, because,
>> well,
 we don't know. This is why we relaxed the rule from *MUST* to *SHOULD*.
 
 On Thu, May 10, 2018 at 3:50 PM, Ilya Kasnacheev <
 ilya.kasnach...@gmail.com> wrote:
 
> I don't think I quite understand how exception explanations should
>> work.
> 
> Imagine we have the following exception:
> 
> // At least RuntimeException can be thrown by the code above when
> GridCacheContext is cleaned and there is
> // an attempt to use cleaned resources.
> U.error(log, "Unexpected exception during cache update", e);
> 
> I mean, we genuinely don't know what happened here.
> 
> Under new rules, what kind of "workaround" would that exception
>> suggest?
> "Try turning it off and then back on"?
> What explanation how to resolve this exception can we offer? "Please
>>> write
> to d...@apache.ignite.org or to Apache JIRA, and then wait for a
>> release
> with fix?"
> 
> I'm really confused how we can implement 1.6 and 1.7 when dealing with
> messy real-world code.
> 
> Regards,
> 
> 
> --
> Ilya Kasnacheev
> 
> 2018-05-10 11:39 GMT+03:00 Vladimir Ozerov :
> 
>> Andrey, Anton, Alex
>> 
>> Agree, *SHOULD* is more appropriate here.
>> 
>> Please see latest version below. Does anyone want to add or change
>> something? Let's wait for several days for more feedback and then
> publish
>> and announce this list. Note that it would not be carved in stone
>> and
>>> we
>> will be able to change it at any time if needed.
>> 
>> 1) API
>> 1.1) API compatibility *MUST* be maintained between minor releases.
>> Do
> not
>> remove existing methods or change their signatures, deprecate them
> instead
>> 1.2) Default behaviour "SHOULD NOT* be changed between minor
>> releases,
>> unless absolutely needed. If change is made, it *MUST* be described
>> in
>> "Migration Guide"
>> 1.3) New operation *MUST* be well-documented in code (javadoc,
> dotnetdoc):
>> documentation must contain method's purpose, description of
>> parameters
> and
>> how their values affect the outcome, description of return value and
> it's
>> default, behavior in negative cases, interaction with other
>> operations
> and
>> components
>> 1.4) API parity between Java and .NET platforms *SHOULD* be
>> maintained
> when
>> operation makes sense on both platforms. If method cannot be
> implemented in
>> a platform immediately, new JIRA ticket *MUST* be created and linked
>>> to
>> current ticket
>> 1.5) API parity between thin clients (Java, .NET) *SHOULD* be
>>> maintained
>> when operation makes sense on several clients. If method cannot be
>> implemented in a client immediately, new JIRA ticket *MUST* be
>> created
> and
>> linked to current ticket
>> 1.6) All exceptions thrown to a user **SHOULD** have explanation how
>>> to
>> resolve, workaround or debug an error
>> 
>> 2) 

Re: Ticket review checklist

2018-05-24 Thread Vladimir Ozerov
Dmitry,

We cannot add this requirements, because we do have failures on TC. This
requirement implies that all development would stop until TC is green.
We never had old requirement work, neither we need to enforce it now.

On Thu, May 24, 2018 at 2:59 PM, Dmitry Pavlov 
wrote:

> 3.c
>
>1. All test suites *MUST* be run on TeamCity [3] before merge to master,
>there *MUST NOT* be any test failures
>
>
> 'New' word should be removed because we cant separate `new` and `non new`
> failures.
>
> Let's imagine example, we have 50 green runs in master. And PR Run-All
> contains this test failed. Is it new or not new? Actually we don't know.
>
> Existing requirement is about all TC must be green, so let's keep it as is.
>
> ср, 23 мая 2018 г. в 17:02, Vladimir Ozerov :
>
> > Igniters,
> >
> > I created review checklist on WIKI [1] and also fixed related pages (e.g.
> > "How To Contribute"). Please let me know if you have any comments before
> I
> > go with public announce.
> >
> > Vladimir.
> >
> > [1] https://cwiki.apache.org/confluence/display/IGNITE/Review+Checklist
> >
> > On Thu, May 10, 2018 at 5:10 PM, Vladimir Ozerov 
> > wrote:
> >
> > > Ilya,
> > >
> > > We define that exception messages *SHOULD* have clear explanation on
> what
> > > is wrong. *SHOULD* mean that the rule should be followed unless there
> is
> > a
> > > reason not to follow. In your case you refer to some unexpected
> behavior.
> > > I.e. an exceptional situation developer is not aware of. In this case
> for
> > > sure we cannot force contributor to explain what is wrong, because,
> well,
> > > we don't know. This is why we relaxed the rule from *MUST* to *SHOULD*.
> > >
> > > On Thu, May 10, 2018 at 3:50 PM, Ilya Kasnacheev <
> > > ilya.kasnach...@gmail.com> wrote:
> > >
> > >> I don't think I quite understand how exception explanations should
> work.
> > >>
> > >> Imagine we have the following exception:
> > >>
> > >> // At least RuntimeException can be thrown by the code above when
> > >> GridCacheContext is cleaned and there is
> > >> // an attempt to use cleaned resources.
> > >> U.error(log, "Unexpected exception during cache update", e);
> > >>
> > >> I mean, we genuinely don't know what happened here.
> > >>
> > >> Under new rules, what kind of "workaround" would that exception
> suggest?
> > >> "Try turning it off and then back on"?
> > >> What explanation how to resolve this exception can we offer? "Please
> > write
> > >> to d...@apache.ignite.org or to Apache JIRA, and then wait for a
> release
> > >> with fix?"
> > >>
> > >> I'm really confused how we can implement 1.6 and 1.7 when dealing with
> > >> messy real-world code.
> > >>
> > >> Regards,
> > >>
> > >>
> > >> --
> > >> Ilya Kasnacheev
> > >>
> > >> 2018-05-10 11:39 GMT+03:00 Vladimir Ozerov :
> > >>
> > >> > Andrey, Anton, Alex
> > >> >
> > >> > Agree, *SHOULD* is more appropriate here.
> > >> >
> > >> > Please see latest version below. Does anyone want to add or change
> > >> > something? Let's wait for several days for more feedback and then
> > >> publish
> > >> > and announce this list. Note that it would not be carved in stone
> and
> > we
> > >> > will be able to change it at any time if needed.
> > >> >
> > >> > 1) API
> > >> > 1.1) API compatibility *MUST* be maintained between minor releases.
> Do
> > >> not
> > >> > remove existing methods or change their signatures, deprecate them
> > >> instead
> > >> > 1.2) Default behaviour "SHOULD NOT* be changed between minor
> releases,
> > >> > unless absolutely needed. If change is made, it *MUST* be described
> in
> > >> > "Migration Guide"
> > >> > 1.3) New operation *MUST* be well-documented in code (javadoc,
> > >> dotnetdoc):
> > >> > documentation must contain method's purpose, description of
> parameters
> > >> and
> > >> > how their values affect the outcome, description of return value and
> > >> it's
> > >> > default, behavior in negative cases, interaction with other
> operations
> > >> and
> > >> > components
> > >> > 1.4) API parity between Java and .NET platforms *SHOULD* be
> maintained
> > >> when
> > >> > operation makes sense on both platforms. If method cannot be
> > >> implemented in
> > >> > a platform immediately, new JIRA ticket *MUST* be created and linked
> > to
> > >> > current ticket
> > >> > 1.5) API parity between thin clients (Java, .NET) *SHOULD* be
> > maintained
> > >> > when operation makes sense on several clients. If method cannot be
> > >> > implemented in a client immediately, new JIRA ticket *MUST* be
> created
> > >> and
> > >> > linked to current ticket
> > >> > 1.6) All exceptions thrown to a user **SHOULD** have explanation how
> > to
> > >> > resolve, workaround or debug an error
> > >> >
> > >> > 2) Compatibility
> > >> > 2.1) Persistence backward compatibility *MUST* be maintained between
> > >> minor
> > >> > releases. It should be possible to start newer version on data 

Re: Baseline topology documentation clarified: usage scenarios and definition

2018-05-24 Thread Dmitry Pavlov
Hi Ivan, sure, I will.

чт, 24 мая 2018 г. в 14:22, Ivan Rakov :

> Well, let's give it a try.
> Dmitry P., will you help me with organization of gotomeeting webinar and
> announcement? I think, I'll be ready to do it at the end of the next week.
>
> Best Regards,
> Ivan Rakov
>
> On 24.05.2018 14:15, Vyacheslav Daradur wrote:
> > +1 for GoToMeeting webinar.
> >
> > On Thu, May 24, 2018 at 2:09 PM, Dmitriy Govorukhin
> >  wrote:
> >> +1 for a webinar, I would like to see it.
> >>
> >> On Wed, May 23, 2018 at 5:34 PM, Dmitry Pavlov 
> >> wrote:
> >>
> >>> Hi Ivan,
> >>>
> >>> Would you like to run presentation based on this slides for Igniters?
> >>>
> >>> We can set up webinar using gotomeeting if you like this idea.
> >>>
> >>> Sincerely,
> >>> Dmitriy Pavlov
> >>>
> >>> ср, 23 мая 2018 г. в 0:19, Ivan Rakov :
> >>>
>  I used to present some slides introducing baseline topology concept.
>  Please feel free to use or change any pictures.
> 
>  https://docs.google.com/presentation/d/1DVLwlaR0zPIuL5YMkX8MyGLutZw-
> >>> tvK9InDTCvubAEQ/edit?usp=sharing
>  Best Regards,
>  Ivan Rakov
> 
>  On 18.05.2018 15:20, Dmitriy Setrakyan wrote:
> > Great docs! Any chance we could add some pictures to illustrate the
>  concept
> > better.
> >
> > On Fri, May 18, 2018 at 2:15 AM, Denis Magda 
> >>> wrote:
> >> Igniters,
> >>
> >> With the help of Stanislav Lukyanov and Ivan Rakov, we could make
> our
> >> baseline topology documentation much better and vivid. Check up the
> >>> new
> >> sections that did a better job explaining the topology, cover common
>  usage
> >> scenarious and explain how to trigger the rebalancing
> programmatically
>  if
> >> it's not expected that the baseline topology's node count to be
>  recovered
> >> soon:
> >>
> >>  - https://apacheignite.readme.io/v2.4/docs/cluster-
> >>  activation#section-baseline-topology
> >>  <
>  https://apacheignite.readme.io/v2.4/docs/cluster-
> >>> activation#section-baseline-topology
> >>  - https://apacheignite.readme.io/v2.4/docs/cluster-
> >>  activation#section-usage-scenarios
> >>  <
>  https://apacheignite.readme.io/v2.4/docs/cluster-
> >>> activation#section-usage-scenarios
> >>  - https://apacheignite.readme.io/v2.4/docs/cluster-
> >>  activation#section-triggering-rebalancing-programmatically
> >>  <
>  https://apacheignite.readme.io/v2.4/docs/cluster-
> >>> activation#section-triggering-rebalancing-programmatically
> >> --
> >> Denis
> >>
> 
> >
> >
>
>


Re: Ticket review checklist

2018-05-24 Thread Dmitry Pavlov
3.c

   1. All test suites *MUST* be run on TeamCity [3] before merge to master,
   there *MUST NOT* be any test failures


'New' word should be removed because we cant separate `new` and `non new`
failures.

Let's imagine example, we have 50 green runs in master. And PR Run-All
contains this test failed. Is it new or not new? Actually we don't know.

Existing requirement is about all TC must be green, so let's keep it as is.

ср, 23 мая 2018 г. в 17:02, Vladimir Ozerov :

> Igniters,
>
> I created review checklist on WIKI [1] and also fixed related pages (e.g.
> "How To Contribute"). Please let me know if you have any comments before I
> go with public announce.
>
> Vladimir.
>
> [1] https://cwiki.apache.org/confluence/display/IGNITE/Review+Checklist
>
> On Thu, May 10, 2018 at 5:10 PM, Vladimir Ozerov 
> wrote:
>
> > Ilya,
> >
> > We define that exception messages *SHOULD* have clear explanation on what
> > is wrong. *SHOULD* mean that the rule should be followed unless there is
> a
> > reason not to follow. In your case you refer to some unexpected behavior.
> > I.e. an exceptional situation developer is not aware of. In this case for
> > sure we cannot force contributor to explain what is wrong, because, well,
> > we don't know. This is why we relaxed the rule from *MUST* to *SHOULD*.
> >
> > On Thu, May 10, 2018 at 3:50 PM, Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com> wrote:
> >
> >> I don't think I quite understand how exception explanations should work.
> >>
> >> Imagine we have the following exception:
> >>
> >> // At least RuntimeException can be thrown by the code above when
> >> GridCacheContext is cleaned and there is
> >> // an attempt to use cleaned resources.
> >> U.error(log, "Unexpected exception during cache update", e);
> >>
> >> I mean, we genuinely don't know what happened here.
> >>
> >> Under new rules, what kind of "workaround" would that exception suggest?
> >> "Try turning it off and then back on"?
> >> What explanation how to resolve this exception can we offer? "Please
> write
> >> to d...@apache.ignite.org or to Apache JIRA, and then wait for a release
> >> with fix?"
> >>
> >> I'm really confused how we can implement 1.6 and 1.7 when dealing with
> >> messy real-world code.
> >>
> >> Regards,
> >>
> >>
> >> --
> >> Ilya Kasnacheev
> >>
> >> 2018-05-10 11:39 GMT+03:00 Vladimir Ozerov :
> >>
> >> > Andrey, Anton, Alex
> >> >
> >> > Agree, *SHOULD* is more appropriate here.
> >> >
> >> > Please see latest version below. Does anyone want to add or change
> >> > something? Let's wait for several days for more feedback and then
> >> publish
> >> > and announce this list. Note that it would not be carved in stone and
> we
> >> > will be able to change it at any time if needed.
> >> >
> >> > 1) API
> >> > 1.1) API compatibility *MUST* be maintained between minor releases. Do
> >> not
> >> > remove existing methods or change their signatures, deprecate them
> >> instead
> >> > 1.2) Default behaviour "SHOULD NOT* be changed between minor releases,
> >> > unless absolutely needed. If change is made, it *MUST* be described in
> >> > "Migration Guide"
> >> > 1.3) New operation *MUST* be well-documented in code (javadoc,
> >> dotnetdoc):
> >> > documentation must contain method's purpose, description of parameters
> >> and
> >> > how their values affect the outcome, description of return value and
> >> it's
> >> > default, behavior in negative cases, interaction with other operations
> >> and
> >> > components
> >> > 1.4) API parity between Java and .NET platforms *SHOULD* be maintained
> >> when
> >> > operation makes sense on both platforms. If method cannot be
> >> implemented in
> >> > a platform immediately, new JIRA ticket *MUST* be created and linked
> to
> >> > current ticket
> >> > 1.5) API parity between thin clients (Java, .NET) *SHOULD* be
> maintained
> >> > when operation makes sense on several clients. If method cannot be
> >> > implemented in a client immediately, new JIRA ticket *MUST* be created
> >> and
> >> > linked to current ticket
> >> > 1.6) All exceptions thrown to a user **SHOULD** have explanation how
> to
> >> > resolve, workaround or debug an error
> >> >
> >> > 2) Compatibility
> >> > 2.1) Persistence backward compatibility *MUST* be maintained between
> >> minor
> >> > releases. It should be possible to start newer version on data files
> >> > created by the previous version
> >> > 2.2) Thin client forward and backward compatibility *SHOULD* be
> >> maintained
> >> > between two consecutive minor releases. If compatibility cannot be
> >> > maintained it *MUST* be described in "Migration Guide"
> >> > 2.3) JDBC and ODBC forward and backward compatibility *SHOULD* be
> >> > maintained between two consecutive minor releases. If compatibility
> >> cannot
> >> > be maintained it *MUST* be described in "Migration Guide"
> >> >
> >> > 3) Tests
> >> > 3.1) New functionality *MUST* be covered with 

[GitHub] ignite pull request #3046: IGNITE-6612: Wrap ack methods in their own class

2018-05-24 Thread 1vanan
Github user 1vanan closed the pull request at:

https://github.com/apache/ignite/pull/3046


---


Re: Ticket review checklist

2018-05-24 Thread Vladimir Ozerov
Dmitry,

Could you please formulate this requirement?

On Wed, May 23, 2018 at 5:06 PM, Dmitry Pavlov 
wrote:

> Hi Vladimir,
>
> I've replied in separate thread.
>
> I would like to keep requirement of Green TC instead of separation to
> new/old test failures.
>
> Once you allow just one test failure, it would be more failures after some
> time.
>
> Sincererly,
> Dmitriy Pavlov
>
> ср, 23 мая 2018 г. в 17:02, Vladimir Ozerov :
>
> > Igniters,
> >
> > I created review checklist on WIKI [1] and also fixed related pages (e.g.
> > "How To Contribute"). Please let me know if you have any comments before
> I
> > go with public announce.
> >
> > Vladimir.
> >
> > [1] https://cwiki.apache.org/confluence/display/IGNITE/Review+Checklist
> >
> > On Thu, May 10, 2018 at 5:10 PM, Vladimir Ozerov 
> > wrote:
> >
> > > Ilya,
> > >
> > > We define that exception messages *SHOULD* have clear explanation on
> what
> > > is wrong. *SHOULD* mean that the rule should be followed unless there
> is
> > a
> > > reason not to follow. In your case you refer to some unexpected
> behavior.
> > > I.e. an exceptional situation developer is not aware of. In this case
> for
> > > sure we cannot force contributor to explain what is wrong, because,
> well,
> > > we don't know. This is why we relaxed the rule from *MUST* to *SHOULD*.
> > >
> > > On Thu, May 10, 2018 at 3:50 PM, Ilya Kasnacheev <
> > > ilya.kasnach...@gmail.com> wrote:
> > >
> > >> I don't think I quite understand how exception explanations should
> work.
> > >>
> > >> Imagine we have the following exception:
> > >>
> > >> // At least RuntimeException can be thrown by the code above when
> > >> GridCacheContext is cleaned and there is
> > >> // an attempt to use cleaned resources.
> > >> U.error(log, "Unexpected exception during cache update", e);
> > >>
> > >> I mean, we genuinely don't know what happened here.
> > >>
> > >> Under new rules, what kind of "workaround" would that exception
> suggest?
> > >> "Try turning it off and then back on"?
> > >> What explanation how to resolve this exception can we offer? "Please
> > write
> > >> to d...@apache.ignite.org or to Apache JIRA, and then wait for a
> release
> > >> with fix?"
> > >>
> > >> I'm really confused how we can implement 1.6 and 1.7 when dealing with
> > >> messy real-world code.
> > >>
> > >> Regards,
> > >>
> > >>
> > >> --
> > >> Ilya Kasnacheev
> > >>
> > >> 2018-05-10 11:39 GMT+03:00 Vladimir Ozerov :
> > >>
> > >> > Andrey, Anton, Alex
> > >> >
> > >> > Agree, *SHOULD* is more appropriate here.
> > >> >
> > >> > Please see latest version below. Does anyone want to add or change
> > >> > something? Let's wait for several days for more feedback and then
> > >> publish
> > >> > and announce this list. Note that it would not be carved in stone
> and
> > we
> > >> > will be able to change it at any time if needed.
> > >> >
> > >> > 1) API
> > >> > 1.1) API compatibility *MUST* be maintained between minor releases.
> Do
> > >> not
> > >> > remove existing methods or change their signatures, deprecate them
> > >> instead
> > >> > 1.2) Default behaviour "SHOULD NOT* be changed between minor
> releases,
> > >> > unless absolutely needed. If change is made, it *MUST* be described
> in
> > >> > "Migration Guide"
> > >> > 1.3) New operation *MUST* be well-documented in code (javadoc,
> > >> dotnetdoc):
> > >> > documentation must contain method's purpose, description of
> parameters
> > >> and
> > >> > how their values affect the outcome, description of return value and
> > >> it's
> > >> > default, behavior in negative cases, interaction with other
> operations
> > >> and
> > >> > components
> > >> > 1.4) API parity between Java and .NET platforms *SHOULD* be
> maintained
> > >> when
> > >> > operation makes sense on both platforms. If method cannot be
> > >> implemented in
> > >> > a platform immediately, new JIRA ticket *MUST* be created and linked
> > to
> > >> > current ticket
> > >> > 1.5) API parity between thin clients (Java, .NET) *SHOULD* be
> > maintained
> > >> > when operation makes sense on several clients. If method cannot be
> > >> > implemented in a client immediately, new JIRA ticket *MUST* be
> created
> > >> and
> > >> > linked to current ticket
> > >> > 1.6) All exceptions thrown to a user **SHOULD** have explanation how
> > to
> > >> > resolve, workaround or debug an error
> > >> >
> > >> > 2) Compatibility
> > >> > 2.1) Persistence backward compatibility *MUST* be maintained between
> > >> minor
> > >> > releases. It should be possible to start newer version on data files
> > >> > created by the previous version
> > >> > 2.2) Thin client forward and backward compatibility *SHOULD* be
> > >> maintained
> > >> > between two consecutive minor releases. If compatibility cannot be
> > >> > maintained it *MUST* be described in "Migration Guide"
> > >> > 2.3) JDBC and ODBC forward and backward compatibility *SHOULD* 

Re: Baseline topology documentation clarified: usage scenarios and definition

2018-05-24 Thread Ivan Rakov

Well, let's give it a try.
Dmitry P., will you help me with organization of gotomeeting webinar and 
announcement? I think, I'll be ready to do it at the end of the next week.


Best Regards,
Ivan Rakov

On 24.05.2018 14:15, Vyacheslav Daradur wrote:

+1 for GoToMeeting webinar.

On Thu, May 24, 2018 at 2:09 PM, Dmitriy Govorukhin
 wrote:

+1 for a webinar, I would like to see it.

On Wed, May 23, 2018 at 5:34 PM, Dmitry Pavlov 
wrote:


Hi Ivan,

Would you like to run presentation based on this slides for Igniters?

We can set up webinar using gotomeeting if you like this idea.

Sincerely,
Dmitriy Pavlov

ср, 23 мая 2018 г. в 0:19, Ivan Rakov :


I used to present some slides introducing baseline topology concept.
Please feel free to use or change any pictures.

https://docs.google.com/presentation/d/1DVLwlaR0zPIuL5YMkX8MyGLutZw-

tvK9InDTCvubAEQ/edit?usp=sharing

Best Regards,
Ivan Rakov

On 18.05.2018 15:20, Dmitriy Setrakyan wrote:

Great docs! Any chance we could add some pictures to illustrate the

concept

better.

On Fri, May 18, 2018 at 2:15 AM, Denis Magda 

wrote:

Igniters,

With the help of Stanislav Lukyanov and Ivan Rakov, we could make our
baseline topology documentation much better and vivid. Check up the

new

sections that did a better job explaining the topology, cover common

usage

scenarious and explain how to trigger the rebalancing programmatically

if

it's not expected that the baseline topology's node count to be

recovered

soon:

 - https://apacheignite.readme.io/v2.4/docs/cluster-
 activation#section-baseline-topology
 <

https://apacheignite.readme.io/v2.4/docs/cluster-

activation#section-baseline-topology

 - https://apacheignite.readme.io/v2.4/docs/cluster-
 activation#section-usage-scenarios
 <

https://apacheignite.readme.io/v2.4/docs/cluster-

activation#section-usage-scenarios

 - https://apacheignite.readme.io/v2.4/docs/cluster-
 activation#section-triggering-rebalancing-programmatically
 <

https://apacheignite.readme.io/v2.4/docs/cluster-

activation#section-triggering-rebalancing-programmatically

--
Denis










Re: Baseline topology documentation clarified: usage scenarios and definition

2018-05-24 Thread Vyacheslav Daradur
+1 for GoToMeeting webinar.

On Thu, May 24, 2018 at 2:09 PM, Dmitriy Govorukhin
 wrote:
> +1 for a webinar, I would like to see it.
>
> On Wed, May 23, 2018 at 5:34 PM, Dmitry Pavlov 
> wrote:
>
>> Hi Ivan,
>>
>> Would you like to run presentation based on this slides for Igniters?
>>
>> We can set up webinar using gotomeeting if you like this idea.
>>
>> Sincerely,
>> Dmitriy Pavlov
>>
>> ср, 23 мая 2018 г. в 0:19, Ivan Rakov :
>>
>> > I used to present some slides introducing baseline topology concept.
>> > Please feel free to use or change any pictures.
>> >
>> > https://docs.google.com/presentation/d/1DVLwlaR0zPIuL5YMkX8MyGLutZw-
>> tvK9InDTCvubAEQ/edit?usp=sharing
>> >
>> > Best Regards,
>> > Ivan Rakov
>> >
>> > On 18.05.2018 15:20, Dmitriy Setrakyan wrote:
>> > > Great docs! Any chance we could add some pictures to illustrate the
>> > concept
>> > > better.
>> > >
>> > > On Fri, May 18, 2018 at 2:15 AM, Denis Magda 
>> wrote:
>> > >
>> > >> Igniters,
>> > >>
>> > >> With the help of Stanislav Lukyanov and Ivan Rakov, we could make our
>> > >> baseline topology documentation much better and vivid. Check up the
>> new
>> > >> sections that did a better job explaining the topology, cover common
>> > usage
>> > >> scenarious and explain how to trigger the rebalancing programmatically
>> > if
>> > >> it's not expected that the baseline topology's node count to be
>> > recovered
>> > >> soon:
>> > >>
>> > >> - https://apacheignite.readme.io/v2.4/docs/cluster-
>> > >> activation#section-baseline-topology
>> > >> <
>> > https://apacheignite.readme.io/v2.4/docs/cluster-
>> activation#section-baseline-topology
>> > >
>> > >> - https://apacheignite.readme.io/v2.4/docs/cluster-
>> > >> activation#section-usage-scenarios
>> > >> <
>> > https://apacheignite.readme.io/v2.4/docs/cluster-
>> activation#section-usage-scenarios
>> > >
>> > >> - https://apacheignite.readme.io/v2.4/docs/cluster-
>> > >> activation#section-triggering-rebalancing-programmatically
>> > >> <
>> > https://apacheignite.readme.io/v2.4/docs/cluster-
>> activation#section-triggering-rebalancing-programmatically
>> > >
>> > >>
>> > >> --
>> > >> Denis
>> > >>
>> >
>> >
>>



-- 
Best Regards, Vyacheslav D.


[jira] [Created] (IGNITE-8600) SQL: lazy row materialization

2018-05-24 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-8600:
---

 Summary: SQL: lazy row materialization
 Key: IGNITE-8600
 URL: https://issues.apache.org/jira/browse/IGNITE-8600
 Project: Ignite
  Issue Type: Task
  Components: sql
Affects Versions: 2.5
 Environment: Currently our index cursor materializes rows as soon as 
they are encountered in an index page. This is necessary to protect ourselves 
from concurrent data modification. However, materialized rows might be filtered 
out later due to additional filters. In addition, there is a chance that only 
indexed fields is needed by query.

We can do the following:
1) Introduce new mode that will return partially materialized rows, with only 
inline index fields initialized. When some non-initialized attribute is 
requested, we go to data page and materialize the whole row
2) Enable this mode for MVCC by default
3) Optionally enable this mode for non-MVCC read-only mode through additional 
flag.
Reporter: Vladimir Ozerov






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Cache 6 Suite Execution timeouts

2018-05-24 Thread Dmitry Pavlov
Alexey? Could I hope you will pick up
https://issues.apache.org/jira/browse/IGNITE-8509?

чт, 24 мая 2018 г. в 14:00, Maxim Muzafarov :

> Hello,
>
> Just bump this topic up.
>
> 6/10 execution timeouts for the last 10 build in master!
>
> Will anyone help?
>
> [1]
>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Cache6=1=buildTypeHistoryList_IgniteTests24Java8=%3Cdefault%3E
>
> пн, 21 мая 2018 г. в 12:58, Dmitry Pavlov :
>
> > Hi Maxim,
> >
> > I observe the same, Cache 6 critical recent fails: 18,0% [9 fails / 50
> > runs]
> >
> > I hope Aleksei Scherbakov can help here. Alexey, please step in.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > вс, 20 мая 2018 г. в 20:37, Maxim Muzafarov :
> >
> > > Hello Igniters,
> > >
> > > Have anyone noticed a lot of execution timeouts for Cache 6 Suite?
> > > Since recently it fails too much on TxRollbackOnTimeoutNearCacheTest
> with
> > > "Found long running transaction".
> > >
> > > Please refer to TC [1]. It has 4/10 execution timeouts for the last 10
> > > builds in master.
> > >
> > > I've created IGNITE-8509 [2]. Will someone have time to look at? Or
> I'll
> > > try when I'm get free.
> > >
> > > [1]
> > >
> > >
> >
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Cache6_IgniteTests24Java8=%3Cdefault%3E=buildTypeStatusDiv
> > > [2] https://issues.apache.org/jira/browse/IGNITE-8509
> > >
> >
>


Re: IGNITE-8583 - review needed

2018-05-24 Thread Dmitry Pavlov
Hi Dmitriy,

According to upsource
https://reviews.ignite.apache.org/ignite/review/IGNT-CR-623 review was
already started by Alexey G. and  Alexey K.

Sincerely,
Dmitriy Pavlov



чт, 24 мая 2018 г. в 14:04, Dmitriy Govorukhin :

> Igniters,
>
> I fixed DataStorageMetricsMXBean.getOffHeapSize, please review my changes.
>
> https://issues.apache.org/jira/browse/IGNITE-8583
>


Re: Baseline topology documentation clarified: usage scenarios and definition

2018-05-24 Thread Dmitriy Govorukhin
+1 for a webinar, I would like to see it.

On Wed, May 23, 2018 at 5:34 PM, Dmitry Pavlov 
wrote:

> Hi Ivan,
>
> Would you like to run presentation based on this slides for Igniters?
>
> We can set up webinar using gotomeeting if you like this idea.
>
> Sincerely,
> Dmitriy Pavlov
>
> ср, 23 мая 2018 г. в 0:19, Ivan Rakov :
>
> > I used to present some slides introducing baseline topology concept.
> > Please feel free to use or change any pictures.
> >
> > https://docs.google.com/presentation/d/1DVLwlaR0zPIuL5YMkX8MyGLutZw-
> tvK9InDTCvubAEQ/edit?usp=sharing
> >
> > Best Regards,
> > Ivan Rakov
> >
> > On 18.05.2018 15:20, Dmitriy Setrakyan wrote:
> > > Great docs! Any chance we could add some pictures to illustrate the
> > concept
> > > better.
> > >
> > > On Fri, May 18, 2018 at 2:15 AM, Denis Magda 
> wrote:
> > >
> > >> Igniters,
> > >>
> > >> With the help of Stanislav Lukyanov and Ivan Rakov, we could make our
> > >> baseline topology documentation much better and vivid. Check up the
> new
> > >> sections that did a better job explaining the topology, cover common
> > usage
> > >> scenarious and explain how to trigger the rebalancing programmatically
> > if
> > >> it's not expected that the baseline topology's node count to be
> > recovered
> > >> soon:
> > >>
> > >> - https://apacheignite.readme.io/v2.4/docs/cluster-
> > >> activation#section-baseline-topology
> > >> <
> > https://apacheignite.readme.io/v2.4/docs/cluster-
> activation#section-baseline-topology
> > >
> > >> - https://apacheignite.readme.io/v2.4/docs/cluster-
> > >> activation#section-usage-scenarios
> > >> <
> > https://apacheignite.readme.io/v2.4/docs/cluster-
> activation#section-usage-scenarios
> > >
> > >> - https://apacheignite.readme.io/v2.4/docs/cluster-
> > >> activation#section-triggering-rebalancing-programmatically
> > >> <
> > https://apacheignite.readme.io/v2.4/docs/cluster-
> activation#section-triggering-rebalancing-programmatically
> > >
> > >>
> > >> --
> > >> Denis
> > >>
> >
> >
>


[jira] [Created] (IGNITE-8599) Remove LocalWalModeChangeDuringRebalancingSelfTest.testWithExchangesMerge from Direct IO suite

2018-05-24 Thread Dmitriy Pavlov (JIRA)
Dmitriy Pavlov created IGNITE-8599:
--

 Summary: Remove  
LocalWalModeChangeDuringRebalancingSelfTest.testWithExchangesMerge from Direct 
IO suite
 Key: IGNITE-8599
 URL: https://issues.apache.org/jira/browse/IGNITE-8599
 Project: Ignite
  Issue Type: Improvement
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov
 Fix For: 2.6


https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=6346758468206865681=%3Cdefault%3E=testDetails

It falls only in Direct IO

It is necessary to exclude it from direct IO if it gives a lot of load.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


IGNITE-8583 - review needed

2018-05-24 Thread Dmitriy Govorukhin
Igniters,

I fixed DataStorageMetricsMXBean.getOffHeapSize, please review my changes.

https://issues.apache.org/jira/browse/IGNITE-8583


Re: Cache 6 Suite Execution timeouts

2018-05-24 Thread Maxim Muzafarov
Hello,

Just bump this topic up.

6/10 execution timeouts for the last 10 build in master!

Will anyone help?

[1]
https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Cache6=1=buildTypeHistoryList_IgniteTests24Java8=%3Cdefault%3E

пн, 21 мая 2018 г. в 12:58, Dmitry Pavlov :

> Hi Maxim,
>
> I observe the same, Cache 6 critical recent fails: 18,0% [9 fails / 50
> runs]
>
> I hope Aleksei Scherbakov can help here. Alexey, please step in.
>
> Sincerely,
> Dmitriy Pavlov
>
> вс, 20 мая 2018 г. в 20:37, Maxim Muzafarov :
>
> > Hello Igniters,
> >
> > Have anyone noticed a lot of execution timeouts for Cache 6 Suite?
> > Since recently it fails too much on TxRollbackOnTimeoutNearCacheTest with
> > "Found long running transaction".
> >
> > Please refer to TC [1]. It has 4/10 execution timeouts for the last 10
> > builds in master.
> >
> > I've created IGNITE-8509 [2]. Will someone have time to look at? Or I'll
> > try when I'm get free.
> >
> > [1]
> >
> >
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Cache6_IgniteTests24Java8=%3Cdefault%3E=buildTypeStatusDiv
> > [2] https://issues.apache.org/jira/browse/IGNITE-8509
> >
>


Re: async operation is not fair async

2018-05-24 Thread Dmitriy Govorukhin
So, in a current implementation, how I can perform the real async operation
in one thread? Any workaround?
What can I do if I have event loop thread model?

On Wed, May 16, 2018 at 12:14 PM, Alexey Goncharuk <
alexey.goncha...@gmail.com> wrote:

> Dmitriy,
>
> I will add technical details to the ticket, however, looks like there is
> still no consensus on how this change should be presented to a user. It
> would be ok if we changed this behavior in Ignite 3.0, but for one of the
> next point releases we have to agree how this should be enabled/disabled
> (or whether we should delay this change to 3.0 at all).
>
> 2018-05-15 22:13 GMT+03:00 Dmitriy Govorukhin <
> dmitriy.govoruk...@gmail.com>
> :
>
> >  Alexey,
> >
> > Any updates?
> >
> > On Mon, May 14, 2018 at 6:19 PM, Dmitriy Govorukhin <
> > dmitriy.govoruk...@gmail.com> wrote:
> >
> > > Alexey,
> > >
> > > Could you please add more description information for this task? [1]
> > > Perhaps, base steps for implementation.
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-8475
> > >
> > > On Mon, May 14, 2018 at 4:58 PM, Alexey Goncharuk <
> > > alexey.goncha...@gmail.com> wrote:
> > >
> > >> Another +1 for the true asynchronous approach. I remember a while ago
> > one
> > >> of the Ignite users raised a similar question regarding the *async
> > method
> > >> being blocked on establishing a TCP connection.
> > >>
> > >> As far as deadlocks go, I have a counter-example. Currently, we check
> > the
> > >> thread-local chain only for a single cache, so if I run the following
> > >> code:
> > >> cache1.getAsync(k1);
> > >> cache2.getAsync(k2);
> > >> then the deadlock is still possible, and I did not see a single user
> > >> complaining about unexpected deadlocks. Rather than implementing this
> > >> cross-cache chain (which would probably add another overhead), I would
> > >> make
> > >> it consistent and allow operations to be run in parallel.
> > >>
> > >> There are many use-cases when having true async operations
> dramatically
> > >> improve performance. Consider, for example, a streaming example when
> > keys
> > >> are being pushed by a client to a cluster. Currently, to run effective
> > >> processing, the user will have to use a data streamer with custom keys
> > >> receiver which may be a huge usability downside. Async operations can
> > >> utilize the cluster resources very efficiently.
> > >>
> > >> Finally, if we want to be on the safe side, we can keep the operation
> > >> chain
> > >> inside a transaction. I see absolutely no point in maintaining this
> > chain
> > >> outside of transactions.
> > >>
> > >> --AG
> > >>
> > >> 2018-05-14 16:01 GMT+03:00 Dmitriy Govorukhin <
> > >> dmitriy.govoruk...@gmail.com>
> > >> :
> > >>
> > >> > Andrey,
> > >> >
> > >> > Do you prefer change behavior at runtime?
> > >> > I guess will be better have different methods for getting cache
> > instance
> > >> > with fair and not fair sync.
> > >> >
> > >> > On Mon, May 14, 2018 at 3:39 PM, Andrey Gura 
> > wrote:
> > >> >
> > >> > > +1 for fair async operations.
> > >> > >
> > >> > > But I don't like idea use withFairSync() method. We added
> xxxAsync()
> > >> > > methods recently and withAsync() is deprecated.
> > >> > >
> > >> > > I think we should just make methods are async in nature and
> provide
> > >> > > ability of switching to the old behaviour using flag or property.
> > >> > >
> > >> > > On Fri, May 11, 2018 at 11:00 PM, Dmitriy Setrakyan
> > >> > >  wrote:
> > >> > > > Vladimir,
> > >> > > >
> > >> > > > In general I agree, but I do get greatly *close-minded* (pun
> > >> intended)
> > >> > > > whenever users' code that worked for the past several years all
> > of a
> > >> > > sudden
> > >> > > > gets deadlocked after an upgrade. Making this feature optional
> is
> > >> even
> > >> > > > worse and more confusing. In this case the best action is no
> > action
> > >> at
> > >> > > all.
> > >> > > >
> > >> > > > BTW, would be interesting to find out how Oracle async driver
> > >> behaves
> > >> > in
> > >> > > > this case.
> > >> > > >
> > >> > > > D.
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On Fri, May 11, 2018 at 8:29 PM, Vladimir Ozerov <
> > >> voze...@gridgain.com
> > >> > >
> > >> > > > wrote:
> > >> > > >
> > >> > > >> Guys,
> > >> > > >>
> > >> > > >> To build a great product we should be open minded and look to
> the
> > >> > > future,
> > >> > > >> not to the past.
> > >> > > >>
> > >> > > >> Dima raised very valid point - why async is not async? Current
> > >> > > programming
> > >> > > >> culture and demanding performance requirements pushes users
> > towards
> > >> > > >> reactive-style programming. I do not want my thread to ever be
> > >> > blocked.
> > >> > > >> Instead, I want to send a number of concurrent commands and
> > >> optionally
> > >> > > >> subscribe to final result. So trully async API makes total
> sense
> > to
> > >> > me.
> > >> > > >>

Re: IEP-4, Phase 2. Using BL(A)T for in-memory caches.

2018-05-24 Thread Anton Vinogradov
BLAT sounds good to me because it slightly funny in case you speak russian.
"Node joined topology by BLAT (po blaty)" sounds reasonable :)

чт, 24 мая 2018 г. в 1:15, Dmitriy Setrakyan :

> Ed,
>
> Anyone speaking Russian would agree that BLAT is not a good name :) Let's
> steak to BLT.
>
> D.
>
> On Wed, May 23, 2018 at 5:34 AM, Eduard Shangareev <
> eduard.shangar...@gmail.com> wrote:
>
> > Igniters,
> >
> > We have invested too much in explaining BLAT. So, it would hard to change
> > the name.
> > I.e. I propose to save this term.
> >
> >
> > New names for auto-adjust control.
> >
> > 1. org.apache.ignite.IgniteCluster
> >
> > *Add*
> > isBaselineAutoAdjustEnabled()
> > setBaselineAutoAdjustEnabled(boolean enabled);
> > setBaselineAutoAdjustTimeout(long timeoutInMs);
> > setBaselineAutoAdjustMaxTimeout(long timeoutInMs);
> >
> > 2. org.apache.ignite.configuration.IgniteConfiguration
> >
> > *Add*
> > IgniteConfiguration setBaselineAutoAdjustEnabled(boolean enabled);
> > IgniteConfiguration setBaselineAutoAdjustTimeout(long timeoutInMs);
> > IgniteConfiguration setBaselineAutoAdjustMaxTimeout(long timeoutInMs);
> >
> > Any objections?
> >
> >
> >
> > On Fri, May 4, 2018 at 10:01 PM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> > wrote:
> >
> > > I do not like the name "current" on the methods. I think we should just
> > > remove it, e.g. currentAffinityTopology() -> affinityTopology()
> > >
> > > D.
> > >
> > > On Fri, May 4, 2018 at 6:17 AM, Eduard Shangareev <
> > > eduard.shangar...@gmail.com> wrote:
> > >
> > > > Igniters,
> > > >
> > > > With Vladimir's help, we analyzed another solution's approaches.
> > > > And decided to simplify our affinity topology auto-adjusting.
> > > >
> > > > It should be enough to be able to turn on/off auto-adjusting (flag)
> and
> > > set
> > > > 2 timeouts if it is working:
> > > > -soft timeout which would be used if there was no other node
> > joins/exits;
> > > > -hard timeout which we would track from first discovery event and if
> it
> > > > reached then immediately would change affinity topology.
> > > >
> > > > All other strategies could be realized with API usage
> > > (setAffinityTopology)
> > > > and metrics tracking by user's monitoring tools.
> > > >
> > > > So, I suggest next API changes:
> > > >
> > > > org.apache.ignite.IgniteCluster
> > > >
> > > > *Deprecate*:
> > > > Collection currentBaselineTopology();
> > > > void setBaselineTopology(Collection
> > > baselineTop);
> > > > void setBaselineTopology(long topVer);
> > > >
> > > > *Replace them with*
> > > > Collection currentAffinityTopology();
> > > > void setAffinityTopology(Collection
> > > affinityTop);
> > > > void setAffinityTopology(long topVer);
> > > >
> > > > *Add*
> > > > isAffinityTopologyAutoAdjustEnabled()
> > > > setAffinityTopologyAutoAdjustEnabled(boolean enabled);
> > > >
> > > > org.apache.ignite.configuration.IgniteConfiguration
> > > >
> > > > *Add*
> > > > IgniteConfiguration setAffinityTopologyAutoAdjustEnabled(boolean
> > > enabled);
> > > > IgniteConfiguration setAffinityTopologyAutoAdjustTimeout(long
> > > > timeoutInMs);
> > > > IgniteConfiguration setAffinityTopologyAutoAdjustMaxTimeout(long
> > > > timeoutInMs);
> > > >
> > > >
> > > > An open question is could we rename or duplicate BaselineNode with
> > > > AffinityNode?
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Fri, Apr 27, 2018 at 6:56 PM, Ivan Rakov 
> > > wrote:
> > > >
> > > > > Eduard,
> > > > >
> > > > > +1 to your proposed API for configuring Affinity Topology change
> > > > policies.
> > > > > Obviously we should use "auto" as default behavior. I believe,
> > > automatic
> > > > > rebalancing is expected and more convenient for majority of users.
> > > > >
> > > > > Best Regards,
> > > > > Ivan Rakov
> > > > >
> > > > >
> > > > > On 26.04.2018 19:27, Eduard Shangareev wrote:
> > > > >
> > > > >> Igniters,
> > > > >>
> > > > >> Ok, I want to share my thoughts about "affinity topology (AT)
> > changing
> > > > >> policies".
> > > > >>
> > > > >>
> > > > >> There would be three major option:
> > > > >> -auto;
> > > > >> -manual;
> > > > >> -custom.
> > > > >>
> > > > >> 1. Automatic change.
> > > > >> A user could set timeouts for:
> > > > >> a. change AT on any topology change after some timeout
> > > > (setATChangeTimeout
> > > > >> in seconds);
> > > > >> b. change AT on node left after some timeout
> > > > (setATChangeOnNodeLeftTimeout
> > > > >> in seconds);
> > > > >> c. change AT on node join after some timeout
> > > > (setATChangeOnNodeJoinTimeout
> > > > >> in seconds).
> > > > >>
> > > > >> b and c are more specific, so they would override a.
> > > > >>
> > > > >> Also, I want to introduce a mechanism of merging AT changes, which
> > > would
> > > > >> be
> > > > >> turned on by default.
> > > > >> Other words, if we reached timeout than we would change AT to
> > current
> > > > >> topology, not that one which was on timeout schedule.
> > > 

[jira] [Created] (IGNITE-8598) SQL: ability to control partition pruning with explicit affinity column definition

2018-05-24 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-8598:
---

 Summary: SQL: ability to control partition pruning with explicit 
affinity column definition
 Key: IGNITE-8598
 URL: https://issues.apache.org/jira/browse/IGNITE-8598
 Project: Ignite
  Issue Type: Task
  Components: sql
Reporter: Vladimir Ozerov


Affinity functions are applicable to keys only. Sometimes users may have 
complex affinity calculation logic, so that partition pruning optimization is 
not applicable. E.g. this custom {{AffinityKeyMapper}}. However, there is a 
chance that partition could be calculated from some attribute of {{value}}. 

It would be nice to force our engine to treat some attribute as affinity key 
even though it is not marked as {{AffinityKeyMapped}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: [jira] [Created] (IGNITE-8595) SQL: Ability to cancel DDL operations

2018-05-24 Thread Vladimir Ozerov
Hi Ray,

We are working on it, All operations will be cancelled in case of either
explicit user request, or when user session is terminated.

On Thu, May 24, 2018 at 12:51 PM, Ray  wrote:

> Hello Vladimir.
>
> From the https://apacheignite-sql.readme.io/docs/query-cancellation, it is
> only possible to cancel select query in java API.
> So select query submitted from thin client(sqlline)/jdbc/odbc is not
> cancellable, is my understanding correct?
>
>
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>


[jira] [Created] (IGNITE-8597) Direct data load

2018-05-24 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-8597:
---

 Summary: Direct data load
 Key: IGNITE-8597
 URL: https://issues.apache.org/jira/browse/IGNITE-8597
 Project: Ignite
  Issue Type: Task
  Components: general
Reporter: Vladimir Ozerov


We should implement optimized data loading mode, which will bypass as much 
internal Ignite components as possible to improve data loading speed.

Raw design:
1) Direct data load must be performed in exclusive mode, i.e. nobody else are 
allowed to read or write to specific cache/table at this time. I.e. we need to 
implement distributed table/cache locks. 
2) At first we should write to data pages directly skipping free lists, PK hash 
index and secondary indexes
3) Once loading is finished, we should rebuild free lists and indexes 
(bottom-up); external merge implementation will be required
4) We should distinguish between initial load and incremental load. The latter 
would require merge of indexes with previously available data.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: [jira] [Created] (IGNITE-8595) SQL: Ability to cancel DDL operations

2018-05-24 Thread Ray
Hello Vladimir.

>From the https://apacheignite-sql.readme.io/docs/query-cancellation, it is
only possible to cancel select query in java API.
So select query submitted from thin client(sqlline)/jdbc/odbc is not
cancellable, is my understanding correct?





--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/


[jira] [Created] (IGNITE-8596) SQL: remove unnecessary index lookups when query parallelism is enabled

2018-05-24 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-8596:
---

 Summary: SQL: remove unnecessary index lookups when query 
parallelism is enabled
 Key: IGNITE-8596
 URL: https://issues.apache.org/jira/browse/IGNITE-8596
 Project: Ignite
  Issue Type: Task
  Components: sql
Affects Versions: 2.5
Reporter: Vladimir Ozerov
 Fix For: 2.6


See 
{{org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor#onQueryRequest}}
 method. If table is segmented, we will submit as many SQL requests as much 
segments. But consider a case when target cache partition(s) is already defined 
by user or derived through partition pruning. In this case most of segments 
will not contain useful information and return empty result set. At the same 
time these queries may impose index or data page scans, thus consuming 
resources without a reason.

To mitigate the problem we should not submit SQL requests to segments we are 
not interested in.

Note that it is not sufficient to simply skip SQL requests on mapper, because 
reducer expects separate response for every message. We should fix both local 
mapper logic as well as protocol.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-8595) SQL: Ability to cancel DDL operations

2018-05-24 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-8595:
---

 Summary: SQL: Ability to cancel DDL operations
 Key: IGNITE-8595
 URL: https://issues.apache.org/jira/browse/IGNITE-8595
 Project: Ignite
  Issue Type: Task
  Components: sql
Reporter: Vladimir Ozerov
 Fix For: 2.6


At the moment it is impossible to cancel DDL operations. Suppose that you 
started {{CREATE INDEX}} on very heavy table. It took hours to finish, and you 
would like to stop it because it is 9:00AM, and users are about to execute many 
queries and {{CREATE INDEX}} will slow them down or even block.

It should be possible to stop DDL operations in the same way it is done for 
{{SELECT}} queries.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-8594) Make error messages in validate_indexes command report more informative

2018-05-24 Thread Ivan Rakov (JIRA)
Ivan Rakov created IGNITE-8594:
--

 Summary: Make error messages in validate_indexes command report 
more informative
 Key: IGNITE-8594
 URL: https://issues.apache.org/jira/browse/IGNITE-8594
 Project: Ignite
  Issue Type: Improvement
Reporter: Ivan Rakov
Assignee: Ivan Rakov
 Fix For: 2.6


In case index is broken and contains links to missing items in data pages, 
validate_indexes command will show "Item not found" messages in report:
{noformat}
IndexValidationIssue [key=null, cacheName=cache_group_1_028, idxName=_key_PK], 
class java.lang.IllegalStateException: Item not found: 65
IndexValidationIssue [key=null, cacheName=cache_group_1_028, idxName=_key_PK], 
class java.lang.IllegalStateException: Item not found: 15
SQL Index [cache=cache_group_1_028, idx=LONG__VAL_IDX] 
ValidateIndexesPartitionResult [consistentId=node2, sqlIdxName=LONG__VAL_IDX]
IndexValidationIssue [key=null, cacheName=cache_group_1_028, 
idxName=LONG__VAL_IDX], class java.lang.IllegalStateException: Item not found: 
60
IndexValidationIssue [key=null, cacheName=cache_group_1_028, 
idxName=LONG__VAL_IDX], class java.lang.IllegalStateException: Item not found: 
65
IndexValidationIssue [key=null, cacheName=cache_group_1_028, 
idxName=LONG__VAL_IDX], class java.lang.IllegalStateException: Item not found: 
65
IndexValidationIssue [key=null, cacheName=cache_group_1_028, 
idxName=LONG__VAL_IDX], class java.lang.IllegalStateException: Item not found: 
15
{noformat}
It would be better to explain what is happening: key is present in SQL index, 
but missing in corresponding data page.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-8593) The semaphore's isBroken function doesn't work properly.

2018-05-24 Thread Mo (JIRA)
Mo created IGNITE-8593:
--

 Summary: The semaphore's isBroken function doesn't work properly.
 Key: IGNITE-8593
 URL: https://issues.apache.org/jira/browse/IGNITE-8593
 Project: Ignite
  Issue Type: Bug
  Components: data structures
Affects Versions: 2.4
Reporter: Mo


Scenario: Three servers (s1,s2,s3) two clients (c1,c2).

A semaphore with one permit is created. 

Config: {{Release acquired permits if node, that owned them, left topology: set 
to false}}

 
 # c2 acquires the permit.
 # Network failure happens, isolating c2 from the rest of nodes for a period of 
time.
 # Network heals.
 # c2 releases the permit.
 # c2 acquires the permit.
 # Calling semaphore.isBroken() returns false on both c1 and c2.
 # c1 tries to acquire the permit but fails.
 # Now calling isBroken() returns true on both c1 and c2.

 

I think isBroken() should return true before a client tries to acquire a 
permit, and then fails (i.e., in step 6) rather than after acquiring a permit 
fails, as in the latter case, what purpose does the isBroken() function serves?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Node.js Thin Client @ npmjs

2018-05-24 Thread Igor Sapego
1. Sounds OK for me.
2. Since Node.js client is going to be included in Ignite releases,
I guess it make sense to align its version with the version of
Ignite, i.e. 2.6, etc...

Best Regards,
Igor

On Thu, May 24, 2018 at 9:02 AM, Pavel Petroshenko 
wrote:

> Igniters,
>
> I'm finalizing the docs (with regards to installation instructions) for the
> Node.js Thin Client at readme.io. And I've got a couple of questions.
>
> 1. Does the name "apache-ignite-client" for the Node.js Thin Client
> npm module sound good to everyone? This name will be used to install the
> client library, for example:
>
> npm i -g apache-ignite-client
>
> or include it from the source code:
>
> const IgniteClient = require('apache-ignite-client');
>
> 2. Which version should the npm module get: 1.0 or does it need to be
> aligned with the Ignite release version when it's published or compatible
> with, i.e. 2.4, 2.5 or 2.6?
>
> thanks,
> p.
>


[jira] [Created] (IGNITE-8592) Network partitions lead to two independent clusters

2018-05-24 Thread Mo (JIRA)
Mo created IGNITE-8592:
--

 Summary: Network partitions lead to two independent clusters
 Key: IGNITE-8592
 URL: https://issues.apache.org/jira/browse/IGNITE-8592
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Mo


Creating a network partition in a replicated Ignite cluster leads to creating 
two independent clusters, each of which would operate independently from the 
other, even after the network partition is healed.

 

Setup: 3 servers (s1,s2,s3) two clients (c1,c2).

A partition created \{(s1,s2,c1),(s3,c2)}.

--> At this point two independent clusters form; one containing s1 and s2, 
while the other containing s3. The two never rejoin even after the partition is 
healed. 

 

This creates different kinds of problems for the different data structure 
ignite provides, such as the cache (stale reads, and data unavailability), 
atomic types (atomicref and long ) ... etc. 

 

These are the settings used for the replicated cache:

 
cfg.setCacheMode(CacheMode.REPLICATED);
cfg.setAtomicityMode(CacheAtomicityMode.ATOMIC);
cfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
cfg.setReadFromBackup(false);
cfg.setPartitionLossPolicy(PartitionLossPolicy.READ_ONLY_SAFE);



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] ignite pull request #4059: IGNITE-8587 GridToStringBuilder use ConcurrentHas...

2018-05-24 Thread macrergate
GitHub user macrergate opened a pull request:

https://github.com/apache/ignite/pull/4059

IGNITE-8587 GridToStringBuilder use ConcurrentHashMap to avoid global…

… locks on classCache

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8587

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4059.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4059


commit 9db962835f49bc08a63f05b0e3f98ff5bdea
Author: Sergey Kosarev 
Date:   2018-05-24T08:56:12Z

IGNITE-8587 GridToStringBuilder use ConcurrentHashMap to avoid global locks 
on classCache




---


Re: Ability to check and completely fill transactions on creation

2018-05-24 Thread Anton Vinogradov
Dmitriy,

Yakov is against the solution based on event sub-system
>> I think that we should think about some other solution instead of
altering
>> event sub-system.

Also, I checked is there any chances to fix all the issues found by Yakov
and see that solution becomes overcomplicated in that case.
That's why I'm proposing this lightweight solution.

As for me it's a good idea to have such validator since that's a common
problem at huge deployments when more than one team have access to Ignite
cluster and there is no other way to setup tx cretion rules.

Yakov,

Could you please share your thoughts on that?


чт, 24 мая 2018 г. в 8:58, Dmitriy Setrakyan :

> On Wed, May 23, 2018 at 4:08 AM, Anton Vinogradov  wrote:
>
> > Dmitriy, Yakov
> >
> > Are there any objections to updated design taking into account the
> comments
> > I provided?
> >
>
> Anton, I do not like an additional validator. I think you can accomplish
> the same with a transaction event. You just need to design it more cleanly,
> incorporating the feedback from Yakov.
>


[GitHub] ignite pull request #4058: IGNITE-8591

2018-05-24 Thread devozerov
GitHub user devozerov opened a pull request:

https://github.com/apache/ignite/pull/4058

IGNITE-8591



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8591

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4058.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4058


commit 17c11ddbbf3e3d0e0e41e03d04050c1ac27340c7
Author: devozerov 
Date:   2018-05-24T08:22:29Z

IGNITE-8591: Done.




---


[jira] [Created] (IGNITE-8591) SQL: Sort links on index pages in physical page order before row access

2018-05-24 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-8591:
---

 Summary: SQL: Sort links on index pages in physical page order 
before row access
 Key: IGNITE-8591
 URL: https://issues.apache.org/jira/browse/IGNITE-8591
 Project: Ignite
  Issue Type: Task
  Components: sql
Affects Versions: 2.5
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
 Fix For: 2.6






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Node.js Thin Client @ npmjs

2018-05-24 Thread Pavel Petroshenko
Igniters,

I'm finalizing the docs (with regards to installation instructions) for the
Node.js Thin Client at readme.io. And I've got a couple of questions.

1. Does the name "apache-ignite-client" for the Node.js Thin Client
npm module sound good to everyone? This name will be used to install the
client library, for example:

npm i -g apache-ignite-client

or include it from the source code:

const IgniteClient = require('apache-ignite-client');

2. Which version should the npm module get: 1.0 or does it need to be
aligned with the Ignite release version when it's published or compatible
with, i.e. 2.4, 2.5 or 2.6?

thanks,
p.