Re: [VOTE] Apache Ignite 2.5.0 RC1
+1 (binding) On Thu, May 24, 2018 at 10:59 AM, Andrey Gurawrote: > Igniters, > > We've uploaded a 2.5.0 release candidate to > https://dist.apache.org/repos/dist/dev/ignite/2.5.0-rc1/ > > Git tag name is > 2.5.0-rc1 > > This release includes the following changes: > > Ignite: > * Implemented Zookeeper discovery SPI. > * Added Java thin client. > * Added continuous queries with transformers. > * Implemented general approach for critical failures handling. > * Added the new method IgniteMXBean.getCurrentCoordinatorFormatted() > which displays formatted information about current coordiantor. > * Added partition update counters verification on partition map exchange. > * Added ability to disable WAL during initial data rebalancing. > * Added additional data storage metrics > * Added additional transactional metrics. > * Added DEB package and improved RPM package. > * Added information about fields serialization/deserialization errors > in BinaryMarshaller. > * Added configurable automatic rollback timeout for pending > transactions preventing partition map exchange to complete. > * Added preventing infinite iterations of corrupted B+Tree. New > IGNITE_BPLUS_TREE_LOCK_RETRIES property was added, default value is > Integer.MAX_VALUE. > * Implemented handling of broken segment in WAL compaction. > * Fixed affinity to always choose primary node from one of the owners > when persistence is enabled. > * Partition eviction now does not block exchange. > * Fixed the issue that prevented rebalancing on a new baseline node join. > * Fixed potential deadlock during preloading. > * Fixed exception handling in IsolatedUpdater of DataStreamer. > * Fixed the issue when ComputeTaskFuture.get() threw a trimmed > exception chain: now full exception chain is propagated to the caller; > the exception thrown from IgniteCompute methods may now differ from > the previous behavior. > * Fixed null value being passed to entry processor when such entry > actually exists. > * Fixed problem when cluster could not survive after > IgniteOutOfMemoryException. > * Fixed NullPointerException on batch atomic cache operations with cache > store. > * Fixed bug with ScanQuery transformer which applies to the first > result page only. > * Fixed hanging of node stopping in case when node is invalidated and > persistence is enabled. > * BinaryMetadata exchange is triggered only if there are actual > changes in BinaryMetadata. > * Fixed contention while executing parallel atomic single get operations. > * Fixed cache metadata corruption during concurrent checkpoint and > cache destroy. > * Implemented additional synchronization for correct partition > counters update on partition map exchange. > * Optimized checkpoint lock acquisition for demander thread. > * Destroyed caches with enabled persistence now will not survive grid > restart. > * Nodes with incompatible SQL metadata will not be able to join grid. > This fixes known case for index corruption. > * Fixed fillFactor memory metric value. > * Fixed bug with checkpoint buffer overflow with enabled throttling. > * Fixed crash recovery problem when index tree could reference to > removed data pages. > * Fixed error during cluster initialization with partially acceptable > WAL archive. > * Fixed issue with local partitions size changing inside checkpoint write > lock. > * Fixed LOG_ONLY and BACKGROUND modes to be power-crash resistant. > * Fixed incorrect finishing of implicit transactions on unstable topology. > * Fixed incorrect mapping of smallint Cassandra type. > * Introduced log markers support to IgniteLogger and standard DEV_ONLY > marker. > * Changed Log4J and Log4J2 adapters to update active logger > configuration when configuration source file is modified. > * Fixed sqlline.sh script to use Java from JAVA_HOME. > * Improved Docker image build. > * Default and recommended JVM options are updated. > > Ignite .Net: > * Added SSL configuration for node-to-node links. > * Added dynamic service proxies. > * Propagated query metrics to ICache.GetQueryMetrics() and > ICache.ResetQueryMetrics(). > * DML update via LINQ. > * Fixed hang caused by mishandled exception during custom cache store > deserialization. > * Fixed GroupBy and Where issues in LINQ. > * .Net thin client SSL support. > > Ignite C++: > * Fixed bug with LocalSize cache method. > > SQL: > * Implemented COPY command for fast data load. > * Added support of authorization for CREATE TABLE and DROP TABLE commands. > * Improved deadlock handling in DML. > * Added IGNITE_SQL_FORCE_LAZY_RESULT_SET option to force all SQL > queries on node using lazy result set. > * Fixed SQL query hang if cluster contains nodes not in baseline. > > JDBC: > * Added SSL support. > * Added authentication. > * Added streaming mode to thin driver. > * Semicolon now can be used as delimiter for properties in URL string. > * Changed default socket buffer size to 64Kb in thin driver. > * Added FQDN to thin client in order to support multiple
[GitHub] ignite pull request #4067: graceful shutdown test
GitHub user BiryukovVA opened a pull request: https://github.com/apache/ignite/pull/4067 graceful shutdown test You can merge this pull request into a Git repository by running: $ git pull https://github.com/BiryukovVA/ignite graceful_shutdown_test Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4067.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4067 commit c236caf256ab49c0b7ae3de06702c6f41f3b7ef6 Author: Vitaliy BiryukovDate: 2018-05-24T22:47:23Z test ---
Re: Node.js Thin Client @ npmjs
As Denis said, there is no need to download the entire Ignite repo to install the client. Once published the client is going to be installed by users with a command: npm install -g apache-ignite-client The sources are going to be distributed as a part of the ignite repository, yes. But in general, release and installation process for the client and the Ignite technically are completely independent. And moreover: if there is a bug, especially critical, found in the client we shouldn't wait for the next Ignite to be released to get it fixed. We should be flexible enough to push the fixes and release the clients' updates independently at any point in time. Having an independent release/versioning scheme would allow the clients to get bug-fixes (minor version update) or nonbreaking feature-adds or improvements (medium version update) between major Ignite releases (potentially breaking changes and thus - the major version update). But the client and the Ignite versions mapping might be tricky and should be clearly documented. So there are pros and cons. But I believe the release policy should be consistent across all the Thin clients (I'm not talking about the "native" or Thick ones, which heavily depend on the Ignite internals and are a different story). p. On Thu, May 24, 2018 at 1:44 PM, Denis Magdawrote: > Once the client is built it will be uploaded to the npmjs repository, > right? So, a JS developer can download the client from there without > touching the whole Ignite binary release. > > However, those who download the whole Ignite binary distribution will find > node.js there (as well as .NET, C++, JDBC and ODBC). > > -- > Denis > > On Thu, May 24, 2018 at 1:06 PM, Dmitriy Setrakyan > wrote: > > > On Thu, May 24, 2018 at 12:36 PM, Pavel Petroshenko < > pa...@petroshenko.com > > > > > wrote: > > > > > Fair enough. Consistency with the other clients is a good argument. > > > > > > > > Pavel, I would discuss it a bit more. Does it really make sense for a > > node.js user to download the whole Ignite distribution just to get a > > node.js client? > > >
[GitHub] ignite pull request #4066: IGNITE-8612 Debugging NPE in GridCacheTtlManager:...
GitHub user andrewmed opened a pull request: https://github.com/apache/ignite/pull/4066 IGNITE-8612 Debugging NPE in GridCacheTtlManager: add assert You can merge this pull request into a Git repository by running: $ git pull https://github.com/andrewmed/ignite ignite-8612 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4066.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4066 commit 7554b6fc667954b37cc52feb3ccc23b663defdd2 Author: AMedvedevDate: 2018-05-24T20:50:54Z IGNITE-8612 Debugging NPE: add assert ---
Re: Node.js Thin Client @ npmjs
Once the client is built it will be uploaded to the npmjs repository, right? So, a JS developer can download the client from there without touching the whole Ignite binary release. However, those who download the whole Ignite binary distribution will find node.js there (as well as .NET, C++, JDBC and ODBC). -- Denis On Thu, May 24, 2018 at 1:06 PM, Dmitriy Setrakyanwrote: > On Thu, May 24, 2018 at 12:36 PM, Pavel Petroshenko > > wrote: > > > Fair enough. Consistency with the other clients is a good argument. > > > > > Pavel, I would discuss it a bit more. Does it really make sense for a > node.js user to download the whole Ignite distribution just to get a > node.js client? >
[jira] [Created] (IGNITE-8612) NPE in GridCacheTtlManager#expire on commit() or close()
Andrew Medvedev created IGNITE-8612: --- Summary: NPE in GridCacheTtlManager#expire on commit() or close() Key: IGNITE-8612 URL: https://issues.apache.org/jira/browse/IGNITE-8612 Project: Ignite Issue Type: Bug Affects Versions: 2.4 Reporter: Andrew Medvedev We have got NPE in org.apache.ignite.internal.processors.cache.GridCacheTtlManager#expire(int) several times during 4 minutes from tx.close() and tx.commit() here [https://github.com/apache/ignite/blob/40845c67750c300b5568d157ab0ffeaf320802a8/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheTtlManager.java#L203] {{Caused by: java.lang.NullPointerException}} {{at org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:197)}} {{at org.apache.ignite.internal.processors.cache.GridCacheUtils.unwindEvicts(GridCacheUtils.java:834)}} {{at org.apache.ignite.internal.processors.cache.GridCacheUtils.unwindEvicts(GridCacheUtils.java:844)}} {{at org.apache.ignite.internal.processors.cache.transactions.TransactionProxyImpl.leave(TransactionProxyImpl.java:136)}} {{at org.apache.ignite.internal.processors.cache.transactions.TransactionProxyImpl.close(TransactionProxyImpl.java:326)}} This could have been IgniteCacheOffheapManager == null, cctx.offheap() returning null, but I could not reproduce it. To debug this further, a PR with assert added will be submitted -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8611) Binary marshaller documentation should cover how data classes can or can't be changed
Stanislav Lukyanov created IGNITE-8611: -- Summary: Binary marshaller documentation should cover how data classes can or can't be changed Key: IGNITE-8611 URL: https://issues.apache.org/jira/browse/IGNITE-8611 Project: Ignite Issue Type: Improvement Components: documentation Reporter: Stanislav Lukyanov Binary marshaller docs (https://apacheignite.readme.io/docs/binary-marshaller) give an idea that a class structure may be changed (fields can be added or removed) and the marshaller will handle such change. However, not all changes are supported. One corner case is when an enum value is stored in the cache: if the order of the enum constants is changed, or if a new constant is added at the start or at the middle of the constants list, it will lead to an error. This is because the enums are stored as ordinals (integers), and the ordinals of an enum depend on the order of values in the code. The task is to update the documentation with the description of data class changes that are incompatible from binary marshallers point of view. At least the enum case should be covered. If more cases are discovered, they should be documented as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Node.js Thin Client @ npmjs
On Thu, May 24, 2018 at 12:36 PM, Pavel Petroshenkowrote: > Fair enough. Consistency with the other clients is a good argument. > > Pavel, I would discuss it a bit more. Does it really make sense for a node.js user to download the whole Ignite distribution just to get a node.js client?
[GitHub] ignite pull request #4065: IGNITE-8459 Do first checkpoint after all partiti...
GitHub user Jokser opened a pull request: https://github.com/apache/ignite/pull/4065 IGNITE-8459 Do first checkpoint after all partitions have been initialized You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-8459 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4065.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4065 commit df3fb2a24d4d43771094e56fea6680f23e3790e6 Author: Pavel KovalenkoDate: 2018-05-18T09:37:13Z IGNITE-8459 WIP commit 54e88d22bfd4c0b61593b4a8f0d5c319288593b7 Author: Pavel Kovalenko Date: 2018-05-18T16:22:01Z IGNITE-8459 WIP commit e8aeeea9d6c30df2ee03bfada3a8a399cfef7b6b Author: Pavel Kovalenko Date: 2018-05-21T12:42:33Z IGNITE-8459 WIP commit 7d58eb9e3c3fa2358c7937ce3f73715d850a33f1 Author: Pavel Kovalenko Date: 2018-05-24T19:46:24Z IGNITE-8459 Rework. commit bd8ef85034bbf7c03d6c8b40b36fa4398a3b23ca Author: Pavel Kovalenko Date: 2018-05-24T19:50:11Z IGNITE-8459 Remove trash. ---
[jira] [Created] (IGNITE-8610) Searching checkpoint / WAL history for rebalancing is not properly working in case of local/global WAL disabling
Pavel Kovalenko created IGNITE-8610: --- Summary: Searching checkpoint / WAL history for rebalancing is not properly working in case of local/global WAL disabling Key: IGNITE-8610 URL: https://issues.apache.org/jira/browse/IGNITE-8610 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.5 Reporter: Pavel Kovalenko Assignee: Pavel Kovalenko Fix For: 2.6 After implementation IGNITE-6411 and IGNITE-8087 we can face with situation when after some checkpoint, WAL was temporarily disabled and enabled again. In this case we can't treat such checkpoint as start point to rebalance, because WAL history after such checkpoint may contain gaps. We should rework our checkpoint / wal history searching mechanism and ignore such checkpoints. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: NodeJS thin client: full API
Hi Denis, That's a good point, thanks. This should be a part of the "Usage" section. I'll follow up in JIRA. p. On Thu, May 24, 2018 at 10:49 AM, Denis Magdawrote: > Pavel, > > Recalled that we've not described how to authenticate and set up SSL from > the client side. Please consider this for the doc. Left some notes in the > JIRA. > > -- > Denis > > On Wed, May 23, 2018 at 12:25 PM, Denis Magda wrote: > > > Alexey, Pavel, > > > > I've done a preliminary review of the doc and moved it to the readme.io > > page: > > https://apacheignite.readme.io/v2.4/docs/nodejs-thin-client > > > > The page is hidden. I'll grant you access to readme so that you can > update > > the doc taking my suggestions into account: > > https://issues.apache.org/jira/browse/IGNITE-8589 > > > > -- > > Denis > > > > > > On Mon, May 21, 2018 at 6:39 PM, Alexey Kuznetsov > > > wrote: > > > >> Hi, > >> > >> FYI, HZ also has NodeJs client: https://github.com/ > >> hazelcast/hazelcast-nodejs-client > >> May be it is worth to take a look? > >> > >> -- > >> Alexey Kuznetsov > >> > > > > >
Re: Node.js Thin Client @ npmjs
Fair enough. Consistency with the other clients is a good argument. p. On Thu, May 24, 2018 at 11:44 AM, Dmitriy Setrakyanwrote: > On Thu, May 24, 2018 at 11:42 AM, Igor Sapego wrote: > > > Well, all other clients have the same version as Ignite. > > Are there any reasons to make a Node.js client some > > kind of a special case? > > > > > I may have spoken too soon. Do we plan to include the node.js client into > Ignite build? If yes, then it will be built anew with every Ignite release, > so the same versioning makes sense in this case. > > D. >
Re: [VOTE] Apache Ignite 2.5.0 RC1
+1 (binding) Checked .NET side of things: build from sources, run examples. On Thu, May 24, 2018 at 8:59 PM, Andrey Gurawrote: > Igniters, > > We've uploaded a 2.5.0 release candidate to > https://dist.apache.org/repos/dist/dev/ignite/2.5.0-rc1/ > > Git tag name is > 2.5.0-rc1 > > This release includes the following changes: > > Ignite: > * Implemented Zookeeper discovery SPI. > * Added Java thin client. > * Added continuous queries with transformers. > * Implemented general approach for critical failures handling. > * Added the new method IgniteMXBean.getCurrentCoordinatorFormatted() > which displays formatted information about current coordiantor. > * Added partition update counters verification on partition map exchange. > * Added ability to disable WAL during initial data rebalancing. > * Added additional data storage metrics > * Added additional transactional metrics. > * Added DEB package and improved RPM package. > * Added information about fields serialization/deserialization errors > in BinaryMarshaller. > * Added configurable automatic rollback timeout for pending > transactions preventing partition map exchange to complete. > * Added preventing infinite iterations of corrupted B+Tree. New > IGNITE_BPLUS_TREE_LOCK_RETRIES property was added, default value is > Integer.MAX_VALUE. > * Implemented handling of broken segment in WAL compaction. > * Fixed affinity to always choose primary node from one of the owners > when persistence is enabled. > * Partition eviction now does not block exchange. > * Fixed the issue that prevented rebalancing on a new baseline node join. > * Fixed potential deadlock during preloading. > * Fixed exception handling in IsolatedUpdater of DataStreamer. > * Fixed the issue when ComputeTaskFuture.get() threw a trimmed > exception chain: now full exception chain is propagated to the caller; > the exception thrown from IgniteCompute methods may now differ from > the previous behavior. > * Fixed null value being passed to entry processor when such entry > actually exists. > * Fixed problem when cluster could not survive after > IgniteOutOfMemoryException. > * Fixed NullPointerException on batch atomic cache operations with cache > store. > * Fixed bug with ScanQuery transformer which applies to the first > result page only. > * Fixed hanging of node stopping in case when node is invalidated and > persistence is enabled. > * BinaryMetadata exchange is triggered only if there are actual > changes in BinaryMetadata. > * Fixed contention while executing parallel atomic single get operations. > * Fixed cache metadata corruption during concurrent checkpoint and > cache destroy. > * Implemented additional synchronization for correct partition > counters update on partition map exchange. > * Optimized checkpoint lock acquisition for demander thread. > * Destroyed caches with enabled persistence now will not survive grid > restart. > * Nodes with incompatible SQL metadata will not be able to join grid. > This fixes known case for index corruption. > * Fixed fillFactor memory metric value. > * Fixed bug with checkpoint buffer overflow with enabled throttling. > * Fixed crash recovery problem when index tree could reference to > removed data pages. > * Fixed error during cluster initialization with partially acceptable > WAL archive. > * Fixed issue with local partitions size changing inside checkpoint write > lock. > * Fixed LOG_ONLY and BACKGROUND modes to be power-crash resistant. > * Fixed incorrect finishing of implicit transactions on unstable topology. > * Fixed incorrect mapping of smallint Cassandra type. > * Introduced log markers support to IgniteLogger and standard DEV_ONLY > marker. > * Changed Log4J and Log4J2 adapters to update active logger > configuration when configuration source file is modified. > * Fixed sqlline.sh script to use Java from JAVA_HOME. > * Improved Docker image build. > * Default and recommended JVM options are updated. > > Ignite .Net: > * Added SSL configuration for node-to-node links. > * Added dynamic service proxies. > * Propagated query metrics to ICache.GetQueryMetrics() and > ICache.ResetQueryMetrics(). > * DML update via LINQ. > * Fixed hang caused by mishandled exception during custom cache store > deserialization. > * Fixed GroupBy and Where issues in LINQ. > * .Net thin client SSL support. > > Ignite C++: > * Fixed bug with LocalSize cache method. > > SQL: > * Implemented COPY command for fast data load. > * Added support of authorization for CREATE TABLE and DROP TABLE commands. > * Improved deadlock handling in DML. > * Added IGNITE_SQL_FORCE_LAZY_RESULT_SET option to force all SQL > queries on node using lazy result set. > * Fixed SQL query hang if cluster contains nodes not in baseline. > > JDBC: > * Added SSL support. > * Added authentication. > * Added streaming mode to thin driver. > * Semicolon now can be used as delimiter for properties in URL string. > * Changed default socket buffer size to 64Kb in thin
[jira] [Created] (IGNITE-8609) SQL: Get rid of syncronization in mvcc processor.
Pavel Kuznetsov created IGNITE-8609: --- Summary: SQL: Get rid of syncronization in mvcc processor. Key: IGNITE-8609 URL: https://issues.apache.org/jira/browse/IGNITE-8609 Project: Ignite Issue Type: Task Components: sql Reporter: Pavel Kuznetsov Assignee: Igor Seliverstov Currently we have to do synchronized actions in org/apache/ignite/internal/managers/communication/GridIoManager.java:1124 (org.apache.ignite.internal.managers.communication.GridIoManager#processRegularMessage) due to fails on nio thread on load. We are able to optimize this code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Node.js Thin Client @ npmjs
On Thu, May 24, 2018 at 11:42 AM, Igor Sapegowrote: > Well, all other clients have the same version as Ignite. > Are there any reasons to make a Node.js client some > kind of a special case? > > I may have spoken too soon. Do we plan to include the node.js client into Ignite build? If yes, then it will be built anew with every Ignite release, so the same versioning makes sense in this case. D.
Re: Node.js Thin Client @ npmjs
Well, all other clients have the same version as Ignite. Are there any reasons to make a Node.js client some kind of a special case? Best Regards, Igor On Thu, May 24, 2018 at 8:21 PM, Dmitriy Setrakyanwrote: > On Thu, May 24, 2018 at 10:03 AM, Pavel Petroshenko > > wrote: > > > Igor, > > > > Are you proposing to update the Thin Client versions on every Ignite > > release regardless of the changes made to them? > > > > I tend to think that an independent versioning scheme for the Thin > Clients > > might be more flexible/meaningful. > > > > Pavel, I agree, the thin clients should have independent versioning. >
Re: IGNITE-8583 - review needed
Reasonable proposal and I hope most Igniters will follow it. In the same time I've just clicked to the link https://issues.apache.org/jira/browse/IGNITE-8583 provided by Dmitriy to check ticket name, and then I've clicked to review link https://reviews.ignite.apache.org/ignite/review/IGNT-CR-623 to find out review status. So I'm OK if contributor will provide just link to ticket. чт, 24 мая 2018 г. в 20:47, Dmitriy Setrakyan: > I continuously ask not to provide naked ticket numbers in the subject. > Please also add titles or descriptions. I doubt anyone in the community can > tell what this is about just by looking at the number. > > D. > > On Thu, May 24, 2018 at 4:09 AM, Dmitry Pavlov > wrote: > > > Hi Dmitriy, > > > > According to upsource > > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-623 review was > > already started by Alexey G. and Alexey K. > > > > Sincerely, > > Dmitriy Pavlov > > > > > > > > чт, 24 мая 2018 г. в 14:04, Dmitriy Govorukhin < > > dmitriy.govoruk...@gmail.com > > >: > > > > > Igniters, > > > > > > I fixed DataStorageMetricsMXBean.getOffHeapSize, please review my > > changes. > > > > > > https://issues.apache.org/jira/browse/IGNITE-8583 > > > > > >
[GitHub] ignite pull request #4064: TcpDiscoveryMulticastIpFinder changed to TcpDisco...
GitHub user antkr opened a pull request: https://github.com/apache/ignite/pull/4064 TcpDiscoveryMulticastIpFinder changed to TcpDiscoveryVmIpFinder in xm⦠PR for TC purposes. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-6826 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4064.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4064 commit c413c236863202ee902c2f070bc6a2130ddb5201 Author: Anton KurbanovDate: 2018-05-24T18:05:03Z TcpDiscoveryMulticastIpFinder changed to TcpDiscoveryVmIpFinder in xml configurations. ---
[VOTE] Apache Ignite 2.5.0 RC1
Igniters, We've uploaded a 2.5.0 release candidate to https://dist.apache.org/repos/dist/dev/ignite/2.5.0-rc1/ Git tag name is 2.5.0-rc1 This release includes the following changes: Ignite: * Implemented Zookeeper discovery SPI. * Added Java thin client. * Added continuous queries with transformers. * Implemented general approach for critical failures handling. * Added the new method IgniteMXBean.getCurrentCoordinatorFormatted() which displays formatted information about current coordiantor. * Added partition update counters verification on partition map exchange. * Added ability to disable WAL during initial data rebalancing. * Added additional data storage metrics * Added additional transactional metrics. * Added DEB package and improved RPM package. * Added information about fields serialization/deserialization errors in BinaryMarshaller. * Added configurable automatic rollback timeout for pending transactions preventing partition map exchange to complete. * Added preventing infinite iterations of corrupted B+Tree. New IGNITE_BPLUS_TREE_LOCK_RETRIES property was added, default value is Integer.MAX_VALUE. * Implemented handling of broken segment in WAL compaction. * Fixed affinity to always choose primary node from one of the owners when persistence is enabled. * Partition eviction now does not block exchange. * Fixed the issue that prevented rebalancing on a new baseline node join. * Fixed potential deadlock during preloading. * Fixed exception handling in IsolatedUpdater of DataStreamer. * Fixed the issue when ComputeTaskFuture.get() threw a trimmed exception chain: now full exception chain is propagated to the caller; the exception thrown from IgniteCompute methods may now differ from the previous behavior. * Fixed null value being passed to entry processor when such entry actually exists. * Fixed problem when cluster could not survive after IgniteOutOfMemoryException. * Fixed NullPointerException on batch atomic cache operations with cache store. * Fixed bug with ScanQuery transformer which applies to the first result page only. * Fixed hanging of node stopping in case when node is invalidated and persistence is enabled. * BinaryMetadata exchange is triggered only if there are actual changes in BinaryMetadata. * Fixed contention while executing parallel atomic single get operations. * Fixed cache metadata corruption during concurrent checkpoint and cache destroy. * Implemented additional synchronization for correct partition counters update on partition map exchange. * Optimized checkpoint lock acquisition for demander thread. * Destroyed caches with enabled persistence now will not survive grid restart. * Nodes with incompatible SQL metadata will not be able to join grid. This fixes known case for index corruption. * Fixed fillFactor memory metric value. * Fixed bug with checkpoint buffer overflow with enabled throttling. * Fixed crash recovery problem when index tree could reference to removed data pages. * Fixed error during cluster initialization with partially acceptable WAL archive. * Fixed issue with local partitions size changing inside checkpoint write lock. * Fixed LOG_ONLY and BACKGROUND modes to be power-crash resistant. * Fixed incorrect finishing of implicit transactions on unstable topology. * Fixed incorrect mapping of smallint Cassandra type. * Introduced log markers support to IgniteLogger and standard DEV_ONLY marker. * Changed Log4J and Log4J2 adapters to update active logger configuration when configuration source file is modified. * Fixed sqlline.sh script to use Java from JAVA_HOME. * Improved Docker image build. * Default and recommended JVM options are updated. Ignite .Net: * Added SSL configuration for node-to-node links. * Added dynamic service proxies. * Propagated query metrics to ICache.GetQueryMetrics() and ICache.ResetQueryMetrics(). * DML update via LINQ. * Fixed hang caused by mishandled exception during custom cache store deserialization. * Fixed GroupBy and Where issues in LINQ. * .Net thin client SSL support. Ignite C++: * Fixed bug with LocalSize cache method. SQL: * Implemented COPY command for fast data load. * Added support of authorization for CREATE TABLE and DROP TABLE commands. * Improved deadlock handling in DML. * Added IGNITE_SQL_FORCE_LAZY_RESULT_SET option to force all SQL queries on node using lazy result set. * Fixed SQL query hang if cluster contains nodes not in baseline. JDBC: * Added SSL support. * Added authentication. * Added streaming mode to thin driver. * Semicolon now can be used as delimiter for properties in URL string. * Changed default socket buffer size to 64Kb in thin driver. * Added FQDN to thin client in order to support multiple IPs for connection establishment. ODBC: * Added SSL support. * Implemented username/password authentication. * Added support of multiple addresses in connection string. * Added support for SQL_ATTR_LOGIN_TIMEOUT. Web Console: * Added support for persistence in demo mode. * Configuration screen
Re: NodeJS thin client: full API
Pavel, Recalled that we've not described how to authenticate and set up SSL from the client side. Please consider this for the doc. Left some notes in the JIRA. -- Denis On Wed, May 23, 2018 at 12:25 PM, Denis Magdawrote: > Alexey, Pavel, > > I've done a preliminary review of the doc and moved it to the readme.io > page: > https://apacheignite.readme.io/v2.4/docs/nodejs-thin-client > > The page is hidden. I'll grant you access to readme so that you can update > the doc taking my suggestions into account: > https://issues.apache.org/jira/browse/IGNITE-8589 > > -- > Denis > > > On Mon, May 21, 2018 at 6:39 PM, Alexey Kuznetsov > wrote: > >> Hi, >> >> FYI, HZ also has NodeJs client: https://github.com/ >> hazelcast/hazelcast-nodejs-client >> May be it is worth to take a look? >> >> -- >> Alexey Kuznetsov >> > >
Re: IGNITE-8583 - review needed
I continuously ask not to provide naked ticket numbers in the subject. Please also add titles or descriptions. I doubt anyone in the community can tell what this is about just by looking at the number. D. On Thu, May 24, 2018 at 4:09 AM, Dmitry Pavlovwrote: > Hi Dmitriy, > > According to upsource > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-623 review was > already started by Alexey G. and Alexey K. > > Sincerely, > Dmitriy Pavlov > > > > чт, 24 мая 2018 г. в 14:04, Dmitriy Govorukhin < > dmitriy.govoruk...@gmail.com > >: > > > Igniters, > > > > I fixed DataStorageMetricsMXBean.getOffHeapSize, please review my > changes. > > > > https://issues.apache.org/jira/browse/IGNITE-8583 > > >
Re: MTCGA & 6th mass run-all
Great outcomes! Can't wait seeing how many tests were fixed since the previous mass run. -- Denis On Thu, May 24, 2018 at 7:06 AM, Dmitry Pavlovwrote: > Hi, > > A lot of Igniters made their contributions to Make TC Green during 6 last > weeks. It is Aleksey Plekhanov, Maxim Muzafarov, Alexey Goncharuk, Ilya > Lantukh, Ivan Rakov, Alexey Kuznetsov, Taras Ledkov, Andrey Kuznetsov, > Alexei Scherbakov, Ivan Fedotov, Alexander Menshikov, Andrew Medvedev, > Andrew Mashenkov, Nikolay Izhikov, Stanilovsky Evgeny, Vyacheslav Daradur, > Alexey Kuznetsov, Pavel Kovalenko, Vitaliy Biryukov, Alexey Kukushkin, > Pavel Pereslegin, Amelchev Nikita. > > Folks, let me say thank you for your contributions. This effort will > definitely move us towards day with 0 fails. > > It’s time to check progress, so I’m going to schedule mass run-all's during > weekend. I plan to trigger these builds Fri May 25 at 15:00 GMT, 18:00 MSK. > Please reply if you have objections or need TC at this time. Feel free to > move builds to top if you have needed fixes. > > Any finished patch review process for MTCGA PA issues is also appreciated. > > Sincerely, > Dmitriy Pavlov >
Re: Node.js Thin Client @ npmjs
On Thu, May 24, 2018 at 10:03 AM, Pavel Petroshenkowrote: > Igor, > > Are you proposing to update the Thin Client versions on every Ignite > release regardless of the changes made to them? > > I tend to think that an independent versioning scheme for the Thin Clients > might be more flexible/meaningful. > Pavel, I agree, the thin clients should have independent versioning.
[jira] [Created] (IGNITE-8608) .NET: Sign release NuGet packages
Pavel Tupitsyn created IGNITE-8608: -- Summary: .NET: Sign release NuGet packages Key: IGNITE-8608 URL: https://issues.apache.org/jira/browse/IGNITE-8608 Project: Ignite Issue Type: Improvement Components: platforms Reporter: Pavel Tupitsyn Assignee: Pavel Tupitsyn Fix For: 2.6 NuGet signed package submissions has been introduced recently: https://blog.nuget.org/20180522/Introducing-signed-package-submissions.html Sign Ignite.NET packages when releasing them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Node.js Thin Client @ npmjs
Igor, Are you proposing to update the Thin Client versions on every Ignite release regardless of the changes made to them? I tend to think that an independent versioning scheme for the Thin Clients might be more flexible/meaningful. p. On Thu, May 24, 2018 at 2:04 AM, Igor Sapegowrote: > 1. Sounds OK for me. > 2. Since Node.js client is going to be included in Ignite releases, > I guess it make sense to align its version with the version of > Ignite, i.e. 2.6, etc... > > Best Regards, > Igor > > On Thu, May 24, 2018 at 9:02 AM, Pavel Petroshenko > wrote: > > > Igniters, > > > > I'm finalizing the docs (with regards to installation instructions) for > the > > Node.js Thin Client at readme.io. And I've got a couple of questions. > > > > 1. Does the name "apache-ignite-client" for the Node.js Thin Client > > npm module sound good to everyone? This name will be used to install the > > client library, for example: > > > > npm i -g apache-ignite-client > > > > or include it from the source code: > > > > const IgniteClient = require('apache-ignite-client'); > > > > 2. Which version should the npm module get: 1.0 or does it need to be > > aligned with the Ignite release version when it's published or compatible > > with, i.e. 2.4, 2.5 or 2.6? > > > > thanks, > > p. > > >
[GitHub] ignite pull request #4063: IGNITE-8547 Use JVM serialization for enum values...
GitHub user alamar opened a pull request: https://github.com/apache/ignite/pull/4063 IGNITE-8547 Use JVM serialization for enum values with OptimizedMarsh⦠â¦aller, avoid deadlock. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-8547 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4063.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4063 commit 4ddb8bd85c7000356badb7bd8daf233491aaa3bf Author: Ilya KasnacheevDate: 2018-05-22T11:32:12Z IGNITE-8547 Use JVM serialization for enum values with OptimizedMarshaller, avoid deadlock. ---
[GitHub] ignite pull request #4042: IGNITE-8547 Use JVM serialization for enum values...
Github user alamar closed the pull request at: https://github.com/apache/ignite/pull/4042 ---
[GitHub] ignite pull request #4051: IGNITE-8560 checkFirst and checkThrough parameter...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4051 ---
[GitHub] ignite pull request #4062: IGNITE-8567: Imputer and Binarizer
GitHub user zaleslaw opened a pull request: https://github.com/apache/ignite/pull/4062 IGNITE-8567: Imputer and Binarizer You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-8567 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4062.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4062 commit 098d2832b3f4af23c72bc25f43e4ab8a95f2f416 Author: Zinoviev AlexeyDate: 2018-04-11T18:40:27Z IGNITE-7829: Added example commit 78f9ea687bff77ec9f6bef87126569cb92cbe745 Author: Zinoviev Alexey Date: 2018-04-13T15:44:26Z Merge branch 'master' of https://github.com/apache/ignite commit 199e17d19ccbde9f15aba5375d834c3930b3a989 Author: Zinoviev Alexey Date: 2018-04-27T10:12:47Z Merge branch 'master' of https://github.com/apache/ignite commit aca9833df4d3cc4a641dd9109daaf628bc85acdf Author: Zinoviev Alexey Date: 2018-05-08T05:29:49Z Merge branch 'master' of https://github.com/apache/ignite commit bb244de762b89d0a1e5606aa282e34d92752595b Author: Zinoviev Alexey Date: 2018-05-16T11:42:06Z Merge branch 'master' of https://github.com/apache/ignite commit a0f6f0fcc54f8a6da36d4b61836719e19a6bbb4a Author: Zinoviev Alexey Date: 2018-05-16T13:21:47Z IGNITE-8511: Added MultiClass Log Regression commit b4cb1a42d35a0da9f8b762207011a46c6f542a20 Author: Zinoviev Alexey Date: 2018-05-16T16:17:39Z Merge branch 'master' of https://github.com/apache/ignite commit eb6178cd8e4a61fa12d4558b4377d53f94029c86 Author: Zinoviev Alexey Date: 2018-05-16T16:18:17Z Merge branch 'master' into ignite-8511 commit 21cb4433728618c06aa8e16fe681772ea25c9b63 Author: Zinoviev Alexey Date: 2018-05-21T07:47:08Z IGNITE-8511: Added MultiClass Log Regression commit 64d96c8c29e290d7ff98c8018056609c3bab83ed Author: Zinoviev Alexey Date: 2018-05-21T07:56:28Z IGNITE-8511: Added MultiClass Log Regression commit e10fb00ed15ac348f097d0eb91abfa4661769025 Author: Zinoviev Alexey Date: 2018-05-23T09:51:13Z IGNITE-8567: Added examples and 2 preprocessors commit a3cf8b5c35b961fd772dabf350f937425822c6ba Author: zaleslaw Date: 2018-05-24T15:05:21Z ignite-8567: Added tests for Imputer/Binarizer preprocessors ---
Re: [jira] [Created] (IGNITE-8595) SQL: Ability to cancel DDL operations
Thanks for the reply, Vladimir. Is there a ticket to track the cancel feature described by you? -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
[GitHub] ignite pull request #4054: IGNITE-8583 fix checkpoint buffer size metric
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4054 ---
[jira] [Created] (IGNITE-8607) [.NET] Support metrics changes in DataStorageMetricsMXBean
Dmitriy Govorukhin created IGNITE-8607: -- Summary: [.NET] Support metrics changes in DataStorageMetricsMXBean Key: IGNITE-8607 URL: https://issues.apache.org/jira/browse/IGNITE-8607 Project: Ignite Issue Type: Bug Reporter: Dmitriy Govorukhin -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8606) Node hangs on next exchange, when no access to marshaller's folder
Vladislav Pyatkov created IGNITE-8606: - Summary: Node hangs on next exchange, when no access to marshaller's folder Key: IGNITE-8606 URL: https://issues.apache.org/jira/browse/IGNITE-8606 Project: Ignite Issue Type: Bug Reporter: Vladislav Pyatkov {noformat} 2018-05-18 11:12:57.572 [ERROR][tcp-disco-msg-worker-#3%DPL_GRID%DplGridNodeName%][o.a.i.i.MarshallerMappingFileStore] Failed to write class name to file [platformId=0id=1713316383, clsName=com.sbt.dpl.gridgain.affinity.DPLIndexAffinityPrimaryFilter, file=/u01/pprb/work/marshaller/1713316383.classname0] java.io.FileNotFoundException: /u01/pprb/work/marshaller/1713316383.classname0 (No such file or directory) at java.io.FileOutputStream.open0(Native Method) at java.io.FileOutputStream.open(FileOutputStream.java:270) at java.io.FileOutputStream.(FileOutputStream.java:213) at java.io.FileOutputStream.(FileOutputStream.java:162) at org.apache.ignite.internal.MarshallerMappingFileStore.writeMapping(MarshallerMappingFileStore.java:94) at org.apache.ignite.internal.MarshallerMappingFileStore.mergeAndWriteMapping(MarshallerMappingFileStore.java:207) at org.apache.ignite.internal.MarshallerContextImpl.onMappingDataReceived(MarshallerContextImpl.java:201) at org.apache.ignite.internal.processors.marshaller.GridMarshallerMappingProcessor.processIncomingMappings(GridMarshallerMappingProcessor.java:356) at org.apache.ignite.internal.processors.marshaller.GridMarshallerMappingProcessor.onJoiningNodeDataReceived(GridMarshallerMappingProcessor.java:336) at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$5.onExchange(GridDiscoveryManager.java:908) at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.onExchange(TcpDiscoverySpi.java:1939) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processNodeAddedMessage(ServerImpl.java:4220) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2744) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ignite pull request #4061: IGNITE-8605 Specify jline minor version to fix nc...
GitHub user alamar opened a pull request: https://github.com/apache/ignite/pull/4061 IGNITE-8605 Specify jline minor version to fix ncurses issue. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-8605 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4061.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4061 commit 4b7251df22be488b5931f98720ee1d4df9dd9d3d Author: Ilya KasnacheevDate: 2018-05-24T14:55:43Z IGNITE-8605 Specify jline minor version to fix ncurses issue. ---
[jira] [Created] (IGNITE-8605) sqlline can't work with terminal on newer ncurses
Ilya Kasnacheev created IGNITE-8605: --- Summary: sqlline can't work with terminal on newer ncurses Key: IGNITE-8605 URL: https://issues.apache.org/jira/browse/IGNITE-8605 Project: Ignite Issue Type: Bug Components: general Affects Versions: 2.5 Reporter: Ilya Kasnacheev On Ubuntu 18.04: {code} ~/w/incubator-ignite/modules/sqlline% IGNITE_HOME=~/w/incubator-ignite . bin/sqlline.sh [ERROR] Failed to construct terminal; falling back to unsupported java.lang.NumberFormatException: For input string: "0x100" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:580) at java.lang.Integer.valueOf(Integer.java:766) at jline.internal.InfoCmp.parseInfoCmp(InfoCmp.java:59) at jline.UnixTerminal.parseInfoCmp(UnixTerminal.java:242) at jline.UnixTerminal.(UnixTerminal.java:65) at jline.UnixTerminal.(UnixTerminal.java:50) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at jline.TerminalFactory.getFlavor(TerminalFactory.java:211) at jline.TerminalFactory.create(TerminalFactory.java:102) at jline.TerminalFactory.get(TerminalFactory.java:186) at jline.TerminalFactory.get(TerminalFactory.java:192) at sqlline.SqlLineOpts.(SqlLineOpts.java:45) at sqlline.SqlLine.(SqlLine.java:54) at sqlline.SqlLine.start(SqlLine.java:372) at sqlline.SqlLine.main(SqlLine.java:265) [ERROR] Failed to construct terminal; falling back to unsupported java.lang.NumberFormatException: For input string: "0x100" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:580) at java.lang.Integer.valueOf(Integer.java:766) at jline.internal.InfoCmp.parseInfoCmp(InfoCmp.java:59) at jline.UnixTerminal.parseInfoCmp(UnixTerminal.java:242) at jline.UnixTerminal.(UnixTerminal.java:65) at jline.UnixTerminal.(UnixTerminal.java:50) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at jline.TerminalFactory.getFlavor(TerminalFactory.java:211) at jline.TerminalFactory.create(TerminalFactory.java:102) at jline.TerminalFactory.create(TerminalFactory.java:51) at sqlline.SqlLine.getConsoleReader(SqlLine.java:705) at sqlline.SqlLine.begin(SqlLine.java:639) at sqlline.SqlLine.start(SqlLine.java:373) at sqlline.SqlLine.main(SqlLine.java:265) sqlline version 1.3.0 sqlline> ... and then history and command editing won't work {code} See also https://github.com/jline/jline2/issues/281 I think we should manually peg jline verison to 2.14.4 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ignite pull request #4060: IGN-10640 [IGNITE-8603] Add JMX-metric to cluster...
GitHub user dgladkikh opened a pull request: https://github.com/apache/ignite/pull/4060 IGN-10640 [IGNITE-8603] Add JMX-metric to cluster: baseline nodes You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-8603 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4060.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4060 commit 9ce4e57ddafa828d58f692bd4f8a1645d6e61f74 Author: dgladkikhDate: 2018-05-24T14:44:07Z IGN-10640 [IGNITE-8603] Add JMX-metric to cluster: baseline nodes ---
[jira] [Created] (IGNITE-8604) .NET test failures probably after IGNITE-5789 merge
Dmitriy Pavlov created IGNITE-8604: -- Summary: .NET test failures probably after IGNITE-5789 merge Key: IGNITE-8604 URL: https://issues.apache.org/jira/browse/IGNITE-8604 Project: Ignite Issue Type: Improvement Reporter: Dmitriy Pavlov Assignee: Ilya Kasnacheev 62 new failed tests in .NET {noformat} Store.CacheStoreTest Current failure:refs/heads/master #254 Changes (8) 24 May 18 02:16 First failure: refs/heads/master #1 No changes 26 Apr 18 10:36 TearDown method failed. HandleRegistry is not empty in grid '' (expected 4, actual 5): '[2, Apache.Ignite.Core.Impl.Cache.Store.CacheStore] [3, Apache.Ignite.Core.Impl.Cache.Store.CacheStore] [4, Apache.Ignite.Core.Impl.Cache.Store.CacheStore] [5, Apache.Ignite.Core.Impl.Cache.Store.CacheStore]' at NUnit.Framework.Assert.Fail(String message, Object[] args) at Apache.Ignite.Core.Tests.TestUtils.AssertHandleRegistryHasItems(IIgnite grid, Int32 expectedCount, Int32 timeout) in c:\BuildAgent\work\c182b70f2dfa6507\modules\platforms\dotnet\Apache.Ignite.Core.Tests\TestUtils.Common.cs:line 339 at Apache.Ignite.Core.Tests.Cache.Store.CacheStoreTest.AfterTests {noformat} First time these test failed https://ci.ignite.apache.org/viewLog.html?buildId=1323959=buildResultsDiv=IgniteTests24Java8_PlatformNet Probably it is caused by merge: https://issues.apache.org/jira/browse/IGNITE-5789 I've tried to revert commit in ignite-5789-1 branch and results: https://ci.ignite.apache.org/viewLog.html?buildId=1326520=buildResultsDiv=IgniteTests24Java8_PlatformNet Other test failed, but current tests were passed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8603) Add JMX-metric to cluster: baseline nodes
Dmitriy Gladkikh created IGNITE-8603: Summary: Add JMX-metric to cluster: baseline nodes Key: IGNITE-8603 URL: https://issues.apache.org/jira/browse/IGNITE-8603 Project: Ignite Issue Type: Improvement Reporter: Dmitriy Gladkikh Fix For: 2.6 Need to add a baseline nodes on JMX: {code:java} int org.apache.ignite.mxbean.ClusterMetricsMXBean#getBaselineNodes {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
MTCGA & 6th mass run-all
Hi, A lot of Igniters made their contributions to Make TC Green during 6 last weeks. It is Aleksey Plekhanov, Maxim Muzafarov, Alexey Goncharuk, Ilya Lantukh, Ivan Rakov, Alexey Kuznetsov, Taras Ledkov, Andrey Kuznetsov, Alexei Scherbakov, Ivan Fedotov, Alexander Menshikov, Andrew Medvedev, Andrew Mashenkov, Nikolay Izhikov, Stanilovsky Evgeny, Vyacheslav Daradur, Alexey Kuznetsov, Pavel Kovalenko, Vitaliy Biryukov, Alexey Kukushkin, Pavel Pereslegin, Amelchev Nikita. Folks, let me say thank you for your contributions. This effort will definitely move us towards day with 0 fails. It’s time to check progress, so I’m going to schedule mass run-all's during weekend. I plan to trigger these builds Fri May 25 at 15:00 GMT, 18:00 MSK. Please reply if you have objections or need TC at this time. Feel free to move builds to top if you have needed fixes. Any finished patch review process for MTCGA PA issues is also appreciated. Sincerely, Dmitriy Pavlov
Re: Ability to check and completely fill transactions on creation
Anton, why do you need to *alter* event sub-system to introduce a new event? Yakov's issue was that you propagated private interface to public API, which is bad of course. Come up with a clean design and it will be accepted. My problem with TransactionValidator is that it only solves a small problem for transactions. If we do that, then we will have to add cache validators, compute validators, etc, etc, etc. That is why we either should use the existing event subsystem or come up with a holistic design that will work across the whole project. D. On Thu, May 24, 2018 at 1:38 AM, Anton Vinogradovwrote: > Dmitriy, > > Yakov is against the solution based on event sub-system > >> I think that we should think about some other solution instead of > altering > >> event sub-system. > > Also, I checked is there any chances to fix all the issues found by Yakov > and see that solution becomes overcomplicated in that case. > That's why I'm proposing this lightweight solution. > > As for me it's a good idea to have such validator since that's a common > problem at huge deployments when more than one team have access to Ignite > cluster and there is no other way to setup tx cretion rules. > > Yakov, > > Could you please share your thoughts on that? > > > чт, 24 мая 2018 г. в 8:58, Dmitriy Setrakyan : > > > On Wed, May 23, 2018 at 4:08 AM, Anton Vinogradov wrote: > > > > > Dmitriy, Yakov > > > > > > Are there any objections to updated design taking into account the > > comments > > > I provided? > > > > > > > Anton, I do not like an additional validator. I think you can accomplish > > the same with a transaction event. You just need to design it more > cleanly, > > incorporating the feedback from Yakov. > > >
Re: Cache 6 Suite Execution timeouts
Hi, guys. Thanks for pointing this out. I'll look at this then I'll have available time. Possibly next week. Meanwhile I suggest not to use async rollbacks for transactions on near caches. чт, 24 мая 2018 г. в 14:09, Dmitry Pavlov: > Alexey? Could I hope you will pick up > https://issues.apache.org/jira/browse/IGNITE-8509? > > чт, 24 мая 2018 г. в 14:00, Maxim Muzafarov : > >> Hello, >> >> Just bump this topic up. >> >> 6/10 execution timeouts for the last 10 build in master! >> >> Will anyone help? >> >> [1] >> >> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Cache6=1=buildTypeHistoryList_IgniteTests24Java8=%3Cdefault%3E >> >> пн, 21 мая 2018 г. в 12:58, Dmitry Pavlov : >> >> > Hi Maxim, >> > >> > I observe the same, Cache 6 critical recent fails: 18,0% [9 fails / 50 >> > runs] >> > >> > I hope Aleksei Scherbakov can help here. Alexey, please step in. >> > >> > Sincerely, >> > Dmitriy Pavlov >> > >> > вс, 20 мая 2018 г. в 20:37, Maxim Muzafarov : >> > >> > > Hello Igniters, >> > > >> > > Have anyone noticed a lot of execution timeouts for Cache 6 Suite? >> > > Since recently it fails too much on TxRollbackOnTimeoutNearCacheTest >> with >> > > "Found long running transaction". >> > > >> > > Please refer to TC [1]. It has 4/10 execution timeouts for the last 10 >> > > builds in master. >> > > >> > > I've created IGNITE-8509 [2]. Will someone have time to look at? Or >> I'll >> > > try when I'm get free. >> > > >> > > [1] >> > > >> > > >> > >> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Cache6_IgniteTests24Java8=%3Cdefault%3E=buildTypeStatusDiv >> > > [2] https://issues.apache.org/jira/browse/IGNITE-8509 >> > > >> > >> > -- Best regards, Alexei Scherbakov
[jira] [Created] (IGNITE-8602) Add support filter label=null for control.sh tx utility
Dmitry Sherstobitov created IGNITE-8602: --- Summary: Add support filter label=null for control.sh tx utility Key: IGNITE-8602 URL: https://issues.apache.org/jira/browse/IGNITE-8602 Project: Ignite Issue Type: Improvement Affects Versions: 2.5 Reporter: Dmitry Sherstobitov -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: async operation is not fair async
Alexey Goncharuk, I remember we started working on async connection establishment. This should fix latency issue related to network which I believe gives the most contribution to overall latency. Mapping logic and other stuff can be ignored as it can very rarely be an issue at least on stable tolopogies. What is the status with async connections? That would really be a huge improvement! Also please remember that uncontrolled async operations may lead to OOME, therefore at some point when there are too many uncompleted async operations newly invoked async operations should become synchronous, i.e. we should return completed future, ignoring the fact that user expected us to be async. I would like to have very strong reasons to start reapproaching this. --Yakov
[jira] [Created] (IGNITE-8601) Add to control.sh tx utility information about transaction start time
Dmitry Sherstobitov created IGNITE-8601: --- Summary: Add to control.sh tx utility information about transaction start time Key: IGNITE-8601 URL: https://issues.apache.org/jira/browse/IGNITE-8601 Project: Ignite Issue Type: Improvement Affects Versions: 2.5 Reporter: Dmitry Sherstobitov This information will be useful -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Ticket review checklist
Petr, good point. It is more intuitive, we should mark test we can ignore by mute. So Vladimir, you or other Ignite veteran can mute test, if can say it is not important. чт, 24 мая 2018 г. в 15:07, Petr Ivanov: > Why cannot we mute (and file corresponding tickets) all test failures > (including flaky) to some date and start initiative Green TC? > > > > > On 24 May 2018, at 15:04, Vladimir Ozerov wrote: > > > > Dmitry, > > > > We cannot add this requirements, because we do have failures on TC. This > > requirement implies that all development would stop until TC is green. > > We never had old requirement work, neither we need to enforce it now. > > > > On Thu, May 24, 2018 at 2:59 PM, Dmitry Pavlov > > wrote: > > > >> 3.c > >> > >> 1. All test suites *MUST* be run on TeamCity [3] before merge to > master, > >> there *MUST NOT* be any test failures > >> > >> > >> 'New' word should be removed because we cant separate `new` and `non > new` > >> failures. > >> > >> Let's imagine example, we have 50 green runs in master. And PR Run-All > >> contains this test failed. Is it new or not new? Actually we don't know. > >> > >> Existing requirement is about all TC must be green, so let's keep it as > is. > >> > >> ср, 23 мая 2018 г. в 17:02, Vladimir Ozerov : > >> > >>> Igniters, > >>> > >>> I created review checklist on WIKI [1] and also fixed related pages > (e.g. > >>> "How To Contribute"). Please let me know if you have any comments > before > >> I > >>> go with public announce. > >>> > >>> Vladimir. > >>> > >>> [1] > https://cwiki.apache.org/confluence/display/IGNITE/Review+Checklist > >>> > >>> On Thu, May 10, 2018 at 5:10 PM, Vladimir Ozerov > > >>> wrote: > >>> > Ilya, > > We define that exception messages *SHOULD* have clear explanation on > >> what > is wrong. *SHOULD* mean that the rule should be followed unless there > >> is > >>> a > reason not to follow. In your case you refer to some unexpected > >> behavior. > I.e. an exceptional situation developer is not aware of. In this case > >> for > sure we cannot force contributor to explain what is wrong, because, > >> well, > we don't know. This is why we relaxed the rule from *MUST* to > *SHOULD*. > > On Thu, May 10, 2018 at 3:50 PM, Ilya Kasnacheev < > ilya.kasnach...@gmail.com> wrote: > > > I don't think I quite understand how exception explanations should > >> work. > > > > Imagine we have the following exception: > > > > // At least RuntimeException can be thrown by the code above when > > GridCacheContext is cleaned and there is > > // an attempt to use cleaned resources. > > U.error(log, "Unexpected exception during cache update", e); > > > > I mean, we genuinely don't know what happened here. > > > > Under new rules, what kind of "workaround" would that exception > >> suggest? > > "Try turning it off and then back on"? > > What explanation how to resolve this exception can we offer? "Please > >>> write > > to d...@apache.ignite.org or to Apache JIRA, and then wait for a > >> release > > with fix?" > > > > I'm really confused how we can implement 1.6 and 1.7 when dealing > with > > messy real-world code. > > > > Regards, > > > > > > -- > > Ilya Kasnacheev > > > > 2018-05-10 11:39 GMT+03:00 Vladimir Ozerov : > > > >> Andrey, Anton, Alex > >> > >> Agree, *SHOULD* is more appropriate here. > >> > >> Please see latest version below. Does anyone want to add or change > >> something? Let's wait for several days for more feedback and then > > publish > >> and announce this list. Note that it would not be carved in stone > >> and > >>> we > >> will be able to change it at any time if needed. > >> > >> 1) API > >> 1.1) API compatibility *MUST* be maintained between minor releases. > >> Do > > not > >> remove existing methods or change their signatures, deprecate them > > instead > >> 1.2) Default behaviour "SHOULD NOT* be changed between minor > >> releases, > >> unless absolutely needed. If change is made, it *MUST* be described > >> in > >> "Migration Guide" > >> 1.3) New operation *MUST* be well-documented in code (javadoc, > > dotnetdoc): > >> documentation must contain method's purpose, description of > >> parameters > > and > >> how their values affect the outcome, description of return value and > > it's > >> default, behavior in negative cases, interaction with other > >> operations > > and > >> components > >> 1.4) API parity between Java and .NET platforms *SHOULD* be > >> maintained > > when > >> operation makes sense on both platforms. If method cannot be > > implemented in > >> a platform immediately, new JIRA
Re: Ticket review checklist
We cannot change this requirement to be softer because we need to come to sutuation of 0-failed test. If we allow commit with test failures, there will be a lot of mistakes new failures will be considered as existing. All contributors will check only new/not new failures. But actually all failures should be checked. чт, 24 мая 2018 г. в 15:04, Vladimir Ozerov: > Dmitry, > > We cannot add this requirements, because we do have failures on TC. This > requirement implies that all development would stop until TC is green. > We never had old requirement work, neither we need to enforce it now. > > On Thu, May 24, 2018 at 2:59 PM, Dmitry Pavlov > wrote: > > > 3.c > > > >1. All test suites *MUST* be run on TeamCity [3] before merge to > master, > >there *MUST NOT* be any test failures > > > > > > 'New' word should be removed because we cant separate `new` and `non new` > > failures. > > > > Let's imagine example, we have 50 green runs in master. And PR Run-All > > contains this test failed. Is it new or not new? Actually we don't know. > > > > Existing requirement is about all TC must be green, so let's keep it as > is. > > > > ср, 23 мая 2018 г. в 17:02, Vladimir Ozerov : > > > > > Igniters, > > > > > > I created review checklist on WIKI [1] and also fixed related pages > (e.g. > > > "How To Contribute"). Please let me know if you have any comments > before > > I > > > go with public announce. > > > > > > Vladimir. > > > > > > [1] > https://cwiki.apache.org/confluence/display/IGNITE/Review+Checklist > > > > > > On Thu, May 10, 2018 at 5:10 PM, Vladimir Ozerov > > > > wrote: > > > > > > > Ilya, > > > > > > > > We define that exception messages *SHOULD* have clear explanation on > > what > > > > is wrong. *SHOULD* mean that the rule should be followed unless there > > is > > > a > > > > reason not to follow. In your case you refer to some unexpected > > behavior. > > > > I.e. an exceptional situation developer is not aware of. In this case > > for > > > > sure we cannot force contributor to explain what is wrong, because, > > well, > > > > we don't know. This is why we relaxed the rule from *MUST* to > *SHOULD*. > > > > > > > > On Thu, May 10, 2018 at 3:50 PM, Ilya Kasnacheev < > > > > ilya.kasnach...@gmail.com> wrote: > > > > > > > >> I don't think I quite understand how exception explanations should > > work. > > > >> > > > >> Imagine we have the following exception: > > > >> > > > >> // At least RuntimeException can be thrown by the code above when > > > >> GridCacheContext is cleaned and there is > > > >> // an attempt to use cleaned resources. > > > >> U.error(log, "Unexpected exception during cache update", e); > > > >> > > > >> I mean, we genuinely don't know what happened here. > > > >> > > > >> Under new rules, what kind of "workaround" would that exception > > suggest? > > > >> "Try turning it off and then back on"? > > > >> What explanation how to resolve this exception can we offer? "Please > > > write > > > >> to d...@apache.ignite.org or to Apache JIRA, and then wait for a > > release > > > >> with fix?" > > > >> > > > >> I'm really confused how we can implement 1.6 and 1.7 when dealing > with > > > >> messy real-world code. > > > >> > > > >> Regards, > > > >> > > > >> > > > >> -- > > > >> Ilya Kasnacheev > > > >> > > > >> 2018-05-10 11:39 GMT+03:00 Vladimir Ozerov : > > > >> > > > >> > Andrey, Anton, Alex > > > >> > > > > >> > Agree, *SHOULD* is more appropriate here. > > > >> > > > > >> > Please see latest version below. Does anyone want to add or change > > > >> > something? Let's wait for several days for more feedback and then > > > >> publish > > > >> > and announce this list. Note that it would not be carved in stone > > and > > > we > > > >> > will be able to change it at any time if needed. > > > >> > > > > >> > 1) API > > > >> > 1.1) API compatibility *MUST* be maintained between minor > releases. > > Do > > > >> not > > > >> > remove existing methods or change their signatures, deprecate them > > > >> instead > > > >> > 1.2) Default behaviour "SHOULD NOT* be changed between minor > > releases, > > > >> > unless absolutely needed. If change is made, it *MUST* be > described > > in > > > >> > "Migration Guide" > > > >> > 1.3) New operation *MUST* be well-documented in code (javadoc, > > > >> dotnetdoc): > > > >> > documentation must contain method's purpose, description of > > parameters > > > >> and > > > >> > how their values affect the outcome, description of return value > and > > > >> it's > > > >> > default, behavior in negative cases, interaction with other > > operations > > > >> and > > > >> > components > > > >> > 1.4) API parity between Java and .NET platforms *SHOULD* be > > maintained > > > >> when > > > >> > operation makes sense on both platforms. If method cannot be > > > >> implemented in > > > >> > a platform immediately, new JIRA ticket *MUST* be
Re: Ticket review checklist
Why cannot we mute (and file corresponding tickets) all test failures (including flaky) to some date and start initiative Green TC? > On 24 May 2018, at 15:04, Vladimir Ozerovwrote: > > Dmitry, > > We cannot add this requirements, because we do have failures on TC. This > requirement implies that all development would stop until TC is green. > We never had old requirement work, neither we need to enforce it now. > > On Thu, May 24, 2018 at 2:59 PM, Dmitry Pavlov > wrote: > >> 3.c >> >> 1. All test suites *MUST* be run on TeamCity [3] before merge to master, >> there *MUST NOT* be any test failures >> >> >> 'New' word should be removed because we cant separate `new` and `non new` >> failures. >> >> Let's imagine example, we have 50 green runs in master. And PR Run-All >> contains this test failed. Is it new or not new? Actually we don't know. >> >> Existing requirement is about all TC must be green, so let's keep it as is. >> >> ср, 23 мая 2018 г. в 17:02, Vladimir Ozerov : >> >>> Igniters, >>> >>> I created review checklist on WIKI [1] and also fixed related pages (e.g. >>> "How To Contribute"). Please let me know if you have any comments before >> I >>> go with public announce. >>> >>> Vladimir. >>> >>> [1] https://cwiki.apache.org/confluence/display/IGNITE/Review+Checklist >>> >>> On Thu, May 10, 2018 at 5:10 PM, Vladimir Ozerov >>> wrote: >>> Ilya, We define that exception messages *SHOULD* have clear explanation on >> what is wrong. *SHOULD* mean that the rule should be followed unless there >> is >>> a reason not to follow. In your case you refer to some unexpected >> behavior. I.e. an exceptional situation developer is not aware of. In this case >> for sure we cannot force contributor to explain what is wrong, because, >> well, we don't know. This is why we relaxed the rule from *MUST* to *SHOULD*. On Thu, May 10, 2018 at 3:50 PM, Ilya Kasnacheev < ilya.kasnach...@gmail.com> wrote: > I don't think I quite understand how exception explanations should >> work. > > Imagine we have the following exception: > > // At least RuntimeException can be thrown by the code above when > GridCacheContext is cleaned and there is > // an attempt to use cleaned resources. > U.error(log, "Unexpected exception during cache update", e); > > I mean, we genuinely don't know what happened here. > > Under new rules, what kind of "workaround" would that exception >> suggest? > "Try turning it off and then back on"? > What explanation how to resolve this exception can we offer? "Please >>> write > to d...@apache.ignite.org or to Apache JIRA, and then wait for a >> release > with fix?" > > I'm really confused how we can implement 1.6 and 1.7 when dealing with > messy real-world code. > > Regards, > > > -- > Ilya Kasnacheev > > 2018-05-10 11:39 GMT+03:00 Vladimir Ozerov : > >> Andrey, Anton, Alex >> >> Agree, *SHOULD* is more appropriate here. >> >> Please see latest version below. Does anyone want to add or change >> something? Let's wait for several days for more feedback and then > publish >> and announce this list. Note that it would not be carved in stone >> and >>> we >> will be able to change it at any time if needed. >> >> 1) API >> 1.1) API compatibility *MUST* be maintained between minor releases. >> Do > not >> remove existing methods or change their signatures, deprecate them > instead >> 1.2) Default behaviour "SHOULD NOT* be changed between minor >> releases, >> unless absolutely needed. If change is made, it *MUST* be described >> in >> "Migration Guide" >> 1.3) New operation *MUST* be well-documented in code (javadoc, > dotnetdoc): >> documentation must contain method's purpose, description of >> parameters > and >> how their values affect the outcome, description of return value and > it's >> default, behavior in negative cases, interaction with other >> operations > and >> components >> 1.4) API parity between Java and .NET platforms *SHOULD* be >> maintained > when >> operation makes sense on both platforms. If method cannot be > implemented in >> a platform immediately, new JIRA ticket *MUST* be created and linked >>> to >> current ticket >> 1.5) API parity between thin clients (Java, .NET) *SHOULD* be >>> maintained >> when operation makes sense on several clients. If method cannot be >> implemented in a client immediately, new JIRA ticket *MUST* be >> created > and >> linked to current ticket >> 1.6) All exceptions thrown to a user **SHOULD** have explanation how >>> to >> resolve, workaround or debug an error >> >> 2)
Re: Ticket review checklist
Dmitry, We cannot add this requirements, because we do have failures on TC. This requirement implies that all development would stop until TC is green. We never had old requirement work, neither we need to enforce it now. On Thu, May 24, 2018 at 2:59 PM, Dmitry Pavlovwrote: > 3.c > >1. All test suites *MUST* be run on TeamCity [3] before merge to master, >there *MUST NOT* be any test failures > > > 'New' word should be removed because we cant separate `new` and `non new` > failures. > > Let's imagine example, we have 50 green runs in master. And PR Run-All > contains this test failed. Is it new or not new? Actually we don't know. > > Existing requirement is about all TC must be green, so let's keep it as is. > > ср, 23 мая 2018 г. в 17:02, Vladimir Ozerov : > > > Igniters, > > > > I created review checklist on WIKI [1] and also fixed related pages (e.g. > > "How To Contribute"). Please let me know if you have any comments before > I > > go with public announce. > > > > Vladimir. > > > > [1] https://cwiki.apache.org/confluence/display/IGNITE/Review+Checklist > > > > On Thu, May 10, 2018 at 5:10 PM, Vladimir Ozerov > > wrote: > > > > > Ilya, > > > > > > We define that exception messages *SHOULD* have clear explanation on > what > > > is wrong. *SHOULD* mean that the rule should be followed unless there > is > > a > > > reason not to follow. In your case you refer to some unexpected > behavior. > > > I.e. an exceptional situation developer is not aware of. In this case > for > > > sure we cannot force contributor to explain what is wrong, because, > well, > > > we don't know. This is why we relaxed the rule from *MUST* to *SHOULD*. > > > > > > On Thu, May 10, 2018 at 3:50 PM, Ilya Kasnacheev < > > > ilya.kasnach...@gmail.com> wrote: > > > > > >> I don't think I quite understand how exception explanations should > work. > > >> > > >> Imagine we have the following exception: > > >> > > >> // At least RuntimeException can be thrown by the code above when > > >> GridCacheContext is cleaned and there is > > >> // an attempt to use cleaned resources. > > >> U.error(log, "Unexpected exception during cache update", e); > > >> > > >> I mean, we genuinely don't know what happened here. > > >> > > >> Under new rules, what kind of "workaround" would that exception > suggest? > > >> "Try turning it off and then back on"? > > >> What explanation how to resolve this exception can we offer? "Please > > write > > >> to d...@apache.ignite.org or to Apache JIRA, and then wait for a > release > > >> with fix?" > > >> > > >> I'm really confused how we can implement 1.6 and 1.7 when dealing with > > >> messy real-world code. > > >> > > >> Regards, > > >> > > >> > > >> -- > > >> Ilya Kasnacheev > > >> > > >> 2018-05-10 11:39 GMT+03:00 Vladimir Ozerov : > > >> > > >> > Andrey, Anton, Alex > > >> > > > >> > Agree, *SHOULD* is more appropriate here. > > >> > > > >> > Please see latest version below. Does anyone want to add or change > > >> > something? Let's wait for several days for more feedback and then > > >> publish > > >> > and announce this list. Note that it would not be carved in stone > and > > we > > >> > will be able to change it at any time if needed. > > >> > > > >> > 1) API > > >> > 1.1) API compatibility *MUST* be maintained between minor releases. > Do > > >> not > > >> > remove existing methods or change their signatures, deprecate them > > >> instead > > >> > 1.2) Default behaviour "SHOULD NOT* be changed between minor > releases, > > >> > unless absolutely needed. If change is made, it *MUST* be described > in > > >> > "Migration Guide" > > >> > 1.3) New operation *MUST* be well-documented in code (javadoc, > > >> dotnetdoc): > > >> > documentation must contain method's purpose, description of > parameters > > >> and > > >> > how their values affect the outcome, description of return value and > > >> it's > > >> > default, behavior in negative cases, interaction with other > operations > > >> and > > >> > components > > >> > 1.4) API parity between Java and .NET platforms *SHOULD* be > maintained > > >> when > > >> > operation makes sense on both platforms. If method cannot be > > >> implemented in > > >> > a platform immediately, new JIRA ticket *MUST* be created and linked > > to > > >> > current ticket > > >> > 1.5) API parity between thin clients (Java, .NET) *SHOULD* be > > maintained > > >> > when operation makes sense on several clients. If method cannot be > > >> > implemented in a client immediately, new JIRA ticket *MUST* be > created > > >> and > > >> > linked to current ticket > > >> > 1.6) All exceptions thrown to a user **SHOULD** have explanation how > > to > > >> > resolve, workaround or debug an error > > >> > > > >> > 2) Compatibility > > >> > 2.1) Persistence backward compatibility *MUST* be maintained between > > >> minor > > >> > releases. It should be possible to start newer version on data
Re: Baseline topology documentation clarified: usage scenarios and definition
Hi Ivan, sure, I will. чт, 24 мая 2018 г. в 14:22, Ivan Rakov: > Well, let's give it a try. > Dmitry P., will you help me with organization of gotomeeting webinar and > announcement? I think, I'll be ready to do it at the end of the next week. > > Best Regards, > Ivan Rakov > > On 24.05.2018 14:15, Vyacheslav Daradur wrote: > > +1 for GoToMeeting webinar. > > > > On Thu, May 24, 2018 at 2:09 PM, Dmitriy Govorukhin > > wrote: > >> +1 for a webinar, I would like to see it. > >> > >> On Wed, May 23, 2018 at 5:34 PM, Dmitry Pavlov > >> wrote: > >> > >>> Hi Ivan, > >>> > >>> Would you like to run presentation based on this slides for Igniters? > >>> > >>> We can set up webinar using gotomeeting if you like this idea. > >>> > >>> Sincerely, > >>> Dmitriy Pavlov > >>> > >>> ср, 23 мая 2018 г. в 0:19, Ivan Rakov : > >>> > I used to present some slides introducing baseline topology concept. > Please feel free to use or change any pictures. > > https://docs.google.com/presentation/d/1DVLwlaR0zPIuL5YMkX8MyGLutZw- > >>> tvK9InDTCvubAEQ/edit?usp=sharing > Best Regards, > Ivan Rakov > > On 18.05.2018 15:20, Dmitriy Setrakyan wrote: > > Great docs! Any chance we could add some pictures to illustrate the > concept > > better. > > > > On Fri, May 18, 2018 at 2:15 AM, Denis Magda > >>> wrote: > >> Igniters, > >> > >> With the help of Stanislav Lukyanov and Ivan Rakov, we could make > our > >> baseline topology documentation much better and vivid. Check up the > >>> new > >> sections that did a better job explaining the topology, cover common > usage > >> scenarious and explain how to trigger the rebalancing > programmatically > if > >> it's not expected that the baseline topology's node count to be > recovered > >> soon: > >> > >> - https://apacheignite.readme.io/v2.4/docs/cluster- > >> activation#section-baseline-topology > >> < > https://apacheignite.readme.io/v2.4/docs/cluster- > >>> activation#section-baseline-topology > >> - https://apacheignite.readme.io/v2.4/docs/cluster- > >> activation#section-usage-scenarios > >> < > https://apacheignite.readme.io/v2.4/docs/cluster- > >>> activation#section-usage-scenarios > >> - https://apacheignite.readme.io/v2.4/docs/cluster- > >> activation#section-triggering-rebalancing-programmatically > >> < > https://apacheignite.readme.io/v2.4/docs/cluster- > >>> activation#section-triggering-rebalancing-programmatically > >> -- > >> Denis > >> > > > > > > >
Re: Ticket review checklist
3.c 1. All test suites *MUST* be run on TeamCity [3] before merge to master, there *MUST NOT* be any test failures 'New' word should be removed because we cant separate `new` and `non new` failures. Let's imagine example, we have 50 green runs in master. And PR Run-All contains this test failed. Is it new or not new? Actually we don't know. Existing requirement is about all TC must be green, so let's keep it as is. ср, 23 мая 2018 г. в 17:02, Vladimir Ozerov: > Igniters, > > I created review checklist on WIKI [1] and also fixed related pages (e.g. > "How To Contribute"). Please let me know if you have any comments before I > go with public announce. > > Vladimir. > > [1] https://cwiki.apache.org/confluence/display/IGNITE/Review+Checklist > > On Thu, May 10, 2018 at 5:10 PM, Vladimir Ozerov > wrote: > > > Ilya, > > > > We define that exception messages *SHOULD* have clear explanation on what > > is wrong. *SHOULD* mean that the rule should be followed unless there is > a > > reason not to follow. In your case you refer to some unexpected behavior. > > I.e. an exceptional situation developer is not aware of. In this case for > > sure we cannot force contributor to explain what is wrong, because, well, > > we don't know. This is why we relaxed the rule from *MUST* to *SHOULD*. > > > > On Thu, May 10, 2018 at 3:50 PM, Ilya Kasnacheev < > > ilya.kasnach...@gmail.com> wrote: > > > >> I don't think I quite understand how exception explanations should work. > >> > >> Imagine we have the following exception: > >> > >> // At least RuntimeException can be thrown by the code above when > >> GridCacheContext is cleaned and there is > >> // an attempt to use cleaned resources. > >> U.error(log, "Unexpected exception during cache update", e); > >> > >> I mean, we genuinely don't know what happened here. > >> > >> Under new rules, what kind of "workaround" would that exception suggest? > >> "Try turning it off and then back on"? > >> What explanation how to resolve this exception can we offer? "Please > write > >> to d...@apache.ignite.org or to Apache JIRA, and then wait for a release > >> with fix?" > >> > >> I'm really confused how we can implement 1.6 and 1.7 when dealing with > >> messy real-world code. > >> > >> Regards, > >> > >> > >> -- > >> Ilya Kasnacheev > >> > >> 2018-05-10 11:39 GMT+03:00 Vladimir Ozerov : > >> > >> > Andrey, Anton, Alex > >> > > >> > Agree, *SHOULD* is more appropriate here. > >> > > >> > Please see latest version below. Does anyone want to add or change > >> > something? Let's wait for several days for more feedback and then > >> publish > >> > and announce this list. Note that it would not be carved in stone and > we > >> > will be able to change it at any time if needed. > >> > > >> > 1) API > >> > 1.1) API compatibility *MUST* be maintained between minor releases. Do > >> not > >> > remove existing methods or change their signatures, deprecate them > >> instead > >> > 1.2) Default behaviour "SHOULD NOT* be changed between minor releases, > >> > unless absolutely needed. If change is made, it *MUST* be described in > >> > "Migration Guide" > >> > 1.3) New operation *MUST* be well-documented in code (javadoc, > >> dotnetdoc): > >> > documentation must contain method's purpose, description of parameters > >> and > >> > how their values affect the outcome, description of return value and > >> it's > >> > default, behavior in negative cases, interaction with other operations > >> and > >> > components > >> > 1.4) API parity between Java and .NET platforms *SHOULD* be maintained > >> when > >> > operation makes sense on both platforms. If method cannot be > >> implemented in > >> > a platform immediately, new JIRA ticket *MUST* be created and linked > to > >> > current ticket > >> > 1.5) API parity between thin clients (Java, .NET) *SHOULD* be > maintained > >> > when operation makes sense on several clients. If method cannot be > >> > implemented in a client immediately, new JIRA ticket *MUST* be created > >> and > >> > linked to current ticket > >> > 1.6) All exceptions thrown to a user **SHOULD** have explanation how > to > >> > resolve, workaround or debug an error > >> > > >> > 2) Compatibility > >> > 2.1) Persistence backward compatibility *MUST* be maintained between > >> minor > >> > releases. It should be possible to start newer version on data files > >> > created by the previous version > >> > 2.2) Thin client forward and backward compatibility *SHOULD* be > >> maintained > >> > between two consecutive minor releases. If compatibility cannot be > >> > maintained it *MUST* be described in "Migration Guide" > >> > 2.3) JDBC and ODBC forward and backward compatibility *SHOULD* be > >> > maintained between two consecutive minor releases. If compatibility > >> cannot > >> > be maintained it *MUST* be described in "Migration Guide" > >> > > >> > 3) Tests > >> > 3.1) New functionality *MUST* be covered with
[GitHub] ignite pull request #3046: IGNITE-6612: Wrap ack methods in their own class
Github user 1vanan closed the pull request at: https://github.com/apache/ignite/pull/3046 ---
Re: Ticket review checklist
Dmitry, Could you please formulate this requirement? On Wed, May 23, 2018 at 5:06 PM, Dmitry Pavlovwrote: > Hi Vladimir, > > I've replied in separate thread. > > I would like to keep requirement of Green TC instead of separation to > new/old test failures. > > Once you allow just one test failure, it would be more failures after some > time. > > Sincererly, > Dmitriy Pavlov > > ср, 23 мая 2018 г. в 17:02, Vladimir Ozerov : > > > Igniters, > > > > I created review checklist on WIKI [1] and also fixed related pages (e.g. > > "How To Contribute"). Please let me know if you have any comments before > I > > go with public announce. > > > > Vladimir. > > > > [1] https://cwiki.apache.org/confluence/display/IGNITE/Review+Checklist > > > > On Thu, May 10, 2018 at 5:10 PM, Vladimir Ozerov > > wrote: > > > > > Ilya, > > > > > > We define that exception messages *SHOULD* have clear explanation on > what > > > is wrong. *SHOULD* mean that the rule should be followed unless there > is > > a > > > reason not to follow. In your case you refer to some unexpected > behavior. > > > I.e. an exceptional situation developer is not aware of. In this case > for > > > sure we cannot force contributor to explain what is wrong, because, > well, > > > we don't know. This is why we relaxed the rule from *MUST* to *SHOULD*. > > > > > > On Thu, May 10, 2018 at 3:50 PM, Ilya Kasnacheev < > > > ilya.kasnach...@gmail.com> wrote: > > > > > >> I don't think I quite understand how exception explanations should > work. > > >> > > >> Imagine we have the following exception: > > >> > > >> // At least RuntimeException can be thrown by the code above when > > >> GridCacheContext is cleaned and there is > > >> // an attempt to use cleaned resources. > > >> U.error(log, "Unexpected exception during cache update", e); > > >> > > >> I mean, we genuinely don't know what happened here. > > >> > > >> Under new rules, what kind of "workaround" would that exception > suggest? > > >> "Try turning it off and then back on"? > > >> What explanation how to resolve this exception can we offer? "Please > > write > > >> to d...@apache.ignite.org or to Apache JIRA, and then wait for a > release > > >> with fix?" > > >> > > >> I'm really confused how we can implement 1.6 and 1.7 when dealing with > > >> messy real-world code. > > >> > > >> Regards, > > >> > > >> > > >> -- > > >> Ilya Kasnacheev > > >> > > >> 2018-05-10 11:39 GMT+03:00 Vladimir Ozerov : > > >> > > >> > Andrey, Anton, Alex > > >> > > > >> > Agree, *SHOULD* is more appropriate here. > > >> > > > >> > Please see latest version below. Does anyone want to add or change > > >> > something? Let's wait for several days for more feedback and then > > >> publish > > >> > and announce this list. Note that it would not be carved in stone > and > > we > > >> > will be able to change it at any time if needed. > > >> > > > >> > 1) API > > >> > 1.1) API compatibility *MUST* be maintained between minor releases. > Do > > >> not > > >> > remove existing methods or change their signatures, deprecate them > > >> instead > > >> > 1.2) Default behaviour "SHOULD NOT* be changed between minor > releases, > > >> > unless absolutely needed. If change is made, it *MUST* be described > in > > >> > "Migration Guide" > > >> > 1.3) New operation *MUST* be well-documented in code (javadoc, > > >> dotnetdoc): > > >> > documentation must contain method's purpose, description of > parameters > > >> and > > >> > how their values affect the outcome, description of return value and > > >> it's > > >> > default, behavior in negative cases, interaction with other > operations > > >> and > > >> > components > > >> > 1.4) API parity between Java and .NET platforms *SHOULD* be > maintained > > >> when > > >> > operation makes sense on both platforms. If method cannot be > > >> implemented in > > >> > a platform immediately, new JIRA ticket *MUST* be created and linked > > to > > >> > current ticket > > >> > 1.5) API parity between thin clients (Java, .NET) *SHOULD* be > > maintained > > >> > when operation makes sense on several clients. If method cannot be > > >> > implemented in a client immediately, new JIRA ticket *MUST* be > created > > >> and > > >> > linked to current ticket > > >> > 1.6) All exceptions thrown to a user **SHOULD** have explanation how > > to > > >> > resolve, workaround or debug an error > > >> > > > >> > 2) Compatibility > > >> > 2.1) Persistence backward compatibility *MUST* be maintained between > > >> minor > > >> > releases. It should be possible to start newer version on data files > > >> > created by the previous version > > >> > 2.2) Thin client forward and backward compatibility *SHOULD* be > > >> maintained > > >> > between two consecutive minor releases. If compatibility cannot be > > >> > maintained it *MUST* be described in "Migration Guide" > > >> > 2.3) JDBC and ODBC forward and backward compatibility *SHOULD*
Re: Baseline topology documentation clarified: usage scenarios and definition
Well, let's give it a try. Dmitry P., will you help me with organization of gotomeeting webinar and announcement? I think, I'll be ready to do it at the end of the next week. Best Regards, Ivan Rakov On 24.05.2018 14:15, Vyacheslav Daradur wrote: +1 for GoToMeeting webinar. On Thu, May 24, 2018 at 2:09 PM, Dmitriy Govorukhinwrote: +1 for a webinar, I would like to see it. On Wed, May 23, 2018 at 5:34 PM, Dmitry Pavlov wrote: Hi Ivan, Would you like to run presentation based on this slides for Igniters? We can set up webinar using gotomeeting if you like this idea. Sincerely, Dmitriy Pavlov ср, 23 мая 2018 г. в 0:19, Ivan Rakov : I used to present some slides introducing baseline topology concept. Please feel free to use or change any pictures. https://docs.google.com/presentation/d/1DVLwlaR0zPIuL5YMkX8MyGLutZw- tvK9InDTCvubAEQ/edit?usp=sharing Best Regards, Ivan Rakov On 18.05.2018 15:20, Dmitriy Setrakyan wrote: Great docs! Any chance we could add some pictures to illustrate the concept better. On Fri, May 18, 2018 at 2:15 AM, Denis Magda wrote: Igniters, With the help of Stanislav Lukyanov and Ivan Rakov, we could make our baseline topology documentation much better and vivid. Check up the new sections that did a better job explaining the topology, cover common usage scenarious and explain how to trigger the rebalancing programmatically if it's not expected that the baseline topology's node count to be recovered soon: - https://apacheignite.readme.io/v2.4/docs/cluster- activation#section-baseline-topology < https://apacheignite.readme.io/v2.4/docs/cluster- activation#section-baseline-topology - https://apacheignite.readme.io/v2.4/docs/cluster- activation#section-usage-scenarios < https://apacheignite.readme.io/v2.4/docs/cluster- activation#section-usage-scenarios - https://apacheignite.readme.io/v2.4/docs/cluster- activation#section-triggering-rebalancing-programmatically < https://apacheignite.readme.io/v2.4/docs/cluster- activation#section-triggering-rebalancing-programmatically -- Denis
Re: Baseline topology documentation clarified: usage scenarios and definition
+1 for GoToMeeting webinar. On Thu, May 24, 2018 at 2:09 PM, Dmitriy Govorukhinwrote: > +1 for a webinar, I would like to see it. > > On Wed, May 23, 2018 at 5:34 PM, Dmitry Pavlov > wrote: > >> Hi Ivan, >> >> Would you like to run presentation based on this slides for Igniters? >> >> We can set up webinar using gotomeeting if you like this idea. >> >> Sincerely, >> Dmitriy Pavlov >> >> ср, 23 мая 2018 г. в 0:19, Ivan Rakov : >> >> > I used to present some slides introducing baseline topology concept. >> > Please feel free to use or change any pictures. >> > >> > https://docs.google.com/presentation/d/1DVLwlaR0zPIuL5YMkX8MyGLutZw- >> tvK9InDTCvubAEQ/edit?usp=sharing >> > >> > Best Regards, >> > Ivan Rakov >> > >> > On 18.05.2018 15:20, Dmitriy Setrakyan wrote: >> > > Great docs! Any chance we could add some pictures to illustrate the >> > concept >> > > better. >> > > >> > > On Fri, May 18, 2018 at 2:15 AM, Denis Magda >> wrote: >> > > >> > >> Igniters, >> > >> >> > >> With the help of Stanislav Lukyanov and Ivan Rakov, we could make our >> > >> baseline topology documentation much better and vivid. Check up the >> new >> > >> sections that did a better job explaining the topology, cover common >> > usage >> > >> scenarious and explain how to trigger the rebalancing programmatically >> > if >> > >> it's not expected that the baseline topology's node count to be >> > recovered >> > >> soon: >> > >> >> > >> - https://apacheignite.readme.io/v2.4/docs/cluster- >> > >> activation#section-baseline-topology >> > >> < >> > https://apacheignite.readme.io/v2.4/docs/cluster- >> activation#section-baseline-topology >> > > >> > >> - https://apacheignite.readme.io/v2.4/docs/cluster- >> > >> activation#section-usage-scenarios >> > >> < >> > https://apacheignite.readme.io/v2.4/docs/cluster- >> activation#section-usage-scenarios >> > > >> > >> - https://apacheignite.readme.io/v2.4/docs/cluster- >> > >> activation#section-triggering-rebalancing-programmatically >> > >> < >> > https://apacheignite.readme.io/v2.4/docs/cluster- >> activation#section-triggering-rebalancing-programmatically >> > > >> > >> >> > >> -- >> > >> Denis >> > >> >> > >> > >> -- Best Regards, Vyacheslav D.
[jira] [Created] (IGNITE-8600) SQL: lazy row materialization
Vladimir Ozerov created IGNITE-8600: --- Summary: SQL: lazy row materialization Key: IGNITE-8600 URL: https://issues.apache.org/jira/browse/IGNITE-8600 Project: Ignite Issue Type: Task Components: sql Affects Versions: 2.5 Environment: Currently our index cursor materializes rows as soon as they are encountered in an index page. This is necessary to protect ourselves from concurrent data modification. However, materialized rows might be filtered out later due to additional filters. In addition, there is a chance that only indexed fields is needed by query. We can do the following: 1) Introduce new mode that will return partially materialized rows, with only inline index fields initialized. When some non-initialized attribute is requested, we go to data page and materialize the whole row 2) Enable this mode for MVCC by default 3) Optionally enable this mode for non-MVCC read-only mode through additional flag. Reporter: Vladimir Ozerov -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Cache 6 Suite Execution timeouts
Alexey? Could I hope you will pick up https://issues.apache.org/jira/browse/IGNITE-8509? чт, 24 мая 2018 г. в 14:00, Maxim Muzafarov: > Hello, > > Just bump this topic up. > > 6/10 execution timeouts for the last 10 build in master! > > Will anyone help? > > [1] > > https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Cache6=1=buildTypeHistoryList_IgniteTests24Java8=%3Cdefault%3E > > пн, 21 мая 2018 г. в 12:58, Dmitry Pavlov : > > > Hi Maxim, > > > > I observe the same, Cache 6 critical recent fails: 18,0% [9 fails / 50 > > runs] > > > > I hope Aleksei Scherbakov can help here. Alexey, please step in. > > > > Sincerely, > > Dmitriy Pavlov > > > > вс, 20 мая 2018 г. в 20:37, Maxim Muzafarov : > > > > > Hello Igniters, > > > > > > Have anyone noticed a lot of execution timeouts for Cache 6 Suite? > > > Since recently it fails too much on TxRollbackOnTimeoutNearCacheTest > with > > > "Found long running transaction". > > > > > > Please refer to TC [1]. It has 4/10 execution timeouts for the last 10 > > > builds in master. > > > > > > I've created IGNITE-8509 [2]. Will someone have time to look at? Or > I'll > > > try when I'm get free. > > > > > > [1] > > > > > > > > > https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Cache6_IgniteTests24Java8=%3Cdefault%3E=buildTypeStatusDiv > > > [2] https://issues.apache.org/jira/browse/IGNITE-8509 > > > > > >
Re: IGNITE-8583 - review needed
Hi Dmitriy, According to upsource https://reviews.ignite.apache.org/ignite/review/IGNT-CR-623 review was already started by Alexey G. and Alexey K. Sincerely, Dmitriy Pavlov чт, 24 мая 2018 г. в 14:04, Dmitriy Govorukhin: > Igniters, > > I fixed DataStorageMetricsMXBean.getOffHeapSize, please review my changes. > > https://issues.apache.org/jira/browse/IGNITE-8583 >
Re: Baseline topology documentation clarified: usage scenarios and definition
+1 for a webinar, I would like to see it. On Wed, May 23, 2018 at 5:34 PM, Dmitry Pavlovwrote: > Hi Ivan, > > Would you like to run presentation based on this slides for Igniters? > > We can set up webinar using gotomeeting if you like this idea. > > Sincerely, > Dmitriy Pavlov > > ср, 23 мая 2018 г. в 0:19, Ivan Rakov : > > > I used to present some slides introducing baseline topology concept. > > Please feel free to use or change any pictures. > > > > https://docs.google.com/presentation/d/1DVLwlaR0zPIuL5YMkX8MyGLutZw- > tvK9InDTCvubAEQ/edit?usp=sharing > > > > Best Regards, > > Ivan Rakov > > > > On 18.05.2018 15:20, Dmitriy Setrakyan wrote: > > > Great docs! Any chance we could add some pictures to illustrate the > > concept > > > better. > > > > > > On Fri, May 18, 2018 at 2:15 AM, Denis Magda > wrote: > > > > > >> Igniters, > > >> > > >> With the help of Stanislav Lukyanov and Ivan Rakov, we could make our > > >> baseline topology documentation much better and vivid. Check up the > new > > >> sections that did a better job explaining the topology, cover common > > usage > > >> scenarious and explain how to trigger the rebalancing programmatically > > if > > >> it's not expected that the baseline topology's node count to be > > recovered > > >> soon: > > >> > > >> - https://apacheignite.readme.io/v2.4/docs/cluster- > > >> activation#section-baseline-topology > > >> < > > https://apacheignite.readme.io/v2.4/docs/cluster- > activation#section-baseline-topology > > > > > >> - https://apacheignite.readme.io/v2.4/docs/cluster- > > >> activation#section-usage-scenarios > > >> < > > https://apacheignite.readme.io/v2.4/docs/cluster- > activation#section-usage-scenarios > > > > > >> - https://apacheignite.readme.io/v2.4/docs/cluster- > > >> activation#section-triggering-rebalancing-programmatically > > >> < > > https://apacheignite.readme.io/v2.4/docs/cluster- > activation#section-triggering-rebalancing-programmatically > > > > > >> > > >> -- > > >> Denis > > >> > > > > >
[jira] [Created] (IGNITE-8599) Remove LocalWalModeChangeDuringRebalancingSelfTest.testWithExchangesMerge from Direct IO suite
Dmitriy Pavlov created IGNITE-8599: -- Summary: Remove LocalWalModeChangeDuringRebalancingSelfTest.testWithExchangesMerge from Direct IO suite Key: IGNITE-8599 URL: https://issues.apache.org/jira/browse/IGNITE-8599 Project: Ignite Issue Type: Improvement Reporter: Dmitriy Pavlov Assignee: Dmitriy Pavlov Fix For: 2.6 https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=6346758468206865681=%3Cdefault%3E=testDetails It falls only in Direct IO It is necessary to exclude it from direct IO if it gives a lot of load. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
IGNITE-8583 - review needed
Igniters, I fixed DataStorageMetricsMXBean.getOffHeapSize, please review my changes. https://issues.apache.org/jira/browse/IGNITE-8583
Re: Cache 6 Suite Execution timeouts
Hello, Just bump this topic up. 6/10 execution timeouts for the last 10 build in master! Will anyone help? [1] https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Cache6=1=buildTypeHistoryList_IgniteTests24Java8=%3Cdefault%3E пн, 21 мая 2018 г. в 12:58, Dmitry Pavlov: > Hi Maxim, > > I observe the same, Cache 6 critical recent fails: 18,0% [9 fails / 50 > runs] > > I hope Aleksei Scherbakov can help here. Alexey, please step in. > > Sincerely, > Dmitriy Pavlov > > вс, 20 мая 2018 г. в 20:37, Maxim Muzafarov : > > > Hello Igniters, > > > > Have anyone noticed a lot of execution timeouts for Cache 6 Suite? > > Since recently it fails too much on TxRollbackOnTimeoutNearCacheTest with > > "Found long running transaction". > > > > Please refer to TC [1]. It has 4/10 execution timeouts for the last 10 > > builds in master. > > > > I've created IGNITE-8509 [2]. Will someone have time to look at? Or I'll > > try when I'm get free. > > > > [1] > > > > > https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Cache6_IgniteTests24Java8=%3Cdefault%3E=buildTypeStatusDiv > > [2] https://issues.apache.org/jira/browse/IGNITE-8509 > > >
Re: async operation is not fair async
So, in a current implementation, how I can perform the real async operation in one thread? Any workaround? What can I do if I have event loop thread model? On Wed, May 16, 2018 at 12:14 PM, Alexey Goncharuk < alexey.goncha...@gmail.com> wrote: > Dmitriy, > > I will add technical details to the ticket, however, looks like there is > still no consensus on how this change should be presented to a user. It > would be ok if we changed this behavior in Ignite 3.0, but for one of the > next point releases we have to agree how this should be enabled/disabled > (or whether we should delay this change to 3.0 at all). > > 2018-05-15 22:13 GMT+03:00 Dmitriy Govorukhin < > dmitriy.govoruk...@gmail.com> > : > > > Alexey, > > > > Any updates? > > > > On Mon, May 14, 2018 at 6:19 PM, Dmitriy Govorukhin < > > dmitriy.govoruk...@gmail.com> wrote: > > > > > Alexey, > > > > > > Could you please add more description information for this task? [1] > > > Perhaps, base steps for implementation. > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-8475 > > > > > > On Mon, May 14, 2018 at 4:58 PM, Alexey Goncharuk < > > > alexey.goncha...@gmail.com> wrote: > > > > > >> Another +1 for the true asynchronous approach. I remember a while ago > > one > > >> of the Ignite users raised a similar question regarding the *async > > method > > >> being blocked on establishing a TCP connection. > > >> > > >> As far as deadlocks go, I have a counter-example. Currently, we check > > the > > >> thread-local chain only for a single cache, so if I run the following > > >> code: > > >> cache1.getAsync(k1); > > >> cache2.getAsync(k2); > > >> then the deadlock is still possible, and I did not see a single user > > >> complaining about unexpected deadlocks. Rather than implementing this > > >> cross-cache chain (which would probably add another overhead), I would > > >> make > > >> it consistent and allow operations to be run in parallel. > > >> > > >> There are many use-cases when having true async operations > dramatically > > >> improve performance. Consider, for example, a streaming example when > > keys > > >> are being pushed by a client to a cluster. Currently, to run effective > > >> processing, the user will have to use a data streamer with custom keys > > >> receiver which may be a huge usability downside. Async operations can > > >> utilize the cluster resources very efficiently. > > >> > > >> Finally, if we want to be on the safe side, we can keep the operation > > >> chain > > >> inside a transaction. I see absolutely no point in maintaining this > > chain > > >> outside of transactions. > > >> > > >> --AG > > >> > > >> 2018-05-14 16:01 GMT+03:00 Dmitriy Govorukhin < > > >> dmitriy.govoruk...@gmail.com> > > >> : > > >> > > >> > Andrey, > > >> > > > >> > Do you prefer change behavior at runtime? > > >> > I guess will be better have different methods for getting cache > > instance > > >> > with fair and not fair sync. > > >> > > > >> > On Mon, May 14, 2018 at 3:39 PM, Andrey Gura> > wrote: > > >> > > > >> > > +1 for fair async operations. > > >> > > > > >> > > But I don't like idea use withFairSync() method. We added > xxxAsync() > > >> > > methods recently and withAsync() is deprecated. > > >> > > > > >> > > I think we should just make methods are async in nature and > provide > > >> > > ability of switching to the old behaviour using flag or property. > > >> > > > > >> > > On Fri, May 11, 2018 at 11:00 PM, Dmitriy Setrakyan > > >> > > wrote: > > >> > > > Vladimir, > > >> > > > > > >> > > > In general I agree, but I do get greatly *close-minded* (pun > > >> intended) > > >> > > > whenever users' code that worked for the past several years all > > of a > > >> > > sudden > > >> > > > gets deadlocked after an upgrade. Making this feature optional > is > > >> even > > >> > > > worse and more confusing. In this case the best action is no > > action > > >> at > > >> > > all. > > >> > > > > > >> > > > BTW, would be interesting to find out how Oracle async driver > > >> behaves > > >> > in > > >> > > > this case. > > >> > > > > > >> > > > D. > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > On Fri, May 11, 2018 at 8:29 PM, Vladimir Ozerov < > > >> voze...@gridgain.com > > >> > > > > >> > > > wrote: > > >> > > > > > >> > > >> Guys, > > >> > > >> > > >> > > >> To build a great product we should be open minded and look to > the > > >> > > future, > > >> > > >> not to the past. > > >> > > >> > > >> > > >> Dima raised very valid point - why async is not async? Current > > >> > > programming > > >> > > >> culture and demanding performance requirements pushes users > > towards > > >> > > >> reactive-style programming. I do not want my thread to ever be > > >> > blocked. > > >> > > >> Instead, I want to send a number of concurrent commands and > > >> optionally > > >> > > >> subscribe to final result. So trully async API makes total > sense > > to > > >> > me. > > >> > > >>
Re: IEP-4, Phase 2. Using BL(A)T for in-memory caches.
BLAT sounds good to me because it slightly funny in case you speak russian. "Node joined topology by BLAT (po blaty)" sounds reasonable :) чт, 24 мая 2018 г. в 1:15, Dmitriy Setrakyan: > Ed, > > Anyone speaking Russian would agree that BLAT is not a good name :) Let's > steak to BLT. > > D. > > On Wed, May 23, 2018 at 5:34 AM, Eduard Shangareev < > eduard.shangar...@gmail.com> wrote: > > > Igniters, > > > > We have invested too much in explaining BLAT. So, it would hard to change > > the name. > > I.e. I propose to save this term. > > > > > > New names for auto-adjust control. > > > > 1. org.apache.ignite.IgniteCluster > > > > *Add* > > isBaselineAutoAdjustEnabled() > > setBaselineAutoAdjustEnabled(boolean enabled); > > setBaselineAutoAdjustTimeout(long timeoutInMs); > > setBaselineAutoAdjustMaxTimeout(long timeoutInMs); > > > > 2. org.apache.ignite.configuration.IgniteConfiguration > > > > *Add* > > IgniteConfiguration setBaselineAutoAdjustEnabled(boolean enabled); > > IgniteConfiguration setBaselineAutoAdjustTimeout(long timeoutInMs); > > IgniteConfiguration setBaselineAutoAdjustMaxTimeout(long timeoutInMs); > > > > Any objections? > > > > > > > > On Fri, May 4, 2018 at 10:01 PM, Dmitriy Setrakyan < > dsetrak...@apache.org> > > wrote: > > > > > I do not like the name "current" on the methods. I think we should just > > > remove it, e.g. currentAffinityTopology() -> affinityTopology() > > > > > > D. > > > > > > On Fri, May 4, 2018 at 6:17 AM, Eduard Shangareev < > > > eduard.shangar...@gmail.com> wrote: > > > > > > > Igniters, > > > > > > > > With Vladimir's help, we analyzed another solution's approaches. > > > > And decided to simplify our affinity topology auto-adjusting. > > > > > > > > It should be enough to be able to turn on/off auto-adjusting (flag) > and > > > set > > > > 2 timeouts if it is working: > > > > -soft timeout which would be used if there was no other node > > joins/exits; > > > > -hard timeout which we would track from first discovery event and if > it > > > > reached then immediately would change affinity topology. > > > > > > > > All other strategies could be realized with API usage > > > (setAffinityTopology) > > > > and metrics tracking by user's monitoring tools. > > > > > > > > So, I suggest next API changes: > > > > > > > > org.apache.ignite.IgniteCluster > > > > > > > > *Deprecate*: > > > > Collection currentBaselineTopology(); > > > > void setBaselineTopology(Collection > > > baselineTop); > > > > void setBaselineTopology(long topVer); > > > > > > > > *Replace them with* > > > > Collection currentAffinityTopology(); > > > > void setAffinityTopology(Collection > > > affinityTop); > > > > void setAffinityTopology(long topVer); > > > > > > > > *Add* > > > > isAffinityTopologyAutoAdjustEnabled() > > > > setAffinityTopologyAutoAdjustEnabled(boolean enabled); > > > > > > > > org.apache.ignite.configuration.IgniteConfiguration > > > > > > > > *Add* > > > > IgniteConfiguration setAffinityTopologyAutoAdjustEnabled(boolean > > > enabled); > > > > IgniteConfiguration setAffinityTopologyAutoAdjustTimeout(long > > > > timeoutInMs); > > > > IgniteConfiguration setAffinityTopologyAutoAdjustMaxTimeout(long > > > > timeoutInMs); > > > > > > > > > > > > An open question is could we rename or duplicate BaselineNode with > > > > AffinityNode? > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Apr 27, 2018 at 6:56 PM, Ivan Rakov > > > wrote: > > > > > > > > > Eduard, > > > > > > > > > > +1 to your proposed API for configuring Affinity Topology change > > > > policies. > > > > > Obviously we should use "auto" as default behavior. I believe, > > > automatic > > > > > rebalancing is expected and more convenient for majority of users. > > > > > > > > > > Best Regards, > > > > > Ivan Rakov > > > > > > > > > > > > > > > On 26.04.2018 19:27, Eduard Shangareev wrote: > > > > > > > > > >> Igniters, > > > > >> > > > > >> Ok, I want to share my thoughts about "affinity topology (AT) > > changing > > > > >> policies". > > > > >> > > > > >> > > > > >> There would be three major option: > > > > >> -auto; > > > > >> -manual; > > > > >> -custom. > > > > >> > > > > >> 1. Automatic change. > > > > >> A user could set timeouts for: > > > > >> a. change AT on any topology change after some timeout > > > > (setATChangeTimeout > > > > >> in seconds); > > > > >> b. change AT on node left after some timeout > > > > (setATChangeOnNodeLeftTimeout > > > > >> in seconds); > > > > >> c. change AT on node join after some timeout > > > > (setATChangeOnNodeJoinTimeout > > > > >> in seconds). > > > > >> > > > > >> b and c are more specific, so they would override a. > > > > >> > > > > >> Also, I want to introduce a mechanism of merging AT changes, which > > > would > > > > >> be > > > > >> turned on by default. > > > > >> Other words, if we reached timeout than we would change AT to > > current > > > > >> topology, not that one which was on timeout schedule. > > >
[jira] [Created] (IGNITE-8598) SQL: ability to control partition pruning with explicit affinity column definition
Vladimir Ozerov created IGNITE-8598: --- Summary: SQL: ability to control partition pruning with explicit affinity column definition Key: IGNITE-8598 URL: https://issues.apache.org/jira/browse/IGNITE-8598 Project: Ignite Issue Type: Task Components: sql Reporter: Vladimir Ozerov Affinity functions are applicable to keys only. Sometimes users may have complex affinity calculation logic, so that partition pruning optimization is not applicable. E.g. this custom {{AffinityKeyMapper}}. However, there is a chance that partition could be calculated from some attribute of {{value}}. It would be nice to force our engine to treat some attribute as affinity key even though it is not marked as {{AffinityKeyMapped}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [jira] [Created] (IGNITE-8595) SQL: Ability to cancel DDL operations
Hi Ray, We are working on it, All operations will be cancelled in case of either explicit user request, or when user session is terminated. On Thu, May 24, 2018 at 12:51 PM, Raywrote: > Hello Vladimir. > > From the https://apacheignite-sql.readme.io/docs/query-cancellation, it is > only possible to cancel select query in java API. > So select query submitted from thin client(sqlline)/jdbc/odbc is not > cancellable, is my understanding correct? > > > > > > -- > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ >
[jira] [Created] (IGNITE-8597) Direct data load
Vladimir Ozerov created IGNITE-8597: --- Summary: Direct data load Key: IGNITE-8597 URL: https://issues.apache.org/jira/browse/IGNITE-8597 Project: Ignite Issue Type: Task Components: general Reporter: Vladimir Ozerov We should implement optimized data loading mode, which will bypass as much internal Ignite components as possible to improve data loading speed. Raw design: 1) Direct data load must be performed in exclusive mode, i.e. nobody else are allowed to read or write to specific cache/table at this time. I.e. we need to implement distributed table/cache locks. 2) At first we should write to data pages directly skipping free lists, PK hash index and secondary indexes 3) Once loading is finished, we should rebuild free lists and indexes (bottom-up); external merge implementation will be required 4) We should distinguish between initial load and incremental load. The latter would require merge of indexes with previously available data. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [jira] [Created] (IGNITE-8595) SQL: Ability to cancel DDL operations
Hello Vladimir. >From the https://apacheignite-sql.readme.io/docs/query-cancellation, it is only possible to cancel select query in java API. So select query submitted from thin client(sqlline)/jdbc/odbc is not cancellable, is my understanding correct? -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
[jira] [Created] (IGNITE-8596) SQL: remove unnecessary index lookups when query parallelism is enabled
Vladimir Ozerov created IGNITE-8596: --- Summary: SQL: remove unnecessary index lookups when query parallelism is enabled Key: IGNITE-8596 URL: https://issues.apache.org/jira/browse/IGNITE-8596 Project: Ignite Issue Type: Task Components: sql Affects Versions: 2.5 Reporter: Vladimir Ozerov Fix For: 2.6 See {{org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor#onQueryRequest}} method. If table is segmented, we will submit as many SQL requests as much segments. But consider a case when target cache partition(s) is already defined by user or derived through partition pruning. In this case most of segments will not contain useful information and return empty result set. At the same time these queries may impose index or data page scans, thus consuming resources without a reason. To mitigate the problem we should not submit SQL requests to segments we are not interested in. Note that it is not sufficient to simply skip SQL requests on mapper, because reducer expects separate response for every message. We should fix both local mapper logic as well as protocol. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8595) SQL: Ability to cancel DDL operations
Vladimir Ozerov created IGNITE-8595: --- Summary: SQL: Ability to cancel DDL operations Key: IGNITE-8595 URL: https://issues.apache.org/jira/browse/IGNITE-8595 Project: Ignite Issue Type: Task Components: sql Reporter: Vladimir Ozerov Fix For: 2.6 At the moment it is impossible to cancel DDL operations. Suppose that you started {{CREATE INDEX}} on very heavy table. It took hours to finish, and you would like to stop it because it is 9:00AM, and users are about to execute many queries and {{CREATE INDEX}} will slow them down or even block. It should be possible to stop DDL operations in the same way it is done for {{SELECT}} queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8594) Make error messages in validate_indexes command report more informative
Ivan Rakov created IGNITE-8594: -- Summary: Make error messages in validate_indexes command report more informative Key: IGNITE-8594 URL: https://issues.apache.org/jira/browse/IGNITE-8594 Project: Ignite Issue Type: Improvement Reporter: Ivan Rakov Assignee: Ivan Rakov Fix For: 2.6 In case index is broken and contains links to missing items in data pages, validate_indexes command will show "Item not found" messages in report: {noformat} IndexValidationIssue [key=null, cacheName=cache_group_1_028, idxName=_key_PK], class java.lang.IllegalStateException: Item not found: 65 IndexValidationIssue [key=null, cacheName=cache_group_1_028, idxName=_key_PK], class java.lang.IllegalStateException: Item not found: 15 SQL Index [cache=cache_group_1_028, idx=LONG__VAL_IDX] ValidateIndexesPartitionResult [consistentId=node2, sqlIdxName=LONG__VAL_IDX] IndexValidationIssue [key=null, cacheName=cache_group_1_028, idxName=LONG__VAL_IDX], class java.lang.IllegalStateException: Item not found: 60 IndexValidationIssue [key=null, cacheName=cache_group_1_028, idxName=LONG__VAL_IDX], class java.lang.IllegalStateException: Item not found: 65 IndexValidationIssue [key=null, cacheName=cache_group_1_028, idxName=LONG__VAL_IDX], class java.lang.IllegalStateException: Item not found: 65 IndexValidationIssue [key=null, cacheName=cache_group_1_028, idxName=LONG__VAL_IDX], class java.lang.IllegalStateException: Item not found: 15 {noformat} It would be better to explain what is happening: key is present in SQL index, but missing in corresponding data page. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8593) The semaphore's isBroken function doesn't work properly.
Mo created IGNITE-8593: -- Summary: The semaphore's isBroken function doesn't work properly. Key: IGNITE-8593 URL: https://issues.apache.org/jira/browse/IGNITE-8593 Project: Ignite Issue Type: Bug Components: data structures Affects Versions: 2.4 Reporter: Mo Scenario: Three servers (s1,s2,s3) two clients (c1,c2). A semaphore with one permit is created. Config: {{Release acquired permits if node, that owned them, left topology: set to false}} # c2 acquires the permit. # Network failure happens, isolating c2 from the rest of nodes for a period of time. # Network heals. # c2 releases the permit. # c2 acquires the permit. # Calling semaphore.isBroken() returns false on both c1 and c2. # c1 tries to acquire the permit but fails. # Now calling isBroken() returns true on both c1 and c2. I think isBroken() should return true before a client tries to acquire a permit, and then fails (i.e., in step 6) rather than after acquiring a permit fails, as in the latter case, what purpose does the isBroken() function serves? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Node.js Thin Client @ npmjs
1. Sounds OK for me. 2. Since Node.js client is going to be included in Ignite releases, I guess it make sense to align its version with the version of Ignite, i.e. 2.6, etc... Best Regards, Igor On Thu, May 24, 2018 at 9:02 AM, Pavel Petroshenkowrote: > Igniters, > > I'm finalizing the docs (with regards to installation instructions) for the > Node.js Thin Client at readme.io. And I've got a couple of questions. > > 1. Does the name "apache-ignite-client" for the Node.js Thin Client > npm module sound good to everyone? This name will be used to install the > client library, for example: > > npm i -g apache-ignite-client > > or include it from the source code: > > const IgniteClient = require('apache-ignite-client'); > > 2. Which version should the npm module get: 1.0 or does it need to be > aligned with the Ignite release version when it's published or compatible > with, i.e. 2.4, 2.5 or 2.6? > > thanks, > p. >
[jira] [Created] (IGNITE-8592) Network partitions lead to two independent clusters
Mo created IGNITE-8592: -- Summary: Network partitions lead to two independent clusters Key: IGNITE-8592 URL: https://issues.apache.org/jira/browse/IGNITE-8592 Project: Ignite Issue Type: Bug Affects Versions: 2.4 Reporter: Mo Creating a network partition in a replicated Ignite cluster leads to creating two independent clusters, each of which would operate independently from the other, even after the network partition is healed. Setup: 3 servers (s1,s2,s3) two clients (c1,c2). A partition created \{(s1,s2,c1),(s3,c2)}. --> At this point two independent clusters form; one containing s1 and s2, while the other containing s3. The two never rejoin even after the partition is healed. This creates different kinds of problems for the different data structure ignite provides, such as the cache (stale reads, and data unavailability), atomic types (atomicref and long ) ... etc. These are the settings used for the replicated cache: cfg.setCacheMode(CacheMode.REPLICATED); cfg.setAtomicityMode(CacheAtomicityMode.ATOMIC); cfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC); cfg.setReadFromBackup(false); cfg.setPartitionLossPolicy(PartitionLossPolicy.READ_ONLY_SAFE); -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ignite pull request #4059: IGNITE-8587 GridToStringBuilder use ConcurrentHas...
GitHub user macrergate opened a pull request: https://github.com/apache/ignite/pull/4059 IGNITE-8587 GridToStringBuilder use ConcurrentHashMap to avoid global⦠⦠locks on classCache You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-8587 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4059.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4059 commit 9db962835f49bc08a63f05b0e3f98ff5bdea Author: Sergey KosarevDate: 2018-05-24T08:56:12Z IGNITE-8587 GridToStringBuilder use ConcurrentHashMap to avoid global locks on classCache ---
Re: Ability to check and completely fill transactions on creation
Dmitriy, Yakov is against the solution based on event sub-system >> I think that we should think about some other solution instead of altering >> event sub-system. Also, I checked is there any chances to fix all the issues found by Yakov and see that solution becomes overcomplicated in that case. That's why I'm proposing this lightweight solution. As for me it's a good idea to have such validator since that's a common problem at huge deployments when more than one team have access to Ignite cluster and there is no other way to setup tx cretion rules. Yakov, Could you please share your thoughts on that? чт, 24 мая 2018 г. в 8:58, Dmitriy Setrakyan: > On Wed, May 23, 2018 at 4:08 AM, Anton Vinogradov wrote: > > > Dmitriy, Yakov > > > > Are there any objections to updated design taking into account the > comments > > I provided? > > > > Anton, I do not like an additional validator. I think you can accomplish > the same with a transaction event. You just need to design it more cleanly, > incorporating the feedback from Yakov. >
[GitHub] ignite pull request #4058: IGNITE-8591
GitHub user devozerov opened a pull request: https://github.com/apache/ignite/pull/4058 IGNITE-8591 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-8591 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4058.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4058 commit 17c11ddbbf3e3d0e0e41e03d04050c1ac27340c7 Author: devozerovDate: 2018-05-24T08:22:29Z IGNITE-8591: Done. ---
[jira] [Created] (IGNITE-8591) SQL: Sort links on index pages in physical page order before row access
Vladimir Ozerov created IGNITE-8591: --- Summary: SQL: Sort links on index pages in physical page order before row access Key: IGNITE-8591 URL: https://issues.apache.org/jira/browse/IGNITE-8591 Project: Ignite Issue Type: Task Components: sql Affects Versions: 2.5 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 2.6 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Node.js Thin Client @ npmjs
Igniters, I'm finalizing the docs (with regards to installation instructions) for the Node.js Thin Client at readme.io. And I've got a couple of questions. 1. Does the name "apache-ignite-client" for the Node.js Thin Client npm module sound good to everyone? This name will be used to install the client library, for example: npm i -g apache-ignite-client or include it from the source code: const IgniteClient = require('apache-ignite-client'); 2. Which version should the npm module get: 1.0 or does it need to be aligned with the Ignite release version when it's published or compatible with, i.e. 2.4, 2.5 or 2.6? thanks, p.