[GitHub] ignite pull request #4956: IGNITE-9853 option control.sh --cache list was ex...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4956


---

[GitHub] ignite pull request #5072: Ignite 9941IGNITE-9941 Web Console: Add option to...

[akuznetsov-gridgain]

GitHub user akuznetsov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/5072

Ignite 9941IGNITE-9941 Web Console: Add option to disable self-registration



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9941

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5072.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5072


commit 79dc1451b60b7f9b61c672809a13f37d6844df4c
Author: Alexey Kuznetsov 
Date:   2018-10-22T04:03:12Z

IGNITE-9941 WIP on disable self registration.

commit 8a803468d21a92944ba3c163281f9e3317aa13e3
Author: Alexey Kuznetsov 
Date:   2018-10-23T04:32:02Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-9941

commit b63d4aae784354c4e65392ac7882c19612c67296
Author: Alexey Kuznetsov 
Date:   2018-10-23T07:35:32Z

IGNITE-9941 WIP.

commit af1e8c23a2e6f696627ee208e6b7263e93f6ee2a
Author: Ilya Borisov 
Date:   2018-10-24T02:32:56Z

IGNITE-9941 Fix form-field-email validation.

commit e5598d4e80f3c313e8110f0a3ed21d8b9fe209ac
Author: Ilya Borisov 
Date:   2018-10-24T02:34:42Z

IGNITE-9941 Extract signup form into component.

commit 131851e12e93a43a412eb0715d5e5a29ee8103cb
Author: Ilya Borisov 
Date:   2018-10-24T04:09:45Z

IGNITE-9941 Convert AuthService to TS, make auth redirect optional, but 
enabled by default.

commit 57505407c7525bbda5fd5bdd2b54c166cb38924b
Author: Ilya Borisov 
Date:   2018-10-24T04:10:11Z

GC-9941 Add user creation dialog.

commit 961ee6e0b26189a18ae3ca295b7cc8e1aff82707
Author: Alexey Kuznetsov 
Date:   2018-10-24T04:20:35Z

IGNITE-9941 WIP.

commit 4505371e27fc0ad89e43a3a5510bedf852bd4df3
Author: Alexey Kuznetsov 
Date:   2018-10-24T04:20:54Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-9941

commit c705b57e04cd218e6c1cc4fcdbe29fb12b920116
Author: Ilya Borisov 
Date:   2018-10-24T06:56:28Z

IGNITE-9941 Reload registered users list after creating one.

commit a7bdbe2694c693644c29ec43bc4d21afa3da796c
Author: Alexey Kuznetsov 
Date:   2018-10-24T07:03:44Z

Merge remote-tracking branch 'community/ignite-9941' into ignite-9941

# Conflicts:
#   
modules/web-console/frontend/app/components/list-of-registered-users/controller.js

commit f94c84e6bd730b8c7632fab547d0e7743093c47b
Author: Alexey Kuznetsov 
Date:   2018-10-24T07:12:41Z

IGNITE-9941 Fixed user registration on backend.

commit 4db4e19e654bc019c48d79e210438d3bfcc65245
Author: Ilya Borisov 
Date:   2018-10-24T07:22:56Z

IGNITE-9941 Fix checkbox, password and phone input validation.

commit e83ffa873814c5f08e2d02312a9ffeb4d335e308
Author: Alexey Kuznetsov 
Date:   2018-10-24T12:09:32Z

IGNITE-9941 WIP.

commit b4085f1fee44fb888fb61b0baabdc32ac967e272
Author: Alexey Kuznetsov 
Date:   2018-10-25T02:50:46Z

IGNITE-9941 WIP.

commit 2eec71f9ff8ccc8a57108a4ac1c012e0e3f26344
Author: Alexey Kuznetsov 
Date:   2018-10-25T02:53:08Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-9941

commit bb4e3376b46d94f517ecd225506ac567db9a71d0
Author: Alexey Kuznetsov 
Date:   2018-10-25T03:07:26Z

IGNITE-9941 WIP.




---

Re: hello

[Denis Magda]

Hey Scott,

Great to know, welcome to the community!

What exactly do you like to contribute back? Please start a separate
discussion with the most meaningful topic.

Also, if you can post a blog post about your architecture at VMWare will be
glad to mention it on the "Ignite in Production" page:
https://ignite.apache.org/provenusecases.html

--
Denis

On Wed, Oct 24, 2018 at 7:39 PM Scott Feldstein  wrote:

> Hi Everyone,
> I'm following the instructions from
> https://ignite.apache.org/community/contribute.html. I'd like to
> contribute
> some of the code I've been working on in order to continue evolving Ignite.
>
> I've been evangelizing Ignite at VMware and am in the process of locking
> down our next-gen service architecture.  The plan is to use Ignite as a
> multi-faceted caching server on steriods!
>
> Please feel free to reach out to me and say hi as well :)
>
> Scott
>

hello

[Scott Feldstein]

Hi Everyone,
I'm following the instructions from
https://ignite.apache.org/community/contribute.html. I'd like to contribute
some of the code I've been working on in order to continue evolving Ignite.

I've been evangelizing Ignite at VMware and am in the process of locking
down our next-gen service architecture.  The plan is to use Ignite as a
multi-faceted caching server on steriods!

Please feel free to reach out to me and say hi as well :)

Scott

[GitHub] ignite pull request #5071: Ignite 9976 mass run (fs test)

[xtern]

GitHub user xtern opened a pull request:

https://github.com/apache/ignite/pull/5071

Ignite 9976 mass run (fs test)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xtern/ignite IGNITE-9976-FULLSYNC

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5071.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5071


commit ab2db757bef7b6acdfc3bcb7a2f920d7c3db7cfc
Author: pereslegin-pa 
Date:   2018-10-23T13:48:13Z

IGNITE-9976 Flaky test failure fix.

commit 0543743104d7d52e61064dfeec662d5c8b875fc9
Author: pereslegin-pa 
Date:   2018-10-23T19:32:09Z

IGNITE-9976 Flaky failure 2 fix.

commit c6453a482d42bbbd7990ee95bc74a0ac9b1d9133
Author: pereslegin-pa 
Date:   2018-10-24T15:38:02Z

IGNITE-9976 mass run.

commit af029e7a9e9dd15c7c38f24a90c8089abe82be94
Author: pereslegin-pa 
Date:   2018-10-24T20:54:26Z

mass run fullsync.




---

[GitHub] ignite pull request #5070: ignite-9650: Added DECIMAL type to jdbc metadata.

[pavel-kuznetsov]

GitHub user pavel-kuznetsov opened a pull request:

https://github.com/apache/ignite/pull/5070

ignite-9650: Added DECIMAL type to jdbc metadata.

DECIMAL type earlier was treated as OTHER type by jdbc metadata.
Added mapping code for this type. Added regression tests.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9650

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5070.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5070


commit c6ec5902b5ac32028b8d28f0dff7b53bee9367ff
Author: Pavel Kuznetsov 
Date:   2018-10-24T20:46:11Z

ignite-9650: Added DECIMAL type to jdbc metadata.

DECIMAL type earlier was treated as OTHER type by jdbc metadata.
Added mapping code for this type. Added regression tests.




---

[GitHub] ignite pull request #5069: Ignite 9841

[slukyano]

GitHub user slukyano opened a pull request:

https://github.com/apache/ignite/pull/5069

Ignite 9841



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9841

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5069.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5069


commit 10dcf22eb6348d0f6c30897611a48b9bfd91379b
Author: Stanislav Lukyanov 
Date:   2018-10-16T17:35:47Z

IGNITE-9841: 
IgniteCachePartitionLossPolicySelfTest.testIgnoreKillThreeNodes added, warnings 
addressed.

commit 9f3d0e4759a3d8084afbe49acffca459774fe664
Author: Stanislav Lukyanov 
Date:   2018-10-16T17:37:06Z

IGNITE-9841: Persistence support for IgniteCachePartitionLossPolicySelfTest.

commit 399389576f7ada2b54e70b1041aa50365a8639e7
Author: Stanislav Lukyanov 
Date:   2018-10-16T17:39:47Z

IGNITE-9841: Reset partLossPlc in IgniteCachePartitionLossPolicySelfTest.

commit f25564ad81d1ed96822bd067f71e7a7a4d33427a
Author: Stanislav Lukyanov 
Date:   2018-10-17T15:28:06Z

IGNITE-9841: Add persistence test cases.

commit af027083459b5967d68835a8aed083c97e50a2e4
Author: Stanislav Lukyanov 
Date:   2018-10-24T18:57:03Z

IGNITE-9841: Added scan query tests + more query checks + refactoring.




---

[GitHub] ignite pull request #5068: Ignite 2.5.1 p12 snap fix

[macrergate]

GitHub user macrergate opened a pull request:

https://github.com/apache/ignite/pull/5068

Ignite 2.5.1 p12 snap fix



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite 
ignite-2.5.1-p12-snap-fix

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5068.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5068


commit 7457fd319a372d54de68271be7fddbb634cb6070
Author: Anton Kalashnikov 
Date:   2018-04-17T07:30:52Z

IGNITE-8048 Store dynamic indexes to cache data on node join - Fixes #3719.

Signed-off-by: Alexey Goncharuk 

commit 2883ff4e958747916e7d9eec671100c366cad66b
Author: Ilya Borisov 
Date:   2018-04-17T08:01:36Z

IGNITE-8291 Web Console: Fixed Docker file generation.

(cherry picked from commit 5614621)

commit d0997d7740ea1114b4c8236f225d989de98e2f10
Author: YuriBabak 
Date:   2018-04-17T08:22:14Z

IGNITE-8292: Broken yardstick compilation.

this closes #3838

(cherry picked from commit e76fcb4)

commit 3d2556bc73eff6c5ccd52af1bea88b6016358db8
Author: Ilya Borisov 
Date:   2018-04-17T08:46:10Z

IGNITE-8285 Web console: Removed debug output.

(cherry picked from commit 8c80dce)

commit 4846e967e4cb7a174880a2956e807505a78fd441
Author: YuriBabak 
Date:   2018-04-17T08:54:41Z

IGNITE-8292: Broken yardstick compilation.

this closes #3840

(cherry picked from commit 3cebf91)

commit 733a62bcb6c0d9381a496f07417c10c7edea6d7c
Author: Ilya Borisov 
Date:   2018-04-17T07:12:39Z

IGNITE-8287 Change position on signup inputs on page-sign-in.

(cherry picked from commit e5c3f89)

commit 83e54311fce1d46279c6ddd687ced6f7c9f17ff6
Author: Ilya Borisov 
Date:   2018-04-17T10:15:57Z

IGNITE-8200 Web Console: Override clonedCluster in cluster-edit-form if 
caches or models have changed.
This improves interop with "import from DB" feature, which might update 
caches/models of cluster currently opened for editing.
The import dialog works as a separate state, so the form change 
detection mechanism ensures that any changes to the original
cluster are safe and won't interfere with changes made by user in 
cluster edit form.

(cherry picked from commit 7731669)

commit 86d3f196e436095f277bb9b3e2c32293185db634
Author: Sergey Chugunov 
Date:   2018-04-17T11:28:47Z

IGNITE-8210 Fixed custom event handling for baseline topology change - 
Fixes #3814.

Signed-off-by: Alexey Goncharuk 

commit a7dbea16064bbd52907a770bb40c3a2445313db2
Author: Dmitriy Sorokin 
Date:   2018-04-17T11:48:44Z

IGNITE-8255 Possible name collisions in WorkersRegistry.

Signed-off-by: Andrey Gura 

commit b762d681b97ea121a8321eb66bf02f89a1d177cd
Author: Aleksey Plekhanov 
Date:   2018-04-17T12:56:36Z

IGNITE-8166 PME hangs when error occurs during checkpoint

Signed-off-by: Andrey Gura 

commit 8428b0e63e97c10277f6d9e1640e16528a772270
Author: Ivan Daschinskiy 
Date:   2018-04-17T15:05:42Z

IGNITE-8021 Delete cache config files when cache is destroyed - Fixes #3697.

Signed-off-by: Alexey Goncharuk 

commit cd59c8e64f05ca03c7da8dc35d027a14fcebf250
Author: Aleksey Plekhanov 
Date:   2018-04-17T15:27:53Z

IGNITE-8033 Fixed flaky failure of 
TxOptimisticDeadlockDetectionCrossCacheTest

Signed-off-by: Andrey Gura 

commit acfef907db8204ac93fc235770f36bf7f61269c3
Author: Ilya Kasnacheev 
Date:   2018-04-17T16:50:51Z

IGNITE-2766 Fix .net test. - Fixes #3853.

Signed-off-by: dpavlov 

(cherry picked from commit 96cb795)

commit 6cea78e4e13fe43555b78dcd683366f54c6816ff
Author: Andrey Kuznetsov 
Date:   2018-04-17T16:58:43Z

IGNITE-7770 Test testRandomMixedTxConfigurations partialy fixed

Signed-off-by: Andrey Gura 

commit e394693a7389b4daff328827abdb1dcd28783f66
Author: Maxim Muzafarov 
Date:   2018-04-17T18:18:36Z

IGNITE-8301 testReconnectCacheDestroyedAndCreated should excpect recreated 
client cache - Fixes #3856.

Signed-off-by: dpavlov 

(cherry picked from commit 56be24b)

commit 4685ebe5f5dda4023980398806e222fada895e26
Author: Vasiliy Sisko 
Date:   2018-04-18T03:44:44Z

IGNITE-8140 Web Console: Fixed code generation for large numbers in 
configuration params.

(cherry picked from commit eda5fe7)

commit 5efc589fcabffdb29cd6dfe0e7323bc91db47703
Author: Ilya Borisov 
Date:   2018-04-18T04:39:41Z

IGNITE-8294 Web Console: Move "Beta" ribbon to the left.

(cherry picked from commit 69606e4)

commit af46856d7f7ec88aa663865a108900abb8314ffe
Author: oleg-ostanin 
Date:   2018-04-17T17:58:53Z

IGNITE-8274 sqlline.sh script uses JAVA_HOME now

Signed-off-by: Andrey Gura 

(cherry picked from commit c3ff274)

commit 

[jira] [Created] (IGNITE-9993) Values cached in WB store are locked during a batch flush

[Dmitry Konstantinov (JIRA)]

Dmitry Konstantinov created IGNITE-9993:
---

 Summary: Values cached in WB store are locked during a batch flush
 Key: IGNITE-9993
 URL: https://issues.apache.org/jira/browse/IGNITE-9993
 Project: Ignite
  Issue Type: Bug
  Components: cache
Reporter: Dmitry Konstantinov


The following logic is executed under write lock:
{code: java}
applyBatch(pending, true, null);
{code}
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/store/GridCacheWriteBehindStore.java#L1120
In combination with IGNITE-5003 it may cause locking of a put/remove operation 
while the whole batch is applying.

applyBatch(pending, true, null); should be moved out of the lock guarded 
section.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] dspavlov opened a new pull request #43: IGNITE-9848 load all builds

[GitBox]

dspavlov opened a new pull request #43: IGNITE-9848 load all builds
URL: https://github.com/apache/ignite-teamcity-bot/pull/43
 
 
   IGNITE-9848
   [TC Bot] Background upload of all builds from TC to the bot DB


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ignite pull request #5067: IGNITE-9420 Proper recovery

[Jokser]

GitHub user Jokser opened a pull request:

https://github.com/apache/ignite/pull/5067

IGNITE-9420 Proper recovery



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite 
ignite-9420-proper-recovery

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5067.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5067


commit fc0ba2d389fa021055413b9d6596859d3ea838df
Author: Maxim Muzafarov 
Date:   2018-08-24T14:27:26Z

IGNITE-7196: initial commit

commit 07d8c0296476656131ffcf929c6eeaeaec1a3a7f
Author: Maxim Muzafarov 
Date:   2018-08-31T08:11:02Z

Merge branch 'master' into ignite-7196

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheDatabaseSharedManager.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/IgniteCacheDatabaseSharedManager.java

commit b8935632089b6a77fa533089508a062c6933161b
Author: Maxim Muzafarov 
Date:   2018-08-31T08:18:20Z

IGNITE-7196: move down init listener

commit 97bde70641d02621925e85bf46defc6c0c213423
Author: Maxim Muzafarov 
Date:   2018-08-31T08:21:20Z

IGNITE-7196: revert down init listener

commit e9c84f2a9f5ae5f2fe4c9e2275addc6042a635e8
Author: Maxim Muzafarov 
Date:   2018-09-03T16:47:46Z

IGNITE-7196: WIP restore binary on first activate

commit a8e6a59bd680c5e861159c32788022e9c000f019
Author: Maxim Muzafarov 
Date:   2018-09-03T20:24:09Z

IGNITE-7196: WIP remove readAndRestore from PME

commit 71b5efb046bbe13632ec938a06683d6a2c398b4e
Author: Maxim Muzafarov 
Date:   2018-09-03T20:30:17Z

IGNITE-7196: WIP remove readAndRestore from PME 2

commit 9f982cd05ad608317c183e9f5a4008a178dac31a
Author: Maxim Muzafarov 
Date:   2018-09-04T09:41:32Z

Merge branch 'master' into ignite-7196

# Conflicts:
#   
modules/indexing/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/wal/IgniteWalRecoveryTest`.java

commit 161b48d94a574007b37a070e731e963ded6bf431
Author: Maxim Muzafarov 
Date:   2018-09-04T09:46:45Z

IGNITE-7196: WIP remove unused imports

commit 1f286be9e80a484a62c6c355e4141afef55923a6
Author: Maxim Muzafarov 
Date:   2018-09-04T16:18:37Z

IGNITE-7196: WIP make lastRestored flag checkable

commit 2b6b17c3a8e9b254fe9337dc76b51c6ca02dd617
Author: Maxim Muzafarov 
Date:   2018-09-04T16:25:38Z

IGNITE-7196: WIP lastRestored 2

commit 323b0e83af00878afd5286dcf6973973b8188844
Author: Maxim Muzafarov 
Date:   2018-09-04T19:17:42Z

IGNITE-7196: WIP make lastRestored flag checkable 2

commit 087e63a5999a8cd4b06d6ee6085d4d1db837576d
Author: Maxim Muzafarov 
Date:   2018-09-04T19:56:12Z

IGNITE-7196: WIP make lastRestored flag checkable 3

commit 3b40a2ef0aef628af088f3a1a8ab0a9a11b3be4b
Author: Maxim Muzafarov 
Date:   2018-09-04T21:21:53Z

IGNITE-7196: WIP make lastRestored flag checkable 4

commit 6f9b44ed9e1c523c266167ff9f03a6e680c2d9fa
Author: Maxim Muzafarov 
Date:   2018-09-05T09:46:42Z

IGNITE-7196: WIP minor changes

commit bb86b9144c9e863ad86af27823046f992503d00e
Author: Maxim Muzafarov 
Date:   2018-09-05T12:14:51Z

IGNITE-7196: WIP without activate#deactivate 1

commit 87b4f325c5dd8a93f6c4fc1dd21d50c9c3752684
Author: Maxim Muzafarov 
Date:   2018-09-05T13:14:58Z

IGNITE-7196: WIP without activate#deactivate 2

commit 65f83f68f5b5ca1923bcc5a7c4841ca2455cb902
Author: Maxim Muzafarov 
Date:   2018-09-05T14:23:24Z

IGNITE-7196: WIP without activate#deactivate 3

commit 166fd04c3bd5dfdd39c6118c21f0ae39170842eb
Author: Maxim Muzafarov 
Date:   2018-09-05T15:43:35Z

IGNITE-7196: WIP without activate#deactivate 4

commit b60a68cb1275b8c1074ae3d22c4c7f7967c883b6
Author: Maxim Muzafarov 
Date:   2018-09-05T16:32:48Z

IGNITE-7196: WIP without activate#deactivate 5

commit a60fdd67145ee5d6f4bbac81de9e6b1d9bbab6b1
Author: Maxim Muzafarov 
Date:   2018-09-05T16:44:03Z

IGNITE-7196: WIP without activate#deactivate 6

commit 3cf0911f646c4100e0a1654e3dec3c0f4952abf5
Author: Maxim Muzafarov 
Date:   2018-09-05T17:21:02Z

IGNITE-7196: WIP without activate#deactivate 7

commit 65ed786c9d9bf83ab56f60349ac8eaba8099807b
Author: Maxim Muzafarov 
Date:   2018-09-05T18:28:59Z

IGNITE-7196: WIP without activate#deactivate 8

commit 53f81a8ce75785c9b6badf67a93261e8eae2947a
Author: Maxim Muzafarov 
Date:   2018-09-05T18:32:39Z

IGNITE-7196: WIP minor code changes

commit 61633dc9dd0980070b5f1c9ec5a9624b99b32b69
Author: Maxim Muzafarov 
Date:   2018-09-06T17:56:13Z

IGNITE-7196: WIP suspend#resume WAL logging 1

commit 3772cbdcfe9edd90d76d5cb841c4c00123077eac
Author: Maxim Muzafarov 
Date:   2018-09-07T17:56:07Z

IGNITE-7196: WIP deactivate wal

commit 5d7affd2539f2b1fb4ab63c93a7c3fba2104eb2b
Author: Maxim 

[GitHub] ignite pull request #5066: IGNITE-9454 Support CACHE_CREATE, CACHE_DESTROY, ...

[alamar]

GitHub user alamar opened a pull request:

https://github.com/apache/ignite/pull/5066

IGNITE-9454 Support CACHE_CREATE, CACHE_DESTROY, JOIN_AS_SERVER permi…

…ssions in SecurityPermissionSetBuilder.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9454

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5066.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5066


commit 2302baf78fb1d2700b81c29140bf730588cbc05a
Author: Ilya Kasnacheev 
Date:   2018-10-24T16:36:15Z

IGNITE-9454 Support CACHE_CREATE, CACHE_DESTROY, JOIN_AS_SERVER permissions 
in SecurityPermissionSetBuilder.




---

[jira] [Created] (IGNITE-9992) Add some command to calculate hast sum per primary partition in product

[ARomantsov (JIRA)]

ARomantsov created IGNITE-9992:
--

 Summary: Add some command to calculate hast sum per primary 
partition in product
 Key: IGNITE-9992
 URL: https://issues.apache.org/jira/browse/IGNITE-9992
 Project: Ignite
  Issue Type: Improvement
Affects Versions: 2.7
Reporter: ARomantsov


Some util to quick check cluster data is ok



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] asfgit closed pull request #41: IGNITE-9940 [TC Bot] Sort pull requests by update time

[GitBox]

asfgit closed pull request #41: IGNITE-9940 [TC Bot] Sort pull requests by 
update time
URL: https://github.com/apache/ignite-teamcity-bot/pull/41
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/github/PullRequest.java
 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/github/PullRequest.java
index 959e3a1..1763274 100644
--- 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/github/PullRequest.java
+++ 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/github/PullRequest.java
@@ -57,6 +57,12 @@
 
 @SerializedName("base") private GitHubBranch base;
 
+/**
+ * @return Pull Request time update.
+ */
+public String getTimeUpdate() {
+return  updatedAt;
+}
 
 /**
  * @return Pull Request number.
diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/tcbot/visa/ContributionToCheck.java
 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/tcbot/visa/ContributionToCheck.java
index a19e3ca..cd265d3 100644
--- 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/tcbot/visa/ContributionToCheck.java
+++ 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/tcbot/visa/ContributionToCheck.java
@@ -40,4 +40,7 @@
 
 /** JIRA issue without server URL, but with project name */
 public String jiraIssueId;
+
+/** Pr time update. */
+public String prTimeUpdate;
 }
diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/tcbot/visa/TcBotTriggerAndSignOffService.java
 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/tcbot/visa/TcBotTriggerAndSignOffService.java
index e4c957e..6d8f0cf 100644
--- 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/tcbot/visa/TcBotTriggerAndSignOffService.java
+++ 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/tcbot/visa/TcBotTriggerAndSignOffService.java
@@ -219,6 +219,7 @@ public SimpleResult commentJiraEx(
 check.prNumber = pr.getNumber();
 check.prTitle = pr.getTitle();
 check.prHtmlUrl = pr.htmlUrl();
+check.prTimeUpdate = pr.getTimeUpdate();
 
 GitHubUser user = pr.gitHubUser();
 if (user != null) {
diff --git a/ignite-tc-helper-web/src/main/webapp/js/prs-1.0.js 
b/ignite-tc-helper-web/src/main/webapp/js/prs-1.0.js
index df63efd..b9729ab 100644
--- a/ignite-tc-helper-web/src/main/webapp/js/prs-1.0.js
+++ b/ignite-tc-helper-web/src/main/webapp/js/prs-1.0.js
@@ -5,6 +5,7 @@ function drawTable(srvId, suiteId, element) {
 "\n" +
 "\n" +
 ".\n" +
+".\n" +
 "...\n" +
 "Loading\n" +
 "...\n" +
@@ -31,6 +32,10 @@ function requestTableForServer(srvId, suiteId, element) {
 });
 }
 
+function normalizeDateNum(num) {
+return num < 10 ? '0' + num : num;
+}
+
 function showContributionsTable(result, srvId, suiteId) {
 let tableId = 'serverContributions-' + srvId;
 let tableForSrv = $('#' + tableId);
@@ -38,11 +43,18 @@ function showContributionsTable(result, srvId, suiteId) {
 tableForSrv.dataTable().fnDestroy();
 
 var table = tableForSrv.DataTable({
+order: [[1, 'desc']],
 data: result,
 "iDisplayLength": 30, //rows to be shown by default
 //"dom": 'ip>',
 //"dom": '<"wrapper"flipt>',
 stateSave: true,
+columnDefs: [
+{
+targets: 1,
+className: 'dt-body-center'
+}
+],
 columns: [
 {
 "className": 'details-control',
@@ -56,6 +68,21 @@ function showContributionsTable(result, srvId, suiteId) {
 }
 }
 },
+{
+"data": "prTimeUpdate",
+title: "Update Time",
+"render": function (data, type, row, meta) {
+if (type === 'display') {
+let date = new Date(data);
+
+data = normalizeDateNum(date.getFullYear()) + '-' + 
normalizeDateNum(date.getMonth()) +
+'-' + normalizeDateNum(date.getDate()) + "" + 
normalizeDateNum(date.getHours()) +
+':' + normalizeDateNum(date.getMinutes()) + ":" + 
normalizeDateNum(date.getSeconds());
+}
+
+return data;
+}
+},
 {
 "data": "prHtmlUrl",
 title: "PR Number",


 


This is an automated message from the Apache Git Service.
To 

[GitHub] ignite pull request #5048: IGNITE-9961 GridMessageListenSelfTest#testNullTop...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5048


---

[GitHub] asfgit closed pull request #42: IGNITE-9894 [TC Bot] Change of JIRA link name

[GitBox]

asfgit closed pull request #42: IGNITE-9894 [TC Bot] Change of JIRA link name
URL: https://github.com/apache/ignite-teamcity-bot/pull/42
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/TcHelper.java 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/TcHelper.java
index 5e0422d..fc9a8bb 100644
--- a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/TcHelper.java
+++ b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/TcHelper.java
@@ -224,7 +224,7 @@ else if (recent.failures != null && recent.runs != null) {
 }
 }
 
-res.append("\\n").append("[TeamCity Run 
All|").append(webUrl).append(']');
+res.append("\\n").append("[TeamCity RunAll 
Results|").append(webUrl).append(']');
 
 return xmlEscapeText(res.toString());
 }


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ignite pull request #5065: Limit test execution time to 10 sec

[pavlukhin]

GitHub user pavlukhin opened a pull request:

https://github.com/apache/ignite/pull/5065

Limit test execution time to 10 sec



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite 10sec-test-limit

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5065.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5065


commit 5d06a7ef34b4fbb0f18bb958fd9e3f54ef2a0aa4
Author: ipavlukhin 
Date:   2018-10-24T15:40:36Z

limit test execution time to 10 sec




---

Legal advise on including Visual C++ Redistributable package into ODBC installer

[Igor Sapego]

Hi guys,

I need your advice here. As some of you probably know we have
Windows installers for ODBC in our binary releases, so users won't need
to build Ignite C++ if all they want is to install and use our ODBC driver.
However, as we build driver with MSVC 10 when we are preparing our
binary release, user should have Visual C++ 2010 Redistributable package
for it to work properly. If they do not, it just do not work, not giving any
human-readable errors, which is very confusing to a user.

So my idea was to check for required package during installation and
give user a download link to a proper package. However, on the WiX
website there are articles on how to include the package into installer [1].
So I've thought that maybe this is even the better way to solve a problem.

So thus is my question, is it OK from the legal standpoint, if we will
distribute
installer, that includes Visual C++ 2010 Redistributable package? We are not
going to put any binaries under the source control, of course.

Can anyone help?

[1] -
http://wixtoolset.org/documentation/manual/v3/howtos/redistributables_and_install_checks/install_vcredist.html

Best Regards,
Igor

[GitHub] asfgit closed pull request #35: Ignite 9848 load all builds

[GitBox]

asfgit closed pull request #35: Ignite 9848 load all builds
URL: https://github.com/apache/ignite-teamcity-bot/pull/35
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (IGNITE-9991) thin clients: can't get cache by array key for PHP, JS and Python clients

[Stepan Pilschikov (JIRA)]

Stepan Pilschikov created IGNITE-9991:
-

 Summary: thin clients: can't get cache by array key for PHP, JS 
and Python clients
 Key: IGNITE-9991
 URL: https://issues.apache.org/jira/browse/IGNITE-9991
 Project: Ignite
  Issue Type: Bug
  Components: thin client
Affects Versions: 2.7
Reporter: Stepan Pilschikov


Trying to put cache with key as values array

Put Py code
{code}
cache = client.get_or_create_cache("PY_BOOLEAN_ARRAY")
cache.put([True, False], 1, key_hint=BoolArrayObject, value_hint=IntObject)
{code}

Get
{code}
cache = client.get_or_create_cache("PY_BOOLEAN_ARRAY")
print(cache.get([True, False] key_hint=BoolArrayObject))
{code}

output: null

Same thing with PHP, JS clients and all others array data types



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: NodeJS thin client: full API

[Pavel Petroshenko]

Hi Igor,

Yes, it sounds reasonable, agree.

P.

On Wed, Oct 24, 2018 at 7:42 AM Igor Sapego  wrote:

> Pavel,
>
> Can we add a proper check and throw proper exception
> when trying to deserialize enum value? NPE does not say
> anything to a user.
>
> Best Regards,
> Igor
>
>
> On Wed, Oct 24, 2018 at 5:34 PM Pavel Petroshenko 
> wrote:
>
> > Hi Stepan,
> >
> > Nodejs and PHP clients do not support enum type registration (and hence
> no
> > tests). So enum type must be registered from somewhere else in order to
> be
> > put or get from the Thin clients.
> >
> > If you register the type say from Java, then put/get for Enum values
> should
> > work from the Thin clients. Can you please try this and let me know how
> it
> > works.
> >
> > Thanks,
> > p.
> >
> > On Tue, Oct 23, 2018 at 6:30 AM Stepan Pilshchikov <
> > pilshchikov@gmail.com> wrote:
> >
> > > Alexey,
> > >
> > > I'm trying to get Enum from cache which placed by different thin client
> > and
> > > find that i can't get it, only exception
> > >
> > > And more when im trying to get Enum which is placed by nodejs client i
> > also
> > > can't do it, have same exception
> > >
> > > code and output -
> > > https://gist.github.com/pilshchikov/ca9fda160e310f62dd4d1d6dbb2f904c
> > >
> > > Can you please help me to investigate it, maybe something im doing
> wrong,
> > > because its look like a bug
> > >
> > >
> > >
> > > --
> > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> > >
> >
>

[jira] [Created] (IGNITE-9990) Control.sh utility should request a password if necessary.

[Alexand Polyakov (JIRA)]

Alexand Polyakov created IGNITE-9990:


 Summary: Control.sh utility should request a password if necessary.
 Key: IGNITE-9990
 URL: https://issues.apache.org/jira/browse/IGNITE-9990
 Project: Ignite
  Issue Type: New Feature
Affects Versions: 2.5
Reporter: Alexand Polyakov
Assignee: Alexand Polyakov


Since password in line parameters may not be safe (stored in console history in 
open form).
Use hidden input
{code:java}
Console console = System.console();
char passwordArray[] = console.readPassword("password: ");
{code}




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: NodeJS thin client: full API

[Igor Sapego]

Pavel,

Can we add a proper check and throw proper exception
when trying to deserialize enum value? NPE does not say
anything to a user.

Best Regards,
Igor


On Wed, Oct 24, 2018 at 5:34 PM Pavel Petroshenko 
wrote:

> Hi Stepan,
>
> Nodejs and PHP clients do not support enum type registration (and hence no
> tests). So enum type must be registered from somewhere else in order to be
> put or get from the Thin clients.
>
> If you register the type say from Java, then put/get for Enum values should
> work from the Thin clients. Can you please try this and let me know how it
> works.
>
> Thanks,
> p.
>
> On Tue, Oct 23, 2018 at 6:30 AM Stepan Pilshchikov <
> pilshchikov@gmail.com> wrote:
>
> > Alexey,
> >
> > I'm trying to get Enum from cache which placed by different thin client
> and
> > find that i can't get it, only exception
> >
> > And more when im trying to get Enum which is placed by nodejs client i
> also
> > can't do it, have same exception
> >
> > code and output -
> > https://gist.github.com/pilshchikov/ca9fda160e310f62dd4d1d6dbb2f904c
> >
> > Can you please help me to investigate it, maybe something im doing wrong,
> > because its look like a bug
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >
>

Re: NodeJS thin client: full API

[Pavel Petroshenko]

Hi Stepan,

Nodejs and PHP clients do not support enum type registration (and hence no
tests). So enum type must be registered from somewhere else in order to be
put or get from the Thin clients.

If you register the type say from Java, then put/get for Enum values should
work from the Thin clients. Can you please try this and let me know how it
works.

Thanks,
p.

On Tue, Oct 23, 2018 at 6:30 AM Stepan Pilshchikov <
pilshchikov@gmail.com> wrote:

> Alexey,
>
> I'm trying to get Enum from cache which placed by different thin client and
> find that i can't get it, only exception
>
> And more when im trying to get Enum which is placed by nodejs client i also
> can't do it, have same exception
>
> code and output -
> https://gist.github.com/pilshchikov/ca9fda160e310f62dd4d1d6dbb2f904c
>
> Can you please help me to investigate it, maybe something im doing wrong,
> because its look like a bug
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

Problem with reading incomplete payload - IGNITE-7153

[Michael Fong]

Hi, all,


I was trying to fix  IGNITE-7153 which relates to parsing incomplete REDIS
packet larger than 8192 bytes. However, I found a random problem  which is
reproducible on TC as well.
That said, GridNioServerRead.processRead() :
  - int cnt = sockCh.read(readBuf);

sometimes does not read the payload fully as the length field in the header
is larger than the ByteBuffer.limit(). As the result, the
BufferUnderFlowException will be thrown.

For example, in a erroneous round run with my IDE, a REDIS payload (sent by
jedis client) looks like the following:

2a 33 d a 24 33 d a 53 45 54 d a 24 32 d a 62 31 d a 24 {38 31 39 32} d a | 65
d a ...etc

GridRedisProtocolParser.readBulkStr(buf) invokes elCnt(buf) which gets
{8192}. However, the limit of buf is 28 which ends at | position. Obviously,
8192 < limit(), therefore, the logic throws BufferUnderFlow soon after.

I traced back and found  sockCh.read(readBuf) only read 28 bytes for
unknown reason

The problem seems totally random. I wonder if this has anything to do
with other NioWorker
or Selector setting.


Any help would be appreciated!


Regards,


Michael

[GitHub] ololo3000 opened a new pull request #42: IGNITE-9894 Change of JIRA link name

[GitBox]

ololo3000 opened a new pull request #42: IGNITE-9894 Change of JIRA link name
URL: https://github.com/apache/ignite-teamcity-bot/pull/42
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Re: [DISCUSSION] TDE. Phase-2. Master key rotation.

[Nikolay Izhikov]

Hello.

Deisgn updated [1]

Please, share your feedback

[1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652381


В Вт, 23/10/2018 в 21:49 +0300, Nikolay Izhikov пишет:
> Hello, Anton.
> 
> Thank you for your very usefull feedback!
> 
> I accept your proposals.
> Seems it makes this feature more robust and clear.
> 
> Will update design in confluence in a couple of hours.
> 
> В Вт, 23/10/2018 в 19:18 +0300, Anton Vinogradov пишет:
> > Nikolay,
> > 
> > I have some comments.
> > 
> > 1) Master key setup and removal is a responsibility of system administrator.
> > No matter how he will set a new master key or remove an old.
> > The only need it to have both keys, new and old, installed before starting
> > the rotation process.
> > 
> > 2) Master Key rotation is a process of cache's keys re-encryption.
> > 
> > So, we should send a message contains a new master key id only.
> > We also have to check that "Master Key rotation" allowed to the user by
> > checking it has SecurityPermission#ADMIN_OPS permission.
> > 
> > During this message handling, we have to re-encrypt keys and to set new
> > master key id.
> > 
> > 3) We should provide recovery mode for nodes unexpectedly leaved cluster
> > during "Master Key rotation" process.
> > We have to have a special "node start" command allows to change node's
> > master key before joining the cluster.
> > 
> > пн, 22 окт. 2018 г. в 22:39, Nikolay Izhikov :
> > 
> > > Hello, Igniters.
> > > 
> > > As you may know, we successfully implement TDE. Phase-1 feature. [1]
> > > This improvement allows users to use an encrypted cache.
> > > 
> > > To make TDE production ready I propose to extend it with two things:
> > > 
> > > * Master key rotation.
> > > * Cache key rotation.
> > > 
> > > Such features required by many security standards such as PCI DSS [2] and
> > > GDPR [3]
> > > 
> > > I think it would be easier to discuss, implement and review both features
> > > separately.
> > > So my plan is the following:
> > > 
> > > * TDE. Phase-2 - Master key rotation [4]
> > > * TDE. Phase-3 - Cache key rotation. [5]
> > > 
> > > I prepared designs for both parts.
> > > I want to specifically discuss Phase-2 design.
> > > Phase-3 design state is [EARLY DRAFT].
> > > I propose to use Phase-3 design as a reference to make sure we have a
> > > consistent view of all aspects of TDE
> > > and can be implemented without significant changes in earlier parts.
> > > 
> > > Below, my design.
> > > Following changes will be made in confluence [4].
> > > Please, share your feedback.
> > > 
> > > *TDE. PHASE-2. MASTER KEY ROTATION*
> > > Key rotation required in case of it compromising or at the end of
> > > crypto period(key validity period).
> > > 
> > > Goal:
> > > To implement the ability to rotate master encryption key.
> > > 
> > > New processes:
> > > 1. Master key rotation.
> > > 2. Removal of a master key.
> > > 
> > > New administrator commands:
> > > 1. Master keys view: node -> master key hash
> > > 2. Cache group keys view: node -> group name -> encryption key
> > > hash
> > > 
> > > MASTER KEY ROTATION:
> > > Process start:
> > > Administrator initiates key rotation via  some kind of
> > > user interface(CLI, Visor, Web Console, JMX, etc).
> > > 
> > > Process description:
> > > Message is sent by discovery.
> > > A Message should contain:
> > > * Master cache key encrypted with the current
> > > master key.
> > > 
> > > When server node processed message following actions are
> > > executed:
> > > * Blocks creation of encrypted cache key.
> > > * Encrypt cache group keys with new master key.
> > > * Unblock creation of encrypted cache key.
> > > 
> > > New joining node should also change the current master key
> > > with the new one.
> > > 
> > > Process completion:
> > > The administrator initiates process completion via the
> > > interface by using “master key removal” command.
> > > Design assumes an administrator will check that all nodes
> > > successfully change master key and all required nodes are alive.
> > > 
> > > MASTER KEY REMOVAL:
> > > Process start:
> > > Administrator initiates process via some kind of user
> > > interface(CLI, Visor, WebConsole, JMX, etc),
> > > 
> > > Process description:
> > > Message is sent by discovery.
> > > Message should contain:
> > > * New master key hash.
> > > When a server node processed message following actions are
> > > executed:
> > > Received master key hash compared with known master key
> > > hash.
> > > Previous master key removed using configured
> > > 

[GitHub] ololo3000 opened a new pull request #41: IGNITE-9940 Sort pull requests by update time

[GitBox]

ololo3000 opened a new pull request #41: IGNITE-9940 Sort pull requests by 
update time
URL: https://github.com/apache/ignite-teamcity-bot/pull/41
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ignite pull request #5064: IGNITE-7072: IgniteCache.replace(k, v, nv) fix fo...

[isapego]

GitHub user isapego opened a pull request:

https://github.com/apache/ignite/pull/5064

IGNITE-7072: IgniteCache.replace(k, v, nv) fix for binary mode



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7072

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5064.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5064


commit 76445c35b792f7b922ea0a74a5259193dedd4f78
Author: Igor Sapego 
Date:   2018-10-24T13:56:12Z

IGNITE-7072: IgniteCache.replace(k, v, nv) fix for binary mode




---

[GitHub] ignite pull request #5063: IGNITE-9753 Several optimization of validate_inde...

[ivandasch]

GitHub user ivandasch opened a pull request:

https://github.com/apache/ignite/pull/5063

IGNITE-9753 Several optimization of validate_indexes



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9753

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5063.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5063


commit 758a3d513e7e8a1d35d21ccb285d1d39a5663b96
Author: Ivan Daschinskiy 
Date:   2018-10-04T14:48:27Z

IGNITE-9753 Validate indexes closure improvements.

commit 837db1da8441fbb37512baf44d75b607f2bb38a3
Author: Ivan Daschinskiy 
Date:   2018-10-22T15:46:45Z

IGNITE-9753 wip.

commit b7f63ac2321e1f18694c79712c725b51c95ac210
Author: Ivan Daschinskiy 
Date:   2018-10-24T13:45:48Z

IGNITE-9753 wip.




---

[GitHub] ignite pull request #5062: IGNITE-9917 Write proper tests for start/stop cli...

[ibessonov]

GitHub user ibessonov opened a pull request:

https://github.com/apache/ignite/pull/5062

IGNITE-9917 Write proper tests for start/stop client.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9917

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5062.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5062


commit e3f0054d46a3dd0ddd708c47274ec5ddebbf
Author: ibessonov 
Date:   2018-10-22T14:12:13Z

IGNITE-9917 Start client test prototype that flacks all the time.

commit c018c5463935cd2b98c7323c87c9b928f5bb77e8
Author: ibessonov 
Date:   2018-10-24T13:27:42Z

IGNITE-9917 Proper tests for start/stop client, not "muted" yes.




---

[jira] [Created] (IGNITE-9989) JDBC v2: getPrimaryKeys always returns constant COLUMN_NAME, KEY_SEQ, PK_NAME

[Pavel Kuznetsov (JIRA)]

Pavel Kuznetsov created IGNITE-9989:
---

 Summary: JDBC v2: getPrimaryKeys always returns constant 
COLUMN_NAME, KEY_SEQ, PK_NAME
 Key: IGNITE-9989
 URL: https://issues.apache.org/jira/browse/IGNITE-9989
 Project: Ignite
  Issue Type: Bug
Reporter: Pavel Kuznetsov


Jdbc v2 driver has hardcoded values for meta attibutes : 
COLUMN_NAME = _KEY 
KEY_SEQ = 1
PK_NAME = _KEY

But this values should be different for different tables.

how to reproduce: 
1) connect to the cluser using jdbcv2 driver
2) CREATE TABLE TAB (ID LONG, SEC_ID LONG, VAL LONG, PRIMARY KEY(ID, SEC_ID))
3) check result of connection.getMetadata().getPrimaryKeys()




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Pre-touch for Ignite off-heap memory

[Павлухин Иван]

Hi Alex,

I wonder what problem you wish to solve. Pre-touching memory looks like a
solution.
But what is the problem? Am I getting it right that it is desired to avoid
OOM (of some kind)
in unpredictable moment of Ignite process execution? If so there could be
number of ways to tackle it.

I am not aware of means which can reserve physical memory for a process in
such way
that particular process will "own" that memory exclusively. Pre-touching
might not give such guarantees.

Actually, I think that one who deploys failure critical system should
carefully plan resources allocation.

I agree with Andrey here that it is worth adding some foolproof check. In
my mind it could be even preventing
a node startup if a requested data region size is greater than an estimated
amount of available memory.
And if such check (by some unusual reason) is not needed it could be
disabled with some property.


ср, 24 окт. 2018 г. в 14:41, David Harvey :

> Denis,
>
> We run must of our production DBs systems without any swapping space,
> because the 10-100x drop in throughput if such systems start paging makes
> them worse than useless.  However, we don't get OOM on them until all the
> pages are dirty, since LINUX will page out read-only (code) pages or memory
> mapped files.
>
> Dave Harvey
>
>
>
> On Wed, Oct 24, 2018 at 12:12 AM Denis Magda  wrote:
>
> > Alex,
> >
> > Correct me if I'm wrong, but even if an OS runs out of physical memory (X
> > GB in total) an Ignite node process still can request the X GB from
> virtual
> > memory. Yes, virtual memory can involve swapping and disk to satisfy your
> > request but this shouldn't be a reason of the OOM. Shouldn't OOM happen
> if
> > you're trying to allocate beyond the virtual memory capacity (beyond X
> GB)?
> >
> > Denis
> >
> > On Tue, Oct 23, 2018 at 12:08 PM Gerus  wrote:
> >
> > > Hi *Igniters*,
> > > Some time ago I've raised a suggestion for product improvement
> > > https://issues.apache.org/jira/browse/IGNITE-9112
> > >   .  It's all about
> > > off-heap memory allocation. Current implementation can have some
> > > improvements for failure critical systems. Ignite can have OOM in
> > runtime,
> > > because RAM can be used by OS, if it will not be pre-booked by
> operation
> > > system and this proposal is to address that. Common case is offheap and
> > > thats why memory segment cannot be managed by JVM that has
> > +AlwaysPreTouch
> > > option
> > > Obviously this implementation will make startup longer and thats why it
> > is
> > > proposed to use configuration flag to manage this feature
> > > I think, it will be useful to have this option. Are you supporting
> this?
> > >
> > >
> > >
> > > --
> > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> > >
> >
>


-- 
Best regards,
Ivan Pavlukhin

Re: Continuous query deployment

[Denis Mekhanikov]

Val,

I'm working on this issue right now.
I don't see an ultimate problem here. The discovery data can be
deserialized using a system class loader.
Serialized representation is stored in
*CacheContinuousQueryDeployableObject.*
The actual deserialization happens later, when any class loader can be used.

So, I think, peer class loading may be used for continuous queries, but not
for the ones with *autoUnsubscribe=false*

Denis

ср, 24 окт. 2018 г. в 5:35, Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Guys,
>
> Correct me if I am wrong, but to my knowledge peer class loading doesn't
> *really* work for CQs regardless of autoUnsubscribe flag. The issue is that
> filter is distributed via discovery messages when new node joins, but
> discovery uses JDK marshaller that is not aware of p2p loading. So classes
> are remotely loaded only on initial query execution, but not for the new
> nodes that join after it's already deployed.
>
> If that's the case, we should just find a right way of dynamic class
> loading that always works. Deployment SPI seems to be a good candidate,
> just like for services.
>
> -Val
>
> On Mon, Oct 22, 2018 at 1:46 AM Denis Mekhanikov 
> wrote:
>
> > Ilya,
> >
> > Disabling P2P class loading for CQ with *autoUnsubscribe=false* is a
> > minimum, that should be done.
> > But it will only solves one of the existing problems.
> > It's still unclear, how to undeploy such queries.
> > Continuous queries with *autoUnsubscribe=true* have different semantic
> with
> > the ones with *autoUnsubscribe=false*.
> >
> > The first of them is designed to deliver cache events to the subscriber.
> > They can't exist without the subscriber node, so they are automatically
> > undeployed, when the subscriber leaves.
> > I don't see any problems in using  P2P class loading for them.
> >
> > The second one is needed to deploy a set of listeners, that are
> collocated
> > to data.
> > In this case the subscriber is not needed, so we don't undeploy the
> query,
> > when the subscriber leaves.
> > But we still have a dependency on the subscriber, since only him can
> > undeploy the query or redeploy it with different parameters.
> > Such queries should have a name or Id, that other nodes will use to get a
> > handle of them.
> > And P2P class loading is not applicable here, since the query may outlive
> > the initiator.
> >
> > Val,
> >
> > We didn't start working on service hot redeployment yet.
> > We tentatively agreed, that Deployment SPI it a good candidate as a tool
> > for this task.
> >
> > Denis
> >
> > чт, 18 окт. 2018 г. в 18:44, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com>:
> >
> > > Denis,
> > >
> > > I agree that p2p loading is not the best choice for CQs. Do you know if
> > > we've already done anything in that area for the Service Grid? Are we
> > > using/going to use Deployment SPI there as well? Either way, I think it
> > > would make sense if services and CQs should end up using the same
> > mechanism
> > > for dynamic loading.
> > >
> > > -Val
> > >
> > > On Thu, Oct 18, 2018 at 8:36 AM Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hello!
> > > >
> > > > Is it possible to simply disable P2P class loading for remote filter
> of
> > > > continuous queries with autoUnsubscribe = true?
> > > >
> > > > Regards,
> > > > --
> > > > Ilya Kasnacheev
> > > >
> > > >
> > > > чт, 18 окт. 2018 г. в 18:31, Denis Mekhanikov  >:
> > > >
> > > > > Igniters,
> > > > >
> > > > > I'm concerned with our continuous query API and deployment
> procedure.
> > > > >
> > > > > Continuous queries have remote filters, that can be deployed over
> > peer
> > > > > class loading mechanism (this functionality is currently broken
> > though:
> > > > > IGNITE-3653 ,
> > > > > IGNITE-9181
> > > > > ).
> > > > > In usual cases P2P class loading makes sense, but problems begin
> > when a
> > > > > node, that deployed the CQ, leaves cluster. By default the CQ is
> > > > undeployed
> > > > > in this case. But ContinuousQuery has *autoUnsubscribe*
> > > > > <
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/query/ContinuousQuery.html#setAutoUnsubscribe-boolean-
> > > > > >
> > > > > property. If set to true, it lets a continuous query stay in
> cluster,
> > > > when
> > > > > initiator leaves.
> > > > > So, you may end up in a situation, when a continuous query remote
> > > filter
> > > > is
> > > > > deployed in a cluster, but the node, that the class was loaded
> from,
> > is
> > > > > already gone.
> > > > > New nodes won't have where to load the remote filter class from in
> > this
> > > > > case. Also existing nodes may have problems, since class loading of
> > > > > dependencies of the loaded class happens lazily (and cannot happen
> > > > > eagerly), so they also need the initiator node.
> > > > >
> > > > > The 

[jira] [Created] (IGNITE-9987) Do not block reading during client node start/stop.

[Ivan Bessonov (JIRA)]

Ivan Bessonov created IGNITE-9987:
-

 Summary: Do not block reading during client node start/stop.
 Key: IGNITE-9987
 URL: https://issues.apache.org/jira/browse/IGNITE-9987
 Project: Ignite
  Issue Type: Sub-task
Reporter: Ivan Bessonov
Assignee: Ivan Bessonov


Do not block reading during client node start/stop.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #5061: IGNITE-3467: Return "DATABASE" as catalog name.

[pavel-kuznetsov]

GitHub user pavel-kuznetsov opened a pull request:

https://github.com/apache/ignite/pull/5061

IGNITE-3467: Return "DATABASE" as catalog name.

Instead of returning "null" we now return "DATABASE" string constant.
Changes affect only jdbc v2 and thin driver (no changes in server code).
Also tests have been update according to the new behaviour.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-3467

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5061.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5061


commit 815807fa8155e884d25fb8437c554c3743583a30
Author: Pavel Kuznetsov 
Date:   2018-10-23T13:41:10Z

ignite-3467: Return "DATABASE" as catalog name.

Instead of returning "null" we now return "DATABASE" string constant.
Changes affect only jdbc v2 and thin driver (no changes in server code).
Also tests have been update according to the new behaviour.

commit a1cb69590d63c6e89f90e99c6f9fd57e6d5dc427
Author: Pavel Kuznetsov 
Date:   2018-10-24T12:54:43Z

ignite-3467: Added assertion message.




---

[jira] [Created] (IGNITE-9986) TcpDiscoverySelfTest.testNodeShutdownOnRingMessageWorkerStartNotFinished is flaky

[Ryabov Dmitrii (JIRA)]

Ryabov Dmitrii created IGNITE-9986:
--

 Summary:  
TcpDiscoverySelfTest.testNodeShutdownOnRingMessageWorkerStartNotFinished is 
flaky
 Key: IGNITE-9986
 URL: https://issues.apache.org/jira/browse/IGNITE-9986
 Project: Ignite
  Issue Type: Bug
Reporter: Ryabov Dmitrii
Assignee: Ryabov Dmitrii


Fails: 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8==testDetails=-6988766947783986136=TEST_STATUS_DESC=50_IgniteTests24Java8=



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] asfgit closed pull request #36: IGNITE-9645 [TC Bot] Add comparison of failed tests lists in two date intervals

[GitBox]

asfgit closed pull request #36: IGNITE-9645 [TC Bot] Add comparison of failed 
tests lists in two date intervals
URL: https://github.com/apache/ignite-teamcity-bot/pull/36
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITeamcity.java 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITeamcity.java
index 6aa3b60..628581f 100644
--- a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITeamcity.java
+++ b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITeamcity.java
@@ -22,6 +22,7 @@
 import java.util.Date;
 import java.util.List;
 import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.Executor;
 import java.util.concurrent.ExecutorService;
 import javax.annotation.Nonnull;
 import javax.annotation.Nullable;
@@ -36,15 +37,18 @@
 import org.apache.ignite.ci.tcmodel.conf.BuildType;
 import org.apache.ignite.ci.tcmodel.hist.BuildRef;
 import org.apache.ignite.ci.tcmodel.result.Build;
+import org.apache.ignite.ci.tcmodel.result.Configurations;
 import org.apache.ignite.ci.tcmodel.result.issues.IssuesUsagesList;
 import org.apache.ignite.ci.tcmodel.result.problems.ProblemOccurrences;
 import org.apache.ignite.ci.tcmodel.result.stat.Statistics;
 import org.apache.ignite.ci.tcmodel.result.tests.TestOccurrence;
 import org.apache.ignite.ci.tcmodel.result.tests.TestOccurrenceFull;
 import org.apache.ignite.ci.tcmodel.result.tests.TestOccurrences;
+import org.apache.ignite.ci.tcmodel.result.tests.TestRef;
 import org.apache.ignite.ci.tcmodel.user.User;
 import org.apache.ignite.ci.teamcity.pure.ITeamcityConn;
 import org.apache.ignite.ci.util.Base64Util;
+import org.apache.ignite.ci.web.rest.parms.FullQueryParams;
 import org.jetbrains.annotations.NotNull;
 
 import static com.google.common.base.Strings.isNullOrEmpty;
@@ -150,10 +154,12 @@ default Build getBuild(int id) {
  * @param build
  * @return
  */
-ProblemOccurrences getProblems(Build build);
+ProblemOccurrences getProblems(BuildRef build);
 
 TestOccurrences getTests(String href, String normalizedBranch);
 
+TestOccurrences getFailedTests(String href, int count, String 
normalizedBranch);
+
 Statistics getBuildStatistics(String href);
 
 CompletableFuture getTestFull(String href);
@@ -162,6 +168,10 @@ default Build getBuild(int id) {
 
 ChangesList getChangesList(String href);
 
+CompletableFuture getTestRef(FullQueryParams key);
+
+Configurations getConfigurations(FullQueryParams key);
+
 /**
  * List of build's related issues.
  *
@@ -251,6 +261,8 @@ default SingleBuildRunCtx loadTestsAndProblems(@Nonnull 
Build build, @Deprecated
 
 void setExecutor(ExecutorService pool);
 
+Executor getExecutor();
+
 
 /**
  * @param tok TeamCity authorization token.
diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/IgnitePersistentTeamcity.java
 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/IgnitePersistentTeamcity.java
index d1e94e4..60f61ba 100644
--- 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/IgnitePersistentTeamcity.java
+++ 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/IgnitePersistentTeamcity.java
@@ -32,6 +32,7 @@
 import java.util.concurrent.CompletableFuture;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.Executor;
 import java.util.concurrent.ExecutorService;
 import java.util.concurrent.TimeUnit;
 import java.util.concurrent.atomic.AtomicReference;
@@ -66,16 +67,19 @@
 import org.apache.ignite.ci.tcmodel.conf.BuildType;
 import org.apache.ignite.ci.tcmodel.hist.BuildRef;
 import org.apache.ignite.ci.tcmodel.result.Build;
+import org.apache.ignite.ci.tcmodel.result.Configurations;
 import org.apache.ignite.ci.tcmodel.result.issues.IssuesUsagesList;
 import org.apache.ignite.ci.tcmodel.result.problems.ProblemOccurrences;
 import org.apache.ignite.ci.tcmodel.result.stat.Statistics;
 import org.apache.ignite.ci.tcmodel.result.tests.TestOccurrence;
 import org.apache.ignite.ci.tcmodel.result.tests.TestOccurrenceFull;
 import org.apache.ignite.ci.tcmodel.result.tests.TestOccurrences;
+import org.apache.ignite.ci.tcmodel.result.tests.TestRef;
 import org.apache.ignite.ci.tcmodel.user.User;
 import org.apache.ignite.ci.util.CacheUpdateUtil;
 import org.apache.ignite.ci.util.CollectionUtil;
 import org.apache.ignite.ci.util.ObjectInterner;
+import org.apache.ignite.ci.web.rest.parms.FullQueryParams;
 import org.jetbrains.annotations.NotNull;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -104,6 +108,8 @@
 private static final String BUILD_HIST_FINISHED = "buildHistFinished";
 private static final 

[GitHub] ignite pull request #5060: IGNITE-9890 Refactor CachePartitionPartialCounter...

[vldpyatkov]

GitHub user vldpyatkov opened a pull request:

https://github.com/apache/ignite/pull/5060

IGNITE-9890 Refactor CachePartitionPartialCountersMap#fromCountersMap…

…, no need to sort partitions any time it called.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9890

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5060.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5060


commit d28c62f134807da102a31689c383f7ae5fe5e8c3
Author: vd-pyatkov 
Date:   2018-10-24T12:07:15Z

IGNITE-9890 Refactor CachePartitionPartialCountersMap#fromCountersMap, no 
need to sort partitions any time it called.




---

Re: Pre-touch for Ignite off-heap memory

[David Harvey]

Denis,

We run must of our production DBs systems without any swapping space,
because the 10-100x drop in throughput if such systems start paging makes
them worse than useless.  However, we don't get OOM on them until all the
pages are dirty, since LINUX will page out read-only (code) pages or memory
mapped files.

Dave Harvey



On Wed, Oct 24, 2018 at 12:12 AM Denis Magda  wrote:

> Alex,
>
> Correct me if I'm wrong, but even if an OS runs out of physical memory (X
> GB in total) an Ignite node process still can request the X GB from virtual
> memory. Yes, virtual memory can involve swapping and disk to satisfy your
> request but this shouldn't be a reason of the OOM. Shouldn't OOM happen if
> you're trying to allocate beyond the virtual memory capacity (beyond X GB)?
>
> Denis
>
> On Tue, Oct 23, 2018 at 12:08 PM Gerus  wrote:
>
> > Hi *Igniters*,
> > Some time ago I've raised a suggestion for product improvement
> > https://issues.apache.org/jira/browse/IGNITE-9112
> >   .  It's all about
> > off-heap memory allocation. Current implementation can have some
> > improvements for failure critical systems. Ignite can have OOM in
> runtime,
> > because RAM can be used by OS, if it will not be pre-booked by operation
> > system and this proposal is to address that. Common case is offheap and
> > thats why memory segment cannot be managed by JVM that has
> +AlwaysPreTouch
> > option
> > Obviously this implementation will make startup longer and thats why it
> is
> > proposed to use configuration flag to manage this feature
> > I think, it will be useful to have this option. Are you supporting this?
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >
>

[GitHub] ignite pull request #5023: IGNITE-9923: remove unused imports

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5023


---

Re: Abbreviation code-style requirement.

[Eduard Shangareev]

Igniters,
thank you for your feedback.

I haven't seen any arguments against making abbreviation optional and not
mandatory.
So, could we update our wiki with code style to reflect our new vision on
abbreviations?

On Tue, Oct 23, 2018 at 2:01 PM Dmitriy Pavlov 
wrote:

> Hi Ivan
>
> if by conflict we mean arguing and fighting it is definitely should be
> avoided, it never helps the community.
>
> But if we mean different opinions on details (variable namings, method
> structure, etc), such different views are unavoidable and I find it is
> perfectly ok that people with different background have different views.
> The paramount thing here if we can solve such conflicts with a positive
> outcome for all community and for the codebase.
>
> The good friend of mine reminded me some time ago that we all have a common
> goal here: make the community bigger and this project better. If we always
> remember that we are connected by a common interest but we admit each
> contributor may have different preferences in coding and probably different
> opinion. We may build up consensus sharing our arguments if it is really
> needed, or these different opinions/priorities/preferences may co-exist.
>
> In a particular case, if reviewer's concerns are not major, another
> reviewer can agree with your proposal. So it should be always considered
> case-by-case, there is no silver bullet here.
>
> Sincerely,
> Dmitriy Pavlov
>
> вт, 23 окт. 2018 г., 11:32 Maxim Muzafarov :
>
> > Igniters,
> >
> > I think it's easy to disable the code style abbreviation plugin option by
> > switching off
> > the checkbox on - File | Settings | Inspections | Apache Ignite |
> Incorrect
> > Java abbreviation usage.
> >
> > +1 to make abbreviation not mandatory, but I'd like to keep it for common
> > variable names like `context = ctx`.
> >
> > On Mon, 22 Oct 2018 at 14:05 Павлухин Иван  wrote:
> >
> > > Hi all,
> > >
> > > I also think that abbreviations should not be mandatory (point 3).
> > > But what I am worrying about is a conflict resolution between a patch
> > > submitter and a reviewer.
> > > How to come to an agreement when one side is strictly for and another
> > side
> > > is strictly against
> > > using abbreviations in some concrete case?
> > >
> > > вс, 21 окт. 2018 г. в 11:34, Dmitriy Pavlov :
> > >
> > > > +1 for proposal 3.
> > > >
> > > > 1. I'm not sure we need to revisit all abbreviations as a lot of
> people
> > > get
> > > > used to it.
> > > > 2. I'm not sure multiword is always need to be fully named, sometimes
> > it
> > > > may be ok to abbreviate.
> > > > 3. But I agree with abbreviations should not be mandatory.
> > > >
> > > > Abbreviated and short names like i,j,cp and etc. are good for simple
> > > > methods and code blocks; for a fast demonstration of some idea, but
> for
> > > > complex enterprise level software it can hide meaning instead of
> > clearly
> > > > showing it.
> > > >
> > > > As a next step, I would like to propose to contribute an option to
> > > disable
> > > > abbreviation requirements for some cases in ignite-abbrev-plugin.
> > > >
> > > > сб, 20 окт. 2018 г. в 10:47, Zhenya :
> > > >
> > > > > +1 for all proposals.
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
> > --
> > --
> > Maxim Muzafarov
> >
>

[jira] [Created] (IGNITE-9985) MVCC TX: fix backup mappings

[Igor Seliverstov (JIRA)]

Igor Seliverstov created IGNITE-9985:


 Summary: MVCC TX: fix backup mappings
 Key: IGNITE-9985
 URL: https://issues.apache.org/jira/browse/IGNITE-9985
 Project: Ignite
  Issue Type: Task
  Components: mvcc
Reporter: Igor Seliverstov
Assignee: Igor Seliverstov


Fix mappings to update going to be evicted partitions as well to maintain 
consistency during rebalance



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9984) Documentation for EVT_MANAGEMENT_TASK_STARTED will be required.

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-9984:
--

 Summary: Documentation for EVT_MANAGEMENT_TASK_STARTED will be 
required.
 Key: IGNITE-9984
 URL: https://issues.apache.org/jira/browse/IGNITE-9984
 Project: Ignite
  Issue Type: Task
  Components: documentation
Affects Versions: 2.8
Reporter: Andrey Aleksandrov
 Fix For: 2.8


New EVT_MANAGEMENT_TASK_STARTED will be added in future release. Documentation 
for it should be added too.

Next information should be added to web console and visor documentation:

EVT_MANAGEMENT_TASK_STARTED provide the possibility to track next tasks that 
could be started by the user from web console and visor during some operations:

+Baseline:+

VisorBaselineTask - Task that will collect information about baseline topology 
and can change its state

+Binaries:+

VisorBinaryMetadataCollectorTask - Task that collects binary metadata.

+Services:+

VisorCancelServiceTask - Task for cancel services with the specified name.

+Metrics:+

VisorComputeResetMetricsTask - Task for cancel services with specified name.

+Caches:+

VisorCacheLostPartitionsTask - Collect list of lost partitions.
VisorCacheResetLostPartitionsTask - Reset lost partitions for caches.
VisorCacheStartTask - Task that start cache or near cache with specified 
configuration.
VisorCacheStopTask - Task that stop specified caches on specified node.
VisorCacheAffinityNodeTask - Task that will find affinity node for key.
VisorCacheModifyTask - Task that modify value in specified cache.
VisorCacheRebalanceTask - Pre-loads caches. Made callable just to conform 
common pattern.
VisorCacheLoadTask - Task to loads caches.
VisorCacheClearTask - Task that clears specified caches on specified node.

+Queries+:

VisorQueryResetMetricsTask - Reset compute grid query metrics.
VisorQueryTask - Task for executing SQL fields query and get the first page of 
results.
VisorQueryCancelTask - Task to cancel queries.

+Computes:+

VisorComputeResetMetricsTask - Reset compute grid metrics.
VisorComputeCancelSessionsTask - Cancels given tasks sessions.

+DEBUG:+

VisorThreadDumpTask - Creates a thread dump.

+IGFS:+

VisorIgfsFormatTask - Format IGFS instance.
VisorIgfsProfilerClearTask - Remove all IGFS profiler logs.
VisorIgfsResetMetricsTask - Resets IGFS metrics.

+LOGS:+

VisorLogSearchTask - Search text matching in logs

+CLUSTER:+

VisorChangeGridActiveStateTask - Task for changing grid active state.
VisorNodeGcTask - Task to run gc on nodes.
VisorNodeRestartTask - Restarts nodes.
VisorNodeStopTask - Stops nodes.

 

{color:#33}

{color}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #5059: IGNITE-9983: add ignite inspections config for te...

[Mmuzaf]

GitHub user Mmuzaf opened a pull request:

https://github.com/apache/ignite/pull/5059

IGNITE-9983: add ignite inspections config for teamcity



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Mmuzaf/ignite ignite-9983

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5059.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5059


commit 4cfb74dd2cd545bf0148fc546eb6a7bfd657ffed
Author: Maxim Muzafarov 
Date:   2018-10-24T10:18:56Z

IGNITE-9983: add ignite inspections config for teamcity




---

[jira] [Created] (IGNITE-9983) Add an inspection configuration for TC suite with enabled short list of rules

[Maxim Muzafarov (JIRA)]

Maxim Muzafarov created IGNITE-9983:
---

 Summary: Add an inspection configuration for TC suite with enabled 
short list of rules
 Key: IGNITE-9983
 URL: https://issues.apache.org/jira/browse/IGNITE-9983
 Project: Ignite
  Issue Type: Task
Reporter: Maxim Muzafarov
Assignee: Maxim Muzafarov


Currently, some of the inspection rules fixed over the whole Apache Ignite 
Project:
IGNITE-9923
IGNITE-9652
IGNITE-9597
IGNITE-9311

We need to create an inspection configuration for the TC suite `Inspections: 
Core` with enabled only these rules.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] asfgit closed pull request #39: IGNITE-9901 Persistent queue for visas added

[GitBox]

asfgit closed pull request #39: IGNITE-9901 Persistent queue for visas added
URL: https://github.com/apache/ignite-teamcity-bot/pull/39
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITcHelper.java 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITcHelper.java
index 00623f7..356b633 100644
--- a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITcHelper.java
+++ b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITcHelper.java
@@ -17,6 +17,7 @@
 
 package org.apache.ignite.ci;
 
+import java.util.Collection;
 import java.util.List;
 import org.apache.ignite.ci.issue.IssueDetector;
 import org.apache.ignite.ci.issue.IssuesStorage;
@@ -24,8 +25,6 @@
 import org.apache.ignite.ci.user.ICredentialsProv;
 import org.apache.ignite.ci.user.UserAndSessionsStorage;
 
-import java.util.Collection;
-
 /**
  * Teamcity Bot main interface
  */
@@ -52,6 +51,15 @@
 
 List getTrackedBranchesIds();
 
+/** */
+void setServerAuthorizerCreds(ICredentialsProv creds);
+
+/** */
+ICredentialsProv getServerAuthorizerCreds();
+
+/** */
+boolean isServerAuthorized();
+
 /**
  * @param srvId Server id.
  * @param prov Credentials.
diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/TcHelper.java 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/TcHelper.java
index 5e0422d..1bd64e9 100644
--- a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/TcHelper.java
+++ b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/TcHelper.java
@@ -54,6 +54,9 @@
 /** Stop guard. */
 private AtomicBoolean stop = new AtomicBoolean();
 
+/** Server authorizer credentials. */
+private ICredentialsProv serverAuthorizerCreds;
+
 @Inject private IssuesStorage issuesStorage;
 
 @Inject private ITcServerProvider serverProvider;
@@ -67,6 +70,21 @@
 public TcHelper() {
 }
 
+/** {@inheritDoc} */
+@Override public void setServerAuthorizerCreds(ICredentialsProv creds) {
+this.serverAuthorizerCreds = creds;
+}
+
+/** {@inheritDoc} */
+@Override public ICredentialsProv getServerAuthorizerCreds() {
+return serverAuthorizerCreds;
+}
+
+/** {@inheritDoc} */
+@Override public boolean isServerAuthorized() {
+return !Objects.isNull(serverAuthorizerCreds);
+}
+
 /** {@inheritDoc} */
 @Override public IssuesStorage issues() {
 return issuesStorage;
diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/observer/BuildObserver.java
 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/observer/BuildObserver.java
index f8a7ac3..1e08eb8 100644
--- 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/observer/BuildObserver.java
+++ 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/observer/BuildObserver.java
@@ -17,8 +17,8 @@
 
 package org.apache.ignite.ci.observer;
 
+import java.util.Collection;
 import java.util.Objects;
-import java.util.Queue;
 import java.util.Timer;
 import javax.inject.Inject;
 import org.apache.ignite.ci.tcmodel.result.Build;
@@ -61,7 +61,7 @@ public void stop() {
  * @param ticket JIRA ticket name.
  */
 public void observe(String srvId, ICredentialsProv prov, String ticket, 
Build... builds) {
-observerTask.builds.add(new BuildsInfo(srvId, prov, ticket, builds));
+observerTask.addBuild(new BuildsInfo(srvId, prov, ticket, builds));
 }
 
 /**
@@ -70,7 +70,7 @@ public void observe(String srvId, ICredentialsProv prov, 
String ticket, Build...
  */
 public String getObservationStatus(String srvId, String branch) {
 StringBuilder sb = new StringBuilder();
-Queue builds = observerTask.builds;
+Collection builds = observerTask.getBuilds();
 
 for (BuildsInfo bi : builds) {
 if (Objects.equals(bi.branchName, branch)
diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/observer/BuildsInfo.java
 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/observer/BuildsInfo.java
index e318f19..ba8b48a 100644
--- 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/observer/BuildsInfo.java
+++ 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/observer/BuildsInfo.java
@@ -37,9 +37,6 @@
 /** Branch name. */
 public final String branchName;
 
-/** Credentials. */
-public final ICredentialsProv prov;
-
 /** JIRA ticket full name. */
 public final String ticket;
 
@@ -54,7 +51,6 @@
  */
 public BuildsInfo(String srvId, ICredentialsProv prov, String ticket, 
Build[] builds) {
 this.srvId = srvId;
-this.prov = prov;
 this.ticket = ticket;
 

[GitHub] ignite pull request #5006: IGNITE-9881 Management events for webconsole and ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5006


---

Re: [DISCUSSION] Spark Data Frame through Thin Client

[Nikolay Izhikov]

Hello, Valentin.

> What I don't agree with is that replacing thick client with thin client is a 
> way to fix usability issues. 

I think it will fix some of them.

> will potentially compromise the performance

As I mentioned earlier, I want to provide easy way to play with integration.
For maximum performance one should use client nodes.

> What is the difference between thin and thick client from this point of view?

We need only 1 jar file.
All config options we need is list of ip addressed.

> I'm not arguing there are usability issues with thick client. 
> I'm just suggesting to fix those issues first, before we jump reworking the 
> implementation.

> My suggestion is to look at usability issues and try to fix them without 
> getting rid of thick client.

I agree, let's do it!
Can you create some tickets?
I'm ready to look at it and contribute a fix.

В Вт, 23/10/2018 в 19:31 -0700, Valentin Kulichenko пишет:
> Nikolay,
> 
> Please see my comments below. Actually, I haven't made most of the
> assumptions that you mentioned, and I generally agree with you. What I
> don't agree with is that replacing thick client with thin client is a way
> to fix usability issues. Thin client is not going to be issue-free either,
> but will potentially compromise the performance, as well as functionality
> (like streaming, as Stephen mentioned). My suggestion is to look at
> usability issues and try to fix them without getting rid of thick client.
> 
> -Val
> 
> On Sun, Oct 21, 2018 at 1:08 AM Nikolay Izhikov  wrote:
> 
> > Valentin.
> > 
> > Seems, You made several suggestions, which is not always true, from my
> > point of view:
> > 
> > 1. "We have access to Spark cluster installation to perform deployment
> > steps" - this is not true in cloud or enterprise environment.
> > 
> 
> Can you please elaborate on this? What is the difference between thin and
> thick client from this point of view? I understand that the latter would
> generally be more complicated, but how would one use thin client without
> deploying a JAR?
> 
> 
> > 
> > 2. "Spark cluster is used only for Ignite integration".
> > From what I know computational resources for big Spark cluster is divided
> > by many business divisions.
> > And it is not convenient to perform some deployment steps on this cluster.
> > 
> 
> Same as #1. Regardless how we use the Spark cluster, we need to deploy a
> JAR in case of thin client, no?
> 
> 
> > 
> > 3. "When Ignite + Spark are used in real production it's OK to have
> > reasonable deployment overhead"
> > What about developer who want to play with this integration?
> > And want to do it quickly to see how it works in real life examples.
> > Can we do his life much easier?
> > 
> 
> We can and we should :) I'm not arguing there are usability issues with
> thick client. I'm just suggesting to fix those issues first, before we jump
> reworking the implementation.
> 
> 
> > 
> > > First of all, they will exist with thin client either.
> > 
> > Spark have an ability to deploy jars on worker and add it to application
> > tasks classpath.
> > For 2.6 we must deploy 11 additional jars to start using Ignite.
> > Please, see my example on the bottom of documentation page [1]
> > 
> > Does cache-api-1.0.0.jar and h2-1.4.195.jar seems like obvious
> > dependencies for Ignite integration for you?
> > And for our users? :)
> > 
> 
> No, this is not obvious. Absolutely, this is a usability issue and we
> should think how to make user's life easier.
> 
> 
> > 
> > Actually, list of dependencies will be changed in 2.7 - new version of
> > jcache, new version of h2
> > So user should change it in code or perform additional deployment steps.
> > 
> > It overkill for me.
> > 
> > On the other hand - thin client requires only 1 jar.
> > Moreover, thin client protocol have the backward compatibility.
> > So thin client will perform correctly when Ignite cluster will be updated
> > from 2.6 to 2.7.
> > So, with Spark integration via thin client we will be able to update
> > Ignite cluster and Spark integration separately.
> > For now, we should do it in one big step.
> > 
> > What do you think?
> > 
> > [1] https://apacheignite-fs.readme.io/docs/installation-deployment
> > 
> > В Сб, 20/10/2018 в 18:33 -0700, Valentin Kulichenko пишет:
> > > Guys,
> > > 
> > > From my experience, Ignite and Spark clusters typically run in the same
> > > environment, which makes client node a more preferable option. Mainly,
> > > because of performance. BTW, I doubt partition-awareness on thin client
> > > will help either, because in dataframes we only run SQL queries and I
> > > believe thin client will execute them through a proxy anyway. But correct
> > > me if I’m wrong.
> > > 
> > > Either way, it sounds like we just have usability issues with
> > 
> > Ignite/Spark
> > > integration. Why don’t we concentrate on fixing them then? For example,
> > 
> > #3
> > > can be fixed by loading XML content on master and then distributing it to
> > > 

[jira] [Created] (IGNITE-9982) SQLLine: can't run with option --autoCommit=false or true

[Stepan Pilschikov (JIRA)]

Stepan Pilschikov created IGNITE-9982:
-

 Summary: SQLLine: can't run with option --autoCommit=false or true
 Key: IGNITE-9982
 URL: https://issues.apache.org/jira/browse/IGNITE-9982
 Project: Ignite
  Issue Type: Bug
  Components: mvcc, sql
Affects Versions: 2.7
Reporter: Stepan Pilschikov


Trying to run sqlline with additional options
{code}
$ /bin/sqlline.sh --autoCommit=false --color=true --outputFormat=csv 
--showNestedErrs=true --showWarnings=true --verbose=true -u 
jdbc:ignite:thin://127.0.0.1:10800
{code}

SQLline is running but with exception:
{code}
issuing: !connect jdbc:ignite:thin://127.0.0.1:10800 '' '' 
org.apache.ignite.IgniteJdbcThinDriver
Connecting to jdbc:ignite:thin://127.0.0.1:10800
Connected to: Apache Ignite (version 2.7.1#20181023-sha1:0ccde7c4)
Driver: Apache Ignite Thin JDBC Driver (version 2.7.1#20181023-sha1:0ccde7c4)
Error: MVCC must be enabled in order to invoke transactional operation: COMMIT 
(state=25000,code=5002)
java.sql.SQLException: MVCC must be enabled in order to invoke transactional 
operation: COMMIT
at 
org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.sendRequest(JdbcThinConnection.java:750)
at 
org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute0(JdbcThinStatement.java:212)
at 
org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute(JdbcThinStatement.java:475)
at 
org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.doCommit(JdbcThinConnection.java:369)
at 
org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.setAutoCommit(JdbcThinConnection.java:328)
at sqlline.DatabaseConnection.connect(DatabaseConnection.java:178)
at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:204)
at sqlline.Commands.connect(Commands.java:1095)
at sqlline.Commands.connect(Commands.java:1001)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
at sqlline.SqlLine.dispatch(SqlLine.java:791)
at sqlline.SqlLine.initArgs(SqlLine.java:566)
at sqlline.SqlLine.begin(SqlLine.java:643)
at sqlline.SqlLine.start(SqlLine.java:373)
at sqlline.SqlLine.main(SqlLine.java:265)
Transaction isolation: TRANSACTION_REPEATABLE_READ
{code}

Without --autoCommit option sqlline running without this exception



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #5011: IGNITE-9899 Fix processing of TX cache backups by...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5011


---

Re: Apache Ignite 2.7. Last Mile

[Nikolay Izhikov]

Hello, Igniters.

We have 3 ticket mapped to 2.7 today:

Igor Seliverstov - IGNITE-9892 - MVCC: Exchange hangs on mvcc coordinator fail
Roman Kondakov   - IGNITE-9828 - MVCC: Continuous query failover.
Roman Kondakov   - IGNITE-9928 - MVCC TX: Bug in SQL query mapping.

В Вт, 23/10/2018 в 15:01 +0300, Nikolay Izhikov пишет:
> Hello, Dmitriy.
> 
> I'm OK with including this patch to 2.7.
> 
> Can you ensure it completely fix the issue?
> I left comment in ticket.
> 
> https://issues.apache.org/jira/browse/IGNITE-9854?focusedCommentId=16660516=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16660516
> 
> 
> В Вт, 23/10/2018 в 13:10 +0300, Dmitriy Govorukhin пишет:
> > Nikolay,
> > 
> > I have an issue which I want to include in 2.7 release,
> > https://issues.apache.org/jira/browse/IGNITE-9854
> > It is a very small fix but very important, it protects us from NPE in some
> > race scenario.
> > Changes already in master, but the issue still not resolve, need your
> > approval for cherry-picking changes to ignite-2.7.
> > 
> > On Tue, Oct 23, 2018 at 10:51 AM Vladimir Ozerov 
> > wrote:
> > 
> > > Igniters,
> > > 
> > > There are still tickets in the scope as we continue finding new issues
> > > during QA. I propose the following plan: if there are still opened issues
> > > by Friday, then shift vote date for 1 week, to 2nd November. This is 
> > > needed
> > > to ensure that product quality is sufficient. But if the backlog is empty
> > > by Friday, we can go ahead with vote.
> > > 
> > > Thoughts?
> > > 
> > > On Tue, Oct 23, 2018 at 10:41 AM Nikolay Izhikov 
> > > wrote:
> > > 
> > > > Hello, Igniters.
> > > > 
> > > > We have 7 tickets mapped to 2.7 today.
> > > > 
> > > > Igov Seliverstov  - IGNITE-9892 - MVCC: Exchange hangs on mvcc
> > > 
> > > coordinator
> > > > fail
> > > > Igor Seliverstov  - IGNITE-9911 -
> > > > 
> > > 
> > > CacheMvccSelectForUpdateQueryAbstractTest#testSelectForUpdateAfterAbortedTx
> > > > periodically hangs
> > > > Roman Kondakov- IGNITE-9928 - MVCC TX: Bug in SQL query mapping.
> > > > Roman Kondakov- IGNITE-9663 - MVCC: Data node failure can cause TX
> > > > hanging.
> > > > Vladimir Ozerov   - IGNITE-9960 - SQL: Revert and reopen lazy flag
> > > > optimization (IGNITE-9171)
> > > > Pavel Petroshenko - IGNITE-9951 - thin php: Date data type cut nanos
> > > > Peter Ivanov  - IGNITE-9953 - Dropping hadoop accelerator downloads
> > > > 
> > > > 
> > > > В Вс, 21/10/2018 в 11:23 +0300, Nikolay Izhikov пишет:
> > > > > Hell, Denis.
> > > > > 
> > > > > I just filter all 2.7 tickets without documentation.
> > > > > 
> > > > > 
> > > > > 
> > > 
> > > https://issues.apache.org/jira/browse/IGNITE-9932?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20is%20not%20empty%20AND%20fixVersion%20in%20(%272.7%27)%20AND%20status%20NOT%20IN%20(Resolved%2C%20Closed)%20and%20(component%20is%20null%20or%20component%20NOT%20IN%20(documentation)))%20ORDER%20BY%20%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20%20
> > > > > 
> > > > > I've added this view to the release page.
> > > > > See sesction "Unresolved tickets(without documentation)".
> > > > > 
> > > > > В Сб, 20/10/2018 в 16:04 -0700, Denis Magda пишет:
> > > > > > Nikolay,
> > > > > > 
> > > > > > Where do you track those 8 blockers? Can't find them on this wiki
> > > 
> > > page:
> > > > > > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.7
> > > > > > 
> > > > > > --
> > > > > > Denis
> > > > > > 
> > > > > > On Sat, Oct 20, 2018 at 11:31 AM Nikolay Izhikov <
> > > 
> > > nizhi...@apache.org>
> > > > > > wrote:
> > > > > > 
> > > > > > > Hello, Denis.
> > > > > > > 
> > > > > > > As a first time release manager I'm trying to rely on Ignite
> > > 
> > > veterans
> > > > > > > opinion.
> > > > > > > Guys told me that we must fix all blockers and only after it make
> > > 
> > > the
> > > > > > > release.
> > > > > > > 
> > > > > > > Let's fix them all.
> > > > > > > What do you think?
> > > > > > > 
> > > > > > > > When are we sending a release candidate for vote?
> > > > > > > 
> > > > > > > When all blocker bugs will be fixed.
> > > > > > > Currently, we have *8*.
> > > > > > > 
> > > > > > > В Пт, 19/10/2018 в 13:50 -0700, Denis Magda пишет:
> > > > > > > > Guys, as a side observer of the current release, this all looks
> > > > 
> > > > like a
> > > > > > > > never ending story :)
> > > > > > > > 
> > > > > > > > When are we sending a release candidate for vote?
> > > > > > > > 
> > > > > > > > --
> > > > > > > > Denis
> > > > > > > > 
> > > > > > > > On Fri, Oct 19, 2018 at 4:39 AM Nikolay Izhikov <
> > > > 
> > > > nizhi...@apache.org>
> > > > > > > 
> > > > > > > wrote:
> > > > > > > > 
> > > > > > > > > Hello, Igniters.
> > > > > > > > > 
> > > > > > > > > We have 6 tickets for 2.7
> > > > > > > > > 
> > > > > > > > > Roman Kondakov - IGNITE-9892, IGNITE-9663, IGNITE-9928
> > > > > > > > > Igor Seliverstov - IGNITE-9911
> > > > > > > > > Ivan Pavlukhin - IGNITE-5935, 

[GitHub] ignite pull request #5030: IGNITE-9914 logic how to get subjectId for tasks ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5030


---

Re: Pre-touch for Ignite off-heap memory

[Andrey Mashenkov]

Alex,

I think offheap memory preallocation will not be helpful as you expect:
1. We still will not have any guarantee JVM process won't be killed by
OOM-killer due to memory overcommiting e.g. for GC needs.
as JVM allocates memory for internal needs in same process memory space.
2. Startup process will costs as we'll need to touch every offheap page.

We can't try to add some checks on startup to warn a user in case of low
physical memory (based on ram size + swap size + overcommit settings),
but I have no idea how we can estimate memory for JVM internal needs
(threads, GC and others).

On Wed, Oct 24, 2018 at 11:25 AM Andrey Mashenkov <
andrey.mashen...@gmail.com> wrote:

> Docs [1] says, that OOM can also be thrown when native library can't
> allocate memory chunk if physical memory (ram + swap).
>
> >> Shouldn't OOM happen if you're trying to allocate beyond the virtual
> memory capacity (beyond X GB)?
> With 64-bit addressing you have some exabytes of virtual memory, is it
> possible to have such amount of physical memory?
>
> I'd think JVM process should be killed by OS much earlier due to memory
> overcommit. See [2].
>
> [1]
> https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/memleaks002.html
> [2]
> http://engineering.pivotal.io/post/virtual_memory_settings_in_linux_-_the_problem_with_overcommit/
>
> On Wed, Oct 24, 2018 at 7:12 AM Denis Magda  wrote:
>
>> Alex,
>>
>> Correct me if I'm wrong, but even if an OS runs out of physical memory (X
>> GB in total) an Ignite node process still can request the X GB from
>> virtual
>> memory. Yes, virtual memory can involve swapping and disk to satisfy your
>> request but this shouldn't be a reason of the OOM. Shouldn't OOM happen if
>> you're trying to allocate beyond the virtual memory capacity (beyond X
>> GB)?
>>
>> Denis
>>
>> On Tue, Oct 23, 2018 at 12:08 PM Gerus  wrote:
>>
>> > Hi *Igniters*,
>> > Some time ago I've raised a suggestion for product improvement
>> > https://issues.apache.org/jira/browse/IGNITE-9112
>> >   .  It's all about
>> > off-heap memory allocation. Current implementation can have some
>> > improvements for failure critical systems. Ignite can have OOM in
>> runtime,
>> > because RAM can be used by OS, if it will not be pre-booked by operation
>> > system and this proposal is to address that. Common case is offheap and
>> > thats why memory segment cannot be managed by JVM that has
>> +AlwaysPreTouch
>> > option
>> > Obviously this implementation will make startup longer and thats why it
>> is
>> > proposed to use configuration flag to manage this feature
>> > I think, it will be useful to have this option. Are you supporting this?
>> >
>> >
>> >
>> > --
>> > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>> >
>>
>
>
> --
> Best regards,
> Andrey V. Mashenkov
>


-- 
Best regards,
Andrey V. Mashenkov

Re: Pre-touch for Ignite off-heap memory

[Andrey Mashenkov]

Docs [1] says, that OOM can also be thrown when native library can't
allocate memory chunk if physical memory (ram + swap).

>> Shouldn't OOM happen if you're trying to allocate beyond the virtual
memory capacity (beyond X GB)?
With 64-bit addressing you have some exabytes of virtual memory, is it
possible to have such amount of physical memory?

I'd think JVM process should be killed by OS much earlier due to memory
overcommit. See [2].

[1]
https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/memleaks002.html
[2]
http://engineering.pivotal.io/post/virtual_memory_settings_in_linux_-_the_problem_with_overcommit/

On Wed, Oct 24, 2018 at 7:12 AM Denis Magda  wrote:

> Alex,
>
> Correct me if I'm wrong, but even if an OS runs out of physical memory (X
> GB in total) an Ignite node process still can request the X GB from virtual
> memory. Yes, virtual memory can involve swapping and disk to satisfy your
> request but this shouldn't be a reason of the OOM. Shouldn't OOM happen if
> you're trying to allocate beyond the virtual memory capacity (beyond X GB)?
>
> Denis
>
> On Tue, Oct 23, 2018 at 12:08 PM Gerus  wrote:
>
> > Hi *Igniters*,
> > Some time ago I've raised a suggestion for product improvement
> > https://issues.apache.org/jira/browse/IGNITE-9112
> >   .  It's all about
> > off-heap memory allocation. Current implementation can have some
> > improvements for failure critical systems. Ignite can have OOM in
> runtime,
> > because RAM can be used by OS, if it will not be pre-booked by operation
> > system and this proposal is to address that. Common case is offheap and
> > thats why memory segment cannot be managed by JVM that has
> +AlwaysPreTouch
> > option
> > Obviously this implementation will make startup longer and thats why it
> is
> > proposed to use configuration flag to manage this feature
> > I think, it will be useful to have this option. Are you supporting this?
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >
>


-- 
Best regards,
Andrey V. Mashenkov

[GitHub] ignite pull request #4530: IGNITE-5795 Ensure binary metadata registration o...

[dmekhanikov]

Github user dmekhanikov closed the pull request at:

https://github.com/apache/ignite/pull/4530


---