Re: Atomic caches

[Dmitriy Setrakyan]

I would say absolutely YES - we need to have configuration validation.

Igniters, why was the validation skipped in atomic caches?

D.

On Fri, Apr 6, 2018 at 1:43 PM, akurbanov  wrote:

> Hello Igniters,
>
> I want to address a question on AtomicConfiguration validation. I've tested
> in ignite-1.8 branch that it is impossible to start two nodes with
> different
> AtomicConfiguration parameters e.g. different cache modes or numbers of
> backups are provided.
> JIRA link: https://issues.apache.org/jira/browse/IGNITE-2096
>
> In ignite-2.4 AtomicConfiguration validation is completely skipped on node
> startup and this issue is non-reproducible. Node with alternative
> configuration successfully joins, but this configuration is being
> completely
> ignored, all created atomics will reference the same initial configuration
> and belong to the same cache "ignite-sys-atomic-cache@default-ds-group",
> even if configuration is provided in constructor.
>
> Do we need any kind of validation for this configuration?
> Would it be correct to use the same approach for atomic types instances
> caching as used for IgniteQueue/IgniteSet, cache for each unique
> configuration?
>
> Best regards,
> Anton Kurbanov
>
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

[GitHub] ignite pull request #3774: Ignite 7791

[Mmuzaf]

GitHub user Mmuzaf opened a pull request:

https://github.com/apache/ignite/pull/3774

Ignite 7791



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Mmuzaf/ignite ignite-7791

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3774.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3774


commit 73c02562a485ad013d7a80cc358a56f47e7b1432
Author: Maxim Muzafarov 
Date:   2018-03-07T17:02:02Z

IGNITE-7791: add test case for delayed partition exchange after reconnect

commit 824ea5f717147e7e615bf961fb235e5b1207a00f
Author: Maxim Muzafarov 
Date:   2018-03-07T17:14:40Z

IGNITE-7791: awaitPartitionMapExchange before client reconnects

commit 640db004ca522e020512c460713ca66e305669bd
Author: Maxim Muzafarov 
Date:   2018-03-08T13:32:41Z

IGNITE-7791: remove awaitPartitionMapExchange before client reconnects

commit 8677131101c94e0d55d25dd65aae4bba4da9af24
Author: Maxim Muzafarov 
Date:   2018-03-08T15:41:14Z

IGNITE-7791: fix 100 prc issue reproducer

commit e239167d87253919441f5faacf6ee2bbc248b394
Author: Maxim Muzafarov 
Date:   2018-03-19T18:20:26Z

IGNITE-7791: remove client node check

commit 26645e5a241a339b56159b463234cc18f827e4f2
Author: Maxim Muzafarov 
Date:   2018-03-27T18:30:04Z

IGNITE-7791: more logging

commit 2fba4cd60b698db5f27ca9c889fa4ace4aba97fb
Author: Maxim Muzafarov 
Date:   2018-04-04T09:16:26Z

IGNITE-6842: update log details

commit 2e31145ce76392f3ba8374d02fca04d1f983cac0
Author: Maxim Muzafarov 
Date:   2018-04-05T09:52:51Z

IGNITE-7791: more logging

commit 5af630a4b2474e0459c3ee0b985ceb1cd6fa1389
Author: Maxim Muzafarov 
Date:   2018-04-06T22:34:42Z

IGNITE-7791: fix local caches context




---

[GitHub] ignite pull request #3773: IGNITE-8153 Nodes fail to connect each other when...

[mcherkasov]

GitHub user mcherkasov opened a pull request:

https://github.com/apache/ignite/pull/3773

IGNITE-8153 Nodes fail to connect each other when SSL is enabled



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8153

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3773.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3773


commit af30014cb50e5c6f6bf4919beaf04ae39e38cd36
Author: mcherkasov 
Date:   2018-04-04T01:48:27Z

IGNITE-8153 Nodes fail to connect each other when SSL is enabled




---

Re: IGNITE-8167

[Denis Magda]

Pavel, added you to JIRA contributors list.

--
Denis

On Fri, Apr 6, 2018 at 8:12 AM, Pavel Sapezhko 
wrote:

> My JIRA ID: pavel.sapezhko
> As I mentioned above, first we will have only archiver thread crashed and
> absolute wal started from 0, but we will have alive ignite instance. Logs
> can be found in attach.
>
>
> On Fri, Apr 6, 2018 at 5:42 PM Dmitry Pavlov 
> wrote:
>
>> Hi Pavel,
>>
>> Thank you. Could you please attach logs/stacktraces. Now it is not quite
>> clear where Ignite has failed?
>>
>> Could you please share your JIRA ID?
>>
>> Hi PMCs,
>>
>> could you please add Pavel to contributor list so
>> https://issues.apache.org/jira/browse/IGNITE-8167 issue can be assigned?
>>
>> Sincerely,
>> Dmitriy Pavlov
>>
>>
>> пт, 6 апр. 2018 г. в 17:38, Pavel Sapezhko :
>>
>> > Working on it.
>> > --
>> >
>> > С уважением,
>> > Cапежко Павел Александрович
>> > Инженер-программист ООО "Synesis"
>> > Skype: p.sapezhko
>> >
>>
> --
>
> С уважением,
> Cапежко Павел Александрович
> Инженер-программист ООО "Synesis"
> Skype: p.sapezhko
>
>

[jira] [Created] (IGNITE-8171) Document how to rollback transactions to let PME complete

[Denis Magda (JIRA)]

Denis Magda created IGNITE-8171:
---

 Summary: Document how to rollback transactions to let PME complete
 Key: IGNITE-8171
 URL: https://issues.apache.org/jira/browse/IGNITE-8171
 Project: Ignite
  Issue Type: New Feature
  Components: documentation
Reporter: Denis Magda
 Fix For: 2.5


Some Ignite operations provoke partition map exchange process within Ignite to 
ensure the partitions distribution state is synchronized cluster-wide. Topology 
update events and a start of a new distributed cache are examples of those 
operations.

When the partition map exchange starts, Ignite acquires a global lock at a 
particular stage. The lock can't be obtained until pending transactions are 
running in parallel. If there is a transaction that runs for a while, then it 
will prevent the partition map exchange process from the start freezing some 
operations such as a new node join process.

This property allows to rollback such long transactions to let Ignite acquire 
the lock faster and initiate the partition map exchange process. The timeout is 
enforced only at the time of the partition map exchange process.

See {{TransactionConfiguraion}} and 
{{IgniteTransactions.localActiveTransactions and withLabel}} methods.

Original ticket:
https://issues.apache.org/jira/browse/IGNITE-6827



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Thin clients release cycle

[Pavel Petroshenko]

>From our point of view option (3) makes the most sense (if it works for the
ASF, as Denis pointed out).
Option (2) implies too much overhead, which may not be worth it.
(1) is the least convenient approach for such independent projects as
Client libs, however it's clear where it comes from. So it is what it is...

Regards,
Pavel


On Fri, Apr 6, 2018 at 2:22 PM, Denis Magda  wrote:

> Regardless of the path taken, the sources have to be located in ASF
> repositories since we agreed to contribute the clients to the Foundation.
>
> Presently, I'm leaning towards the monolithic approach (1) because that's
> just simpler for an ASF project.
>
> The hybrid way (3.) can work out only if ASF INFRA is ok with that.
> Otherwise, we need to get a permit to host many ignite-...-client projects
> separately. It's unlikely our ASF mates approve that. The reason is weak.
>
> --
> Denis
>
> On Fri, Apr 6, 2018 at 11:32 AM, Pavel Tupitsyn 
> wrote:
>
> > Vladimir, can you clarify your ideas from Apache projects standpoint?
> >
> > Do you propose (1) to create new apache projects for every client (or for
> > all of them)
> > Or (2) move thin clients OUT of Apache ecosystem and simply host them on
> > Github?
> >
> > I think none of these will fly with ASF.
> > I am strongly for Monolith and following ASF guidelines and releases.
> >
> > We can increase release frequency if you want to be "agile".
> >
> > Thanks,
> > Pavel
> >
> > On Fri, Apr 6, 2018 at 4:49 PM, Petr Ivanov  wrote:
> >
> > > I would vote for single repository and the following release scheme: we
> > > build and release everything, but deliver only that modules, which have
> > > actual features / bugfixes, skipping other from release iteration until
> > new
> > > changes come into them.
> > >
> > > Also, I’d propose versioning scheme, that will reflect Apache Ignite’s
> > > compatibility, i.e. if thin client requires features from Apache Ignite
> > > 2.4, then it’s own version will be 2.4.A.B (where A - own major
> version,
> > > that will be reset every new Apache Release, B - minor/fix version for
> > > minor / quick fixes goals that will be reset every new A version).
> > >
> > >
> > >
> > > > On 6 Apr 2018, at 16:15, Vladimir Ozerov 
> wrote:
> > > >
> > > > Igniters,
> > > >
> > > > Over the last year we saw dramatic increase in demand for lightweight
> > > thin
> > > > clients. We already have four: JDBC, ODBC, .NET, Java. In future we
> are
> > > > going to have even more: NodeJS, PHP, Python, Go, whatever. I'd like
> to
> > > > start a discussion on how are we going to host them. There are
> several
> > > > approaches.
> > > >
> > > > 1) *Monolith* - everything is hosted in a single repository and
> > released
> > > as
> > > > a single artifact. This is our current approach.
> > > >
> > > > Pros:
> > > > - Easy to manage
> > > > Cons
> > > > - Long release cycle
> > > > - Client features must be developed in sync with each other which
> would
> > > be
> > > > very hard should we have > 5-6 different clients:
> > > >
> > > > 2) *Modules* - clients are moved to separate repositories with their
> > own
> > > > release cycles. JDBC is released separately, ODBC separately, .NET
> > > > separately, Java (guess what?) - separately, etc..They could have
> > > different
> > > > timelines, different feature sets, different release notes, different
> > > > versions. This is natural approach employed by multitude of vendors.
> > > When
> > > > new feature is added we do not need to wait for Apache Ignite
> release.
> > > > Instead, we release only small client on it's own.
> > > > Pros:
> > > > - Fast and (sorry) agile release cycle!
> > > > - No need to wait for months for new Ignite version
> > > > - No need to sync features between different clients
> > > > Cons:
> > > > - More votes, more artifacts, more release-related code and scripts
> > > >
> > > > 3) *Hybrid* - all clients are hosted in a single separate repository
> > and
> > > > released in sync with each other, but not with Apache Ignite.
> Balanced
> > > mix
> > > > of pros/cons from #1 and #2 approaches.
> > > > Pros:
> > > > - Relatively fast release cycle
> > > > Cons:
> > > > - Still need to sync features between clients
> > > >
> > > > I think that we should start moving our clients to #2 or #3
> approaches
> > to
> > > > get greater momentum from community, What do you think?
> > > >
> > > > Vladimir.
> > >
> > >
> >
>

[GitHub] ignite pull request #3772: Ignite 1.9.12

[slukyano]

GitHub user slukyano opened a pull request:

https://github.com/apache/ignite/pull/3772

Ignite 1.9.12



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-1.9.12

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3772.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3772


commit 6ab152a85f70c05847823f65f8e095ab9eb6b1f7
Author: sboikov 
Date:   2017-04-19T09:46:31Z

Attempt to fix awaitPartitionMapExchange: wait for last exchange completion 
to avoid races with cache destroy.

(cherry picked from commit d383484)

commit 1525c6cf2cb015289392eb54fec4029e9b53b438
Author: Andrey V. Mashenkov 
Date:   2017-06-30T15:36:50Z

Fixed service deployment tests.

commit a24ca24dd7aaa34707d74a4e660d769d3d5b0ed8
Author: sboikov 
Date:   2017-04-19T09:46:31Z

Attempt to fix awaitPartitionMapExchange: wait for last exchange completion 
to avoid races with cache destroy.

(cherry picked from commit d383484)

commit 05fac7e8a0bf9e08cd758bed7bd35ec85b914592
Author: Evgenii Zhuravlev 
Date:   2017-04-19T11:01:21Z

Backported IGNITE-4925 Fix 
IgniteCacheBinaryObjectsScanSelfTest.testScanNoClasses - Fixes #1750.

(cherry picked from commit b47f29d)

commit ef0a874ceb5c8bfa53e16337f6fd1699afaf2a39
Author: nikolay_tikhonov 
Date:   2017-06-30T17:39:01Z

Fixed CacheSerializableTransactionsTest#testTxConflictRemoveWithOldValue 
test.

Signed-off-by: nikolay_tikhonov 

commit 4dce965ea86374cba7265cb5d22e975aeac7d480
Author: nikolay_tikhonov 
Date:   2017-06-30T18:36:02Z

Fixed org.jsr107.tck.PutTest tests.

Signed-off-by: nikolay_tikhonov 

commit 32f1e394c222a6f4a2c111d6b6284c8626442b68
Author: Andrey V. Mashenkov 
Date:   2017-07-03T09:28:12Z

Merge branch 'ignite-1.8.8' into ignite-1.9.4

commit 50887fed508e03a8b7df32569afb6d84ab3eb670
Author: Igor Sapego 
Date:   2017-07-04T17:01:01Z

IGNITE-5663: ODBC: Closing cursor do not reset prepared statement anymore

commit da290cee855ef45a90ad539515e039f2826a6c00
Author: Igor Sapego 
Date:   2017-07-05T10:21:12Z

IGNITE-5663: Fix for test

commit 024a01d6bf91b4f301c4aee7f4a747e810a9a30b
Author: nikolay_tikhonov 
Date:   2017-07-05T15:58:00Z

Merged 1.7.12 into 1.8.9

Signed-off-by: nikolay_tikhonov 

commit 3536a58982e4c264bb72b2ccc1953049d2b5c67f
Author: Alexey Kukushkin 
Date:   2017-07-05T16:36:41Z

IGNITE-4901 Decrease logging level for DataStremer retry

commit 6d3a3ff2d99697882232070e715928336a9180cd
Author: Alexey Kukushkin 
Date:   2017-07-05T17:05:02Z

Fixed merge conflicts

commit aedc6aa8b17a39a6460c4b7f69255cd07d635bfb
Author: nikolay_tikhonov 
Date:   2017-07-05T17:42:15Z

Merge branch 'ignite-1.7.12' into ignite-1.9.4

Signed-off-by: nikolay_tikhonov 

commit acfc400b22738fa46397d392f88d49614e687969
Author: nikolay_tikhonov 
Date:   2017-07-05T17:42:48Z

Merge branch 'ignite-1.7.12' into ignite-1.9.4

Signed-off-by: nikolay_tikhonov 

commit 8dea19ba41bb9eda16f47933b2c46a081116d5f7
Author: Andrey V. Mashenkov 
Date:   2017-07-06T09:02:07Z

Minor fix.

commit f208f434f944196d531a1b51066dfe8d6394d739
Author: Andrey V. Mashenkov 
Date:   2017-07-06T12:17:50Z

Test fixed "IGNITE-5390: Bug in 
IgniteCacheTxStoreSessionWriteBehindCoalescingTest."

commit 355a5283559c885f57c4557bba2c6d9170a9b5fc
Author: mcherkasov 
Date:   2017-06-30T17:23:55Z

IGNITE-5554 ServiceProcessor may process failed reassignments in timeout 
thread

commit 92aa7c6e3c0d9b5cc68002433861b175d54f9421
Author: agura 
Date:   2017-07-04T13:56:40Z

ignite-5685 JDBC prepared statement shouldn't clear parameters after 
execution

commit 9165a0f93b5173b543cc6b4fad5fde37bd215f91
Author: Slava Koptilin 
Date:   2017-07-07T12:35:33Z

ignite-5562: assert statements were changed to the 'if' blocks

commit d9fc20a61d5ac0a6e63b26faa7fa0af753b2fa06
Author: Dmitriy Govorukhin 
Date:   2017-04-07T11:28:22Z

IGNITE-4889 - Changed Hibernate integration to use custom keys

(cherry picked from commit 6b62a20)

commit 16067300c9124b79bfee42139eb881ae585c0914
Author: Dmitriy Govorukhin 
Date:   2017-04-07T11:28:22Z

IGNITE-4889 - Changed Hibernate integration to use custom keys

(cherry picked from commit 6b62a20)

commit 

Re: Thin clients release cycle

[Denis Magda]

Regardless of the path taken, the sources have to be located in ASF
repositories since we agreed to contribute the clients to the Foundation.

Presently, I'm leaning towards the monolithic approach (1) because that's
just simpler for an ASF project.

The hybrid way (3.) can work out only if ASF INFRA is ok with that.
Otherwise, we need to get a permit to host many ignite-...-client projects
separately. It's unlikely our ASF mates approve that. The reason is weak.

--
Denis

On Fri, Apr 6, 2018 at 11:32 AM, Pavel Tupitsyn 
wrote:

> Vladimir, can you clarify your ideas from Apache projects standpoint?
>
> Do you propose (1) to create new apache projects for every client (or for
> all of them)
> Or (2) move thin clients OUT of Apache ecosystem and simply host them on
> Github?
>
> I think none of these will fly with ASF.
> I am strongly for Monolith and following ASF guidelines and releases.
>
> We can increase release frequency if you want to be "agile".
>
> Thanks,
> Pavel
>
> On Fri, Apr 6, 2018 at 4:49 PM, Petr Ivanov  wrote:
>
> > I would vote for single repository and the following release scheme: we
> > build and release everything, but deliver only that modules, which have
> > actual features / bugfixes, skipping other from release iteration until
> new
> > changes come into them.
> >
> > Also, I’d propose versioning scheme, that will reflect Apache Ignite’s
> > compatibility, i.e. if thin client requires features from Apache Ignite
> > 2.4, then it’s own version will be 2.4.A.B (where A - own major version,
> > that will be reset every new Apache Release, B - minor/fix version for
> > minor / quick fixes goals that will be reset every new A version).
> >
> >
> >
> > > On 6 Apr 2018, at 16:15, Vladimir Ozerov  wrote:
> > >
> > > Igniters,
> > >
> > > Over the last year we saw dramatic increase in demand for lightweight
> > thin
> > > clients. We already have four: JDBC, ODBC, .NET, Java. In future we are
> > > going to have even more: NodeJS, PHP, Python, Go, whatever. I'd like to
> > > start a discussion on how are we going to host them. There are several
> > > approaches.
> > >
> > > 1) *Monolith* - everything is hosted in a single repository and
> released
> > as
> > > a single artifact. This is our current approach.
> > >
> > > Pros:
> > > - Easy to manage
> > > Cons
> > > - Long release cycle
> > > - Client features must be developed in sync with each other which would
> > be
> > > very hard should we have > 5-6 different clients:
> > >
> > > 2) *Modules* - clients are moved to separate repositories with their
> own
> > > release cycles. JDBC is released separately, ODBC separately, .NET
> > > separately, Java (guess what?) - separately, etc..They could have
> > different
> > > timelines, different feature sets, different release notes, different
> > > versions. This is natural approach employed by multitude of vendors.
> > When
> > > new feature is added we do not need to wait for Apache Ignite release.
> > > Instead, we release only small client on it's own.
> > > Pros:
> > > - Fast and (sorry) agile release cycle!
> > > - No need to wait for months for new Ignite version
> > > - No need to sync features between different clients
> > > Cons:
> > > - More votes, more artifacts, more release-related code and scripts
> > >
> > > 3) *Hybrid* - all clients are hosted in a single separate repository
> and
> > > released in sync with each other, but not with Apache Ignite. Balanced
> > mix
> > > of pros/cons from #1 and #2 approaches.
> > > Pros:
> > > - Relatively fast release cycle
> > > Cons:
> > > - Still need to sync features between clients
> > >
> > > I think that we should start moving our clients to #2 or #3 approaches
> to
> > > get greater momentum from community, What do you think?
> > >
> > > Vladimir.
> >
> >
>

Re: Service grid redesign

[Valentin Kulichenko]

Yes, the class deployment itself has to be explicit. I.e., there has to be
a manual step where user updates the class, and the exact step required
would depend on DeploymentSpi implementation. But then Ignite takes care of
everything else - service redeployment and restart is automatic.

Dmitriy Pavlov, all this is going to be disabled if DeploymentSpi is not
configured. In this case service class definitions have to be deployed on
local classpath and can't be updated in runtime. Just like it works right
now.

-Val

On Fri, Apr 6, 2018 at 10:20 AM, Dmitriy Setrakyan 
wrote:

> On Fri, Apr 6, 2018 at 9:13 AM, Dmitry Pavlov 
> wrote:
>
> > Hi Igniters,
> >
> > I like automatic redeploy which can be disabled by config if user wants
> to
> > control this process. What do you think?
> >
>
> I do not think we should have anything automatic when it comes to
> deployment, everything should be explicit. However, if we use the
> deployment SPI, then a user should be able to do "hot" redeploy, where a
> new service will be deployed if the user drops an updated jar.
>
> We should not create anything new here. Ignite already has a deployment SPI
> and it already works in a certain way. Let's not change it.
>
> D.
>

Re: IEP-18: Transparent Data Encryption

[Denis Magda]

Nikolay, Dmitriy R.,

Thanks for the research and for writing down a summary in the IEP form.

Please answer several high-level questions:

   - Is it necessary to have CEP keys for every cache? Not sure how all the
   keys will be managed if the user wants to encrypt 10-100 caches. Is it
   possible to use a single CEP by default with an option of having a unique
   one for a cache with more sensitive information?
   - It's not written, but I guess it would be up to me which caches to
   encrypt, right? In practice, you don't need to have all the data encrypted.
   Usually, companies look to hide personal, payments history, etc.
   - Should we think of procedures of CEP keys regeneration? A key can be
   lost or stolen.
   - Similar question goes for MEP key.

--
Denis

On Thu, Apr 5, 2018 at 2:15 PM, Dmitriy Setrakyan 
wrote:

> Here is a correct link to IEP:
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> 18%3A+Transparent+Data+Encryption
>
> On Thu, Apr 5, 2018 at 12:01 PM, Nikolay Izhikov 
> wrote:
>
> > Hello, Igniters.
> >
> > Based on previous discussion [1] we've created "IEP-18: Transparent Data
> > Encryption" [2]
> > I've planned to start implementation of TDE in few weeks.
> > I will create JIRA ticket for each piece of implementation.
> >
> > So, please, see IEP-18 and give us feedback.
> >
> > Dima Ryabov, huge thanks for pushing TDE IEP forward.
> >
> > [1] http://apache-ignite-developers.2346864.n4.nabble.
> > com/Transparent-Data-Encryption-TDE-in-Apache-Ignite-td18957.html
> > [2] https://cwiki.apache.org/confluence/pages/viewpage.
> > action?pageId=75979078
> >
>

Re: Data Loss while upgrading custom jar from old jar in server and client nodes

[Denis Magda]

Thanks, Ivan,

It will be good if we fix usability issues like that as soon as possible
because usually, they don't involve much effort. Igniters, is anyone
interested to implement the improvement for 2.5 release?


--
Denis

On Fri, Apr 6, 2018 at 4:12 AM, Ivan Rakov  wrote:

> By the way, ticket is marked with "newbie" label.
> If there's anyone who wants to start contributing in Ignite, feel free to
> take this issue.
>
> Best Regards,
> Ivan Rakov
>
>
> On 06.04.2018 14:06, Ivan Rakov wrote:
>
>> Denis,
>>
>> I'm sure we can: https://issues.apache.org/jira/browse/IGNITE-8162
>>
>> Best Regards,
>> Ivan Rakov
>>
>> On 05.04.2018 22:39, Denis Magda wrote:
>>
>>> Ivan,
>>>
>>> How can we facilitate the user here? Can we generate a meaningful
>>> exception
>>> that explains how to tackle the issue?
>>>
>>> --
>>> Denis
>>>
>>> On Thu, Apr 5, 2018 at 2:49 AM, Ivan Rakov 
>>> wrote:
>>>
>>> Hi,

 Cache configuration is persisted in db\{consistent-ID}\cache-{cach
 e-name}\cache_data.dat
 file. It's just CacheConfiguration serialized by JDK marshaller.
 Try to delete this file and start cache with new configuration (with
 correct factory class names). All your data will persist as long as old
 ignite data is compatible with new configuration.

 Best Regards,
 Ivan Rakov


 On 04.04.2018 23:08, Denis Magda wrote:

 Ivan R., Alex G., persistence experts,
>
> Please have a look at this question.
>
> --
> Denis
>
> On Mon, Apr 2, 2018 at 6:15 AM, Andrey Mashenkov <
> andrey.mashen...@gmail.com
>
> wrote:
>> Crossposting to dev.
>>
>> Guys,
>> User use Ignite native persistence with CacheStore configured.
>> Seems, changing CacheStore requires cache recreation in his case to
>> changes
>> can be applied.
>>
>> Does anybody has idea how it can be fixed?
>>
>> Do we allow to rewrite cache config in native store if it has minor
>> changes
>> that will not cause issues? E.g. changing cacheStore factory or
>> changing
>> number of backups?
>> Otherwise, I'd suggest to deprecate such configuration.
>>
>>
>> On Tue, Mar 27, 2018 at 4:18 PM, siva 
>> wrote:
>>
>> Hi,
>>
>>> Thanks for checking out issue. I think its not about IDE issue.
>>> Again same thing i have reproduce like bellow steps
>>>
>>> 1.run the program from IDE as client
>>> 2.copy jar to ignite libs folder and start the server
>>> 3.Then check previous message steps so that can able to reproduce the
>>> issue.
>>> https://www.youtube.com/watch?v=COQiu2gi8ag=43s
>>> 
>>> I have attached the video link.Can you go through that video and
>>> check
>>>
>>> the
>>
>> issue.
>>>
>>>
>>> Thanks
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>>
>>> --
>> Best regards,
>> Andrey V. Mashenkov
>>
>>
>>
>> --
>> Best regards,
>> Andrey V. Mashenkov
>>
>>
>>
>>
>

Re: Memory usage per cache

[Denis Magda]

Alex,

Why not return cache group metrics from this method by default and properly
> document it?


What do you think about Dmitry's suggestion? It sounds reasonable to me.

--
Denis

On Wed, Apr 4, 2018 at 12:22 PM, Dmitriy Setrakyan 
wrote:

> On Wed, Apr 4, 2018 at 5:27 AM, Alexey Goncharuk <
> alexey.goncha...@gmail.com
> > wrote:
>
> > Denis,
> >
> > I think this particular metric should be deprecated. The most we can do
> > about it is to return the actual allocated size when a cache is the only
> > cache in a group and return -1 if there are multiple caches in a group.
> > However, this does not look like a consistent approach to me, so I would
> > prefer to always return -1 and suggest that users use corresponding cache
> > group metrics.
> >
>
> Why not return cache group metrics from this method by default and properly
> document it?
>

Re: Thin clients release cycle

[Pavel Tupitsyn]

Vladimir, can you clarify your ideas from Apache projects standpoint?

Do you propose (1) to create new apache projects for every client (or for
all of them)
Or (2) move thin clients OUT of Apache ecosystem and simply host them on
Github?

I think none of these will fly with ASF.
I am strongly for Monolith and following ASF guidelines and releases.

We can increase release frequency if you want to be "agile".

Thanks,
Pavel

On Fri, Apr 6, 2018 at 4:49 PM, Petr Ivanov  wrote:

> I would vote for single repository and the following release scheme: we
> build and release everything, but deliver only that modules, which have
> actual features / bugfixes, skipping other from release iteration until new
> changes come into them.
>
> Also, I’d propose versioning scheme, that will reflect Apache Ignite’s
> compatibility, i.e. if thin client requires features from Apache Ignite
> 2.4, then it’s own version will be 2.4.A.B (where A - own major version,
> that will be reset every new Apache Release, B - minor/fix version for
> minor / quick fixes goals that will be reset every new A version).
>
>
>
> > On 6 Apr 2018, at 16:15, Vladimir Ozerov  wrote:
> >
> > Igniters,
> >
> > Over the last year we saw dramatic increase in demand for lightweight
> thin
> > clients. We already have four: JDBC, ODBC, .NET, Java. In future we are
> > going to have even more: NodeJS, PHP, Python, Go, whatever. I'd like to
> > start a discussion on how are we going to host them. There are several
> > approaches.
> >
> > 1) *Monolith* - everything is hosted in a single repository and released
> as
> > a single artifact. This is our current approach.
> >
> > Pros:
> > - Easy to manage
> > Cons
> > - Long release cycle
> > - Client features must be developed in sync with each other which would
> be
> > very hard should we have > 5-6 different clients:
> >
> > 2) *Modules* - clients are moved to separate repositories with their own
> > release cycles. JDBC is released separately, ODBC separately, .NET
> > separately, Java (guess what?) - separately, etc..They could have
> different
> > timelines, different feature sets, different release notes, different
> > versions. This is natural approach employed by multitude of vendors.
> When
> > new feature is added we do not need to wait for Apache Ignite release.
> > Instead, we release only small client on it's own.
> > Pros:
> > - Fast and (sorry) agile release cycle!
> > - No need to wait for months for new Ignite version
> > - No need to sync features between different clients
> > Cons:
> > - More votes, more artifacts, more release-related code and scripts
> >
> > 3) *Hybrid* - all clients are hosted in a single separate repository and
> > released in sync with each other, but not with Apache Ignite. Balanced
> mix
> > of pros/cons from #1 and #2 approaches.
> > Pros:
> > - Relatively fast release cycle
> > Cons:
> > - Still need to sync features between clients
> >
> > I think that we should start moving our clients to #2 or #3 approaches to
> > get greater momentum from community, What do you think?
> >
> > Vladimir.
>
>

Re: Service grid redesign

[Dmitriy Setrakyan]

On Fri, Apr 6, 2018 at 9:13 AM, Dmitry Pavlov  wrote:

> Hi Igniters,
>
> I like automatic redeploy which can be disabled by config if user wants to
> control this process. What do you think?
>

I do not think we should have anything automatic when it comes to
deployment, everything should be explicit. However, if we use the
deployment SPI, then a user should be able to do "hot" redeploy, where a
new service will be deployed if the user drops an updated jar.

We should not create anything new here. Ignite already has a deployment SPI
and it already works in a certain way. Let's not change it.

D.

[GitHub] ignite pull request #3700: IGNITE-6839 Ignite Compatibility: flaky test test...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3700


---

[GitHub] ignite pull request #3734: IGNITE-8114 Add fail recovery mechanism to tracki...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3734


---

Re: Service grid redesign

[Dmitry Pavlov]

Hi Igniters,

I like automatic redeploy which can be disabled by config if user wants to
control this process. What do you think?

Sincerely,
Dmitriy Pavlov

пт, 6 апр. 2018 г. в 18:29, Denis Mekhanikov :

> Val,
>
> I don't really like the idea of automatic redeployment of services when
> classes change.
> Different nodes may detect these changes at different moments in time, so
> there won't be any guarantee, that all nodes have the same version.
> And if redeployment fails, then there won't be a way to notify user code
> about it.
> Also service fields may change between versions, so already deployed
> services won't be able to be deserialized, using new classes.
>
> I think, it would be better if user could trigger redeployment manually. It
> would solve the mentioned problems and let the user redeploy services, even
> when only their field parameters change without implementation changes.
>
> What do you think?
>
> Denis
>
> чт, 5 апр. 2018 г. в 22:37, Denis Magda :
>
> > Val,
> >
> > Sounds like a great solution. I'm totally for it.
> >
> > --
> > Denis
> >
> > On Thu, Apr 5, 2018 at 12:32 PM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> > > Denis,
> > >
> > > This is why I'm suggesting to use DeploymentSpi for this. The way I see
> > > this is that instead of deploying classes on local classpath, user can
> > > deploy them in the storage that SPI points to. If class is updated in
> the
> > > storage, Ignite detects this and automatically restarts the service.
> This
> > > is a very simple and straightforward approach that doesn't required a
> lot
> > > of changes on our side and allows to reuse existing implementation of
> > > DeploymentSpi.
> > >
> > > -Val
> > >
> > > On Thu, Apr 5, 2018 at 12:13 PM, Denis Magda 
> > wrote:
> > >
> > > > >
> > > > > There is no need to deserialize services on the coordinator. It
> > should
> > > > only
> > > > > be able to calculate the assignments.
> > > > > *LazyServiceConfiguration *should be used to deliver the service
> > > > > configurations, just like it is done right now.
> > > >
> > > >
> > > > Can that configuration be tweaked over the time requiring to update
> the
> > > > class on all the nodes (if, for instance, someone wants to deploy the
> > > next
> > > > version of a service)? Just want to be sure we don't need to restart
> > the
> > > > cluster nodes (that won't be used for service deployments) on
> > > > services-related configurational changes.
> > > >
> > > > --
> > > > Denis
> > > >
> > > > On Thu, Apr 5, 2018 at 8:18 AM, Denis Mekhanikov <
> > dmekhani...@gmail.com>
> > > > wrote:
> > > >
> > > > > Denis,
> > > > > There is no need to deserialize services on the coordinator. It
> > should
> > > > only
> > > > > be able to calculate the assignments.
> > > > > *LazyServiceConfiguration *should be used to deliver the service
> > > > > configurations, just like it is done right now.
> > > > >
> > > > > Val,
> > > > > Usage of DeploymentSpi is a good idea, I didn't think about this
> > > > > possibility.
> > > > > This is a viable alternative to peer-class-loading, not that
> > > > user-friendly
> > > > > though.
> > > > > But if peer-class-loading is that hard to implement, then I vote
> for
> > > > > DeploymentSpi.
> > > > > As far as I understand, it won't require us to do any additional
> > > changes
> > > > in
> > > > > Ignite, but will make users think about using a proper
> DeploymentSpi.
> > > > > Please correct me, if I'm wrong.
> > > > > It would be good, though, to add some examples on service
> > redeployment,
> > > > > when implementation class changes.
> > > > >
> > > > > Denis
> > > > >
> > > > > чт, 5 апр. 2018 г. в 2:33, Valentin Kulichenko <
> > > > > valentin.kuliche...@gmail.com>:
> > > > >
> > > > > > I don't think peer class loading is even possible for services. I
> > > > believe
> > > > > > we should reuse DeploymentSpi [1] for versioning.
> > > > > >
> > > > > > [1] https://apacheignite.readme.io/docs/deployment-spi
> > > > > >
> > > > > > -Val
> > > > > >
> > > > > > On Wed, Apr 4, 2018 at 12:52 PM, Denis Magda <
> dma...@gridgain.com>
> > > > > wrote:
> > > > > >
> > > > > > > Sorry, that was me who renamed the IEP to "Oil Change in
> Service
> > > > Grid".
> > > > > > Was
> > > > > > > writing this email after the renaming. Like that title more
> > because
> > > > > it's
> > > > > > > fun and highlights what we're intended to do - cleaning of our
> > > > service
> > > > > > grid
> > > > > > > engine and powering it up with new "liquid" (new communication
> > and
> > > > > > > deployment approach not available before).
> > > > > > >
> > > > > > > Denis
> > > > > > >
> > > > > > >
> > > > > > > > This message contains serialized service instance and its
> > > > > > configuration.
> > > > > > > > It is delivered to the coordinator node first, that
> calculates
> > > the
> > > > > > > service
> > > > > > > > deployment assignments and 

[GitHub] ignite pull request #3740: ignite-8049 Limit the number of operation cycles ...

[SpiderRus]

Github user SpiderRus closed the pull request at:

https://github.com/apache/ignite/pull/3740


---

[GitHub] ignite pull request #3769: Ignite 8049

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3769


---

Re: Service grid redesign

[Denis Mekhanikov]

Val,

I don't really like the idea of automatic redeployment of services when
classes change.
Different nodes may detect these changes at different moments in time, so
there won't be any guarantee, that all nodes have the same version.
And if redeployment fails, then there won't be a way to notify user code
about it.
Also service fields may change between versions, so already deployed
services won't be able to be deserialized, using new classes.

I think, it would be better if user could trigger redeployment manually. It
would solve the mentioned problems and let the user redeploy services, even
when only their field parameters change without implementation changes.

What do you think?

Denis

чт, 5 апр. 2018 г. в 22:37, Denis Magda :

> Val,
>
> Sounds like a great solution. I'm totally for it.
>
> --
> Denis
>
> On Thu, Apr 5, 2018 at 12:32 PM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > Denis,
> >
> > This is why I'm suggesting to use DeploymentSpi for this. The way I see
> > this is that instead of deploying classes on local classpath, user can
> > deploy them in the storage that SPI points to. If class is updated in the
> > storage, Ignite detects this and automatically restarts the service. This
> > is a very simple and straightforward approach that doesn't required a lot
> > of changes on our side and allows to reuse existing implementation of
> > DeploymentSpi.
> >
> > -Val
> >
> > On Thu, Apr 5, 2018 at 12:13 PM, Denis Magda 
> wrote:
> >
> > > >
> > > > There is no need to deserialize services on the coordinator. It
> should
> > > only
> > > > be able to calculate the assignments.
> > > > *LazyServiceConfiguration *should be used to deliver the service
> > > > configurations, just like it is done right now.
> > >
> > >
> > > Can that configuration be tweaked over the time requiring to update the
> > > class on all the nodes (if, for instance, someone wants to deploy the
> > next
> > > version of a service)? Just want to be sure we don't need to restart
> the
> > > cluster nodes (that won't be used for service deployments) on
> > > services-related configurational changes.
> > >
> > > --
> > > Denis
> > >
> > > On Thu, Apr 5, 2018 at 8:18 AM, Denis Mekhanikov <
> dmekhani...@gmail.com>
> > > wrote:
> > >
> > > > Denis,
> > > > There is no need to deserialize services on the coordinator. It
> should
> > > only
> > > > be able to calculate the assignments.
> > > > *LazyServiceConfiguration *should be used to deliver the service
> > > > configurations, just like it is done right now.
> > > >
> > > > Val,
> > > > Usage of DeploymentSpi is a good idea, I didn't think about this
> > > > possibility.
> > > > This is a viable alternative to peer-class-loading, not that
> > > user-friendly
> > > > though.
> > > > But if peer-class-loading is that hard to implement, then I vote for
> > > > DeploymentSpi.
> > > > As far as I understand, it won't require us to do any additional
> > changes
> > > in
> > > > Ignite, but will make users think about using a proper DeploymentSpi.
> > > > Please correct me, if I'm wrong.
> > > > It would be good, though, to add some examples on service
> redeployment,
> > > > when implementation class changes.
> > > >
> > > > Denis
> > > >
> > > > чт, 5 апр. 2018 г. в 2:33, Valentin Kulichenko <
> > > > valentin.kuliche...@gmail.com>:
> > > >
> > > > > I don't think peer class loading is even possible for services. I
> > > believe
> > > > > we should reuse DeploymentSpi [1] for versioning.
> > > > >
> > > > > [1] https://apacheignite.readme.io/docs/deployment-spi
> > > > >
> > > > > -Val
> > > > >
> > > > > On Wed, Apr 4, 2018 at 12:52 PM, Denis Magda 
> > > > wrote:
> > > > >
> > > > > > Sorry, that was me who renamed the IEP to "Oil Change in Service
> > > Grid".
> > > > > Was
> > > > > > writing this email after the renaming. Like that title more
> because
> > > > it's
> > > > > > fun and highlights what we're intended to do - cleaning of our
> > > service
> > > > > grid
> > > > > > engine and powering it up with new "liquid" (new communication
> and
> > > > > > deployment approach not available before).
> > > > > >
> > > > > > Denis
> > > > > >
> > > > > >
> > > > > > > This message contains serialized service instance and its
> > > > > configuration.
> > > > > > > It is delivered to the coordinator node first, that calculates
> > the
> > > > > > service
> > > > > > > deployment assignments and adds this information to the
> message.
> > > > > >
> > > > > >
> > > > > > I would consider using a NodeFilter first to decide where a
> service
> > > can
> > > > > be
> > > > > > potentially deployed.  Otherwise, we would require service
> classes
> > to
> > > > be
> > > > > on
> > > > > > every node (every node might become a coordinator) which is not
> the
> > > > > desired
> > > > > > requirement.
> > > > > >
> > > > > >
> > > > > > As for the peer-class-loading, I would backup up Dmitriy here.
> > 

[jira] [Created] (IGNITE-8170) [ML] Adopt KMeans example to the Partitioned Dataset

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-8170:


 Summary: [ML] Adopt KMeans example to the Partitioned Dataset
 Key: IGNITE-8170
 URL: https://issues.apache.org/jira/browse/IGNITE-8170
 Project: Ignite
  Issue Type: Sub-task
  Components: ml
Reporter: Aleksey Zinoviev
Assignee: Aleksey Zinoviev






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8169) [ML] Implement Model-Trainer pair for KMeans based on Partitioned Dataset

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-8169:


 Summary: [ML] Implement Model-Trainer pair for KMeans based on 
Partitioned Dataset
 Key: IGNITE-8169
 URL: https://issues.apache.org/jira/browse/IGNITE-8169
 Project: Ignite
  Issue Type: Sub-task
  Components: ml
Reporter: Aleksey Zinoviev






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8168) [ML] Add KMeans version for Partitioned Datasets

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-8168:


 Summary: [ML] Add KMeans version for Partitioned Datasets
 Key: IGNITE-8168
 URL: https://issues.apache.org/jira/browse/IGNITE-8168
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Aleksey Zinoviev
Assignee: Aleksey Zinoviev






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: IGNITE-8167

[Pavel Sapezhko]

My JIRA ID: pavel.sapezhko
As I mentioned above, first we will have only archiver thread crashed and
absolute wal started from 0, but we will have alive ignite instance. Logs
can be found in attach.

On Fri, Apr 6, 2018 at 5:42 PM Dmitry Pavlov  wrote:

> Hi Pavel,
>
> Thank you. Could you please attach logs/stacktraces. Now it is not quite
> clear where Ignite has failed?
>
> Could you please share your JIRA ID?
>
> Hi PMCs,
>
> could you please add Pavel to contributor list so
> https://issues.apache.org/jira/browse/IGNITE-8167 issue can be assigned?
>
> Sincerely,
> Dmitriy Pavlov
>
>
> пт, 6 апр. 2018 г. в 17:38, Pavel Sapezhko :
>
> > Working on it.
> > --
> >
> > С уважением,
> > Cапежко Павел Александрович
> > Инженер-программист ООО "Synesis"
> > Skype: p.sapezhko
> >
>
-- 

С уважением,
Cапежко Павел Александрович
Инженер-программист ООО "Synesis"
Skype: p.sapezhko
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.ignite.internal.util.GridUnsafe$2 (file:/home/pavel.sapezhko/src/ignite/modules/core/target/classes/) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of org.apache.ignite.internal.util.GridUnsafe$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
[06-04-2018 18:08:17][INFO ][main][IgniteKernal] 

>>>__    
>>>   /  _/ ___/ |/ /  _/_  __/ __/  
>>>  _/ // (7 7// /  / / / _/
>>> /___/\___/_/|_/___/ /_/ /___/   
>>> 
>>> ver. 2.4.0#20180305-sha1:aa342270
>>> 2018 Copyright(C) Apache Software Foundation
>>> 
>>> Ignite documentation: http://ignite.apache.org

[06-04-2018 18:08:17][INFO ][main][IgniteKernal] Config URL: file:/home/pavel.sapezhko/src/ignite/examples/config/persistentstore/example-persistent-store.xml
[06-04-2018 18:08:17][INFO ][main][IgniteKernal] IgniteConfiguration [igniteInstanceName=null, pubPoolSize=8, svcPoolSize=8, callbackPoolSize=8, stripedPoolSize=8, sysPoolSize=8, mgmtPoolSize=4, igfsPoolSize=8, dataStreamerPoolSize=8, utilityCachePoolSize=8, utilityCacheKeepAliveTime=6, p2pPoolSize=2, qryPoolSize=8, igniteHome=/home/pavel.sapezhko/src/ignite, igniteWorkDir=/home/pavel.sapezhko/src/ignite/work, mbeanSrv=com.sun.jmx.mbeanserver.JmxMBeanServer@66d1af89, nodeId=e8d21e4a-5725-446e-b310-5f682079639c, marsh=org.apache.ignite.internal.binary.BinaryMarshaller@3e34ace1, marshLocJobs=false, daemon=false, p2pEnabled=false, netTimeout=5000, sndRetryDelay=1000, sndRetryCnt=3, metricsHistSize=1, metricsUpdateFreq=2000, metricsExpTime=9223372036854775807, discoSpi=TcpDiscoverySpi [addrRslvr=null, sockTimeout=0, ackTimeout=0, marsh=null, reconCnt=10, reconDelay=2000, maxAckTimeout=60, forceSrvMode=false, clientReconnectDisabled=false], segPlc=STOP, segResolveAttempts=2, waitForSegOnStart=true, allResolversPassReq=true, segChkFreq=1, commSpi=TcpCommunicationSpi [connectGate=null, connPlc=null, enableForcibleNodeKill=false, enableTroubleshootingLog=false, srvLsnr=org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$2@6cbcf243, locAddr=null, locHost=null, locPort=47100, locPortRange=100, shmemPort=-1, directBuf=true, directSndBuf=false, idleConnTimeout=60, connTimeout=5000, maxConnTimeout=60, reconCnt=10, sockSndBuf=32768, sockRcvBuf=32768, msgQueueLimit=0, slowClientQueueLimit=0, nioSrvr=null, shmemSrv=null, usePairedConnections=false, connectionsPerNode=1, tcpNoDelay=true, filterReachableAddresses=false, ackSndThreshold=32, unackedMsgsBufSize=0, sockWriteTimeout=2000, lsnr=null, boundTcpPort=-1, boundTcpShmemPort=-1, selectorsCnt=4, selectorSpins=0, addrRslvr=null, ctxInitLatch=java.util.concurrent.CountDownLatch@29e6eb25[Count = 1], stopping=false, metricsLsnr=org.apache.ignite.spi.communication.tcp.TcpCommunicationMetricsListener@339bf286], evtSpi=org.apache.ignite.spi.eventstorage.NoopEventStorageSpi@38be305c, colSpi=NoopCollisionSpi [], deploySpi=LocalDeploymentSpi [lsnr=null], indexingSpi=org.apache.ignite.spi.indexing.noop.NoopIndexingSpi@269f4bad, addrRslvr=null, clientMode=false, rebalanceThreadPoolSize=1, txCfg=org.apache.ignite.configuration.TransactionConfiguration@5ed731d0, cacheSanityCheckEnabled=true, discoStartupDelay=6, deployMode=SHARED, p2pMissedCacheSize=100, locHost=null, timeSrvPortBase=31100, timeSrvPortRange=100, failureDetectionTimeout=1, clientFailureDetectionTimeout=3, metricsLogFreq=6, hadoopCfg=null, connectorCfg=org.apache.ignite.configuration.ConnectorConfiguration@3234f74e, odbcCfg=null, warmupClos=null, atomicCfg=AtomicConfiguration [seqReserveSize=1000, cacheMode=PARTITIONED, backups=1, aff=null, grpName=null], classLdr=null, sslCtxFactory=null, platformCfg=null, binaryCfg=null, memCfg=null, pstCfg=null, dsCfg=DataStorageConfiguration [sysRegionInitSize=41943040, sysCacheMaxSize=104857600, 

[GitHub] ignite pull request #3771: IGNITE-8167: Fix inconsistent last record pointer...

[amelius0712]

GitHub user amelius0712 opened a pull request:

https://github.com/apache/ignite/pull/3771

IGNITE-8167: Fix inconsistent last record pointer in case of recovery from 
corrupted WAL

Let's look at this peace of code from 
GridCacheDatabaseSharedManager.readCheckpointAndRestoreMemory

`
WALPointer restore = restoreMemory(status);

// First, bring memory to the last consistent checkpoint state 
if needed.
// This method should return a pointer to the last valid record 
in the WAL.

cctx.wal().resumeLogging(restore);
`
In case of `restore == null`. Logging will be resuming from 0 absolute WAL 
index.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Synesis-LLC/ignite ignite-8167

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3771.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3771


commit 40b9c8e227783f8d90fff5f2db4688e63be3dd37
Author: Pavel Sapezhko 
Date:   2018-04-06T14:36:23Z

IGNITE-8167: Fix inconsistent last record pointer in case of recovery from 
corrupted WAL




---

Re: IGNITE-8167

[Dmitry Pavlov]

Hi Pavel,

Thank you. Could you please attach logs/stacktraces. Now it is not quite
clear where Ignite has failed?

Could you please share your JIRA ID?

Hi PMCs,

could you please add Pavel to contributor list so
https://issues.apache.org/jira/browse/IGNITE-8167 issue can be assigned?

Sincerely,
Dmitriy Pavlov


пт, 6 апр. 2018 г. в 17:38, Pavel Sapezhko :

> Working on it.
> --
>
> С уважением,
> Cапежко Павел Александрович
> Инженер-программист ООО "Synesis"
> Skype: p.sapezhko
>

IGNITE-8167

[Pavel Sapezhko]

Working on it.
-- 

С уважением,
Cапежко Павел Александрович
Инженер-программист ООО "Synesis"
Skype: p.sapezhko

Re: IGNITE-6879

[Dmitry Pavlov]

Excellend picture. I remember about this change.

If Denis M. would be able to look througt the changes faster than me, I can
merge without detailed review.

пт, 6 апр. 2018 г. в 16:15, Роман Меерсон :

> OK
>
> [image: 1486924635147168240.jpg]
>
>
> пт, 6 апр. 2018 г. в 17:08, Igor Sapego :
>
>> Hi,
>> Well, Dmitry has said he's going to merge it in 3-4 days 2 days ago,
>> so I guess, the merge is going to happen in 1-2 days or so.
>>
>>
>> Best Regards,
>> Igor
>>
>> On Fri, Apr 6, 2018 at 3:48 PM, Роман Меерсон 
>> wrote:
>>
>> > Hi all!
>> >
>> > As i see everything is awesome and there is no objections, so when my PR
>> > would be merged?
>> >
>> > чт, 5 апр. 2018 г. в 18:58, Вячеслав Коптилин > >:
>> >
>> > > Thank you, Roman!
>> > >
>> > > 2018-04-05 17:49 GMT+03:00 Роман Меерсон :
>> > >
>> > > > Hi Slava,
>> > > >
>> > > > Fixed
>> > > >
>> > > > чт, 5 апр. 2018 г. в 18:41, Вячеслав Коптилин <
>> > slava.kopti...@gmail.com
>> > > >:
>> > > >
>> > > > > Hi Roman,
>> > > > >
>> > > > > please take into account my comment IgniteQueryGenerator.java
>> > > > > <
>> > > > > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-541?
>> > > > commentId=de43c65f-9ac7-4080-9904-aec119138c94=/
>> > > > modules/spring-data-2.0/src/main/java/org/apache/ignite/
>> > > > springdata/repository/query/IgniteQueryGenerator.java
>> > > > > >
>> > > > >
>> > > > > Best regards,
>> > > > > Slava.
>> > > > >
>> > > > > 2018-04-05 14:59 GMT+03:00 Роман Меерсон :
>> > > > >
>> > > > > > Ok, so waiting for accept and commit
>> > > > > >
>> > > > > > чт, 5 апр. 2018 г. в 15:29, Alexey Kukushkin <
>> > > > kukushkinale...@gmail.com
>> > > > > >:
>> > > > > >
>> > > > > > > Roman,
>> > > > > > >
>> > > > > > > Just pay commiter's (Dmitry Pavlov will most likely commit
>> your
>> > > code)
>> > > > > > > attention to include the new test suite to TeamCity
>> > configuration.
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>

[jira] [Created] (IGNITE-8167) Recovery after crash sometimes leads to starting from beginning absolute wal segment index

[Pavel Sapezhko (JIRA)]

Pavel Sapezhko created IGNITE-8167:
--

 Summary: Recovery after crash sometimes leads to starting from 
beginning absolute wal segment index
 Key: IGNITE-8167
 URL: https://issues.apache.org/jira/browse/IGNITE-8167
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
 Environment: Doesn't meter. We saw these behavior in k8s deployment as 
in local deployment too. Using any of WAL_MOD.
Reporter: Pavel Sapezhko
 Fix For: 2.5


When we are trying to restore after crash using wal log, sometimes we can find 
corrupted wal messages which can leads to starting from beginning absolute wal 
index. So, we will have broken wal archiver thread due to assertation error(but 
we still having working Ignite instance. I think we need to discuss if we are 
really want it) and as a result on next restart we can see "Wal history is too 
short" message.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8166) stopGrid() hangs in some cases when node is invalidated and PDS is enabled

[Aleksey Plekhanov (JIRA)]

Aleksey Plekhanov created IGNITE-8166:
-

 Summary: stopGrid() hangs in some cases when node is invalidated 
and PDS is enabled
 Key: IGNITE-8166
 URL: https://issues.apache.org/jira/browse/IGNITE-8166
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.5
Reporter: Aleksey Plekhanov


Node invalidation via FailureProcessor can hang {{exchange-worker}} and 
{{stopGrid()}} when PDS is enabled.

Reproducer (reproducer is racy, sometimes finished without hang):
{code:java}
public class StopNodeHangsTest extends GridCommonAbstractTest {
/** Offheap size for memory policy. */
private static final int SIZE = 10 * 1024 * 1024;

/** Page size. */
static final int PAGE_SIZE = 2048;

/** Number of entries. */
static final int ENTRIES = 2_000;

/** {@inheritDoc} */
@Override protected IgniteConfiguration getConfiguration(String 
igniteInstanceName) throws Exception {
IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);

DataStorageConfiguration dsCfg = new DataStorageConfiguration();

DataRegionConfiguration dfltPlcCfg = new DataRegionConfiguration();

dfltPlcCfg.setName("dfltPlc");
dfltPlcCfg.setInitialSize(SIZE);
dfltPlcCfg.setMaxSize(SIZE);
dfltPlcCfg.setPersistenceEnabled(true);

dsCfg.setDefaultDataRegionConfiguration(dfltPlcCfg);
dsCfg.setPageSize(PAGE_SIZE);

cfg.setDataStorageConfiguration(dsCfg);

cfg.setFailureHandler(new FailureHandler() {
@Override public boolean onFailure(Ignite ignite, FailureContext 
failureCtx) {
return true;
}
});

return cfg;
}

public void testStopNodeHangs() throws Exception {
cleanPersistenceDir();

IgniteEx ignite0 = startGrid(0);
IgniteEx ignite1 = startGrid(1);

ignite1.cluster().active(true);

awaitPartitionMapExchange();

IgniteCache cache = ignite1.getOrCreateCache("TEST");

Map entries = new HashMap<>();

for (int i = 0; i < ENTRIES; i++)
entries.put(i, new byte[PAGE_SIZE * 2 / 3]);

cache.putAll(entries);

ignite1.context().failure().process(new 
FailureContext(FailureType.CRITICAL_ERROR, null));

stopGrid(0);
stopGrid(1);
}
}
{code}

{{stopGrid(1)}} waiting until exchange finished, {{exchange-worker}} waits on 
method {{GridCacheDatabaseSharedManager#checkpointReadLock}} for 
{{CheckpointProgressSnapshot#cpBeginFut}}, but this future is never done 
because {{db-checkpoint-thread}} got exception at 
{{GridCacheDatabaseSharedManager.Checkpointer#markCheckpointBegin}} thrown by 
{{FileWriteAheadLogManager#checkNode}} and leave method {{markCheckpointBegin}} 
before future is done ({{curr.cpBeginFut.onDone();}})



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8165) Spark Dataset Write intermittent "Failed to map key to node" error

[mark pettovello (JIRA)]

mark pettovello created IGNITE-8165:
---

 Summary: Spark Dataset Write intermittent "Failed to map key to 
node" error 
 Key: IGNITE-8165
 URL: https://issues.apache.org/jira/browse/IGNITE-8165
 Project: Ignite
  Issue Type: Bug
  Components: jdbc, spark
Affects Versions: 2.4
 Environment: Spark 2.1.0

Java 1.8.0_152

Ignite-core-2.4.0.jar

ignite-spark_2.10-2.4.0.jar

Scala 2.11.8

 

 
Reporter: mark pettovello


Inserts partially fail when issuing a Dataset  write() operation.  
Rerunning write operation causes different sets of rows fail to insert.  Not 
all of the rows in dsCity.show() are inserted into Ignite.  All random missing 
rows encountered "Failed to map key to node" exception.

 

SparkSession spark = SparkSession
 .builder()
 .appName("IgniteSQLDataSource example")
.master("local[4]")//run local PC using Winutils
.config("spark.local.dir","/tmp")
 .getOrCreate();

 ... create about 10 \{(int) ID, (string) NAME} tuples and add them to the 
dsCity dataset ...

Dataset dsCity = spark.createDataset(...).toDF("ID","NAME");

dsCity.show(1000);

String tblName = "CITY";
String jdbcURL = "jdbc:ignite:thin://127.0.0.1/";

 

dsCity.write()
 .format("jdbc")
.option("primary_key_fields", "ID")
 .option("url", jdbcURL)
 .option("driver", "org.apache.ignite.IgniteJdbcThinDriver")
 .option("batchsize", 1000)
 .option("dbtable", tblName)
.mode(SaveMode.Append)
 .save();

 

18/04/06 09:33:23 ERROR Executor: Exception in task 3.0 in stage 2.0 (TID 5)
java.sql.BatchUpdateException: Failed to map key to node.
 at 
org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.executeBatch(JdbcThinStatement.java:435)
 at 
org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.savePartition(JdbcUtils.scala:597)
 at 
org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$saveTable$1.apply(JdbcUtils.scala:670)
 at 
org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$saveTable$1.apply(JdbcUtils.scala:670)
 at 
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:925)
 at 
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:925)
 at 
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1944)
 at 
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1944)
 at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
 at org.apache.spark.scheduler.Task.run(Task.scala:99)
 at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Thin clients release cycle

[Petr Ivanov]

I would vote for single repository and the following release scheme: we build 
and release everything, but deliver only that modules, which have actual 
features / bugfixes, skipping other from release iteration until new changes 
come into them.

Also, I’d propose versioning scheme, that will reflect Apache Ignite’s 
compatibility, i.e. if thin client requires features from Apache Ignite 2.4, 
then it’s own version will be 2.4.A.B (where A - own major version, that will 
be reset every new Apache Release, B - minor/fix version for minor / quick 
fixes goals that will be reset every new A version). 



> On 6 Apr 2018, at 16:15, Vladimir Ozerov  wrote:
> 
> Igniters,
> 
> Over the last year we saw dramatic increase in demand for lightweight thin
> clients. We already have four: JDBC, ODBC, .NET, Java. In future we are
> going to have even more: NodeJS, PHP, Python, Go, whatever. I'd like to
> start a discussion on how are we going to host them. There are several
> approaches.
> 
> 1) *Monolith* - everything is hosted in a single repository and released as
> a single artifact. This is our current approach.
> 
> Pros:
> - Easy to manage
> Cons
> - Long release cycle
> - Client features must be developed in sync with each other which would be
> very hard should we have > 5-6 different clients:
> 
> 2) *Modules* - clients are moved to separate repositories with their own
> release cycles. JDBC is released separately, ODBC separately, .NET
> separately, Java (guess what?) - separately, etc..They could have different
> timelines, different feature sets, different release notes, different
> versions. This is natural approach employed by multitude of vendors.  When
> new feature is added we do not need to wait for Apache Ignite release.
> Instead, we release only small client on it's own.
> Pros:
> - Fast and (sorry) agile release cycle!
> - No need to wait for months for new Ignite version
> - No need to sync features between different clients
> Cons:
> - More votes, more artifacts, more release-related code and scripts
> 
> 3) *Hybrid* - all clients are hosted in a single separate repository and
> released in sync with each other, but not with Apache Ignite. Balanced mix
> of pros/cons from #1 and #2 approaches.
> Pros:
> - Relatively fast release cycle
> Cons:
> - Still need to sync features between clients
> 
> I think that we should start moving our clients to #2 or #3 approaches to
> get greater momentum from community, What do you think?
> 
> Vladimir.

Thin clients release cycle

[Vladimir Ozerov]

Igniters,

Over the last year we saw dramatic increase in demand for lightweight thin
clients. We already have four: JDBC, ODBC, .NET, Java. In future we are
going to have even more: NodeJS, PHP, Python, Go, whatever. I'd like to
start a discussion on how are we going to host them. There are several
approaches.

1) *Monolith* - everything is hosted in a single repository and released as
a single artifact. This is our current approach.

Pros:
- Easy to manage
Cons
- Long release cycle
- Client features must be developed in sync with each other which would be
very hard should we have > 5-6 different clients:

2) *Modules* - clients are moved to separate repositories with their own
release cycles. JDBC is released separately, ODBC separately, .NET
separately, Java (guess what?) - separately, etc..They could have different
timelines, different feature sets, different release notes, different
versions. This is natural approach employed by multitude of vendors.  When
new feature is added we do not need to wait for Apache Ignite release.
Instead, we release only small client on it's own.
Pros:
- Fast and (sorry) agile release cycle!
- No need to wait for months for new Ignite version
- No need to sync features between different clients
Cons:
- More votes, more artifacts, more release-related code and scripts

3) *Hybrid* - all clients are hosted in a single separate repository and
released in sync with each other, but not with Apache Ignite. Balanced mix
of pros/cons from #1 and #2 approaches.
Pros:
- Relatively fast release cycle
Cons:
- Still need to sync features between clients

I think that we should start moving our clients to #2 or #3 approaches to
get greater momentum from community, What do you think?

Vladimir.

Re: IGNITE-6879

[Роман Меерсон]

OK

[image: 1486924635147168240.jpg]


пт, 6 апр. 2018 г. в 17:08, Igor Sapego :

> Hi,
> Well, Dmitry has said he's going to merge it in 3-4 days 2 days ago,
> so I guess, the merge is going to happen in 1-2 days or so.
>
>
> Best Regards,
> Igor
>
> On Fri, Apr 6, 2018 at 3:48 PM, Роман Меерсон 
> wrote:
>
> > Hi all!
> >
> > As i see everything is awesome and there is no objections, so when my PR
> > would be merged?
> >
> > чт, 5 апр. 2018 г. в 18:58, Вячеслав Коптилин  >:
> >
> > > Thank you, Roman!
> > >
> > > 2018-04-05 17:49 GMT+03:00 Роман Меерсон :
> > >
> > > > Hi Slava,
> > > >
> > > > Fixed
> > > >
> > > > чт, 5 апр. 2018 г. в 18:41, Вячеслав Коптилин <
> > slava.kopti...@gmail.com
> > > >:
> > > >
> > > > > Hi Roman,
> > > > >
> > > > > please take into account my comment IgniteQueryGenerator.java
> > > > > <
> > > > > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-541?
> > > > commentId=de43c65f-9ac7-4080-9904-aec119138c94=/
> > > > modules/spring-data-2.0/src/main/java/org/apache/ignite/
> > > > springdata/repository/query/IgniteQueryGenerator.java
> > > > > >
> > > > >
> > > > > Best regards,
> > > > > Slava.
> > > > >
> > > > > 2018-04-05 14:59 GMT+03:00 Роман Меерсон :
> > > > >
> > > > > > Ok, so waiting for accept and commit
> > > > > >
> > > > > > чт, 5 апр. 2018 г. в 15:29, Alexey Kukushkin <
> > > > kukushkinale...@gmail.com
> > > > > >:
> > > > > >
> > > > > > > Roman,
> > > > > > >
> > > > > > > Just pay commiter's (Dmitry Pavlov will most likely commit your
> > > code)
> > > > > > > attention to include the new test suite to TeamCity
> > configuration.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: IGNITE-6879

[Igor Sapego]

Hi,
Well, Dmitry has said he's going to merge it in 3-4 days 2 days ago,
so I guess, the merge is going to happen in 1-2 days or so.


Best Regards,
Igor

On Fri, Apr 6, 2018 at 3:48 PM, Роман Меерсон  wrote:

> Hi all!
>
> As i see everything is awesome and there is no objections, so when my PR
> would be merged?
>
> чт, 5 апр. 2018 г. в 18:58, Вячеслав Коптилин :
>
> > Thank you, Roman!
> >
> > 2018-04-05 17:49 GMT+03:00 Роман Меерсон :
> >
> > > Hi Slava,
> > >
> > > Fixed
> > >
> > > чт, 5 апр. 2018 г. в 18:41, Вячеслав Коптилин <
> slava.kopti...@gmail.com
> > >:
> > >
> > > > Hi Roman,
> > > >
> > > > please take into account my comment IgniteQueryGenerator.java
> > > > <
> > > > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-541?
> > > commentId=de43c65f-9ac7-4080-9904-aec119138c94=/
> > > modules/spring-data-2.0/src/main/java/org/apache/ignite/
> > > springdata/repository/query/IgniteQueryGenerator.java
> > > > >
> > > >
> > > > Best regards,
> > > > Slava.
> > > >
> > > > 2018-04-05 14:59 GMT+03:00 Роман Меерсон :
> > > >
> > > > > Ok, so waiting for accept and commit
> > > > >
> > > > > чт, 5 апр. 2018 г. в 15:29, Alexey Kukushkin <
> > > kukushkinale...@gmail.com
> > > > >:
> > > > >
> > > > > > Roman,
> > > > > >
> > > > > > Just pay commiter's (Dmitry Pavlov will most likely commit your
> > code)
> > > > > > attention to include the new test suite to TeamCity
> configuration.
> > > > > >
> > > > >
> > > >
> > >
> >
>

[GitHub] ignite pull request #3770: IGNITE-8017 (2)

[ilantukh]

GitHub user ilantukh opened a pull request:

https://github.com/apache/ignite/pull/3770

IGNITE-8017 (2)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8017-8122

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3770.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3770


commit 604ee719b304d0b4cf4caabaa6fa16b5a980e04e
Author: Pavel Kovalenko 
Date:   2018-04-04T09:33:10Z

IGNITE-8122 Restore partition state to OWNING if unable to read from page 
memory.

commit 67a48943b876b193067625fb564ab44d57a83f56
Author: Ilya Lantukh 
Date:   2018-03-23T13:10:41Z

ignite-8017 : Simple test.

commit 0021e39fe4a232a402c93aa6431048cdd00ca74c
Author: Ilya Lantukh 
Date:   2018-03-23T13:46:29Z

ignite-8017 : Basic implementation.

commit 3d446aa7bf78d53511725eb33e2546293e641167
Author: Ilya Lantukh 
Date:   2018-03-23T14:05:55Z

ignite-8017 : Configuration property.

commit bda2ba4e25e6699974520cc378638be22f4e1db3
Author: Ilya Lantukh 
Date:   2018-03-23T14:09:04Z

ignite-8017 : Property value check.

commit 9f50edc7a68dbf3253ec19ef15a8530d8484377a
Author: Ilya Lantukh 
Date:   2018-03-23T14:46:19Z

ignite-8017 : Fixed test finalization.

commit da2d400f578cd57c129c9a410e48b4586310a728
Author: Ilya Lantukh 
Date:   2018-03-26T12:34:14Z

ignite-8017 : Fixed parameter handling.

commit 34aa645d93c67ab7cea75edd4357339aab785bc8
Author: Ilya Lantukh 
Date:   2018-03-26T16:25:00Z

ignite-8017 : WIP.

commit 620b15b97511ade697ed7ec3255f4312afe857c5
Author: Ilya Lantukh 
Date:   2018-03-27T15:04:03Z

ignite-8017 : Test refactoring.

commit adab75755da9dcdec82d27e8d4d4099491c6d2f3
Author: Ilya Lantukh 
Date:   2018-03-27T15:19:26Z

ignite-8017 : Test for local and global WAL state.

commit 95492f025381521b76b480d4871a88a5fc9e3b40
Author: Ilya Lantukh 
Date:   2018-03-27T15:27:15Z

ignite-8017 : Separate handling for global and local WAL state.

commit 6da9bd39a28a6aee21613c2fb660ad6a9c6b62fb
Author: Ilya Lantukh 
Date:   2018-03-29T13:58:58Z

ignite-8017 : WIP.

commit 66968f98a3579c939ea0166008c36f15dbd0d484
Author: Ilya Lantukh 
Date:   2018-04-03T13:43:07Z

ignite-8017 : WIP.

commit d008fde6ded45d3f31ca3848bbbe347d75a05f30
Author: Ilya Lantukh 
Date:   2018-04-03T14:04:32Z

ignite-8017 : WIP.

commit 06d820950d87c0c4405c5b063596edcde37b0bf3
Author: Ilya Lantukh 
Date:   2018-04-04T16:01:55Z

ignite-8017 : WIP.

commit 4d8b7bdd0724da39d4cc45b403624c479984d1f8
Author: Ilya Lantukh 
Date:   2018-04-04T16:06:31Z

ignite-8017 : WIP.

commit 5022a554aa2ac73200f9180314f2ef69b54afb38
Author: Ilya Lantukh 
Date:   2018-04-05T11:54:12Z

ignite-8017 : WIP.

commit c43c598916bd9bda2f624a214cefcc28b0380a5c
Author: Pavel Kovalenko 
Date:   2018-04-05T14:26:53Z

IGNITE-8122 Restore partitions when persistence is enabled with OWNING 
default state.

commit a061cdba3a46f5384910fecf89366101f904ccdf
Author: Pavel Kovalenko 
Date:   2018-04-05T14:50:20Z

IGNITE-8122 Move OWN logic to GridDhtLocalPartition constructor.

commit 9259407a462633a68de6dcd7d3ae135c8c7c0b37
Author: Pavel Kovalenko 
Date:   2018-04-05T17:15:39Z

IGNITE-8122 Docs.

commit 20979dc4874cfeca649b29d8827d522f3e57ee67
Author: Pavel Kovalenko 
Date:   2018-04-05T17:16:54Z

Merge branch 'master' into ignite-8122

commit 96ac09a87690fbe8a0d9623b01e12715905c568a
Author: Ilya Lantukh 
Date:   2018-04-06T12:28:05Z

Merge branch 'ignite-8017' of https://github.com/gridgain/apache-ignite 
into ignite-8017-8122

commit fb079afe7d96aa0c5511d58c7a9db3b77ee8813f
Author: Ilya Lantukh 
Date:   2018-04-06T13:02:16Z

ignite-8017 : WIP.




---

Re: Breaking change in JDBC connection string format

[Igor Sapego]

ODBC uses semicolon and this semantics are defined by ODBC specification.

Best Regards,
Igor

On Thu, Apr 5, 2018 at 10:35 PM, Denis Magda  wrote:

> Vladimir, Igor,
>
> Shouldn't we do the same for ODBC?
>
> --
> Denis
>
> On Thu, Apr 5, 2018 at 5:53 AM, Vladimir Ozerov 
> wrote:
>
> > I think colon is not very good candidate as it clashes with other
> > properties (e.g. host-port delimiter). Semicolon looks good to me. I'll
> > created a ticket [1] to address this.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-8148
> >
> > On Wed, Apr 4, 2018 at 12:35 AM, Andrey Gura  wrote:
> >
> > > Denis,
> > >
> > > actually params (key-value pairs) are separated by colon in provided
> > > JIRA issue comment.
> > >
> > > On Tue, Apr 3, 2018 at 7:55 PM, Denis Magda  wrote:
> > > > Vladimir, Taras,
> > > >
> > > > Will "@" work for us as the delimiter? I would agree with Andrey to
> > reuse
> > > > this approach for the thin client.
> > > >
> > > > As for the breaking changes, if update the delimiter for the next
> > driver
> > > > version and make sure that version understands 2 delimiters for some
> > time
> > > > (& and the new one), then this would be ideal and not disrupting.
> > > >
> > > > --
> > > > Denis
> > > >
> > > > On Tue, Apr 3, 2018 at 2:39 AM, Andrey Gura 
> wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> We've been solve this problem during JDBC2 driver implementation.
> And
> > > >> I don't know any complains about connection string format. Why we
> can
> > > >> just use the same approach? [1]
> > > >>
> > > >> [1] https://issues.apache.org/jira/browse/IGNITE-1250?
> > > >> focusedCommentId=14706511=com.atlassian.jira.
> > > >> plugin.system.issuetabpanels:comment-tabpanel#comment-14706511
> > > >>
> > > >> On Tue, Apr 3, 2018 at 12:20 PM, Ilya Kasnacheev
> > > >>  wrote:
> > > >> > Hello!
> > > >> >
> > > >> > I think semicolon is the way to go, since round brackets are often
> > > >> > interpreted by shells and will need escaping on their own. Let's
> get
> > > rid
> > > >> of
> > > >> > & and ?.
> > > >> >
> > > >> > jdbc:ignite:thin://host:port/schema:param1=val1:param2=val2
> > > >> >
> > > >> > --
> > > >> > Ilya Kasnacheev
> > > >> >
> > > >> > 2018-04-03 11:30 GMT+03:00 Vladimir Ozerov  >:
> > > >> >
> > > >> >> Igniters,
> > > >> >>
> > > >> >> Thanks to Alex Kuznetsov, we revealed serious usability issue in
> > our
> > > >> thin
> > > >> >> JDBC driver - we user ampersand character to delimit properties:
> > > >> >>
> > > >> >> jdbc:ignite:thin://host:port/schema?param1=val1*&*param2=val2
> > > >> >>
> > > >> >> This leads to multiple problems:
> > > >> >> 1) This format is unusable from many console environments and
> > require
> > > >> >> special escaping (PowerShell, bash)
> > > >> >> 2) Also this causing issues when writing connection string is
> > passed
> > > to
> > > >> >> some 3rd-party applications as sometimes they cannot escape it as
> > > well.
> > > >> >>
> > > >> >> I performed investigation on how other vendors do that. Bottom
> > line -
> > > >> >> nobody use ampersand. Instead semicolon or parentheses are used:
> > > >> >>
> > > >> >> jdbc:ignite:thin://host:port/schema?param1=val1=val2
> > > >> >> jdbc:ignite:thin://host:port/schema?(param1=val1)(param2=val2)
> > > >> >>
> > > >> >> I propose to do a breaking change in AI 2.5 and change the format
> > to
> > > >> >> *parentheses*. This would be a disruptive experience for existing
> > > users,
> > > >> >> but in the long term benefits will outweight. Also remember that
> we
> > > do
> > > >> not
> > > >> >> offered officially any backward compatibility for our JDBC
> driver.
> > > >> >>
> > > >> >> Alternatively we may allow users to use previous format with help
> > of
> > > >> system
> > > >> >> property or environment variable.
> > > >> >>
> > > >> >> Thoughts?
> > > >> >>
> > > >> >> Vladimir.
> > > >> >>
> > > >>
> > >
> >
>

Re: IGNITE-6879

[Роман Меерсон]

Hi all!

As i see everything is awesome and there is no objections, so when my PR
would be merged?

чт, 5 апр. 2018 г. в 18:58, Вячеслав Коптилин :

> Thank you, Roman!
>
> 2018-04-05 17:49 GMT+03:00 Роман Меерсон :
>
> > Hi Slava,
> >
> > Fixed
> >
> > чт, 5 апр. 2018 г. в 18:41, Вячеслав Коптилин  >:
> >
> > > Hi Roman,
> > >
> > > please take into account my comment IgniteQueryGenerator.java
> > > <
> > > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-541?
> > commentId=de43c65f-9ac7-4080-9904-aec119138c94=/
> > modules/spring-data-2.0/src/main/java/org/apache/ignite/
> > springdata/repository/query/IgniteQueryGenerator.java
> > > >
> > >
> > > Best regards,
> > > Slava.
> > >
> > > 2018-04-05 14:59 GMT+03:00 Роман Меерсон :
> > >
> > > > Ok, so waiting for accept and commit
> > > >
> > > > чт, 5 апр. 2018 г. в 15:29, Alexey Kukushkin <
> > kukushkinale...@gmail.com
> > > >:
> > > >
> > > > > Roman,
> > > > >
> > > > > Just pay commiter's (Dmitry Pavlov will most likely commit your
> code)
> > > > > attention to include the new test suite to TeamCity configuration.
> > > > >
> > > >
> > >
> >
>

Re: Upsource update required

[Anton Vinogradov]

Anyway,

We have to find the reason why merge from master breaks upsource review
when PR is ok.

2018-04-06 15:28 GMT+03:00 Dmitry Pavlov :

> Defenetely, it may be reason of my PR / CR problem because I've merged
> master into my branch. Thank you.
>
> пт, 6 апр. 2018 г. в 15:14, Vyacheslav Daradur :
>
> > Dmitry, I confirm that a problem existed.
> >
> > Upsource can't handle situations of merging master to PR branch, in
> > this case, Upsorce shows changes which are not related to a pull
> > request.
> >
> > I know only one workaround solution: rebasing branch on master and
> > never merge it, but in this case, we lost mapping between existing
> > comments and the commits in a pull request.
> >
> >
> > On Fri, Apr 6, 2018 at 3:05 PM, Dmitry Pavlov 
> > wrote:
> > > Hi Igniters, Anton,
> > >
> > > According to my experience upsource works well, except 1 suspicious
> case
> > > with my PR.
> > >
> > > So I'm not sure if there are Upsource problems, probably there are
> > problems
> > > with some of our PRs?
> > >
> > > My example is https://github.com/apache/ignite/pull/3243 PR and
> > > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-513 ,- it is
> > quite
> > > old PR, so I can suppose it is some commit problem.
> > >
> > > Sinerely,
> > > Dmitriy Pavlov
> > >
> > > пт, 6 апр. 2018 г. в 14:23, Anton Vinogradov :
> > >
> > >> Igniters.
> > >>
> > >> Who is responsible for Upsource [1]?
> > >> I see some strange things at reviews, a lot of fake changes inside
> > reviews.
> > >> I don't see such changes at PRs.
> > >> Could you please check we're using stable version and update if
> > necessary?
> > >>
> > >>
> > >> [1] https://reviews.ignite.apache.org
> > >>
> >
> >
> >
> > --
> > Best Regards, Vyacheslav D.
> >
>

[GitHub] ignite pull request #3769: Ignite 8049

[SpiderRus]

GitHub user SpiderRus opened a pull request:

https://github.com/apache/ignite/pull/3769

Ignite 8049



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8049

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3769.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3769


commit 035c5c44d008b79cfa2a9a29ca1775e3f06bef55
Author: Алексей Стельмак 
Date:   2018-04-03T11:33:04Z

Limit the number of operation cycles in B+Tree

commit 140f448f0856c10d07bd4f4bf4384830d176ac7b
Author: Алексей Стельмак 
Date:   2018-04-06T10:06:39Z

Limit the number of operation cycles in B+Tree




---

Re: Upsource update required

[Dmitry Pavlov]

Defenetely, it may be reason of my PR / CR problem because I've merged
master into my branch. Thank you.

пт, 6 апр. 2018 г. в 15:14, Vyacheslav Daradur :

> Dmitry, I confirm that a problem existed.
>
> Upsource can't handle situations of merging master to PR branch, in
> this case, Upsorce shows changes which are not related to a pull
> request.
>
> I know only one workaround solution: rebasing branch on master and
> never merge it, but in this case, we lost mapping between existing
> comments and the commits in a pull request.
>
>
> On Fri, Apr 6, 2018 at 3:05 PM, Dmitry Pavlov 
> wrote:
> > Hi Igniters, Anton,
> >
> > According to my experience upsource works well, except 1 suspicious case
> > with my PR.
> >
> > So I'm not sure if there are Upsource problems, probably there are
> problems
> > with some of our PRs?
> >
> > My example is https://github.com/apache/ignite/pull/3243 PR and
> > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-513 ,- it is
> quite
> > old PR, so I can suppose it is some commit problem.
> >
> > Sinerely,
> > Dmitriy Pavlov
> >
> > пт, 6 апр. 2018 г. в 14:23, Anton Vinogradov :
> >
> >> Igniters.
> >>
> >> Who is responsible for Upsource [1]?
> >> I see some strange things at reviews, a lot of fake changes inside
> reviews.
> >> I don't see such changes at PRs.
> >> Could you please check we're using stable version and update if
> necessary?
> >>
> >>
> >> [1] https://reviews.ignite.apache.org
> >>
>
>
>
> --
> Best Regards, Vyacheslav D.
>

[GitHub] ignite pull request #3766: fix to avoid race between auto-activation and exp...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3766


---

Re: Upsource update required

[Vyacheslav Daradur]

Dmitry, I confirm that a problem existed.

Upsource can't handle situations of merging master to PR branch, in
this case, Upsorce shows changes which are not related to a pull
request.

I know only one workaround solution: rebasing branch on master and
never merge it, but in this case, we lost mapping between existing
comments and the commits in a pull request.


On Fri, Apr 6, 2018 at 3:05 PM, Dmitry Pavlov  wrote:
> Hi Igniters, Anton,
>
> According to my experience upsource works well, except 1 suspicious case
> with my PR.
>
> So I'm not sure if there are Upsource problems, probably there are problems
> with some of our PRs?
>
> My example is https://github.com/apache/ignite/pull/3243 PR and
> https://reviews.ignite.apache.org/ignite/review/IGNT-CR-513 ,- it is quite
> old PR, so I can suppose it is some commit problem.
>
> Sinerely,
> Dmitriy Pavlov
>
> пт, 6 апр. 2018 г. в 14:23, Anton Vinogradov :
>
>> Igniters.
>>
>> Who is responsible for Upsource [1]?
>> I see some strange things at reviews, a lot of fake changes inside reviews.
>> I don't see such changes at PRs.
>> Could you please check we're using stable version and update if necessary?
>>
>>
>> [1] https://reviews.ignite.apache.org
>>



-- 
Best Regards, Vyacheslav D.

[GitHub] ignite pull request #3768: IGNITE-8111 Add extra validation for WAL segment ...

[dgarus]

GitHub user dgarus opened a pull request:

https://github.com/apache/ignite/pull/3768

IGNITE-8111 Add extra validation for WAL segment size



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dgarus/ignite ignite-8111

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3768.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3768


commit f6a76f18246a6af3f69c126c2fbf3c4680319754
Author: denis.garus 
Date:   2018-04-06T12:06:56Z

IGNITE-8111 Add extra validation for WAL segment size




---

[GitHub] ignite pull request #3767: IGNITE-8085

[1vanan]

GitHub user 1vanan opened a pull request:

https://github.com/apache/ignite/pull/3767

IGNITE-8085



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/1vanan/ignite IGNITE-8085

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3767.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3767


commit 192dbd515dcf81638ce52d6ec1f3239f97e29538
Author: Fedotov 
Date:   2018-04-06T08:38:06Z

make more timeout and kick off Thread.sleep

commit f97bae5b3e43b29d04c3806dc56201364d065649
Author: Fedotov 
Date:   2018-04-06T11:57:50Z

fix timeout and foreach

commit 7cd3e0dda7b569f91fea9e8cb669df89199da287
Author: Fedotov 
Date:   2018-04-06T12:04:13Z

testSuite




---

Re: Upsource update required

[Dmitry Pavlov]

Hi Igniters, Anton,

According to my experience upsource works well, except 1 suspicious case
with my PR.

So I'm not sure if there are Upsource problems, probably there are problems
with some of our PRs?

My example is https://github.com/apache/ignite/pull/3243 PR and
https://reviews.ignite.apache.org/ignite/review/IGNT-CR-513 ,- it is quite
old PR, so I can suppose it is some commit problem.

Sinerely,
Dmitriy Pavlov

пт, 6 апр. 2018 г. в 14:23, Anton Vinogradov :

> Igniters.
>
> Who is responsible for Upsource [1]?
> I see some strange things at reviews, a lot of fake changes inside reviews.
> I don't see such changes at PRs.
> Could you please check we're using stable version and update if necessary?
>
>
> [1] https://reviews.ignite.apache.org
>

[jira] [Created] (IGNITE-8164) SQL TX: JDBC driver meta data update.

[Roman Kondakov (JIRA)]

Roman Kondakov created IGNITE-8164:
--

 Summary: SQL TX: JDBC driver meta data update.
 Key: IGNITE-8164
 URL: https://issues.apache.org/jira/browse/IGNITE-8164
 Project: Ignite
  Issue Type: Bug
  Components: jdbc, sql
Reporter: Roman Kondakov


Since we've implemented a transactional SQL, we need to reflect it in our JDBC 
driver. At the moment our implementation of {{java.sql.DatabaseMetaData}} 
returns incorrect data. For example, methods

{{boolean supportsTransactions()}}

and 

{{boolean supportsTransactionIsolationLevel(int level)}}

return hardcoded  {{false }}value, which is incorrect if MVCC is enabled. We 
need to fix it.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3765: Ignite 8049

[SpiderRus]

Github user SpiderRus closed the pull request at:

https://github.com/apache/ignite/pull/3765


---

[GitHub] ignite pull request #3766: fix to avoid race between auto-activation and exp...

[sergey-chugunov-1985]

GitHub user sergey-chugunov-1985 opened a pull request:

https://github.com/apache/ignite/pull/3766

fix to avoid race between auto-activation and explicit activation



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8163

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3766.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3766


commit a765ef4d0fd9de0d011f7f4346844116c29c4c27
Author: Sergey Chugunov 
Date:   2018-04-05T10:07:13Z

pds-hang-fix fix to avoid race between auto-activation and explicit 
activation




---

[GitHub] ignite pull request #3765: Ignite 8049

[SpiderRus]

GitHub user SpiderRus opened a pull request:

https://github.com/apache/ignite/pull/3765

Ignite 8049



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8049

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3765.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3765


commit 035c5c44d008b79cfa2a9a29ca1775e3f06bef55
Author: Алексей Стельмак 
Date:   2018-04-03T11:33:04Z

Limit the number of operation cycles in B+Tree

commit 140f448f0856c10d07bd4f4bf4384830d176ac7b
Author: Алексей Стельмак 
Date:   2018-04-06T10:06:39Z

Limit the number of operation cycles in B+Tree




---

[jira] [Created] (IGNITE-8163) PDS Indexing suite is hanging on TC in different branches including master

[Sergey Chugunov (JIRA)]

Sergey Chugunov created IGNITE-8163:
---

 Summary: PDS Indexing suite is hanging on TC in different branches 
including master
 Key: IGNITE-8163
 URL: https://issues.apache.org/jira/browse/IGNITE-8163
 Project: Ignite
  Issue Type: Bug
Reporter: Sergey Chugunov
Assignee: Sergey Chugunov






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Upsource update required

[Anton Vinogradov]

Igniters.

Who is responsible for Upsource [1]?
I see some strange things at reviews, a lot of fake changes inside reviews.
I don't see such changes at PRs.
Could you please check we're using stable version and update if necessary?


[1] https://reviews.ignite.apache.org

Re: Ability to check and completely fill transactions on creation

[Anton Vinogradov]

 >> But I have concern
>> about performance. How can you estimate impact to performance ?
We have to benchmark result.

>> How about to set label name with some useful info if user does not
provide
>> custom name?
You can set custom listener which will do that

>>  For example thread name + global counter?
Or even full stacktrace

>>  how the user is expected to use this event?
Event will be used to validate tx on creation.
Since listner will be invoked at same thread (Am I right?) it will have all
requred info for validation.


2018-04-05 22:06 GMT+03:00 Denis Magda :

> Guys,
>
> Sorry for a dumb question but how the user is expected to use this event?
>
> --
> Denis
>
> On Thu, Apr 5, 2018 at 6:06 AM, Anton Vinogradov  wrote:
>
> > Igniters,
> >
> > As far as I know we're working on additional 'label' field for
> transactions
> > [1].
> > That's great and will be helpful for customers with huge deploymens to
> see
> > reason of each transaction.
> > But, since 'label' is optional field, there is no way to guarantee it
> will
> > be filled.
> >
> > I'd like to propose an idea of brand new event EVT_USR_TX_CREATED (local
> > transaction created).
> >
> > Local listener on such event will allow to guarantee tx's 'label' field
> > filled, timeout is correct and so on.
> >
> > Thoughts?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-6827
> >
>

Re: Data Loss while upgrading custom jar from old jar in server and client nodes

[Ivan Rakov]

Denis,

I'm sure we can: https://issues.apache.org/jira/browse/IGNITE-8162

Best Regards,
Ivan Rakov

On 05.04.2018 22:39, Denis Magda wrote:

Ivan,

How can we facilitate the user here? Can we generate a meaningful exception
that explains how to tackle the issue?

--
Denis

On Thu, Apr 5, 2018 at 2:49 AM, Ivan Rakov  wrote:


Hi,

Cache configuration is persisted in 
db\{consistent-ID}\cache-{cache-name}\cache_data.dat
file. It's just CacheConfiguration serialized by JDK marshaller.
Try to delete this file and start cache with new configuration (with
correct factory class names). All your data will persist as long as old
ignite data is compatible with new configuration.

Best Regards,
Ivan Rakov


On 04.04.2018 23:08, Denis Magda wrote:


Ivan R., Alex G., persistence experts,

Please have a look at this question.

--
Denis

On Mon, Apr 2, 2018 at 6:15 AM, Andrey Mashenkov <
andrey.mashen...@gmail.com


wrote:
Crossposting to dev.

Guys,
User use Ignite native persistence with CacheStore configured.
Seems, changing CacheStore requires cache recreation in his case to
changes
can be applied.

Does anybody has idea how it can be fixed?

Do we allow to rewrite cache config in native store if it has minor
changes
that will not cause issues? E.g. changing cacheStore factory or changing
number of backups?
Otherwise, I'd suggest to deprecate such configuration.


On Tue, Mar 27, 2018 at 4:18 PM, siva  wrote:

Hi,

Thanks for checking out issue. I think its not about IDE issue.
Again same thing i have reproduce like bellow steps

1.run the program from IDE as client
2.copy jar to ignite libs folder and start the server
3.Then check previous message steps so that can able to reproduce the
issue.
https://www.youtube.com/watch?v=COQiu2gi8ag=43s

I have attached the video link.Can you go through that video and check


the


issue.


Thanks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/



--
Best regards,
Andrey V. Mashenkov



--
Best regards,
Andrey V. Mashenkov

[jira] [Created] (IGNITE-8162) Handle ClassNotFoundException during deserialization of persisted cache configuration

[Ivan Rakov (JIRA)]

Ivan Rakov created IGNITE-8162:
--

 Summary: Handle ClassNotFoundException during deserialization of 
persisted cache configuration 
 Key: IGNITE-8162
 URL: https://issues.apache.org/jira/browse/IGNITE-8162
 Project: Ignite
  Issue Type: Improvement
  Components: general
Affects Versions: 2.4
Reporter: Ivan Rakov
 Fix For: 2.6


Ticket is created according to dev list discussion: 
http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-Data-Loss-while-upgrading-custom-jar-from-old-jar-in-server-and-client-nodes-td28808.html
Cache configuration is serialized by JDK marshaller and persisted in 
cache_data.dat file. It may contain instances of classes that disappeared from 
runtime classpath (e.g. implementation of CacheStore has been renamed). In such 
case, node will fail on start.
We should handle this and show meaningful message with instruction how to 
overcome this issue - delete cache_data.dat and restart cache.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3764: GG-13379 Extended security for Huawei

[kukushal]

GitHub user kukushal opened a pull request:

https://github.com/apache/ignite/pull/3764

GG-13379 Extended security for Huawei



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite gridgain-13379

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3764.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3764


commit 4925ffc10ce8e8287980eaec38b587512568a302
Author: Alexey Goncharuk 
Date:   2018-01-17T12:26:03Z

IGNITE-7453 Use GridUnsafe.cleanDirectBuffer in PageSnapshot

commit bcd68a058683b2f17b7ac60471b6e7aab3e4f6de
Author: Pavel Tupitsyn 
Date:   2018-01-17T12:38:20Z

IGNITE-7301 .NET: Baseline topology

This closes #3352

commit 66b96316a7775ce8a6e2ff4475185d5929e4998b
Author: devozerov 
Date:   2018-01-17T12:54:17Z

Merge branch 'master' into ignite-2.4

commit 268481c1cf7fe57df24be130eb67c3e3a13afe01
Author: Alexey Goncharuk 
Date:   2018-01-17T13:50:34Z

IGNITE-7453 Use GridUnsafe.cleanDirectBuffer in WalStat

commit db0cd105719c8ae713b13b34d9dca0a8cd45d377
Author: Pavel Tupitsyn 
Date:   2018-01-17T14:05:25Z

IGNITE-6776 .NET: Thin client: Add SQL & LINQ example

This closes #3390

commit c214db879101aa5660e2a50b11cd20964c0bc114
Author: Andrey Gura 
Date:   2018-01-17T12:42:41Z

ignite-7450 FileWriteAheadLogManager always uses RandomAccessFileIOFactory 
now

commit 75c27d5e49d7458e46eb46e6f87a445c3f1320ea
Author: Alexey Kuznetsov 
Date:   2018-01-18T02:25:19Z

IGNITE-7274 Web Console: Support multiple statements on Queries screen.
(cherry picked from commit 1926783)

commit 36cc822935387b2150e690c29bc6992dca0563f7
Author: Dmitriy Shabalin 
Date:   2018-01-18T04:49:08Z

IGNITE-7306 Web Console: Fixed export data from tables.
(cherry picked from commit 1bb60ec)

commit d753298b4012894b56f5c9218325211cd84a21d5
Author: Peter Ivanov 
Date:   2018-01-18T06:18:53Z

IGNITE-7107 Apache Ignite RPM packages

* added changelog to package specification

This closes #3396

commit 63445893f1bc75dc9777184499f7eabc1d4e51b1
Author: Denis Mekhanikov 
Date:   2018-01-18T08:36:18Z

IGNITE-3935 Use PeerDeployAware for streamer transformer - Fixes #3378.

Signed-off-by: Alexey Goncharuk 

commit f3f9f2a24b23027cf0c835140322e41a788932ae
Author: Pavel Tupitsyn 
Date:   2018-01-18T09:05:12Z

IGNITE-7413 Fix SqlDmlExample

This closes #3389

commit 1daa7c41bf1460a4d9a2b0c26a7a317f2fca3fb7
Author: Alexey Kuznetsov 
Date:   2018-01-18T10:14:53Z

IGNITE-7461 UI tools: Actualized data storage configuration.
(cherry picked from commit 577e632)

commit cf0080210d24d9dd8b057f959446fac5f8a4ca01
Author: dpavlov 
Date:   2018-01-18T10:53:29Z

IGNITE-7380 Implemented pluggable Direct IO - Fixes #3226.

Signed-off-by: Alexey Goncharuk 

commit dd06d0bd7ef266bfbe156e858b312d1ac86e8982
Author: Pavel Tupitsyn 
Date:   2018-01-18T12:55:49Z

IGNITE-7465 .NET: Fix SqlDdlExample failure with standalone node

commit 57479ec564e1761716da3d5f9feb7a64c396a9f9
Author: Pavel Tupitsyn 
Date:   2018-01-18T13:45:54Z

.NET: Fix CacheLocalTest.TestTxDeadlockDetection

commit bd6be8a4653322905a3b63850c7e033ce3801ce5
Author: Pavel Tupitsyn 
Date:   2018-01-18T18:25:05Z

.NET: Thin client: Fix OP_BINARY_TYPE_GET schema passing format

commit 97564d160586d6d57d300937e6b8877994e35fc7
Author: rkondakov 
Date:   2018-01-19T08:24:51Z

IGNITE-6456: Ability to separately enable or disable JDBC, ODBC and thin 
client endpoints. This closes #3309.

commit d50274ca8875c9680c12e8786ac355a787ba95e0
Author: Yakov Zhdanov 
Date:   2018-01-18T17:57:17Z

Javadoc enhancements - added @see

commit cb2d3cf22388ab19fb2d34ae5bdfc8f1b608db75
Author: Dmitriy Govorukhin 
Date:   2018-01-18T14:14:26Z

IGNITE-7471 Use soft reference for checkpoint entry contents to avoid 
excessive memory usage

commit 3965923369870bb4e8e57e3332c1a1eb1e5f5ed3
Author: rkondakov 
Date:   2018-01-19T09:00:55Z

IGNITE-6772: SQL exception messages became more informative. This closes 
#3342.

commit ba68cb0fa87f776fcd0499d030c333f182611f41
Author: devozerov 
Date:   2018-01-19T09:03:52Z

Merge remote-tracking branch 'origin/ignite-2.4' into ignite-2.4

commit b54c0c8786bd74aa0abb208f537c29f0c4be4b1e
Author: tledkov-gridgain 
Date:   2018-01-19T09:09:34Z

IGNITE-7248: JDBC: fixed schema resolution for streaming 

[GitHub] ignite pull request #3686: IGNITE-8018 : Optimized initialValue(...)

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3686


---

[jira] [Created] (IGNITE-8161) Suspend-resume TX test is flaky on TC (~5% fail rate)

[Dmitriy Pavlov (JIRA)]

Dmitriy Pavlov created IGNITE-8161:
--

 Summary: Suspend-resume TX test is flaky on TC (~5% fail rate)
 Key: IGNITE-8161
 URL: https://issues.apache.org/jira/browse/IGNITE-8161
 Project: Ignite
  Issue Type: Test
  Components: cache
Reporter: Dmitriy Pavlov


https://ci.ignite.apache.org/viewLog.html?buildId=1176294=buildResultsDiv=IgniteTests24Java8_Cache6#testNameId-7194341254453895210

Causal chani java.lang.RuntimeException: javax.cache.CacheException: class 
org.apache.ignite.transactions.TransactionTimeoutException: Cache transaction 
timed out: GridNearTxLocal 

First exception in log
{noformat}
validParts=null, state=MARKED_ROLLBACK, timedOut=true, 
topVer=AffinityTopologyVersion [topVer=-1, minorTopVer=0], duration=172ms, 
onePhaseCommit=false], size=0]]]
at 
org.apache.ignite.internal.processors.cache.distributed.IgniteOptimisticTxSuspendResumeTest$CI2Exc.apply(IgniteOptimisticTxSuspendResumeTest.java:759)
at 
org.apache.ignite.internal.processors.cache.distributed.IgniteOptimisticTxSuspendResumeTest.executeTestForAllCaches(IgniteOptimisticTxSuspendResumeTest.java:728)
at 
org.apache.ignite.internal.processors.cache.distributed.IgniteOptimisticTxSuspendResumeTest.testTxTimeoutOnResumed(IgniteOptimisticTxSuspendResumeTest.java:431)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at junit.framework.TestCase.runTest(TestCase.java:176)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2018)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:136)
at 
org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1933)
at java.lang.Thread.run(Thread.java:745)
{noformat}

Test history
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-7194341254453895210=testDetails




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8160) GridCacheAbstractDataStructuresFailoverSelfTest#testAtomicInitialization flaky-fails on TC

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-8160:


 Summary: 
GridCacheAbstractDataStructuresFailoverSelfTest#testAtomicInitialization 
flaky-fails on TC
 Key: IGNITE-8160
 URL: https://issues.apache.org/jira/browse/IGNITE-8160
 Project: Ignite
  Issue Type: Test
Reporter: Alexey Goncharuk
Assignee: Alexey Goncharuk
 Fix For: 2.5


The test fails because sequence initialization compute is sent to nodes being 
stopped. The test should be changed to test:
1) If the closure may be sent to a stopping node, then NodeStoppingException 
should be ignored
2) Another test should send closures only to stable nodes and should not 
tolerate any failures



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8159) control.sh Failed with NPE in case of adding not online node in base line

[Alexey Kuznetsov (JIRA)]

Alexey Kuznetsov created IGNITE-8159:


 Summary: control.sh Failed with NPE in case of adding not online 
node in base line
 Key: IGNITE-8159
 URL: https://issues.apache.org/jira/browse/IGNITE-8159
 Project: Ignite
  Issue Type: Task
Affects Versions: 2.4
Reporter: Alexey Kuznetsov
Assignee: Alexey Kuznetsov
 Fix For: 2.5


{code}

Caused by: java.lang.NullPointerException
 at 
org.apache.ignite.internal.visor.baseline.VisorBaselineTask$VisorBaselineJob.currentBaseLine(VisorBaselineTask.java:100)
 at 
org.apache.ignite.internal.visor.baseline.VisorBaselineTask$VisorBaselineJob.add(VisorBaselineTask.java:148)
 at 
org.apache.ignite.internal.visor.baseline.VisorBaselineTask$VisorBaselineJob.run(VisorBaselineTask.java:203)
 at 
org.apache.ignite.internal.visor.baseline.VisorBaselineTask$VisorBaselineJob.run(VisorBaselineTask.java:52)
 at org.apache.ignite.internal.visor.VisorJob.execute(VisorJob.java:69)
 at 
org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:566)
 at 
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6623)
 at 
org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:560)
 at 
org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:489)
 at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
 at 
org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1123)
 at 
org.apache.ignite.internal.processors.task.GridTaskWorker.sendRequest(GridTaskWorker.java:1407)
 at 
org.apache.ignite.internal.processors.task.GridTaskWorker.processMappedJobs(GridTaskWorker.java:660)
 at 
org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:532)
 at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
 at 
org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:760)
 at 
org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:509)
 at 
org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:489)
 at 
org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler.handleAsyncUnsafe(GridTaskCommandHandler.java:227)
 ... 8 more

{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8158) Missed cleanups if afterTestsStop throws exception

[Maxim Muzafarov (JIRA)]

Maxim Muzafarov created IGNITE-8158:
---

 Summary: Missed cleanups if afterTestsStop throws exception
 Key: IGNITE-8158
 URL: https://issues.apache.org/jira/browse/IGNITE-8158
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Maxim Muzafarov
 Fix For: 2.6


Method {{afterTestsStopped}} might throw exception. Contibutor should provide 
solution for ensuring that all cleanups in {{tearDown}} method would be 
executed in this case.

{code:java|title=GridAbstractTest.java}
/** {@inheritDoc} */
@Override protected void tearDown() throws Exception {
...

try {
afterTest();
}
finally {
serializedObj.clear();

if (isLastTest()) {

...

afterTestsStopped();

if (startGrid)
G.stop(getTestIgniteInstanceName(), true);

// Remove counters.
tests.remove(getClass());

// Remove resources cached in static, if any.
GridClassLoaderCache.clear();
U.clearClassCache();
MarshallerExclusions.clearCache();
BinaryEnumCache.clear();
}

Thread.currentThread().setContextClassLoader(clsLdr);

clsLdr = null;

cleanReferences();
}
}
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8157) Remove boilerplate and unused code due to grids stopping by default

[Maxim Muzafarov (JIRA)]

Maxim Muzafarov created IGNITE-8157:
---

 Summary: Remove boilerplate and unused code due to grids stopping 
by default
 Key: IGNITE-8157
 URL: https://issues.apache.org/jira/browse/IGNITE-8157
 Project: Ignite
  Issue Type: Task
Reporter: Maxim Muzafarov
Assignee: Maxim Muzafarov
 Fix For: 2.5


Changes IGNITE-6842 guarantee us stopping all Ignite instances after test 
completion by default.
 # We should remove a lot of bolerplate code. 
e.g.
{code:java|title=BaseDecisionTreeTest.java|borderStyle=solid}
/** {@inheritDoc} */
@Override protected void afterTestsStopped() throws Exception {
stopAllGrids();
}
{code}
 # We shuold carefully review whole usages of stopAllGrids method and remove if 
it not necessary anymore or change for using in proper way\position. 
e.g.
{code:java|title=TcpClientDiscoverySpiSelfTest.java|borderStyle=solid}
/** {@inheritDoc} */
@Override protected void afterTest() throws Exception {
stopAllClients(true);
stopAllServers(true);

...
}
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8156) Ignite Compatibility: common improvements

[Vyacheslav Daradur (JIRA)]

Vyacheslav Daradur created IGNITE-8156:
--

 Summary: Ignite Compatibility: common improvements
 Key: IGNITE-8156
 URL: https://issues.apache.org/jira/browse/IGNITE-8156
 Project: Ignite
  Issue Type: Task
Reporter: Vyacheslav Daradur
Assignee: Vyacheslav Daradur
 Fix For: 2.6


Found issues in the 'compatibility' module:
 * classpath building process should be simplified
 * {{DummyPersistenceCompatibilityTest}} should be renamed or split
 * needs resolve code style issues
 * {{IgniteUuidCompatibilityTest}} should be removed. {{UUID type}} should be 
checked in common test
 * PDS's directory cleaning logic should be moved to 
{{IgniteCompatibilityAbstractTest}}
 * etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3633: IGNITE-7933 Checkpoint markers writing issues

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3633


---

Re: Apache Ignite 2.5 release

[Petr Ivanov]

Added corresponding triggers for ignite-2.5 in Ignite Tests 2.4+ project in TC.



> On 5 Apr 2018, at 21:55, Denis Magda  wrote:
> 
> Thanks Andrey!
> 
> Folks, if you'd like to add anything to 2.5 please make sure it gets merged
> into 2.5 branch.
> 
> --
> Denis
> 
> On Thu, Apr 5, 2018 at 11:29 AM, Andrey Gura  wrote:
> 
>> Hi,
>> 
>> I've created branch ignite-2.5 for Apache Ignite 2.5 release.
>> 
>> As always please follow the rules below when merging new commits to master:
>> 
>> 1) If ticket is targeted for 2.5 release, please merge to master, then
>> cherry-pick to ignite-2.5
>> 2) Otherwise just merge to master.
>> 
>> 
>> 
>> On Wed, Apr 4, 2018 at 9:11 PM, Andrey Gura  wrote:
>>> Igniters,
>>> 
>>> It's time to create branch for upcoming Apache Ignite 2.5 release in
>>> order to start stabilization process.
>>> 
>>> If there are no any objections I'll create ignite-2.5 branch tomorrow.
>>> 
>>> Also please check JIRA issues assigned to you and move it to the next
>>> version if this issues shouldn't be included to 2.5 release.
>>> 
>>> Release page on wiki [1] contains all issues targeted to 2.5 (fix
>>> version field).
>>> 
>>> [1] https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.5
>>