Re: IgniteSet implementation: changes required

[Dmitriy Setrakyan]

On Thu, Mar 15, 2018 at 12:24 AM, Andrey Kuznetsov 
wrote:

> Dmitriy,
>
> It's technically possible to produce both kinds of sets with
> {{Ignite.set()}} call, but this will require to one more argument ('small'
> vs 'large'). Doesn't it look less inuitive than separate
> {{IgniteCache.asSet()}} ?
>
> And of course, we don't want to leave existing implementation broken. Pavel
> P. has prepared the fix as part of [1].
>
> [1] https://issues.apache.org/jira/browse/IGNITE-5553


Andrey, I am suggesting that we change all non-collocated sets to be based
on IgniteCache. In this case you do not need any additional parameters.

Makes sense?

D.

Re: Reconsider default WAL mode: we need something between LOG_ONLY and FSYNC

[Dmitriy Setrakyan]

Ivan,

Is there a performance difference between LOG_ONLY and LOG_ONLY_SAFE?

D.

On Thu, Mar 15, 2018 at 4:23 PM, Ivan Rakov  wrote:

> Igniters and especially Native Persistence experts,
>
> We decided to change default WAL mode from DEFAULT(FSYNC) to LOG_ONLY in
> 2.4 release. That was difficult decision: we sacrificed power loss / OS
> crash tolerance, but gained significant performance boost. From my
> perspective, LOG_ONLY is right choice, but it still misses some critical
> features that default mode should have.
>
> Let's focus on exact guarantees each mode provides. Documentation explains
> it in pretty simple manner: LOG_ONLY - writes survive process crash, FSYNC
> - writes survive power loss scenarios. I have to notice that documentation
> doesn't describe what exactly can happen to node in LOG_ONLY mode in case
> of power loss / OS crash scenario. Basically, there are two possible
> negative outcomes: loss of several last updates (it's exactly what can
> happen in BACKGROUND mode in case of process crash) and total storage
> corruption (not only last updates, but all data will be lost). I've made a
> quick research on this and came into conclusion that power loss in LOG_ONLY
> can lead to storage corruption. There are several explanations for this:
> 1) IgniteWriteAheadLogManager#fsync is kind of broken - it doesn't
> perform actual fsync unless current WAL mode is FSYNC. We call this method
> when we write checkpoint marker to WAL. As long as part of WAL before
> checkpoint marker can be not synced, "physical" records that are necessary
> for crash recovery in "Node stopped in the middle of checkpoint" scenario
> may be corrupted after power loss. If that happens, we won't be able to
> recover internal data structures, which means loss of all data.
> 2) We don't fsync WAL archive files unless current WAL mode is FSYNC. WAL
> archive can contain necessary "physical" records as well, which leads us to
> the case described above.
> 3) We do perform fsync on rollover (switch of current WAL segment) in all
> modes, but only when there's enough space to write switch segment record -
> see FileWriteHandle#close. So there's a little chance that we'll skip fsync
> and bump into the same case.
>
> Enforcing fsync on that three situations will give us a guarantee that
> LOG_ONLY will survive power loss scenarios with possibility of losing
> several last updates. There still can be a total binary mess in the last
> part of WAL, but as long as we perform CRC check during WAL replay, we'll
> detect start of that mess. Extra fsyncs may cause slight performance
> degradation - all writes will have to await for one fsync on every rollover
> and checkpoint. It's still much faster than fsync on every write in WAL - I
> expect a few percent (0-5%) drop comparing to current LOG_ONLY. But
> degradation is degradation, and LOG_ONLY mode without extra fsyncs makes
> sense as well - that's why we need to introduce "LOG_ONLY + extra fsyncs"
> as separate WAL mode. I think, we should make it default - it provides
> significant durability bonus for the cost of one extra fsync for each WAL
> segment written.
>
> To sum it up, I propose a new set of possible WAL modes:
> NONE - both process crash and power loss can lead to corruption
> BACKGROUND - process crash can lead to last updates loss, power loss can
> lead to corruption
> LOG_ONLY - writes survive process crash, power loss can lead to corruption
> LOG_ONLY_SAFE (default) - writes survive process crash, power loss can
> lead to last updates loss
> FSYNC - writes survive both process crash and power loss
>
> Thoughts?
>
>
> Best Regards,
> Ivan Rakov
>
>

[GitHub] ignite pull request #3644: IGNITE-7800: Correction for IgniteSystemPropertie...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3644


---

[GitHub] ignite pull request #3644: IGNITE-7800: Correction for IgniteSystemPropertie...

[shroman]

GitHub user shroman opened a pull request:

https://github.com/apache/ignite/pull/3644

IGNITE-7800: Correction for IgniteSystemProperties.IGNITE_NO_SHUTDOWN…

…_HOOK javadoc.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shroman/ignite IGNITE-7800

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3644.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3644


commit e68e0b1ee9ffeb28b0cdd7eb87f1165b2fecdefd
Author: shroman 
Date:   2018-03-16T03:10:32Z

IGNITE-7800: Correction for IgniteSystemProperties.IGNITE_NO_SHUTDOWN_HOOK 
javadoc.




---

[jira] [Created] (IGNITE-7970) Fix E2E test with user notifications

[Alexander Kalinin (JIRA)]

Alexander Kalinin created IGNITE-7970:
-

 Summary: Fix E2E test with user notifications
 Key: IGNITE-7970
 URL: https://issues.apache.org/jira/browse/IGNITE-7970
 Project: Ignite
  Issue Type: Bug
  Components: wizards
Reporter: Alexander Kalinin
Assignee: Alexander Kalinin


Currently Web Console E2E test on setting user notifications is failing. Should 
be fixed and passing.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Fwd: [ANNOUNCE] Apache Ignite 2.4.0 Released: Machine Learning GA and Spark DataFrames

[Denis Magda]

Igniters,

Please use your social media accounts to promote the release we were
working on for a while:
https://twitter.com/denismagda/status/974438862465351681

It will be great if you can translate the article into your native language
and publish it in your country of origin.

--
Denis

-- Forwarded message --
From: Denis Magda 
Date: Thu, Mar 15, 2018 at 5:09 PM
Subject: [ANNOUNCE] Apache Ignite 2.4.0 Released: Machine Learning GA and
Spark DataFrames
To: annou...@apache.org, pr...@apache.org, dev@ignite.apache.org


Usually, Ignite community rolls out a new version once in 3 months, but we
had to make an exception for Apache Ignite 2.4 that consumed five months in
total.

We could easily blame Thanksgiving, Christmas and New Year holidays for the
delay and would be forgiven, but, in fact, we were forging the release you
can't just pass by.

Let's dive in and look for a big fish: https://blogs.apache.
org/ignite/entry/apache-ignite-2-4-brings

The full list of the changes can be found here: https://ignite.apache.org/
releases/2.4.0/release_notes.html

Ready to try then navigate to our downloads page:
https://ignite.apache.org/download.cgi

--
Denis

[ANNOUNCE] Apache Ignite 2.4.0 Released: Machine Learning GA and Spark DataFrames

[Denis Magda]

Usually, Ignite community rolls out a new version once in 3 months, but we
had to make an exception for Apache Ignite 2.4 that consumed five months in
total.

We could easily blame Thanksgiving, Christmas and New Year holidays for the
delay and would be forgiven, but, in fact, we were forging the release you
can't just pass by.

Let's dive in and look for a big fish:
https://blogs.apache.org/ignite/entry/apache-ignite-2-4-brings

The full list of the changes can be found here:
https://ignite.apache.org/releases/2.4.0/release_notes.html

Ready to try then navigate to our downloads page:
https://ignite.apache.org/download.cgi

--
Denis

Reconsider default WAL mode: we need something between LOG_ONLY and FSYNC

[Ivan Rakov]

Igniters and especially Native Persistence experts,

We decided to change default WAL mode from DEFAULT(FSYNC) to LOG_ONLY in 
2.4 release. That was difficult decision: we sacrificed power loss / OS 
crash tolerance, but gained significant performance boost. From my 
perspective, LOG_ONLY is right choice, but it still misses some critical 
features that default mode should have.


Let's focus on exact guarantees each mode provides. Documentation 
explains it in pretty simple manner: LOG_ONLY - writes survive process 
crash, FSYNC - writes survive power loss scenarios. I have to notice 
that documentation doesn't describe what exactly can happen to node in 
LOG_ONLY mode in case of power loss / OS crash scenario. Basically, 
there are two possible negative outcomes: loss of several last updates 
(it's exactly what can happen in BACKGROUND mode in case of process 
crash) and total storage corruption (not only last updates, but all data 
will be lost). I've made a quick research on this and came into 
conclusion that power loss in LOG_ONLY can lead to storage corruption. 
There are several explanations for this:
1) IgniteWriteAheadLogManager#fsync is kind of broken - it doesn't 
perform actual fsync unless current WAL mode is FSYNC. We call this 
method when we write checkpoint marker to WAL. As long as part of WAL 
before checkpoint marker can be not synced, "physical" records that are 
necessary for crash recovery in "Node stopped in the middle of 
checkpoint" scenario may be corrupted after power loss. If that happens, 
we won't be able to recover internal data structures, which means loss 
of all data.
2) We don't fsync WAL archive files unless current WAL mode is FSYNC. 
WAL archive can contain necessary "physical" records as well, which 
leads us to the case described above.
3) We do perform fsync on rollover (switch of current WAL segment) in 
all modes, but only when there's enough space to write switch segment 
record - see FileWriteHandle#close. So there's a little chance that 
we'll skip fsync and bump into the same case.


Enforcing fsync on that three situations will give us a guarantee that 
LOG_ONLY will survive power loss scenarios with possibility of losing 
several last updates. There still can be a total binary mess in the last 
part of WAL, but as long as we perform CRC check during WAL replay, 
we'll detect start of that mess. Extra fsyncs may cause slight 
performance degradation - all writes will have to await for one fsync on 
every rollover and checkpoint. It's still much faster than fsync on 
every write in WAL - I expect a few percent (0-5%) drop comparing to 
current LOG_ONLY. But degradation is degradation, and LOG_ONLY mode 
without extra fsyncs makes sense as well - that's why we need to 
introduce "LOG_ONLY + extra fsyncs" as separate WAL mode. I think, we 
should make it default - it provides significant durability bonus for 
the cost of one extra fsync for each WAL segment written.


To sum it up, I propose a new set of possible WAL modes:
NONE - both process crash and power loss can lead to corruption
BACKGROUND - process crash can lead to last updates loss, power loss can 
lead to corruption

LOG_ONLY - writes survive process crash, power loss can lead to corruption
LOG_ONLY_SAFE (default) - writes survive process crash, power loss can 
lead to last updates loss

FSYNC - writes survive both process crash and power loss

Thoughts?


Best Regards,
Ivan Rakov

Re: IEP-14: Ignite failures handling (Discussion)

[Dmitriy Setrakyan]

On Thu, Mar 15, 2018 at 5:21 AM, Dmitry Pavlov 
wrote:

> Hi Dmitriy,
>
> It seems, here everyone agrees that killing the process will give a more
> guaranteed result. The question is that the majority in the community does
> not consider this to be acceptable in case Ignite as started as embedded
> lib (e.g. from Java, using Ignition.start())
>
> What can help to accept the community's opinion? Let's remember Apache
> principle: "community first".
>

I am still confused about the problem the majority of the community is
trying to solve. If our priority is to keep the cluster in frozen state,
then what is the reason for this task altogether?

The priority should be to keep the cluster operational, not frozen. The
only solution here is "kill" or "stop+kill". If the community does not
accept this option as a default, then I propose to drop this task
altogether, because we do not have to do anything to keep the cluster
frozen.


> If release 2.5 will show us it was inpractical, we will change default to
> kill even for library. What do you think?
>

See above. I do not see a reason to continue with this task if the end
result is identical to what we have today.

I want to give the community another chance to speak up and voice their
opinions again, having fully understood the context and the problem being
solved here.

D.

Re: Partition eviction failed, this can cause grid hang. (Caused by: java.lang.IllegalStateException: Failed to get page IO instance (page content is corrupted))

[Arseny Kovalchuk]

Hi, guys.

I've got a reproducer for a problem which is generally reported as "Caused
by: java.lang.IllegalStateException: Failed to get page IO instance (page
content is corrupted)". Actually it reproduces the result. I don't have an
idea how the data has been corrupted, but the cluster node doesn't want to
start with this data.

We got the issue again when some of server nodes were restarted several
times by kubernetes. I suspect that the data got corrupted during such
restarts. But the main functionality that we really desire to have, that
the cluster DOESN'T HANG during next restart even if the data is corrupted!
Anyway, there is no a tool that can help to correct such data, and as a
result we wipe all data manually to start the cluster. So, having warnings
about corrupted data in logs and just working cluster is the expected
behavior.

How to reproduce:
1. Download the data from here
https://storage.googleapis.com/pub-data-0/data5.tar.gz (~200Mb)
2. Download and import Gradle project
https://storage.googleapis.com/pub-data-0/project.tar.gz (~100Kb)
3. Unpack the data to the home folder, say /home/user1. You should get the
path like */home/user1/data5*. Inside data5 you should have binary_meta,
db, marshaller.
4. Open *src/main/resources/data-test.xml* and put the absolute path of
unpacked data into *workDirectory* property of *igniteCfg5* bean. In this
example it should be */home/user1/data5.* Do not edit consistentId!
The consistentId is ignite-instance-5, so the real data is in
the data5/db/ignite_instance_5 folder
5. Start application from ru.synesis.kipod.DataTestBootApp
6. Enjoy

Hope it will help.


​
Arseny Kovalchuk

Senior Software Engineer at Synesis
skype: arseny.kovalchuk
mobile: +375 (29) 666-16-16
​LinkedIn Profile ​

On 26 December 2017 at 21:15, Denis Magda  wrote:

> Cross-posting to the dev list.
>
> Ignite persistence maintainers please chime in.
>
> —
> Denis
>
> On Dec 26, 2017, at 2:17 AM, Arseny Kovalchuk 
> wrote:
>
> Hi guys.
>
> Another issue when using Ignite 2.3 with native persistence enabled. See
> details below.
>
> We deploy Ignite along with our services in Kubernetes (v 1.8) on
> premises. Ignite cluster is a StatefulSet of 5 Pods (5 instances) of Ignite
> version 2.3. Each Pod mounts PersistentVolume backed by CEPH RBD.
>
> We put about 230 events/second into Ignite, 70% of events are ~200KB in
> size and 30% are 5000KB. Smaller events have indexed fields and we query
> them via SQL.
>
> The cluster is activated from a client node which also streams events into
> Ignite from Kafka. We use custom implementation of streamer which uses
> cache.putAll() API.
>
> We started cluster from scratch without any persistent data. After a while
> we got corrupted data with the error message.
>
> [2017-12-26 07:44:14,251] ERROR [sys-#127%ignite-instance-2%]
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader:
> - Partition eviction failed, this can cause grid hang.
> class org.apache.ignite.IgniteException: Runtime failure on search row:
> Row@5b1479d6[ key: 171:1513946618964:3008806055072854, val:
> ru.synesis.kipod.event.KipodEvent [idHash=510912646, hash=-387621419,
> face_last_name=null, face_list_id=null, channel=171, source=,
> face_similarity=null, license_plate_number=null, descriptors=null,
> cacheName=kipod_events, cacheKey=171:1513946618964:3008806055072854,
> stream=171, alarm=false, processed_at=0, face_id=null, id=3008806055072854,
> persistent=false, face_first_name=null, license_plate_first_name=null,
> face_full_name=null, level=0, module=Kpx.Synesis.Outdoor,
> end_time=1513946624379, params=null, commented_at=0, tags=[vehicle, 0,
> human, 0, truck, 0, start_time=1513946618964, processed=false,
> kafka_offset=111259, license_plate_last_name=null, armed=false,
> license_plate_country=null, topic=MovingObject, comment=,
> expiration=1514033024000, original_id=null, license_plate_lists=null], ver:
> GridCacheVersion [topVer=125430590, order=1513955001926, nodeOrder=3] ][
> 3008806055072854, MovingObject, Kpx.Synesis.Outdoor, 0, , 1513946618964,
> 1513946624379, 171, 171, FALSE, FALSE, , FALSE, FALSE, 0, 0, 111259,
> 1514033024000, (vehicle, 0, human, 0, truck, 0), null, null, null, null,
> null, null, null, null, null, null, null, null ]
> at org.apache.ignite.internal.processors.cache.persistence.tree
> .BPlusTree.doRemove(BPlusTree.java:1787)
> at org.apache.ignite.internal.processors.cache.persistence.tree
> .BPlusTree.remove(BPlusTree.java:1578)
> at org.apache.ignite.internal.processors.query.h2.database.H2Tr
> eeIndex.remove(H2TreeIndex.java:216)
> at org.apache.ignite.internal.processors.query.h2.opt.GridH2Tab
> le.doUpdate(GridH2Table.java:496)
> at org.apache.ignite.internal.processors.query.h2.opt.GridH2Tab
> le.update(GridH2Table.java:423)
> at org.apache.ignite.internal.processors.query.h2.IgniteH2Index
> ing.remove(IgniteH2Indexing.java:580)
> at org.apache.ignite.interna

Re: Data eviction/expiration from Ignite persistence

[Dmitriy Setrakyan]

On Thu, Mar 15, 2018 at 12:31 PM, Denis Magda  wrote:

> >
> > Agree with AG. There is a difference between expiration and eviction. If
> an
> > entry is expired, then it should be removed from the store, regardless if
> > it is in memory or on disk.
>
>
> Well, then it works this way now depending on a memory configuration:
>
>- memory only mode: expired entry removed from memory storage
>- memory + Ignite persistence: expired entry removed from both memory
>and disk tiers
>- memory + 3rd party: expired entry is removed from the memory storage
>only.
>

I think it works correctly. Whenever Ignite is configured with a 3rd party
database, entries are usually expired from memory exactly because the
database may have a more recent value. This way, the more recent value will
be reloaded form database on next access.


> Let me know if I'm wrong somewhere. Otherwise, I'll improve the docs to
> bring more clarity.
>

I believe you are correct.


> > However, evicting from memory because there is not enough space does not
> remove an entry from the store.
>
>
> That was another topic of the discussion. How can we support the *eviction*
> from disk (by using a particular configuration parameter)?
>

I do not think we need to evict from disk. None of the other databases
support it, and I do not think it is a required feature. Also, it will be
hard to implement in my view.

Re: readme.io weird interface element

[Denis Magda]

Please create a ticket and let's fix it in the nearest future.

--
Denis

On Thu, Mar 15, 2018 at 12:47 PM, Prachi Garg  wrote:

> That's the download button. Tables on feature pages are downloadable in
> pdf, csv format. There is a javascript code embedded with the table
> element. Not sure why it doesn't work on the download page; works fine on
> other pages.
>
> On Thu, Mar 15, 2018 at 12:16 PM, Denis Magda  wrote:
>
> > Have no idea, just makes the table looks nicer :)
> >
> > Prachi, what is this for? It's not clear from the HTML sources.
> >
> > --
> > Denis
> >
> > On Thu, Mar 15, 2018 at 12:09 PM, Petr Ivanov 
> wrote:
> >
> >> Sorry for misleading. I meant https://ignite.apache.org/download.cgi
> >> indeed.
> >>
> >>
> >>
> >> > On 15 Mar 2018, at 22:05, Denis Magda  wrote:
> >> >
> >> > Petr,
> >> >
> >> > What's that? It doesn't look like the readme.io doc that hosts all
> >> Ignite
> >> > docs:
> >> > https://apacheignite.readme.io/docs
> >> >
> >> > --
> >> > Denis
> >> >
> >> > On Thu, Mar 15, 2018 at 1:52 AM, vveider  wrote:
> >> >
> >> >> Hi, all!
> >> >>
> >> >>
> >> >> Does anyone know what is this button with arrow and disk at the right
> >> >> corner
> >> >> of versions list header? [1]
> >> >> Seems that it does nothing.
> >> >>
> >> >> [1] https://ibb.co/eJf5uc
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >> >>
> >>
> >>
> >
>

Re: readme.io weird interface element

[Prachi Garg]

That's the download button. Tables on feature pages are downloadable in
pdf, csv format. There is a javascript code embedded with the table
element. Not sure why it doesn't work on the download page; works fine on
other pages.

On Thu, Mar 15, 2018 at 12:16 PM, Denis Magda  wrote:

> Have no idea, just makes the table looks nicer :)
>
> Prachi, what is this for? It's not clear from the HTML sources.
>
> --
> Denis
>
> On Thu, Mar 15, 2018 at 12:09 PM, Petr Ivanov  wrote:
>
>> Sorry for misleading. I meant https://ignite.apache.org/download.cgi
>> indeed.
>>
>>
>>
>> > On 15 Mar 2018, at 22:05, Denis Magda  wrote:
>> >
>> > Petr,
>> >
>> > What's that? It doesn't look like the readme.io doc that hosts all
>> Ignite
>> > docs:
>> > https://apacheignite.readme.io/docs
>> >
>> > --
>> > Denis
>> >
>> > On Thu, Mar 15, 2018 at 1:52 AM, vveider  wrote:
>> >
>> >> Hi, all!
>> >>
>> >>
>> >> Does anyone know what is this button with arrow and disk at the right
>> >> corner
>> >> of versions list header? [1]
>> >> Seems that it does nothing.
>> >>
>> >> [1] https://ibb.co/eJf5uc
>> >>
>> >>
>> >>
>> >> --
>> >> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>> >>
>>
>>
>

Re: Partition loss policy - how to use?

[Denis Magda]

The monitoring screen that should show the partition loss metrics is not in
open source. It's a plugin GridGain built on top of the console foundation.

However, we can use it for free via that deployment even for Ignite
clusters: https://console.gridgain.com/

--
Denis

On Wed, Mar 14, 2018 at 8:29 PM, Gaurav Bajaj 
wrote:

> Alexey,
>
> Thanks. I wonder why not webconsole?
>
> Thanks,
> Gaurav
>
> On 14-Mar-2018 1:28 AM, "Alexey Kuznetsov"  wrote:
>
> >  Gaurav,
> >
> > I think it make sense to add this for tools.
> > Created issue: https://issues.apache.org/jira/browse/IGNITE-7940
> >
> > On Wed, Mar 14, 2018 at 1:44 AM, Gaurav Bajaj 
> > wrote:
> >
> > > Hi Denis,
> > > Thanks. Document certainly looks useful. Do we have ticket for
> > improvement
> > > in Webconsole/Visor for marking resetLostPartitions()?
> > >
> > >
> > > Regards,
> > > Gaurav
> > >
> > > On 13-Mar-2018 7:42 PM, "Denis Magda"  wrote:
> > >
> > > For those interested, here is a doc we put together for the partition
> > > policies which considers extra improvements released in 2.4:
> > > https://apacheignite.readme.io/v2.4/docs/partition-loss-policies
> > >
> > > --
> > > Denis
> > >
> > > On Tue, Mar 6, 2018 at 11:19 AM, Denis Magda 
> wrote:
> > >
> > > > Hi,
> > > >
> > > > Here is documentation we prepared for 2.4 release:
> > https://apacheignite.
> > > > readme.io/v2.3/docs/cache-modes-24#partition-loss-policies
> > > >
> > > > It's hidden for now and will become visible to everyone once Ignite
> 2.4
> > > > vote passes (in progress).
> > > >
> > > > --
> > > > Denis
> > > >
> > > > On Tue, Mar 6, 2018 at 6:59 AM, gauravhb 
> > wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> Is there any update on this topic?
> > > >> Any tickets created for points mentioned by Valentin?
> > > >>
> > > >> Thanks.
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Alexey Kuznetsov
> >
>

Re: Data eviction/expiration from Ignite persistence

[Denis Magda]

>
> Agree with AG. There is a difference between expiration and eviction. If an
> entry is expired, then it should be removed from the store, regardless if
> it is in memory or on disk.


Well, then it works this way now depending on a memory configuration:

   - memory only mode: expired entry removed from memory storage
   - memory + Ignite persistence: expired entry removed from both memory
   and disk tiers
   - memory + 3rd party: expired entry is removed from the memory storage
   only.

Let me know if I'm wrong somewhere. Otherwise, I'll improve the docs to
bring more clarity.

However, evicting from memory because there is not enough space does not
> remove an entry from the store.


That was another topic of the discussion. How can we support the *eviction*
from disk (by using a particular configuration parameter)?

--
Denis

On Wed, Mar 14, 2018 at 7:01 PM, Dmitriy Setrakyan 
wrote:

> On Wed, Mar 14, 2018 at 1:41 PM, Alexey Goncharuk <
> alexey.goncha...@gmail.com> wrote:
>
> > Denis,
> >
> > With the approach of Ignite Durable Memory there is no difference between
> > 'memory' and 'disk'. The data is expired from the Ignite data storage
> which
> > can be persisted or not. Before persistence was introduced, TTL was
> mostly
> > used when write-through was enabled, otherwise data was cleared from
> Ignite
> > data storage. Currently, the situation stays the same - if an entry is
> > expired, it is removed from the Ignite storage, which looks absolutely
> > consistent to me.
> >
>
> Agree with AG. There is a difference between expiration and eviction. If an
> entry is expired, then it should be removed from the store, regardless if
> it is in memory or on disk.
>
> However, evicting from memory because there is not enough space does not
> remove an entry from the store.
>
> D.
>

Re: Timeline for support of compute functions by thin clients

[Denis Magda]

Pavel,

I just don't see a substantial reason why we need to support the
compute APIs.

As you properly mentioned, it's not easy to copy all the APIs and, again,
for what. It's right that the thin client allows decoupling .NET from JVM,
but its implementation won't be more performant than the regular client's
one.

So, personally, a thin client (.NET, Node.JS, Java, Python, etc.) is a
lightweight connection to the cluster that supports classic client-server
request-response operations. If someone needs more (compute, services,
streaming, ML), then go for the regular client which is battle-tested and
available for usage.

--
Denis



On Wed, Mar 14, 2018 at 1:33 PM, Pavel Tupitsyn 
wrote:

> Hi Denis,
>
> > There are no any plans for that level of support
> Why do you think so?
> We already have ScanQuery with filter in .NET Thin Client, which involves
> remote code execution on server nodes.
> It is quite similar to Compute.Broadcast and such.
>
> Thanks,
> Pavel
>
>
> On Wed, Mar 14, 2018 at 11:32 PM, Denis Magda  wrote:
>
> > Raymond,
> >
> > Then I would suggest you keep using the regular .NET client that supports
> > and optimized for computations. Is there any reason why you can't use the
> > regular one?
> >
> > --
> > Denis
> >
> > On Wed, Mar 14, 2018 at 12:53 PM, Raymond Wilson <
> > raymond_wil...@trimble.com
> > > wrote:
> >
> > > Hi Denis,
> > >
> > > We are using Ignite.Net and are planning to use 2.4 + .Net Core + thin
> > > client support to enable lightweight containerisable services that
> > interact
> > > with the main Ignite compute grid.
> > >
> > > These work flows are less about Get/Put style semantics, and more about
> > > using grid compute.
> > >
> > > Eg: Here's an example where a client context asks a remote context to
> > > render
> > > a bitmap tile in an ICompute:
> > >
> > > public Bitmap Execute(TileRenderRequestArgument arg)
> > > {
> > > IComputeFunc func = new
> > > TileRenderRequestComputeFunc();
> > >
> > > return
> > > _ignite.GetCluster().ForRemotes().GetCompute().Apply(func, arg);
> > > }
> > >
> > > In this example, the calling context here could be a lightweight
> Kestrel
> > > web
> > > service end point delegating rendering to a remote service.
> > >
> > > Thanks,
> > > Raymond.
> > >
> > > -Original Message-
> > > From: Denis Magda [mailto:dma...@apache.org]
> > > Sent: Thursday, March 15, 2018 8:31 AM
> > > To: dev@ignite.apache.org
> > > Subject: Re: Timeline for support of compute functions by thin clients
> > >
> > > Hi Raymond,
> > >
> > > There are no any plans for that level of support. The thin clients are
> > > targeted for classic client-server processing use cases when a client
> > > request data from a server, does something with it locally and
> > potentially
> > > writes changes back to the server. ICache, SQL fall under this
> category.
> > >
> > > Are you intended to use .NET thin client or anyone else?
> > >
> > > --
> > > Denis
> > >
> > > On Wed, Mar 14, 2018 at 12:25 PM, Raymond Wilson <
> > > raymond_wil...@trimble.com
> > > > wrote:
> > >
> > > > Hi,
> > > >
> > > >
> > > >
> > > > The thin client implementation in Ignite 2.4 only covers a subset of
> > > > the ICache interface.
> > > >
> > > >
> > > >
> > > > When will we see thin client support for compute, messaging etc?
> > > >
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Raymond.
> > > >
> > >
> >
>

Re: readme.io weird interface element

[Denis Magda]

Have no idea, just makes the table looks nicer :)

Prachi, what is this for? It's not clear from the HTML sources.

--
Denis

On Thu, Mar 15, 2018 at 12:09 PM, Petr Ivanov  wrote:

> Sorry for misleading. I meant https://ignite.apache.org/download.cgi
> indeed.
>
>
>
> > On 15 Mar 2018, at 22:05, Denis Magda  wrote:
> >
> > Petr,
> >
> > What's that? It doesn't look like the readme.io doc that hosts all
> Ignite
> > docs:
> > https://apacheignite.readme.io/docs
> >
> > --
> > Denis
> >
> > On Thu, Mar 15, 2018 at 1:52 AM, vveider  wrote:
> >
> >> Hi, all!
> >>
> >>
> >> Does anyone know what is this button with arrow and disk at the right
> >> corner
> >> of versions list header? [1]
> >> Seems that it does nothing.
> >>
> >> [1] https://ibb.co/eJf5uc
> >>
> >>
> >>
> >> --
> >> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >>
>
>

Re: Apache Ignite RPM packaging (Stage II)

[Petr Ivanov]

I suppose that most everything if not all from libs/options will go to OPTIONAL 
(I’d call it simply ‘apache-ignite-libs').
More precise lib selection (if something from optional would better to have in 
core package) will be discussed right after preliminary split architecture 
agreement.



> On 15 Mar 2018, at 22:11, Dmitry Pavlov  wrote:
> 
> I like idea of keeping simple system of modules, so +1 from me.
> 
> Where optional libs (e.g Direct IO plugin) would be included, would it be
> core or optional?
> 
> чт, 15 мар. 2018 г. в 22:09, Denis Magda :
> 
>>> 
 
 How big would be a final core module?
>>> Around 30M. Can be shrinked to ~15M if separate Visor and create it’s own
>>> package.
>> 
>> 
>> Guys, 30 vs 280M is a hge difference.  I would agree with Petr and
>> propose the simplest modular system:
>> 
>>   - core module that includes basic Ignite capabilities including SQL,
>>   compute grid, service grid, k/v
>>   - optional module hosts the rest - ML, streamers integration (kafka,
>>   flink), kubernetes, etc.
>> 
>> What do you think?
>> 
>> --
>> Denis
>> 
>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov  wrote:
>> 
>>> *DEB package
>>> 
>>> 
 On 15 Mar 2018, at 10:35, Petr Ivanov  wrote:
 
 Considering that DEV package for now is almost platform independent
>> (its
>>> a java application more or less), that package will work almost on any
>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc.
 The only restriction is existence of systemctl (systemd) service
>> manager
>>> — we are dependent on it.
 
 Thats why, for instance, our RPM repository is called simply ‘rpm’ and
>>> package has no arch or dist suffix — it will work on CentOS, RHEL,
>> Fedora,
>>> etc. with presence of aforementioned systemd.
 
 
 
> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan 
>>> wrote:
> 
> Will Debian package work for Ubuntu?
> 
> D.
> 
> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov 
>>> wrote:
> 
>> Not a problem, rather nuisance. Also, when we will move to official
>> repositories, there can be a problem from OS community.
>> 
>> Concerning DEB packages — I plan to use RPM as base for DEB package
>>> build
>> (package layout / install scripts) for speeding up things and
>> excluding
>> possible duplication and desynchronisation, so its a matter of ’sit
>>> and do’
>> rather then some technical research. Thats why I rose discussion
>> about
>> future package architecture, so that after agreement I'm be able to
>>> pack
>> both RPM and DEB identically.
>> 
>> Yet, if you insist, I can create DEB package according to current RPM
>> layout in no time.
>> 
>> 
>> 
>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan 
>> wrote:
>>> 
>>> Peter,
>>> 
>>> I don't think the package size of 280M is going to be a problem at
>>> all,
>> but
>>> what you are suggesting can be an improvement down the road.
>>> 
>>> In the mean time, I think our top priority should be to provide
>>> packages
>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
>>> 
>>> Agree?
>>> 
>>> D.
>>> 
>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider 
>> wrote:
>>> 
 Hi, Igniters!
 
 
 Release 2.4 is almost there, at least binary part of it, so I'd
>> like
>>> to
 move
 forward to further improve and widen AI delivery through packages.
 As of now, Apache Ignite ships in RPM package weighing about 280M+
>>> and,
>> to
 improve usability and significantly reduce required download
>> sizes, I
 purpose that in 2.5 release we introduce splitted delivery as
>>> follows:
 - CORE
 - bin
 - config
 - libs (!optional)
 - OPTIONAL LIBS
 - BENCHMARKS
 - DOCS (?)
 - EXAMPLES
 - .NET PLATFORM FILES
 - C++ PLATFORM FILES
 
 This architecture, as I assume, will add flexibility (no reason to
>> download
 all 280M+ of binaries where you are to run only core node
>>> functionality)
 and
 maintainability (you are in full control of what is installed on
>> your
 system).
 
 After successful architecture choice, same scheme are planned to be
>> used in
 DEB packages as well.
 
 WDYT?
 
 
 
 --
 Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
 
>> 
>> 
 
>>> 
>>> 
>> 

Re: Apache Ignite RPM packaging (Stage II)

[Dmitry Pavlov]

I like idea of keeping simple system of modules, so +1 from me.

Where optional libs (e.g Direct IO plugin) would be included, would it be
core or optional?

чт, 15 мар. 2018 г. в 22:09, Denis Magda :

> >
> > >
> > > How big would be a final core module?
> > Around 30M. Can be shrinked to ~15M if separate Visor and create it’s own
> > package.
>
>
> Guys, 30 vs 280M is a hge difference.  I would agree with Petr and
> propose the simplest modular system:
>
>- core module that includes basic Ignite capabilities including SQL,
>compute grid, service grid, k/v
>- optional module hosts the rest - ML, streamers integration (kafka,
>flink), kubernetes, etc.
>
> What do you think?
>
> --
> Denis
>
> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov  wrote:
>
> > *DEB package
> >
> >
> > > On 15 Mar 2018, at 10:35, Petr Ivanov  wrote:
> > >
> > > Considering that DEV package for now is almost platform independent
> (its
> > a java application more or less), that package will work almost on any
> > DEB-based linux, including but not limited to Ubuntu, Debian, etc.
> > > The only restriction is existence of systemctl (systemd) service
> manager
> > — we are dependent on it.
> > >
> > > Thats why, for instance, our RPM repository is called simply ‘rpm’ and
> > package has no arch or dist suffix — it will work on CentOS, RHEL,
> Fedora,
> > etc. with presence of aforementioned systemd.
> > >
> > >
> > >
> > >> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan 
> > wrote:
> > >>
> > >> Will Debian package work for Ubuntu?
> > >>
> > >> D.
> > >>
> > >> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov 
> > wrote:
> > >>
> > >>> Not a problem, rather nuisance. Also, when we will move to official
> > >>> repositories, there can be a problem from OS community.
> > >>>
> > >>> Concerning DEB packages — I plan to use RPM as base for DEB package
> > build
> > >>> (package layout / install scripts) for speeding up things and
> excluding
> > >>> possible duplication and desynchronisation, so its a matter of ’sit
> > and do’
> > >>> rather then some technical research. Thats why I rose discussion
> about
> > >>> future package architecture, so that after agreement I'm be able to
> > pack
> > >>> both RPM and DEB identically.
> > >>>
> > >>> Yet, if you insist, I can create DEB package according to current RPM
> > >>> layout in no time.
> > >>>
> > >>>
> > >>>
> >  On 15 Mar 2018, at 04:53, Dmitriy Setrakyan 
> > >>> wrote:
> > 
> >  Peter,
> > 
> >  I don't think the package size of 280M is going to be a problem at
> > all,
> > >>> but
> >  what you are suggesting can be an improvement down the road.
> > 
> >  In the mean time, I think our top priority should be to provide
> > packages
> >  for Debian and Ubuntu. Having only RPMs is not nearly enough.
> > 
> >  Agree?
> > 
> >  D.
> > 
> >  On Wed, Mar 14, 2018 at 5:36 AM, vveider 
> wrote:
> > 
> > > Hi, Igniters!
> > >
> > >
> > > Release 2.4 is almost there, at least binary part of it, so I'd
> like
> > to
> > > move
> > > forward to further improve and widen AI delivery through packages.
> > > As of now, Apache Ignite ships in RPM package weighing about 280M+
> > and,
> > >>> to
> > > improve usability and significantly reduce required download
> sizes, I
> > > purpose that in 2.5 release we introduce splitted delivery as
> > follows:
> > > - CORE
> > > - bin
> > > - config
> > > - libs (!optional)
> > > - OPTIONAL LIBS
> > > - BENCHMARKS
> > > - DOCS (?)
> > > - EXAMPLES
> > > - .NET PLATFORM FILES
> > > - C++ PLATFORM FILES
> > >
> > > This architecture, as I assume, will add flexibility (no reason to
> > >>> download
> > > all 280M+ of binaries where you are to run only core node
> > functionality)
> > > and
> > > maintainability (you are in full control of what is installed on
> your
> > > system).
> > >
> > > After successful architecture choice, same scheme are planned to be
> > >>> used in
> > > DEB packages as well.
> > >
> > > WDYT?
> > >
> > >
> > >
> > > --
> > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> > >
> > >>>
> > >>>
> > >
> >
> >
>

Re: readme.io weird interface element

[Petr Ivanov]

Sorry for misleading. I meant https://ignite.apache.org/download.cgi indeed.



> On 15 Mar 2018, at 22:05, Denis Magda  wrote:
> 
> Petr,
> 
> What's that? It doesn't look like the readme.io doc that hosts all Ignite
> docs:
> https://apacheignite.readme.io/docs
> 
> --
> Denis
> 
> On Thu, Mar 15, 2018 at 1:52 AM, vveider  wrote:
> 
>> Hi, all!
>> 
>> 
>> Does anyone know what is this button with arrow and disk at the right
>> corner
>> of versions list header? [1]
>> Seems that it does nothing.
>> 
>> [1] https://ibb.co/eJf5uc
>> 
>> 
>> 
>> --
>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>> 

Re: Apache Ignite RPM packaging (Stage II)

[Denis Magda]

>
> >
> > How big would be a final core module?
> Around 30M. Can be shrinked to ~15M if separate Visor and create it’s own
> package.


Guys, 30 vs 280M is a hge difference.  I would agree with Petr and
propose the simplest modular system:

   - core module that includes basic Ignite capabilities including SQL,
   compute grid, service grid, k/v
   - optional module hosts the rest - ML, streamers integration (kafka,
   flink), kubernetes, etc.

What do you think?

--
Denis

On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov  wrote:

> *DEB package
>
>
> > On 15 Mar 2018, at 10:35, Petr Ivanov  wrote:
> >
> > Considering that DEV package for now is almost platform independent (its
> a java application more or less), that package will work almost on any
> DEB-based linux, including but not limited to Ubuntu, Debian, etc.
> > The only restriction is existence of systemctl (systemd) service manager
> — we are dependent on it.
> >
> > Thats why, for instance, our RPM repository is called simply ‘rpm’ and
> package has no arch or dist suffix — it will work on CentOS, RHEL, Fedora,
> etc. with presence of aforementioned systemd.
> >
> >
> >
> >> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan 
> wrote:
> >>
> >> Will Debian package work for Ubuntu?
> >>
> >> D.
> >>
> >> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov 
> wrote:
> >>
> >>> Not a problem, rather nuisance. Also, when we will move to official
> >>> repositories, there can be a problem from OS community.
> >>>
> >>> Concerning DEB packages — I plan to use RPM as base for DEB package
> build
> >>> (package layout / install scripts) for speeding up things and excluding
> >>> possible duplication and desynchronisation, so its a matter of ’sit
> and do’
> >>> rather then some technical research. Thats why I rose discussion about
> >>> future package architecture, so that after agreement I'm be able to
> pack
> >>> both RPM and DEB identically.
> >>>
> >>> Yet, if you insist, I can create DEB package according to current RPM
> >>> layout in no time.
> >>>
> >>>
> >>>
>  On 15 Mar 2018, at 04:53, Dmitriy Setrakyan 
> >>> wrote:
> 
>  Peter,
> 
>  I don't think the package size of 280M is going to be a problem at
> all,
> >>> but
>  what you are suggesting can be an improvement down the road.
> 
>  In the mean time, I think our top priority should be to provide
> packages
>  for Debian and Ubuntu. Having only RPMs is not nearly enough.
> 
>  Agree?
> 
>  D.
> 
>  On Wed, Mar 14, 2018 at 5:36 AM, vveider  wrote:
> 
> > Hi, Igniters!
> >
> >
> > Release 2.4 is almost there, at least binary part of it, so I'd like
> to
> > move
> > forward to further improve and widen AI delivery through packages.
> > As of now, Apache Ignite ships in RPM package weighing about 280M+
> and,
> >>> to
> > improve usability and significantly reduce required download sizes, I
> > purpose that in 2.5 release we introduce splitted delivery as
> follows:
> > - CORE
> > - bin
> > - config
> > - libs (!optional)
> > - OPTIONAL LIBS
> > - BENCHMARKS
> > - DOCS (?)
> > - EXAMPLES
> > - .NET PLATFORM FILES
> > - C++ PLATFORM FILES
> >
> > This architecture, as I assume, will add flexibility (no reason to
> >>> download
> > all 280M+ of binaries where you are to run only core node
> functionality)
> > and
> > maintainability (you are in full control of what is installed on your
> > system).
> >
> > After successful architecture choice, same scheme are planned to be
> >>> used in
> > DEB packages as well.
> >
> > WDYT?
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >
> >>>
> >>>
> >
>
>

Re: readme.io weird interface element

[Denis Magda]

Petr,

What's that? It doesn't look like the readme.io doc that hosts all Ignite
docs:
https://apacheignite.readme.io/docs

--
Denis

On Thu, Mar 15, 2018 at 1:52 AM, vveider  wrote:

> Hi, all!
>
>
> Does anyone know what is this button with arrow and disk at the right
> corner
> of versions list header? [1]
> Seems that it does nothing.
>
> [1] https://ibb.co/eJf5uc
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

Re: TeamCity test names refactoring

[Petr Ivanov]

I think I agree about digits. Will fix.


> On 15 Mar 2018, at 20:09, Dmitry Pavlov  wrote:
> 
> In using digits in suite name, there is nothing to do with
> short-sightedness.
> 
> I think this is a matter of taste, so I would like to hear the opinion of
> all.
> 
> чт, 15 мар. 2018 г. в 20:02, Petr Ivanov :
> 
>> I understand conservatism and inertia of such changes.
>> But thus we are sticking to arrogance and shortsightedness of previous
>> engineers who worked on this project, while trying to make it better.
>> My point is - without some legacy cleaning and approaches review it
>> becomes impossible to introduce any meaningful improvements to the project
>> we are so hardly working on.
>> 
>> 
>> 
>>> On 15 Mar 2018, at 19:45, Dmitry Pavlov  wrote:
>>> 
>>> I think it is better to keep existing naming because Igniters are used to
>>> it.
>>> 
>>> Moreover a lot of issues use suite name as reference on where test is
>>> located. So I prefer existing naming.
>>> 
>>> чт, 15 мар. 2018 г. в 19:43, vveider :
>>> 
 Hi, Igniters!
 
 
 Took the liberty to introduce minor refactoring into test build names in
 our
 (now) main test project Ignite Tests 2.4+ (Java 8) [1]
 Changes affected the numeration of tests with common name, i.e
   Ignite Basic => Ignite Basic [I]
   Ignite Basic 2 => Ignite Basic [II]
 And so on.
 My main motivation is common design for the test project, including
>> naming
 policy. It will help maintain this project in future with much less
 efforts.
 
 I earnestly encourage community to be a little bit patient while works
>> on
 test improvements are held and counting on your support.
 
 As always - open to questions and constructive criticism.
 
 
 [1]
>> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8
 
 
 
 --
 Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
 
>> 
>> 

Re: Ignite contibutors page

[Denis Magda]

Dmitriy, Alexey,

Done, now your names are engraved on the page ;)

--
Denis

On Thu, Mar 15, 2018 at 2:34 AM, Dmitriy Shabalin 
wrote:

> I'm not mentioned on the community contributors page:
> https://ignite.apache.org/community/resources.html
>
> pls add me, dmitriyff (Dmitriy Shabalin)
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

Re: [RESULT] [VOTE] Apache Ignite 2.4.0 Release (RC1)

[Denis Magda]

Looks like we're good to go with the announce. I'll blast it out today.

--
Denis

On Thu, Mar 15, 2018 at 11:19 AM, Pavel Tupitsyn 
wrote:

> Igniters, is there still something blocking the release announcement?
>
> On Thu, Mar 15, 2018 at 7:00 AM, Petr Ivanov  wrote:
>
> > I am sure about AWS and GC images now — there is only script inside that
> > have IGNITE_VERSION on input and runs apacheignite/ignite: VERSION>
> > docker image inside the container.
> > No changes except date are not required except that, maybe, we are to
> > update docs at readme.io  for more latest version of
> > Apache Ignite as example.
> >
> >
> > > On 14 Mar 2018, at 20:16, Denis Magda  wrote:
> > >
> > > Petr,
> > >
> > > Fixed the page. As for the cloud images, please give a call to Nikolay
> to
> > > sort things out. It blocks us from the release announcement.
> > >
> > > --
> > > Denis
> > >
> > > On Tue, Mar 13, 2018 at 11:43 PM, Petr Ivanov 
> > wrote:
> > >
> > >> 1. RPM Package Installation
> > >>* missing ‘[ignite]’ header;
> > >>* rest of text is indented by 1 space, while should not.
> > >> 2. Building the Binaries
> > >>* I guess mentioning Java 7 in 2.4.0 release is obsolete — that
> block
> > >> should be either split (which AI requires which JDK for compiling) or
> > just
> > >> get rid of Java 7 mentioning.
> > >> 3. Docker and Cloud Images
> > >>* still not clear what to do with AWS / Google Cloud — how to
> update
> > >> those images. Maybe Nikolay Tikhonov can help?
> > >>
> > >>
> > >>
> > >>> On 13 Mar 2018, at 22:27, Denis Magda  wrote:
> > >>>
> > >>> Then tell me what to correct and I'll merge the changes.
> > >>>
> > >>> Alternatively, you can send me an SVN patch with changes if it's
> > simples:
> > >>> https://cwiki.apache.org/confluence/display/IGNITE/
> Website+Development
> > >>>
> > >>> --
> > >>> Denis
> > >>>
> > >>> On Tue, Mar 13, 2018 at 12:25 PM, Petr Ivanov 
> > >> wrote:
> > >>>
> >  I have no access to site and readme.io  has no
> > >> errors.
> > 
> > 
> > > On 13 Mar 2018, at 22:23, Denis Magda  wrote:
> > >
> > > Petr,
> > >
> > > Have you corrected the site and readme.io docs (if it was
> required)?
> > >
> > > --
> > > Denis
> > >
> > > On Mon, Mar 12, 2018 at 11:14 PM, Petr Ivanov  >
> >  wrote:
> > >
> > >> There is an error in RPM documentation on site — missing header
> for
> > >> ignite.repo file and formatting is also wrong.
> > >> Otherwise — I’m able to install and run Apache Ignite from RPM
> from
> > >> ASF
> > >> according to manual.
> > >>
> > >>
> > >>
> > >>> On 12 Mar 2018, at 23:25, Denis Magda  wrote:
> > >>>
> > >>> Released RPM installation section on the downloads page:
> > >>> https://ignite.apache.org/download.cgi#rpm-package
> > >>>
> > >>> *Petr*, please double check that the section and the readme
> getting
> > >> started
> > >>> look good:
> > >>> https://apacheignite.readme.io/docs/getting-started#
> >  section-rpm-package-
> > >> installation
> > >>>
> > >>> *Mauricio*, could you please create a patch that points out
> > "latest"
> >  docs
> > >>> to "2.4.0"?
> > >>>
> > >>> --
> > >>> Denis
> > >>>
> > >>>
> > >>>
> > >>> On Mon, Mar 12, 2018 at 7:36 AM, Pavel Tupitsyn <
> > >> ptupit...@apache.org>
> > >>> wrote:
> > >>>
> >  NuGet packages uploaded: https://www.nuget.org/
> >  packages?q=Apache.Ignite
> > 
> >  Thanks,
> >  Pavel
> > 
> > 
> >  On Mon, Mar 12, 2018 at 1:54 PM, Vladimir Ozerov <
> >  voze...@gridgain.com>
> >  wrote:
> > 
> > > Correct, it is a matter of time. Download links already work on
> > my
> >  side
> > > [1].
> > >
> > > [1] http://mirror.linux-ia64.org/apache/ignite/2.4.0/
> > >
> > > On Mon, Mar 12, 2018 at 1:29 PM, Petr Ivanov <
> > mr.wei...@gmail.com>
> >  wrote:
> > >
> > >> Link to 2.4.0 is pointing to RBC mirror, not Apache Dist
> itself.
> > >>
> > >> I do not know how often RBC updates its mirror, but I think we
> > >> have
> >  to
> > >> devise better solution for pointing to fresh release
> artifacts.
> > >>
> > >>
> > >>
> > >>> On 12 Mar 2018, at 13:24, Pavel Tupitsyn <
> ptupit...@apache.org
> > >
> >  wrote:
> > >>>
> > >>> Hi Vladimir,
> > >>>
> > >>> I see the link for 2.4.0 on https://ignite.apache.org/
> > >> download.cgi#binaries,
> > >>> but it does not work.
> > >>>
> > >>> On Mon, Mar 12, 2018 at 10:55 AM, Vladimir Ozerov <
> > > voze...@gridgain.com>
> > >>> wrote:
> > >>>
> >  Correction: " Apache Ignite *2.4.0* release (RC1) has been
> >  acc

Re: [RESULT] [VOTE] Apache Ignite 2.4.0 Release (RC1)

[Pavel Tupitsyn]

Igniters, is there still something blocking the release announcement?

On Thu, Mar 15, 2018 at 7:00 AM, Petr Ivanov  wrote:

> I am sure about AWS and GC images now — there is only script inside that
> have IGNITE_VERSION on input and runs apacheignite/ignite:
> docker image inside the container.
> No changes except date are not required except that, maybe, we are to
> update docs at readme.io  for more latest version of
> Apache Ignite as example.
>
>
> > On 14 Mar 2018, at 20:16, Denis Magda  wrote:
> >
> > Petr,
> >
> > Fixed the page. As for the cloud images, please give a call to Nikolay to
> > sort things out. It blocks us from the release announcement.
> >
> > --
> > Denis
> >
> > On Tue, Mar 13, 2018 at 11:43 PM, Petr Ivanov 
> wrote:
> >
> >> 1. RPM Package Installation
> >>* missing ‘[ignite]’ header;
> >>* rest of text is indented by 1 space, while should not.
> >> 2. Building the Binaries
> >>* I guess mentioning Java 7 in 2.4.0 release is obsolete — that block
> >> should be either split (which AI requires which JDK for compiling) or
> just
> >> get rid of Java 7 mentioning.
> >> 3. Docker and Cloud Images
> >>* still not clear what to do with AWS / Google Cloud — how to update
> >> those images. Maybe Nikolay Tikhonov can help?
> >>
> >>
> >>
> >>> On 13 Mar 2018, at 22:27, Denis Magda  wrote:
> >>>
> >>> Then tell me what to correct and I'll merge the changes.
> >>>
> >>> Alternatively, you can send me an SVN patch with changes if it's
> simples:
> >>> https://cwiki.apache.org/confluence/display/IGNITE/Website+Development
> >>>
> >>> --
> >>> Denis
> >>>
> >>> On Tue, Mar 13, 2018 at 12:25 PM, Petr Ivanov 
> >> wrote:
> >>>
>  I have no access to site and readme.io  has no
> >> errors.
> 
> 
> > On 13 Mar 2018, at 22:23, Denis Magda  wrote:
> >
> > Petr,
> >
> > Have you corrected the site and readme.io docs (if it was required)?
> >
> > --
> > Denis
> >
> > On Mon, Mar 12, 2018 at 11:14 PM, Petr Ivanov 
>  wrote:
> >
> >> There is an error in RPM documentation on site — missing header for
> >> ignite.repo file and formatting is also wrong.
> >> Otherwise — I’m able to install and run Apache Ignite from RPM from
> >> ASF
> >> according to manual.
> >>
> >>
> >>
> >>> On 12 Mar 2018, at 23:25, Denis Magda  wrote:
> >>>
> >>> Released RPM installation section on the downloads page:
> >>> https://ignite.apache.org/download.cgi#rpm-package
> >>>
> >>> *Petr*, please double check that the section and the readme getting
> >> started
> >>> look good:
> >>> https://apacheignite.readme.io/docs/getting-started#
>  section-rpm-package-
> >> installation
> >>>
> >>> *Mauricio*, could you please create a patch that points out
> "latest"
>  docs
> >>> to "2.4.0"?
> >>>
> >>> --
> >>> Denis
> >>>
> >>>
> >>>
> >>> On Mon, Mar 12, 2018 at 7:36 AM, Pavel Tupitsyn <
> >> ptupit...@apache.org>
> >>> wrote:
> >>>
>  NuGet packages uploaded: https://www.nuget.org/
>  packages?q=Apache.Ignite
> 
>  Thanks,
>  Pavel
> 
> 
>  On Mon, Mar 12, 2018 at 1:54 PM, Vladimir Ozerov <
>  voze...@gridgain.com>
>  wrote:
> 
> > Correct, it is a matter of time. Download links already work on
> my
>  side
> > [1].
> >
> > [1] http://mirror.linux-ia64.org/apache/ignite/2.4.0/
> >
> > On Mon, Mar 12, 2018 at 1:29 PM, Petr Ivanov <
> mr.wei...@gmail.com>
>  wrote:
> >
> >> Link to 2.4.0 is pointing to RBC mirror, not Apache Dist itself.
> >>
> >> I do not know how often RBC updates its mirror, but I think we
> >> have
>  to
> >> devise better solution for pointing to fresh release artifacts.
> >>
> >>
> >>
> >>> On 12 Mar 2018, at 13:24, Pavel Tupitsyn  >
>  wrote:
> >>>
> >>> Hi Vladimir,
> >>>
> >>> I see the link for 2.4.0 on https://ignite.apache.org/
> >> download.cgi#binaries,
> >>> but it does not work.
> >>>
> >>> On Mon, Mar 12, 2018 at 10:55 AM, Vladimir Ozerov <
> > voze...@gridgain.com>
> >>> wrote:
> >>>
>  Correction: " Apache Ignite *2.4.0* release (RC1) has been
>  accepted."
> 
>  On Mon, Mar 12, 2018 at 10:51 AM, Vladimir Ozerov <
> > voze...@gridgain.com
> >>>
>  wrote:
> 
> > Igniters,
> >
> > Apache Ignite 2.3.0 release (RC1) has been accepted.
> >
> > 5 "+1" binding votes received:
> > - Alexey Goncharuk
> > - Alexey Kuznetsov
> > - Anton Vinogradov
> > - Denis Magda
> 

Re: Partition recovery issue on partition loss.

[Denis Magda]

I dared to set fix version to 2.5 and increased the severity. It's
important to fix the race since we've just released the partition loss
functionality in 2.4 and it's already broken.

Andrey, please keep us posted. If you didn't fix it, we would need to find
another contributor.

--
Denis

On Thu, Mar 15, 2018 at 7:29 AM, Dmitry Pavlov 
wrote:

> Hi Andrew Mashenkov,
>
> would you like to pick up issue?
>
> Sincerely,
> Dmitriy Pavlov
>
> чт, 15 мар. 2018 г. в 6:23, Dmitriy Setrakyan :
>
> > Completely agree, we must fix this. I like the proposed design. We should
> > also specify that resetLostPartitions() method should return true and
> > false.
> >
> > Val, do you mind updating the ticket with new design?
> > https://issues.apache.org/jira/browse/IGNITE-7832
> >
> > D.
> >
> > On Tue, Mar 13, 2018 at 5:31 PM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> > > This indeed looks like a bigger issue. Basically, there is no clear way
> > (or
> > > no way at all) to synchronize code that listens to partition loss
> event,
> > > and the code that calls resetLostPartitions() method. Example scenario:
> > >
> > > 1. Cache is configured with 3rd party persistence.
> > > 2. One or more nodes fail causing loss of several partitions in memory.
> > > 3. Ignite blocks access to those partitions according to partition loss
> > > policy and fires an event.
> > > 4. Application listens to the event and starts reloading the data from
> > > store.
> > > 5. When reloading is complete, application calls resetLostPartitions()
> to
> > > restore access.
> > > 6. Nodes fail again causing another partition loss, new event is fired.
> > >
> > > There is race between steps 5 and 6. If 2nd failure happens BEFORE
> > > resetLostPartitions() is called, we end up with inconsistent data.
> > >
> > > I believe the only way to fix this is to add corresponding topology
> > version
> > > to partition loss event, and also add it as a parameter for
> > > resetLostPartitions().
> > > This way if resetLostPartitions() is invoked with a version that is not
> > the
> > > latest anymore, the invocation will be ignored.
> > >
> > > The only problem with this approach  is that topology version itself is
> > > currently not a part of public API. It needs to be properly exposed
> there
> > > first.
> > >
> > > -Val
> > >
> > > On Mon, Mar 12, 2018 at 1:07 PM, Denis Magda 
> wrote:
> > >
> > > > Just in case here is you can find the present documentation:
> > > >
> > https://apacheignite.readme.io/docs/cache-modes#partition-loss-policies
> > > >
> > > > Let us know what needs to be updated once the issues reported by you
> > are
> > > > addressed.
> > > >
> > > > --
> > > > Denis
> > > >
> > > > On Mon, Mar 12, 2018 at 3:33 AM, Andrey Mashenkov <
> > > > andrey.mashen...@gmail.com> wrote:
> > > >
> > > > > Hi Igniters,
> > > > >
> > > > > I've found we no documentation how user can recover cache from
> > > cacheStore
> > > > > in case of partition loss.
> > > > > Ignite provides some instruments (methods and events) that should
> > help
> > > > user
> > > > > to solve this problem,
> > > > > but looks like these instruments have an architecture lack.
> > > > >
> > > > > The first one is an usability issue. Ignite provides partition loss
> > > event
> > > > > to user can handle this, but Ignite fires an event per partition.
> > > > > Why we can't have an event with list of lost partitions?
> > > > >
> > > > > The second one is a bug. Ignite.resetLostPartitions() method
> doesn't
> > > care
> > > > > about what topology version recovered partitions belonged to.
> > > > > Tthere is a race, when user call this method after a node was
> failed,
> > > but
> > > > > right before Ignite fire an event.
> > > > > So, it is possible state of just lost partitions will be reseted
> > > > > unexpectedly.
> > > > >
> > > > >
> > > > > I've created a ticket for this [1] and think we should rethink the
> > > > > architecture of the partition recovery mechanics and improve
> > > > documentation.
> > > > > Any thoughts?
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-7832
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Andrey V. Mashenkov
> > > > >
> > > >
> > >
> >
>

[GitHub] ignite pull request #3622: IGNITE-7770 For TC tests only

[andrey-kuznetsov]

Github user andrey-kuznetsov closed the pull request at:

https://github.com/apache/ignite/pull/3622


---

Re: TeamCity test names refactoring

[Dmitry Pavlov]

In using digits in suite name, there is nothing to do with
short-sightedness.

I think this is a matter of taste, so I would like to hear the opinion of
all.

чт, 15 мар. 2018 г. в 20:02, Petr Ivanov :

> I understand conservatism and inertia of such changes.
> But thus we are sticking to arrogance and shortsightedness of previous
> engineers who worked on this project, while trying to make it better.
> My point is - without some legacy cleaning and approaches review it
> becomes impossible to introduce any meaningful improvements to the project
> we are so hardly working on.
>
>
>
> > On 15 Mar 2018, at 19:45, Dmitry Pavlov  wrote:
> >
> > I think it is better to keep existing naming because Igniters are used to
> > it.
> >
> > Moreover a lot of issues use suite name as reference on where test is
> > located. So I prefer existing naming.
> >
> > чт, 15 мар. 2018 г. в 19:43, vveider :
> >
> >> Hi, Igniters!
> >>
> >>
> >> Took the liberty to introduce minor refactoring into test build names in
> >> our
> >> (now) main test project Ignite Tests 2.4+ (Java 8) [1]
> >> Changes affected the numeration of tests with common name, i.e
> >>Ignite Basic => Ignite Basic [I]
> >>Ignite Basic 2 => Ignite Basic [II]
> >> And so on.
> >> My main motivation is common design for the test project, including
> naming
> >> policy. It will help maintain this project in future with much less
> >> efforts.
> >>
> >> I earnestly encourage community to be a little bit patient while works
> on
> >> test improvements are held and counting on your support.
> >>
> >> As always - open to questions and constructive criticism.
> >>
> >>
> >> [1]
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8
> >>
> >>
> >>
> >> --
> >> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >>
>
>

[GitHub] ignite pull request #3575: OSGI tests fail

[daradurvs]

Github user daradurvs closed the pull request at:

https://github.com/apache/ignite/pull/3575


---

Re: TeamCity test names refactoring

[Petr Ivanov]

I understand conservatism and inertia of such changes.
But thus we are sticking to arrogance and shortsightedness of previous 
engineers who worked on this project, while trying to make it better.
My point is - without some legacy cleaning and approaches review it becomes 
impossible to introduce any meaningful improvements to the project we are so 
hardly working on.



> On 15 Mar 2018, at 19:45, Dmitry Pavlov  wrote:
> 
> I think it is better to keep existing naming because Igniters are used to
> it.
> 
> Moreover a lot of issues use suite name as reference on where test is
> located. So I prefer existing naming.
> 
> чт, 15 мар. 2018 г. в 19:43, vveider :
> 
>> Hi, Igniters!
>> 
>> 
>> Took the liberty to introduce minor refactoring into test build names in
>> our
>> (now) main test project Ignite Tests 2.4+ (Java 8) [1]
>> Changes affected the numeration of tests with common name, i.e
>>Ignite Basic => Ignite Basic [I]
>>Ignite Basic 2 => Ignite Basic [II]
>> And so on.
>> My main motivation is common design for the test project, including naming
>> policy. It will help maintain this project in future with much less
>> efforts.
>> 
>> I earnestly encourage community to be a little bit patient while works on
>> test improvements are held and counting on your support.
>> 
>> As always - open to questions and constructive criticism.
>> 
>> 
>> [1] https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8
>> 
>> 
>> 
>> --
>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>> 

[GitHub] ignite pull request #3643: IGNITE-7811: ODBC: Implemented connection failove...

[isapego]

GitHub user isapego opened a pull request:

https://github.com/apache/ignite/pull/3643

IGNITE-7811: ODBC: Implemented connection failover



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7811

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3643.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3643


commit 3446fa362ded005b159fecb63769152d458963c2
Author: Igor Sapego 
Date:   2018-02-27T14:35:00Z

IGNITE-7811: Minor changes

commit de9e0062c947a401aecb097f853f94c86b58c09f
Author: Igor Sapego 
Date:   2018-03-05T16:19:27Z

IGNITE-7811: Fixed SslMode

commit 3ac07ac59ba7fcbed0684ba9a34efbb1e7568b8f
Author: Igor Sapego 
Date:   2018-03-05T16:20:59Z

IGNITE-7811: Moved connect string parsing to separate class.

commit 62689c454fd9c6761a71d939eae4de76a7e4606c
Author: Igor Sapego 
Date:   2018-03-05T16:27:36Z

IGNITE-7811: Removed ConnectionStringBase

commit c43a2a4026f2cb5cad5d3b9956a293cd0e9c0f73
Author: Igor Sapego 
Date:   2018-03-05T17:10:52Z

IGNITE-7811: Implemented SettableValue

commit 3bfa4b3dd9192a4248254e9d2fc77d6c78df969e
Author: Igor Sapego 
Date:   2018-03-06T15:07:07Z

IGNITE-7811: Configuration adjusted

commit bc1bffedf964c6c44d55ceb7959a258ecacbf748
Author: Igor Sapego 
Date:   2018-03-07T15:04:13Z

IGNITE-7811: Fixed compiliation

commit 7603cb8aea4bd5248c95a6da5d6fcfffabca9d14
Author: Igor Sapego 
Date:   2018-03-13T15:02:10Z

IGNITE-7811: Fixed tests

commit 4a85dcd8cb95f386c7a47a09c0eacf21d7f529d4
Author: Igor Sapego 
Date:   2018-03-13T16:24:59Z

IGNITE-7811: Added test

commit 4ce91db5b51b7e0c2670cd3c14cff2446ccf9e24
Author: Igor Sapego 
Date:   2018-03-14T11:39:39Z

IGNITE-7811: Fixed tests

commit f8cc48e5f035e14990dfaa7edf9dafbb48c5980c
Author: Igor Sapego 
Date:   2018-03-15T13:00:55Z

IGNITE-7811: Changed random number generation algorithm.

commit aeefaa1905aa56fb0d1272c4d4ccca996526a365
Author: Igor Sapego 
Date:   2018-03-15T14:35:10Z

IGNITE-7811: Added connection restoring

commit db80848bdf56a18d3dfd36c2e6fff671e4d73d5f
Author: Igor Sapego 
Date:   2018-03-15T15:39:45Z

IGNITE-7811: Added port range parsing

commit bdeab5edccd77ab67c3ec31a787a1ad1a3b45363
Author: Igor Sapego 
Date:   2018-03-15T15:49:29Z

IGNITE-7811: Implemented connection to the range

commit 572987854d1d66af2d1a87c16ce7bd2c2bb2a3c3
Author: Igor Sapego 
Date:   2018-03-15T16:08:15Z

IGNITE-7811: Refactored tests

commit 673ce3a95b8789281e28da9692e687b102066726
Author: Igor Sapego 
Date:   2018-03-15T16:16:25Z

IGNITE-7811: Added tests

commit 33c82ae50f41e6cd44ddf840bd2a02f46f022497
Author: Igor Sapego 
Date:   2018-03-15T16:47:05Z

IGNITE-7811: Fixes for Linux




---

Re: TeamCity test names refactoring

[Dmitry Pavlov]

I think it is better to keep existing naming because Igniters are used to
it.

Moreover a lot of issues use suite name as reference on where test is
located. So I prefer existing naming.

чт, 15 мар. 2018 г. в 19:43, vveider :

> Hi, Igniters!
>
>
> Took the liberty to introduce minor refactoring into test build names in
> our
> (now) main test project Ignite Tests 2.4+ (Java 8) [1]
> Changes affected the numeration of tests with common name, i.e
> Ignite Basic => Ignite Basic [I]
> Ignite Basic 2 => Ignite Basic [II]
> And so on.
> My main motivation is common design for the test project, including naming
> policy. It will help maintain this project in future with much less
> efforts.
>
> I earnestly encourage community to be a little bit patient while works on
> test improvements are held and counting on your support.
>
> As always - open to questions and constructive criticism.
>
>
> [1] https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

TeamCity test names refactoring

[vveider]

Hi, Igniters!


Took the liberty to introduce minor refactoring into test build names in our
(now) main test project Ignite Tests 2.4+ (Java 8) [1]
Changes affected the numeration of tests with common name, i.e
Ignite Basic => Ignite Basic [I]
Ignite Basic 2 => Ignite Basic [II]
And so on.
My main motivation is common design for the test project, including naming
policy. It will help maintain this project in future with much less efforts.

I earnestly encourage community to be a little bit patient while works on
test improvements are held and counting on your support.

As always - open to questions and constructive criticism.


[1] https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: Partition eviction failed, this can cause grid hang. (Caused by: java.lang.IllegalStateException: Failed to get page IO instance (page content is corrupted))

[Dmitry Pavlov]

Hi Alexey,

It may be serious issue. Could you recommend expert here who can pick up
this?

Sincerely,
Dmitriy Pavlov

чт, 15 мар. 2018 г. в 19:25, Arseny Kovalchuk :

> Hi, guys.
>
> I've got a reproducer for a problem which is generally reported as "Caused
> by: java.lang.IllegalStateException: Failed to get page IO instance (page
> content is corrupted)". Actually it reproduces the result. I don't have an
> idea how the data has been corrupted, but the cluster node doesn't want to
> start with this data.
>
> We got the issue again when some of server nodes were restarted several
> times by kubernetes. I suspect that the data got corrupted during such
> restarts. But the main functionality that we really desire to have, that
> the cluster DOESN'T HANG during next restart even if the data is corrupted!
> Anyway, there is no a tool that can help to correct such data, and as a
> result we wipe all data manually to start the cluster. So, having warnings
> about corrupted data in logs and just working cluster is the expected
> behavior.
>
> How to reproduce:
> 1. Download the data from here
> https://storage.googleapis.com/pub-data-0/data5.tar.gz (~200Mb)
> 2. Download and import Gradle project
> https://storage.googleapis.com/pub-data-0/project.tar.gz (~100Kb)
> 3. Unpack the data to the home folder, say /home/user1. You should get the
> path like */home/user1/data5*. Inside data5 you should have binary_meta,
> db, marshaller.
> 4. Open *src/main/resources/data-test.xml* and put the absolute path of
> unpacked data into *workDirectory* property of *igniteCfg5* bean. In this
> example it should be */home/user1/data5.* Do not edit consistentId!
> The consistentId is ignite-instance-5, so the real data is in
> the data5/db/ignite_instance_5 folder
> 5. Start application from ru.synesis.kipod.DataTestBootApp
> 6. Enjoy
>
> Hope it will help.
>
>
> ​
> Arseny Kovalchuk
>
> Senior Software Engineer at Synesis
> skype: arseny.kovalchuk
> mobile: +375 (29) 666-16-16 <+375%2029%20666-16-16>
> ​LinkedIn Profile ​
>
> On 26 December 2017 at 21:15, Denis Magda  wrote:
>
>> Cross-posting to the dev list.
>>
>> Ignite persistence maintainers please chime in.
>>
>> —
>> Denis
>>
> On Dec 26, 2017, at 2:17 AM, Arseny Kovalchuk 
>> wrote:
>>
>> Hi guys.
>>
>> Another issue when using Ignite 2.3 with native persistence enabled. See
>> details below.
>>
>> We deploy Ignite along with our services in Kubernetes (v 1.8) on
>> premises. Ignite cluster is a StatefulSet of 5 Pods (5 instances) of Ignite
>> version 2.3. Each Pod mounts PersistentVolume backed by CEPH RBD.
>>
>> We put about 230 events/second into Ignite, 70% of events are ~200KB in
>> size and 30% are 5000KB. Smaller events have indexed fields and we query
>> them via SQL.
>>
>> The cluster is activated from a client node which also streams events
>> into Ignite from Kafka. We use custom implementation of streamer which uses
>> cache.putAll() API.
>>
>> We started cluster from scratch without any persistent data. After a
>> while we got corrupted data with the error message.
>>
>> [2017-12-26 07:44:14,251] ERROR [sys-#127%ignite-instance-2%]
>> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader:
>> - Partition eviction failed, this can cause grid hang.
>> class org.apache.ignite.IgniteException: Runtime failure on search row:
>> Row@5b1479d6[ key: 171:1513946618964:3008806055072854, val:
>> ru.synesis.kipod.event.KipodEvent [idHash=510912646, hash=-387621419,
>> face_last_name=null, face_list_id=null, channel=171, source=,
>> face_similarity=null, license_plate_number=null, descriptors=null,
>> cacheName=kipod_events, cacheKey=171:1513946618964:3008806055072854,
>> stream=171, alarm=false, processed_at=0, face_id=null, id=3008806055072854,
>> persistent=false, face_first_name=null, license_plate_first_name=null,
>> face_full_name=null, level=0, module=Kpx.Synesis.Outdoor,
>> end_time=1513946624379, params=null, commented_at=0, tags=[vehicle, 0,
>> human, 0, truck, 0, start_time=1513946618964, processed=false,
>> kafka_offset=111259, license_plate_last_name=null, armed=false,
>> license_plate_country=null, topic=MovingObject, comment=,
>> expiration=1514033024000, original_id=null, license_plate_lists=null], ver:
>> GridCacheVersion [topVer=125430590, order=1513955001926, nodeOrder=3] ][
>> 3008806055072854, MovingObject, Kpx.Synesis.Outdoor, 0, , 1513946618964,
>> 1513946624379, 171, 171, FALSE, FALSE, , FALSE, FALSE, 0, 0, 111259,
>> 1514033024000, (vehicle, 0, human, 0, truck, 0), null, null, null, null,
>> null, null, null, null, null, null, null, null ]
>> at org.apache.ignite.internal.pro
>> cessors.cache.persistence.tree.BPlusTree.doRemove(BPlusTree.java:1787)
>> at org.apache.ignite.internal.pro
>> cessors.cache.persistence.tree.BPlusTree.remove(BPlusTree.java:1578)
>> at org.apache.ignite.internal.pro
>> cessors.query.h2.database.H2TreeIndex.remove(H2TreeIndex.java:216)
>> at 

Re: Nodes which started in separate JVM couldn't stop properly (in tests)

[Dmitry Pavlov]

I see now. Thank you.

Nikolay, could you please merge this change?

чт, 15 мар. 2018 г. в 18:48, Vyacheslav Daradur :

> In brief:
> Nodes in *separate* JVMs are shutting down by the computing task
> *StopGridTask* which has sent from *local* JVM *synchronously* that
> means *local* node must wait for task's finish.
>
> At the same time when a node in *separate* JVM executes the received
> *StopGridTask* which *synchronously* calls *G.stop(igniteInstanceName,
> FALSE)* which is waiting for all computing task's finish, including
> *StopGridTask* which has invoked it.
>
> We have some kind of deadlock:
> *Local* node is waiting for the computing task's finish which is
> waiting for finish of execution *G.stop* which is waiting for all
> computing tasks finish including *StopGridTask*.
>
> We have not noticed that before because we use only stopAllGrids() in
> out tests which stop local JVM without waiting for nodes in other
> JVMs.
>
>
>
> On Thu, Mar 15, 2018 at 6:11 PM, Dmitry Pavlov 
> wrote:
> > Please address comments in PR.
> >
> > I did not fully understood why sync GridStopMessage message was lost, but
> > async will be successfull. Probably we need discuss it briefly.
> >
> > чт, 1 мар. 2018 г. в 12:11, Vyacheslav Daradur :
> >>
> >> Thank you, Dmitry!
> >>
> >> I'll join this review soon.
> >>
> >> On Thu, Mar 1, 2018 at 12:07 PM, Dmitry Pavlov 
> >> wrote:
> >> > Hi Vyacheslav,
> >> >
> >> > I will take a look, but first of all I am going to review
> >> > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502  - it is
> >> > impact
> >> > change in testing framework. Hope you also will join to this review .
> >> >
> >> > Sincerely,
> >> > Dmitiry Pavlov
> >> >
> >> >
> >> > чт, 1 мар. 2018 г. в 11:13, Vyacheslav Daradur :
> >> >>
> >> >> Hi, Dmitry, could you please review it, because you are one of the
> >> >> most experienced people in the testing framework.
> >> >>
> >> >> Please see comment in Jira, because it is in pretty-format there.
> >> >>
> >> >> On Thu, Feb 22, 2018 at 11:56 AM, Vyacheslav Daradur
> >> >>  wrote:
> >> >> > Hi Igniters!
> >> >> >
> >> >> > I have investigated the issue [1] and found that stopping node in
> >> >> > separate JVM may stuck thread or leave system process alive after
> >> >> > test
> >> >> > finished.
> >> >> > The main reason is *StopGridTask* that we send from node in local
> JVM
> >> >> > to node in separate JVM via remote computing.
> >> >> > We send job synchronously to be sure that node will be stopped, but
> >> >> > job calls synchronously *G.stop(igniteInstanceName, cancel))* with
> >> >> > *cancel = false*, that means node must wait to compute jobs before
> it
> >> >> > goes down what leads to some kind of deadlock. Using of *cancel =
> >> >> > true* would solve the issue but may break some tests’ logic, for
> this
> >> >> > reason, I've reworked the method’s synchronization logic [2].
> >> >> >
> >> >> > We have not noticed that before because we use only
> *stopAllGrids()*
> >> >> > in out tests which stop local JVM without waiting for nodes in
> other
> >> >> > JVMs.
> >> >> > I believe this fix should reduce the number of flaky tests on
> >> >> > TeamCity, especially which fails because of a cluster from the
> >> >> > previous test has not been stopped properly.
> >> >> >
> >> >> > Ci.tests [3] look a bit better than in master.
> >> >> > Please review prepared PR [2] and share your thoughts.
> >> >> >
> >> >> > [1] https://issues.apache.org/jira/browse/IGNITE-5910
> >> >> > [2] https://github.com/apache/ignite/pull/2382
> >> >> > [3] https://ci.ignite.apache.org/viewLog.html?buildId=1105939
> >> >> >
> >> >> >
> >> >> > On Fri, Aug 4, 2017 at 11:41 AM, Vyacheslav Daradur
> >> >> >  wrote:
> >> >> >> Hi Igniters,
> >> >> >>
> >> >> >> Working on my task I found a bug at call the method
> #stopGrid(name),
> >> >> >> it produced ClassCastException. I created a ticket[1].
> >> >> >>
> >> >> >> After it was fixed[2] I saw that nodes which was started in a
> >> >> >> separate
> >> >> >> JVM
> >> >> >> could stay in process of operation system.
> >> >> >> It was fixed too, but not sure is it fixed in proper way or not.
> >> >> >>
> >> >> >> Could someone review it?
> >> >> >>
> >> >> >> [1] https://issues.apache.org/jira/browse/IGNITE-5910
> >> >> >> [2] https://github.com/apache/ignite/pull/2382
> >> >> >>
> >> >> >> --
> >> >> >> Best Regards, Vyacheslav D.
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Best Regards, Vyacheslav D.
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Best Regards, Vyacheslav D.
> >>
> >>
> >>
> >> --
> >> Best Regards, Vyacheslav D.
>
>
>
> --
> Best Regards, Vyacheslav D.
>

Fwd: [jira] [Resolved] (IGNITE-7646) OSGI tests fail to unlock build directory causing following builds to fail to start

[Dmitry Pavlov]

Hi Peter, Vyacheslav, thank you for solving this issue.

Each contribution like this helps Ignite to achieve stable test runs.


-- Forwarded message -
From: Peter Ivanov (JIRA) 
Date: чт, 15 мар. 2018 г. в 18:37
Subject: [jira] [Resolved] (IGNITE-7646) OSGI tests fail to unlock build
directory causing following builds to fail to start
To: 



 [
https://issues.apache.org/jira/browse/IGNITE-7646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Peter Ivanov resolved IGNITE-7646.
--
Resolution: Fixed

> OSGI tests fail to unlock build directory causing following builds to
fail to start
>
---
>
> Key: IGNITE-7646
> URL: https://issues.apache.org/jira/browse/IGNITE-7646
> Project: Ignite
>  Issue Type: Task
>Reporter: Peter Ivanov
>Assignee: Peter Ivanov
>Priority: Critical
>  Labels: MakeTeamcityGreenAgain
>
> After running test suite [Ignite Tests 2.4+ \[Java 8\] Ignite OSGi|
https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_IgniteOSGi]
under Windows based agent, following builds fail to start with error
> {code}
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\data\cache\bundle1\version0.0\bundle.jar
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\data\cache\bundle2\version0.0\bundle.jar
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\data\cache\bundle3\version0.0\bundle.jar
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\data\cache\bundle4\version0.0\bundle.jar
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\data\cache\bundle5\version0.0\bundle.jar
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\data\cache\bundle6\version0.0\bundle.jar
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\data\cache\bundle7\version0.0\bundle.jar
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\data\cache\bundle8\version0.0\bundle.jar
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\data\cache\cache.lock
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\data\log\karaf.log
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\lib\boot\org.apache.karaf.diagnostic.boot-4.0.2.jar
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\lib\boot\org.apache.karaf.jaas.boot-4.0.2.jar
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\lib\boot\org.apache.karaf.main-4.0.2.jar
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\lib\boot\org.osgi.core-6.0.0.jar
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\lib\endorsed\javax.annotation-api-1.2.jar
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\lib\endorsed\org.apache.karaf.exception-4.0.2.jar
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\lib\endorsed\org.apache.servicemix.bundles.xalan-2.7.2_2.jar
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\lib\endorsed\org.apache.servicemix.bundles.xalan-serializer-2.7.2_1.jar
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\lib\endorsed\org.apache.servicemix.specs.activation-api-1.1-2.5.0.jar
> [16:30:24]Failed to delete file:
C:\BuildAgent\work\bd85361428dcdb1\modules\osgi\target\paxexam\unpack\f12af441-dfed-411b-9784-87dafedfac4f\lib\endorsed\org

Re: Nodes which started in separate JVM couldn't stop properly (in tests)

[Vyacheslav Daradur]

In brief:
Nodes in *separate* JVMs are shutting down by the computing task
*StopGridTask* which has sent from *local* JVM *synchronously* that
means *local* node must wait for task's finish.

At the same time when a node in *separate* JVM executes the received
*StopGridTask* which *synchronously* calls *G.stop(igniteInstanceName,
FALSE)* which is waiting for all computing task's finish, including
*StopGridTask* which has invoked it.

We have some kind of deadlock:
*Local* node is waiting for the computing task's finish which is
waiting for finish of execution *G.stop* which is waiting for all
computing tasks finish including *StopGridTask*.

We have not noticed that before because we use only stopAllGrids() in
out tests which stop local JVM without waiting for nodes in other
JVMs.



On Thu, Mar 15, 2018 at 6:11 PM, Dmitry Pavlov  wrote:
> Please address comments in PR.
>
> I did not fully understood why sync GridStopMessage message was lost, but
> async will be successfull. Probably we need discuss it briefly.
>
> чт, 1 мар. 2018 г. в 12:11, Vyacheslav Daradur :
>>
>> Thank you, Dmitry!
>>
>> I'll join this review soon.
>>
>> On Thu, Mar 1, 2018 at 12:07 PM, Dmitry Pavlov 
>> wrote:
>> > Hi Vyacheslav,
>> >
>> > I will take a look, but first of all I am going to review
>> > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502  - it is
>> > impact
>> > change in testing framework. Hope you also will join to this review .
>> >
>> > Sincerely,
>> > Dmitiry Pavlov
>> >
>> >
>> > чт, 1 мар. 2018 г. в 11:13, Vyacheslav Daradur :
>> >>
>> >> Hi, Dmitry, could you please review it, because you are one of the
>> >> most experienced people in the testing framework.
>> >>
>> >> Please see comment in Jira, because it is in pretty-format there.
>> >>
>> >> On Thu, Feb 22, 2018 at 11:56 AM, Vyacheslav Daradur
>> >>  wrote:
>> >> > Hi Igniters!
>> >> >
>> >> > I have investigated the issue [1] and found that stopping node in
>> >> > separate JVM may stuck thread or leave system process alive after
>> >> > test
>> >> > finished.
>> >> > The main reason is *StopGridTask* that we send from node in local JVM
>> >> > to node in separate JVM via remote computing.
>> >> > We send job synchronously to be sure that node will be stopped, but
>> >> > job calls synchronously *G.stop(igniteInstanceName, cancel))* with
>> >> > *cancel = false*, that means node must wait to compute jobs before it
>> >> > goes down what leads to some kind of deadlock. Using of *cancel =
>> >> > true* would solve the issue but may break some tests’ logic, for this
>> >> > reason, I've reworked the method’s synchronization logic [2].
>> >> >
>> >> > We have not noticed that before because we use only *stopAllGrids()*
>> >> > in out tests which stop local JVM without waiting for nodes in other
>> >> > JVMs.
>> >> > I believe this fix should reduce the number of flaky tests on
>> >> > TeamCity, especially which fails because of a cluster from the
>> >> > previous test has not been stopped properly.
>> >> >
>> >> > Ci.tests [3] look a bit better than in master.
>> >> > Please review prepared PR [2] and share your thoughts.
>> >> >
>> >> > [1] https://issues.apache.org/jira/browse/IGNITE-5910
>> >> > [2] https://github.com/apache/ignite/pull/2382
>> >> > [3] https://ci.ignite.apache.org/viewLog.html?buildId=1105939
>> >> >
>> >> >
>> >> > On Fri, Aug 4, 2017 at 11:41 AM, Vyacheslav Daradur
>> >> >  wrote:
>> >> >> Hi Igniters,
>> >> >>
>> >> >> Working on my task I found a bug at call the method #stopGrid(name),
>> >> >> it produced ClassCastException. I created a ticket[1].
>> >> >>
>> >> >> After it was fixed[2] I saw that nodes which was started in a
>> >> >> separate
>> >> >> JVM
>> >> >> could stay in process of operation system.
>> >> >> It was fixed too, but not sure is it fixed in proper way or not.
>> >> >>
>> >> >> Could someone review it?
>> >> >>
>> >> >> [1] https://issues.apache.org/jira/browse/IGNITE-5910
>> >> >> [2] https://github.com/apache/ignite/pull/2382
>> >> >>
>> >> >> --
>> >> >> Best Regards, Vyacheslav D.
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Best Regards, Vyacheslav D.
>> >>
>> >>
>> >>
>> >> --
>> >> Best Regards, Vyacheslav D.
>>
>>
>>
>> --
>> Best Regards, Vyacheslav D.



-- 
Best Regards, Vyacheslav D.

[jira] [Created] (IGNITE-7969) Fluky failure of IgniteTopologyValidatorGridSplitCacheTest.testTopologyValidatorWithCacheGroup

[Aleksey Plekhanov (JIRA)]

Aleksey Plekhanov created IGNITE-7969:
-

 Summary: Fluky failure of 
IgniteTopologyValidatorGridSplitCacheTest.testTopologyValidatorWithCacheGroup
 Key: IGNITE-7969
 URL: https://issues.apache.org/jira/browse/IGNITE-7969
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov


Test fails on TeamCity with exception:
{noformat}
java.lang.AssertionError: Successful tryPut after failure [gridIdx=2, sucessful 
puts = 50]

at 
org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:491)
at 
org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.testTopologyValidator0(IgniteTopologyValidatorGridSplitCacheTest.java:281)
at 
org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.testTopologyValidatorWithCacheGroup(IgniteTopologyValidatorGridSplitCacheTest.java:234)
...
Caused by: class org.apache.ignite.IgniteException: Failed to put entry
at 
org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:498)
... 11 more
Suppressed: class org.apache.ignite.IgniteException: Failed to put 
entry [node=cache.IgniteTopologyValidatorGridSplitCacheTest0]
at 
org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:463)
at 
org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:488)
... 11 more
Suppressed: junit.framework.AssertionFailedError: expected:<1> 
but was:<2>
at junit.framework.Assert.fail(Assert.java:57)
at junit.framework.Assert.failNotEquals(Assert.java:329)
at junit.framework.Assert.assertEquals(Assert.java:78)
at junit.framework.Assert.assertEquals(Assert.java:234)
at junit.framework.Assert.assertEquals(Assert.java:241)
at 
junit.framework.TestCase.assertEquals(TestCase.java:409)
at 
org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:448)
... 12 more
...
{noformat}
Sometimes reproduces locally.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3640: IGNITE-7965: Robin-hood hashing may fail with neg...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3640


---

Re: Nodes which started in separate JVM couldn't stop properly (in tests)

[Dmitry Pavlov]

Please address comments in PR.

I did not fully understood why sync GridStopMessage message was lost, but
async will be successfull. Probably we need discuss it briefly.

чт, 1 мар. 2018 г. в 12:11, Vyacheslav Daradur :

> Thank you, Dmitry!
>
> I'll join this review soon.
>
> On Thu, Mar 1, 2018 at 12:07 PM, Dmitry Pavlov 
> wrote:
> > Hi Vyacheslav,
> >
> > I will take a look, but first of all I am going to review
> > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502  - it is
> impact
> > change in testing framework. Hope you also will join to this review .
> >
> > Sincerely,
> > Dmitiry Pavlov
> >
> >
> > чт, 1 мар. 2018 г. в 11:13, Vyacheslav Daradur :
> >>
> >> Hi, Dmitry, could you please review it, because you are one of the
> >> most experienced people in the testing framework.
> >>
> >> Please see comment in Jira, because it is in pretty-format there.
> >>
> >> On Thu, Feb 22, 2018 at 11:56 AM, Vyacheslav Daradur
> >>  wrote:
> >> > Hi Igniters!
> >> >
> >> > I have investigated the issue [1] and found that stopping node in
> >> > separate JVM may stuck thread or leave system process alive after test
> >> > finished.
> >> > The main reason is *StopGridTask* that we send from node in local JVM
> >> > to node in separate JVM via remote computing.
> >> > We send job synchronously to be sure that node will be stopped, but
> >> > job calls synchronously *G.stop(igniteInstanceName, cancel))* with
> >> > *cancel = false*, that means node must wait to compute jobs before it
> >> > goes down what leads to some kind of deadlock. Using of *cancel =
> >> > true* would solve the issue but may break some tests’ logic, for this
> >> > reason, I've reworked the method’s synchronization logic [2].
> >> >
> >> > We have not noticed that before because we use only *stopAllGrids()*
> >> > in out tests which stop local JVM without waiting for nodes in other
> >> > JVMs.
> >> > I believe this fix should reduce the number of flaky tests on
> >> > TeamCity, especially which fails because of a cluster from the
> >> > previous test has not been stopped properly.
> >> >
> >> > Ci.tests [3] look a bit better than in master.
> >> > Please review prepared PR [2] and share your thoughts.
> >> >
> >> > [1] https://issues.apache.org/jira/browse/IGNITE-5910
> >> > [2] https://github.com/apache/ignite/pull/2382
> >> > [3] https://ci.ignite.apache.org/viewLog.html?buildId=1105939
> >> >
> >> >
> >> > On Fri, Aug 4, 2017 at 11:41 AM, Vyacheslav Daradur
> >> >  wrote:
> >> >> Hi Igniters,
> >> >>
> >> >> Working on my task I found a bug at call the method #stopGrid(name),
> >> >> it produced ClassCastException. I created a ticket[1].
> >> >>
> >> >> After it was fixed[2] I saw that nodes which was started in a
> separate
> >> >> JVM
> >> >> could stay in process of operation system.
> >> >> It was fixed too, but not sure is it fixed in proper way or not.
> >> >>
> >> >> Could someone review it?
> >> >>
> >> >> [1] https://issues.apache.org/jira/browse/IGNITE-5910
> >> >> [2] https://github.com/apache/ignite/pull/2382
> >> >>
> >> >> --
> >> >> Best Regards, Vyacheslav D.
> >> >
> >> >
> >> >
> >> > --
> >> > Best Regards, Vyacheslav D.
> >>
> >>
> >>
> >> --
> >> Best Regards, Vyacheslav D.
>
>
>
> --
> Best Regards, Vyacheslav D.
>

Re: Partition recovery issue on partition loss.

[Dmitry Pavlov]

Hi Andrew Mashenkov,

would you like to pick up issue?

Sincerely,
Dmitriy Pavlov

чт, 15 мар. 2018 г. в 6:23, Dmitriy Setrakyan :

> Completely agree, we must fix this. I like the proposed design. We should
> also specify that resetLostPartitions() method should return true and
> false.
>
> Val, do you mind updating the ticket with new design?
> https://issues.apache.org/jira/browse/IGNITE-7832
>
> D.
>
> On Tue, Mar 13, 2018 at 5:31 PM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > This indeed looks like a bigger issue. Basically, there is no clear way
> (or
> > no way at all) to synchronize code that listens to partition loss event,
> > and the code that calls resetLostPartitions() method. Example scenario:
> >
> > 1. Cache is configured with 3rd party persistence.
> > 2. One or more nodes fail causing loss of several partitions in memory.
> > 3. Ignite blocks access to those partitions according to partition loss
> > policy and fires an event.
> > 4. Application listens to the event and starts reloading the data from
> > store.
> > 5. When reloading is complete, application calls resetLostPartitions() to
> > restore access.
> > 6. Nodes fail again causing another partition loss, new event is fired.
> >
> > There is race between steps 5 and 6. If 2nd failure happens BEFORE
> > resetLostPartitions() is called, we end up with inconsistent data.
> >
> > I believe the only way to fix this is to add corresponding topology
> version
> > to partition loss event, and also add it as a parameter for
> > resetLostPartitions().
> > This way if resetLostPartitions() is invoked with a version that is not
> the
> > latest anymore, the invocation will be ignored.
> >
> > The only problem with this approach  is that topology version itself is
> > currently not a part of public API. It needs to be properly exposed there
> > first.
> >
> > -Val
> >
> > On Mon, Mar 12, 2018 at 1:07 PM, Denis Magda  wrote:
> >
> > > Just in case here is you can find the present documentation:
> > >
> https://apacheignite.readme.io/docs/cache-modes#partition-loss-policies
> > >
> > > Let us know what needs to be updated once the issues reported by you
> are
> > > addressed.
> > >
> > > --
> > > Denis
> > >
> > > On Mon, Mar 12, 2018 at 3:33 AM, Andrey Mashenkov <
> > > andrey.mashen...@gmail.com> wrote:
> > >
> > > > Hi Igniters,
> > > >
> > > > I've found we no documentation how user can recover cache from
> > cacheStore
> > > > in case of partition loss.
> > > > Ignite provides some instruments (methods and events) that should
> help
> > > user
> > > > to solve this problem,
> > > > but looks like these instruments have an architecture lack.
> > > >
> > > > The first one is an usability issue. Ignite provides partition loss
> > event
> > > > to user can handle this, but Ignite fires an event per partition.
> > > > Why we can't have an event with list of lost partitions?
> > > >
> > > > The second one is a bug. Ignite.resetLostPartitions() method doesn't
> > care
> > > > about what topology version recovered partitions belonged to.
> > > > Tthere is a race, when user call this method after a node was failed,
> > but
> > > > right before Ignite fire an event.
> > > > So, it is possible state of just lost partitions will be reseted
> > > > unexpectedly.
> > > >
> > > >
> > > > I've created a ticket for this [1] and think we should rethink the
> > > > architecture of the partition recovery mechanics and improve
> > > documentation.
> > > > Any thoughts?
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-7832
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Andrey V. Mashenkov
> > > >
> > >
> >
>

[GitHub] ignite pull request #3642: IGNITE-7961

[ilantukh]

GitHub user ilantukh opened a pull request:

https://github.com/apache/ignite/pull/3642

IGNITE-7961

Benchmarks for rebalance.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7961

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3642.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3642


commit fdb4e19ef150bc71ac6dd5b1511f5a8bf8ff94b4
Author: Ilya Lantukh 
Date:   2018-03-15T10:04:52Z

ignite-7961 : Local JMH benchmark.

commit 3414e91537823b50869864c132ed5e67784b1c25
Author: Ilya Lantukh 
Date:   2018-03-15T14:23:56Z

ignite-7961 : Potential optimizations




---

Re: Atomic sequence and topology exception

[Dmitry Pavlov]

Hi Pavel,

would you like to prepare fix/PR for this ticket?

Sincerely,
Dmitriy Pavlov

чт, 15 мар. 2018 г. в 16:36, Vinokurov Pavel :

> Alexey,
>
> Thank you for your explanation.
> I have created the ticket
> https://issues.apache.org/jira/browse/IGNITE-7968
> with attached reproducer.
>
> 2018-03-15 10:23 GMT+03:00 Alexey Goncharuk :
>
> > Pavel,
> >
> > The exception from an AtomicSequence looks like a bug to me. Ignite
> should
> > retry all operations that do not involve user's logic, this stands for
> > AtomicSequence. I believe such handling already present in AtomicLong, so
> > it should not be difficult to fix.
> >
> > The only case when a user must handle TopologyException is an explicit
> > transaction. In this case, the transaction involves particular user logic
> > that cannot be captured and retried by Ignite, so the exception handling
> > should be covered by the user.
> >
> > The way you handled the topology exception looks correct to me.
> >
> > --AG
> >
> > 2018-03-14 20:47 GMT+03:00 Dmitry Pavlov :
> >
> > > Hi Nikolay,
> > >
> > > How do you think which was idea / the best practice to handling
> > exceptions
> > > here?
> > >
> > > Why exception here may have difference?
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > -- Forwarded message -
> > > From: Vinokurov Pavel 
> > > Date: вт, 13 мар. 2018 г. в 5:52
> > > Subject: Atomic sequence and topology exception
> > > To: 
> > >
> > >
> > > Igniters!
> > >
> > > I  have a few questions related to a replicated atomic sequence and an
> > > topology exception.
> > > In my case after one server node has left cluster, on a client node the
> > > execution of the *IgniteAtomicSequence#incrementAndGet()* throws
> > > exception:
> > >
> > > org.apache.ignite.cluster.ClusterTopologyException: Failed to acquire
> > lock
> > > for keys (primary node left grid, retry transaction if possible)
> > > [keys=[UserKeyCacheObjectImpl [part=94, val=GridCacheInternalKeyImpl
> > > [name=seq, grpName=default-ds-group], hasValBytes=true]],
> > > node=a047ec4b-7de6-4503-9691-a5d7e03e267f]
> > >
> > > I handle exception in that way:
> > >
> > > IgniteAtomicSequence seq = ignite.atomicSequence(Const.SEQ, new
> > > AtomicConfiguration().setAtomicSequenceReserveSize(0)
> > > .setCacheMode(CacheMode.REPLICATED),0, Boolean.TRUE);
> > > while(true){
> > > //do some logic
> > > try {
> > > *seq.incrementAndGet();*
> > > }
> > > catch (ClusterTopologyException cte) {
> > > *cte.retryReadyFuture().get();*
> > > }
> > > }
> > >
> > > At the same time the *IgniteAtomicLong* doesn't throw such exception
> (at
> > > least I can't reproduce it).
> > >
> > > Please help me to clarify flowing questions:
> > > 1. Is it recommended to handle topology exception in the same way? Is
> > there
> > > any public documentation about that?
> > > 2. What kind of distributed operations(cache updates, open data stream,
> > > atomic) are recommended to handle the ClusterTopologyException?
> > >
> > > --
> > >
> > > Regards
> > >
> > > Pavel Vinokurov
> > >
> >
>
>
>
> --
>
> Regards
>
> Pavel Vinokurov
>

Re: Ignite Platform .NET Inspections problems

[Dmitry Pavlov]

Hi Pavel,

Let me say thank you from me too. All test fixes helps us to run review
more easy.

Sincerely,
Dmitriy Pavlov

чт, 15 мар. 2018 г., 16:49 Pavel Tupitsyn :

> Suppressed these two as well
>
> On Thu, Mar 15, 2018 at 4:45 PM, Petr Ivanov  wrote:
>
> > BTW, preliminary run still shows 2 failed inspections [1]
> >
> >
> > [1] https://ci.ignite.apache.org/viewLog.html?buildId=1138826
> >
> >
> > > On 15 Mar 2018, at 15:41, Petr Ivanov  wrote:
> > >
> > > I see. Will wait for result tomorrow.
> > >
> > >
> > >
> > >> On 15 Mar 2018, at 15:40, Pavel Tupitsyn 
> wrote:
> > >>
> > >> Just waiting for automatic build in master branch
> > >>
> > >> On Thu, Mar 15, 2018 at 3:39 PM, Petr Ivanov 
> > wrote:
> > >>
> > >>> Nice news, Pavel!
> > >>>
> > >>> Can you point to the exact build where you are checking changes,
> > please?
> > >>>
> > >>>
> > >>>
> >  On 15 Mar 2018, at 15:34, Pavel Tupitsyn 
> > wrote:
> > 
> >  Hi Petr,
> > 
> >  I've fixed that yesterday [1], waiting for the build to verify.
> > 
> >  [1]
> >  https://github.com/apache/ignite/commit/
> > a69e1acba8a69f1818fa9182a11673
> > >>> 02f81b93ec
> > 
> >  On Thu, Mar 15, 2018 at 1:07 PM, Petr Ivanov 
> > >>> wrote:
> > 
> > > HI, Pavel.
> > >
> > >
> > > After TeamCity update from 2017.1.3 to 2017.2.2 build Ignite
> Platform
> > >>> .NET
> > > Inspections [1] started to show 12-14 failed inspections.
> > > Can you help and check settings for this build, please? Or give any
> > > recommendations on how to fix this?
> > >
> > > Thanks!
> > >
> > >
> > > [1] https://ci.ignite.apache.org/viewType.html?buildTypeId=
> > > IgniteTests24Java8_IgnitePlatformNetInspections
> > >
> > >>>
> > >>>
> > >
> >
> >
>

Re: Ignite Platform .NET Inspections problems

[Pavel Tupitsyn]

Suppressed these two as well

On Thu, Mar 15, 2018 at 4:45 PM, Petr Ivanov  wrote:

> BTW, preliminary run still shows 2 failed inspections [1]
>
>
> [1] https://ci.ignite.apache.org/viewLog.html?buildId=1138826
>
>
> > On 15 Mar 2018, at 15:41, Petr Ivanov  wrote:
> >
> > I see. Will wait for result tomorrow.
> >
> >
> >
> >> On 15 Mar 2018, at 15:40, Pavel Tupitsyn  wrote:
> >>
> >> Just waiting for automatic build in master branch
> >>
> >> On Thu, Mar 15, 2018 at 3:39 PM, Petr Ivanov 
> wrote:
> >>
> >>> Nice news, Pavel!
> >>>
> >>> Can you point to the exact build where you are checking changes,
> please?
> >>>
> >>>
> >>>
>  On 15 Mar 2018, at 15:34, Pavel Tupitsyn 
> wrote:
> 
>  Hi Petr,
> 
>  I've fixed that yesterday [1], waiting for the build to verify.
> 
>  [1]
>  https://github.com/apache/ignite/commit/
> a69e1acba8a69f1818fa9182a11673
> >>> 02f81b93ec
> 
>  On Thu, Mar 15, 2018 at 1:07 PM, Petr Ivanov 
> >>> wrote:
> 
> > HI, Pavel.
> >
> >
> > After TeamCity update from 2017.1.3 to 2017.2.2 build Ignite Platform
> >>> .NET
> > Inspections [1] started to show 12-14 failed inspections.
> > Can you help and check settings for this build, please? Or give any
> > recommendations on how to fix this?
> >
> > Thanks!
> >
> >
> > [1] https://ci.ignite.apache.org/viewType.html?buildTypeId=
> > IgniteTests24Java8_IgnitePlatformNetInspections
> >
> >>>
> >>>
> >
>
>

Re: Ignite Platform .NET Inspections problems

[Petr Ivanov]

BTW, preliminary run still shows 2 failed inspections [1]


[1] https://ci.ignite.apache.org/viewLog.html?buildId=1138826


> On 15 Mar 2018, at 15:41, Petr Ivanov  wrote:
> 
> I see. Will wait for result tomorrow.
> 
> 
> 
>> On 15 Mar 2018, at 15:40, Pavel Tupitsyn  wrote:
>> 
>> Just waiting for automatic build in master branch
>> 
>> On Thu, Mar 15, 2018 at 3:39 PM, Petr Ivanov  wrote:
>> 
>>> Nice news, Pavel!
>>> 
>>> Can you point to the exact build where you are checking changes, please?
>>> 
>>> 
>>> 
 On 15 Mar 2018, at 15:34, Pavel Tupitsyn  wrote:
 
 Hi Petr,
 
 I've fixed that yesterday [1], waiting for the build to verify.
 
 [1]
 https://github.com/apache/ignite/commit/a69e1acba8a69f1818fa9182a11673
>>> 02f81b93ec
 
 On Thu, Mar 15, 2018 at 1:07 PM, Petr Ivanov 
>>> wrote:
 
> HI, Pavel.
> 
> 
> After TeamCity update from 2017.1.3 to 2017.2.2 build Ignite Platform
>>> .NET
> Inspections [1] started to show 12-14 failed inspections.
> Can you help and check settings for this build, please? Or give any
> recommendations on how to fix this?
> 
> Thanks!
> 
> 
> [1] https://ci.ignite.apache.org/viewType.html?buildTypeId=
> IgniteTests24Java8_IgnitePlatformNetInspections
> 
>>> 
>>> 
> 

Re: Atomic sequence and topology exception

[Vinokurov Pavel]

Alexey,

Thank you for your explanation.
I have created the ticket https://issues.apache.org/jira/browse/IGNITE-7968
with attached reproducer.

2018-03-15 10:23 GMT+03:00 Alexey Goncharuk :

> Pavel,
>
> The exception from an AtomicSequence looks like a bug to me. Ignite should
> retry all operations that do not involve user's logic, this stands for
> AtomicSequence. I believe such handling already present in AtomicLong, so
> it should not be difficult to fix.
>
> The only case when a user must handle TopologyException is an explicit
> transaction. In this case, the transaction involves particular user logic
> that cannot be captured and retried by Ignite, so the exception handling
> should be covered by the user.
>
> The way you handled the topology exception looks correct to me.
>
> --AG
>
> 2018-03-14 20:47 GMT+03:00 Dmitry Pavlov :
>
> > Hi Nikolay,
> >
> > How do you think which was idea / the best practice to handling
> exceptions
> > here?
> >
> > Why exception here may have difference?
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > -- Forwarded message -
> > From: Vinokurov Pavel 
> > Date: вт, 13 мар. 2018 г. в 5:52
> > Subject: Atomic sequence and topology exception
> > To: 
> >
> >
> > Igniters!
> >
> > I  have a few questions related to a replicated atomic sequence and an
> > topology exception.
> > In my case after one server node has left cluster, on a client node the
> > execution of the *IgniteAtomicSequence#incrementAndGet()* throws
> > exception:
> >
> > org.apache.ignite.cluster.ClusterTopologyException: Failed to acquire
> lock
> > for keys (primary node left grid, retry transaction if possible)
> > [keys=[UserKeyCacheObjectImpl [part=94, val=GridCacheInternalKeyImpl
> > [name=seq, grpName=default-ds-group], hasValBytes=true]],
> > node=a047ec4b-7de6-4503-9691-a5d7e03e267f]
> >
> > I handle exception in that way:
> >
> > IgniteAtomicSequence seq = ignite.atomicSequence(Const.SEQ, new
> > AtomicConfiguration().setAtomicSequenceReserveSize(0)
> > .setCacheMode(CacheMode.REPLICATED),0, Boolean.TRUE);
> > while(true){
> > //do some logic
> > try {
> > *seq.incrementAndGet();*
> > }
> > catch (ClusterTopologyException cte) {
> > *cte.retryReadyFuture().get();*
> > }
> > }
> >
> > At the same time the *IgniteAtomicLong* doesn't throw such exception (at
> > least I can't reproduce it).
> >
> > Please help me to clarify flowing questions:
> > 1. Is it recommended to handle topology exception in the same way? Is
> there
> > any public documentation about that?
> > 2. What kind of distributed operations(cache updates, open data stream,
> > atomic) are recommended to handle the ClusterTopologyException?
> >
> > --
> >
> > Regards
> >
> > Pavel Vinokurov
> >
>



-- 

Regards

Pavel Vinokurov

[jira] [Created] (IGNITE-7968) IgniteAtomicSequence.incrementAndGet throws ClusterTopologyException: Failed to acquire lock for keys

[Pavel Vinokurov (JIRA)]

Pavel Vinokurov created IGNITE-7968:
---

 Summary: IgniteAtomicSequence.incrementAndGet throws 
ClusterTopologyException: Failed to acquire lock for keys
 Key: IGNITE-7968
 URL: https://issues.apache.org/jira/browse/IGNITE-7968
 Project: Ignite
  Issue Type: Bug
  Components: data structures
Affects Versions: 2.4, 2.3
Reporter: Pavel Vinokurov
 Attachments: AtomicSeqAndClusterTopologyReproducer.java

When a primary node for a atomic sequnce has left cluster, 
IgniteAtomicSequence.incrementAndGet  could throw ClusterTopologyException.
The reproducer is attached. 




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3641: Keep marshaller mappings in consistent state: bac...

[dmekhanikov]

GitHub user dmekhanikov opened a pull request:

https://github.com/apache/ignite/pull/3641

Keep marshaller mappings in consistent state: backport to 2.3



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.3.3.b1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3641.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3641


commit be91bbf8bcb7e9c71fe6d3bea0f79763f9606558
Author: Krzysztof Chmielewski 
Date:   2017-10-10T14:50:59Z

Fixed "IGNITE-6234 Initialize schemaIds to empty set if schemas field is 
null during the deserialization".

Signed-off-by: nikolay_tikhonov 

commit 08389601728512dc4e7fa5b953f5afe34ae4506f
Author: AMRepo 
Date:   2017-10-10T08:57:20Z

IGNITE-6545: Failure during Ignite Service.cancel() can break normal 
shutdown process. This closes #2807.

(cherry picked from commit 8ffa109)

commit 57547b5afae059a0a6dfa13c08b2e0b6c0e96ebd
Author: devozerov 
Date:   2017-10-13T09:34:35Z

Merge branch 'ignite-2.3.1' into ignite-2.3.2

commit 08798f8e47bdfdd68a557385ed2ce98b4bb1609a
Author: devozerov 
Date:   2017-10-13T11:12:44Z

IGNITE-6605: SQL: common backup filter. This closes #2836.

commit 2b59a241de3935a338842b8fc3221aedc8e11e1d
Author: devozerov 
Date:   2017-10-16T07:33:36Z

IGNITE-6631: Minor improvements to GridH2KeyValueRowOnheap. This closes 
#2855.

commit 98438c954c5f9a08634cf3132361268456397864
Author: devozerov 
Date:   2017-10-16T09:38:54Z

IGNITE-6632: SQL: simplified GridH2Row inheritance tree. This closes #2856.

commit 95b7ab518dd3c3db6fcc5142c2ee85da2516c2b6
Author: devozerov 
Date:   2017-10-16T10:37:11Z

IGNITE-6634: Removed IgniteDistributedJoinTestSuite. It's tests are 
distributed between "Query" and "Query 2" suites. This closes #2857.

commit 9c91deff877ebc0eed84559d4abca71408e3cb0a
Author: devozerov 
Date:   2017-10-16T13:46:13Z

Merge branch 'ignite-2.3.1' into ignite-2.3.2

commit 911ab7ab7a8a6968d219b053adb2338477738506
Author: Alexey Popov 
Date:   2017-10-17T11:45:42Z

IGNITE-6627 .NET: Fix serialization of enums within generic collections

* Fix EnumEqualityComparer serialization
* Fix enum arrays serialization
* Fix empty objects missing metadata

This closes #2864

commit 3ba374c319ac7048a05871692060e2f143d6acdf
Author: Alexey Kuznetsov 
Date:   2017-10-06T17:11:37Z

IGNITE-6463 Web Console: Fixed output of big numbers in SQL query results.
(cherry picked from commit 35589a7)

commit b67feb0f175bfbd6ffbefe82a8d693c8ab7d4213
Author: Vasiliy Sisko 
Date:   2017-10-09T10:55:23Z

IGNITE-5767 Web console: Use byte array type instead of java.lang.Object 
for binary JDBC types.
(cherry picked from commit 3184437)

commit 8e1560322b87d79b3d3250832a3969ac4032d6fc
Author: Alexey Kuznetsov 
Date:   2017-10-06T18:10:08Z

IGNITE-6574 Remove pending requests in case STATUS_AUTH_FAILURE && 
credentials == null.
(cherry picked from commit 85261a3)

commit 7a0300ae35894c389b126e95615f720a99a3d360
Author: devozerov 
Date:   2017-10-18T11:18:08Z

Merge branch 'ignite-2.3.1' into ignite-2.3.2

commit ad01f9b099d0bf92537378859ad6d5a52de57748
Author: Alexey Kuznetsov 
Date:   2017-10-19T02:43:20Z

IGNITE-6647 Web Console: Implemented support of schema migration scripts.
(cherry picked from commit c65399c)

commit 0c66344bc752dac98b256dd140fcab95d1662862
Author: Pavel Tupitsyn 
Date:   2017-10-19T09:36:39Z

IGNITE-6627 .NET: Fix repeated known metadata updates

This closes #2876

commit 1b8abd214ed2afcd3fd1f6a4c71a19d6fe1a4b01
Author: Alexey Kuznetsov 
Date:   2017-10-20T04:23:23Z

IGNITE-6647 Added missing Mongo injector.
(cherry picked from commit 173ecef)

commit a221066b3d029afc392be704a810c0e830fc0c49
Author: Alexey Kuznetsov 
Date:   2017-10-20T14:15:02Z

IGNITE-6647 Web Console: Added folder for modules migrations.
(cherry picked from commit 3700717)

commit da8a9d5a968ba071697a28adb01bc59f80d1893c
Author: Pavel Tupitsyn 
Date:   2017-10-23T08:55:33Z

Merge branch 'ignite-2.3.1' into ignite-2.3.2

# Conflicts:
#   
modules/platforms/dotnet/Apache.Ignite.Core.Tests/Apache.Ignite.Core.Tests.csproj

commit 69fdac3acf768ecb9df80d4412c4de5ffd5bc4f5
Author: Dmitriy Shabalin 
Date:   2017-10-23T09:09:47Z

IGNITE-5909 Added list editable component.
(cherry picked from commit 01daee6)

commit 4a2c38333c112d4956d6394667672c1470503435
Author: apopov 
Date:   2017-10-24T08:56:33Z

IGNITE-6362 NPE in Log4J2Logger

commit 089ebecb3e5962c7a38afd01bd18c77feb23d155
Author: vsisko 
Date:   2017-10-25T04:23:11Z

IGNITE-6671 Web Agent: Fixed data type conversion for Oracle NUMBER(N) data 
types.
(cherry picked from commit 93be8ea)

commit 1e56de86525a79c895eb

[jira] [Created] (IGNITE-7967) DataRegionMetrics#totalAllocatedPages is invalid if metrics were enabled during runtime

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-7967:


 Summary: DataRegionMetrics#totalAllocatedPages is invalid if 
metrics were enabled during runtime
 Key: IGNITE-7967
 URL: https://issues.apache.org/jira/browse/IGNITE-7967
 Project: Ignite
  Issue Type: Bug
  Components: persistence
Affects Versions: 2.3
Reporter: Alexey Goncharuk
 Fix For: 2.5


{{totalAllocatedPages}} is updated during runtime via callback. When metrics 
are enabled at runtime, some pages may be already allocated, thus the metric 
shows invalid value.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Ignite Platform .NET Inspections problems

[Petr Ivanov]

I see. Will wait for result tomorrow.



> On 15 Mar 2018, at 15:40, Pavel Tupitsyn  wrote:
> 
> Just waiting for automatic build in master branch
> 
> On Thu, Mar 15, 2018 at 3:39 PM, Petr Ivanov  wrote:
> 
>> Nice news, Pavel!
>> 
>> Can you point to the exact build where you are checking changes, please?
>> 
>> 
>> 
>>> On 15 Mar 2018, at 15:34, Pavel Tupitsyn  wrote:
>>> 
>>> Hi Petr,
>>> 
>>> I've fixed that yesterday [1], waiting for the build to verify.
>>> 
>>> [1]
>>> https://github.com/apache/ignite/commit/a69e1acba8a69f1818fa9182a11673
>> 02f81b93ec
>>> 
>>> On Thu, Mar 15, 2018 at 1:07 PM, Petr Ivanov 
>> wrote:
>>> 
 HI, Pavel.
 
 
 After TeamCity update from 2017.1.3 to 2017.2.2 build Ignite Platform
>> .NET
 Inspections [1] started to show 12-14 failed inspections.
 Can you help and check settings for this build, please? Or give any
 recommendations on how to fix this?
 
 Thanks!
 
 
 [1] https://ci.ignite.apache.org/viewType.html?buildTypeId=
 IgniteTests24Java8_IgnitePlatformNetInspections
 
>> 
>> 

Re: Ignite Platform .NET Inspections problems

[Pavel Tupitsyn]

Just waiting for automatic build in master branch

On Thu, Mar 15, 2018 at 3:39 PM, Petr Ivanov  wrote:

> Nice news, Pavel!
>
> Can you point to the exact build where you are checking changes, please?
>
>
>
> > On 15 Mar 2018, at 15:34, Pavel Tupitsyn  wrote:
> >
> > Hi Petr,
> >
> > I've fixed that yesterday [1], waiting for the build to verify.
> >
> > [1]
> > https://github.com/apache/ignite/commit/a69e1acba8a69f1818fa9182a11673
> 02f81b93ec
> >
> > On Thu, Mar 15, 2018 at 1:07 PM, Petr Ivanov 
> wrote:
> >
> >> HI, Pavel.
> >>
> >>
> >> After TeamCity update from 2017.1.3 to 2017.2.2 build Ignite Platform
> .NET
> >> Inspections [1] started to show 12-14 failed inspections.
> >> Can you help and check settings for this build, please? Or give any
> >> recommendations on how to fix this?
> >>
> >> Thanks!
> >>
> >>
> >> [1] https://ci.ignite.apache.org/viewType.html?buildTypeId=
> >> IgniteTests24Java8_IgnitePlatformNetInspections
> >>
>
>

Re: Ignite Platform .NET Inspections problems

[Petr Ivanov]

Nice news, Pavel!

Can you point to the exact build where you are checking changes, please?



> On 15 Mar 2018, at 15:34, Pavel Tupitsyn  wrote:
> 
> Hi Petr,
> 
> I've fixed that yesterday [1], waiting for the build to verify.
> 
> [1]
> https://github.com/apache/ignite/commit/a69e1acba8a69f1818fa9182a1167302f81b93ec
> 
> On Thu, Mar 15, 2018 at 1:07 PM, Petr Ivanov  wrote:
> 
>> HI, Pavel.
>> 
>> 
>> After TeamCity update from 2017.1.3 to 2017.2.2 build Ignite Platform .NET
>> Inspections [1] started to show 12-14 failed inspections.
>> Can you help and check settings for this build, please? Or give any
>> recommendations on how to fix this?
>> 
>> Thanks!
>> 
>> 
>> [1] https://ci.ignite.apache.org/viewType.html?buildTypeId=
>> IgniteTests24Java8_IgnitePlatformNetInspections
>> 

Re: Ignite Platform .NET Inspections problems

[Pavel Tupitsyn]

Hi Petr,

I've fixed that yesterday [1], waiting for the build to verify.

[1]
https://github.com/apache/ignite/commit/a69e1acba8a69f1818fa9182a1167302f81b93ec

On Thu, Mar 15, 2018 at 1:07 PM, Petr Ivanov  wrote:

> HI, Pavel.
>
>
> After TeamCity update from 2017.1.3 to 2017.2.2 build Ignite Platform .NET
> Inspections [1] started to show 12-14 failed inspections.
> Can you help and check settings for this build, please? Or give any
> recommendations on how to fix this?
>
> Thanks!
>
>
> [1] https://ci.ignite.apache.org/viewType.html?buildTypeId=
> IgniteTests24Java8_IgnitePlatformNetInspections
>

Re: IEP-14: Ignite failures handling (Discussion)

[Dmitry Pavlov]

Hi Dmitriy,

It seems, here everyone agrees that killing the process will give a more
guaranteed result. The question is that the majority in the community does
not consider this to be acceptable in case Ignite as started as embedded
lib (e.g. from Java, using Ignition.start())

What can help to accept the community's opinion? Let's remember Apache
principle: "community first".

If release 2.5 will show us it was inpractical, we will change default to
kill even for library. What do you think?

Sincerely,
Dmitriy Pavlov

чт, 15 мар. 2018 г. в 5:48, Dmitriy Setrakyan :

> On Wed, Mar 14, 2018 at 7:12 PM, Andrey Kornev 
> wrote:
>
> > I'm not disagreeing with you, Dmitriy.
> >
> > What I'm trying to say is that if we assume that a serious enough bug or
> > some environmental issue prevents Ignite node from functioning correctly,
> > then it's only logical to assume that Ignite process is completely hosed
> > (for example, due to a very very long STW pause) and can't make any
> > progress at all. In a situation like this the application can't reason
> > about the process state, and the process itself may not be able to even
> > kill itself. The only reliable way to handle cases like that is to have
> an
> > external observer (a health monitoring tool) that is not itself affected
> by
> > the bug or the env issue and can either make a decision by itself or
> send a
> > notification to the SRE team.
> >
>
> Agree about the external observers, but that is something a user should do
> outside of Ignite.
>
>
> > In my previous post I only suggest to go easy on the "cleverness" of the
> > self-monitoring implementation as IMHO it won't be used much in
> production
> > environment. I think Ignite as it is already provides sufficient means
> > of monitoring its health (they may or may not be robust enough, which is
> a
> > different issue).
> >
>
> The approach I am suggesting is pretty simple - "kill" the process in case
> of a critical error. The only intelligence I would like to add is to
> attempt shutting down the Ignite node gracefully before the "kill" is
> executed. If a node is shutdown gracefully, then the restart procedure will
> be faster, so it is worthwhile to try.
>
> Some of the critical errors include running out of disk, memory, loosing
> Ignite system threads, etc... These errors are truly unrecoverable from the
> application stand point and should mostly be handled with a process restart
> anyway.
>
> D.
>

[jira] [Created] (IGNITE-7966) MVCC TX Review cache/Ignite configuration for MVCC needs

[Igor Seliverstov (JIRA)]

Igor Seliverstov created IGNITE-7966:


 Summary: MVCC TX Review cache/Ignite configuration for MVCC needs
 Key: IGNITE-7966
 URL: https://issues.apache.org/jira/browse/IGNITE-7966
 Project: Ignite
  Issue Type: Task
  Components: cache, sql
Reporter: Igor Seliverstov


We need to rethink Ignite and Cache configuration to have a control on such 
parameters as VACUUM frequency, TxLog data-region parameters, whether MVCC 
enabled or not for particular cache etc. Most of such parameters are hardcoded 
now.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3640: IGNITE-7965: Robin-hood hashing may fail with neg...

[dspavlov]

GitHub user dspavlov opened a pull request:

https://github.com/apache/ignite/pull/3640

IGNITE-7965: Robin-hood hashing may fail with negative index in case …

…backward shift finished with full table scan

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7965

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3640.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3640


commit a1ec7b9b9d9cafd6ab9169c4955c08ff7921c503
Author: dpavlov 
Date:   2018-03-15T10:32:17Z

IGNITE-7965: Robin-hood hashing may fail with negative index in case 
backward shift finished with full table scan




---

Re: IGNITE-5357 is ready for review (Replicated cache reads load balancing)

[Vyacheslav Daradur]

Dmitry, I'm looking forward to the news.

Thanks in advance!

On Thu, Mar 15, 2018 at 1:10 PM, Dmitry Pavlov  wrote:
> I would like to take a look to code now, if you don't mind.
>
> But I need to check if there is critical bug introduced to Ignite by my
> recent fix. So I need some time to research bug, and then can come back to
> review.
>
> чт, 15 мар. 2018 г. в 13:04, Vyacheslav Daradur :
>>
>> So, what's the next step?
>>
>> On Thu, Mar 15, 2018 at 12:27 AM, Dmitry Pavlov 
>> wrote:
>> > Yes, I think I could move IgniteReproducingSuite to dev-utils module
>> > later.
>> > Thank you for this idea.
>> >
>> > Yes, It is probably it was Queries test flaky'ness.
>> >
>> > I hope Vladimir, you will find some time to make query tests more
>> > stable. It
>> > is not friendly to community members if their patches are rejected by
>> > reasons not related to their change.
>> >
>> > Any assistance from the rest of community here is also appreciated.
>> >
>> > ср, 14 мар. 2018 г. в 22:24, Vyacheslav Daradur :
>> >>
>> >> Thank you for the advice!
>> >>
>> >> Unfortunately, *IgniteReproducingSuite* is in the core module while
>> >> *IgniteSqlSplitterSelfTest* in the ignite-indexing module that means I
>> >> am not able to add the test in this test suite without addition
>> >> cycling dependency.
>> >>
>> >> I'd recommend you detaching *IgniteReproducingSuite* as a separate
>> >> module in the project to include the test suites from any module in
>> >> the project.
>> >>
>> >>
>> >> But I've prepared *Ignite Queries* in the same way as you suggested in
>> >> *IgniteReproducingSuite* [1] and ran all tests in
>> >> *IgniteSqlSplitterSelfTest* 100 times [2].
>> >>
>> >> >> IgniteBinaryCacheQueryTestSuite:
>> >> >>
>> >> >> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
>> >> >> (fail rate 0,0%)
>> >> For this test "Green lite" 100 times of 100.
>> >>
>> >> Green lite for all tests in *IgniteSqlSplitterSelfTest* in the latest
>> >> build of main PR [3].
>> >>
>> >>
>> >> [1]
>> >>
>> >> https://github.com/daradurvs/ignite/blob/fd6abc915838599c2ebab3f803f90f2e641e8892/modules/indexing/src/test/java/org/apache/ignite/testsuites/IgniteCacheQuerySelfTestSuite.java
>> >> [2] https://ci.ignite.apache.org/viewLog.html?buildId=1136780
>> >> [3] https://ci.ignite.apache.org/viewLog.html?buildId=1136685
>> >>
>> >> On Wed, Mar 14, 2018 at 7:55 PM, Dmitry Pavlov 
>> >> wrote:
>> >> > It is possible that test is failing only on agents and is always
>> >> > successfull
>> >> > locally.
>> >> >
>> >> > For researching such test there was "Ignite reproducing suite"
>> >> > introduced
>> >> > early. This suite intentionally left blank on TC. Correspondent suite
>> >> > in
>> >> > code is IgniteReproducingSuite.
>> >> >
>> >> > You may add some extra debug info into test. Add this test in
>> >> > IgniteReproducingSuite in code and then start suite on TC several
>> >> > times.
>> >> >
>> >> > ср, 14 мар. 2018 г. в 19:42, Vyacheslav Daradur
>> >> > :
>> >> >>
>> >> >> Dmitry, as I've written here before: I checked this test locally,
>> >> >> many
>> >> >> times (didn't have any falling on 100 starts).
>> >> >>
>> >> >> On Wed, Mar 14, 2018 at 7:31 PM, Dmitry Pavlov
>> >> >> 
>> >> >> wrote:
>> >> >> > Hi, I've found test which never failed on master, but fails in
>> >> >> > branch
>> >> >> >
>> >> >> >  Ignite Queries [ tests 1 ]
>> >> >> >
>> >> >> >  IgniteBinaryCacheQueryTestSuite:
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
>> >> >> > (fail rate 0,0%)
>> >> >> >
>> >> >> >
>> >> >> > ср, 14 мар. 2018 г. в 19:26, Dmitry Pavlov
>> >> >> > :
>> >> >> >>
>> >> >> >> Hi, let me check TC run
>> >> >> >>
>> >> >> >> вт, 13 мар. 2018 г. в 9:22, Vyacheslav Daradur
>> >> >> >> :
>> >> >> >>>
>> >> >> >>> Dmitry,
>> >> >> >>>
>> >> >> >>> Nickolay accepted PR changes at Upsource [1].
>> >> >> >>>
>> >> >> >>> Latest ci.build [2] looks good in comparison with master [3].
>> >> >> >>>
>> >> >> >>> Following tests passed locally:
>> >> >> >>> CacheAffinityCallSelfTest.testAffinityCallFromClientRestartNode
>> >> >> >>> CacheAffinityCallSelfTest.testAffinityCallRestartNode
>> >> >> >>>
>> >> >> >>>
>> >> >> >>> IgniteOptimisticTxSuspendResumeMultiServerTest.testTxTimeoutOnSuspend
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
>> >> >> >>>
>> >> >> >>>
>> >> >> >>> [1] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-509
>> >> >> >>> [2] https://ci.ignite.apache.org/viewLog.html?buildId=1134466
>> >> >> >>> [3] https://ci.ignite.apache.org/viewLog.html?buildId=1134372
>> >> >> >>>
>> >> >> >>> On Mon, Mar 5, 2018 at 7:16 PM, Vyacheslav Daradur
>> >> >> >>> 
>> >> >> >>> wrote:
>> >> >> >>> > Dmitry, I saw them, but it looks like just randomness.
>> >> >> >>> >
>> >> >> >>> > I've checked it locally several times.
>> >> >> >>> >

Re: Stop nodes after test by default - IGNITE-6842

[Maxim Muzafarov]

Dmtry,

Can we proceed with this change?
I've done with fixing review comments and tests that you mentioned before.

TC: https://ci.ignite.apache.org/viewLog.html?buildId=1138151
JIRA: https://issues.apache.org/jira/browse/IGNITE-6842
Upsource: https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
PR: https://github.com/apache/ignite/pull/3542



вт, 6 мар. 2018 г. в 20:42, Dmitry Pavlov :

> Ok, thank you.
>
> Please let me know when we can proceed with review
> https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
>
>
> вт, 6 мар. 2018 г. в 20:17, Maxim Muzafarov :
>
> > Hello Dmitry,
> >
> > Yes, I've updated test classes as you metioned before.
> > Now i'm fixing review comments. Within next few days I'll prepare final
> > version of this PR.
> >
> > вт, 6 мар. 2018 г. в 20:12, Dmitry Pavlov :
> >
> > > Hi Maxim,
> > >
> > > are there any news on these test fails?
> > >
> > > Is issue ready for review?
> > >
> > > Sincerely,
> > > Dmitiry Pavlov
> > >
> > > вт, 27 февр. 2018 г. в 17:12, Dmitry Pavlov :
> > >
> > > > Hi, thank you!
> > > >
> > > > I've found several suspicious fails: such test fails have rate less
> > than
> > > > 1%, it is probably new failures.
> > > >
> > > > It would be great if we can fix it before merge. Could you address
> this
> > > > fails?
> > > >
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > > >
> > > > IgniteCacheTestSuite5: IgniteCacheStoreCollectionTest.testStoreMap
> > (fail
> > > > rate 0,0%)
> > > > IgniteCacheTestSuite5:
> > > > CacheLateAffinityAssignmentTest.testDelayAssignmentClientJoin (fail
> > rate
> > > > 0,0%)
> > > > IgniteCacheWithIndexingTestSuite:
> > > > CacheRandomOperationsMultithreadedTest.testAtomicOffheapEviction
> (fail
> > > rate
> > > > 0,0%)
> > > > IgniteCacheWithIndexingTestSuite:
> > > >
> > CacheRandomOperationsMultithreadedTest.testAtomicOffheapEvictionIndexing
> > > > (fail rate 0,0%)
> > > > IgniteCacheWithIndexingTestSuite:
> > > > CacheRandomOperationsMultithreadedTest.testTxOffheapEviction (fail
> rate
> > > > 0,0%)
> > > > IgniteCacheWithIndexingTestSuite:
> > > > CacheRandomOperationsMultithreadedTest.testTxOffheapEvictionIndexing
> > > (fail
> > > > rate 0,0%)
> > > >
> > > > IgniteBinarySimpleNameMapperCacheFullApiTestSuite:
> > > >
> > >
> >
> GridCachePartitionedNearDisabledMultiNodeWithGroupFullApiSelfTest.testWithSkipStoreTx
> > > > (fail rate 0,0%)
> > > >
> > > > вт, 27 февр. 2018 г. в 17:04, Maxim Muzafarov :
> > > >
> > > >> Yep, link may not be correct.
> > > >>
> > > >> Here is correct version:
> > > >> TC: *
> > > >>
> > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&branch_IgniteTests24Java8=pull%2F3542%2Fhead
> > > >> <
> > > >>
> > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&branch_IgniteTests24Java8=pull%2F3542%2Fhead
> > > >> >*
> > > >>
> > > >>
> > > >> вт, 27 февр. 2018 г. в 16:41, Dmitry Pavlov  >:
> > > >>
> > > >> > Hi Maxim,
> > > >> >
> > > >> > could you please provide link to TC run on your PR? It seems link
> > > >> provided
> > > >> > points to run of master. In changes field you may select
> > > pull/3542/head
> > > >> > before starting RunAll.
> > > >> >
> > > >> > Igniters,
> > > >> >
> > > >> > this change is related to our test framework, so change may affect
> > > your
> > > >> > tests. Please join to review
> > > >> > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
> > > >> >
> > > >> > Sincerely,
> > > >> > Dmitriy Pavlov
> > > >> >
> > > >> > вт, 27 февр. 2018 г. в 16:14, Maxim Muzafarov  >:
> > > >> >
> > > >> > > Hi all,
> > > >> > >
> > > >> > > I think, I've done with this issue, what should we do next?
> > > >> > >
> > > >> > > PR: https://github.com/apache/ignite/pull/3542
> > > >> > > Upsource:
> > > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
> > > >> > > TC:
> > > >> > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://ci.ignite.apache.org/viewModification.html?modId=723895&personal=false&buildTypeId=&tab=vcsModificationTests
> > > >> > > JIRA: https://issues.apache.org/jira/browse/IGNITE-6842
> > > >> > >
> > > >> > >
> > > >> > > чт, 22 февр. 2018 г. в 14:12, Dmitry Pavlov <
> > dpavlov@gmail.com
> > > >:
> > > >> > >
> > > >> > > > Hi Maxim,
> > > >> > > >
> > > >> > > > Thank you.
> > > >> > > >
> > > >> > > > To my mind stopAllGrids call should be kept in
> afterTestsStop().
> > > If
> > > >> > > > developer forgot to call super(), he will see exception from
> > > >> > > > beforeTestsStart()
> > > >> > > > added by you.
> > > >> > > >
> > > >> > > > If you think patch is ready to be reviewed, please change JIRA
> > > >> status
> > > >> > to
> > > >> > > > "Patch Available".
> > > >> > > >
> > > >> > > > Optionally you can create Upsource review. It would be easier
> > for
> > > >> > > multiple
> > > >> > > > reviewers to handle this patch. This test framework is used by
> > all
> > > >> > > Igniters
> > > >> > > > so Upsource would be good option (
> > > >> https://revie

[jira] [Created] (IGNITE-7965) Robin-hood hashing may fail with negative index in case backward shift finished with full table scan

[Dmitriy Pavlov (JIRA)]

Dmitriy Pavlov created IGNITE-7965:
--

 Summary: Robin-hood hashing may fail with negative index in case 
backward shift finished with full table scan
 Key: IGNITE-7965
 URL: https://issues.apache.org/jira/browse/IGNITE-7965
 Project: Ignite
  Issue Type: Bug
  Components: persistence
Affects Versions: 2.5
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov
 Fix For: 2.5


Introduced in [IGNITE-7638] Implemented robin-hood hashing for FullPageIdTable


Found by [~Jokser]

test
{noformat}
 @Test
public void testShortSize() throws Exception {
withMap(map -> {
  map.put(1, 1, 0, 0);
  map.put(2, 0, 1, 1);
  map.remove(1, 1);
}, 2);
}
{noformat}

Problematic code
org/apache/ignite/internal/processors/cache/persistence/pagemem/RobinHoodBackwardShiftHashMap.java:321



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Why does DataStreamer swallow exceptions?

[Ilya Kasnacheev]

Hello!

I have filed an issue https://issues.apache.org/jira/browse/IGNITE-7962 -
going to work on it soon.

I have also filed a ticket about usability issue:
https://issues.apache.org/jira/browse/IGNITE-7963
Feedback is appreciated.

Regards,

-- 
Ilya Kasnacheev

2018-03-14 23:35 GMT+03:00 Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Ilya,
>
> Yes, this is definitely a bug, please create a ticket.
>
> -Val
>
> On Wed, Mar 14, 2018 at 2:16 AM, Ilya Kasnacheev <
> ilya.kasnach...@gmail.com>
> wrote:
>
> > Hello Val!
> >
> > No, this does NOT happen. If you collect future, call get() on it later,
> it
> > will finish normally despite exceptions in server log and entry not being
> > written. I will do some digging to figure out why this happens exactly.
> >
> > There's also another huge problems with DataStreamer's futures. They
> never
> > finish unless you call flush() on DS before calling get() on futures.
> > I think this is a colossal usability problem (I'm pretty sure I've seen
> > numerous messages about it on userlist) and I'll fill an issue if nobody
> is
> > objecting.
> >
> > Ilya.
> >
> > --
> > Ilya Kasnacheev
> >
> > 2018-03-05 22:50 GMT+03:00 Valentin Kulichenko <
> > valentin.kuliche...@gmail.com>:
> >
> > > Ilya,
> > >
> > > IgniteDataStreamer#addData method returns future which should be
> > completed
> > > with error if one is thrown on server side. Does this happen or not?
> > >
> > > -Val
> > >
> > > On Mon, Mar 5, 2018 at 4:10 AM, Nikolay Izhikov 
> > > wrote:
> > >
> > > > Hello, Ilya.
> > > >
> > > > > I think it's time to end this, if that was the case. DataStreamer
> > > should
> > > > > not be a special case and it should guarantee data safety. WDYT?
> > > >
> > > > +1 from me.
> > > >
> > > > I'm also facing this issue.
> > > >
> > > > Ticket - https://issues.apache.org/jira/browse/IGNITE-7756
> > > > Discussion - http://apache-ignite-developers.2346864.n4.nabble.
> > > > com/IgniteDataStreamer-silently-fails-on-a-server-node-td27239.html
> > > >
> > > >
> > > >
> > > > В Пн, 05/03/2018 в 14:46 +0300, Ilya Kasnacheev пишет:
> > > > > Dear Igniters, why do I have a hunch that DataStreamer would
> readily
> > > > > swallow exceptions?
> > > > >
> > > > > DataStreamerImpl:1756 swallows marshalling error, lines 1774 & 1781
> > eat
> > > > > deployment errors.
> > > > >
> > > > > Some people are worried they can fill a leaking vessel without
> > noticing
> > > > > anything off.
> > > > > Also in line 2156 fsync() on WAL can throw exceptions, which will
> be
> > > > > swallowed, and IMP this fsync doesn't belong to DataStreamer code.
> > > > >
> > > > > This question is not purely theoretical, we have also replaced one
> of
> > > > these
> > > > > eaters with throw with IGNITE-7519.
> > > > >
> > > > > There was a fault in PDS implementation, which did not propagate to
> > > > client.
> > > > > A serious issue IMHO.
> > > > >
> > > > > As a data streamer user I will expect that flush()/close() will
> throw
> > > any
> > > > > pending exceptions and will only be silent if all data landed
> safely
> > in
> > > > the
> > > > > cluster.
> > > > >
> > > > > I also have this feeling that DataStreamer was written using very
> > > > internal
> > > > > APIs so that it can compromise guarantees that cache and SQL APIs
> are
> > > > bound
> > > > > by. I think I've heard something about not recovering properly in
> > case
> > > of
> > > > > node failures.
> > > > > I think it's time to end this, if that was the case. DataStreamer
> > > should
> > > > > not be a special case and it should guarantee data safety. WDYT?
> > > > >
> > > >
> > >
> >
>

[jira] [Created] (IGNITE-7963) Futures from DataStreamer.addData() fail to complete if DataStreamer.flush() is never called

[Ilya Kasnacheev (JIRA)]

Ilya Kasnacheev created IGNITE-7963:
---

 Summary: Futures from DataStreamer.addData() fail to complete if 
DataStreamer.flush() is never called
 Key: IGNITE-7963
 URL: https://issues.apache.org/jira/browse/IGNITE-7963
 Project: Ignite
  Issue Type: Task
  Components: cache
Affects Versions: 2.3
Reporter: Ilya Kasnacheev
Assignee: Ilya Kasnacheev


DataStreamer.addData() will return futures for operation.
Thus the naive use of DataStreamer will look like this:

{code}
for (Data d : data)
 futs.add(dataStreamer.addData(d));

for (IgniteFuture f : futs)
f.get();

dataStreamer.close();
{code}

However, this does not work. Unless flush is called (manual or otherwisE), 
futures are not being processed. This code will likely hang on f.get().

The solution, IMHO, is to introduce dataStreamer-flushing clause in f.get().



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7964) Read a key from metastorage may hang on node restart

[Sergey Chugunov (JIRA)]

Sergey Chugunov created IGNITE-7964:
---

 Summary: Read a key from metastorage may hang on node restart
 Key: IGNITE-7964
 URL: https://issues.apache.org/jira/browse/IGNITE-7964
 Project: Ignite
  Issue Type: Bug
  Components: persistence
Affects Versions: 2.4
Reporter: Sergey Chugunov
Assignee: Sergey Chugunov
 Fix For: 2.5


MetaStorage handles *rmvId* counter incorrectly: it is not propagated to 
metapage and is not persisted to disk; on restart *rmvId* is set to 0 which 
leads to incorrect initialization of BPlusTree where all keys live.

As a consequence if there are more than 52 keys and more than 31 of them get 
updated than on next restart node hangs on attempt to read BaselineTopology 
info from metastore (because of endless looping inside BPlusTree starting from 
the root over and over again).
h2. Steps to reproduce
 # Start single node with persistence enabled.
 # Put 60 keys to metastore, update 40 of them.
 # Restart node.

h2. Expected behavior

Node starts just fine.
h2. Actual behavior

Node hangs, thread dump of starting thread shows looping on reading 
BaselineTopology from metastore.
h2. Workaround

No impact if no more than 52 keys are stored in metastore or keys are never 
updated.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7962) More cases of suppressed exceptions in IsolatedUpdater

[Ilya Kasnacheev (JIRA)]

Ilya Kasnacheev created IGNITE-7962:
---

 Summary: More cases of suppressed exceptions in IsolatedUpdater
 Key: IGNITE-7962
 URL: https://issues.apache.org/jira/browse/IGNITE-7962
 Project: Ignite
  Issue Type: Task
  Components: cache
Affects Versions: 2.4
Reporter: Ilya Kasnacheev
Assignee: Ilya Kasnacheev


IsolatedUpdater is a StreamReciever.

The contract for StreamReciever.recieve() is, @throws 
org.apache.ignite.IgniteException If failed.

However, there's a number of cases where this doesn't happen and exceptions are 
suppressed after being written in server log. Should replace (or augment) 
log.error()'s with throw new IgniteException().

See maillist discussion.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: IGNITE-5357 is ready for review (Replicated cache reads load balancing)

[Dmitry Pavlov]

I would like to take a look to code now, if you don't mind.

But I need to check if there is critical bug introduced to Ignite by my
recent fix. So I need some time to research bug, and then can come back to
review.

чт, 15 мар. 2018 г. в 13:04, Vyacheslav Daradur :

> So, what's the next step?
>
> On Thu, Mar 15, 2018 at 12:27 AM, Dmitry Pavlov 
> wrote:
> > Yes, I think I could move IgniteReproducingSuite to dev-utils module
> later.
> > Thank you for this idea.
> >
> > Yes, It is probably it was Queries test flaky'ness.
> >
> > I hope Vladimir, you will find some time to make query tests more
> stable. It
> > is not friendly to community members if their patches are rejected by
> > reasons not related to their change.
> >
> > Any assistance from the rest of community here is also appreciated.
> >
> > ср, 14 мар. 2018 г. в 22:24, Vyacheslav Daradur :
> >>
> >> Thank you for the advice!
> >>
> >> Unfortunately, *IgniteReproducingSuite* is in the core module while
> >> *IgniteSqlSplitterSelfTest* in the ignite-indexing module that means I
> >> am not able to add the test in this test suite without addition
> >> cycling dependency.
> >>
> >> I'd recommend you detaching *IgniteReproducingSuite* as a separate
> >> module in the project to include the test suites from any module in
> >> the project.
> >>
> >>
> >> But I've prepared *Ignite Queries* in the same way as you suggested in
> >> *IgniteReproducingSuite* [1] and ran all tests in
> >> *IgniteSqlSplitterSelfTest* 100 times [2].
> >>
> >> >> IgniteBinaryCacheQueryTestSuite:
> >> >>
> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
> >> >> (fail rate 0,0%)
> >> For this test "Green lite" 100 times of 100.
> >>
> >> Green lite for all tests in *IgniteSqlSplitterSelfTest* in the latest
> >> build of main PR [3].
> >>
> >>
> >> [1]
> >>
> https://github.com/daradurvs/ignite/blob/fd6abc915838599c2ebab3f803f90f2e641e8892/modules/indexing/src/test/java/org/apache/ignite/testsuites/IgniteCacheQuerySelfTestSuite.java
> >> [2] https://ci.ignite.apache.org/viewLog.html?buildId=1136780
> >> [3] https://ci.ignite.apache.org/viewLog.html?buildId=1136685
> >>
> >> On Wed, Mar 14, 2018 at 7:55 PM, Dmitry Pavlov 
> >> wrote:
> >> > It is possible that test is failing only on agents and is always
> >> > successfull
> >> > locally.
> >> >
> >> > For researching such test there was "Ignite reproducing suite"
> >> > introduced
> >> > early. This suite intentionally left blank on TC. Correspondent suite
> in
> >> > code is IgniteReproducingSuite.
> >> >
> >> > You may add some extra debug info into test. Add this test in
> >> > IgniteReproducingSuite in code and then start suite on TC several
> times.
> >> >
> >> > ср, 14 мар. 2018 г. в 19:42, Vyacheslav Daradur  >:
> >> >>
> >> >> Dmitry, as I've written here before: I checked this test locally,
> many
> >> >> times (didn't have any falling on 100 starts).
> >> >>
> >> >> On Wed, Mar 14, 2018 at 7:31 PM, Dmitry Pavlov <
> dpavlov@gmail.com>
> >> >> wrote:
> >> >> > Hi, I've found test which never failed on master, but fails in
> branch
> >> >> >
> >> >> >  Ignite Queries [ tests 1 ]
> >> >> >
> >> >> >  IgniteBinaryCacheQueryTestSuite:
> >> >> >
> >> >> >
> >> >> >
> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
> >> >> > (fail rate 0,0%)
> >> >> >
> >> >> >
> >> >> > ср, 14 мар. 2018 г. в 19:26, Dmitry Pavlov  >:
> >> >> >>
> >> >> >> Hi, let me check TC run
> >> >> >>
> >> >> >> вт, 13 мар. 2018 г. в 9:22, Vyacheslav Daradur
> >> >> >> :
> >> >> >>>
> >> >> >>> Dmitry,
> >> >> >>>
> >> >> >>> Nickolay accepted PR changes at Upsource [1].
> >> >> >>>
> >> >> >>> Latest ci.build [2] looks good in comparison with master [3].
> >> >> >>>
> >> >> >>> Following tests passed locally:
> >> >> >>> CacheAffinityCallSelfTest.testAffinityCallFromClientRestartNode
> >> >> >>> CacheAffinityCallSelfTest.testAffinityCallRestartNode
> >> >> >>>
> >> >> >>>
> IgniteOptimisticTxSuspendResumeMultiServerTest.testTxTimeoutOnSuspend
> >> >> >>>
> >> >> >>>
> >> >> >>>
> >> >> >>>
> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
> >> >> >>>
> >> >> >>>
> >> >> >>> [1] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-509
> >> >> >>> [2] https://ci.ignite.apache.org/viewLog.html?buildId=1134466
> >> >> >>> [3] https://ci.ignite.apache.org/viewLog.html?buildId=1134372
> >> >> >>>
> >> >> >>> On Mon, Mar 5, 2018 at 7:16 PM, Vyacheslav Daradur
> >> >> >>> 
> >> >> >>> wrote:
> >> >> >>> > Dmitry, I saw them, but it looks like just randomness.
> >> >> >>> >
> >> >> >>> > I've checked it locally several times.
> >> >> >>> > They failed only in one TeamCity's build of four.
> >> >> >>> >
> >> >> >>> > Started build once again to be sure.
> >> >> >>> >
> >> >> >>> > On Mon, Mar 5, 2018 at 6:59 PM, Dmitry Pavlov
> >> >> >>> > 
> >> >> >>> > wrote:
> >> >> >>> >> I can see Nikolay Izhikov as reviewer in Upsource.
> >> >> >>> >>
> >> >> >>> 

Ignite Platform .NET Inspections problems

[Petr Ivanov]

HI, Pavel.


After TeamCity update from 2017.1.3 to 2017.2.2 build Ignite Platform .NET 
Inspections [1] started to show 12-14 failed inspections.
Can you help and check settings for this build, please? Or give any 
recommendations on how to fix this?

Thanks!


[1] 
https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_IgnitePlatformNetInspections
 

Re: IGNITE-5357 is ready for review (Replicated cache reads load balancing)

[Vyacheslav Daradur]

So, what's the next step?

On Thu, Mar 15, 2018 at 12:27 AM, Dmitry Pavlov  wrote:
> Yes, I think I could move IgniteReproducingSuite to dev-utils module later.
> Thank you for this idea.
>
> Yes, It is probably it was Queries test flaky'ness.
>
> I hope Vladimir, you will find some time to make query tests more stable. It
> is not friendly to community members if their patches are rejected by
> reasons not related to their change.
>
> Any assistance from the rest of community here is also appreciated.
>
> ср, 14 мар. 2018 г. в 22:24, Vyacheslav Daradur :
>>
>> Thank you for the advice!
>>
>> Unfortunately, *IgniteReproducingSuite* is in the core module while
>> *IgniteSqlSplitterSelfTest* in the ignite-indexing module that means I
>> am not able to add the test in this test suite without addition
>> cycling dependency.
>>
>> I'd recommend you detaching *IgniteReproducingSuite* as a separate
>> module in the project to include the test suites from any module in
>> the project.
>>
>>
>> But I've prepared *Ignite Queries* in the same way as you suggested in
>> *IgniteReproducingSuite* [1] and ran all tests in
>> *IgniteSqlSplitterSelfTest* 100 times [2].
>>
>> >> IgniteBinaryCacheQueryTestSuite:
>> >> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
>> >> (fail rate 0,0%)
>> For this test "Green lite" 100 times of 100.
>>
>> Green lite for all tests in *IgniteSqlSplitterSelfTest* in the latest
>> build of main PR [3].
>>
>>
>> [1]
>> https://github.com/daradurvs/ignite/blob/fd6abc915838599c2ebab3f803f90f2e641e8892/modules/indexing/src/test/java/org/apache/ignite/testsuites/IgniteCacheQuerySelfTestSuite.java
>> [2] https://ci.ignite.apache.org/viewLog.html?buildId=1136780
>> [3] https://ci.ignite.apache.org/viewLog.html?buildId=1136685
>>
>> On Wed, Mar 14, 2018 at 7:55 PM, Dmitry Pavlov 
>> wrote:
>> > It is possible that test is failing only on agents and is always
>> > successfull
>> > locally.
>> >
>> > For researching such test there was "Ignite reproducing suite"
>> > introduced
>> > early. This suite intentionally left blank on TC. Correspondent suite in
>> > code is IgniteReproducingSuite.
>> >
>> > You may add some extra debug info into test. Add this test in
>> > IgniteReproducingSuite in code and then start suite on TC several times.
>> >
>> > ср, 14 мар. 2018 г. в 19:42, Vyacheslav Daradur :
>> >>
>> >> Dmitry, as I've written here before: I checked this test locally, many
>> >> times (didn't have any falling on 100 starts).
>> >>
>> >> On Wed, Mar 14, 2018 at 7:31 PM, Dmitry Pavlov 
>> >> wrote:
>> >> > Hi, I've found test which never failed on master, but fails in branch
>> >> >
>> >> >  Ignite Queries [ tests 1 ]
>> >> >
>> >> >  IgniteBinaryCacheQueryTestSuite:
>> >> >
>> >> >
>> >> > IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
>> >> > (fail rate 0,0%)
>> >> >
>> >> >
>> >> > ср, 14 мар. 2018 г. в 19:26, Dmitry Pavlov :
>> >> >>
>> >> >> Hi, let me check TC run
>> >> >>
>> >> >> вт, 13 мар. 2018 г. в 9:22, Vyacheslav Daradur
>> >> >> :
>> >> >>>
>> >> >>> Dmitry,
>> >> >>>
>> >> >>> Nickolay accepted PR changes at Upsource [1].
>> >> >>>
>> >> >>> Latest ci.build [2] looks good in comparison with master [3].
>> >> >>>
>> >> >>> Following tests passed locally:
>> >> >>> CacheAffinityCallSelfTest.testAffinityCallFromClientRestartNode
>> >> >>> CacheAffinityCallSelfTest.testAffinityCallRestartNode
>> >> >>>
>> >> >>> IgniteOptimisticTxSuspendResumeMultiServerTest.testTxTimeoutOnSuspend
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
>> >> >>>
>> >> >>>
>> >> >>> [1] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-509
>> >> >>> [2] https://ci.ignite.apache.org/viewLog.html?buildId=1134466
>> >> >>> [3] https://ci.ignite.apache.org/viewLog.html?buildId=1134372
>> >> >>>
>> >> >>> On Mon, Mar 5, 2018 at 7:16 PM, Vyacheslav Daradur
>> >> >>> 
>> >> >>> wrote:
>> >> >>> > Dmitry, I saw them, but it looks like just randomness.
>> >> >>> >
>> >> >>> > I've checked it locally several times.
>> >> >>> > They failed only in one TeamCity's build of four.
>> >> >>> >
>> >> >>> > Started build once again to be sure.
>> >> >>> >
>> >> >>> > On Mon, Mar 5, 2018 at 6:59 PM, Dmitry Pavlov
>> >> >>> > 
>> >> >>> > wrote:
>> >> >>> >> I can see Nikolay Izhikov as reviewer in Upsource.
>> >> >>> >>
>> >> >>> >> Nikolay, would you run review first?
>> >> >>> >>
>> >> >>> >> I've found several suspicious tests : Test fail rate is less
>> >> >>> >> than
>> >> >>> >> 1%,
>> >> >>> >> it is
>> >> >>> >> probably new failure
>> >> >>> >> IgniteCacheTestSuite2:
>> >> >>> >>
>> >> >>> >>
>> >> >>> >>
>> >> >>> >> GridCachePartitionedTxSingleThreadedSelfTest.testOptimisticReadCommittedRollback
>> >> >>> >> (fail rate 0,0%)
>> >> >>> >> IgniteCacheTestSuite2:
>> >> >>> >>
>> >> >>> >>
>> >> >>> >>
>> >> >>> >> GridCachePartitionedTxSingleThreadedSelfTest.testOptimisticRepe

[jira] [Created] (IGNITE-7961) Rebalance throughput requires optimization

[Ilya Lantukh (JIRA)]

Ilya Lantukh created IGNITE-7961:


 Summary: Rebalance throughput requires optimization
 Key: IGNITE-7961
 URL: https://issues.apache.org/jira/browse/IGNITE-7961
 Project: Ignite
  Issue Type: Improvement
Affects Versions: 2.4
Reporter: Ilya Lantukh
Assignee: Ilya Lantukh






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3639: IGNITE-7608

[SomeFire]

GitHub user SomeFire opened a pull request:

https://github.com/apache/ignite/pull/3639

IGNITE-7608



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/SomeFire/ignite ignite-7608

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3639.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3639


commit af134f9b9b20a95cacf9291b3010dff42b3dbe72
Author: Dmitrii Ryabov 
Date:   2018-03-05T15:14:14Z

IGNITE-7608: Sort keys in putAll/removeAll methods.

commit 5e419ca5e576486f928af7c1fa6e46b3d64c7152
Author: Dmitrii Ryabov 
Date:   2018-03-15T08:14:02Z

-lambdas

commit 28652a35c29dc145eef3f46c06fd20d23b220399
Author: Dmitrii Ryabov 
Date:   2018-03-15T09:50:42Z

+ other *All() methods




---

Re: Ignite contibutors page

[Dmitriy Shabalin]

I'm not mentioned on the community contributors page: 
https://ignite.apache.org/community/resources.html

pls add me, dmitriyff (Dmitriy Shabalin)



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

readme.io weird interface element

[vveider]

Hi, all!


Does anyone know what is this button with arrow and disk at the right corner
of versions list header? [1]
Seems that it does nothing.

[1] https://ibb.co/eJf5uc



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[jira] [Created] (IGNITE-7960) Wrong type name in cache metadata on query execution

[Vasiliy Sisko (JIRA)]

Vasiliy Sisko created IGNITE-7960:
-

 Summary: Wrong type name in cache metadata on query execution
 Key: IGNITE-7960
 URL: https://issues.apache.org/jira/browse/IGNITE-7960
 Project: Ignite
  Issue Type: Bug
  Components: cache, sql
Affects Versions: 2.5
Reporter: Vasiliy Sisko
Assignee: Vladimir Ozerov
 Attachments: Country.java, TestQuery.java

1) On run of demo mode in Web console in Queries page for every query showed 
error. F.e.:
Error: Table "CAR" not found.
After debug I notice that org.h2.schema.Schema#tablesAndViews map contains key 
with type name stored as "Car". In that case query parser require "CAR" name 
and query fail.
Caches created by invocation of java config 
org.apache.ignite.Ignite#getOrCreateCaches

I create simple reproducer (See files TestQuery.java and Country.java), but 
metadata and parser in that case use the same name "Car". 
Caches created by invocation of java config 
org.apache.ignite.Ignite#getOrCreateCaches

Configured in XML cache has normalised name of type "CAR". In that case parser 
too require "CAR" name and work without error.
Caches created in process of cluster activation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7959) Deadlock at GridDhtAtomicCache.lockEntries during parallel SQL delete

[Pavel Vinokurov (JIRA)]

Pavel Vinokurov created IGNITE-7959:
---

 Summary: Deadlock at GridDhtAtomicCache.lockEntries during 
parallel SQL delete
 Key: IGNITE-7959
 URL: https://issues.apache.org/jira/browse/IGNITE-7959
 Project: Ignite
  Issue Type: Bug
  Components: cache, sql
Affects Versions: 2.3
Reporter: Pavel Vinokurov
 Attachments: DeadlockOnDelete.java

Reproduce steps:
1. Run insert operations from single thread.
2. Run "delete from table limit 50" in 5 threads.

The reproducer is attached.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: IGNITE-6879

[Alexey Kukushkin]

Just found the fix is ready - I will review it today or tomorrow.

Re: IGNITE-6879

[Alexey Kukushkin]

Dmitry, Roman, I can review the fix - send me a code review link when it is
ready.

On Wed, Mar 14, 2018 at 5:14 PM, Dmitry Pavlov 
wrote:

> Igniters,
>
> it is about spring-data integration support.
>
> Who has intereset about styding this technology?
>
> Alexey Kukushkin would you like to be reviewer of this issue?
>
> Sincerely,
> Dmitriy Pavlov
>
> ср, 14 мар. 2018 г. в 1:08, Denis Magda :
>
>> Hello Roman,
>>
>> It's not a big deal, I've added you to the contributors' list in JIRA. Go
>> ahead and assign the ticket to yourself.
>>
>> Folks, anyway, who can review Roman's contribution that brings Spring 2.0
>> support to Ignite?
>>
>> --
>> Denis
>>
>>
>>
>> On Tue, Mar 13, 2018 at 2:21 PM, Роман Меерсон 
>> wrote:
>>
>> > Hello!
>> >
>> > I want to work on https://issues.apache.org/jira/browse/IGNITE-6879
>> issue.
>> > Following the rules here
>> > https://ignite.apache.org/community/contribute.html#contribute my Jira
>> > username is "homich" so assign me to this ticket please.
>> >
>> > P.S. I found this rules page after I made PR, so sorry for this
>> > inconvenience.
>> >
>> > Regards.
>> >
>>
>


-- 
Best regards,
Alexey

Re: Ignite contibutors page

[Alexey Zinoviev]

Dear Denis, could you please include me in list of contributors too?
https://github.com/zaleslaw


2018-03-14 22:04 GMT+03:00 Denis Magda :

> Turik,
>
> Done, you're in ;)
>
> On Tue, Mar 13, 2018 at 10:11 PM, techbysample  wrote:
>
> > Denis,
> >
> > Please add my name as well.
> >
> > Best
> > Turik Campbell
> >
> >
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >
>

Re: Apache Ignite RPM packaging (Stage II)

[Petr Ivanov]

*DEB package


> On 15 Mar 2018, at 10:35, Petr Ivanov  wrote:
> 
> Considering that DEV package for now is almost platform independent (its a 
> java application more or less), that package will work almost on any 
> DEB-based linux, including but not limited to Ubuntu, Debian, etc.
> The only restriction is existence of systemctl (systemd) service manager — we 
> are dependent on it.
> 
> Thats why, for instance, our RPM repository is called simply ‘rpm’ and 
> package has no arch or dist suffix — it will work on CentOS, RHEL, Fedora, 
> etc. with presence of aforementioned systemd.
> 
> 
> 
>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan  wrote:
>> 
>> Will Debian package work for Ubuntu?
>> 
>> D.
>> 
>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov  wrote:
>> 
>>> Not a problem, rather nuisance. Also, when we will move to official
>>> repositories, there can be a problem from OS community.
>>> 
>>> Concerning DEB packages — I plan to use RPM as base for DEB package build
>>> (package layout / install scripts) for speeding up things and excluding
>>> possible duplication and desynchronisation, so its a matter of ’sit and do’
>>> rather then some technical research. Thats why I rose discussion about
>>> future package architecture, so that after agreement I'm be able to pack
>>> both RPM and DEB identically.
>>> 
>>> Yet, if you insist, I can create DEB package according to current RPM
>>> layout in no time.
>>> 
>>> 
>>> 
 On 15 Mar 2018, at 04:53, Dmitriy Setrakyan 
>>> wrote:
 
 Peter,
 
 I don't think the package size of 280M is going to be a problem at all,
>>> but
 what you are suggesting can be an improvement down the road.
 
 In the mean time, I think our top priority should be to provide packages
 for Debian and Ubuntu. Having only RPMs is not nearly enough.
 
 Agree?
 
 D.
 
 On Wed, Mar 14, 2018 at 5:36 AM, vveider  wrote:
 
> Hi, Igniters!
> 
> 
> Release 2.4 is almost there, at least binary part of it, so I'd like to
> move
> forward to further improve and widen AI delivery through packages.
> As of now, Apache Ignite ships in RPM package weighing about 280M+ and,
>>> to
> improve usability and significantly reduce required download sizes, I
> purpose that in 2.5 release we introduce splitted delivery as follows:
> - CORE
> - bin
> - config
> - libs (!optional)
> - OPTIONAL LIBS
> - BENCHMARKS
> - DOCS (?)
> - EXAMPLES
> - .NET PLATFORM FILES
> - C++ PLATFORM FILES
> 
> This architecture, as I assume, will add flexibility (no reason to
>>> download
> all 280M+ of binaries where you are to run only core node functionality)
> and
> maintainability (you are in full control of what is installed on your
> system).
> 
> After successful architecture choice, same scheme are planned to be
>>> used in
> DEB packages as well.
> 
> WDYT?
> 
> 
> 
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> 
>>> 
>>> 
> 

Re: Apache Ignite RPM packaging (Stage II)

[Petr Ivanov]

Considering that DEV package for now is almost platform independent (its a java 
application more or less), that package will work almost on any DEB-based 
linux, including but not limited to Ubuntu, Debian, etc.
The only restriction is existence of systemctl (systemd) service manager — we 
are dependent on it.

Thats why, for instance, our RPM repository is called simply ‘rpm’ and package 
has no arch or dist suffix — it will work on CentOS, RHEL, Fedora, etc. with 
presence of aforementioned systemd.



> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan  wrote:
> 
> Will Debian package work for Ubuntu?
> 
> D.
> 
> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov  wrote:
> 
>> Not a problem, rather nuisance. Also, when we will move to official
>> repositories, there can be a problem from OS community.
>> 
>> Concerning DEB packages — I plan to use RPM as base for DEB package build
>> (package layout / install scripts) for speeding up things and excluding
>> possible duplication and desynchronisation, so its a matter of ’sit and do’
>> rather then some technical research. Thats why I rose discussion about
>> future package architecture, so that after agreement I'm be able to pack
>> both RPM and DEB identically.
>> 
>> Yet, if you insist, I can create DEB package according to current RPM
>> layout in no time.
>> 
>> 
>> 
>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan 
>> wrote:
>>> 
>>> Peter,
>>> 
>>> I don't think the package size of 280M is going to be a problem at all,
>> but
>>> what you are suggesting can be an improvement down the road.
>>> 
>>> In the mean time, I think our top priority should be to provide packages
>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
>>> 
>>> Agree?
>>> 
>>> D.
>>> 
>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider  wrote:
>>> 
 Hi, Igniters!
 
 
 Release 2.4 is almost there, at least binary part of it, so I'd like to
 move
 forward to further improve and widen AI delivery through packages.
 As of now, Apache Ignite ships in RPM package weighing about 280M+ and,
>> to
 improve usability and significantly reduce required download sizes, I
 purpose that in 2.5 release we introduce splitted delivery as follows:
 - CORE
  - bin
  - config
  - libs (!optional)
 - OPTIONAL LIBS
 - BENCHMARKS
 - DOCS (?)
 - EXAMPLES
 - .NET PLATFORM FILES
 - C++ PLATFORM FILES
 
 This architecture, as I assume, will add flexibility (no reason to
>> download
 all 280M+ of binaries where you are to run only core node functionality)
 and
 maintainability (you are in full control of what is installed on your
 system).
 
 After successful architecture choice, same scheme are planned to be
>> used in
 DEB packages as well.
 
 WDYT?
 
 
 
 --
 Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
 
>> 
>> 

Re: IgniteSet implementation: changes required

[Andrey Kuznetsov]

Dmitriy,

It's technically possible to produce both kinds of sets with
{{Ignite.set()}} call, but this will require to one more argument ('small'
vs 'large'). Doesn't it look less inuitive than separate
{{IgniteCache.asSet()}} ?

And of course, we don't want to leave existing implementation broken. Pavel
P. has prepared the fix as part of [1].

[1] https://issues.apache.org/jira/browse/IGNITE-5553

2018-03-15 5:40 GMT+03:00 Dmitriy Setrakyan :

> I am not sure I like the "asSet()" method. We already have Ignite.set(...)
> method and now introducing yet another one. The new design should fix the
> existing implementation. We cannot keep the broken implementation around
> and introduce yet another one. To be consistent, we should also stick to
> the IgniteSet abstraction.
>
> Just to be clear, I am in agreement that a non-collocated set should be
> based on a cache, but I do not see why the existing API cannot be re-used.
>
> D.
>
>
-- 
Best regards,
  Andrey Kuznetsov.

Re: Atomic sequence and topology exception

[Alexey Goncharuk]

Pavel,

The exception from an AtomicSequence looks like a bug to me. Ignite should
retry all operations that do not involve user's logic, this stands for
AtomicSequence. I believe such handling already present in AtomicLong, so
it should not be difficult to fix.

The only case when a user must handle TopologyException is an explicit
transaction. In this case, the transaction involves particular user logic
that cannot be captured and retried by Ignite, so the exception handling
should be covered by the user.

The way you handled the topology exception looks correct to me.

--AG

2018-03-14 20:47 GMT+03:00 Dmitry Pavlov :

> Hi Nikolay,
>
> How do you think which was idea / the best practice to handling exceptions
> here?
>
> Why exception here may have difference?
>
> Sincerely,
> Dmitriy Pavlov
>
> -- Forwarded message -
> From: Vinokurov Pavel 
> Date: вт, 13 мар. 2018 г. в 5:52
> Subject: Atomic sequence and topology exception
> To: 
>
>
> Igniters!
>
> I  have a few questions related to a replicated atomic sequence and an
> topology exception.
> In my case after one server node has left cluster, on a client node the
> execution of the *IgniteAtomicSequence#incrementAndGet()* throws
> exception:
>
> org.apache.ignite.cluster.ClusterTopologyException: Failed to acquire lock
> for keys (primary node left grid, retry transaction if possible)
> [keys=[UserKeyCacheObjectImpl [part=94, val=GridCacheInternalKeyImpl
> [name=seq, grpName=default-ds-group], hasValBytes=true]],
> node=a047ec4b-7de6-4503-9691-a5d7e03e267f]
>
> I handle exception in that way:
>
> IgniteAtomicSequence seq = ignite.atomicSequence(Const.SEQ, new
> AtomicConfiguration().setAtomicSequenceReserveSize(0)
> .setCacheMode(CacheMode.REPLICATED),0, Boolean.TRUE);
> while(true){
> //do some logic
> try {
> *seq.incrementAndGet();*
> }
> catch (ClusterTopologyException cte) {
> *cte.retryReadyFuture().get();*
> }
> }
>
> At the same time the *IgniteAtomicLong* doesn't throw such exception (at
> least I can't reproduce it).
>
> Please help me to clarify flowing questions:
> 1. Is it recommended to handle topology exception in the same way? Is there
> any public documentation about that?
> 2. What kind of distributed operations(cache updates, open data stream,
> atomic) are recommended to handle the ClusterTopologyException?
>
> --
>
> Regards
>
> Pavel Vinokurov
>