Re: Apache Ignite RPM packaging (Stage II)

[Dmitriy Setrakyan]

Will Debian package work for Ubuntu?

D.

On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov  wrote:

> Not a problem, rather nuisance. Also, when we will move to official
> repositories, there can be a problem from OS community.
>
> Concerning DEB packages — I plan to use RPM as base for DEB package build
> (package layout / install scripts) for speeding up things and excluding
> possible duplication and desynchronisation, so its a matter of ’sit and do’
> rather then some technical research. Thats why I rose discussion about
> future package architecture, so that after agreement I'm be able to pack
> both RPM and DEB identically.
>
> Yet, if you insist, I can create DEB package according to current RPM
> layout in no time.
>
>
>
> > On 15 Mar 2018, at 04:53, Dmitriy Setrakyan 
> wrote:
> >
> > Peter,
> >
> > I don't think the package size of 280M is going to be a problem at all,
> but
> > what you are suggesting can be an improvement down the road.
> >
> > In the mean time, I think our top priority should be to provide packages
> > for Debian and Ubuntu. Having only RPMs is not nearly enough.
> >
> > Agree?
> >
> > D.
> >
> > On Wed, Mar 14, 2018 at 5:36 AM, vveider  wrote:
> >
> >> Hi, Igniters!
> >>
> >>
> >> Release 2.4 is almost there, at least binary part of it, so I'd like to
> >> move
> >> forward to further improve and widen AI delivery through packages.
> >> As of now, Apache Ignite ships in RPM package weighing about 280M+ and,
> to
> >> improve usability and significantly reduce required download sizes, I
> >> purpose that in 2.5 release we introduce splitted delivery as follows:
> >> - CORE
> >>   - bin
> >>   - config
> >>   - libs (!optional)
> >> - OPTIONAL LIBS
> >> - BENCHMARKS
> >> - DOCS (?)
> >> - EXAMPLES
> >> - .NET PLATFORM FILES
> >> - C++ PLATFORM FILES
> >>
> >> This architecture, as I assume, will add flexibility (no reason to
> download
> >> all 280M+ of binaries where you are to run only core node functionality)
> >> and
> >> maintainability (you are in full control of what is installed on your
> >> system).
> >>
> >> After successful architecture choice, same scheme are planned to be
> used in
> >> DEB packages as well.
> >>
> >> WDYT?
> >>
> >>
> >>
> >> --
> >> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >>
>
>

Re: Apache Ignite RPM packaging (Stage II)

[Petr Ivanov]

> On 14 Mar 2018, at 22:12, Denis Magda  wrote:
> 
> Petr,
> 
> How big would be a final core module?

Around 30M. Can be shrinked to ~15M if separate Visor and create it’s own 
package.


> As for the optional libs do you
> suggest installing them with a single command or each module can be
> installed separately?

Eventually — separately. If user wishes to try some new integration or lib, 
there is no need to download all of them.
Also it will be clearer what new integrations / libs we add (if we add) each 
release.


> 
> --
> Denis
> 
> On Wed, Mar 14, 2018 at 2:36 AM, vveider  wrote:
> 
>> Hi, Igniters!
>> 
>> 
>> Release 2.4 is almost there, at least binary part of it, so I'd like to
>> move
>> forward to further improve and widen AI delivery through packages.
>> As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to
>> improve usability and significantly reduce required download sizes, I
>> purpose that in 2.5 release we introduce splitted delivery as follows:
>> - CORE
>>   - bin
>>   - config
>>   - libs (!optional)
>> - OPTIONAL LIBS
>> - BENCHMARKS
>> - DOCS (?)
>> - EXAMPLES
>> - .NET PLATFORM FILES
>> - C++ PLATFORM FILES
>> 
>> This architecture, as I assume, will add flexibility (no reason to download
>> all 280M+ of binaries where you are to run only core node functionality)
>> and
>> maintainability (you are in full control of what is installed on your
>> system).
>> 
>> After successful architecture choice, same scheme are planned to be used in
>> DEB packages as well.
>> 
>> WDYT?
>> 
>> 
>> 
>> --
>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>> 

Re: [RESULT] [VOTE] Apache Ignite 2.4.0 Release (RC1)

[Petr Ivanov]

I am sure about AWS and GC images now — there is only script inside that have 
IGNITE_VERSION on input and runs apacheignite/ignite: docker 
image inside the container.
No changes except date are not required except that, maybe, we are to update 
docs at readme.io  for more latest version of Apache Ignite 
as example.


> On 14 Mar 2018, at 20:16, Denis Magda  wrote:
> 
> Petr,
> 
> Fixed the page. As for the cloud images, please give a call to Nikolay to
> sort things out. It blocks us from the release announcement.
> 
> --
> Denis
> 
> On Tue, Mar 13, 2018 at 11:43 PM, Petr Ivanov  wrote:
> 
>> 1. RPM Package Installation
>>* missing ‘[ignite]’ header;
>>* rest of text is indented by 1 space, while should not.
>> 2. Building the Binaries
>>* I guess mentioning Java 7 in 2.4.0 release is obsolete — that block
>> should be either split (which AI requires which JDK for compiling) or just
>> get rid of Java 7 mentioning.
>> 3. Docker and Cloud Images
>>* still not clear what to do with AWS / Google Cloud — how to update
>> those images. Maybe Nikolay Tikhonov can help?
>> 
>> 
>> 
>>> On 13 Mar 2018, at 22:27, Denis Magda  wrote:
>>> 
>>> Then tell me what to correct and I'll merge the changes.
>>> 
>>> Alternatively, you can send me an SVN patch with changes if it's simples:
>>> https://cwiki.apache.org/confluence/display/IGNITE/Website+Development
>>> 
>>> --
>>> Denis
>>> 
>>> On Tue, Mar 13, 2018 at 12:25 PM, Petr Ivanov 
>> wrote:
>>> 
 I have no access to site and readme.io  has no
>> errors.
 
 
> On 13 Mar 2018, at 22:23, Denis Magda  wrote:
> 
> Petr,
> 
> Have you corrected the site and readme.io docs (if it was required)?
> 
> --
> Denis
> 
> On Mon, Mar 12, 2018 at 11:14 PM, Petr Ivanov 
 wrote:
> 
>> There is an error in RPM documentation on site — missing header for
>> ignite.repo file and formatting is also wrong.
>> Otherwise — I’m able to install and run Apache Ignite from RPM from
>> ASF
>> according to manual.
>> 
>> 
>> 
>>> On 12 Mar 2018, at 23:25, Denis Magda  wrote:
>>> 
>>> Released RPM installation section on the downloads page:
>>> https://ignite.apache.org/download.cgi#rpm-package
>>> 
>>> *Petr*, please double check that the section and the readme getting
>> started
>>> look good:
>>> https://apacheignite.readme.io/docs/getting-started#
 section-rpm-package-
>> installation
>>> 
>>> *Mauricio*, could you please create a patch that points out "latest"
 docs
>>> to "2.4.0"?
>>> 
>>> --
>>> Denis
>>> 
>>> 
>>> 
>>> On Mon, Mar 12, 2018 at 7:36 AM, Pavel Tupitsyn <
>> ptupit...@apache.org>
>>> wrote:
>>> 
 NuGet packages uploaded: https://www.nuget.org/
 packages?q=Apache.Ignite
 
 Thanks,
 Pavel
 
 
 On Mon, Mar 12, 2018 at 1:54 PM, Vladimir Ozerov <
 voze...@gridgain.com>
 wrote:
 
> Correct, it is a matter of time. Download links already work on my
 side
> [1].
> 
> [1] http://mirror.linux-ia64.org/apache/ignite/2.4.0/
> 
> On Mon, Mar 12, 2018 at 1:29 PM, Petr Ivanov 
 wrote:
> 
>> Link to 2.4.0 is pointing to RBC mirror, not Apache Dist itself.
>> 
>> I do not know how often RBC updates its mirror, but I think we
>> have
 to
>> devise better solution for pointing to fresh release artifacts.
>> 
>> 
>> 
>>> On 12 Mar 2018, at 13:24, Pavel Tupitsyn 
 wrote:
>>> 
>>> Hi Vladimir,
>>> 
>>> I see the link for 2.4.0 on https://ignite.apache.org/
>> download.cgi#binaries,
>>> but it does not work.
>>> 
>>> On Mon, Mar 12, 2018 at 10:55 AM, Vladimir Ozerov <
> voze...@gridgain.com>
>>> wrote:
>>> 
 Correction: " Apache Ignite *2.4.0* release (RC1) has been
 accepted."
 
 On Mon, Mar 12, 2018 at 10:51 AM, Vladimir Ozerov <
> voze...@gridgain.com
>>> 
 wrote:
 
> Igniters,
> 
> Apache Ignite 2.3.0 release (RC1) has been accepted.
> 
> 5 "+1" binding votes received:
> - Alexey Goncharuk
> - Alexey Kuznetsov
> - Anton Vinogradov
> - Denis Magda
> - Pavel Tupitsyn
> 
> Vote thread:
> 
> *http://apache-ignite-developers.2346864.n4.nabble.
 com/VOTE-Apache-Ignite-2-4-0-RC1-td27687.html
> 

Re: Partition loss policy - how to use?

[Gaurav Bajaj]

Alexey,

Thanks. I wonder why not webconsole?

Thanks,
Gaurav

On 14-Mar-2018 1:28 AM, "Alexey Kuznetsov"  wrote:

>  Gaurav,
>
> I think it make sense to add this for tools.
> Created issue: https://issues.apache.org/jira/browse/IGNITE-7940
>
> On Wed, Mar 14, 2018 at 1:44 AM, Gaurav Bajaj 
> wrote:
>
> > Hi Denis,
> > Thanks. Document certainly looks useful. Do we have ticket for
> improvement
> > in Webconsole/Visor for marking resetLostPartitions()?
> >
> >
> > Regards,
> > Gaurav
> >
> > On 13-Mar-2018 7:42 PM, "Denis Magda"  wrote:
> >
> > For those interested, here is a doc we put together for the partition
> > policies which considers extra improvements released in 2.4:
> > https://apacheignite.readme.io/v2.4/docs/partition-loss-policies
> >
> > --
> > Denis
> >
> > On Tue, Mar 6, 2018 at 11:19 AM, Denis Magda  wrote:
> >
> > > Hi,
> > >
> > > Here is documentation we prepared for 2.4 release:
> https://apacheignite.
> > > readme.io/v2.3/docs/cache-modes-24#partition-loss-policies
> > >
> > > It's hidden for now and will become visible to everyone once Ignite 2.4
> > > vote passes (in progress).
> > >
> > > --
> > > Denis
> > >
> > > On Tue, Mar 6, 2018 at 6:59 AM, gauravhb 
> wrote:
> > >
> > >> Hi,
> > >>
> > >> Is there any update on this topic?
> > >> Any tickets created for points mentioned by Valentin?
> > >>
> > >> Thanks.
> > >>
> > >>
> > >>
> > >> --
> > >> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> > >>
> > >
> > >
> >
>
>
>
> --
> Alexey Kuznetsov
>

Re: Partition recovery issue on partition loss.

[Dmitriy Setrakyan]

Completely agree, we must fix this. I like the proposed design. We should
also specify that resetLostPartitions() method should return true and false.

Val, do you mind updating the ticket with new design?
https://issues.apache.org/jira/browse/IGNITE-7832

D.

On Tue, Mar 13, 2018 at 5:31 PM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> This indeed looks like a bigger issue. Basically, there is no clear way (or
> no way at all) to synchronize code that listens to partition loss event,
> and the code that calls resetLostPartitions() method. Example scenario:
>
> 1. Cache is configured with 3rd party persistence.
> 2. One or more nodes fail causing loss of several partitions in memory.
> 3. Ignite blocks access to those partitions according to partition loss
> policy and fires an event.
> 4. Application listens to the event and starts reloading the data from
> store.
> 5. When reloading is complete, application calls resetLostPartitions() to
> restore access.
> 6. Nodes fail again causing another partition loss, new event is fired.
>
> There is race between steps 5 and 6. If 2nd failure happens BEFORE
> resetLostPartitions() is called, we end up with inconsistent data.
>
> I believe the only way to fix this is to add corresponding topology version
> to partition loss event, and also add it as a parameter for
> resetLostPartitions().
> This way if resetLostPartitions() is invoked with a version that is not the
> latest anymore, the invocation will be ignored.
>
> The only problem with this approach  is that topology version itself is
> currently not a part of public API. It needs to be properly exposed there
> first.
>
> -Val
>
> On Mon, Mar 12, 2018 at 1:07 PM, Denis Magda  wrote:
>
> > Just in case here is you can find the present documentation:
> > https://apacheignite.readme.io/docs/cache-modes#partition-loss-policies
> >
> > Let us know what needs to be updated once the issues reported by you are
> > addressed.
> >
> > --
> > Denis
> >
> > On Mon, Mar 12, 2018 at 3:33 AM, Andrey Mashenkov <
> > andrey.mashen...@gmail.com> wrote:
> >
> > > Hi Igniters,
> > >
> > > I've found we no documentation how user can recover cache from
> cacheStore
> > > in case of partition loss.
> > > Ignite provides some instruments (methods and events) that should help
> > user
> > > to solve this problem,
> > > but looks like these instruments have an architecture lack.
> > >
> > > The first one is an usability issue. Ignite provides partition loss
> event
> > > to user can handle this, but Ignite fires an event per partition.
> > > Why we can't have an event with list of lost partitions?
> > >
> > > The second one is a bug. Ignite.resetLostPartitions() method doesn't
> care
> > > about what topology version recovered partitions belonged to.
> > > Tthere is a race, when user call this method after a node was failed,
> but
> > > right before Ignite fire an event.
> > > So, it is possible state of just lost partitions will be reseted
> > > unexpectedly.
> > >
> > >
> > > I've created a ticket for this [1] and think we should rethink the
> > > architecture of the partition recovery mechanics and improve
> > documentation.
> > > Any thoughts?
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-7832
> > >
> > >
> > > --
> > > Best regards,
> > > Andrey V. Mashenkov
> > >
> >
>

[jira] [Created] (IGNITE-7958) Web Console: Optimize on-focus-out directive.

[Alexey Kuznetsov (JIRA)]

Alexey Kuznetsov created IGNITE-7958:


 Summary: Web Console: Optimize on-focus-out directive.
 Key: IGNITE-7958
 URL: https://issues.apache.org/jira/browse/IGNITE-7958
 Project: Ignite
  Issue Type: Task
Reporter: Alexey Kuznetsov
Assignee: Alexey Kuznetsov


on-focus-out directive can produce significant GC pressure and make UI 
unresponsive.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: IEP-14: Ignite failures handling (Discussion)

[Dmitriy Setrakyan]

On Wed, Mar 14, 2018 at 7:12 PM, Andrey Kornev 
wrote:

> I'm not disagreeing with you, Dmitriy.
>
> What I'm trying to say is that if we assume that a serious enough bug or
> some environmental issue prevents Ignite node from functioning correctly,
> then it's only logical to assume that Ignite process is completely hosed
> (for example, due to a very very long STW pause) and can't make any
> progress at all. In a situation like this the application can't reason
> about the process state, and the process itself may not be able to even
> kill itself. The only reliable way to handle cases like that is to have an
> external observer (a health monitoring tool) that is not itself affected by
> the bug or the env issue and can either make a decision by itself or send a
> notification to the SRE team.
>

Agree about the external observers, but that is something a user should do
outside of Ignite.


> In my previous post I only suggest to go easy on the "cleverness" of the
> self-monitoring implementation as IMHO it won't be used much in production
> environment. I think Ignite as it is already provides sufficient means
> of monitoring its health (they may or may not be robust enough, which is a
> different issue).
>

The approach I am suggesting is pretty simple - "kill" the process in case
of a critical error. The only intelligence I would like to add is to
attempt shutting down the Ignite node gracefully before the "kill" is
executed. If a node is shutdown gracefully, then the restart procedure will
be faster, so it is worthwhile to try.

Some of the critical errors include running out of disk, memory, loosing
Ignite system threads, etc... These errors are truly unrecoverable from the
application stand point and should mostly be handled with a process restart
anyway.

D.

Re: IgniteSet implementation: changes required

[Dmitriy Setrakyan]

I am not sure I like the "asSet()" method. We already have Ignite.set(...)
method and now introducing yet another one. The new design should fix the
existing implementation. We cannot keep the broken implementation around
and introduce yet another one. To be consistent, we should also stick to
the IgniteSet abstraction.

Just to be clear, I am in agreement that a non-collocated set should be
based on a cache, but I do not see why the existing API cannot be re-used.

D.


On Wed, Mar 14, 2018 at 1:26 PM, Andrey Kuznetsov  wrote:

> Hi, Dmitry.
>
> The primary goal of the ticket is to implement {{IgniteCache::asSet}} view.
> The rationale is introduced earlier in this talk by Vladimir O. I've also
> mentioned the need to document that new method properly: it should be used
> to create large sets. Existing {{IgniteSets}} are good to represent small
> sets.
>
> 2018-03-14 20:05 GMT+03:00 Dmitry Pavlov :
>
> > Hi Pavel,
> >
> > I did not quite understand the purpose of the new collection and the
> > purpose of introduced new set.
> >
> > Is ticket descriptoin states, that it is for better documentation?
> >
> > Can we improve the task description
> > https://issues.apache.org/jira/browse/IGNITE-7823 ?
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > ср, 14 мар. 2018 г. в 12:31, Pavel Pereslegin :
> >
> > > Hello Igniters.
> > >
> > > I'm working on the implementation of the IgniteCache#asSet method [1]
> > > and I think it should return Set (not IgniteSet). Because IgniteSet
> > > was introduced mainly to add methods for the collocated version of
> > > IgniteSet.
> > >
> > > Any thoughts?
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-7823
> > >
> > >
> > > 2018-02-27 14:59 GMT+03:00 Andrey Kuznetsov :
> > > > As far as I know, Pavel P. is working on fixing existing sets
> > currently.
> > > >
> > > > As for {{asSet}} cache adapter, I filed the ticket [1].
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-7823
> > > >
> > > > 2018-02-27 11:20 GMT+03:00 Vladimir Ozerov :
> > > >
> > > >> I think the root issue is that we are trying to mix different cases
> > in a
> > > >> single solution. What is the common usage patterns of sets?
> > > >> 1) Small mostly-read sets - current implementation is ideal for
> them -
> > > >> everything is available locally, on-heap and in deserialized form
> > > >> 2) Big data sets - IgniteCache API is the best candidate here; we
> can
> > > >> simply add a method "Set IgniteCache.asSet()" which will return
> > thin
> > > >> wrapper for Set interface around cache. An you will have recovery
> and
> > > >> heap-resistance with almost no additional efforts.
> > > >> There is no need to choose between one or another solution. We can
> > > support
> > > >> both.
> > > >>
> > > >> As far as current igniteSet implementation - all issues described
> > above
> > > >> will go away if you use approach suggested by Dmitry. I do not see a
> > > need
> > > >> for any major changes.
> > > >>
> > > >> Vladimir.
> > > >>
> > > >
> > > > --
> > > > Best regards,
> > > >   Andrey Kuznetsov.
> > >
> >
>
>
>
> --
> Best regards,
>   Andrey Kuznetsov.
>

Re: IEP-14: Ignite failures handling (Discussion)

[Andrey Kornev]

I'm not disagreeing with you, Dmitriy.

What I'm trying to say is that if we assume that a serious enough bug or some 
environmental issue prevents Ignite node from functioning correctly, then it's 
only logical to assume that Ignite process is completely hosed (for example, 
due to a very very long STW pause) and can't make any progress at all. In a 
situation like this the application can't reason about the process state, and 
the process itself may not be able to even kill itself. The only reliable way 
to handle cases like that is to have an external observer (a health monitoring 
tool) that is not itself affected by the bug or the env issue and can either 
make a decision by itself or send a notification to the SRE team.

In my previous post I only suggest to go easy on the "cleverness" of the 
self-monitoring implementation as IMHO it won't be used much in production 
environment. I think Ignite as it is already provides sufficient means of 
monitoring its health (they may or may not be robust enough, which is a 
different issue).

Regards
Andrey


From: Dmitriy Setrakyan 
Sent: Wednesday, March 14, 2018 6:22 PM
To: dev@ignite.apache.org
Subject: Re: IEP-14: Ignite failures handling (Discussion)

On Wed, Mar 14, 2018 at 3:36 PM, Andrey Kornev 
wrote:

> If I were the one responsible for running Ignite-based applications (be it
> embedded or standalone Ignite) in my company's datacenter, I'd prefer the
> application nodes simply make their current state readily available to
> external tools (via JMX, health checks, etc.) and leave the decision of
> when to die and when to continue to run up to me. The last thing I need in
> production is a too clever an application that decides to kill itself based
> on its local (perhaps confused) state.
>
> Usually SRE teams build all sorts of technology-specific tools to monitor
> health of the applications and they like to be as much in control as
> possible when it comes to killing processes.
>
> I guess what I'm saying is this: keep things simple. Do not over engineer.
> In real production environments the companies will most likely have this
> feature disabled (I know I would) and instead rely on their own tooling for
> handling failures.
>
>
Andrey, our priority should be to keep the cluster operational. If a frozen
Ignite node is kept around, the whole cluster becomes un-operational. I bet
this is not what you would prefer in production either. However, if we kill
the process, then the cluster should continue to operate.

We are talking about a distributed system in which a failure of one node
should not matter. If we want to keep this promise to the users, then we
must kill the process if Ignite node freezes.

Also, keep in mind that we are talking about the "default" behavior. If you
are not happy with the "default" mode, then you will be able to configure
other behaviors, like keeping the frozen Ignite node around, if you like.

D.

Re: Data eviction/expiration from Ignite persistence

[Dmitriy Setrakyan]

On Wed, Mar 14, 2018 at 1:41 PM, Alexey Goncharuk <
alexey.goncha...@gmail.com> wrote:

> Denis,
>
> With the approach of Ignite Durable Memory there is no difference between
> 'memory' and 'disk'. The data is expired from the Ignite data storage which
> can be persisted or not. Before persistence was introduced, TTL was mostly
> used when write-through was enabled, otherwise data was cleared from Ignite
> data storage. Currently, the situation stays the same - if an entry is
> expired, it is removed from the Ignite storage, which looks absolutely
> consistent to me.
>

Agree with AG. There is a difference between expiration and eviction. If an
entry is expired, then it should be removed from the store, regardless if
it is in memory or on disk.

However, evicting from memory because there is not enough space does not
remove an entry from the store.

D.

Re: Apache Ignite RPM packaging (Stage II)

[Dmitriy Setrakyan]

Peter,

I don't think the package size of 280M is going to be a problem at all, but
what you are suggesting can be an improvement down the road.

In the mean time, I think our top priority should be to provide packages
for Debian and Ubuntu. Having only RPMs is not nearly enough.

Agree?

D.

On Wed, Mar 14, 2018 at 5:36 AM, vveider  wrote:

> Hi, Igniters!
>
>
> Release 2.4 is almost there, at least binary part of it, so I'd like to
> move
> forward to further improve and widen AI delivery through packages.
> As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to
> improve usability and significantly reduce required download sizes, I
> purpose that in 2.5 release we introduce splitted delivery as follows:
>  - CORE
>- bin
>- config
>- libs (!optional)
>  - OPTIONAL LIBS
>  - BENCHMARKS
>  - DOCS (?)
>  - EXAMPLES
>  - .NET PLATFORM FILES
>  - C++ PLATFORM FILES
>
> This architecture, as I assume, will add flexibility (no reason to download
> all 280M+ of binaries where you are to run only core node functionality)
> and
> maintainability (you are in full control of what is installed on your
> system).
>
> After successful architecture choice, same scheme are planned to be used in
> DEB packages as well.
>
> WDYT?
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

Re: IEP-14: Ignite failures handling (Discussion)

[Dmitriy Setrakyan]

On Wed, Mar 14, 2018 at 3:36 PM, Andrey Kornev 
wrote:

> If I were the one responsible for running Ignite-based applications (be it
> embedded or standalone Ignite) in my company's datacenter, I'd prefer the
> application nodes simply make their current state readily available to
> external tools (via JMX, health checks, etc.) and leave the decision of
> when to die and when to continue to run up to me. The last thing I need in
> production is a too clever an application that decides to kill itself based
> on its local (perhaps confused) state.
>
> Usually SRE teams build all sorts of technology-specific tools to monitor
> health of the applications and they like to be as much in control as
> possible when it comes to killing processes.
>
> I guess what I'm saying is this: keep things simple. Do not over engineer.
> In real production environments the companies will most likely have this
> feature disabled (I know I would) and instead rely on their own tooling for
> handling failures.
>
>
Andrey, our priority should be to keep the cluster operational. If a frozen
Ignite node is kept around, the whole cluster becomes un-operational. I bet
this is not what you would prefer in production either. However, if we kill
the process, then the cluster should continue to operate.

We are talking about a distributed system in which a failure of one node
should not matter. If we want to keep this promise to the users, then we
must kill the process if Ignite node freezes.

Also, keep in mind that we are talking about the "default" behavior. If you
are not happy with the "default" mode, then you will be able to configure
other behaviors, like keeping the frozen Ignite node around, if you like.

D.

Re: IEP-14: Ignite failures handling (Discussion)

[Dmitriy Setrakyan]

On Tue, Mar 13, 2018 at 11:17 PM, Nick Pordash 
wrote:

> I can tell you as a user that if any library I was using in my application
> called System.exit without my consent would result in a lot of frustration.
>
> If ignite enters an unrecoverable state then I think that is something that
> should be observable locally, similar to node segmentation and then the
> application can decide the best course of action.
>

Nick, you would be a lot more frustrated if Ignite was frozen and every
call to Ignite would freeze the application threads as well. Again, if you
prefer to keep the process around, even if Ignite freezes, then you can
always configure this behavior, but I still believe that the default should
be to kill the process.

Ignite is a horizontally scalable system, so killing of one node should not
be a significant event and should not disrupt the cluster. However, a
freeze of one node is a significant event and can bring the whole cluster
to a halt.

D.

[GitHub] ignite pull request #3638: IGNITE-7916: GA Grid examples should be ready for...

[techbysample]

GitHub user techbysample opened a pull request:

https://github.com/apache/ignite/pull/3638

IGNITE-7916: GA Grid examples should be ready for auto run on TeamCity

 Removed Ignition.stop(true) to prevent shutdown of examples within build.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/techbysample/ignite ignite-7916

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3638.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3638


commit 6da25a491721121490a5b868295e5496392fed22
Author: Turik Campbell 
Date:   2018-03-14T23:15:04Z

IGNITE-7916: Removed Ignition.stop(true) to prevent shutdown of examples 
within build.

commit 7d5e74eb97e8a5ebbe9f2a640f9f36aae5adb2be
Author: Turik Campbell 
Date:   2018-03-14T23:19:16Z

IGNITE-7916: Merge branch 'master' into ignite-7916




---

Re: IGNITE-5357 is ready for review (Replicated cache reads load balancing)

[Dmitry Pavlov]

Yes, I think I could move IgniteReproducingSuite to dev-utils module later.
Thank you for this idea.

Yes, It is probably it was Queries test flaky'ness.

I hope Vladimir, you will find some time to make query tests more stable.
It is not friendly to community members if their patches are rejected by
reasons not related to their change.

Any assistance from the rest of community here is also appreciated.

ср, 14 мар. 2018 г. в 22:24, Vyacheslav Daradur :

> Thank you for the advice!
>
> Unfortunately, *IgniteReproducingSuite* is in the core module while
> *IgniteSqlSplitterSelfTest* in the ignite-indexing module that means I
> am not able to add the test in this test suite without addition
> cycling dependency.
>
> I'd recommend you detaching *IgniteReproducingSuite* as a separate
> module in the project to include the test suites from any module in
> the project.
>
>
> But I've prepared *Ignite Queries* in the same way as you suggested in
> *IgniteReproducingSuite* [1] and ran all tests in
> *IgniteSqlSplitterSelfTest* 100 times [2].
>
> >> IgniteBinaryCacheQueryTestSuite:
> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
> (fail rate 0,0%)
> For this test "Green lite" 100 times of 100.
>
> Green lite for all tests in *IgniteSqlSplitterSelfTest* in the latest
> build of main PR [3].
>
>
> [1]
> https://github.com/daradurvs/ignite/blob/fd6abc915838599c2ebab3f803f90f2e641e8892/modules/indexing/src/test/java/org/apache/ignite/testsuites/IgniteCacheQuerySelfTestSuite.java
> [2] https://ci.ignite.apache.org/viewLog.html?buildId=1136780
> [3] https://ci.ignite.apache.org/viewLog.html?buildId=1136685
>
> On Wed, Mar 14, 2018 at 7:55 PM, Dmitry Pavlov 
> wrote:
> > It is possible that test is failing only on agents and is always
> successfull
> > locally.
> >
> > For researching such test there was "Ignite reproducing suite" introduced
> > early. This suite intentionally left blank on TC. Correspondent suite in
> > code is IgniteReproducingSuite.
> >
> > You may add some extra debug info into test. Add this test in
> > IgniteReproducingSuite in code and then start suite on TC several times.
> >
> > ср, 14 мар. 2018 г. в 19:42, Vyacheslav Daradur :
> >>
> >> Dmitry, as I've written here before: I checked this test locally, many
> >> times (didn't have any falling on 100 starts).
> >>
> >> On Wed, Mar 14, 2018 at 7:31 PM, Dmitry Pavlov 
> >> wrote:
> >> > Hi, I've found test which never failed on master, but fails in branch
> >> >
> >> >  Ignite Queries [ tests 1 ]
> >> >
> >> >  IgniteBinaryCacheQueryTestSuite:
> >> >
> >> >
> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
> >> > (fail rate 0,0%)
> >> >
> >> >
> >> > ср, 14 мар. 2018 г. в 19:26, Dmitry Pavlov :
> >> >>
> >> >> Hi, let me check TC run
> >> >>
> >> >> вт, 13 мар. 2018 г. в 9:22, Vyacheslav Daradur  >:
> >> >>>
> >> >>> Dmitry,
> >> >>>
> >> >>> Nickolay accepted PR changes at Upsource [1].
> >> >>>
> >> >>> Latest ci.build [2] looks good in comparison with master [3].
> >> >>>
> >> >>> Following tests passed locally:
> >> >>> CacheAffinityCallSelfTest.testAffinityCallFromClientRestartNode
> >> >>> CacheAffinityCallSelfTest.testAffinityCallRestartNode
> >> >>>
> IgniteOptimisticTxSuspendResumeMultiServerTest.testTxTimeoutOnSuspend
> >> >>>
> >> >>>
> >> >>>
> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
> >> >>>
> >> >>>
> >> >>> [1] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-509
> >> >>> [2] https://ci.ignite.apache.org/viewLog.html?buildId=1134466
> >> >>> [3] https://ci.ignite.apache.org/viewLog.html?buildId=1134372
> >> >>>
> >> >>> On Mon, Mar 5, 2018 at 7:16 PM, Vyacheslav Daradur
> >> >>> 
> >> >>> wrote:
> >> >>> > Dmitry, I saw them, but it looks like just randomness.
> >> >>> >
> >> >>> > I've checked it locally several times.
> >> >>> > They failed only in one TeamCity's build of four.
> >> >>> >
> >> >>> > Started build once again to be sure.
> >> >>> >
> >> >>> > On Mon, Mar 5, 2018 at 6:59 PM, Dmitry Pavlov
> >> >>> > 
> >> >>> > wrote:
> >> >>> >> I can see Nikolay Izhikov as reviewer in Upsource.
> >> >>> >>
> >> >>> >> Nikolay, would you run review first?
> >> >>> >>
> >> >>> >> I've found several suspicious tests : Test fail rate is less than
> >> >>> >> 1%,
> >> >>> >> it is
> >> >>> >> probably new failure
> >> >>> >> IgniteCacheTestSuite2:
> >> >>> >>
> >> >>> >>
> >> >>> >>
> GridCachePartitionedTxSingleThreadedSelfTest.testOptimisticReadCommittedRollback
> >> >>> >> (fail rate 0,0%)
> >> >>> >> IgniteCacheTestSuite2:
> >> >>> >>
> >> >>> >>
> >> >>> >>
> GridCachePartitionedTxSingleThreadedSelfTest.testOptimisticRepeatableReadRollback
> >> >>> >> (fail rate 0,0%)
> >> >>> >> IgniteCacheTestSuite2:
> >> >>> >>
> >> >>> >>
> >> >>> >>

RE: Timeline for support of compute functions by thin clients

[Raymond Wilson]

There is no particular reason why we cannot use the regular one, so this is
not a blocker for us.

The Thin Client was attractive in that it allowed client contexts to be very
lightweight which aids aspects such as fine grained container scalability
(think AWS::Fargate/EKS).

-Original Message-
From: Denis Magda [mailto:dma...@apache.org]
Sent: Thursday, March 15, 2018 9:32 AM
To: dev@ignite.apache.org
Subject: Re: Timeline for support of compute functions by thin clients

Raymond,

Then I would suggest you keep using the regular .NET client that supports
and optimized for computations. Is there any reason why you can't use the
regular one?

--
Denis

On Wed, Mar 14, 2018 at 12:53 PM, Raymond Wilson  wrote:

> Hi Denis,
>
> We are using Ignite.Net and are planning to use 2.4 + .Net Core + thin
> client support to enable lightweight containerisable services that
> interact with the main Ignite compute grid.
>
> These work flows are less about Get/Put style semantics, and more
> about using grid compute.
>
> Eg: Here's an example where a client context asks a remote context to
> render a bitmap tile in an ICompute:
>
> public Bitmap Execute(TileRenderRequestArgument arg)
> {
> IComputeFunc func = new
> TileRenderRequestComputeFunc();
>
> return
> _ignite.GetCluster().ForRemotes().GetCompute().Apply(func, arg);
> }
>
> In this example, the calling context here could be a lightweight
> Kestrel web service end point delegating rendering to a remote
> service.
>
> Thanks,
> Raymond.
>
> -Original Message-
> From: Denis Magda [mailto:dma...@apache.org]
> Sent: Thursday, March 15, 2018 8:31 AM
> To: dev@ignite.apache.org
> Subject: Re: Timeline for support of compute functions by thin clients
>
> Hi Raymond,
>
> There are no any plans for that level of support. The thin clients are
> targeted for classic client-server processing use cases when a client
> request data from a server, does something with it locally and
> potentially writes changes back to the server. ICache, SQL fall under this
> category.
>
> Are you intended to use .NET thin client or anyone else?
>
> --
> Denis
>
> On Wed, Mar 14, 2018 at 12:25 PM, Raymond Wilson <
> raymond_wil...@trimble.com
> > wrote:
>
> > Hi,
> >
> >
> >
> > The thin client implementation in Ignite 2.4 only covers a subset of
> > the ICache interface.
> >
> >
> >
> > When will we see thin client support for compute, messaging etc?
> >
> >
> >
> > Thanks,
> >
> > Raymond.
> >
>

Re: Timeline for support of compute functions by thin clients

[Pavel Tupitsyn]

I think there were plans to eventually cover whole set of existing APIs in
Thin Client mode.
Some APIs may be difficult to add (like Compute Task), but basic compute
methods (non-Task based) are easy to do.


On Wed, Mar 14, 2018 at 11:33 PM, Pavel Tupitsyn 
wrote:

> Hi Denis,
>
> > There are no any plans for that level of support
> Why do you think so?
> We already have ScanQuery with filter in .NET Thin Client, which involves
> remote code execution on server nodes.
> It is quite similar to Compute.Broadcast and such.
>
> Thanks,
> Pavel
>
>
> On Wed, Mar 14, 2018 at 11:32 PM, Denis Magda  wrote:
>
>> Raymond,
>>
>> Then I would suggest you keep using the regular .NET client that supports
>> and optimized for computations. Is there any reason why you can't use the
>> regular one?
>>
>> --
>> Denis
>>
>> On Wed, Mar 14, 2018 at 12:53 PM, Raymond Wilson <
>> raymond_wil...@trimble.com
>> > wrote:
>>
>> > Hi Denis,
>> >
>> > We are using Ignite.Net and are planning to use 2.4 + .Net Core + thin
>> > client support to enable lightweight containerisable services that
>> interact
>> > with the main Ignite compute grid.
>> >
>> > These work flows are less about Get/Put style semantics, and more about
>> > using grid compute.
>> >
>> > Eg: Here's an example where a client context asks a remote context to
>> > render
>> > a bitmap tile in an ICompute:
>> >
>> > public Bitmap Execute(TileRenderRequestArgument arg)
>> > {
>> > IComputeFunc func = new
>> > TileRenderRequestComputeFunc();
>> >
>> > return
>> > _ignite.GetCluster().ForRemotes().GetCompute().Apply(func, arg);
>> > }
>> >
>> > In this example, the calling context here could be a lightweight Kestrel
>> > web
>> > service end point delegating rendering to a remote service.
>> >
>> > Thanks,
>> > Raymond.
>> >
>> > -Original Message-
>> > From: Denis Magda [mailto:dma...@apache.org]
>> > Sent: Thursday, March 15, 2018 8:31 AM
>> > To: dev@ignite.apache.org
>> > Subject: Re: Timeline for support of compute functions by thin clients
>> >
>> > Hi Raymond,
>> >
>> > There are no any plans for that level of support. The thin clients are
>> > targeted for classic client-server processing use cases when a client
>> > request data from a server, does something with it locally and
>> potentially
>> > writes changes back to the server. ICache, SQL fall under this category.
>> >
>> > Are you intended to use .NET thin client or anyone else?
>> >
>> > --
>> > Denis
>> >
>> > On Wed, Mar 14, 2018 at 12:25 PM, Raymond Wilson <
>> > raymond_wil...@trimble.com
>> > > wrote:
>> >
>> > > Hi,
>> > >
>> > >
>> > >
>> > > The thin client implementation in Ignite 2.4 only covers a subset of
>> > > the ICache interface.
>> > >
>> > >
>> > >
>> > > When will we see thin client support for compute, messaging etc?
>> > >
>> > >
>> > >
>> > > Thanks,
>> > >
>> > > Raymond.
>> > >
>> >
>>
>
>

Re: Why does DataStreamer swallow exceptions?

[Valentin Kulichenko]

Ilya,

Yes, this is definitely a bug, please create a ticket.

-Val

On Wed, Mar 14, 2018 at 2:16 AM, Ilya Kasnacheev 
wrote:

> Hello Val!
>
> No, this does NOT happen. If you collect future, call get() on it later, it
> will finish normally despite exceptions in server log and entry not being
> written. I will do some digging to figure out why this happens exactly.
>
> There's also another huge problems with DataStreamer's futures. They never
> finish unless you call flush() on DS before calling get() on futures.
> I think this is a colossal usability problem (I'm pretty sure I've seen
> numerous messages about it on userlist) and I'll fill an issue if nobody is
> objecting.
>
> Ilya.
>
> --
> Ilya Kasnacheev
>
> 2018-03-05 22:50 GMT+03:00 Valentin Kulichenko <
> valentin.kuliche...@gmail.com>:
>
> > Ilya,
> >
> > IgniteDataStreamer#addData method returns future which should be
> completed
> > with error if one is thrown on server side. Does this happen or not?
> >
> > -Val
> >
> > On Mon, Mar 5, 2018 at 4:10 AM, Nikolay Izhikov 
> > wrote:
> >
> > > Hello, Ilya.
> > >
> > > > I think it's time to end this, if that was the case. DataStreamer
> > should
> > > > not be a special case and it should guarantee data safety. WDYT?
> > >
> > > +1 from me.
> > >
> > > I'm also facing this issue.
> > >
> > > Ticket - https://issues.apache.org/jira/browse/IGNITE-7756
> > > Discussion - http://apache-ignite-developers.2346864.n4.nabble.
> > > com/IgniteDataStreamer-silently-fails-on-a-server-node-td27239.html
> > >
> > >
> > >
> > > В Пн, 05/03/2018 в 14:46 +0300, Ilya Kasnacheev пишет:
> > > > Dear Igniters, why do I have a hunch that DataStreamer would readily
> > > > swallow exceptions?
> > > >
> > > > DataStreamerImpl:1756 swallows marshalling error, lines 1774 & 1781
> eat
> > > > deployment errors.
> > > >
> > > > Some people are worried they can fill a leaking vessel without
> noticing
> > > > anything off.
> > > > Also in line 2156 fsync() on WAL can throw exceptions, which will be
> > > > swallowed, and IMP this fsync doesn't belong to DataStreamer code.
> > > >
> > > > This question is not purely theoretical, we have also replaced one of
> > > these
> > > > eaters with throw with IGNITE-7519.
> > > >
> > > > There was a fault in PDS implementation, which did not propagate to
> > > client.
> > > > A serious issue IMHO.
> > > >
> > > > As a data streamer user I will expect that flush()/close() will throw
> > any
> > > > pending exceptions and will only be silent if all data landed safely
> in
> > > the
> > > > cluster.
> > > >
> > > > I also have this feeling that DataStreamer was written using very
> > > internal
> > > > APIs so that it can compromise guarantees that cache and SQL APIs are
> > > bound
> > > > by. I think I've heard something about not recovering properly in
> case
> > of
> > > > node failures.
> > > > I think it's time to end this, if that was the case. DataStreamer
> > should
> > > > not be a special case and it should guarantee data safety. WDYT?
> > > >
> > >
> >
>

Re: Timeline for support of compute functions by thin clients

[Pavel Tupitsyn]

Hi Denis,

> There are no any plans for that level of support
Why do you think so?
We already have ScanQuery with filter in .NET Thin Client, which involves
remote code execution on server nodes.
It is quite similar to Compute.Broadcast and such.

Thanks,
Pavel


On Wed, Mar 14, 2018 at 11:32 PM, Denis Magda  wrote:

> Raymond,
>
> Then I would suggest you keep using the regular .NET client that supports
> and optimized for computations. Is there any reason why you can't use the
> regular one?
>
> --
> Denis
>
> On Wed, Mar 14, 2018 at 12:53 PM, Raymond Wilson <
> raymond_wil...@trimble.com
> > wrote:
>
> > Hi Denis,
> >
> > We are using Ignite.Net and are planning to use 2.4 + .Net Core + thin
> > client support to enable lightweight containerisable services that
> interact
> > with the main Ignite compute grid.
> >
> > These work flows are less about Get/Put style semantics, and more about
> > using grid compute.
> >
> > Eg: Here's an example where a client context asks a remote context to
> > render
> > a bitmap tile in an ICompute:
> >
> > public Bitmap Execute(TileRenderRequestArgument arg)
> > {
> > IComputeFunc func = new
> > TileRenderRequestComputeFunc();
> >
> > return
> > _ignite.GetCluster().ForRemotes().GetCompute().Apply(func, arg);
> > }
> >
> > In this example, the calling context here could be a lightweight Kestrel
> > web
> > service end point delegating rendering to a remote service.
> >
> > Thanks,
> > Raymond.
> >
> > -Original Message-
> > From: Denis Magda [mailto:dma...@apache.org]
> > Sent: Thursday, March 15, 2018 8:31 AM
> > To: dev@ignite.apache.org
> > Subject: Re: Timeline for support of compute functions by thin clients
> >
> > Hi Raymond,
> >
> > There are no any plans for that level of support. The thin clients are
> > targeted for classic client-server processing use cases when a client
> > request data from a server, does something with it locally and
> potentially
> > writes changes back to the server. ICache, SQL fall under this category.
> >
> > Are you intended to use .NET thin client or anyone else?
> >
> > --
> > Denis
> >
> > On Wed, Mar 14, 2018 at 12:25 PM, Raymond Wilson <
> > raymond_wil...@trimble.com
> > > wrote:
> >
> > > Hi,
> > >
> > >
> > >
> > > The thin client implementation in Ignite 2.4 only covers a subset of
> > > the ICache interface.
> > >
> > >
> > >
> > > When will we see thin client support for compute, messaging etc?
> > >
> > >
> > >
> > > Thanks,
> > >
> > > Raymond.
> > >
> >
>

Re: Timeline for support of compute functions by thin clients

[Denis Magda]

Raymond,

Then I would suggest you keep using the regular .NET client that supports
and optimized for computations. Is there any reason why you can't use the
regular one?

--
Denis

On Wed, Mar 14, 2018 at 12:53 PM, Raymond Wilson  wrote:

> Hi Denis,
>
> We are using Ignite.Net and are planning to use 2.4 + .Net Core + thin
> client support to enable lightweight containerisable services that interact
> with the main Ignite compute grid.
>
> These work flows are less about Get/Put style semantics, and more about
> using grid compute.
>
> Eg: Here's an example where a client context asks a remote context to
> render
> a bitmap tile in an ICompute:
>
> public Bitmap Execute(TileRenderRequestArgument arg)
> {
> IComputeFunc func = new
> TileRenderRequestComputeFunc();
>
> return
> _ignite.GetCluster().ForRemotes().GetCompute().Apply(func, arg);
> }
>
> In this example, the calling context here could be a lightweight Kestrel
> web
> service end point delegating rendering to a remote service.
>
> Thanks,
> Raymond.
>
> -Original Message-
> From: Denis Magda [mailto:dma...@apache.org]
> Sent: Thursday, March 15, 2018 8:31 AM
> To: dev@ignite.apache.org
> Subject: Re: Timeline for support of compute functions by thin clients
>
> Hi Raymond,
>
> There are no any plans for that level of support. The thin clients are
> targeted for classic client-server processing use cases when a client
> request data from a server, does something with it locally and potentially
> writes changes back to the server. ICache, SQL fall under this category.
>
> Are you intended to use .NET thin client or anyone else?
>
> --
> Denis
>
> On Wed, Mar 14, 2018 at 12:25 PM, Raymond Wilson <
> raymond_wil...@trimble.com
> > wrote:
>
> > Hi,
> >
> >
> >
> > The thin client implementation in Ignite 2.4 only covers a subset of
> > the ICache interface.
> >
> >
> >
> > When will we see thin client support for compute, messaging etc?
> >
> >
> >
> > Thanks,
> >
> > Raymond.
> >
>

[GitHub] ignite pull request #3637: IGNITE-7727: Test unmuted. Fixed by #70ca86a

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3637


---

RE: Timeline for support of compute functions by thin clients

[Raymond Wilson]

Hi Denis,

We are using Ignite.Net and are planning to use 2.4 + .Net Core + thin
client support to enable lightweight containerisable services that interact
with the main Ignite compute grid.

These work flows are less about Get/Put style semantics, and more about
using grid compute.

Eg: Here's an example where a client context asks a remote context to render
a bitmap tile in an ICompute:

public Bitmap Execute(TileRenderRequestArgument arg)
{
IComputeFunc func = new
TileRenderRequestComputeFunc();

return
_ignite.GetCluster().ForRemotes().GetCompute().Apply(func, arg);
}

In this example, the calling context here could be a lightweight Kestrel web
service end point delegating rendering to a remote service.

Thanks,
Raymond.

-Original Message-
From: Denis Magda [mailto:dma...@apache.org]
Sent: Thursday, March 15, 2018 8:31 AM
To: dev@ignite.apache.org
Subject: Re: Timeline for support of compute functions by thin clients

Hi Raymond,

There are no any plans for that level of support. The thin clients are
targeted for classic client-server processing use cases when a client
request data from a server, does something with it locally and potentially
writes changes back to the server. ICache, SQL fall under this category.

Are you intended to use .NET thin client or anyone else?

--
Denis

On Wed, Mar 14, 2018 at 12:25 PM, Raymond Wilson  wrote:

> Hi,
>
>
>
> The thin client implementation in Ignite 2.4 only covers a subset of
> the ICache interface.
>
>
>
> When will we see thin client support for compute, messaging etc?
>
>
>
> Thanks,
>
> Raymond.
>

[GitHub] ignite pull request #3637: IGNITE-7727: Test unmuted. Fixed by #70ca86a

[nizhikov]

GitHub user nizhikov opened a pull request:

https://github.com/apache/ignite/pull/3637

IGNITE-7727: Test unmuted. Fixed by #70ca86a



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nizhikov/ignite IGNITE-7727

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3637.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3637


commit fbc554ca46056d2e869c989a4ad64b4294106784
Author: Nikolay Izhikov 
Date:   2018-03-14T19:44:07Z

IGNITE-7727: Test unmuted. Fixed by #70ca86a




---

[GitHub] ignite pull request #3550: IGNITE-7756: IgniteUuid added to predefined types...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3550


---

Re: Timeline for support of compute functions by thin clients

[Denis Magda]

Hi Raymond,

There are no any plans for that level of support. The thin clients are
targeted for classic client-server processing use cases when a client
request data from a server, does something with it locally and potentially
writes changes back to the server. ICache, SQL fall under this category.

Are you intended to use .NET thin client or anyone else?

--
Denis

On Wed, Mar 14, 2018 at 12:25 PM, Raymond Wilson  wrote:

> Hi,
>
>
>
> The thin client implementation in Ignite 2.4 only covers a subset of the
> ICache interface.
>
>
>
> When will we see thin client support for compute, messaging etc?
>
>
>
> Thanks,
>
> Raymond.
>

Timeline for support of compute functions by thin clients

[Raymond Wilson]

Hi,



The thin client implementation in Ignite 2.4 only covers a subset of the
ICache interface.



When will we see thin client support for compute, messaging etc?



Thanks,

Raymond.

Re: IGNITE-5357 is ready for review (Replicated cache reads load balancing)

[Vyacheslav Daradur]

Thank you for the advice!

Unfortunately, *IgniteReproducingSuite* is in the core module while
*IgniteSqlSplitterSelfTest* in the ignite-indexing module that means I
am not able to add the test in this test suite without addition
cycling dependency.

I'd recommend you detaching *IgniteReproducingSuite* as a separate
module in the project to include the test suites from any module in
the project.


But I've prepared *Ignite Queries* in the same way as you suggested in
*IgniteReproducingSuite* [1] and ran all tests in
*IgniteSqlSplitterSelfTest* 100 times [2].

>> IgniteBinaryCacheQueryTestSuite: 
>> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
>>  (fail rate 0,0%)
For this test "Green lite" 100 times of 100.

Green lite for all tests in *IgniteSqlSplitterSelfTest* in the latest
build of main PR [3].


[1] 
https://github.com/daradurvs/ignite/blob/fd6abc915838599c2ebab3f803f90f2e641e8892/modules/indexing/src/test/java/org/apache/ignite/testsuites/IgniteCacheQuerySelfTestSuite.java
[2] https://ci.ignite.apache.org/viewLog.html?buildId=1136780
[3] https://ci.ignite.apache.org/viewLog.html?buildId=1136685

On Wed, Mar 14, 2018 at 7:55 PM, Dmitry Pavlov  wrote:
> It is possible that test is failing only on agents and is always successfull
> locally.
>
> For researching such test there was "Ignite reproducing suite" introduced
> early. This suite intentionally left blank on TC. Correspondent suite in
> code is IgniteReproducingSuite.
>
> You may add some extra debug info into test. Add this test in
> IgniteReproducingSuite in code and then start suite on TC several times.
>
> ср, 14 мар. 2018 г. в 19:42, Vyacheslav Daradur :
>>
>> Dmitry, as I've written here before: I checked this test locally, many
>> times (didn't have any falling on 100 starts).
>>
>> On Wed, Mar 14, 2018 at 7:31 PM, Dmitry Pavlov 
>> wrote:
>> > Hi, I've found test which never failed on master, but fails in branch
>> >
>> >  Ignite Queries [ tests 1 ]
>> >
>> >  IgniteBinaryCacheQueryTestSuite:
>> >
>> > IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
>> > (fail rate 0,0%)
>> >
>> >
>> > ср, 14 мар. 2018 г. в 19:26, Dmitry Pavlov :
>> >>
>> >> Hi, let me check TC run
>> >>
>> >> вт, 13 мар. 2018 г. в 9:22, Vyacheslav Daradur :
>> >>>
>> >>> Dmitry,
>> >>>
>> >>> Nickolay accepted PR changes at Upsource [1].
>> >>>
>> >>> Latest ci.build [2] looks good in comparison with master [3].
>> >>>
>> >>> Following tests passed locally:
>> >>> CacheAffinityCallSelfTest.testAffinityCallFromClientRestartNode
>> >>> CacheAffinityCallSelfTest.testAffinityCallRestartNode
>> >>> IgniteOptimisticTxSuspendResumeMultiServerTest.testTxTimeoutOnSuspend
>> >>>
>> >>>
>> >>> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
>> >>>
>> >>>
>> >>> [1] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-509
>> >>> [2] https://ci.ignite.apache.org/viewLog.html?buildId=1134466
>> >>> [3] https://ci.ignite.apache.org/viewLog.html?buildId=1134372
>> >>>
>> >>> On Mon, Mar 5, 2018 at 7:16 PM, Vyacheslav Daradur
>> >>> 
>> >>> wrote:
>> >>> > Dmitry, I saw them, but it looks like just randomness.
>> >>> >
>> >>> > I've checked it locally several times.
>> >>> > They failed only in one TeamCity's build of four.
>> >>> >
>> >>> > Started build once again to be sure.
>> >>> >
>> >>> > On Mon, Mar 5, 2018 at 6:59 PM, Dmitry Pavlov
>> >>> > 
>> >>> > wrote:
>> >>> >> I can see Nikolay Izhikov as reviewer in Upsource.
>> >>> >>
>> >>> >> Nikolay, would you run review first?
>> >>> >>
>> >>> >> I've found several suspicious tests : Test fail rate is less than
>> >>> >> 1%,
>> >>> >> it is
>> >>> >> probably new failure
>> >>> >> IgniteCacheTestSuite2:
>> >>> >>
>> >>> >>
>> >>> >> GridCachePartitionedTxSingleThreadedSelfTest.testOptimisticReadCommittedRollback
>> >>> >> (fail rate 0,0%)
>> >>> >> IgniteCacheTestSuite2:
>> >>> >>
>> >>> >>
>> >>> >> GridCachePartitionedTxSingleThreadedSelfTest.testOptimisticRepeatableReadRollback
>> >>> >> (fail rate 0,0%)
>> >>> >> IgniteCacheTestSuite2:
>> >>> >>
>> >>> >>
>> >>> >> GridCachePartitionedTxSingleThreadedSelfTest.testPessimisticReadCommittedCommit
>> >>> >> (fail rate 0,0%)
>> >>> >> IgniteCacheTestSuite2:
>> >>> >>
>> >>> >>
>> >>> >> GridCachePartitionedTxSingleThreadedSelfTest.testPessimisticReadCommittedRollback
>> >>> >> (fail rate 0,0%)
>> >>> >> IgniteCacheTestSuite2:
>> >>> >>
>> >>> >>
>> >>> >> GridCachePartitionedTxSingleThreadedSelfTest.testPessimisticSerializableCommit
>> >>> >> (fail rate 0,0%)
>> >>> >>
>> >>> >> Vyacheslav, could you please check if these failures are related to
>> >>> >> the new
>> >>> >> changes?
>> >>> >>
>> >>> >>
>> >>> >> пн, 5 мар. 2018 г. в 18:50, Vyacheslav Daradur
>> >>> >> :
>> >>> >>
>> >>> >>> I've done 

Re: Apache Ignite RPM packaging (Stage II)

[Denis Magda]

Petr,

How big would be a final core module? As for the optional libs do you
suggest installing them with a single command or each module can be
installed separately?

--
Denis

On Wed, Mar 14, 2018 at 2:36 AM, vveider  wrote:

> Hi, Igniters!
>
>
> Release 2.4 is almost there, at least binary part of it, so I'd like to
> move
> forward to further improve and widen AI delivery through packages.
> As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to
> improve usability and significantly reduce required download sizes, I
> purpose that in 2.5 release we introduce splitted delivery as follows:
>  - CORE
>- bin
>- config
>- libs (!optional)
>  - OPTIONAL LIBS
>  - BENCHMARKS
>  - DOCS (?)
>  - EXAMPLES
>  - .NET PLATFORM FILES
>  - C++ PLATFORM FILES
>
> This architecture, as I assume, will add flexibility (no reason to download
> all 280M+ of binaries where you are to run only core node functionality)
> and
> maintainability (you are in full control of what is installed on your
> system).
>
> After successful architecture choice, same scheme are planned to be used in
> DEB packages as well.
>
> WDYT?
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

Re: Ignite contibutors page

[Denis Magda]

Turik,

Done, you're in ;)

On Tue, Mar 13, 2018 at 10:11 PM, techbysample  wrote:

> Denis,
>
> Please add my name as well.
>
> Best
> Turik Campbell
>
>
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

[GitHub] ignite pull request #3636: IGNITE-7912 First stage of unused WAL segments co...

[ivandasch]

GitHub user ivandasch opened a pull request:

https://github.com/apache/ignite/pull/3636

IGNITE-7912 First stage of unused WAL segments command. Now print

from all nodes.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ivandasch/ignite ignite-7912

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3636.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3636


commit 76817db04de34c85d9fd78a33f1f26657ab91e7b
Author: Ivan Daschinskiy 
Date:   2018-03-14T17:43:34Z

IGNITE-7912 First stage of unused WAL segments command. Now print
from all nodes.




---

Fwd: Atomic sequence and topology exception

[Dmitry Pavlov]

Hi Nikolay,

How do you think which was idea / the best practice to handling exceptions
here?

Why exception here may have difference?

Sincerely,
Dmitriy Pavlov

-- Forwarded message -
From: Vinokurov Pavel 
Date: вт, 13 мар. 2018 г. в 5:52
Subject: Atomic sequence and topology exception
To: 


Igniters!

I  have a few questions related to a replicated atomic sequence and an
topology exception.
In my case after one server node has left cluster, on a client node the
execution of the *IgniteAtomicSequence#incrementAndGet()* throws exception:

org.apache.ignite.cluster.ClusterTopologyException: Failed to acquire lock
for keys (primary node left grid, retry transaction if possible)
[keys=[UserKeyCacheObjectImpl [part=94, val=GridCacheInternalKeyImpl
[name=seq, grpName=default-ds-group], hasValBytes=true]],
node=a047ec4b-7de6-4503-9691-a5d7e03e267f]

I handle exception in that way:

IgniteAtomicSequence seq = ignite.atomicSequence(Const.SEQ, new
AtomicConfiguration().setAtomicSequenceReserveSize(0)
.setCacheMode(CacheMode.REPLICATED),0, Boolean.TRUE);
while(true){
//do some logic
try {
*seq.incrementAndGet();*
}
catch (ClusterTopologyException cte) {
*cte.retryReadyFuture().get();*
}
}

At the same time the *IgniteAtomicLong* doesn't throw such exception (at
least I can't reproduce it).

Please help me to clarify flowing questions:
1. Is it recommended to handle topology exception in the same way? Is there
any public documentation about that?
2. What kind of distributed operations(cache updates, open data stream,
atomic) are recommended to handle the ClusterTopologyException?

--

Regards

Pavel Vinokurov

Re: Data eviction/expiration from Ignite persistence

[Alexey Goncharuk]

Denis,

With the approach of Ignite Durable Memory there is no difference between
'memory' and 'disk'. The data is expired from the Ignite data storage which
can be persisted or not. Before persistence was introduced, TTL was mostly
used when write-through was enabled, otherwise data was cleared from Ignite
data storage. Currently, the situation stays the same - if an entry is
expired, it is removed from the Ignite storage, which looks absolutely
consistent to me.

2018-03-13 21:30 GMT+03:00 Denis Magda :

> Alexey,
>
> My understanding was that the expiration policies worked for data in RAM
> only. Ok, if an expired entry is removed from both RAM and Ignite
> persistence then what happens if a cache store is used instead of Ignite
> storage? Do we remove expired entries from RDBMs, Cassandra, etc? My guess
> that we don't which doesn't look consistent product wide.
>
> --
> Denis
>
> On Tue, Mar 13, 2018 at 12:50 AM, Alexey Goncharuk <
> alexey.goncha...@gmail.com> wrote:
>
> > Denis,
> >
> > What do you mean by 'current behavior when data is evicted from the
> memory
> > only'? TTL expiration effectively means that the corresponding key-value
> > pairs are destroyed. If you are talking about page replacement, then
> there
> > is no way to do this on per-key basis because a page must be replaced as
> a
> > whole and it makes no sense to track keys.
> >
> > --AG
> >
> > 2018-03-13 0:03 GMT+03:00 Denis Magda :
> >
> > > Dmitriy,
> > >
> > > It will break the current default behavior when data is evicted from
> the
> > > memory only, and I would disagree that it's a right decision overall.
> > >
> > > There are many scenarios when users need to have the eviction in the
> > memory
> > > layer and preserve data on disk for later usage. So, can we keep what
> we
> > > have now and merely expand the eviction to disk if the user requires
> it?
> > >
> > > --
> > > Denis
> > >
> > >
> > > On Mon, Mar 12, 2018 at 1:35 PM, Dmitry Pavlov 
> > > wrote:
> > >
> > > > Denis,
> > > >
> > > > I suppose there is no configuration will be required. If TTL
> configured
> > > > entry will be removed from disk & memory both.
> > > >
> > > > SIncerely,
> > > > Dmitriy Pavlov
> > > >
> > > > пн, 12 мар. 2018 г. в 23:32, Denis Magda :
> > > >
> > > > > Alexey, Dmitriy,
> > > > >
> > > > > What would be the configuration parameter that defines if the
> > eviction
> > > > > should happen in the in-memory layer only or in both memory and
> disk?
> > > > >
> > > > > --
> > > > > Denis
> > > > >
> > > > > On Mon, Mar 12, 2018 at 9:22 AM, Alexey Goncharuk <
> > > > > alexey.goncha...@gmail.com> wrote:
> > > > >
> > > > > > Val,
> > > > > >
> > > > > > Yes, the entries will be removed from both memory and
> persistence.
> > > > > >
> > > > > > 2018-03-12 19:20 GMT+03:00 Valentin Kulichenko <
> > > > > > valentin.kuliche...@gmail.com>:
> > > > > >
> > > > > > > Alex,
> > > > > > >
> > > > > > > What is behavior going to be after IGNITE-5874 is fixed? Will
> > > expired
> > > > > > entry
> > > > > > > be removed from both memory and persistence?
> > > > > > >
> > > > > > > -Val
> > > > > > >
> > > > > > > On Sat, Mar 10, 2018 at 12:06 AM, Alexey Goncharuk <
> > > > > > > alexey.goncha...@gmail.com> wrote:
> > > > > > >
> > > > > > > > The ticket [1] is in patch available state looks good, the
> only
> > > > thing
> > > > > > > left
> > > > > > > > there is to transfer old entries to new storage. I hope
> Andrey
> > > will
> > > > > > have
> > > > > > > > time to finish this soon, so we can target the fix for 2.5.
> > > > > > > >
> > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-5874
> > > > > > > >
> > > > > > > > 2018-03-09 22:51 GMT+03:00 Denis Magda :
> > > > > > > >
> > > > > > > > > Val,
> > > > > > > > >
> > > > > > > > > I'd like to hear Alexey G. opinion on this? Alex, please
> > chime
> > > > in.
> > > > > > > > >
> > > > > > > > > In general, the more deployments the persistence will get
> the
> > > > more
> > > > > > > demand
> > > > > > > > > we will see for that capability. Personally, I'd create a
> > > ticket
> > > > > for
> > > > > > > now
> > > > > > > > > and put it off to our backlog.
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Denis
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Fri, Mar 9, 2018 at 9:34 AM, Dmitriy Setrakyan <
> > > > > > > dsetrak...@apache.org
> > > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > On Fri, Mar 9, 2018 at 3:43 AM, Dmitry Pavlov <
> > > > > > dpavlov@gmail.com
> > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > As far as I know there is no plans. Denis please
> correct
> > me
> > > > if
> > > > > > I'm
> > > > > > > > > wrong.
> > > > > > > > > > >
> > > > > > > > > > > But users found these or similar bugs, it seems we need
> > to
> > > > > > support
> > > > > > > > PDS
> > > > > > > > > 

[GitHub] ignite pull request #3635: TeamCity reproducing tests

[daradurvs]

GitHub user daradurvs opened a pull request:

https://github.com/apache/ignite/pull/3635

TeamCity reproducing tests



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/daradurvs/ignite ignite-5357-reproducing-tests

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3635.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3635


commit fc6cdf8cff5236254c9f3d4805561ca0baa8518d
Author: Vyacheslav Daradur 
Date:   2018-02-27T15:58:38Z

ignite-5357: neineighborhoods selection logic was added

commit 7fe3b609bd5c5adfcca26516958da32796fc0be6
Author: Vyacheslav Daradur 
Date:   2018-02-27T16:01:02Z

ignite-5357: fix statement with 'canRemap' flag

commit 9c07ad741c1fcc2a051b77207d7f939b03a7d668
Author: Vyacheslav Daradur 
Date:   2018-02-27T16:09:29Z

ignite-5357: code style fix

commit 65546ae4f60d13bcbd49fa2c526ed00d63dd2b90
Author: Vyacheslav Daradur 
Date:   2018-02-27T16:30:09Z

ignite-5357: replaced parallelStream with stream

commit f6197a0e3994a671debb802474cde042f4a1749d
Author: Vyacheslav Daradur 
Date:   2018-02-27T17:18:22Z

ignite-5357: fix nullable logic

commit b35ddb9cf8d15e3739ee55ce830cf9e442687a7e
Author: Vyacheslav Daradur 
Date:   2018-02-27T17:20:12Z

ignite-5357: remove unnecessary imports

commit 518a083913d7a4dbd26faf96cc0d3ce1f89dc60d
Author: Vyacheslav Daradur 
Date:   2018-02-27T17:36:16Z

ignite-5357: skip primary node condition was added

commit ded2441863ff9582947ae20a48161b366b793ca7
Author: Vyacheslav Daradur 
Date:   2018-02-28T15:36:24Z

ignite-5357: refactoring

commit 883dbbff5d257c4eb8a3de65020c84c5c444ae54
Author: Vyacheslav Daradur 
Date:   2018-02-28T17:50:01Z

ignite-5357: javadoc fix

commit e76013f40f17fc5ee07370b3d0d1ee7b73580460
Author: Vyacheslav Daradur 
Date:   2018-02-28T19:22:13Z

ignite-5357: refactoring

commit eaaa0318dc0242dc901ad9b447bfdbd96e9c7698
Author: Vyacheslav Daradur 
Date:   2018-02-28T19:33:37Z

ignite-5357: back logic

commit 66097c4998e799df980591d1fa0126096a27267e
Author: Vyacheslav Daradur 
Date:   2018-02-28T19:36:24Z

ignite-5357: changed logic

commit f6f0db629bc19b414ae15f7aea63cec3e08f9749
Author: Vyacheslav Daradur 
Date:   2018-02-28T19:37:45Z

ignite-5357: changed logic 2

commit 91c75ef77093e8747fd90d1b505d47aedacd711f
Author: Vyacheslav Daradur 
Date:   2018-02-28T19:54:36Z

ignite-5357: removed utils method

commit 26c639ef9ba30b58b18fb21a56a644f14021dbb1
Author: Vyacheslav Daradur 
Date:   2018-02-28T20:43:09Z

Merge remote-tracking branch 'origin/ignite-5357-2' into ignite-5357

commit 09ba7b2effd29fb9c6202958d4d019daf22e9800
Author: Vyacheslav Daradur 
Date:   2018-03-01T23:59:16Z

ignite-5357: requests distribution tests were added

commit 8a685e10f15a89eb48b4db731d6035c31dae9176
Author: Vyacheslav Daradur 
Date:   2018-03-02T06:02:33Z

ignite-5357: fix tests classes names

commit 56579c8f5194744e6ce83103d33d034f95ee219b
Author: Vyacheslav Daradur 
Date:   2018-03-02T09:32:39Z

ignite-5357: fix node for getting primary keys

commit a5fca8ab25bc66c39ec46843a0c65ce09ba9334a
Author: Vyacheslav Daradur 
Date:   2018-03-02T10:08:00Z

ignite-5357: code style fix: long line split

commit d85fd75315973b12b9d60f3ddc6ceba84147f869
Author: Vyacheslav Daradur 
Date:   2018-03-02T10:45:42Z

ignite-5357: javadoc comment was extended

commit 2de55a8954cd10b55613a4229f390c5d03d2984e
Author: Vyacheslav Daradur 
Date:   2018-03-02T14:39:20Z

ignite-5357: generator tests were added

commit c0c95e0715f80c3148b77927a93ed7b75518d971
Author: Vyacheslav Daradur 
Date:   2018-03-02T15:21:34Z

ignite-5357: lamda expressions were unwraped

commit d12e14bd143e56674f9fc5d8e4f44ebbeb728dcf
Author: Vyacheslav Daradur 
Date:   2018-03-02T20:23:58Z

ignite-5357: minor fix

commit 62093d240568f18618517d9cdfb6767c1d5543a2
Author: Vyacheslav Daradur 
Date:   2018-03-11T21:08:32Z

ignite-5357: review notes fixes; refactoring

commit 43f485ea505e45112cf6f95321ae89c0b9c1c1ad
Author: Vyacheslav Daradur 
Date:   2018-03-12T08:28:43Z

ignite-5357: moved random generator to method

commit c0fc039eba4b5596d496bb69c84fe2df10df680c
Author: Vyacheslav Daradur 
Date:   2018-03-12T08:55:09Z

ignite-5357: review notes fixes

commit 5a98fb8bcef6c6134221d0d7e418e0f325b0e02f
Author: Vyacheslav Daradur 
Date:   2018-03-12T11:06:03Z

ignite-5357: review 

Re: IgniteSet implementation: changes required

[Andrey Kuznetsov]

Hi, Dmitry.

The primary goal of the ticket is to implement {{IgniteCache::asSet}} view.
The rationale is introduced earlier in this talk by Vladimir O. I've also
mentioned the need to document that new method properly: it should be used
to create large sets. Existing {{IgniteSets}} are good to represent small
sets.

2018-03-14 20:05 GMT+03:00 Dmitry Pavlov :

> Hi Pavel,
>
> I did not quite understand the purpose of the new collection and the
> purpose of introduced new set.
>
> Is ticket descriptoin states, that it is for better documentation?
>
> Can we improve the task description
> https://issues.apache.org/jira/browse/IGNITE-7823 ?
>
> Sincerely,
> Dmitriy Pavlov
>
> ср, 14 мар. 2018 г. в 12:31, Pavel Pereslegin :
>
> > Hello Igniters.
> >
> > I'm working on the implementation of the IgniteCache#asSet method [1]
> > and I think it should return Set (not IgniteSet). Because IgniteSet
> > was introduced mainly to add methods for the collocated version of
> > IgniteSet.
> >
> > Any thoughts?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-7823
> >
> >
> > 2018-02-27 14:59 GMT+03:00 Andrey Kuznetsov :
> > > As far as I know, Pavel P. is working on fixing existing sets
> currently.
> > >
> > > As for {{asSet}} cache adapter, I filed the ticket [1].
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-7823
> > >
> > > 2018-02-27 11:20 GMT+03:00 Vladimir Ozerov :
> > >
> > >> I think the root issue is that we are trying to mix different cases
> in a
> > >> single solution. What is the common usage patterns of sets?
> > >> 1) Small mostly-read sets - current implementation is ideal for them -
> > >> everything is available locally, on-heap and in deserialized form
> > >> 2) Big data sets - IgniteCache API is the best candidate here; we can
> > >> simply add a method "Set IgniteCache.asSet()" which will return
> thin
> > >> wrapper for Set interface around cache. An you will have recovery and
> > >> heap-resistance with almost no additional efforts.
> > >> There is no need to choose between one or another solution. We can
> > support
> > >> both.
> > >>
> > >> As far as current igniteSet implementation - all issues described
> above
> > >> will go away if you use approach suggested by Dmitry. I do not see a
> > need
> > >> for any major changes.
> > >>
> > >> Vladimir.
> > >>
> > >
> > > --
> > > Best regards,
> > >   Andrey Kuznetsov.
> >
>



-- 
Best regards,
  Andrey Kuznetsov.

[jira] [Created] (IGNITE-7957) TestDistributedJoins fails in CPP Win32 suite

[Dmitriy Pavlov (JIRA)]

Dmitriy Pavlov created IGNITE-7957:
--

 Summary: TestDistributedJoins fails in CPP Win32 suite
 Key: IGNITE-7957
 URL: https://issues.apache.org/jira/browse/IGNITE-7957
 Project: Ignite
  Issue Type: Task
  Components: platforms
Reporter: Dmitriy Pavlov


https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=417754190398412=%3Cdefault%3E=testDetails

   Ignite Platform CPP Win32 [ tests 1 TC_EXIT_CODE ]  
  IgniteOdbcTest: QueriesTestSuite: TestDistributedJoins (fail rate 9,8%) 




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [RESULT] [VOTE] Apache Ignite 2.4.0 Release (RC1)

[Denis Magda]

Petr,

Fixed the page. As for the cloud images, please give a call to Nikolay to
sort things out. It blocks us from the release announcement.

--
Denis

On Tue, Mar 13, 2018 at 11:43 PM, Petr Ivanov  wrote:

> 1. RPM Package Installation
> * missing ‘[ignite]’ header;
> * rest of text is indented by 1 space, while should not.
> 2. Building the Binaries
> * I guess mentioning Java 7 in 2.4.0 release is obsolete — that block
> should be either split (which AI requires which JDK for compiling) or just
> get rid of Java 7 mentioning.
> 3. Docker and Cloud Images
> * still not clear what to do with AWS / Google Cloud — how to update
> those images. Maybe Nikolay Tikhonov can help?
>
>
>
> > On 13 Mar 2018, at 22:27, Denis Magda  wrote:
> >
> > Then tell me what to correct and I'll merge the changes.
> >
> > Alternatively, you can send me an SVN patch with changes if it's simples:
> > https://cwiki.apache.org/confluence/display/IGNITE/Website+Development
> >
> > --
> > Denis
> >
> > On Tue, Mar 13, 2018 at 12:25 PM, Petr Ivanov 
> wrote:
> >
> >> I have no access to site and readme.io  has no
> errors.
> >>
> >>
> >>> On 13 Mar 2018, at 22:23, Denis Magda  wrote:
> >>>
> >>> Petr,
> >>>
> >>> Have you corrected the site and readme.io docs (if it was required)?
> >>>
> >>> --
> >>> Denis
> >>>
> >>> On Mon, Mar 12, 2018 at 11:14 PM, Petr Ivanov 
> >> wrote:
> >>>
>  There is an error in RPM documentation on site — missing header for
>  ignite.repo file and formatting is also wrong.
>  Otherwise — I’m able to install and run Apache Ignite from RPM from
> ASF
>  according to manual.
> 
> 
> 
> > On 12 Mar 2018, at 23:25, Denis Magda  wrote:
> >
> > Released RPM installation section on the downloads page:
> > https://ignite.apache.org/download.cgi#rpm-package
> >
> > *Petr*, please double check that the section and the readme getting
>  started
> > look good:
> > https://apacheignite.readme.io/docs/getting-started#
> >> section-rpm-package-
>  installation
> >
> > *Mauricio*, could you please create a patch that points out "latest"
> >> docs
> > to "2.4.0"?
> >
> > --
> > Denis
> >
> >
> >
> > On Mon, Mar 12, 2018 at 7:36 AM, Pavel Tupitsyn <
> ptupit...@apache.org>
> > wrote:
> >
> >> NuGet packages uploaded: https://www.nuget.org/
> >> packages?q=Apache.Ignite
> >>
> >> Thanks,
> >> Pavel
> >>
> >>
> >> On Mon, Mar 12, 2018 at 1:54 PM, Vladimir Ozerov <
> >> voze...@gridgain.com>
> >> wrote:
> >>
> >>> Correct, it is a matter of time. Download links already work on my
> >> side
> >>> [1].
> >>>
> >>> [1] http://mirror.linux-ia64.org/apache/ignite/2.4.0/
> >>>
> >>> On Mon, Mar 12, 2018 at 1:29 PM, Petr Ivanov 
> >> wrote:
> >>>
>  Link to 2.4.0 is pointing to RBC mirror, not Apache Dist itself.
> 
>  I do not know how often RBC updates its mirror, but I think we
> have
> >> to
>  devise better solution for pointing to fresh release artifacts.
> 
> 
> 
> > On 12 Mar 2018, at 13:24, Pavel Tupitsyn 
> >> wrote:
> >
> > Hi Vladimir,
> >
> > I see the link for 2.4.0 on https://ignite.apache.org/
>  download.cgi#binaries,
> > but it does not work.
> >
> > On Mon, Mar 12, 2018 at 10:55 AM, Vladimir Ozerov <
> >>> voze...@gridgain.com>
> > wrote:
> >
> >> Correction: " Apache Ignite *2.4.0* release (RC1) has been
> >> accepted."
> >>
> >> On Mon, Mar 12, 2018 at 10:51 AM, Vladimir Ozerov <
> >>> voze...@gridgain.com
> >
> >> wrote:
> >>
> >>> Igniters,
> >>>
> >>> Apache Ignite 2.3.0 release (RC1) has been accepted.
> >>>
> >>> 5 "+1" binding votes received:
> >>> - Alexey Goncharuk
> >>> - Alexey Kuznetsov
> >>> - Anton Vinogradov
> >>> - Denis Magda
> >>> - Pavel Tupitsyn
> >>>
> >>> Vote thread:
> >>>
> >>> *http://apache-ignite-developers.2346864.n4.nabble.
> >> com/VOTE-Apache-Ignite-2-4-0-RC1-td27687.html
> >>>  >> com/VOTE-Apache-Ignite-2-4-0-RC1-td27687.html>*
> >>>
> >>> Ignite 2.4.0 will be released soon.
> >>>
> >>> Vladimir.
> >>>
> >>
> 
> 
> >>>
> >>
> 
> 
> >>
> >>
>
>

Re: IGNITE-6879

[Dmitry Pavlov]

Igniters,

it is about spring-data integration support.

Who has intereset about styding this technology?

Alexey Kukushkin would you like to be reviewer of this issue?

Sincerely,
Dmitriy Pavlov

ср, 14 мар. 2018 г. в 1:08, Denis Magda :

> Hello Roman,
>
> It's not a big deal, I've added you to the contributors' list in JIRA. Go
> ahead and assign the ticket to yourself.
>
> Folks, anyway, who can review Roman's contribution that brings Spring 2.0
> support to Ignite?
>
> --
> Denis
>
>
>
> On Tue, Mar 13, 2018 at 2:21 PM, Роман Меерсон 
> wrote:
>
> > Hello!
> >
> > I want to work on https://issues.apache.org/jira/browse/IGNITE-6879
> issue.
> > Following the rules here
> > https://ignite.apache.org/community/contribute.html#contribute my Jira
> > username is "homich" so assign me to this ticket please.
> >
> > P.S. I found this rules page after I made PR, so sorry for this
> > inconvenience.
> >
> > Regards.
> >
>

Re: IGNITE-5357 is ready for review (Replicated cache reads load balancing)

[Dmitry Pavlov]

It is possible that test is failing only on agents and is always
successfull locally.

For researching such test there was "Ignite reproducing suite" introduced
early. This suite intentionally left blank on TC. Correspondent suite in
code is IgniteReproducingSuite.

You may add some extra debug info into test. Add this test in
IgniteReproducingSuite in code and then start suite on TC several times.

ср, 14 мар. 2018 г. в 19:42, Vyacheslav Daradur :

> Dmitry, as I've written here before: I checked this test locally, many
> times (didn't have any falling on 100 starts).
>
> On Wed, Mar 14, 2018 at 7:31 PM, Dmitry Pavlov 
> wrote:
> > Hi, I've found test which never failed on master, but fails in branch
> >
> >  Ignite Queries [ tests 1 ]
> >
> >  IgniteBinaryCacheQueryTestSuite:
> >
> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
> > (fail rate 0,0%)
> >
> >
> > ср, 14 мар. 2018 г. в 19:26, Dmitry Pavlov :
> >>
> >> Hi, let me check TC run
> >>
> >> вт, 13 мар. 2018 г. в 9:22, Vyacheslav Daradur :
> >>>
> >>> Dmitry,
> >>>
> >>> Nickolay accepted PR changes at Upsource [1].
> >>>
> >>> Latest ci.build [2] looks good in comparison with master [3].
> >>>
> >>> Following tests passed locally:
> >>> CacheAffinityCallSelfTest.testAffinityCallFromClientRestartNode
> >>> CacheAffinityCallSelfTest.testAffinityCallRestartNode
> >>> IgniteOptimisticTxSuspendResumeMultiServerTest.testTxTimeoutOnSuspend
> >>>
> >>>
> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
> >>>
> >>>
> >>> [1] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-509
> >>> [2] https://ci.ignite.apache.org/viewLog.html?buildId=1134466
> >>> [3] https://ci.ignite.apache.org/viewLog.html?buildId=1134372
> >>>
> >>> On Mon, Mar 5, 2018 at 7:16 PM, Vyacheslav Daradur <
> daradu...@gmail.com>
> >>> wrote:
> >>> > Dmitry, I saw them, but it looks like just randomness.
> >>> >
> >>> > I've checked it locally several times.
> >>> > They failed only in one TeamCity's build of four.
> >>> >
> >>> > Started build once again to be sure.
> >>> >
> >>> > On Mon, Mar 5, 2018 at 6:59 PM, Dmitry Pavlov  >
> >>> > wrote:
> >>> >> I can see Nikolay Izhikov as reviewer in Upsource.
> >>> >>
> >>> >> Nikolay, would you run review first?
> >>> >>
> >>> >> I've found several suspicious tests : Test fail rate is less than
> 1%,
> >>> >> it is
> >>> >> probably new failure
> >>> >> IgniteCacheTestSuite2:
> >>> >>
> >>> >>
> GridCachePartitionedTxSingleThreadedSelfTest.testOptimisticReadCommittedRollback
> >>> >> (fail rate 0,0%)
> >>> >> IgniteCacheTestSuite2:
> >>> >>
> >>> >>
> GridCachePartitionedTxSingleThreadedSelfTest.testOptimisticRepeatableReadRollback
> >>> >> (fail rate 0,0%)
> >>> >> IgniteCacheTestSuite2:
> >>> >>
> >>> >>
> GridCachePartitionedTxSingleThreadedSelfTest.testPessimisticReadCommittedCommit
> >>> >> (fail rate 0,0%)
> >>> >> IgniteCacheTestSuite2:
> >>> >>
> >>> >>
> GridCachePartitionedTxSingleThreadedSelfTest.testPessimisticReadCommittedRollback
> >>> >> (fail rate 0,0%)
> >>> >> IgniteCacheTestSuite2:
> >>> >>
> >>> >>
> GridCachePartitionedTxSingleThreadedSelfTest.testPessimisticSerializableCommit
> >>> >> (fail rate 0,0%)
> >>> >>
> >>> >> Vyacheslav, could you please check if these failures are related to
> >>> >> the new
> >>> >> changes?
> >>> >>
> >>> >>
> >>> >> пн, 5 мар. 2018 г. в 18:50, Vyacheslav Daradur  >:
> >>> >>
> >>> >>> I've done some test-builds iteration on the weekends.
> >>> >>>
> >>> >>> Tests [1] look well.
> >>> >>>
> >>> >>> Does anyone have time to do the final review [2][3] and merge it?
> >>> >>>
> >>> >>>
> >>> >>> [1] https://ci.ignite.apache.org/viewLog.html?buildId=1125676
> >>> >>> [2] https://github.com/apache/ignite/pull/3578
> >>> >>> [3] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-509
> >>> >>>
> >>> >>>
> >>> >>> On Fri, Mar 2, 2018 at 10:17 PM, Vyacheslav Daradur
> >>> >>> 
> >>> >>> wrote:
> >>> >>> > Hi, Igniters!
> >>> >>> >
> >>> >>> > This task [1] is about 'get' requests distribution between
> primary
> >>> >>> > and
> >>> >>> > backup nodes in the replicated cache if 'readFromBackup' flag is
> >>> >>> > enabled.
> >>> >>> >
> >>> >>> > I've prepared a solution [2] suggested by Alexei Scherbakov in
> Jira
> >>> >>> > comments. It passed prereviews by Alexei and Nikolay Izhikov.
> >>> >>> >
> >>> >>> > TeamCity tests look similar with the master branch.
> >>> >>> >
> >>> >>> > Could someone of core module maintainers do the final review
> >>> >>> > [2][3]?
> >>> >>> >
> >>> >>> >
> >>> >>> > [1] https://issues.apache.org/jira/browse/IGNITE-5357
> >>> >>> > [2] https://github.com/apache/ignite/pull/3578
> >>> >>> > [3] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-509
> >>> >>> >
> >>> >>> > --
> >>> >>> > Best Regards, Vyacheslav D.
> >>> >>>
> >>> 

Re: IGNITE-5357 is ready for review (Replicated cache reads load balancing)

[Vyacheslav Daradur]

Dmitry, as I've written here before: I checked this test locally, many
times (didn't have any falling on 100 starts).

On Wed, Mar 14, 2018 at 7:31 PM, Dmitry Pavlov  wrote:
> Hi, I've found test which never failed on master, but fails in branch
>
>  Ignite Queries [ tests 1 ]
>
>  IgniteBinaryCacheQueryTestSuite:
> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
> (fail rate 0,0%)
>
>
> ср, 14 мар. 2018 г. в 19:26, Dmitry Pavlov :
>>
>> Hi, let me check TC run
>>
>> вт, 13 мар. 2018 г. в 9:22, Vyacheslav Daradur :
>>>
>>> Dmitry,
>>>
>>> Nickolay accepted PR changes at Upsource [1].
>>>
>>> Latest ci.build [2] looks good in comparison with master [3].
>>>
>>> Following tests passed locally:
>>> CacheAffinityCallSelfTest.testAffinityCallFromClientRestartNode
>>> CacheAffinityCallSelfTest.testAffinityCallRestartNode
>>> IgniteOptimisticTxSuspendResumeMultiServerTest.testTxTimeoutOnSuspend
>>>
>>> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
>>>
>>>
>>> [1] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-509
>>> [2] https://ci.ignite.apache.org/viewLog.html?buildId=1134466
>>> [3] https://ci.ignite.apache.org/viewLog.html?buildId=1134372
>>>
>>> On Mon, Mar 5, 2018 at 7:16 PM, Vyacheslav Daradur 
>>> wrote:
>>> > Dmitry, I saw them, but it looks like just randomness.
>>> >
>>> > I've checked it locally several times.
>>> > They failed only in one TeamCity's build of four.
>>> >
>>> > Started build once again to be sure.
>>> >
>>> > On Mon, Mar 5, 2018 at 6:59 PM, Dmitry Pavlov 
>>> > wrote:
>>> >> I can see Nikolay Izhikov as reviewer in Upsource.
>>> >>
>>> >> Nikolay, would you run review first?
>>> >>
>>> >> I've found several suspicious tests : Test fail rate is less than 1%,
>>> >> it is
>>> >> probably new failure
>>> >> IgniteCacheTestSuite2:
>>> >>
>>> >> GridCachePartitionedTxSingleThreadedSelfTest.testOptimisticReadCommittedRollback
>>> >> (fail rate 0,0%)
>>> >> IgniteCacheTestSuite2:
>>> >>
>>> >> GridCachePartitionedTxSingleThreadedSelfTest.testOptimisticRepeatableReadRollback
>>> >> (fail rate 0,0%)
>>> >> IgniteCacheTestSuite2:
>>> >>
>>> >> GridCachePartitionedTxSingleThreadedSelfTest.testPessimisticReadCommittedCommit
>>> >> (fail rate 0,0%)
>>> >> IgniteCacheTestSuite2:
>>> >>
>>> >> GridCachePartitionedTxSingleThreadedSelfTest.testPessimisticReadCommittedRollback
>>> >> (fail rate 0,0%)
>>> >> IgniteCacheTestSuite2:
>>> >>
>>> >> GridCachePartitionedTxSingleThreadedSelfTest.testPessimisticSerializableCommit
>>> >> (fail rate 0,0%)
>>> >>
>>> >> Vyacheslav, could you please check if these failures are related to
>>> >> the new
>>> >> changes?
>>> >>
>>> >>
>>> >> пн, 5 мар. 2018 г. в 18:50, Vyacheslav Daradur :
>>> >>
>>> >>> I've done some test-builds iteration on the weekends.
>>> >>>
>>> >>> Tests [1] look well.
>>> >>>
>>> >>> Does anyone have time to do the final review [2][3] and merge it?
>>> >>>
>>> >>>
>>> >>> [1] https://ci.ignite.apache.org/viewLog.html?buildId=1125676
>>> >>> [2] https://github.com/apache/ignite/pull/3578
>>> >>> [3] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-509
>>> >>>
>>> >>>
>>> >>> On Fri, Mar 2, 2018 at 10:17 PM, Vyacheslav Daradur
>>> >>> 
>>> >>> wrote:
>>> >>> > Hi, Igniters!
>>> >>> >
>>> >>> > This task [1] is about 'get' requests distribution between primary
>>> >>> > and
>>> >>> > backup nodes in the replicated cache if 'readFromBackup' flag is
>>> >>> > enabled.
>>> >>> >
>>> >>> > I've prepared a solution [2] suggested by Alexei Scherbakov in Jira
>>> >>> > comments. It passed prereviews by Alexei and Nikolay Izhikov.
>>> >>> >
>>> >>> > TeamCity tests look similar with the master branch.
>>> >>> >
>>> >>> > Could someone of core module maintainers do the final review
>>> >>> > [2][3]?
>>> >>> >
>>> >>> >
>>> >>> > [1] https://issues.apache.org/jira/browse/IGNITE-5357
>>> >>> > [2] https://github.com/apache/ignite/pull/3578
>>> >>> > [3] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-509
>>> >>> >
>>> >>> > --
>>> >>> > Best Regards, Vyacheslav D.
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Best Regards, Vyacheslav D.
>>> >>>
>>> >
>>> >
>>> >
>>> > --
>>> > Best Regards, Vyacheslav D.
>>>
>>>
>>>
>>> --
>>> Best Regards, Vyacheslav D.



-- 
Best Regards, Vyacheslav D.

Re: IGNITE-5357 is ready for review (Replicated cache reads load balancing)

[Dmitry Pavlov]

Hi, let me check TC run

вт, 13 мар. 2018 г. в 9:22, Vyacheslav Daradur :

> Dmitry,
>
> Nickolay accepted PR changes at Upsource [1].
>
> Latest ci.build [2] looks good in comparison with master [3].
>
> Following tests passed locally:
> CacheAffinityCallSelfTest.testAffinityCallFromClientRestartNode
> CacheAffinityCallSelfTest.testAffinityCallRestartNode
> IgniteOptimisticTxSuspendResumeMultiServerTest.testTxTimeoutOnSuspend
>
> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
>
>
> [1] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-509
> [2] https://ci.ignite.apache.org/viewLog.html?buildId=1134466
> [3] https://ci.ignite.apache.org/viewLog.html?buildId=1134372
>
> On Mon, Mar 5, 2018 at 7:16 PM, Vyacheslav Daradur 
> wrote:
> > Dmitry, I saw them, but it looks like just randomness.
> >
> > I've checked it locally several times.
> > They failed only in one TeamCity's build of four.
> >
> > Started build once again to be sure.
> >
> > On Mon, Mar 5, 2018 at 6:59 PM, Dmitry Pavlov 
> wrote:
> >> I can see Nikolay Izhikov as reviewer in Upsource.
> >>
> >> Nikolay, would you run review first?
> >>
> >> I've found several suspicious tests : Test fail rate is less than 1%,
> it is
> >> probably new failure
> >> IgniteCacheTestSuite2:
> >>
> GridCachePartitionedTxSingleThreadedSelfTest.testOptimisticReadCommittedRollback
> >> (fail rate 0,0%)
> >> IgniteCacheTestSuite2:
> >>
> GridCachePartitionedTxSingleThreadedSelfTest.testOptimisticRepeatableReadRollback
> >> (fail rate 0,0%)
> >> IgniteCacheTestSuite2:
> >>
> GridCachePartitionedTxSingleThreadedSelfTest.testPessimisticReadCommittedCommit
> >> (fail rate 0,0%)
> >> IgniteCacheTestSuite2:
> >>
> GridCachePartitionedTxSingleThreadedSelfTest.testPessimisticReadCommittedRollback
> >> (fail rate 0,0%)
> >> IgniteCacheTestSuite2:
> >>
> GridCachePartitionedTxSingleThreadedSelfTest.testPessimisticSerializableCommit
> >> (fail rate 0,0%)
> >>
> >> Vyacheslav, could you please check if these failures are related to the
> new
> >> changes?
> >>
> >>
> >> пн, 5 мар. 2018 г. в 18:50, Vyacheslav Daradur :
> >>
> >>> I've done some test-builds iteration on the weekends.
> >>>
> >>> Tests [1] look well.
> >>>
> >>> Does anyone have time to do the final review [2][3] and merge it?
> >>>
> >>>
> >>> [1] https://ci.ignite.apache.org/viewLog.html?buildId=1125676
> >>> [2] https://github.com/apache/ignite/pull/3578
> >>> [3] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-509
> >>>
> >>>
> >>> On Fri, Mar 2, 2018 at 10:17 PM, Vyacheslav Daradur <
> daradu...@gmail.com>
> >>> wrote:
> >>> > Hi, Igniters!
> >>> >
> >>> > This task [1] is about 'get' requests distribution between primary
> and
> >>> > backup nodes in the replicated cache if 'readFromBackup' flag is
> >>> > enabled.
> >>> >
> >>> > I've prepared a solution [2] suggested by Alexei Scherbakov in Jira
> >>> > comments. It passed prereviews by Alexei and Nikolay Izhikov.
> >>> >
> >>> > TeamCity tests look similar with the master branch.
> >>> >
> >>> > Could someone of core module maintainers do the final review [2][3]?
> >>> >
> >>> >
> >>> > [1] https://issues.apache.org/jira/browse/IGNITE-5357
> >>> > [2] https://github.com/apache/ignite/pull/3578
> >>> > [3] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-509
> >>> >
> >>> > --
> >>> > Best Regards, Vyacheslav D.
> >>>
> >>>
> >>>
> >>> --
> >>> Best Regards, Vyacheslav D.
> >>>
> >
> >
> >
> > --
> > Best Regards, Vyacheslav D.
>
>
>
> --
> Best Regards, Vyacheslav D.
>

Re: IGNITE-5357 is ready for review (Replicated cache reads load balancing)

[Dmitry Pavlov]

Hi, I've found test which never failed on master, but fails in branch

 Ignite Queries [ tests 1 ]

 IgniteBinaryCacheQueryTestSuite:
IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
(fail rate 0,0%)


ср, 14 мар. 2018 г. в 19:26, Dmitry Pavlov :

> Hi, let me check TC run
>
> вт, 13 мар. 2018 г. в 9:22, Vyacheslav Daradur :
>
>> Dmitry,
>>
>> Nickolay accepted PR changes at Upsource [1].
>>
>> Latest ci.build [2] looks good in comparison with master [3].
>>
>> Following tests passed locally:
>> CacheAffinityCallSelfTest.testAffinityCallFromClientRestartNode
>> CacheAffinityCallSelfTest.testAffinityCallRestartNode
>> IgniteOptimisticTxSuspendResumeMultiServerTest.testTxTimeoutOnSuspend
>>
>> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
>>
>>
>> [1] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-509
>> [2] https://ci.ignite.apache.org/viewLog.html?buildId=1134466
>> [3] https://ci.ignite.apache.org/viewLog.html?buildId=1134372
>>
>> On Mon, Mar 5, 2018 at 7:16 PM, Vyacheslav Daradur 
>> wrote:
>> > Dmitry, I saw them, but it looks like just randomness.
>> >
>> > I've checked it locally several times.
>> > They failed only in one TeamCity's build of four.
>> >
>> > Started build once again to be sure.
>> >
>> > On Mon, Mar 5, 2018 at 6:59 PM, Dmitry Pavlov 
>> wrote:
>> >> I can see Nikolay Izhikov as reviewer in Upsource.
>> >>
>> >> Nikolay, would you run review first?
>> >>
>> >> I've found several suspicious tests : Test fail rate is less than 1%,
>> it is
>> >> probably new failure
>> >> IgniteCacheTestSuite2:
>> >>
>> GridCachePartitionedTxSingleThreadedSelfTest.testOptimisticReadCommittedRollback
>> >> (fail rate 0,0%)
>> >> IgniteCacheTestSuite2:
>> >>
>> GridCachePartitionedTxSingleThreadedSelfTest.testOptimisticRepeatableReadRollback
>> >> (fail rate 0,0%)
>> >> IgniteCacheTestSuite2:
>> >>
>> GridCachePartitionedTxSingleThreadedSelfTest.testPessimisticReadCommittedCommit
>> >> (fail rate 0,0%)
>> >> IgniteCacheTestSuite2:
>> >>
>> GridCachePartitionedTxSingleThreadedSelfTest.testPessimisticReadCommittedRollback
>> >> (fail rate 0,0%)
>> >> IgniteCacheTestSuite2:
>> >>
>> GridCachePartitionedTxSingleThreadedSelfTest.testPessimisticSerializableCommit
>> >> (fail rate 0,0%)
>> >>
>> >> Vyacheslav, could you please check if these failures are related to
>> the new
>> >> changes?
>> >>
>> >>
>> >> пн, 5 мар. 2018 г. в 18:50, Vyacheslav Daradur :
>> >>
>> >>> I've done some test-builds iteration on the weekends.
>> >>>
>> >>> Tests [1] look well.
>> >>>
>> >>> Does anyone have time to do the final review [2][3] and merge it?
>> >>>
>> >>>
>> >>> [1] https://ci.ignite.apache.org/viewLog.html?buildId=1125676
>> >>> [2] https://github.com/apache/ignite/pull/3578
>> >>> [3] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-509
>> >>>
>> >>>
>> >>> On Fri, Mar 2, 2018 at 10:17 PM, Vyacheslav Daradur <
>> daradu...@gmail.com>
>> >>> wrote:
>> >>> > Hi, Igniters!
>> >>> >
>> >>> > This task [1] is about 'get' requests distribution between primary
>> and
>> >>> > backup nodes in the replicated cache if 'readFromBackup' flag is
>> >>> > enabled.
>> >>> >
>> >>> > I've prepared a solution [2] suggested by Alexei Scherbakov in Jira
>> >>> > comments. It passed prereviews by Alexei and Nikolay Izhikov.
>> >>> >
>> >>> > TeamCity tests look similar with the master branch.
>> >>> >
>> >>> > Could someone of core module maintainers do the final review [2][3]?
>> >>> >
>> >>> >
>> >>> > [1] https://issues.apache.org/jira/browse/IGNITE-5357
>> >>> > [2] https://github.com/apache/ignite/pull/3578
>> >>> > [3] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-509
>> >>> >
>> >>> > --
>> >>> > Best Regards, Vyacheslav D.
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Best Regards, Vyacheslav D.
>> >>>
>> >
>> >
>> >
>> > --
>> > Best Regards, Vyacheslav D.
>>
>>
>>
>> --
>> Best Regards, Vyacheslav D.
>>
>

[jira] [Created] (IGNITE-7956) MVCC TX: cache eviction operations for key-value API

[Alexander Paschenko (JIRA)]

Alexander Paschenko created IGNITE-7956:
---

 Summary: MVCC TX: cache eviction operations for key-value API
 Key: IGNITE-7956
 URL: https://issues.apache.org/jira/browse/IGNITE-7956
 Project: Ignite
  Issue Type: Task
  Components: cache
Reporter: Alexander Paschenko
 Fix For: 2.5


We need to implement MVCC-compatible cache eviction operations for key-value 
API.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7955) MVCC TX: cache peek for key-value API

[Alexander Paschenko (JIRA)]

Alexander Paschenko created IGNITE-7955:
---

 Summary: MVCC TX: cache peek for key-value API
 Key: IGNITE-7955
 URL: https://issues.apache.org/jira/browse/IGNITE-7955
 Project: Ignite
  Issue Type: Task
  Components: cache
Reporter: Alexander Paschenko
 Fix For: 2.5


We need to implement MVCC-compatible cache peek operation for key-value API.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3626: IGNITE-7898 Fixed IgniteCachePartitionLossPolicy ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3626


---

[GitHub] ignite pull request #3632: IGNITE-7947 Not all OWNING partitions saved in Pa...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3632


---

[GitHub] ignite pull request #3634: IGNITE-7879: Don't push down expressions with agg...

[tledkov-gridgain]

GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/3634

IGNITE-7879: Don't push down expressions with aggregate function



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7879

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3634.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3634


commit 7b5a3423886fe0de68362fce679d7bc32f0c78f6
Author: tledkov-gridgain 
Date:   2018-03-07T13:35:21Z

IGNITE-7879: Don't push down expressions with aggregate function




---

[jira] [Created] (IGNITE-7953) MVCC TX: continuous queries

[Alexander Paschenko (JIRA)]

Alexander Paschenko created IGNITE-7953:
---

 Summary: MVCC TX: continuous queries
 Key: IGNITE-7953
 URL: https://issues.apache.org/jira/browse/IGNITE-7953
 Project: Ignite
  Issue Type: Task
  Components: cache
Reporter: Alexander Paschenko
 Fix For: 2.5


We need to implement MVCC-compatible continuous queries.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7952) MVCC TX: cache clear routines for key-value API

[Alexander Paschenko (JIRA)]

Alexander Paschenko created IGNITE-7952:
---

 Summary: MVCC TX: cache clear routines for key-value API
 Key: IGNITE-7952
 URL: https://issues.apache.org/jira/browse/IGNITE-7952
 Project: Ignite
  Issue Type: Task
  Components: cache
Reporter: Alexander Paschenko
 Fix For: 2.5






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7951) Add metrics for remains to evict keys/partitions

[Alexander Belyak (JIRA)]

Alexander Belyak created IGNITE-7951:


 Summary: Add metrics for remains to evict keys/partitions
 Key: IGNITE-7951
 URL: https://issues.apache.org/jira/browse/IGNITE-7951
 Project: Ignite
  Issue Type: New Feature
  Components: general
Affects Versions: 2.4
Reporter: Alexander Belyak


Need to add some metrics for remains to evict keys/partitions to indicate total 
amount of evicting work. In some cases we have synchronous eviction and it's 
critically important to know how many keys need to be evicted before exchange 
process end and cluster became working again. In some other cases we just wanna 
know what happens in cluster now (background eviction without workload) and 
when cluster will became 100% healthy. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Deploying 2.4 artifacts to maven repo

[aaksenov]

it was uploaded to maven central no 05.03 and it seems it was most probably
just a local maven issue with cached metadata. Just not having it in
mvnrepository become confusing, sorry



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: Deploying 2.4 artifacts to maven repo

[aaksenov]

 just yesterday it couldn't be found (in http://repo.maven.apache.org) by
running mvn. Now that's there and ok!  thank you!



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[GitHub] ignite pull request #3633: IGNITE-7933 Checkpoint markers writing issues

[Jokser]

GitHub user Jokser opened a pull request:

https://github.com/apache/ignite/pull/3633

IGNITE-7933 Checkpoint markers writing issues

* Write checkpoint markers through temporary files
* Stop node if markCheckpointBegin is failed
* Reduced stop timeout
* Use Runtime.halt() to kill JVM if node is not stopped within timeout

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7933

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3633.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3633


commit c9138c3ea9d6d79141aeedf02c7e27c3e2a487fe
Author: Pavel Kovalenko 
Date:   2018-03-14T11:51:01Z

IGNITE-7933 Checkpoint markers writing issues:
* Write checkpoint markers through temporary files
* Stop node if markCheckpointBegin is failed
* Reduced stop timeout
* Use Runtime.halt() to kill JVM if node is not stopped within timeout




---

[jira] [Created] (IGNITE-7950) SQL: Add upload benchmarks for sql streamer feature

[Pavel Kuznetsov (JIRA)]

Pavel Kuznetsov created IGNITE-7950:
---

 Summary: SQL: Add upload benchmarks for sql streamer feature
 Key: IGNITE-7950
 URL: https://issues.apache.org/jira/browse/IGNITE-7950
 Project: Ignite
  Issue Type: Task
  Components: sql, yardstick
Reporter: Pavel Kuznetsov
Assignee: Pavel Kuznetsov


IGNITE-7253 introduces streaming mode. 
We need to update/create benchmarks to measure total upload time.
Data model remains the same as in IGNITE-7531 : incrementaly generated long 
primary key, 5 long and 5 strings (contains random long value in string 
representation)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3631: IGNITE-7946 Fail test with known issue

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3631


---

[jira] [Created] (IGNITE-7949) Web Console: Refactor post validation on sign in / sign up.

[Alexey Kuznetsov (JIRA)]

Alexey Kuznetsov created IGNITE-7949:


 Summary: Web Console: Refactor post validation on sign in / sign 
up.
 Key: IGNITE-7949
 URL: https://issues.apache.org/jira/browse/IGNITE-7949
 Project: Ignite
  Issue Type: Task
  Components: wizards
Reporter: Alexey Kuznetsov
Assignee: Ilya Borisov
 Fix For: 2.5


In current implementation we show ugly popover in case of server error or when 
backend unavailable at all.

 

Lets show errors as we do for input validation errors.

And when server not available lets show proper message.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3632: IGNITE-7947 Not all OWNING partitions saved in Pa...

[EdShangGG]

GitHub user EdShangGG opened a pull request:

https://github.com/apache/ignite/pull/3632

IGNITE-7947 Not all OWNING partitions saved in PartitionAllocationMap…

… during checkpoint

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7947

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3632.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3632


commit f7f419c42e9bb6d2b9e72c2fe1b8e007c82655fc
Author: Eduard Shangareev 
Date:   2018-03-14T10:56:24Z

IGNITE-7947 Not all OWNING partitions saved in PartitionAllocationMap 
during checkpoint




---

[jira] [Created] (IGNITE-7948) SQL: read only necessary fields into the row when possible

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-7948:
---

 Summary: SQL: read only necessary fields into the row when possible
 Key: IGNITE-7948
 URL: https://issues.apache.org/jira/browse/IGNITE-7948
 Project: Ignite
  Issue Type: Task
  Components: sql
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov


When H2 row is read, we always fill it with data eagerly through link 
materialization. Materialization is performed under page "read lock" what 
guarantees row-level consistency. This may lead to excessive memory pressure 
due to memory copying. For example, consider a class with 50 fields and a query 
which reads only 2 of them. 48 other fields will be copied without a reason. 
Lazy initialization is not an option because it will only defer memcpy, but not 
eliminate it. 

Instead we can try using H2. It passes {{TableFilter}} class to some of index 
access methods*. We can analyze this class and create the list of required 
fields. Then we can read these fields under read lock from offheap and put them 
to the row.

In addition to saved memcpy this could give us more benefits:
1) No more need for field cache ({{GridH2KeyValueRowOnheap#valCache}})
2) No more need to read {{_VER}} column and possibly {{_KEY}} or {{_VAL}}

But there are a number of drawbacks as well. E.g. it is impossible to read 
strings from offheap efficiently, so queries with VARCHAR will definitely 
suffer from this change.

\* {{org.h2.index.Index#find(org.h2.table.TableFilter, org.h2.result.SearchRow, 
org.h2.result.SearchRow)}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7947) Not all OWNING partitions saved in PartitionAllocationMap during checkpoint

[Eduard Shangareev (JIRA)]

Eduard Shangareev created IGNITE-7947:
-

 Summary: Not all OWNING partitions saved in PartitionAllocationMap 
during checkpoint
 Key: IGNITE-7947
 URL: https://issues.apache.org/jira/browse/IGNITE-7947
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Eduard Shangareev
Assignee: Eduard Shangareev
 Fix For: 2.5


In checkpoint begin listeners we collect PartitionAllocationMap in offheap 
managers. 
But if rowStore is null than we ignore this partition in any case even if we 
own it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3631: IGNITE-7946 Fail test with known issue

[Jokser]

GitHub user Jokser opened a pull request:

https://github.com/apache/ignite/pull/3631

IGNITE-7946 Fail test with known issue



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7946

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3631.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3631


commit 2dbc5efe69a02a07f670149c7ed2acfe507d6e9c
Author: Pavel Kovalenko 
Date:   2018-03-14T10:32:23Z

IGNITE-7946 Fail test with known issue




---

[jira] [Created] (IGNITE-7946) IgniteCacheClientQueryReplicatedNodeRestartSelfTest#testRestarts can hang on TC

[Pavel Kovalenko (JIRA)]

Pavel Kovalenko created IGNITE-7946:
---

 Summary: 
IgniteCacheClientQueryReplicatedNodeRestartSelfTest#testRestarts can hang on TC
 Key: IGNITE-7946
 URL: https://issues.apache.org/jira/browse/IGNITE-7946
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.4
Reporter: Pavel Kovalenko


According to test logs there can be unfinished rebalance:

{noformat}
[01:21:23] : [Step 4/5] [2018-03-13 22:21:23,327][INFO 
][exchange-worker-#456665%near.IgniteCacheClientQueryReplicatedNodeRestartSelfTest3%][GridDhtPartitionDemander]
 Cancelled rebalancing from all nodes [topology=AffinityTopologyVersion 
[topVer=103, minorTopVer=0]]
[01:21:23] : [Step 4/5] [2018-03-13 22:21:23,327][INFO 
][exchange-worker-#456665%near.IgniteCacheClientQueryReplicatedNodeRestartSelfTest3%][GridDhtPartitionDemander]
 Completed rebalance future: RebalanceFuture [grp=CacheGroupContext [grp=pr], 
topVer=AffinityTopologyVersion [topVer=103, minorTopVer=0], rebalanceId=1]
[01:21:23] : [Step 4/5] [2018-03-13 22:21:23,328][INFO 
][exchange-worker-#456665%near.IgniteCacheClientQueryReplicatedNodeRestartSelfTest3%][GridCachePartitionExchangeManager]
 Rebalancing scheduled [order=[pe, pr]]
[01:21:23] : [Step 4/5] [2018-03-13 22:21:23,328][INFO 
][exchange-worker-#456665%near.IgniteCacheClientQueryReplicatedNodeRestartSelfTest3%][GridCachePartitionExchangeManager]
 Rebalancing started [top=AffinityTopologyVersion [topVer=104, minorTopVer=0], 
evt=NODE_LEFT, node=04d02ea1-286c-4d8c-8870-e147c552]
[01:21:23] : [Step 4/5] [2018-03-13 22:21:23,328][INFO 
][exchange-worker-#456665%near.IgniteCacheClientQueryReplicatedNodeRestartSelfTest3%][GridDhtPartitionDemander]
 Starting rebalancing [grp=pe, mode=SYNC, 
fromNode=31193890-bf8f-4c85-af76-342efb31, partitionsCount=15, 
topology=AffinityTopologyVersion [topVer=104, minorTopVer=0], rebalanceId=2]
[01:21:23] : [Step 4/5] [2018-03-13 22:21:23,328][INFO 
][exchange-worker-#456665%near.IgniteCacheClientQueryReplicatedNodeRestartSelfTest3%][GridDhtPartitionDemander]
 Starting rebalancing [grp=pe, mode=SYNC, 
fromNode=517f4efb-4433-489a-8c8e-e91f9e70, partitionsCount=16, 
topology=AffinityTopologyVersion [topVer=104, minorTopVer=0], rebalanceId=2]
[01:21:23] : [Step 4/5] [2018-03-13 22:21:23,328][INFO 
][exchange-worker-#455983%near.IgniteCacheClientQueryReplicatedNodeRestartSelfTest0%][GridCachePartitionExchangeManager]
 Skipping rebalancing (nothing scheduled) [top=AffinityTopologyVersion 
[topVer=104, minorTopVer=0], evt=NODE_LEFT, 
node=04d02ea1-286c-4d8c-8870-e147c552]
[01:21:23] : [Step 4/5] [2018-03-13 22:21:23,332][INFO 
][sys-#456730%near.IgniteCacheClientQueryReplicatedNodeRestartSelfTest3%][GridDhtPartitionDemander]
 Completed rebalancing [fromNode=517f4efb-4433-489a-8c8e-e91f9e70, 
cacheOrGroup=pe, topology=AffinityTopologyVersion [topVer=104, minorTopVer=0], 
time=0 ms]

{noformat}


It can be cause of test hanging.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Apache Ignite nightly release builds

[vveider]

Prepared corresponding task [1], will start preparing build in the nearest
future.


[1] https://issues.apache.org/jira/browse/IGNITE-7945



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[jira] [Created] (IGNITE-7945) Apache Ignite nightly releases

[Peter Ivanov (JIRA)]

Peter Ivanov created IGNITE-7945:


 Summary: Apache Ignite nightly releases
 Key: IGNITE-7945
 URL: https://issues.apache.org/jira/browse/IGNITE-7945
 Project: Ignite
  Issue Type: Task
Reporter: Peter Ivanov
Assignee: Peter Ivanov


# As discussed 
[here|http://apache-ignite-developers.2346864.n4.nabble.com/Apache-Ignite-nightly-release-builds-td27966.html],
 it is necessary to prepare build on [AI TeamCity|ttps://ci.ignite.apache.org] 
which will run every night and will build release binaries for *{{master}}* 
branch.
# Prepare corresponding [Apache Ignite 
Documentation|https://ignite.apache.org/download.cgi] section for link to 
latest nightly release binaries.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Apache Ignite RPM packaging (Stage II)

[Petr Ivanov]

> On 14 Mar 2018, at 12:46, Dmitry Pavlov  wrote:
> 
> Hi,
> 
> Did not understand fully that splitted delivery means.
> Is it correct,that user will have install
> install ignite:core
> install ignite:opt_libs
> with separated commands?

It will be one command with several arguments, i.e.
yum install ignite-core ignite-optional

Also, I considered splitting optional libs per package (like php / perl / etc. 
do, for example) but not sure whether these efforts are appropriate at this 
moment.
I guess this scheme must have huge support from the community.


> 
> Or there will be some aggregating package like ignite:full and several
> options like ignite:core.

Nice idea! If community will agree, I can add aggregating virtual package with 
dependencies on all other for corresponding release. It can even keep current 
name ‘apache-ignite’ :)


> 
> Sincerely,
> Dmitriy Pavlov
> 
> ср, 14 мар. 2018 г. в 12:36, vveider :
> 
>> Hi, Igniters!
>> 
>> 
>> Release 2.4 is almost there, at least binary part of it, so I'd like to
>> move
>> forward to further improve and widen AI delivery through packages.
>> As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to
>> improve usability and significantly reduce required download sizes, I
>> purpose that in 2.5 release we introduce splitted delivery as follows:
>> - CORE
>>   - bin
>>   - config
>>   - libs (!optional)
>> - OPTIONAL LIBS
>> - BENCHMARKS
>> - DOCS (?)
>> - EXAMPLES
>> - .NET PLATFORM FILES
>> - C++ PLATFORM FILES
>> 
>> This architecture, as I assume, will add flexibility (no reason to download
>> all 280M+ of binaries where you are to run only core node functionality)
>> and
>> maintainability (you are in full control of what is installed on your
>> system).
>> 
>> After successful architecture choice, same scheme are planned to be used in
>> DEB packages as well.
>> 
>> WDYT?
>> 
>> 
>> 
>> --
>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>> 

[jira] [Created] (IGNITE-7944) Disconnected client node tries to send JOB_CANCEL message

[Roman Guseinov (JIRA)]

Roman Guseinov created IGNITE-7944:
--

 Summary: Disconnected client node tries to send JOB_CANCEL message
 Key: IGNITE-7944
 URL: https://issues.apache.org/jira/browse/IGNITE-7944
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.3, 1.9
 Environment: In case the network is blocked (socket connections not 
closed) and failure is detected, tcp-client-disco-msg-worker thread can be 
stuck in process of TcpClient creating:
{code:java}
"tcp-client-disco-msg-worker-#4%wd5prsvtots0016a-tg-QueryFabric%" #494 prio=5 
os_prio=0 tid=0x7f94c067c800 nid=0x2bdf runnable [0x7f960ecf1000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.Net.poll(Native Method)
at sun.nio.ch.SocketChannelImpl.poll(SocketChannelImpl.java:954)
- locked <0x7fa140f520c0> (a java.lang.Object)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:110)
- locked <0x7fa140f520b0> (a java.lang.Object)
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2950)
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:2681)
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2568)
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2429)
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:2393)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1590)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1659)
at 
org.apache.ignite.internal.processors.task.GridTaskWorker.cancelChildren(GridTaskWorker.java:1305)
at 
org.apache.ignite.internal.processors.task.GridTaskWorker.finishTask(GridTaskWorker.java:1609)
at 
org.apache.ignite.internal.processors.task.GridTaskWorker.finishTask(GridTaskWorker.java:1581)
at 
org.apache.ignite.internal.processors.task.GridTaskProcessor.onDisconnected(GridTaskProcessor.java:168)
at 
org.apache.ignite.internal.IgniteKernal.onDisconnected(IgniteKernal.java:3460)
at 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$4.onDiscovery(GridDiscoveryManager.java:601)
at 
org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.notifyDiscovery(ClientImpl.java:2407)
at 
org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.notifyDiscovery(ClientImpl.java:2386)
at 
org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.body(ClientImpl.java:1714)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
{code}
It looks like msg-worker is trying to send JOB_CANCEL message for each job with 
timeout equals failureDetectionTimeout.

Reproducer is attached.
Reporter: Roman Guseinov






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7943) Node hangs on start after failing during checkpoint.

[Oleg Ostanin (JIRA)]

Oleg Ostanin created IGNITE-7943:


 Summary: Node hangs on start after failing during checkpoint. 
 Key: IGNITE-7943
 URL: https://issues.apache.org/jira/browse/IGNITE-7943
 Project: Ignite
  Issue Type: Bug
Reporter: Oleg Ostanin


Node can restart more than 20 minutes after failing in the middle of a 
checkpoint.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Apache Ignite RPM packaging (Stage II)

[Dmitry Pavlov]

Hi,

Did not understand fully that splitted delivery means.
Is it correct,that user will have install
install ignite:core
install ignite:opt_libs
with separated commands?

Or there will be some aggregating package like ignite:full and several
options like ignite:core.

Sincerely,
Dmitriy Pavlov

ср, 14 мар. 2018 г. в 12:36, vveider :

> Hi, Igniters!
>
>
> Release 2.4 is almost there, at least binary part of it, so I'd like to
> move
> forward to further improve and widen AI delivery through packages.
> As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to
> improve usability and significantly reduce required download sizes, I
> purpose that in 2.5 release we introduce splitted delivery as follows:
>  - CORE
>- bin
>- config
>- libs (!optional)
>  - OPTIONAL LIBS
>  - BENCHMARKS
>  - DOCS (?)
>  - EXAMPLES
>  - .NET PLATFORM FILES
>  - C++ PLATFORM FILES
>
> This architecture, as I assume, will add flexibility (no reason to download
> all 280M+ of binaries where you are to run only core node functionality)
> and
> maintainability (you are in full control of what is installed on your
> system).
>
> After successful architecture choice, same scheme are planned to be used in
> DEB packages as well.
>
> WDYT?
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

Apache Ignite RPM packaging (Stage II)

[vveider]

Hi, Igniters!


Release 2.4 is almost there, at least binary part of it, so I'd like to move
forward to further improve and widen AI delivery through packages.
As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to
improve usability and significantly reduce required download sizes, I
purpose that in 2.5 release we introduce splitted delivery as follows:
 - CORE
   - bin
   - config
   - libs (!optional)
 - OPTIONAL LIBS
 - BENCHMARKS
 - DOCS (?)
 - EXAMPLES
 - .NET PLATFORM FILES
 - C++ PLATFORM FILES

This architecture, as I assume, will add flexibility (no reason to download
all 280M+ of binaries where you are to run only core node functionality) and
maintainability (you are in full control of what is installed on your
system).

After successful architecture choice, same scheme are planned to be used in
DEB packages as well.

WDYT?



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: IgniteSet implementation: changes required

[Pavel Pereslegin]

Hello Igniters.

I'm working on the implementation of the IgniteCache#asSet method [1]
and I think it should return Set (not IgniteSet). Because IgniteSet
was introduced mainly to add methods for the collocated version of
IgniteSet.

Any thoughts?

[1] https://issues.apache.org/jira/browse/IGNITE-7823


2018-02-27 14:59 GMT+03:00 Andrey Kuznetsov :
> As far as I know, Pavel P. is working on fixing existing sets currently.
>
> As for {{asSet}} cache adapter, I filed the ticket [1].
>
> [1] https://issues.apache.org/jira/browse/IGNITE-7823
>
> 2018-02-27 11:20 GMT+03:00 Vladimir Ozerov :
>
>> I think the root issue is that we are trying to mix different cases in a
>> single solution. What is the common usage patterns of sets?
>> 1) Small mostly-read sets - current implementation is ideal for them -
>> everything is available locally, on-heap and in deserialized form
>> 2) Big data sets - IgniteCache API is the best candidate here; we can
>> simply add a method "Set IgniteCache.asSet()" which will return thin
>> wrapper for Set interface around cache. An you will have recovery and
>> heap-resistance with almost no additional efforts.
>> There is no need to choose between one or another solution. We can support
>> both.
>>
>> As far as current igniteSet implementation - all issues described above
>> will go away if you use approach suggested by Dmitry. I do not see a need
>> for any major changes.
>>
>> Vladimir.
>>
>
> --
> Best regards,
>   Andrey Kuznetsov.

[GitHub] ignite pull request #3591: IGNITE-7253

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3591


---

[GitHub] ignite pull request #3630: IGNITE-7831 Throw Exceptions instead of Assertion...

[alex-plekhanov]

GitHub user alex-plekhanov opened a pull request:

https://github.com/apache/ignite/pull/3630

IGNITE-7831 Throw Exceptions instead of AssertionErrors when reading from 
corrupted persistence



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/alex-plekhanov/ignite ignite-7831

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3630.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3630


commit 1698c15ce0d47d5fbb8dbc633fbf007fb5e7c0e3
Author: Aleksey Plekhanov 
Date:   2018-03-13T11:09:54Z

IGNITE-7831 Throw Exceptions instead of AssertionErrors when reading from 
corrupted persistence

commit 905a1a0065eb853f70e5bb86bb850c27707f4b8d
Author: Aleksey Plekhanov 
Date:   2018-03-13T11:46:10Z

IGNITE-7831 WIP

commit fd94878610d70697181e6ffcfe939073d8bc273f
Author: Aleksey Plekhanov 
Date:   2018-03-14T08:39:49Z

IGNITE-7831 WIP




---

[jira] [Created] (IGNITE-7942) Document streaming in thin JDBC driver

[Alexander Paschenko (JIRA)]

Alexander Paschenko created IGNITE-7942:
---

 Summary: Document streaming in thin JDBC driver
 Key: IGNITE-7942
 URL: https://issues.apache.org/jira/browse/IGNITE-7942
 Project: Ignite
  Issue Type: Task
  Components: documentation, jdbc, sql
Reporter: Alexander Paschenko
 Fix For: 2.5






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Why does DataStreamer swallow exceptions?

[Ilya Kasnacheev]

Hello Val!

No, this does NOT happen. If you collect future, call get() on it later, it
will finish normally despite exceptions in server log and entry not being
written. I will do some digging to figure out why this happens exactly.

There's also another huge problems with DataStreamer's futures. They never
finish unless you call flush() on DS before calling get() on futures.
I think this is a colossal usability problem (I'm pretty sure I've seen
numerous messages about it on userlist) and I'll fill an issue if nobody is
objecting.

Ilya.

-- 
Ilya Kasnacheev

2018-03-05 22:50 GMT+03:00 Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Ilya,
>
> IgniteDataStreamer#addData method returns future which should be completed
> with error if one is thrown on server side. Does this happen or not?
>
> -Val
>
> On Mon, Mar 5, 2018 at 4:10 AM, Nikolay Izhikov 
> wrote:
>
> > Hello, Ilya.
> >
> > > I think it's time to end this, if that was the case. DataStreamer
> should
> > > not be a special case and it should guarantee data safety. WDYT?
> >
> > +1 from me.
> >
> > I'm also facing this issue.
> >
> > Ticket - https://issues.apache.org/jira/browse/IGNITE-7756
> > Discussion - http://apache-ignite-developers.2346864.n4.nabble.
> > com/IgniteDataStreamer-silently-fails-on-a-server-node-td27239.html
> >
> >
> >
> > В Пн, 05/03/2018 в 14:46 +0300, Ilya Kasnacheev пишет:
> > > Dear Igniters, why do I have a hunch that DataStreamer would readily
> > > swallow exceptions?
> > >
> > > DataStreamerImpl:1756 swallows marshalling error, lines 1774 & 1781 eat
> > > deployment errors.
> > >
> > > Some people are worried they can fill a leaking vessel without noticing
> > > anything off.
> > > Also in line 2156 fsync() on WAL can throw exceptions, which will be
> > > swallowed, and IMP this fsync doesn't belong to DataStreamer code.
> > >
> > > This question is not purely theoretical, we have also replaced one of
> > these
> > > eaters with throw with IGNITE-7519.
> > >
> > > There was a fault in PDS implementation, which did not propagate to
> > client.
> > > A serious issue IMHO.
> > >
> > > As a data streamer user I will expect that flush()/close() will throw
> any
> > > pending exceptions and will only be silent if all data landed safely in
> > the
> > > cluster.
> > >
> > > I also have this feeling that DataStreamer was written using very
> > internal
> > > APIs so that it can compromise guarantees that cache and SQL APIs are
> > bound
> > > by. I think I've heard something about not recovering properly in case
> of
> > > node failures.
> > > I think it's time to end this, if that was the case. DataStreamer
> should
> > > not be a special case and it should guarantee data safety. WDYT?
> > >
> >
>

Re: [RESULT] [VOTE] Apache Ignite 2.4.0 Release (RC1)

[Petr Ivanov]

Guess, I figured out cloud images matter: those images have embedded scripts 
for running corresponding docker container, so it looks like no update for them 
is required except date on site, which update policy I’m an unaware of.



> On 14 Mar 2018, at 09:43, Petr Ivanov  wrote:
> 
> 1. RPM Package Installation
>* missing ‘[ignite]’ header;
>* rest of text is indented by 1 space, while should not.
> 2. Building the Binaries
>* I guess mentioning Java 7 in 2.4.0 release is obsolete — that block 
> should be either split (which AI requires which JDK for compiling) or just 
> get rid of Java 7 mentioning.
> 3. Docker and Cloud Images
>* still not clear what to do with AWS / Google Cloud — how to update those 
> images. Maybe Nikolay Tikhonov can help?
> 
> 
> 
>> On 13 Mar 2018, at 22:27, Denis Magda  wrote:
>> 
>> Then tell me what to correct and I'll merge the changes.
>> 
>> Alternatively, you can send me an SVN patch with changes if it's simples:
>> https://cwiki.apache.org/confluence/display/IGNITE/Website+Development
>> 
>> --
>> Denis
>> 
>> On Tue, Mar 13, 2018 at 12:25 PM, Petr Ivanov  wrote:
>> 
>>> I have no access to site and readme.io  has no errors.
>>> 
>>> 
 On 13 Mar 2018, at 22:23, Denis Magda  wrote:
 
 Petr,
 
 Have you corrected the site and readme.io docs (if it was required)?
 
 --
 Denis
 
 On Mon, Mar 12, 2018 at 11:14 PM, Petr Ivanov 
>>> wrote:
 
> There is an error in RPM documentation on site — missing header for
> ignite.repo file and formatting is also wrong.
> Otherwise — I’m able to install and run Apache Ignite from RPM from ASF
> according to manual.
> 
> 
> 
>> On 12 Mar 2018, at 23:25, Denis Magda  wrote:
>> 
>> Released RPM installation section on the downloads page:
>> https://ignite.apache.org/download.cgi#rpm-package
>> 
>> *Petr*, please double check that the section and the readme getting
> started
>> look good:
>> https://apacheignite.readme.io/docs/getting-started#
>>> section-rpm-package-
> installation
>> 
>> *Mauricio*, could you please create a patch that points out "latest"
>>> docs
>> to "2.4.0"?
>> 
>> --
>> Denis
>> 
>> 
>> 
>> On Mon, Mar 12, 2018 at 7:36 AM, Pavel Tupitsyn 
>> wrote:
>> 
>>> NuGet packages uploaded: https://www.nuget.org/
>>> packages?q=Apache.Ignite
>>> 
>>> Thanks,
>>> Pavel
>>> 
>>> 
>>> On Mon, Mar 12, 2018 at 1:54 PM, Vladimir Ozerov <
>>> voze...@gridgain.com>
>>> wrote:
>>> 
 Correct, it is a matter of time. Download links already work on my
>>> side
 [1].
 
 [1] http://mirror.linux-ia64.org/apache/ignite/2.4.0/
 
 On Mon, Mar 12, 2018 at 1:29 PM, Petr Ivanov 
>>> wrote:
 
> Link to 2.4.0 is pointing to RBC mirror, not Apache Dist itself.
> 
> I do not know how often RBC updates its mirror, but I think we have
>>> to
> devise better solution for pointing to fresh release artifacts.
> 
> 
> 
>> On 12 Mar 2018, at 13:24, Pavel Tupitsyn 
>>> wrote:
>> 
>> Hi Vladimir,
>> 
>> I see the link for 2.4.0 on https://ignite.apache.org/
> download.cgi#binaries,
>> but it does not work.
>> 
>> On Mon, Mar 12, 2018 at 10:55 AM, Vladimir Ozerov <
 voze...@gridgain.com>
>> wrote:
>> 
>>> Correction: " Apache Ignite *2.4.0* release (RC1) has been
>>> accepted."
>>> 
>>> On Mon, Mar 12, 2018 at 10:51 AM, Vladimir Ozerov <
 voze...@gridgain.com
>> 
>>> wrote:
>>> 
 Igniters,
 
 Apache Ignite 2.3.0 release (RC1) has been accepted.
 
 5 "+1" binding votes received:
 - Alexey Goncharuk
 - Alexey Kuznetsov
 - Anton Vinogradov
 - Denis Magda
 - Pavel Tupitsyn
 
 Vote thread:
 
 *http://apache-ignite-developers.2346864.n4.nabble.
>>> com/VOTE-Apache-Ignite-2-4-0-RC1-td27687.html
 >> com/VOTE-Apache-Ignite-2-4-0-RC1-td27687.html>*
 
 Ignite 2.4.0 will be released soon.
 
 Vladimir.
 
>>> 
> 
> 
 
>>> 
> 
> 
>>> 
>>> 
> 

Re: [RESULT] [VOTE] Apache Ignite 2.4.0 Release (RC1)

[Petr Ivanov]

1. RPM Package Installation
* missing ‘[ignite]’ header;
* rest of text is indented by 1 space, while should not.
2. Building the Binaries
* I guess mentioning Java 7 in 2.4.0 release is obsolete — that block 
should be either split (which AI requires which JDK for compiling) or just get 
rid of Java 7 mentioning.
3. Docker and Cloud Images
* still not clear what to do with AWS / Google Cloud — how to update those 
images. Maybe Nikolay Tikhonov can help?



> On 13 Mar 2018, at 22:27, Denis Magda  wrote:
> 
> Then tell me what to correct and I'll merge the changes.
> 
> Alternatively, you can send me an SVN patch with changes if it's simples:
> https://cwiki.apache.org/confluence/display/IGNITE/Website+Development
> 
> --
> Denis
> 
> On Tue, Mar 13, 2018 at 12:25 PM, Petr Ivanov  wrote:
> 
>> I have no access to site and readme.io  has no errors.
>> 
>> 
>>> On 13 Mar 2018, at 22:23, Denis Magda  wrote:
>>> 
>>> Petr,
>>> 
>>> Have you corrected the site and readme.io docs (if it was required)?
>>> 
>>> --
>>> Denis
>>> 
>>> On Mon, Mar 12, 2018 at 11:14 PM, Petr Ivanov 
>> wrote:
>>> 
 There is an error in RPM documentation on site — missing header for
 ignite.repo file and formatting is also wrong.
 Otherwise — I’m able to install and run Apache Ignite from RPM from ASF
 according to manual.
 
 
 
> On 12 Mar 2018, at 23:25, Denis Magda  wrote:
> 
> Released RPM installation section on the downloads page:
> https://ignite.apache.org/download.cgi#rpm-package
> 
> *Petr*, please double check that the section and the readme getting
 started
> look good:
> https://apacheignite.readme.io/docs/getting-started#
>> section-rpm-package-
 installation
> 
> *Mauricio*, could you please create a patch that points out "latest"
>> docs
> to "2.4.0"?
> 
> --
> Denis
> 
> 
> 
> On Mon, Mar 12, 2018 at 7:36 AM, Pavel Tupitsyn 
> wrote:
> 
>> NuGet packages uploaded: https://www.nuget.org/
>> packages?q=Apache.Ignite
>> 
>> Thanks,
>> Pavel
>> 
>> 
>> On Mon, Mar 12, 2018 at 1:54 PM, Vladimir Ozerov <
>> voze...@gridgain.com>
>> wrote:
>> 
>>> Correct, it is a matter of time. Download links already work on my
>> side
>>> [1].
>>> 
>>> [1] http://mirror.linux-ia64.org/apache/ignite/2.4.0/
>>> 
>>> On Mon, Mar 12, 2018 at 1:29 PM, Petr Ivanov 
>> wrote:
>>> 
 Link to 2.4.0 is pointing to RBC mirror, not Apache Dist itself.
 
 I do not know how often RBC updates its mirror, but I think we have
>> to
 devise better solution for pointing to fresh release artifacts.
 
 
 
> On 12 Mar 2018, at 13:24, Pavel Tupitsyn 
>> wrote:
> 
> Hi Vladimir,
> 
> I see the link for 2.4.0 on https://ignite.apache.org/
 download.cgi#binaries,
> but it does not work.
> 
> On Mon, Mar 12, 2018 at 10:55 AM, Vladimir Ozerov <
>>> voze...@gridgain.com>
> wrote:
> 
>> Correction: " Apache Ignite *2.4.0* release (RC1) has been
>> accepted."
>> 
>> On Mon, Mar 12, 2018 at 10:51 AM, Vladimir Ozerov <
>>> voze...@gridgain.com
> 
>> wrote:
>> 
>>> Igniters,
>>> 
>>> Apache Ignite 2.3.0 release (RC1) has been accepted.
>>> 
>>> 5 "+1" binding votes received:
>>> - Alexey Goncharuk
>>> - Alexey Kuznetsov
>>> - Anton Vinogradov
>>> - Denis Magda
>>> - Pavel Tupitsyn
>>> 
>>> Vote thread:
>>> 
>>> *http://apache-ignite-developers.2346864.n4.nabble.
>> com/VOTE-Apache-Ignite-2-4-0-RC1-td27687.html
>>> > com/VOTE-Apache-Ignite-2-4-0-RC1-td27687.html>*
>>> 
>>> Ignite 2.4.0 will be released soon.
>>> 
>>> Vladimir.
>>> 
>> 
 
 
>>> 
>> 
 
 
>> 
>>