Re: NodeJS thin client: full API

[Denis Magda]

Alexey,

Presently, Ignite hosts all the docs in readme.io without exception. It
means that once your contribution is accepted by the community the Node.JS
docs should be placed on readme.io.

You're right saying that we're planning to migrate from readme.io to
another documentation engine that would allow us storing doc sources in
Ignite repo. It might happen by 2.6 or might take longer.

Thus, we need to host the Node.JS docs on readme.io and edit them there
once your pull-request is merged (it means there wouldn't be docs' copy
added to Ignite repo for now). It's easy to move the docs to readme.io
which understands the standard markdown. I'll ask Prachi to assist here.

--
Denis

On Fri, May 11, 2018 at 1:45 PM, Alexey Kosenchuk <
alexey.kosenc...@nobitlost.com> wrote:

> Denis,
>
> > As for the docs, are you ready to bring them to readme.io? Just let me
> know
> > and I'll be happy to arrange an account for you and discuss the
> structure.
>
> I remember some discussion regarding moving the docs from readme.io to
> GitHub pages in 2.6.
> No?
>
> In any case, in my opinion, a readme near the code is a right primary
> place for the docs for thin clients.
> Is there any script/automation to convert .md to readme.io?
> Or maybe just place a link from the readme.io to the repo readme?
> Manual support of the same docs in two places seems not an effective
> solution.
>
> The docs for NodeJS client is ready for review in the repo.
> The links and the installation procedure will have to be updated when the
> client is integrated into the apache repo and released on npmjs.
>
> -Alexey
>

Re: NodeJS thin client: full API

[Alexey Kosenchuk]

Denis,

> As for the docs, are you ready to bring them to readme.io? Just let 
me know
> and I'll be happy to arrange an account for you and discuss the 
structure.


I remember some discussion regarding moving the docs from readme.io to 
GitHub pages in 2.6.

No?

In any case, in my opinion, a readme near the code is a right primary 
place for the docs for thin clients.

Is there any script/automation to convert .md to readme.io?
Or maybe just place a link from the readme.io to the repo readme?
Manual support of the same docs in two places seems not an effective 
solution.


The docs for NodeJS client is ready for review in the repo.
The links and the installation procedure will have to be updated when 
the client is integrated into the apache repo and released on npmjs.


-Alexey

Re: async operation is not fair async

[Dmitriy Setrakyan]

Vladimir,

In general I agree, but I do get greatly *close-minded* (pun intended)
whenever users' code that worked for the past several years all of a sudden
gets deadlocked after an upgrade. Making this feature optional is even
worse and more confusing. In this case the best action is no action at all.

BTW, would be interesting to find out how Oracle async driver behaves in
this case.

D.





On Fri, May 11, 2018 at 8:29 PM, Vladimir Ozerov 
wrote:

> Guys,
>
> To build a great product we should be open minded and look to the future,
> not to the past.
>
> Dima raised very valid point - why async is not async? Current programming
> culture and demanding performance requirements pushes users towards
> reactive-style programming. I do not want my thread to ever be blocked.
> Instead, I want to send a number of concurrent commands and optionally
> subscribe to final result. So trully async API makes total sense to me.
>
> But personally, my primary interest in this area is SQL. Oracle is
> preparing new async driver. ADBA - async database access. It was presented
> on recent JavaOne [1]. It is under active development right now - juse
> weave through the mailing list [2]. Some prototypes are already there [3].
> PostgreSQL community even started adopted it [4]!
>
> I am not pushing for immediate actions, but at least we should understand
> which way the wind is blowing. As a mid-term goals I would propose to
> finally remove thread ID from our PESSIMISTIC transactions to allow for
> suspend/resume in different threads. And as a next step I would think on
> adopting async cache and SQL APIs.
>
> Vladimir.
>
> [1]
> http://www.oracle.com/technetwork/database/application-development/jdbc/
> con1491-3961036.pdf
> [2] http://mail.openjdk.java.net/pipermail/jdbc-spec-discuss/
> [3] https://github.com/oracle/oracle-db-examples/tree/master/java/AoJ
> [4] https://github.com/pgjdbc/pgjdbc/issues/978
>
> On Fri, May 11, 2018 at 9:48 PM, Dmitriy Setrakyan 
> wrote:
>
> > On Fri, May 11, 2018 at 7:46 PM, Dmitriy Govorukhin <
> > dmitriy.govoruk...@gmail.com> wrote:
> >
> > > I will edit IGNITE-8475, and remove all part that belong to the public
> > api.
> > > Is it acceptable for you?
> > >
> >
> > Everything is acceptable, as long as the public API is safe :)
> >
>

Re: async operation is not fair async

[Vladimir Ozerov]

Guys,

To build a great product we should be open minded and look to the future,
not to the past.

Dima raised very valid point - why async is not async? Current programming
culture and demanding performance requirements pushes users towards
reactive-style programming. I do not want my thread to ever be blocked.
Instead, I want to send a number of concurrent commands and optionally
subscribe to final result. So trully async API makes total sense to me.

But personally, my primary interest in this area is SQL. Oracle is
preparing new async driver. ADBA - async database access. It was presented
on recent JavaOne [1]. It is under active development right now - juse
weave through the mailing list [2]. Some prototypes are already there [3].
PostgreSQL community even started adopted it [4]!

I am not pushing for immediate actions, but at least we should understand
which way the wind is blowing. As a mid-term goals I would propose to
finally remove thread ID from our PESSIMISTIC transactions to allow for
suspend/resume in different threads. And as a next step I would think on
adopting async cache and SQL APIs.

Vladimir.

[1]
http://www.oracle.com/technetwork/database/application-development/jdbc/con1491-3961036.pdf
[2] http://mail.openjdk.java.net/pipermail/jdbc-spec-discuss/
[3] https://github.com/oracle/oracle-db-examples/tree/master/java/AoJ
[4] https://github.com/pgjdbc/pgjdbc/issues/978

On Fri, May 11, 2018 at 9:48 PM, Dmitriy Setrakyan 
wrote:

> On Fri, May 11, 2018 at 7:46 PM, Dmitriy Govorukhin <
> dmitriy.govoruk...@gmail.com> wrote:
>
> > I will edit IGNITE-8475, and remove all part that belong to the public
> api.
> > Is it acceptable for you?
> >
>
> Everything is acceptable, as long as the public API is safe :)
>

Re: NodeJS thin client: full API

[Denis Magda]

Alexey,

Amazing progress and the overall state of the contribution! I'll try to
install and play with the client in the nearest days. As for now, please
consider the following suggestions.

Please do the following changes:

   - Let's rename SqlDataProcessingExample.js to just SqlExample.js and
   describe it as an example that shows primary APIs to use with Ignite as
   with an SQL database.
   - Let's rename SqlQueryExample.js to SqlQueryEntiriesExample.js.

As for the docs, are you ready to bring them to readme.io? Just let me know
and I'll be happy to arrange an account for you and discuss the structure.

--
Denis

On Thu, May 10, 2018 at 4:07 PM, Alexey Kosenchuk <
alexey.kosenc...@nobitlost.com> wrote:

> Folks,
>
> The next version is ready -
> in the pull request [1] or directly in the repo [2].
>
> The version includes:
> - full API
> - full implementation
> - examples
> - tests (cover the full API but might need to be updated/extended)
> - docs
>
> The details are in the readme [3]
>
> Regards,
> -Alexey
>
> [1] https://github.com/apache/ignite/pull/3978
> [2] https://github.com/nobitlost/ignite/tree/master/modules/plat
> forms/nodejs
> [3] https://github.com/nobitlost/ignite/blob/master/modules/plat
> forms/nodejs/README.md
>

Re: NodeJS thin client: full API

[Alexey Kosenchuk]

Hi Igor,

all types from the protocol v.2.4 are supported for both read and write.
See [1] for the mappings and explanation.

The idea is to minimize additional non-standard and/or 3rd-party 
JavaScript Objects for that.


Timestamp and Enum require additional classes - no better options were 
found. So, Timestamp and EnumItem classes have been introduced.


For Decimal - a 3rd-party Decimal type [2] seems the best choice and is 
utilized.


For all other Ignite types - only standard JavaScript types and Objects 
are used.

Eg. for UUID - it is simply an array of numbers (16-elements).

Regards,
-Alexey

[1] 
https://rawgit.com/nobitlost/ignite/master/modules/platforms/nodejs/api_spec/ObjectType.html

[2] https://github.com/MikeMcl/decimal.js


11.05.2018 19:38, Igor Sapego пишет:

Alexey,

I've checked out the code. Looks good to me. Great job!

What about data types support? I can see Timestamp.
Are you planning to implement other types, e.g. Decimal,
Guid?

Best Regards,
Igor

On Fri, May 11, 2018 at 11:43 AM, Dmitriy Setrakyan 
wrote:


On Fri, May 11, 2018 at 9:14 AM, Alexey Kosenchuk <
alexey.kosenc...@nobitlost.com> wrote:


Not yet. Need a help with that.



I think we definitely need a load test before we merge to master. Can
anyone in the community assist Alexey?

Re: Ignite v2.6 release dates

[Denis Magda]

Right, it depends on the community. I'm just sharing how the process works
in general.

--
Denis

On Fri, May 11, 2018 at 11:27 AM, Dmitry Pavlov 
wrote:

> Hi Pavel,
>
> Denis, please correct me if I'm wrong,
>
> Recently we have removed baseline topology usability fixes from 2.5 scope.
>
> If usability fix would be ready soon, there is possibility that community
> will agree with 2.6 to be prepared early.
>
> Sincerely,
> Dmitriy Pavlov
>
> пт, 11 мая 2018 г. в 21:23, Denis Magda :
>
> > Pavel,
> >
> > The scope has not been discussed or defined for 2.6 yet. Usually, we do
> > this right after releasing a version that is in progress.
> >
> > As a rule of thumb, Ignite is released once in 3 months. So, if Ignite
> 2.5
> > is released in May, then 2.6 will be projected towards the end of August.
> >
> > --
> > Denis
> >
> > On Fri, May 11, 2018 at 11:08 AM, Pavel Petroshenko <
> pa...@petroshenko.com
> > >
> > wrote:
> >
> > > Igniters,
> > >
> > > Is the schedule for v2.6 finalized already? If I want a
> feature/component
> > > I'm working on to get into the release, when should it be
> (code/feature)
> > > frozen? Thanks.
> > >
> > > p.
> > >
> >
>

Re: async operation is not fair async

[Dmitriy Setrakyan]

On Fri, May 11, 2018 at 7:46 PM, Dmitriy Govorukhin <
dmitriy.govoruk...@gmail.com> wrote:

> I will edit IGNITE-8475, and remove all part that belong to the public api.
> Is it acceptable for you?
>

Everything is acceptable, as long as the public API is safe :)

Re: async operation is not fair async

[Dmitriy Setrakyan]

On Fri, May 11, 2018 at 7:23 PM, Dmitriy Govorukhin <
dmitriy.govoruk...@gmail.com> wrote:

> Dmitriy S,
>
> If it will be in the internal package, and only for internal usage, are you
> agree with changes?
>

Yes, but please be careful not to create deadlocks for ourselves.

Can you please close the ticket?

D.

Re: async operation is not fair async

[Dmitriy Govorukhin]

Dmitriy S,

If it will be in the internal package, and only for internal usage, are you
agree with changes?

пт, 11 мая 2018 г., 21:12 Dmitriy Setrakyan :

> On Fri, May 11, 2018 at 6:49 PM, Dmitriy Govorukhin <
> dmitriy.govoruk...@gmail.com> wrote:
>
> > Dmitriy S,
> >
> > It is not broke existing code, because for use this ability you must use
> > decorator "withFairSycn()".
> >
> > What about the argument of Vladimir?
> >
>
> Here is Vladimir's quote:
>
>
> *This would also be helpful for transactional SQL as it would allow to
> hide**network
> latency. But there is a problem - deadlocks. We need to inform user*
> *that this mode should be used with great care. *
>
>
> I would rather not change anything instead of increasing the probability of
> deadlocks. This was the main reason for the current behavior to begin with.
>
> In my view, if something is needed for the transactional SQL, please add it
> internally. Let's not corrupt the public API by adding dangerous methods.
>
> D.
>

[GitHub] ignite pull request #3986: IGNITE-8469: release memory in case initializatio...

[Mmuzaf]

GitHub user Mmuzaf opened a pull request:

https://github.com/apache/ignite/pull/3986

IGNITE-8469: release memory in case initialization multi times



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Mmuzaf/ignite ignite-8469

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3986.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3986


commit 38c0b2e3d26f3190f6d981ffa5f9a5862610f795
Author: Maxim Muzafarov 
Date:   2018-05-11T18:23:28Z

IGNITE-8469: release memory in case initialization multi times




---

Re: Ignite v2.6 release dates

[Denis Magda]

Pavel,

The scope has not been discussed or defined for 2.6 yet. Usually, we do
this right after releasing a version that is in progress.

As a rule of thumb, Ignite is released once in 3 months. So, if Ignite 2.5
is released in May, then 2.6 will be projected towards the end of August.

--
Denis

On Fri, May 11, 2018 at 11:08 AM, Pavel Petroshenko 
wrote:

> Igniters,
>
> Is the schedule for v2.6 finalized already? If I want a feature/component
> I'm working on to get into the release, when should it be (code/feature)
> frozen? Thanks.
>
> p.
>

Re: async operation is not fair async

[Dmitriy Setrakyan]

On Fri, May 11, 2018 at 6:49 PM, Dmitriy Govorukhin <
dmitriy.govoruk...@gmail.com> wrote:

> Dmitriy S,
>
> It is not broke existing code, because for use this ability you must use
> decorator "withFairSycn()".
>
> What about the argument of Vladimir?
>

Here is Vladimir's quote:


*This would also be helpful for transactional SQL as it would allow to
hide**network
latency. But there is a problem - deadlocks. We need to inform user*
*that this mode should be used with great care. *


I would rather not change anything instead of increasing the probability of
deadlocks. This was the main reason for the current behavior to begin with.

In my view, if something is needed for the transactional SQL, please add it
internally. Let's not corrupt the public API by adding dangerous methods.

D.

Ignite v2.6 release dates

[Pavel Petroshenko]

Igniters,

Is the schedule for v2.6 finalized already? If I want a feature/component
I'm working on to get into the release, when should it be (code/feature)
frozen? Thanks.

p.

[GitHub] ignite pull request #3985: IGNITE-8320 Corrupted indexes fix

[Jokser]

GitHub user Jokser opened a pull request:

https://github.com/apache/ignite/pull/3985

IGNITE-8320 Corrupted indexes fix



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8320-reproduce

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3985.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3985


commit ffb362929decc431b325bccc8c612a049f85063f
Author: Pavel Kovalenko 
Date:   2018-05-11T15:32:32Z

IGNITE-8320 Reproducer.

commit 37765277286a18198255bcbc2286073706ef6048
Author: Pavel Kovalenko 
Date:   2018-05-11T15:33:42Z

IGNITE-8320 Reproducer.

commit d1d265ae98ab79d6d80a667e4a844ea86f724e32
Author: Pavel Kovalenko 
Date:   2018-05-11T15:34:47Z

IGNITE-8320 Docs fix.

commit 234b1f8fcf24d849227e5e73e26fb81e0768cf21
Author: Pavel Kovalenko 
Date:   2018-05-11T15:36:01Z

IGNITE-8320 Docs fix.

commit 951d67e93677358470416a5faabe238b6e2bb21a
Author: Pavel Kovalenko 
Date:   2018-05-11T16:46:15Z

IGNITE-8320 Fix WIP.

commit a1acab629dfce81e904bdc6fac92458b60a7ac48
Author: Pavel Kovalenko 
Date:   2018-05-11T17:51:00Z

IGNITE-8320 Fix WIP.




---

Re: async operation is not fair async

[Dmitriy Govorukhin]

Dmitriy S,

It is not broke existing code, because for use this ability you must use
decorator "withFairSycn()".

What about the argument of Vladimir?

On Fri, May 11, 2018 at 8:41 PM, Dmitriy Setrakyan 
wrote:

> On Fri, May 11, 2018 at 6:38 PM, Dmitriy Govorukhin <
> dmitriy.govoruk...@gmail.com> wrote:
>
> > Dmitriy S,
> >
> > Why method named as "async" but does not work as async? This is
> misleading.
> >
> > getAllAsync() is a special case. Not always you can use getAllAsync()
> > instead
> > of multiple getAsync().
> > In this topic, I wanna discuss problem not only for the GET operation but
> > also all async operation behavior in the one thread.
> >
> > In compute grid we can run multiple async compute operation in the one
> > thread, so why we can not do this for cache?
>
>
> Because it will break a lot of existing code and create bugs we cannot even
> predict at this point. I am not sure why has this become a problem. Is it
> preventing us from accomplishing some other task? If not, then I propose to
> drop it.
>
> D.
>

[GitHub] ignite pull request #3984: IGNITE-8471 Dependencies upgraded

[agura]

GitHub user agura opened a pull request:

https://github.com/apache/ignite/pull/3984

IGNITE-8471 Dependencies upgraded



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/agura/incubator-ignite ignite-8471

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3984.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3984


commit 50c69ace2834a2ed6cbb4d828ee75a0dc157e208
Author: Andrey Gura 
Date:   2018-05-11T17:41:30Z

IGNITE-8471 Dependencies upgraded




---

Re: async operation is not fair async

[Dmitriy Setrakyan]

On Fri, May 11, 2018 at 6:38 PM, Dmitriy Govorukhin <
dmitriy.govoruk...@gmail.com> wrote:

> Dmitriy S,
>
> Why method named as "async" but does not work as async? This is misleading.
>
> getAllAsync() is a special case. Not always you can use getAllAsync()
> instead
> of multiple getAsync().
> In this topic, I wanna discuss problem not only for the GET operation but
> also all async operation behavior in the one thread.
>
> In compute grid we can run multiple async compute operation in the one
> thread, so why we can not do this for cache?


Because it will break a lot of existing code and create bugs we cannot even
predict at this point. I am not sure why has this become a problem. Is it
preventing us from accomplishing some other task? If not, then I propose to
drop it.

D.

Re: async operation is not fair async

[Dmitriy Govorukhin]

Dmitriy S,

Why method named as "async" but does not work as async? This is misleading.

getAllAsync() is a special case. Not always you can use getAllAsync() instead
of multiple getAsync().
In this topic, I wanna discuss problem not only for the GET operation but
also all async operation behavior in the one thread.

In compute grid we can run multiple async compute operation in the one
thread, so why we can not do this for cache?


On Fri, May 11, 2018 at 8:33 PM, Dmitriy Setrakyan 
wrote:

> On Fri, May 11, 2018 at 6:29 PM, Dmitry Pavlov 
> wrote:
>
> > IMO you can complete async operations one before another if these
> > operations are related to independent data.
> >
> > It is strange why Ignite users are not confused by current API. So I
> > support Dmitriy's G. suggestion.
> >
>
> Again, this is a solution looking for a problem. Nobody complains about
> this, so there really isn't any issue. There are so many other tasks we
> could focus on. Let's not fix what's not broken.
>
> D.
>

Re: async operation is not fair async

[Dmitriy Setrakyan]

On Fri, May 11, 2018 at 6:29 PM, Dmitry Pavlov 
wrote:

> IMO you can complete async operations one before another if these
> operations are related to independent data.
>
> It is strange why Ignite users are not confused by current API. So I
> support Dmitriy's G. suggestion.
>

Again, this is a solution looking for a problem. Nobody complains about
this, so there really isn't any issue. There are so many other tasks we
could focus on. Let's not fix what's not broken.

D.

Re: async operation is not fair async

[Dmitry Pavlov]

IMO you can complete async operations one before another if these
operations are related to independent data.

It is strange why Ignite users are not confused by current API. So I
support Dmitriy's G. suggestion.

пт, 11 мая 2018 г. в 20:24, Dmitriy Setrakyan :

> Guys,
>
> I am not sure I like this approach, especially for this code:
>
> f1=cache.getAsync(key1);
> f2=cache.getAsync(key2);
>
> You cannot complete f2 before f1. If you do, the code is unusable and it is
> impossible to predict anything. If you need to get 2 elements
> asynchronously, use getAllAsync() instead.
>
> Let's not introduce such usability issues into our API and stop trying to
> fix what's not broken. I propose to close the ticket as "Won't Fix".
>
> D.
>
> On Fri, May 11, 2018 at 6:14 PM, Dmitriy Govorukhin <
> dmitriy.govoruk...@gmail.com> wrote:
>
> > Igniters,
> >
> > I created the issue. IGNITE-8475
> > 
> >
> > Any comments are welcome.
> >
> > On Fri, May 11, 2018 at 6:32 PM, Ivan Rakov 
> wrote:
> >
> > > Agree. "fair" is more descriptive.
> > >
> > > Best Regards,
> > > Ivan Rakov
> > >
> > >
> > > On 11.05.2018 18:30, Dmitriy Govorukhin wrote:
> > >
> > >> Ivan,
> > >>
> > >> My suggestion "withFairAsync()". What do you think?
> > >>
> > >> On Fri, May 11, 2018 at 6:23 PM, Ivan Rakov 
> > >> wrote:
> > >>
> > >> I think, the best option from API side is to add decorating
> > >>> withExplicitAsync() method.
> > >>> We already have withKeepBinary, withExpiryPolicy and so on.
> > >>>
> > >>> Best Regards,
> > >>> Ivan Rakov
> > >>>
> > >>>
> > >>> On 11.05.2018 18:18, Dmitriy Govorukhin wrote:
> > >>>
> > >>> Vladimir,
> > 
> >  Should we create the new cache adapter or rework GridCacheAdapter?
> > 
> >  On Fri, May 11, 2018 at 5:52 PM, Vladimir Ozerov <
> > voze...@gridgain.com>
> >  wrote:
> > 
> >  +1
> > 
> > > This would also be helpful for transactional SQL as it would allow
> to
> > > hide
> > > network latency. But there is a problem - deadlocks. We need to
> > inform
> > > user
> > > that this mode should be used with great care.
> > >
> > > On Fri, May 11, 2018 at 5:21 PM, Dmitriy Govorukhin <
> > > dmitriy.govoruk...@gmail.com> wrote:
> > >
> > > Hi Igniters,
> > >
> > >> I have a question. Why our async operation in not really async?
> > >>
> > >> GridCacheAdapter.syncOp has awaitLastFut(); this call wait last
> > async
> > >> operation completed.
> > >>
> > >> This means all async operation in one thread will be executed one
> by
> > >> one
> > >> but not in parallel. Async operation is not async.
> > >>
> > >> Example for atomic cache
> > >>
> > >> f1=cache.getAsync(key1);
> > >> f2=cache.getAsync(key2);
> > >>
> > >> f1 always will be complete before f2.
> > >>
> > >> I want to have the ability run multiple async operations in one
> > >> thread.
> > >> What do you think?
> > >>
> > >> Maybe we can add new cache adapter with fair async operations?
> > >>
> > >>
> > >>
> > >
> >
>

Re: async operation is not fair async

[Dmitriy Setrakyan]

Guys,

I am not sure I like this approach, especially for this code:

f1=cache.getAsync(key1);
f2=cache.getAsync(key2);

You cannot complete f2 before f1. If you do, the code is unusable and it is
impossible to predict anything. If you need to get 2 elements
asynchronously, use getAllAsync() instead.

Let's not introduce such usability issues into our API and stop trying to
fix what's not broken. I propose to close the ticket as "Won't Fix".

D.

On Fri, May 11, 2018 at 6:14 PM, Dmitriy Govorukhin <
dmitriy.govoruk...@gmail.com> wrote:

> Igniters,
>
> I created the issue. IGNITE-8475
> 
>
> Any comments are welcome.
>
> On Fri, May 11, 2018 at 6:32 PM, Ivan Rakov  wrote:
>
> > Agree. "fair" is more descriptive.
> >
> > Best Regards,
> > Ivan Rakov
> >
> >
> > On 11.05.2018 18:30, Dmitriy Govorukhin wrote:
> >
> >> Ivan,
> >>
> >> My suggestion "withFairAsync()". What do you think?
> >>
> >> On Fri, May 11, 2018 at 6:23 PM, Ivan Rakov 
> >> wrote:
> >>
> >> I think, the best option from API side is to add decorating
> >>> withExplicitAsync() method.
> >>> We already have withKeepBinary, withExpiryPolicy and so on.
> >>>
> >>> Best Regards,
> >>> Ivan Rakov
> >>>
> >>>
> >>> On 11.05.2018 18:18, Dmitriy Govorukhin wrote:
> >>>
> >>> Vladimir,
> 
>  Should we create the new cache adapter or rework GridCacheAdapter?
> 
>  On Fri, May 11, 2018 at 5:52 PM, Vladimir Ozerov <
> voze...@gridgain.com>
>  wrote:
> 
>  +1
> 
> > This would also be helpful for transactional SQL as it would allow to
> > hide
> > network latency. But there is a problem - deadlocks. We need to
> inform
> > user
> > that this mode should be used with great care.
> >
> > On Fri, May 11, 2018 at 5:21 PM, Dmitriy Govorukhin <
> > dmitriy.govoruk...@gmail.com> wrote:
> >
> > Hi Igniters,
> >
> >> I have a question. Why our async operation in not really async?
> >>
> >> GridCacheAdapter.syncOp has awaitLastFut(); this call wait last
> async
> >> operation completed.
> >>
> >> This means all async operation in one thread will be executed one by
> >> one
> >> but not in parallel. Async operation is not async.
> >>
> >> Example for atomic cache
> >>
> >> f1=cache.getAsync(key1);
> >> f2=cache.getAsync(key2);
> >>
> >> f1 always will be complete before f2.
> >>
> >> I want to have the ability run multiple async operations in one
> >> thread.
> >> What do you think?
> >>
> >> Maybe we can add new cache adapter with fair async operations?
> >>
> >>
> >>
> >
>

Re: async operation is not fair async

[Dmitriy Govorukhin]

Igniters,

I created the issue. IGNITE-8475


Any comments are welcome.

On Fri, May 11, 2018 at 6:32 PM, Ivan Rakov  wrote:

> Agree. "fair" is more descriptive.
>
> Best Regards,
> Ivan Rakov
>
>
> On 11.05.2018 18:30, Dmitriy Govorukhin wrote:
>
>> Ivan,
>>
>> My suggestion "withFairAsync()". What do you think?
>>
>> On Fri, May 11, 2018 at 6:23 PM, Ivan Rakov 
>> wrote:
>>
>> I think, the best option from API side is to add decorating
>>> withExplicitAsync() method.
>>> We already have withKeepBinary, withExpiryPolicy and so on.
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>>
>>> On 11.05.2018 18:18, Dmitriy Govorukhin wrote:
>>>
>>> Vladimir,

 Should we create the new cache adapter or rework GridCacheAdapter?

 On Fri, May 11, 2018 at 5:52 PM, Vladimir Ozerov 
 wrote:

 +1

> This would also be helpful for transactional SQL as it would allow to
> hide
> network latency. But there is a problem - deadlocks. We need to inform
> user
> that this mode should be used with great care.
>
> On Fri, May 11, 2018 at 5:21 PM, Dmitriy Govorukhin <
> dmitriy.govoruk...@gmail.com> wrote:
>
> Hi Igniters,
>
>> I have a question. Why our async operation in not really async?
>>
>> GridCacheAdapter.syncOp has awaitLastFut(); this call wait last async
>> operation completed.
>>
>> This means all async operation in one thread will be executed one by
>> one
>> but not in parallel. Async operation is not async.
>>
>> Example for atomic cache
>>
>> f1=cache.getAsync(key1);
>> f2=cache.getAsync(key2);
>>
>> f1 always will be complete before f2.
>>
>> I want to have the ability run multiple async operations in one
>> thread.
>> What do you think?
>>
>> Maybe we can add new cache adapter with fair async operations?
>>
>>
>>
>

[jira] [Created] (IGNITE-8475) Create new IgniteCache decorator with fair async methonds

[Dmitriy Govorukhin (JIRA)]

Dmitriy Govorukhin created IGNITE-8475:
--

 Summary: Create new IgniteCache decorator with fair async methonds
 Key: IGNITE-8475
 URL: https://issues.apache.org/jira/browse/IGNITE-8475
 Project: Ignite
  Issue Type: Improvement
  Components: cache
Affects Versions: 2.4
Reporter: Dmitriy Govorukhin
 Fix For: None


GridCacheAdapter.syncOp has awaitLastFut(); this call wait last async 
operation completed. 

This means all async operation in one thread will be executed one by one but
not in parallel. Async operation is not async. 

Example for atomic cache 

f1=cache.getAsync(key1); 
f2=cache.getAsync(key2); 

f1 always will be complete before f2. 

Need to create a new decorator for IgniteCache, and return IgniteCache proxy 
with fair async 

operations.

 IgniteCache.withFairAsync()



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: NodeJS thin client: full API

[Igor Sapego]

Alexey,

I've checked out the code. Looks good to me. Great job!

What about data types support? I can see Timestamp.
Are you planning to implement other types, e.g. Decimal,
Guid?

Best Regards,
Igor

On Fri, May 11, 2018 at 11:43 AM, Dmitriy Setrakyan 
wrote:

> On Fri, May 11, 2018 at 9:14 AM, Alexey Kosenchuk <
> alexey.kosenc...@nobitlost.com> wrote:
>
> > Not yet. Need a help with that.
> >
>
> I think we definitely need a load test before we merge to master. Can
> anyone in the community assist Alexey?
>

[GitHub] ignite pull request #3983: IGNITE-7999: for 2.4

[tledkov-gridgain]

GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/3983

IGNITE-7999: for 2.4



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7999-2.4-next

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3983.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3983


commit 915dd2966084d78f7b4f3d482e6bd25f860c1e23
Author: Alexey Goncharuk 
Date:   2018-01-31T08:22:26Z

IGNITE-7569 Fixed index rebuild future - Fixes #3454.

commit 8ea8609259039852ab0c26f26ac528c1ffae7c94
Author: Alexey Goncharuk 
Date:   2018-01-31T08:24:57Z

IGNITE-7577 Fixing public API active flag on baseline changes - Fixes #3455.

commit c8ce1f66e98b3174d771a3b801a2538499dc2c3d
Author: Ivan Rakov 
Date:   2018-01-31T09:51:09Z

IGNITE-7475 Improved VerifyBackupPartitionsTask to calculate partition 
hashes in parallel - Fixes #3407.

Signed-off-by: Alexey Goncharuk 

commit 258ff4299da20122d7c387cb8579264035c93c18
Author: Alexey Goncharuk 
Date:   2018-01-31T13:52:24Z

IGNITE-7573 Fixed full API tests to be compliant with baseline topology

commit 254ed3a9c32d092702a0461509bf867cbd7cdee6
Author: Alexey Kuznetsov 
Date:   2018-02-01T08:22:53Z

ignite-2.4.0 Update version.

(cherry picked from commit 2e43749)

commit c1a9c0a404d77fba08170bedf14844f87abe3028
Author: Alexey Goncharuk 
Date:   2018-02-01T10:17:28Z

IGNITE-7569 Fixing index rebuild future

commit e43799ce70cdbe03d9e206381d1d5138b820b075
Author: Alexey Goncharuk 
Date:   2018-02-01T13:39:17Z

IGNITE-7520 Provide util-methods to get baseline from context - Fixes #3431.

commit 8f5fc7cfb0624cf2048efad38dfff32f782116e8
Author: Sergey Chugunov 
Date:   2018-02-02T08:24:29Z

IGNITE-7580 Fix compatibilityMode flag consistency

This closes #3466

(cherry picked from commit 8f2045e)

commit d3ddd50cb2b889173176b6c47c9ff61410e1d909
Author: Ilya Lantukh 
Date:   2018-02-07T10:33:28Z

IGNITE-7514 Affinity assignment should be recalculated when primary node is 
not OWNER

(cherry picked from commit faf50f1)

commit d3745e9d0a3ff5a64fba494889b7e2605f3af6bb
Author: Alexey Goncharuk 
Date:   2018-02-07T18:10:32Z

IGNITE-7639 Fixed NPE

commit f7c16855ba802d9d47048521aec7e14285e4a281
Author: Pavel Kovalenko 
Date:   2018-02-09T13:55:15Z

IGNITE-7540 Prevent page memory metadata corruption during checkpoint and 
group destroying. - Fixes #3490.

Signed-off-by: Alexey Goncharuk 

commit c92f167fc491078f02b9f94fe89edafc2902ebc2
Author: ilantukh 
Date:   2018-02-14T12:40:13Z

Updated version in properties.

commit 1ecf348dd429cf7861b414e0e5a7776b72dba281
Author: Sergey Chugunov 
Date:   2018-02-16T13:21:12Z

IGNITE-7699 BinaryMetadata exchange should not be triggered if metadata was 
not updated - Fixes #3523.

Signed-off-by: Alexey Goncharuk 

(cherry-picked from commit bcd3881)

commit 2458bd08a5b501b3eeb5caf0ae6dcaa2bcccd915
Author: EdShangGG 
Date:   2018-02-16T13:29:49Z

IGNITE-7676 Add affinity version to snapshot plugin stub - Fixes #3510.

Signed-off-by: Alexey Goncharuk 
(cherry picked from commit b6d21fb)

commit bfdcda7a2a6b5cf64f15ed169d2beb886f131fac
Author: EdShangGG 
Date:   2018-02-12T16:36:30Z

IGNITE-7626 Unify code in test which cleans up persistence directories - 
Fixes #3477.

Signed-off-by: Alexey Goncharuk 
(cherry picked from commit a0997b9)

commit 2e92e0094b270aa8489e66d94bfcf15eadabfb4f
Author: EdShangGG 
Date:   2018-02-12T18:44:10Z

IGNITE-7626 Unify code in test which clean up persistence directories - 
Fixes #3512.

Signed-off-by: Alexey Goncharuk 
(cherry picked from commit 6f6f8dd)

commit 3f86c127c78065999663a4fc4eaedb5e5d4bee1c
Author: EdShangGG 
Date:   2018-02-12T18:26:31Z

compilation fix

commit 0b9322c566f9b464291854142ac02495bd1817e4
Author: gg-shq 
Date:   2018-02-07T11:28:04Z

IGNITE-6917: Implemented SQL COPY command.

commit c5e386ca96750213bddcd98d0af0c589fee476ca
Author: gg-shq 
Date:   2018-02-07T15:31:27Z

IGNITE-7586: Added COPY command into the JDBC example.

This closes #3485

commit d8203e2d81f8fbf0f7fbe5e710c9908f2fcb8307
Author: shq 
Date:   

[GitHub] ignite pull request #3231: ignite-5874: Store TTL expire times in B+ tree on...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3231


---

[GitHub] ignite pull request #3982: IGNITE-8464 removed file format after archive

[akalash]

GitHub user akalash opened a pull request:

https://github.com/apache/ignite/pull/3982

IGNITE-8464 removed file format after archive



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8464

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3982.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3982


commit 502e52d687e3b796b351cbb8e916d331a64675c6
Author: Anton Kalashnikov 
Date:   2018-05-11T15:40:13Z

IGNITE-8464 removed file format after archive




---

Re: Disable WAL for several cache groups within one exchange

[Ivan Rakov]

Agree about collections.

Regarding return type: it's a tricky question. Maybe author of this 
feature may help.

Anton V., in which case enableWal/disableWal can return false?

Best Regards,
Ivan Rakov

On 11.05.2018 18:19, Vladimir Ozerov wrote:

Ivan,

This proven to be too hard to understand.  It is better to have a lot small
methods with clear and compact semantics. Also arrays are harder to manage
than collections, users typically prefer the latest.
Also we need to think on what would be a result of this operation. Current
methods with a single cache return true/false based on whether they changed
something or not. Should we continue returning a single boolean for batch
operations as well?

On Fri, May 11, 2018 at 6:13 PM, Ivan Rakov  wrote:


It would be six methods in total (3 for enabling, 3 for disabling).
What about accepting null in *enableWAL(String... caches)* as wildcard?

Best Regards,
Ivan Rakov


On 11.05.2018 17:52, Andrey Mashenkov wrote:


Ivan,

Huge +1 for this improvement.

I think we can have 2 overloaded method enableWal() with no args to enable
WAL for all caches
and enableWAL(String... caches) for one or multiple caches. (and same for
disable wal)



On Fri, May 11, 2018 at 5:25 PM, Dmitriy Govorukhin <
dmitriy.govoruk...@gmail.com> wrote:

Ivan,

Agree, if we have the batch method for cache create, we should have the
ability to enable/disable WAL in the batch too.

On Fri, May 11, 2018 at 5:17 PM, Ivan Rakov 
wrote:

Igniters,

API method for disabling WAL in IgniteCluster accepts only one cache


name.


Every call triggers exchange and checkpoints cluster-wide - it takes


plenty


of time to disable/enable WAL for multiple caches.
I think, we should add option to disable/enable WAL for several caches
with single command.

Thoughts?

--
Best Regards,
Ivan Rakov

Re: async operation is not fair async

[Ivan Rakov]

Agree. "fair" is more descriptive.

Best Regards,
Ivan Rakov

On 11.05.2018 18:30, Dmitriy Govorukhin wrote:

Ivan,

My suggestion "withFairAsync()". What do you think?

On Fri, May 11, 2018 at 6:23 PM, Ivan Rakov  wrote:


I think, the best option from API side is to add decorating
withExplicitAsync() method.
We already have withKeepBinary, withExpiryPolicy and so on.

Best Regards,
Ivan Rakov


On 11.05.2018 18:18, Dmitriy Govorukhin wrote:


Vladimir,

Should we create the new cache adapter or rework GridCacheAdapter?

On Fri, May 11, 2018 at 5:52 PM, Vladimir Ozerov 
wrote:

+1

This would also be helpful for transactional SQL as it would allow to
hide
network latency. But there is a problem - deadlocks. We need to inform
user
that this mode should be used with great care.

On Fri, May 11, 2018 at 5:21 PM, Dmitriy Govorukhin <
dmitriy.govoruk...@gmail.com> wrote:

Hi Igniters,

I have a question. Why our async operation in not really async?

GridCacheAdapter.syncOp has awaitLastFut(); this call wait last async
operation completed.

This means all async operation in one thread will be executed one by one
but not in parallel. Async operation is not async.

Example for atomic cache

f1=cache.getAsync(key1);
f2=cache.getAsync(key2);

f1 always will be complete before f2.

I want to have the ability run multiple async operations in one thread.
What do you think?

Maybe we can add new cache adapter with fair async operations?

Re: async operation is not fair async

[Dmitriy Govorukhin]

Ivan,

My suggestion "withFairAsync()". What do you think?

On Fri, May 11, 2018 at 6:23 PM, Ivan Rakov  wrote:

> I think, the best option from API side is to add decorating
> withExplicitAsync() method.
> We already have withKeepBinary, withExpiryPolicy and so on.
>
> Best Regards,
> Ivan Rakov
>
>
> On 11.05.2018 18:18, Dmitriy Govorukhin wrote:
>
>> Vladimir,
>>
>> Should we create the new cache adapter or rework GridCacheAdapter?
>>
>> On Fri, May 11, 2018 at 5:52 PM, Vladimir Ozerov 
>> wrote:
>>
>> +1
>>>
>>> This would also be helpful for transactional SQL as it would allow to
>>> hide
>>> network latency. But there is a problem - deadlocks. We need to inform
>>> user
>>> that this mode should be used with great care.
>>>
>>> On Fri, May 11, 2018 at 5:21 PM, Dmitriy Govorukhin <
>>> dmitriy.govoruk...@gmail.com> wrote:
>>>
>>> Hi Igniters,

 I have a question. Why our async operation in not really async?

 GridCacheAdapter.syncOp has awaitLastFut(); this call wait last async
 operation completed.

 This means all async operation in one thread will be executed one by one
 but not in parallel. Async operation is not async.

 Example for atomic cache

 f1=cache.getAsync(key1);
 f2=cache.getAsync(key2);

 f1 always will be complete before f2.

 I want to have the ability run multiple async operations in one thread.
 What do you think?

 Maybe we can add new cache adapter with fair async operations?


>

Re: async operation is not fair async

[Ivan Rakov]

I think, the best option from API side is to add decorating 
withExplicitAsync() method.

We already have withKeepBinary, withExpiryPolicy and so on.

Best Regards,
Ivan Rakov

On 11.05.2018 18:18, Dmitriy Govorukhin wrote:

Vladimir,

Should we create the new cache adapter or rework GridCacheAdapter?

On Fri, May 11, 2018 at 5:52 PM, Vladimir Ozerov 
wrote:


+1

This would also be helpful for transactional SQL as it would allow to hide
network latency. But there is a problem - deadlocks. We need to inform user
that this mode should be used with great care.

On Fri, May 11, 2018 at 5:21 PM, Dmitriy Govorukhin <
dmitriy.govoruk...@gmail.com> wrote:


Hi Igniters,

I have a question. Why our async operation in not really async?

GridCacheAdapter.syncOp has awaitLastFut(); this call wait last async
operation completed.

This means all async operation in one thread will be executed one by one
but not in parallel. Async operation is not async.

Example for atomic cache

f1=cache.getAsync(key1);
f2=cache.getAsync(key2);

f1 always will be complete before f2.

I want to have the ability run multiple async operations in one thread.
What do you think?

Maybe we can add new cache adapter with fair async operations?

Re: Disable WAL for several cache groups within one exchange

[Vladimir Ozerov]

Ivan,

This proven to be too hard to understand.  It is better to have a lot small
methods with clear and compact semantics. Also arrays are harder to manage
than collections, users typically prefer the latest.
Also we need to think on what would be a result of this operation. Current
methods with a single cache return true/false based on whether they changed
something or not. Should we continue returning a single boolean for batch
operations as well?

On Fri, May 11, 2018 at 6:13 PM, Ivan Rakov  wrote:

> It would be six methods in total (3 for enabling, 3 for disabling).
> What about accepting null in *enableWAL(String... caches)* as wildcard?
>
> Best Regards,
> Ivan Rakov
>
>
> On 11.05.2018 17:52, Andrey Mashenkov wrote:
>
>> Ivan,
>>
>> Huge +1 for this improvement.
>>
>> I think we can have 2 overloaded method enableWal() with no args to enable
>> WAL for all caches
>> and enableWAL(String... caches) for one or multiple caches. (and same for
>> disable wal)
>>
>>
>>
>> On Fri, May 11, 2018 at 5:25 PM, Dmitriy Govorukhin <
>> dmitriy.govoruk...@gmail.com> wrote:
>>
>> Ivan,
>>>
>>> Agree, if we have the batch method for cache create, we should have the
>>> ability to enable/disable WAL in the batch too.
>>>
>>> On Fri, May 11, 2018 at 5:17 PM, Ivan Rakov 
>>> wrote:
>>>
>>> Igniters,

 API method for disabling WAL in IgniteCluster accepts only one cache

>>> name.
>>>
 Every call triggers exchange and checkpoints cluster-wide - it takes

>>> plenty
>>>
 of time to disable/enable WAL for multiple caches.
 I think, we should add option to disable/enable WAL for several caches
 with single command.

 Thoughts?

 --
 Best Regards,
 Ivan Rakov



>>
>>
>

Re: async operation is not fair async

[Dmitriy Govorukhin]

Vladimir,

Should we create the new cache adapter or rework GridCacheAdapter?

On Fri, May 11, 2018 at 5:52 PM, Vladimir Ozerov 
wrote:

> +1
>
> This would also be helpful for transactional SQL as it would allow to hide
> network latency. But there is a problem - deadlocks. We need to inform user
> that this mode should be used with great care.
>
> On Fri, May 11, 2018 at 5:21 PM, Dmitriy Govorukhin <
> dmitriy.govoruk...@gmail.com> wrote:
>
> > Hi Igniters,
> >
> > I have a question. Why our async operation in not really async?
> >
> > GridCacheAdapter.syncOp has awaitLastFut(); this call wait last async
> > operation completed.
> >
> > This means all async operation in one thread will be executed one by one
> > but not in parallel. Async operation is not async.
> >
> > Example for atomic cache
> >
> > f1=cache.getAsync(key1);
> > f2=cache.getAsync(key2);
> >
> > f1 always will be complete before f2.
> >
> > I want to have the ability run multiple async operations in one thread.
> > What do you think?
> >
> > Maybe we can add new cache adapter with fair async operations?
> >
>

Re: Disable WAL for several cache groups within one exchange

[Ivan Rakov]

It would be six methods in total (3 for enabling, 3 for disabling).
What about accepting null in *enableWAL(String... caches)* as wildcard?

Best Regards,
Ivan Rakov

On 11.05.2018 17:52, Andrey Mashenkov wrote:

Ivan,

Huge +1 for this improvement.

I think we can have 2 overloaded method enableWal() with no args to enable
WAL for all caches
and enableWAL(String... caches) for one or multiple caches. (and same for
disable wal)



On Fri, May 11, 2018 at 5:25 PM, Dmitriy Govorukhin <
dmitriy.govoruk...@gmail.com> wrote:


Ivan,

Agree, if we have the batch method for cache create, we should have the
ability to enable/disable WAL in the batch too.

On Fri, May 11, 2018 at 5:17 PM, Ivan Rakov  wrote:


Igniters,

API method for disabling WAL in IgniteCluster accepts only one cache

name.

Every call triggers exchange and checkpoints cluster-wide - it takes

plenty

of time to disable/enable WAL for multiple caches.
I think, we should add option to disable/enable WAL for several caches
with single command.

Thoughts?

--
Best Regards,
Ivan Rakov

Re: Disable WAL for several cache groups within one exchange

[Ivan Rakov]

Vladimir,

I fully agree. We don't expose groups to public API (except for cache 
configuration), so we shouldn't start doing it here.


Ignite issue for this improvement: 
https://issues.apache.org/jira/browse/IGNITE-8473


Best Regards,
Ivan Rakov

On 11.05.2018 17:49, Vladimir Ozerov wrote:

Ivan,

WAL disable for multiple caches was not implemented in the first place for
a reason. We striven to maintain API consistency,
"IgniteCluster.disableWal(cacheName)" maps exactly to "ALTER TABLE
tableName NOLOGGING" command.  There is no SQL equivalent for multiple
caches. WAL disable for multiple tables is typically handled through
tablespace which is an arbitrary logical group of objects sharing the same
physical files. We have only cache group notion which is a set of tables
with the same backup and affinity properties sharing the same files. As I
hope we would replace groups with tablespaces my preference is not to
expose any SQL commands for cache groups. Neither I would expose cache
groups to public API unless absolutely needed.

I would suggest to extend native API with a command accepting multiple
cache names, not cache groups:

IgniteCluster.disableWal(Collection cacheNames)
IgniteCluster.enableWal(Collection cacheNames)

Makes sense?

Vladimir.

On Fri, May 11, 2018 at 5:25 PM, Dmitriy Govorukhin <
dmitriy.govoruk...@gmail.com> wrote:


Ivan,

Agree, if we have the batch method for cache create, we should have the
ability to enable/disable WAL in the batch too.

On Fri, May 11, 2018 at 5:17 PM, Ivan Rakov  wrote:


Igniters,

API method for disabling WAL in IgniteCluster accepts only one cache

name.

Every call triggers exchange and checkpoints cluster-wide - it takes

plenty

of time to disable/enable WAL for multiple caches.
I think, we should add option to disable/enable WAL for several caches
with single command.

Thoughts?

--
Best Regards,
Ivan Rakov

[jira] [Created] (IGNITE-8474) WalStateNodeLeaveExchangeTask prevents merge exchanges on leaving many nodes

[Sergey Chugunov (JIRA)]

Sergey Chugunov created IGNITE-8474:
---

 Summary: WalStateNodeLeaveExchangeTask prevents merge exchanges on 
leaving many nodes
 Key: IGNITE-8474
 URL: https://issues.apache.org/jira/browse/IGNITE-8474
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.5
Reporter: Sergey Chugunov
Assignee: Sergey Chugunov
 Fix For: 2.6


Exchange merge mechanism provides huge optimization when a lot of nodes join or 
leave cluster simultaneously.

In IGNITE-7003 WalStateNodeLeaveExchangeTask custom exchange task was added 
which is generated for each NODE_LEFT/NODE_FAILED event.
Because of property skipForExchangeMerge set to false on this task it prohibits 
exchange merge optimization.

The property needs to be reconsidered and changed if it is not crucial for this 
functionality.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8473) Add option to enable/disable WAL for several caches with single command

[Ivan Rakov (JIRA)]

Ivan Rakov created IGNITE-8473:
--

 Summary: Add option to enable/disable WAL for several caches with 
single command
 Key: IGNITE-8473
 URL: https://issues.apache.org/jira/browse/IGNITE-8473
 Project: Ignite
  Issue Type: Improvement
Reporter: Ivan Rakov
 Fix For: 2.6


API method for disabling WAL in IgniteCluster accepts only one cache name. 
Every call triggers exchange and checkpoints cluster-wide - it takes plenty of 
time to disable/enable WAL for multiple caches.
We should add option to disable/enable WAL for several caches with single 
command. 

New proposed API methods:
IgniteCluster.disableWal(Collection cacheNames)
IgniteCluster.enableWal(Collection cacheNames)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3789: IGNITE-7999 Enhance performance of the thin JDBC ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3789


---

Re: async operation is not fair async

[Vladimir Ozerov]

+1

This would also be helpful for transactional SQL as it would allow to hide
network latency. But there is a problem - deadlocks. We need to inform user
that this mode should be used with great care.

On Fri, May 11, 2018 at 5:21 PM, Dmitriy Govorukhin <
dmitriy.govoruk...@gmail.com> wrote:

> Hi Igniters,
>
> I have a question. Why our async operation in not really async?
>
> GridCacheAdapter.syncOp has awaitLastFut(); this call wait last async
> operation completed.
>
> This means all async operation in one thread will be executed one by one
> but not in parallel. Async operation is not async.
>
> Example for atomic cache
>
> f1=cache.getAsync(key1);
> f2=cache.getAsync(key2);
>
> f1 always will be complete before f2.
>
> I want to have the ability run multiple async operations in one thread.
> What do you think?
>
> Maybe we can add new cache adapter with fair async operations?
>

Re: Disable WAL for several cache groups within one exchange

[Andrey Mashenkov]

Ivan,

Huge +1 for this improvement.

I think we can have 2 overloaded method enableWal() with no args to enable
WAL for all caches
and enableWAL(String... caches) for one or multiple caches. (and same for
disable wal)



On Fri, May 11, 2018 at 5:25 PM, Dmitriy Govorukhin <
dmitriy.govoruk...@gmail.com> wrote:

> Ivan,
>
> Agree, if we have the batch method for cache create, we should have the
> ability to enable/disable WAL in the batch too.
>
> On Fri, May 11, 2018 at 5:17 PM, Ivan Rakov  wrote:
>
> > Igniters,
> >
> > API method for disabling WAL in IgniteCluster accepts only one cache
> name.
> > Every call triggers exchange and checkpoints cluster-wide - it takes
> plenty
> > of time to disable/enable WAL for multiple caches.
> > I think, we should add option to disable/enable WAL for several caches
> > with single command.
> >
> > Thoughts?
> >
> > --
> > Best Regards,
> > Ivan Rakov
> >
> >
>



-- 
Best regards,
Andrey V. Mashenkov

Re: Disable WAL for several cache groups within one exchange

[Vladimir Ozerov]

Ivan,

WAL disable for multiple caches was not implemented in the first place for
a reason. We striven to maintain API consistency,
"IgniteCluster.disableWal(cacheName)" maps exactly to "ALTER TABLE
tableName NOLOGGING" command.  There is no SQL equivalent for multiple
caches. WAL disable for multiple tables is typically handled through
tablespace which is an arbitrary logical group of objects sharing the same
physical files. We have only cache group notion which is a set of tables
with the same backup and affinity properties sharing the same files. As I
hope we would replace groups with tablespaces my preference is not to
expose any SQL commands for cache groups. Neither I would expose cache
groups to public API unless absolutely needed.

I would suggest to extend native API with a command accepting multiple
cache names, not cache groups:

IgniteCluster.disableWal(Collection cacheNames)
IgniteCluster.enableWal(Collection cacheNames)

Makes sense?

Vladimir.

On Fri, May 11, 2018 at 5:25 PM, Dmitriy Govorukhin <
dmitriy.govoruk...@gmail.com> wrote:

> Ivan,
>
> Agree, if we have the batch method for cache create, we should have the
> ability to enable/disable WAL in the batch too.
>
> On Fri, May 11, 2018 at 5:17 PM, Ivan Rakov  wrote:
>
> > Igniters,
> >
> > API method for disabling WAL in IgniteCluster accepts only one cache
> name.
> > Every call triggers exchange and checkpoints cluster-wide - it takes
> plenty
> > of time to disable/enable WAL for multiple caches.
> > I think, we should add option to disable/enable WAL for several caches
> > with single command.
> >
> > Thoughts?
> >
> > --
> > Best Regards,
> > Ivan Rakov
> >
> >
>

Re: Disable WAL for several cache groups within one exchange

[Dmitriy Govorukhin]

Ivan,

Agree, if we have the batch method for cache create, we should have the
ability to enable/disable WAL in the batch too.

On Fri, May 11, 2018 at 5:17 PM, Ivan Rakov  wrote:

> Igniters,
>
> API method for disabling WAL in IgniteCluster accepts only one cache name.
> Every call triggers exchange and checkpoints cluster-wide - it takes plenty
> of time to disable/enable WAL for multiple caches.
> I think, we should add option to disable/enable WAL for several caches
> with single command.
>
> Thoughts?
>
> --
> Best Regards,
> Ivan Rakov
>
>

async operation is not fair async

[Dmitriy Govorukhin]

Hi Igniters,

I have a question. Why our async operation in not really async?

GridCacheAdapter.syncOp has awaitLastFut(); this call wait last async
operation completed.

This means all async operation in one thread will be executed one by one
but not in parallel. Async operation is not async.

Example for atomic cache

f1=cache.getAsync(key1);
f2=cache.getAsync(key2);

f1 always will be complete before f2.

I want to have the ability run multiple async operations in one thread.
What do you think?

Maybe we can add new cache adapter with fair async operations?

Disable WAL for several cache groups within one exchange

[Ivan Rakov]

Igniters,

API method for disabling WAL in IgniteCluster accepts only one cache 
name. Every call triggers exchange and checkpoints cluster-wide - it 
takes plenty of time to disable/enable WAL for multiple caches.
I think, we should add option to disable/enable WAL for several caches 
with single command.


Thoughts?

--
Best Regards,
Ivan Rakov

[jira] [Created] (IGNITE-8472) Apache ignite for .NET has security vulnerabilities

[Harendra Rai (JIRA)]

Harendra Rai created IGNITE-8472:


 Summary: Apache ignite for .NET has security vulnerabilities
 Key: IGNITE-8472
 URL: https://issues.apache.org/jira/browse/IGNITE-8472
 Project: Ignite
  Issue Type: Bug
  Components: security
Affects Versions: 2.4
Reporter: Harendra Rai
 Fix For: 2.5


There are two vulnerabilities in the latest 2.3.0 version.
 # commons-beanutils-1.8.3.jar.  Here is the vulnerability id CVE-2014-0114 :  
[https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-0114]

*Resolution to this issue is*: All Commons BeanUtils users should upgrade to 
the latest version >= commons-beanutils-1.9.2
 # commons-codec-1.6.jar: Here is the vulnerability detail 
https://issues.apache.org/jira/browse/CODEC-96

*Resolution to this issue is:* To upgrade to the latest available Version 1.11



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8471) Apache ignite for .NET has security vulnerabilities

[Harendra Rai (JIRA)]

Harendra Rai created IGNITE-8471:


 Summary: Apache ignite for .NET has security vulnerabilities
 Key: IGNITE-8471
 URL: https://issues.apache.org/jira/browse/IGNITE-8471
 Project: Ignite
  Issue Type: Improvement
  Components: security
Affects Versions: 2.4
Reporter: Harendra Rai
 Fix For: 2.5


There are two security vulnerabilities in the latest 2.4.0 version.
 # commons-beanutils-1.8.3.jar.  Here is the vulnerability id CVE-2014-0114 :  
[https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-0114]

*Resolution to this issue is*: All Commons BeanUtils users should upgrade to 
the latest version >= commons-beanutils-1.9.2
 # commons-codec-1.6.jar: Here is the vulnerability detail 
https://issues.apache.org/jira/browse/CODEC-96

*Resolution* *to this issue is:* To upgrade to the latest available Version 1.11



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

REMINDER: Apache EU Roadshow 2018 schedule announced!

[sharan]

Hello Apache Supporters and Enthusiasts

This is a reminder that the schedule for the Apache EU Roadshow 2018 in 
Berlin has been announced.


http://apachecon.com/euroadshow18/schedule.html

Please note that we will not be running an ApacheCon in Europe this year 
which means that this Apache EU Roadshow will be the main Apache event 
in Europe for 2018.


The Apache EU Roadshow tracks take place on the 13th and 14th June 2018, 
and will feature 28 sessions across the following themes; Apache Tomcat, 
IoT , Cloud Technologies, Microservices and Apache Httpd Server.


Please note that the Apache EU Roadshow is co-located with FOSS 
Backstage and their schedule (https://foss-backstage.de/sessions) 
includes many Apache related sessions such as Incubator, Apache Way, 
Open Source Governance, Legal, Trademarks as well as a full range 
community related presentations and panel discussions.


One single registration gives you access to both events - the Apache EU 
Roadshow and FOSS Backstage.


Registration includes catering (breakfast & lunch both days) and also an 
attendee evening event. And if you want to have a project meet-up, hack 
or simply spend time and relax in our on-site Apache Lounge between 
sessions, then you are more than welcome.


We look forward to seeing you in Berlin!

Thanks
Sharan Foga, VP Apache Community Development

PLEASE NOTE: You are receiving this message because you are subscribed 
to a user@ or dev@ list of one or more Apache Software Foundation projects.

[jira] [Created] (IGNITE-8470) NPE when starting LOCAL cache on a client with no data regions

[Stanislav Lukyanov (JIRA)]

Stanislav Lukyanov created IGNITE-8470:
--

 Summary: NPE when starting LOCAL cache on a client with no data 
regions
 Key: IGNITE-8470
 URL: https://issues.apache.org/jira/browse/IGNITE-8470
 Project: Ignite
  Issue Type: Bug
Reporter: Stanislav Lukyanov


If a LOCAL cache is started on a client and the client has no data regions 
configured then an NPE is thrown with no error message:

{code}
class org.apache.ignite.IgniteCheckedException: null
at 
org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7284)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:259)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:232)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:159)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:151)
at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.onKernalStart(GridCachePartitionExchangeManager.java:632)
at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.onKernalStart(GridCacheProcessor.java:829)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1043)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1973)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1716)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1144)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:642)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:849)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:798)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapIndexScanTest.beforeTestsStarted(IgniteCacheOffheapIndexScanTest.java:78)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.setUp(GridAbstractTest.java:599)
at 
org.apache.ignite.testframework.junits.common.GridCommonAbstractTest.setUp(GridCommonAbstractTest.java:485)
at junit.framework.TestCase.runBare(TestCase.java:139)
at junit.framework.TestResult$1.protect(TestResult.java:122)
at junit.framework.TestResult.runProtected(TestResult.java:142)
at junit.framework.TestResult.run(TestResult.java:125)
at junit.framework.TestCase.run(TestCase.java:129)
at junit.framework.TestSuite.runTest(TestSuite.java:255)
at junit.framework.TestSuite.run(TestSuite.java:250)
at 
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at 
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at 
com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at 
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at 
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.lang.NullPointerException
at 
org.apache.ignite.internal.processors.cache.CacheGroupContext.(CacheGroupContext.java:207)
at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCacheGroup(GridCacheProcessor.java:1983)
at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1881)
at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnLocalJoin(GridCacheProcessor.java:1773)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:784)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:666)
at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2344)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:748)
{code}

If assertions are enabled (-ea), an AssertionError is thrown instead:
{code}
java.lang.AssertionError
at 
org.apache.ignite.internal.processors.cache.CacheGroupContext.(CacheGroupContext.java:185)
at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCacheGroup(GridCacheProcessor.java:1983)
at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1881)
at 

Re: NodeJS thin client: full API

[Dmitriy Setrakyan]

On Fri, May 11, 2018 at 9:14 AM, Alexey Kosenchuk <
alexey.kosenc...@nobitlost.com> wrote:

> Not yet. Need a help with that.
>

I think we definitely need a load test before we merge to master. Can
anyone in the community assist Alexey?

[GitHub] ignite pull request #3738: patch

[voipp]

Github user voipp closed the pull request at:

https://github.com/apache/ignite/pull/3738


---

Re: NodeJS thin client: full API

[Alexey Kosenchuk]

Not yet. Need a help with that.

-Alexey

11.05.2018 10:58, Dmitriy Setrakyan пишет:

This is great! Finally a native NodeJS client for Ignite.

Alexey, in addition to the functional tests, were you able to perform any
load tests?

D.

On Fri, May 11, 2018 at 12:07 AM, Alexey Kosenchuk <
alexey.kosenc...@nobitlost.com> wrote:


Folks,

The next version is ready -
in the pull request [1] or directly in the repo [2].

The version includes:
- full API
- full implementation
- examples
- tests (cover the full API but might need to be updated/extended)
- docs

The details are in the readme [3]

Regards,
-Alexey

[1] https://github.com/apache/ignite/pull/3978
[2] https://github.com/nobitlost/ignite/tree/master/modules/plat
forms/nodejs
[3] https://github.com/nobitlost/ignite/blob/master/modules/plat
forms/nodejs/README.md

Re: NodeJS thin client: full API

[Dmitriy Setrakyan]

This is great! Finally a native NodeJS client for Ignite.

Alexey, in addition to the functional tests, were you able to perform any
load tests?

D.

On Fri, May 11, 2018 at 12:07 AM, Alexey Kosenchuk <
alexey.kosenc...@nobitlost.com> wrote:

> Folks,
>
> The next version is ready -
> in the pull request [1] or directly in the repo [2].
>
> The version includes:
> - full API
> - full implementation
> - examples
> - tests (cover the full API but might need to be updated/extended)
> - docs
>
> The details are in the readme [3]
>
> Regards,
> -Alexey
>
> [1] https://github.com/apache/ignite/pull/3978
> [2] https://github.com/nobitlost/ignite/tree/master/modules/plat
> forms/nodejs
> [3] https://github.com/nobitlost/ignite/blob/master/modules/plat
> forms/nodejs/README.md
>