[MTCGA]: new failures in builds [6144444] needs to be handled

[dpavlov . tasks]

Hi Igniters,

 I've detected some new issue on TeamCity to be handled. You are more than 
welcomed to help.

 *New Critical Failure in master Cache (Restarts) 1 
https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_CacheRestarts1?branch=%3Cdefault%3E
 No changes in the build

 - Here's a reminder of what contributors were agreed to do 
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute 
 - Should you have any questions please contact dev@ignite.apache.org 

Best Regards,
Apache Ignite TeamCity Bot 
https://github.com/apache/ignite-teamcity-bot
Notification generated at 02:18:37 21-08-2021 

Re: Re[2]: Google Guava in Ignite 3

[Alexander Polovtcev]

Guys,

Thanks again for your responses. I've created a ticket about using the
Shade plugin in order to understand if it is possible to shade and minimize
Guava, I will start working on it shortly:
https://issues.apache.org/jira/browse/IGNITE-15354

On Tue, Aug 10, 2021 at 1:51 PM Courtney Robinson 
wrote:

> I think since Calcite brings it in already then your arguments make sense.
> Would it be pinned to the same version as Calcite? Risk NoSuchMethodError
> at runtime if not.
> +1
>
> On Mon, Aug 9, 2021 at 9:56 AM Alexander Polovtcev <
> alexpolovt...@gmail.com>
> wrote:
>
> > Zhenya, Courtney, Andrey,
> >
> > What do you think about my arguments, was I able to convince you? I would
> > like to reach some consensus here. At the moment, my original points
> still
> > stand, I'm also ok with shading Guava if needed, though I think it is not
> > necessary at this point.
> >
> > On Fri, Aug 6, 2021 at 12:45 PM Alexander Polovtcev <
> > alexpolovt...@gmail.com>
> > wrote:
> >
> > > Zhenya,
> > >
> > > > But there is no restrictions from running ignite server nodes from
> some
> > > other code with it`s own guava version seems we obtain fast path to jar
> > > hell here?
> > >
> > > I'm not sure if I fully understand your question, but it looks like we
> > are
> > > in this situation already, because we have some dependencies that use
> > > Guava. That's why I propose to add Guava explicitly to at least have a
> > > deterministic runtime configuration (see this link
> > > <
> >
> https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Management
> > >
> > > for an explanation).
> > >
> > > On Fri, Aug 6, 2021 at 12:25 PM Zhenya Stanilovsky
> > >  wrote:
> > >
> > >>
> > >> Alexander, first of all looks like Ivan Daschinsky approach about thin
> > >> client use only and shadow plugin are cover all Andrey Mashenkov
> listing
> > >> problems.
> > >> But there is no restrictions from running ignite server nodes from
> some
> > >> other code with it`s own guava version seems we obtain fast path to
> jar
> > >> hell here?
> > >>
> > >>
> > >> >Zhenya,
> > >> >
> > >> >My intentions are the following:
> > >> >
> > >> >1. Remove some copy-pasted code (like the "bytecode" module or some
> > >> utility
> > >> >methods). Please see my original message for the links to the code.
> > >> >2. Explicitly pin the Guava version to avoid conflicts in the
> runtime.
> > >> >
> > >> >About allowing to use Guava in the codebase, my thoughts are the
> > >> following:
> > >> >
> > >> >1. We *already* use some code from Guava either directly (like in the
> > >> >"calcite" module) or by copy-pasting it into a utility class.
> > >> >2. I understand that some Guava methods are obsolete as of Java 11,
> but
> > >> >some of them still don't have any standard library counterparts, in
> > which
> > >> >case I think using Guava is justified (which is supported by point
> 1).
> > >> >
> > >> >Can you please explain why you would disapprove of my proposal?
> > >> >
> > >> >On Thu, Aug 5, 2021 at 7:56 PM Zhenya Stanilovsky
> > >> >< arzamas...@mail.ru.invalid > wrote:
> > >> >
> > >> >>
> > >> >> alexpolovtcev please clarify what do you mean under : «possibility
> of
> > >> >> using Guava in Ignite 3», using how necessary dependency of calcite
> > or
> > >> >> using like «using in our code» ? If using in code, i -1 here.
> > >> >> thanks.
> > >> >>
> > >> >>
> > >> >> >Hello, dear Igniters!
> > >> >> >
> > >> >> >I would like to discuss the possibility of using Guava
> > >> >> ><  https://github.com/google/guava > in Ignite 3. I know about
> the
> > >> >> restrictive
> > >> >> >policy of using it in Ignite 2, but I have the following reasons:
> > >> >> >
> > >> >> >1. We are de-facto using it already as an implicit dependency,
> since
> > >> the
> > >> >> >Calcite module depends on it, and Calcite is going to stay for a
> > >> while =)
> > >> >> >2. AFAIK, the "bytecode" module is copied into the codebase only
> to
> > >> strip
> > >> >> >Guava away from it. We can remove this module, which will improve
> > the
> > >> >> >maintainability of the project.
> > >> >> >3. We have some copy-paste of Guava code in the project. For
> > example,
> > >> see
> > >> >> >this
> > >> >> ><
> > >> >>
> > >>
> >
> https://github.com/apache/ignite-3/blob/main/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L136
> > >> >> >
> > >> >> >and this
> > >> >> ><
> > >> >>
> > >>
> >
> https://github.com/apache/ignite-3/blob/main/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L428
> > >> >> >
> > >> >> >.
> > >> >> >4. Regarding security concerns, this report
> > >> >> ><
> > >> >>
> > >>
> >
> https://www.cvedetails.com/product/52274/Google-Guava.html?vendor_id=1224
> > >> >> >
> > >> >> >shows no major vulnerability issues for the last three years.
> > >> >> >
> > >> >> >Taking these points into account, I propose to allow using Guava
> > both
> > >> in
> > >> >> 

Re: TeamCity (ci.ignite.apache.org) is not available.

[Petr Ivanov]

Second IP address is missing from domain record, and first is currently down.
The issue is being resolved by INFRA [1]



[1] https://issues.apache.org/jira/browse/INFRA-22230

> On 20 Aug 2021, at 16:10, Pavel Pereslegin  wrote:
> 
> Hello, Igniters!
> 
> Can someone check why TeamCity (ci.ignite.apache.org) is currently 
> unavailable?

Re: [ANNOUNCE] New committer: Petr Ivanov

[Petr Ivanov]

Hi, Igniters.


This is an honour to be among committers of Apache Ignite indeed.
Thanks for your trust, will do my best to meet it.

> On 19 Aug 2021, at 12:58, Dmitry Pavlov  wrote:
> 
> Hello Ignite community,
> 
> The Project Management Committee (PMC) for Apache Ignite
> has invited Petr Ivanov (vveider) to become a committer and we are pleased 
> to announce that he has accepted.
> 
> Petr is community veteran, who supports TeamCity instance, as well as TC Bot, 
> and other services. He contributes also to supporting our 
> checkstyle/PMD/maven scripts, docker images, DEB, and RPM packages, Ignite 
> start scripts, and many more things, which are crucial for Apache Ignite 
> development process and running in producition. 
> 
> Please join me in congratulating Petr with his new role in the community.
> 
> Being a committer enables easier contribution to the
> project since there is no need to go via the patch
> submission process. This should enable better productivity.
> 
> Best Regards,
> Dmitriy Pavlov
> on behalf of Apache Ignite PMC

Re: [DISCUSSION] Code style for Ignite 3

[Pavel Tupitsyn]

+1, as long as 100% of the rules are checked automatically.

On Fri, Aug 20, 2021 at 4:00 PM Andrey Gura  wrote:

> Looks good to me. But Idea configuration for style check is not
> enough. It helps developers but does not automate style checking.
>
> Checkstyle project provides ready to use config based on Google Code
> Style [1]. I hope it matches well with Idea config and we'll avoid any
> confusing incidents.
>
> Let's start with this convention and adopt it to our needs if needed.
>
> [1]
> https://github.com/checkstyle/checkstyle/blob/master/src/main/resources/google_checks.xml
>
> On Fri, Aug 20, 2021 at 12:02 PM Alexei Scherbakov
>  wrote:
> >
> > +1
> >
> > пт, 20 авг. 2021 г. в 10:54, Alexander Polovtcev <
> alexpolovt...@gmail.com>:
> >
> > > Hi, Val. This is an extremely welcome change, thank you!
> > >
> > > On Fri, Aug 20, 2021 at 12:17 AM Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com> wrote:
> > >
> > > > Igniters,
> > > >
> > > > I would like to discuss a potential change to the coding guidelines
> for
> > > > Ignite 3. Currently, we're using the existing guidelines inherited
> from
> > > > Ignite 2, which are described here:
> > > > https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines
> > > >
> > > > Current guidelines, however, exist for many years and have several
> > > issues.
> > > > They are cumbersome, carry a lot of legacy stuff, and can't be
> automated.
> > > > Every now and then, they seem to cause questions and confusion.
> > > >
> > > > While it's hard to make drastic changes in Ignite 2, we still have a
> > > great
> > > > opportunity to update the guidelines in Ignite 3. I would identify
> two
> > > > major goals here:
> > > >
> > > >1. Simplification. Having too many rules and restrictions tend to
> > > >complicate development rather than providing any value. We should
> come
> > > > up
> > > >with a minimum set of rules and then make amends one by one if
> needed.
> > > >2. The ability for automation. I hold a strong belief that code
> style
> > > >checking has to become a part of the build. Therefore, we need to
> make
> > > > sure
> > > >that any rule that ends up in the guideline can be automatically
> > > > verified.
> > > >
> > > > I propose the following process to define the new guideline:
> > > >
> > > >1. Use Google code style as the starting point:
> > > >https://google.github.io/styleguide/javaguide.html
> > > >2. Replace the 100 column limit with 140. The latter is the value
> we
> > > >already use in Ignite 2, and it seems to be more reasonable, in my
> > > > opinion.
> > > >3. Use 4 spaces block indentation and 8 spaces for continuation
> (as
> > > >opposed to 2 and 4). Nothing wrong with 2 spaces, in my view, but
> 4
> > > > spaces
> > > >should provide a smoother transition, as we're really used to this
> > > > style.
> > > >4. For any other changes, initiate separate discussions going
> forward.
> > > >
> > > > Several reasons why I specifically propose Google style:
> > > >
> > > >1. This is essentially the standard for many projects. I don't
> think
> > > >there is a need for us to reinvent the wheel.
> > > >2. It's minimalistic and developer-friendly. No overcomplication.
> > > >3. (probably the most important) It comes with all the required
> tools
> > > >and configurations for automation (e.g., here is the config for
> IDEA:
> > > >
> > >
> https://github.com/google/error-prone/blob/master/.idea/GoogleStyle.xml
> > > > )
> > > >
> > > > Please let me know what you think. If there are no objections, I will
> > > start
> > > > the process.
> > > >
> > > > -Val
> > > >
> > >
> > >
> > > --
> > > With regards,
> > > Aleksandr Polovtcev
> > >
> >
> >
> > --
> >
> > Best regards,
> > Alexei Scherbakov
>

TeamCity (ci.ignite.apache.org) is not available.

[Pavel Pereslegin]

Hello, Igniters!

Can someone check why TeamCity (ci.ignite.apache.org) is currently unavailable?

A new feedback has been added : 5

[Bugyard]

A new feedback has been added, go to bugyard.io to see all the details...

https://bugyard.io

A new feedback has been added 

"This didn't worked for me, I had to include the full config file starting by 
the root element  and not just the  node"   by diego.ruiz 

View feedback 
https://app.bugyard.io/web/app/rycqZJDyY/f/611fa11b1602950014ea078e

Re: [DISCUSSION] Code style for Ignite 3

[Andrey Gura]

Looks good to me. But Idea configuration for style check is not
enough. It helps developers but does not automate style checking.

Checkstyle project provides ready to use config based on Google Code
Style [1]. I hope it matches well with Idea config and we'll avoid any
confusing incidents.

Let's start with this convention and adopt it to our needs if needed.

[1] 
https://github.com/checkstyle/checkstyle/blob/master/src/main/resources/google_checks.xml

On Fri, Aug 20, 2021 at 12:02 PM Alexei Scherbakov
 wrote:
>
> +1
>
> пт, 20 авг. 2021 г. в 10:54, Alexander Polovtcev :
>
> > Hi, Val. This is an extremely welcome change, thank you!
> >
> > On Fri, Aug 20, 2021 at 12:17 AM Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> > > Igniters,
> > >
> > > I would like to discuss a potential change to the coding guidelines for
> > > Ignite 3. Currently, we're using the existing guidelines inherited from
> > > Ignite 2, which are described here:
> > > https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines
> > >
> > > Current guidelines, however, exist for many years and have several
> > issues.
> > > They are cumbersome, carry a lot of legacy stuff, and can't be automated.
> > > Every now and then, they seem to cause questions and confusion.
> > >
> > > While it's hard to make drastic changes in Ignite 2, we still have a
> > great
> > > opportunity to update the guidelines in Ignite 3. I would identify two
> > > major goals here:
> > >
> > >1. Simplification. Having too many rules and restrictions tend to
> > >complicate development rather than providing any value. We should come
> > > up
> > >with a minimum set of rules and then make amends one by one if needed.
> > >2. The ability for automation. I hold a strong belief that code style
> > >checking has to become a part of the build. Therefore, we need to make
> > > sure
> > >that any rule that ends up in the guideline can be automatically
> > > verified.
> > >
> > > I propose the following process to define the new guideline:
> > >
> > >1. Use Google code style as the starting point:
> > >https://google.github.io/styleguide/javaguide.html
> > >2. Replace the 100 column limit with 140. The latter is the value we
> > >already use in Ignite 2, and it seems to be more reasonable, in my
> > > opinion.
> > >3. Use 4 spaces block indentation and 8 spaces for continuation (as
> > >opposed to 2 and 4). Nothing wrong with 2 spaces, in my view, but 4
> > > spaces
> > >should provide a smoother transition, as we're really used to this
> > > style.
> > >4. For any other changes, initiate separate discussions going forward.
> > >
> > > Several reasons why I specifically propose Google style:
> > >
> > >1. This is essentially the standard for many projects. I don't think
> > >there is a need for us to reinvent the wheel.
> > >2. It's minimalistic and developer-friendly. No overcomplication.
> > >3. (probably the most important) It comes with all the required tools
> > >and configurations for automation (e.g., here is the config for IDEA:
> > >
> > https://github.com/google/error-prone/blob/master/.idea/GoogleStyle.xml
> > > )
> > >
> > > Please let me know what you think. If there are no objections, I will
> > start
> > > the process.
> > >
> > > -Val
> > >
> >
> >
> > --
> > With regards,
> > Aleksandr Polovtcev
> >
>
>
> --
>
> Best regards,
> Alexei Scherbakov

Re: [DISCUSSION] Code style for Ignite 3

[Alexei Scherbakov]

+1

пт, 20 авг. 2021 г. в 10:54, Alexander Polovtcev :

> Hi, Val. This is an extremely welcome change, thank you!
>
> On Fri, Aug 20, 2021 at 12:17 AM Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > Igniters,
> >
> > I would like to discuss a potential change to the coding guidelines for
> > Ignite 3. Currently, we're using the existing guidelines inherited from
> > Ignite 2, which are described here:
> > https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines
> >
> > Current guidelines, however, exist for many years and have several
> issues.
> > They are cumbersome, carry a lot of legacy stuff, and can't be automated.
> > Every now and then, they seem to cause questions and confusion.
> >
> > While it's hard to make drastic changes in Ignite 2, we still have a
> great
> > opportunity to update the guidelines in Ignite 3. I would identify two
> > major goals here:
> >
> >1. Simplification. Having too many rules and restrictions tend to
> >complicate development rather than providing any value. We should come
> > up
> >with a minimum set of rules and then make amends one by one if needed.
> >2. The ability for automation. I hold a strong belief that code style
> >checking has to become a part of the build. Therefore, we need to make
> > sure
> >that any rule that ends up in the guideline can be automatically
> > verified.
> >
> > I propose the following process to define the new guideline:
> >
> >1. Use Google code style as the starting point:
> >https://google.github.io/styleguide/javaguide.html
> >2. Replace the 100 column limit with 140. The latter is the value we
> >already use in Ignite 2, and it seems to be more reasonable, in my
> > opinion.
> >3. Use 4 spaces block indentation and 8 spaces for continuation (as
> >opposed to 2 and 4). Nothing wrong with 2 spaces, in my view, but 4
> > spaces
> >should provide a smoother transition, as we're really used to this
> > style.
> >4. For any other changes, initiate separate discussions going forward.
> >
> > Several reasons why I specifically propose Google style:
> >
> >1. This is essentially the standard for many projects. I don't think
> >there is a need for us to reinvent the wheel.
> >2. It's minimalistic and developer-friendly. No overcomplication.
> >3. (probably the most important) It comes with all the required tools
> >and configurations for automation (e.g., here is the config for IDEA:
> >
> https://github.com/google/error-prone/blob/master/.idea/GoogleStyle.xml
> > )
> >
> > Please let me know what you think. If there are no objections, I will
> start
> > the process.
> >
> > -Val
> >
>
>
> --
> With regards,
> Aleksandr Polovtcev
>


-- 

Best regards,
Alexei Scherbakov

Re: [DISCUSSION] Code style for Ignite 3

[Alexander Polovtcev]

Hi, Val. This is an extremely welcome change, thank you!

On Fri, Aug 20, 2021 at 12:17 AM Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Igniters,
>
> I would like to discuss a potential change to the coding guidelines for
> Ignite 3. Currently, we're using the existing guidelines inherited from
> Ignite 2, which are described here:
> https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines
>
> Current guidelines, however, exist for many years and have several issues.
> They are cumbersome, carry a lot of legacy stuff, and can't be automated.
> Every now and then, they seem to cause questions and confusion.
>
> While it's hard to make drastic changes in Ignite 2, we still have a great
> opportunity to update the guidelines in Ignite 3. I would identify two
> major goals here:
>
>1. Simplification. Having too many rules and restrictions tend to
>complicate development rather than providing any value. We should come
> up
>with a minimum set of rules and then make amends one by one if needed.
>2. The ability for automation. I hold a strong belief that code style
>checking has to become a part of the build. Therefore, we need to make
> sure
>that any rule that ends up in the guideline can be automatically
> verified.
>
> I propose the following process to define the new guideline:
>
>1. Use Google code style as the starting point:
>https://google.github.io/styleguide/javaguide.html
>2. Replace the 100 column limit with 140. The latter is the value we
>already use in Ignite 2, and it seems to be more reasonable, in my
> opinion.
>3. Use 4 spaces block indentation and 8 spaces for continuation (as
>opposed to 2 and 4). Nothing wrong with 2 spaces, in my view, but 4
> spaces
>should provide a smoother transition, as we're really used to this
> style.
>4. For any other changes, initiate separate discussions going forward.
>
> Several reasons why I specifically propose Google style:
>
>1. This is essentially the standard for many projects. I don't think
>there is a need for us to reinvent the wheel.
>2. It's minimalistic and developer-friendly. No overcomplication.
>3. (probably the most important) It comes with all the required tools
>and configurations for automation (e.g., here is the config for IDEA:
>https://github.com/google/error-prone/blob/master/.idea/GoogleStyle.xml
> )
>
> Please let me know what you think. If there are no objections, I will start
> the process.
>
> -Val
>


-- 
With regards,
Aleksandr Polovtcev

Re: [ANNOUNCE] New committer: Petr Ivanov

[Konstantin Orlov]

Well deserved!

-- 
Regards,
Konstantin Orlov


> On 19 Aug 2021, at 19:11, Kevin Corbett  wrote:
> 
> Congratulations Petr!! :)
> 
>> On Aug 19, 2021, at 5:58 AM, Dmitry Pavlov  wrote:
>> 
>> Hello Ignite community,
>> 
>> The Project Management Committee (PMC) for Apache Ignite
>> has invited Petr Ivanov (vveider) to become a committer and we are pleased 
>> to announce that he has accepted.
>> 
>> Petr is community veteran, who supports TeamCity instance, as well as TC 
>> Bot, and other services. He contributes also to supporting our 
>> checkstyle/PMD/maven scripts, docker images, DEB, and RPM packages, Ignite 
>> start scripts, and many more things, which are crucial for Apache Ignite 
>> development process and running in producition. 
>> 
>> Please join me in congratulating Petr with his new role in the community.
>> 
>> Being a committer enables easier contribution to the
>> project since there is no need to go via the patch
>> submission process. This should enable better productivity.
>> 
>> Best Regards,
>> Dmitriy Pavlov
>> on behalf of Apache Ignite PMC
> 

Re: Ignite 3 async continuation executor

[Pavel Tupitsyn]

Courtney,

Ignite 3 uses CompletableFuture.
Yes, it has thenApplyAsync and other methods which accept an Executor,
but we can't force user code to use those methods and not thenApply.

Thanks for describing your use case with custom thread pools,
we'll figure out how to support that in 3.x



On Fri, Aug 20, 2021 at 9:31 AM Pavel Tupitsyn  wrote:

> Alexander,
>
> > it is not expected that a user may want to specify their own custom
> executor
>
> That would be nice, but I'm not sure if this fits into Ignite 3
> configuration approach.
> I'd like to hear more opinions on this.
>
> On Fri, Aug 20, 2021 at 8:10 AM Courtney Robinson <
> courtney.robin...@hypi.io> wrote:
>
>> Pavel I would really welcome this - when we first started with Ignite we
>> were constantly getting the Ignite threads blocked because we'd perform
>> other work on it.
>>
>> I don't know about the configuration part however because this isn't a
>> static thing I'd argue.
>> Is Ignite 3 still using its own types or is it switching to
>> CompletableFuture
>> <
>> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html
>> >
>> ?
>> The key APIs in CompletableFuture (acceptEitherAsync,applyToEitherAsync,
>> handleAsync, thenAcceptASync, thenComposeAsync, whenCompleteAsync) all
>> already accept an Executor argument so returning CompletableFuture solves
>> the problem, it'd just need documentation.
>>
>> If Ignite 3 still uses its own types then I'd suggest what's needed is an
>> argument to accept a custom Executor.
>> We have dedicated pools configured now with custom
>> UncaughtExceptionHandler
>> and ThreadFactory
>> <
>> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ThreadFactory.html
>> >
>> that
>> we have various metrics and customisations around. If the only option is
>> the default ForkJoinPool#commonPool we'd lose this when eventually moving
>> to 3.
>>
>> Regards,
>> Courtney Robinson
>> Founder and CEO, Hypi
>> Tel: ++44 208 123 2413 (GMT+0) 
>>
>> 
>> https://hypi.io
>>
>>
>> On Thu, Aug 19, 2021 at 5:08 PM Alexander Polovtcev <
>> alexpolovt...@gmail.com>
>> wrote:
>>
>> > Pavel, thanks for the response. Do I understand correctly that it is not
>> > expected that a user may want to specify their own custom executor?
>> >
>> > On Thu, Aug 19, 2021 at 6:55 PM Pavel Tupitsyn 
>> > wrote:
>> >
>> > > Hi Alexander,
>> > >
>> > > To be honest, I'm not sure yet - just getting to know this new
>> > > configuration mechanism and format.
>> > >
>> > > Since we can't use a property of type Executor, we'll have to provide
>> > > predefined values.
>> > > It can either be "bool executeAsyncContinuationsDirectly": false
>> > (default)
>> > > => commonPool, true => Runnable::run,
>> > > or "String asyncContinuationExecutor" which allows two values "direct"
>> > and
>> > > "commonPool".
>> > >
>> > > I'm leaning towards the latter:
>> > >
>> > > {
>> > > "node": {
>> > > "metastorageNodes": [ "node-0" ],
>> > > "asyncContinuationExecutor": "commonPool"
>> > > },
>> > > "network": { ... }
>> > > }
>> > >
>> > >
>> > >
>> > > On Thu, Aug 19, 2021 at 6:29 PM Alexander Polovtcev <
>> > > alexpolovt...@gmail.com>
>> > > wrote:
>> > >
>> > > > Hi, Pavel!
>> > > >
>> > > > Can you please provide an example (e.g. HOCON snippet) of how this
>> > > > configuration is going to look like in Ignite 3? Or how is this
>> > property
>> > > > going to be set?
>> > > >
>> > > >
>> > > > On Thu, Aug 19, 2021 at 6:00 PM Pavel Tupitsyn <
>> ptupit...@apache.org>
>> > > > wrote:
>> > > >
>> > > > > Igniters,
>> > > > >
>> > > > > I propose to add a configurable async continuation executor for
>> > public
>> > > > APIs
>> > > > > to Ignite 3
>> > > > > like we have in Ignite 2.x [1]
>> > > > >
>> > > > > In short, currently, async APIs return a future to the user code.
>> > > > > Continuations like "myCode" in
>> "table.getAsync().thenApply(myCode)"
>> > > will
>> > > > be
>> > > > > executed by the same thread that completes the future, which will
>> be
>> > a
>> > > > > Netty thread or some other Ignite thread.
>> > > > >
>> > > > > This is dangerous because user code can be blocking or
>> long-running,
>> > > and
>> > > > > system threads become unavailable.
>> > > > >
>> > > > > Proposal:
>> > > > > 1. Add asyncContinuationExecutor configuration property, defaults
>> to
>> > > > > ForkJoinPool#commonPool - both for server and thin client
>> > > > > 2. Use this executor to complete all public API futures
>> > > > >
>> > > > > This means safe default behavior and a possibility to enable
>> unsafe
>> > but
>> > > > > faster behavior with Runnable::run executor.
>> > > > >
>> > > > > Thoughts?
>> > > > >
>> > > > >
>> > > > > [1]
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-70%3A+Async+Continuation+Executor
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > With regards,
>> > > > Aleksandr Polovtcev
>> > 

Re: Ignite 3 async continuation executor

[Pavel Tupitsyn]

Alexander,

> it is not expected that a user may want to specify their own custom
executor

That would be nice, but I'm not sure if this fits into Ignite 3
configuration approach.
I'd like to hear more opinions on this.

On Fri, Aug 20, 2021 at 8:10 AM Courtney Robinson 
wrote:

> Pavel I would really welcome this - when we first started with Ignite we
> were constantly getting the Ignite threads blocked because we'd perform
> other work on it.
>
> I don't know about the configuration part however because this isn't a
> static thing I'd argue.
> Is Ignite 3 still using its own types or is it switching to
> CompletableFuture
> <
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html
> >
> ?
> The key APIs in CompletableFuture (acceptEitherAsync,applyToEitherAsync,
> handleAsync, thenAcceptASync, thenComposeAsync, whenCompleteAsync) all
> already accept an Executor argument so returning CompletableFuture solves
> the problem, it'd just need documentation.
>
> If Ignite 3 still uses its own types then I'd suggest what's needed is an
> argument to accept a custom Executor.
> We have dedicated pools configured now with custom UncaughtExceptionHandler
> and ThreadFactory
> <
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ThreadFactory.html
> >
> that
> we have various metrics and customisations around. If the only option is
> the default ForkJoinPool#commonPool we'd lose this when eventually moving
> to 3.
>
> Regards,
> Courtney Robinson
> Founder and CEO, Hypi
> Tel: ++44 208 123 2413 (GMT+0) 
>
> 
> https://hypi.io
>
>
> On Thu, Aug 19, 2021 at 5:08 PM Alexander Polovtcev <
> alexpolovt...@gmail.com>
> wrote:
>
> > Pavel, thanks for the response. Do I understand correctly that it is not
> > expected that a user may want to specify their own custom executor?
> >
> > On Thu, Aug 19, 2021 at 6:55 PM Pavel Tupitsyn 
> > wrote:
> >
> > > Hi Alexander,
> > >
> > > To be honest, I'm not sure yet - just getting to know this new
> > > configuration mechanism and format.
> > >
> > > Since we can't use a property of type Executor, we'll have to provide
> > > predefined values.
> > > It can either be "bool executeAsyncContinuationsDirectly": false
> > (default)
> > > => commonPool, true => Runnable::run,
> > > or "String asyncContinuationExecutor" which allows two values "direct"
> > and
> > > "commonPool".
> > >
> > > I'm leaning towards the latter:
> > >
> > > {
> > > "node": {
> > > "metastorageNodes": [ "node-0" ],
> > > "asyncContinuationExecutor": "commonPool"
> > > },
> > > "network": { ... }
> > > }
> > >
> > >
> > >
> > > On Thu, Aug 19, 2021 at 6:29 PM Alexander Polovtcev <
> > > alexpolovt...@gmail.com>
> > > wrote:
> > >
> > > > Hi, Pavel!
> > > >
> > > > Can you please provide an example (e.g. HOCON snippet) of how this
> > > > configuration is going to look like in Ignite 3? Or how is this
> > property
> > > > going to be set?
> > > >
> > > >
> > > > On Thu, Aug 19, 2021 at 6:00 PM Pavel Tupitsyn  >
> > > > wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > I propose to add a configurable async continuation executor for
> > public
> > > > APIs
> > > > > to Ignite 3
> > > > > like we have in Ignite 2.x [1]
> > > > >
> > > > > In short, currently, async APIs return a future to the user code.
> > > > > Continuations like "myCode" in "table.getAsync().thenApply(myCode)"
> > > will
> > > > be
> > > > > executed by the same thread that completes the future, which will
> be
> > a
> > > > > Netty thread or some other Ignite thread.
> > > > >
> > > > > This is dangerous because user code can be blocking or
> long-running,
> > > and
> > > > > system threads become unavailable.
> > > > >
> > > > > Proposal:
> > > > > 1. Add asyncContinuationExecutor configuration property, defaults
> to
> > > > > ForkJoinPool#commonPool - both for server and thin client
> > > > > 2. Use this executor to complete all public API futures
> > > > >
> > > > > This means safe default behavior and a possibility to enable unsafe
> > but
> > > > > faster behavior with Runnable::run executor.
> > > > >
> > > > > Thoughts?
> > > > >
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-70%3A+Async+Continuation+Executor
> > > > >
> > > >
> > > >
> > > > --
> > > > With regards,
> > > > Aleksandr Polovtcev
> > > >
> > >
> >
> >
> > --
> > With regards,
> > Aleksandr Polovtcev
> >
>