[MTCGA]: new failures in builds [4480005] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. If your changes can lead to this failure(s): We're grateful that you were a volunteer to make the contribution to this project, but things change and you may no longer be able to finalize your contribution. Could you respond to this email and indicate if you wish to continue and fix test failures or step down and some committer may revert you commit. *Recently contributed test failed in master IgniteNodeStoppedDuringDisableWALTest.test[AFTER_DISABLE_WAL] https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-4415134685125470652=%3Cdefault%3E=testDetails Changes may lead to failure were done by - dmitriy govorukhin https://ci.ignite.apache.org/viewModification.html?modId=889238 - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 07:03:39 09-08-2019
SQL query timeout: in progress or abandoned
Hey Saikat, Are you still working on this ticket? https://issues.apache.org/jira/browse/IGNITE-7285 Seems that's the last API that doesn't support timeouts - JDBC and ODBC drivers already go with it. If you don't have time to complete the changes then someone else from the community can take over. We see a lot of demand for this API and here is one example: https://stackoverflow.com/questions/57275301/how-to-set-a-query-timeout-for-apache-ignite-cache - Denis
Re: Ignite commits atomicity & GG tickets
Ok, thank you. I forgot to mention atomicity. I discourage contributors from so-called bulk commits when several unrelated changes are merged into one commit. In case of any issues, it is not possible to find out reasons why it was changed, it is not possible to easily revert. чт, 8 авг. 2019 г. в 23:58, Denis Magda : > > > > Do you find the presence of GG's tickets in the commit is a reason for > > revert? > > > No, GG employees just need to follow "commit messages" guidelines removing > GG-specific details from the messages. > > > > - > Denis > > > On Thu, Aug 8, 2019 at 1:52 PM Dmitriy Pavlov wrote: > > > Hi Igniters, > > > > I little bit upset because I sometimes find GG tickets mentioned in > Ignite > > source code as a reason for Ignoring tests, todos, etc. > > > > I am personally grateful to all GG's employes for contributing to Ignite > > code base. > > > > I just want to some accuracy for commits provided to Ignite community, it > > should contain ONLY tickets which is available to all members. > > > > Do you find the presence of GG's tickets in the commit is a reason for > > revert? > > > > Sincerely, > > Dmitriy Pavlov > > >
Re: Ignite commits atomicity & GG tickets
> > Do you find the presence of GG's tickets in the commit is a reason for > revert? No, GG employees just need to follow "commit messages" guidelines removing GG-specific details from the messages. - Denis On Thu, Aug 8, 2019 at 1:52 PM Dmitriy Pavlov wrote: > Hi Igniters, > > I little bit upset because I sometimes find GG tickets mentioned in Ignite > source code as a reason for Ignoring tests, todos, etc. > > I am personally grateful to all GG's employes for contributing to Ignite > code base. > > I just want to some accuracy for commits provided to Ignite community, it > should contain ONLY tickets which is available to all members. > > Do you find the presence of GG's tickets in the commit is a reason for > revert? > > Sincerely, > Dmitriy Pavlov >
Re: Apache Ignite 2.7.6 (Time, Scope, and Release manager)
Hi Denis, sure, we can include. Just one point, a wider scope can move very optimistic dates from the first letter little bit forward. But since Apache communities are about solutions made without time bounds, it is perfectly ok for me. чт, 8 авг. 2019 г. в 22:46, Raymond Wilson : > +1 for IGNITE-10451 > > Thanks, > Raymond. > > Sent from my iPhone > > > On 9/08/2019, at 5:40 AM, Dmitriy Pavlov wrote: > > > > Hi Ivan, Ilya, Igniters, > > > > > > > > I would like this release would be as minimal as possible. > > > > > > > > According to dates proposed we could freeze scope at 12.08, 4 days is > more > > than enough to stand up and say, ‘Hey, I have an urgent fix’. But it is > > also ok for me if we decide to have more relaxed dates. > > > > > > > > For now, I suppose the following fixed should be cherry-picked: > > > > https://issues.apache.org/jira/browse/IGNITE-11767 (Blocker) > > GridDhtPartitionsFullMessage retains huge maps on heap in exchange > history > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-10451 (Major Bug) .NET: > > Persistence does not work with custom affinity function > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-9562 (Critical Bug) > Destroyed > > cache that resurrected on an old offline node breaks PME > > > > > > > > But I will continue to research JIRA. > > > > > > > > Sincerely, > > > > Dmitriy Pavlov > > > > > > чт, 8 авг. 2019 г. в 17:30, Павлухин Иван : > > > >>> What's the scope for this release? > >> > >> Same question. > >> > >> On the other hand an idea of 2.7.6 release attracts me because having > >> a practice of doing frequent minor releases can help us to build > >> reliable and predictable release rails. > >> > >> чт, 8 авг. 2019 г. в 15:09, Ilya Kasnacheev >: > >>> > >>> Hello! > >>> > >>> What's the scope for this release? > >>> > >>> Regards, > >>> -- > >>> Ilya Kasnacheev > >>> > >>> > >>> чт, 8 авг. 2019 г. в 15:07, Dmitriy Pavlov : > >>> > Hi Apache Ignite Developers, > > > > We seem to be on the same page about 2.8 release, but we’ve started > new > practice - minor releases, the first release was 2.7.5. Unfortunately, > there is a couple of issues still not fixed in that release, so I > would > like to propose to prepare one more minor release for Apache Ignite. > > > > I propose my candidacy to be release manager once again. > > > > I could (of course with help from Peter Ivanov) do some additional > >> efforts > to complete and improve process doc > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process > > > > I propose (most optimistic) dates for release stages: > > Scope Freeze: August 12, 2019 > > Code Freeze: August 15, 2019 > > Voting Date: August 22, 2019 > > Release Date: August 27, 2019 > > > WDYT? > > > If nobody minds, I will create branch 2.7.6 based on 2.7.5 and set up > >> in > the TC Bot during the weekend. > > > > Sincerely, > > Dmitriy Pavlov > > >> > >> > >> > >> -- > >> Best regards, > >> Ivan Pavlukhin > >> >
Ignite commits atomicity & GG tickets
Hi Igniters, I little bit upset because I sometimes find GG tickets mentioned in Ignite source code as a reason for Ignoring tests, todos, etc. I am personally grateful to all GG's employes for contributing to Ignite code base. I just want to some accuracy for commits provided to Ignite community, it should contain ONLY tickets which is available to all members. Do you find the presence of GG's tickets in the commit is a reason for revert? Sincerely, Dmitriy Pavlov
Re: Apache Ignite 2.7.6 (Time, Scope, and Release manager)
+1 for IGNITE-10451 Thanks, Raymond. Sent from my iPhone > On 9/08/2019, at 5:40 AM, Dmitriy Pavlov wrote: > > Hi Ivan, Ilya, Igniters, > > > > I would like this release would be as minimal as possible. > > > > According to dates proposed we could freeze scope at 12.08, 4 days is more > than enough to stand up and say, ‘Hey, I have an urgent fix’. But it is > also ok for me if we decide to have more relaxed dates. > > > > For now, I suppose the following fixed should be cherry-picked: > > https://issues.apache.org/jira/browse/IGNITE-11767 (Blocker) > GridDhtPartitionsFullMessage retains huge maps on heap in exchange history > > > > https://issues.apache.org/jira/browse/IGNITE-10451 (Major Bug) .NET: > Persistence does not work with custom affinity function > > > > https://issues.apache.org/jira/browse/IGNITE-9562 (Critical Bug) Destroyed > cache that resurrected on an old offline node breaks PME > > > > But I will continue to research JIRA. > > > > Sincerely, > > Dmitriy Pavlov > > > чт, 8 авг. 2019 г. в 17:30, Павлухин Иван : > >>> What's the scope for this release? >> >> Same question. >> >> On the other hand an idea of 2.7.6 release attracts me because having >> a practice of doing frequent minor releases can help us to build >> reliable and predictable release rails. >> >> чт, 8 авг. 2019 г. в 15:09, Ilya Kasnacheev : >>> >>> Hello! >>> >>> What's the scope for this release? >>> >>> Regards, >>> -- >>> Ilya Kasnacheev >>> >>> >>> чт, 8 авг. 2019 г. в 15:07, Dmitriy Pavlov : >>> Hi Apache Ignite Developers, We seem to be on the same page about 2.8 release, but we’ve started new practice - minor releases, the first release was 2.7.5. Unfortunately, there is a couple of issues still not fixed in that release, so I would like to propose to prepare one more minor release for Apache Ignite. I propose my candidacy to be release manager once again. I could (of course with help from Peter Ivanov) do some additional >> efforts to complete and improve process doc https://cwiki.apache.org/confluence/display/IGNITE/Release+Process I propose (most optimistic) dates for release stages: Scope Freeze: August 12, 2019 Code Freeze: August 15, 2019 Voting Date: August 22, 2019 Release Date: August 27, 2019 WDYT? If nobody minds, I will create branch 2.7.6 based on 2.7.5 and set up >> in the TC Bot during the weekend. Sincerely, Dmitriy Pavlov >> >> >> >> -- >> Best regards, >> Ivan Pavlukhin >>
Re: Apache Ignite 2.7.6 (Time, Scope, and Release manager)
Dmitry, Please add this BTree corruption fix to the scope: https://issues.apache.org/jira/browse/IGNITE-11953 Plus, I would upgrade our Spark integration to version 2.4 as long as 2.3 goes with limitations reported by our users: https://issues.apache.org/jira/browse/IGNITE-12054 Nickolay, could you check if that's a quick upgrade? - Denis On Thu, Aug 8, 2019 at 10:40 AM Dmitriy Pavlov wrote: > Hi Ivan, Ilya, Igniters, > > > > I would like this release would be as minimal as possible. > > > > According to dates proposed we could freeze scope at 12.08, 4 days is more > than enough to stand up and say, ‘Hey, I have an urgent fix’. But it is > also ok for me if we decide to have more relaxed dates. > > > > For now, I suppose the following fixed should be cherry-picked: > > https://issues.apache.org/jira/browse/IGNITE-11767 (Blocker) > GridDhtPartitionsFullMessage retains huge maps on heap in exchange history > > > > https://issues.apache.org/jira/browse/IGNITE-10451 (Major Bug) .NET: > Persistence does not work with custom affinity function > > > > https://issues.apache.org/jira/browse/IGNITE-9562 (Critical Bug) Destroyed > cache that resurrected on an old offline node breaks PME > > > > But I will continue to research JIRA. > > > > Sincerely, > > Dmitriy Pavlov > > > чт, 8 авг. 2019 г. в 17:30, Павлухин Иван : > > > > What's the scope for this release? > > > > Same question. > > > > On the other hand an idea of 2.7.6 release attracts me because having > > a practice of doing frequent minor releases can help us to build > > reliable and predictable release rails. > > > > чт, 8 авг. 2019 г. в 15:09, Ilya Kasnacheev : > > > > > > Hello! > > > > > > What's the scope for this release? > > > > > > Regards, > > > -- > > > Ilya Kasnacheev > > > > > > > > > чт, 8 авг. 2019 г. в 15:07, Dmitriy Pavlov : > > > > > > > Hi Apache Ignite Developers, > > > > > > > > > > > > > > > > We seem to be on the same page about 2.8 release, but we’ve started > new > > > > practice - minor releases, the first release was 2.7.5. > Unfortunately, > > > > there is a couple of issues still not fixed in that release, so I > would > > > > like to propose to prepare one more minor release for Apache Ignite. > > > > > > > > > > > > > > > > I propose my candidacy to be release manager once again. > > > > > > > > > > > > > > > > I could (of course with help from Peter Ivanov) do some additional > > efforts > > > > to complete and improve process doc > > > > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process > > > > > > > > > > > > > > > > I propose (most optimistic) dates for release stages: > > > > > > > > Scope Freeze: August 12, 2019 > > > > > > > > Code Freeze: August 15, 2019 > > > > > > > > Voting Date: August 22, 2019 > > > > > > > > Release Date: August 27, 2019 > > > > > > > > > > > > WDYT? > > > > > > > > > > > > If nobody minds, I will create branch 2.7.6 based on 2.7.5 and set up > > in > > > > the TC Bot during the weekend. > > > > > > > > > > > > > > > > Sincerely, > > > > > > > > Dmitriy Pavlov > > > > > > > > > > > > -- > > Best regards, > > Ivan Pavlukhin > > >
[jira] [Created] (IGNITE-12054) Upgrade Spark module to 2.4
Denis Magda created IGNITE-12054: Summary: Upgrade Spark module to 2.4 Key: IGNITE-12054 URL: https://issues.apache.org/jira/browse/IGNITE-12054 Project: Ignite Issue Type: Task Components: spark Affects Versions: 2.7.5 Reporter: Denis Magda Fix For: 2.7.6 Users can't use APIs that are already available in Spark 2.4: https://stackoverflow.com/questions/57392143/persisting-spark-dataframe-to-ignite Let's upgrade Spark from 2.3 to 2.4 until we extract the Spark Integration as a separate module that can support multiple Spark versions. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
Re: Apache Ignite 2.7.6 (Time, Scope, and Release manager)
Hi Ivan, Ilya, Igniters, I would like this release would be as minimal as possible. According to dates proposed we could freeze scope at 12.08, 4 days is more than enough to stand up and say, ‘Hey, I have an urgent fix’. But it is also ok for me if we decide to have more relaxed dates. For now, I suppose the following fixed should be cherry-picked: https://issues.apache.org/jira/browse/IGNITE-11767 (Blocker) GridDhtPartitionsFullMessage retains huge maps on heap in exchange history https://issues.apache.org/jira/browse/IGNITE-10451 (Major Bug) .NET: Persistence does not work with custom affinity function https://issues.apache.org/jira/browse/IGNITE-9562 (Critical Bug) Destroyed cache that resurrected on an old offline node breaks PME But I will continue to research JIRA. Sincerely, Dmitriy Pavlov чт, 8 авг. 2019 г. в 17:30, Павлухин Иван : > > What's the scope for this release? > > Same question. > > On the other hand an idea of 2.7.6 release attracts me because having > a practice of doing frequent minor releases can help us to build > reliable and predictable release rails. > > чт, 8 авг. 2019 г. в 15:09, Ilya Kasnacheev : > > > > Hello! > > > > What's the scope for this release? > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > чт, 8 авг. 2019 г. в 15:07, Dmitriy Pavlov : > > > > > Hi Apache Ignite Developers, > > > > > > > > > > > > We seem to be on the same page about 2.8 release, but we’ve started new > > > practice - minor releases, the first release was 2.7.5. Unfortunately, > > > there is a couple of issues still not fixed in that release, so I would > > > like to propose to prepare one more minor release for Apache Ignite. > > > > > > > > > > > > I propose my candidacy to be release manager once again. > > > > > > > > > > > > I could (of course with help from Peter Ivanov) do some additional > efforts > > > to complete and improve process doc > > > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process > > > > > > > > > > > > I propose (most optimistic) dates for release stages: > > > > > > Scope Freeze: August 12, 2019 > > > > > > Code Freeze: August 15, 2019 > > > > > > Voting Date: August 22, 2019 > > > > > > Release Date: August 27, 2019 > > > > > > > > > WDYT? > > > > > > > > > If nobody minds, I will create branch 2.7.6 based on 2.7.5 and set up > in > > > the TC Bot during the weekend. > > > > > > > > > > > > Sincerely, > > > > > > Dmitriy Pavlov > > > > > > > -- > Best regards, > Ivan Pavlukhin >
Re: Coding guidelines. Useless JavaDoc comments.
I can agree that some part of javadocs we have is useless. It relates to
DTOs, getters/setters without side-effects, short self-descriptive methods.
In an ideal world, proper modularization of architecture, leading to
KISS/SOLID/DRY/etc. principles, writing self-documented code should result
in javadocs disappearing, as they become not needed.
We live in a not ideal world. We don't have good architecture and can't
always lead to mentioned principles, because we need sometimes sacrifice
readability for optimization, fixing a critical bug, etc.
I think that API of our core internal things like PageMemory, WAL, all
existing managers and processors should be as well documented as possible.
If a developer uses some module / manager / processor without looking
inside, reading the only description of public methods, it's a good sign
that this part of the functionality is well documented.
Internal implementation should be also clear for a developer who likes to
make a change inside it. Every optimization, avoiding race-condition, not
obvious thing and especially crutch must be documented as detailed as
possible.
Documentation should be a result of a proper code review. If a reviewer has
questions regarding any code line it should be either refactored to make
this thing obvious or well documented.
If a class or method is self-documented and obvious there is no need to
document it anyway.
if each person takes the code review as seriously as possible,
documentation and code will be better automatically.
Mandatory documentation in places where it's really not needed looks like a
burden. A developer will avoid write it properly everywhere or do it "just
for check" and this will influence on documentation with the negative side.
Flexible approach with mandatory / optional javadocs with good code review
will result in readability improvement overall.
чт, 8 авг. 2019 г. в 17:52, Maxim Muzafarov :
> Ivan,
>
> It is not a problem to check Javadocs at the moment code syle checking
> performed, but do we really need this? And the human-factor you
> mentioned above is also related to the `self-descriptive` names. I
> assume, that someone now is desiring to use single-letter variables
> and single-letter class names to save space an time. We will always
> have such an opinion race.
>
> I'd prefer to leave the current situation with Javadoc as it is and
> just to ask to apply the patch [1][2]. Can you? :-)
>
> [1] https://issues.apache.org/jira/browse/IGNITE-12051
> [2] https://github.com/apache/ignite/pull/6760
>
> On Thu, 8 Aug 2019 at 17:24, Павлухин Иван wrote:
> >
> > Maxim,
> >
> > My main concern here is a human factor. Humans are usually not so good
> > in keeping documentation always in sync with a code. Examples from an
> > actual PR:
> > /**
> > * @param nodeId The remote node id.
> > * @param channel The channel to notify listeners with.
> > */
> > private void onChannelOpened0(UUID nodeId, GridIoMessage initMsg,
> > Channel channel)
> >
> > First, there is a mismatch between number of parameters in javadoc and
> > code. Second, e.g. nodeId name can be made self-descriptive rmtNodeId
> > name.
> >
> > Mandatory javadocs do not imply good javadocs. Good javadocs do not
> > imply javadocs for every method and field. For me, mandatory and good
> > javadocs are like communism. Sounds quite nice in theory, but not
> > feasible in practice.
> >
> > чт, 8 авг. 2019 г. в 16:55, Denis Garus :
> > >
> > > Thank you, Maxim.
> > >
> > > >>On what we are trying to save by making Javadoc optional?
> > >
> > > Space and time.
> > >
> > > I doubt that we need a verbal description of what private method make.
> > > Why we need it if we just can read the code?
> > >
> > > Bright examples:
> > >
> > > /**
> > > * @param helper Helper to attach to kernal context.
> > > */
> > > private void addHelper(Object helper) {
> > > ctx.addHelper(helper);
> > > }
> > >
> > > /**
> > > * Gets "on" or "off" string for given boolean value.
> > > *
> > > * @param b Boolean value to convert.
> > > * @return Result string.
> > > */
> > > private String onOff(boolean b) {
> > > return b ? "on" : "off";
> > > }
> > >
> > > You have to support both code of method and their JavaDoc description,
> but
> > > it doesn't get any sake.
> > >
> > > >>Let's just help them to read the source code.
> > >
> > > But at the same time, public method IgniteKernal#start doesn't have any
> > > description except enumeration of attributes with apparent notes.
> > > Maybe to give it a verbal description needs one or two pages of the
> screen.
> > > Does it make sense?
> > >
> > > чт, 8 авг. 2019 г. в 15:41, Maxim Muzafarov :
> > >
> > > > Igniters,
> > > >
> > > > I'm -1 with making Javadoc optional (except tests).
> > > >
> > > > Here is my patch [1] with PR [2] which fixes all the Javadoc comments
> > > > for the IgniteKernal class mentioned above. Please, take a look. The
> > > > review is very appreciated.
> > > >
> > > > On what we are trying to save by
Re: Coding guidelines. Useless JavaDoc comments.
Hello Igniters,
[A quick introduction: My name is Garrett and I manage documentation at
GridGain. I'm relatively new here — I've been at GridGain for 4 months —
but I've been in Documentation for most of my career.]
I put together some _proposed_ guidelines for Javadocs as a starting point
for us to discuss, *modify*, and hopefully eventually adopt.
These are based on what I've seen succeed at other companies/on the
internet:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=123900577
For me, the reason I prefer optional comments is that I don't want users to
have to wade through the obvious stuff to find the useful/insightful
information. If we can reduce the amount of comments, but still retain the
useful information, the readers will be rewarded.
Thanks!
===
Garrett Alley
Documentation
GridGain Systems
On Thu, Aug 8, 2019 at 7:52 AM Maxim Muzafarov wrote:
> Ivan,
>
> It is not a problem to check Javadocs at the moment code syle checking
> performed, but do we really need this? And the human-factor you
> mentioned above is also related to the `self-descriptive` names. I
> assume, that someone now is desiring to use single-letter variables
> and single-letter class names to save space an time. We will always
> have such an opinion race.
>
> I'd prefer to leave the current situation with Javadoc as it is and
> just to ask to apply the patch [1][2]. Can you? :-)
>
> [1] https://issues.apache.org/jira/browse/IGNITE-12051
> [2] https://github.com/apache/ignite/pull/6760
>
> On Thu, 8 Aug 2019 at 17:24, Павлухин Иван wrote:
> >
> > Maxim,
> >
> > My main concern here is a human factor. Humans are usually not so good
> > in keeping documentation always in sync with a code. Examples from an
> > actual PR:
> > /**
> > * @param nodeId The remote node id.
> > * @param channel The channel to notify listeners with.
> > */
> > private void onChannelOpened0(UUID nodeId, GridIoMessage initMsg,
> > Channel channel)
> >
> > First, there is a mismatch between number of parameters in javadoc and
> > code. Second, e.g. nodeId name can be made self-descriptive rmtNodeId
> > name.
> >
> > Mandatory javadocs do not imply good javadocs. Good javadocs do not
> > imply javadocs for every method and field. For me, mandatory and good
> > javadocs are like communism. Sounds quite nice in theory, but not
> > feasible in practice.
> >
> > чт, 8 авг. 2019 г. в 16:55, Denis Garus :
> > >
> > > Thank you, Maxim.
> > >
> > > >>On what we are trying to save by making Javadoc optional?
> > >
> > > Space and time.
> > >
> > > I doubt that we need a verbal description of what private method make.
> > > Why we need it if we just can read the code?
> > >
> > > Bright examples:
> > >
> > > /**
> > > * @param helper Helper to attach to kernal context.
> > > */
> > > private void addHelper(Object helper) {
> > > ctx.addHelper(helper);
> > > }
> > >
> > > /**
> > > * Gets "on" or "off" string for given boolean value.
> > > *
> > > * @param b Boolean value to convert.
> > > * @return Result string.
> > > */
> > > private String onOff(boolean b) {
> > > return b ? "on" : "off";
> > > }
> > >
> > > You have to support both code of method and their JavaDoc description,
> but
> > > it doesn't get any sake.
> > >
> > > >>Let's just help them to read the source code.
> > >
> > > But at the same time, public method IgniteKernal#start doesn't have any
> > > description except enumeration of attributes with apparent notes.
> > > Maybe to give it a verbal description needs one or two pages of the
> screen.
> > > Does it make sense?
> > >
> > > чт, 8 авг. 2019 г. в 15:41, Maxim Muzafarov :
> > >
> > > > Igniters,
> > > >
> > > > I'm -1 with making Javadoc optional (except tests).
> > > >
> > > > Here is my patch [1] with PR [2] which fixes all the Javadoc comments
> > > > for the IgniteKernal class mentioned above. Please, take a look. The
> > > > review is very appreciated.
> > > >
> > > > On what we are trying to save by making Javadoc optional? In my
> humble
> > > > opinion - Javadoc comments are mostly concern users who debug the
> > > > Ignite when they have faced with some unhandled exception or related
> > > > to the community members with an average involvement (produces a few
> > > > small patches during the year) but not to the experienced one. Let's
> > > > just help them to read the source code.
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-12051
> > > > [2] https://github.com/apache/ignite/pull/6760
> > > >
> > > >
> > > >
> > > > On Wed, 7 Aug 2019 at 13:54, Andrey Kuznetsov
> wrote:
> > > > >
> > > > > +1 for making javadoc comments optional.
> > > > >
> > > > > - Empty and tautological comments are kind of garbage that reduce
> > > > > readability.
> > > > > - It's better to leave the entity undocumented, than write
> > > > > unexpressive/misleading comment.
> > > > > - Even classes may not require javadocs, e.g. simple DTOs.
> > > > >
> > > > > ср, 7 авг. 2019 г., 13:39 Denis Garus :
> >
[jira] [Created] (IGNITE-12053) Total time threads parked if checkpoint throttling occurred
Maxim Muzafarov created IGNITE-12053: Summary: Total time threads parked if checkpoint throttling occurred Key: IGNITE-12053 URL: https://issues.apache.org/jira/browse/IGNITE-12053 Project: Ignite Issue Type: Improvement Reporter: Maxim Muzafarov Fix For: 2.8 Currently, threads that generate dirty pages during an ongoing checkpoint are throttled. User should have the ability to see total time threads are parked during the checkpoint writing marked pages. Such a metric must be dropped down at checkpoint end. See `PagesWriteThrottle` class as a start point of investigation. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
Re: Coding guidelines. Useless JavaDoc comments.
Ivan,
It is not a problem to check Javadocs at the moment code syle checking
performed, but do we really need this? And the human-factor you
mentioned above is also related to the `self-descriptive` names. I
assume, that someone now is desiring to use single-letter variables
and single-letter class names to save space an time. We will always
have such an opinion race.
I'd prefer to leave the current situation with Javadoc as it is and
just to ask to apply the patch [1][2]. Can you? :-)
[1] https://issues.apache.org/jira/browse/IGNITE-12051
[2] https://github.com/apache/ignite/pull/6760
On Thu, 8 Aug 2019 at 17:24, Павлухин Иван wrote:
>
> Maxim,
>
> My main concern here is a human factor. Humans are usually not so good
> in keeping documentation always in sync with a code. Examples from an
> actual PR:
> /**
> * @param nodeId The remote node id.
> * @param channel The channel to notify listeners with.
> */
> private void onChannelOpened0(UUID nodeId, GridIoMessage initMsg,
> Channel channel)
>
> First, there is a mismatch between number of parameters in javadoc and
> code. Second, e.g. nodeId name can be made self-descriptive rmtNodeId
> name.
>
> Mandatory javadocs do not imply good javadocs. Good javadocs do not
> imply javadocs for every method and field. For me, mandatory and good
> javadocs are like communism. Sounds quite nice in theory, but not
> feasible in practice.
>
> чт, 8 авг. 2019 г. в 16:55, Denis Garus :
> >
> > Thank you, Maxim.
> >
> > >>On what we are trying to save by making Javadoc optional?
> >
> > Space and time.
> >
> > I doubt that we need a verbal description of what private method make.
> > Why we need it if we just can read the code?
> >
> > Bright examples:
> >
> > /**
> > * @param helper Helper to attach to kernal context.
> > */
> > private void addHelper(Object helper) {
> > ctx.addHelper(helper);
> > }
> >
> > /**
> > * Gets "on" or "off" string for given boolean value.
> > *
> > * @param b Boolean value to convert.
> > * @return Result string.
> > */
> > private String onOff(boolean b) {
> > return b ? "on" : "off";
> > }
> >
> > You have to support both code of method and their JavaDoc description, but
> > it doesn't get any sake.
> >
> > >>Let's just help them to read the source code.
> >
> > But at the same time, public method IgniteKernal#start doesn't have any
> > description except enumeration of attributes with apparent notes.
> > Maybe to give it a verbal description needs one or two pages of the screen.
> > Does it make sense?
> >
> > чт, 8 авг. 2019 г. в 15:41, Maxim Muzafarov :
> >
> > > Igniters,
> > >
> > > I'm -1 with making Javadoc optional (except tests).
> > >
> > > Here is my patch [1] with PR [2] which fixes all the Javadoc comments
> > > for the IgniteKernal class mentioned above. Please, take a look. The
> > > review is very appreciated.
> > >
> > > On what we are trying to save by making Javadoc optional? In my humble
> > > opinion - Javadoc comments are mostly concern users who debug the
> > > Ignite when they have faced with some unhandled exception or related
> > > to the community members with an average involvement (produces a few
> > > small patches during the year) but not to the experienced one. Let's
> > > just help them to read the source code.
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-12051
> > > [2] https://github.com/apache/ignite/pull/6760
> > >
> > >
> > >
> > > On Wed, 7 Aug 2019 at 13:54, Andrey Kuznetsov wrote:
> > > >
> > > > +1 for making javadoc comments optional.
> > > >
> > > > - Empty and tautological comments are kind of garbage that reduce
> > > > readability.
> > > > - It's better to leave the entity undocumented, than write
> > > > unexpressive/misleading comment.
> > > > - Even classes may not require javadocs, e.g. simple DTOs.
> > > >
> > > > ср, 7 авг. 2019 г., 13:39 Denis Garus :
> > > >
> > > > > Thx for feedback!
> > > > >
> > > > > >> we have to write proper javadoc for all production classes,
> > > including
> > > > > internal.
> > > > >
> > > > > Nikolay, I cannot agree with it.
> > > > >
> > > > > What should be the best comment for the next fields?
> > > > > /** */
> > > > > private static final long serialVersionUID = 0L;
> > > > > or
> > > > > /** */
> > > > > @LoggerResource
> > > > > private IgniteLogger log;
> > > > >
> > > > > There are more than 8000 lines of /** */ only at the ignite-core
> > > module (do
> > > > > not include tests).
> > > > >
> > > > > Any comments will be redundant and just noise. Obvious comment learns
> > > > > readers skip all comments.
> > > > >
> > > > >
> > > > > >> identical distance/padding/margin between fields in a class - is
> > > really
> > > > > cool
> > > > >
> > > > > Vyacheslav, but we have a blank line between fields. Why is one blank
> > > line
> > > > > not enough?
> > > > >
> > > > > ср, 7 авг. 2019 г. в 12:58, Павлухин Иван :
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Denis, thank you for starting this discussion!
> > > > >
Re: Apache Ignite 2.7.6 (Time, Scope, and Release manager)
> What's the scope for this release? Same question. On the other hand an idea of 2.7.6 release attracts me because having a practice of doing frequent minor releases can help us to build reliable and predictable release rails. чт, 8 авг. 2019 г. в 15:09, Ilya Kasnacheev : > > Hello! > > What's the scope for this release? > > Regards, > -- > Ilya Kasnacheev > > > чт, 8 авг. 2019 г. в 15:07, Dmitriy Pavlov : > > > Hi Apache Ignite Developers, > > > > > > > > We seem to be on the same page about 2.8 release, but we’ve started new > > practice - minor releases, the first release was 2.7.5. Unfortunately, > > there is a couple of issues still not fixed in that release, so I would > > like to propose to prepare one more minor release for Apache Ignite. > > > > > > > > I propose my candidacy to be release manager once again. > > > > > > > > I could (of course with help from Peter Ivanov) do some additional efforts > > to complete and improve process doc > > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process > > > > > > > > I propose (most optimistic) dates for release stages: > > > > Scope Freeze: August 12, 2019 > > > > Code Freeze: August 15, 2019 > > > > Voting Date: August 22, 2019 > > > > Release Date: August 27, 2019 > > > > > > WDYT? > > > > > > If nobody minds, I will create branch 2.7.6 based on 2.7.5 and set up in > > the TC Bot during the weekend. > > > > > > > > Sincerely, > > > > Dmitriy Pavlov > > -- Best regards, Ivan Pavlukhin
Re: Coding guidelines. Useless JavaDoc comments.
Maxim,
My main concern here is a human factor. Humans are usually not so good
in keeping documentation always in sync with a code. Examples from an
actual PR:
/**
* @param nodeId The remote node id.
* @param channel The channel to notify listeners with.
*/
private void onChannelOpened0(UUID nodeId, GridIoMessage initMsg,
Channel channel)
First, there is a mismatch between number of parameters in javadoc and
code. Second, e.g. nodeId name can be made self-descriptive rmtNodeId
name.
Mandatory javadocs do not imply good javadocs. Good javadocs do not
imply javadocs for every method and field. For me, mandatory and good
javadocs are like communism. Sounds quite nice in theory, but not
feasible in practice.
чт, 8 авг. 2019 г. в 16:55, Denis Garus :
>
> Thank you, Maxim.
>
> >>On what we are trying to save by making Javadoc optional?
>
> Space and time.
>
> I doubt that we need a verbal description of what private method make.
> Why we need it if we just can read the code?
>
> Bright examples:
>
> /**
> * @param helper Helper to attach to kernal context.
> */
> private void addHelper(Object helper) {
> ctx.addHelper(helper);
> }
>
> /**
> * Gets "on" or "off" string for given boolean value.
> *
> * @param b Boolean value to convert.
> * @return Result string.
> */
> private String onOff(boolean b) {
> return b ? "on" : "off";
> }
>
> You have to support both code of method and their JavaDoc description, but
> it doesn't get any sake.
>
> >>Let's just help them to read the source code.
>
> But at the same time, public method IgniteKernal#start doesn't have any
> description except enumeration of attributes with apparent notes.
> Maybe to give it a verbal description needs one or two pages of the screen.
> Does it make sense?
>
> чт, 8 авг. 2019 г. в 15:41, Maxim Muzafarov :
>
> > Igniters,
> >
> > I'm -1 with making Javadoc optional (except tests).
> >
> > Here is my patch [1] with PR [2] which fixes all the Javadoc comments
> > for the IgniteKernal class mentioned above. Please, take a look. The
> > review is very appreciated.
> >
> > On what we are trying to save by making Javadoc optional? In my humble
> > opinion - Javadoc comments are mostly concern users who debug the
> > Ignite when they have faced with some unhandled exception or related
> > to the community members with an average involvement (produces a few
> > small patches during the year) but not to the experienced one. Let's
> > just help them to read the source code.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-12051
> > [2] https://github.com/apache/ignite/pull/6760
> >
> >
> >
> > On Wed, 7 Aug 2019 at 13:54, Andrey Kuznetsov wrote:
> > >
> > > +1 for making javadoc comments optional.
> > >
> > > - Empty and tautological comments are kind of garbage that reduce
> > > readability.
> > > - It's better to leave the entity undocumented, than write
> > > unexpressive/misleading comment.
> > > - Even classes may not require javadocs, e.g. simple DTOs.
> > >
> > > ср, 7 авг. 2019 г., 13:39 Denis Garus :
> > >
> > > > Thx for feedback!
> > > >
> > > > >> we have to write proper javadoc for all production classes,
> > including
> > > > internal.
> > > >
> > > > Nikolay, I cannot agree with it.
> > > >
> > > > What should be the best comment for the next fields?
> > > > /** */
> > > > private static final long serialVersionUID = 0L;
> > > > or
> > > > /** */
> > > > @LoggerResource
> > > > private IgniteLogger log;
> > > >
> > > > There are more than 8000 lines of /** */ only at the ignite-core
> > module (do
> > > > not include tests).
> > > >
> > > > Any comments will be redundant and just noise. Obvious comment learns
> > > > readers skip all comments.
> > > >
> > > >
> > > > >> identical distance/padding/margin between fields in a class - is
> > really
> > > > cool
> > > >
> > > > Vyacheslav, but we have a blank line between fields. Why is one blank
> > line
> > > > not enough?
> > > >
> > > > ср, 7 авг. 2019 г. в 12:58, Павлухин Иван :
> > > >
> > > > > Hi,
> > > > >
> > > > > Denis, thank you for starting this discussion!
> > > > >
> > > > > My opinion here is that having a good javadoc for every class and
> > > > > method is not feasible in the real world. I am quite curious to see a
> > > > > non-trivial project which follows it. Also, all comments and javadocs
> > > > > are prone to become misleading when code changes (human factor). In
> > my
> > > > > experience good method and variable names and clear code organization
> > > > > often are more helpful than javadocs.
> > > > >
> > > > > ср, 7 авг. 2019 г. в 12:49, Vyacheslav Daradur > >:
> > > > > >
> > > > > > I agree that useless comments look weird in the codebase.
> > > > > >
> > > > > > But, identical distance/padding/margin between fields in a class -
> > is
> > > > > > really cool, and helps read the class very fast.
> > > > > >
> > > > > > On Wed, Aug 7, 2019 at 12:26 PM Nikolay Izhikov <
> > nizhi...@apache.org>
> > > > > wrote:
> > > > > > >
> > > > > > >
Re: Coding guidelines. Useless JavaDoc comments.
Thank you, Maxim.
>>On what we are trying to save by making Javadoc optional?
Space and time.
I doubt that we need a verbal description of what private method make.
Why we need it if we just can read the code?
Bright examples:
/**
* @param helper Helper to attach to kernal context.
*/
private void addHelper(Object helper) {
ctx.addHelper(helper);
}
/**
* Gets "on" or "off" string for given boolean value.
*
* @param b Boolean value to convert.
* @return Result string.
*/
private String onOff(boolean b) {
return b ? "on" : "off";
}
You have to support both code of method and their JavaDoc description, but
it doesn't get any sake.
>>Let's just help them to read the source code.
But at the same time, public method IgniteKernal#start doesn't have any
description except enumeration of attributes with apparent notes.
Maybe to give it a verbal description needs one or two pages of the screen.
Does it make sense?
чт, 8 авг. 2019 г. в 15:41, Maxim Muzafarov :
> Igniters,
>
> I'm -1 with making Javadoc optional (except tests).
>
> Here is my patch [1] with PR [2] which fixes all the Javadoc comments
> for the IgniteKernal class mentioned above. Please, take a look. The
> review is very appreciated.
>
> On what we are trying to save by making Javadoc optional? In my humble
> opinion - Javadoc comments are mostly concern users who debug the
> Ignite when they have faced with some unhandled exception or related
> to the community members with an average involvement (produces a few
> small patches during the year) but not to the experienced one. Let's
> just help them to read the source code.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-12051
> [2] https://github.com/apache/ignite/pull/6760
>
>
>
> On Wed, 7 Aug 2019 at 13:54, Andrey Kuznetsov wrote:
> >
> > +1 for making javadoc comments optional.
> >
> > - Empty and tautological comments are kind of garbage that reduce
> > readability.
> > - It's better to leave the entity undocumented, than write
> > unexpressive/misleading comment.
> > - Even classes may not require javadocs, e.g. simple DTOs.
> >
> > ср, 7 авг. 2019 г., 13:39 Denis Garus :
> >
> > > Thx for feedback!
> > >
> > > >> we have to write proper javadoc for all production classes,
> including
> > > internal.
> > >
> > > Nikolay, I cannot agree with it.
> > >
> > > What should be the best comment for the next fields?
> > > /** */
> > > private static final long serialVersionUID = 0L;
> > > or
> > > /** */
> > > @LoggerResource
> > > private IgniteLogger log;
> > >
> > > There are more than 8000 lines of /** */ only at the ignite-core
> module (do
> > > not include tests).
> > >
> > > Any comments will be redundant and just noise. Obvious comment learns
> > > readers skip all comments.
> > >
> > >
> > > >> identical distance/padding/margin between fields in a class - is
> really
> > > cool
> > >
> > > Vyacheslav, but we have a blank line between fields. Why is one blank
> line
> > > not enough?
> > >
> > > ср, 7 авг. 2019 г. в 12:58, Павлухин Иван :
> > >
> > > > Hi,
> > > >
> > > > Denis, thank you for starting this discussion!
> > > >
> > > > My opinion here is that having a good javadoc for every class and
> > > > method is not feasible in the real world. I am quite curious to see a
> > > > non-trivial project which follows it. Also, all comments and javadocs
> > > > are prone to become misleading when code changes (human factor). In
> my
> > > > experience good method and variable names and clear code organization
> > > > often are more helpful than javadocs.
> > > >
> > > > ср, 7 авг. 2019 г. в 12:49, Vyacheslav Daradur >:
> > > > >
> > > > > I agree that useless comments look weird in the codebase.
> > > > >
> > > > > But, identical distance/padding/margin between fields in a class -
> is
> > > > > really cool, and helps read the class very fast.
> > > > >
> > > > > On Wed, Aug 7, 2019 at 12:26 PM Nikolay Izhikov <
> nizhi...@apache.org>
> > > > wrote:
> > > > > >
> > > > > > Hello, Denis.
> > > > > >
> > > > > > Thanks for starting this discussion.
> > > > > >
> > > > > > I think we have to write proper javadoc for all production
> classes,
> > > > including internal.
> > > > > > We should fix useless javadoc you provide.
> > > > > > We should not accept patches without good javadocs.
> > > > > >
> > > > > > As for the tests, seems, we can make javadoc optional.
> > > > > >
> > > > > > What do you think?
> > > > > >
> > > > > >
> > > > > > В Ср, 07/08/2019 в 11:41 +0300, Denis Garus пишет:
> > > > > > > Igniters!
> > > > > > >
> > > > > > > I think it's time to change coding guidelines in part of
> JavaDoc
> > > > comments
> > > > > > > [1]:
> > > > > > > > > Every method, field or initializer public, private or
> protected
> > > > in
> > > > > > >
> > > > > > > top-level,
> > > > > > > > > inner or anonymous type should have at least minimal
> Javadoc
> > > > comments
> > > > > > >
> > > > > > > including
> > > > > > > > > description and description of parameters using
Re: Coding guidelines. Useless JavaDoc comments.
Igniters, I'm -1 with making Javadoc optional (except tests). Here is my patch [1] with PR [2] which fixes all the Javadoc comments for the IgniteKernal class mentioned above. Please, take a look. The review is very appreciated. On what we are trying to save by making Javadoc optional? In my humble opinion - Javadoc comments are mostly concern users who debug the Ignite when they have faced with some unhandled exception or related to the community members with an average involvement (produces a few small patches during the year) but not to the experienced one. Let's just help them to read the source code. [1] https://issues.apache.org/jira/browse/IGNITE-12051 [2] https://github.com/apache/ignite/pull/6760 On Wed, 7 Aug 2019 at 13:54, Andrey Kuznetsov wrote: > > +1 for making javadoc comments optional. > > - Empty and tautological comments are kind of garbage that reduce > readability. > - It's better to leave the entity undocumented, than write > unexpressive/misleading comment. > - Even classes may not require javadocs, e.g. simple DTOs. > > ср, 7 авг. 2019 г., 13:39 Denis Garus : > > > Thx for feedback! > > > > >> we have to write proper javadoc for all production classes, including > > internal. > > > > Nikolay, I cannot agree with it. > > > > What should be the best comment for the next fields? > > /** */ > > private static final long serialVersionUID = 0L; > > or > > /** */ > > @LoggerResource > > private IgniteLogger log; > > > > There are more than 8000 lines of /** */ only at the ignite-core module (do > > not include tests). > > > > Any comments will be redundant and just noise. Obvious comment learns > > readers skip all comments. > > > > > > >> identical distance/padding/margin between fields in a class - is really > > cool > > > > Vyacheslav, but we have a blank line between fields. Why is one blank line > > not enough? > > > > ср, 7 авг. 2019 г. в 12:58, Павлухин Иван : > > > > > Hi, > > > > > > Denis, thank you for starting this discussion! > > > > > > My opinion here is that having a good javadoc for every class and > > > method is not feasible in the real world. I am quite curious to see a > > > non-trivial project which follows it. Also, all comments and javadocs > > > are prone to become misleading when code changes (human factor). In my > > > experience good method and variable names and clear code organization > > > often are more helpful than javadocs. > > > > > > ср, 7 авг. 2019 г. в 12:49, Vyacheslav Daradur : > > > > > > > > I agree that useless comments look weird in the codebase. > > > > > > > > But, identical distance/padding/margin between fields in a class - is > > > > really cool, and helps read the class very fast. > > > > > > > > On Wed, Aug 7, 2019 at 12:26 PM Nikolay Izhikov > > > wrote: > > > > > > > > > > Hello, Denis. > > > > > > > > > > Thanks for starting this discussion. > > > > > > > > > > I think we have to write proper javadoc for all production classes, > > > including internal. > > > > > We should fix useless javadoc you provide. > > > > > We should not accept patches without good javadocs. > > > > > > > > > > As for the tests, seems, we can make javadoc optional. > > > > > > > > > > What do you think? > > > > > > > > > > > > > > > В Ср, 07/08/2019 в 11:41 +0300, Denis Garus пишет: > > > > > > Igniters! > > > > > > > > > > > > I think it's time to change coding guidelines in part of JavaDoc > > > comments > > > > > > [1]: > > > > > > > > Every method, field or initializer public, private or protected > > > in > > > > > > > > > > > > top-level, > > > > > > > > inner or anonymous type should have at least minimal Javadoc > > > comments > > > > > > > > > > > > including > > > > > > > > description and description of parameters using @param, @return > > > and > > > > > > > > > > > > @throws Javadoc tags, > > > > > > > > where applicable. > > > > > > > > > > > > Let's look at JavaDoc comments in the IgniteKernal class: > > > > > > > > > > > > Why? > > > > > > > > > > > > /** */ - 15 matches. > > > > > > > > > > > > What can you get new from these comments? > > > > > > > > > > > > /** Periodic starvation check interval. */ > > > > > > private static final long PERIODIC_STARVATION_CHECK_FREQ = 1000 * > > 30; > > > > > > > > > > > > /** Long jvm pause detector. */ > > > > > > private LongJVMPauseDetector longJVMPauseDetector; > > > > > > > > > > > > /** Scheduler. */ > > > > > > private IgniteScheduler scheduler; > > > > > > > > > > > > /** Stop guard. */ > > > > > > private final AtomicBoolean stopGuard = new AtomicBoolean(); > > > > > > > > > > > > /** > > > > > > * @param cfg Configuration to use. > > > > > > * @param utilityCachePool Utility cache pool. > > > > > > * @param execSvc Executor service. > > > > > > * @param sysExecSvc System executor service. > > > > > > * @param stripedExecSvc Striped executor. > > > > > > * @param p2pExecSvc P2P executor service. > > > > > > * @param mgmtExecSvc Management executor
[jira] [Created] (IGNITE-12052) GridDhtTxPrepareFuture should print transaction in case of assertion error
Sergey Antonov created IGNITE-12052:
---
Summary: GridDhtTxPrepareFuture should print transaction in case
of assertion error
Key: IGNITE-12052
URL: https://issues.apache.org/jira/browse/IGNITE-12052
Project: Ignite
Issue Type: Improvement
Reporter: Sergey Antonov
Assignee: Sergey Antonov
Fix For: 2.8
At the moment we getting exception, if assertion error occurs:
{noformat}
2019-07-13
20:23:00.769[ERROR][srvc-deploy-#299%xxx_GRID%xxxGridNodeName%][o.a.i.i.p.s.GridServiceProcessor]
Error when executing service: null
java.lang.AssertionError: Got removed exception on entry with dht local
candidate: IgniteTxEntry []
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.sendPrepareRequests(GridDhtTxPrepareFuture.java:1368)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare0(GridDhtTxPrepareFuture.java:1276)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.access$000(GridDhtTxPrepareFuture.java:109)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture$2.apply(GridDhtTxPrepareFuture.java:704)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture$2.apply(GridDhtTxPrepareFuture.java:699)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:383)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:347)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:335)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:495)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:474)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture.onDone(GridDhtForceKeysFuture.java:153)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture.onDone(GridDhtForceKeysFuture.java:69)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:451)
at
org.apache.ignite.internal.util.future.GridCompoundFuture.checkComplete(GridCompoundFuture.java:285)
at
org.apache.ignite.internal.util.future.GridCompoundFuture.markInitialized(GridCompoundFuture.java:276)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture.init(GridDhtForceKeysFuture.java:217)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader$2.apply(GridDhtPreloader.java:523)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader$2.apply(GridDhtPreloader.java:521)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:383)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:347)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:335)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:495)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:474)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:451)
at
org.apache.ignite.internal.util.future.GridCompoundFuture.checkComplete(GridCompoundFuture.java:285)
at
org.apache.ignite.internal.util.future.GridCompoundFuture.apply(GridCompoundFuture.java:144)
at
org.apache.ignite.internal.util.future.GridCompoundFuture.apply(GridCompoundFuture.java:45)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:383)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:347)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:335)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:495)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:474)
at
org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache$AffinityReadyFuture.onDone(GridAffinityAssignmentCache.java:850)
at
org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache$AffinityReadyFuture.onDone(GridAffinityAssignmentCache.java:834)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:451)
at
Re: Replacing NodeFilter functionality with label approach
I agree that attribute-based filtering is enough. We should get rid of predicates in configuration as much as possible: they introduce a lot of complexity for other platforms (.NET), among other things mentioned above. On Thu, Aug 8, 2019 at 2:04 PM Pavel Kovalenko wrote: > Ivan, > > > And there is also one idea (I am not fan of it but still). Can we use > > some kind of scripting for nodes filtering? In that case node filter > > is represented by script string, e.g. javascript. > > I guess it can lead to the same situation as in Java NodeFilter's. We can't > control what happens in a filter in this case. > We can consider regex as an option instead of just labels. It's still > string and can be validated on correctness during node start. > But we still don't have any real examples that require more flexibility > than labels have. > > вт, 6 авг. 2019 г. в 14:46, Павлухин Иван : > > > Alexey, > > > > It seems that a problem has a solution with using 2 attributes or 2 > > labels. Is not it more clear than using custom code? > > > > Folks, > > > > > I don't think we should take "hard to implement" as an argument in this > > discussion :) > > Did not fully get the point. KISS principle is not true anymore? Or is > > this discussion somehow special? I believe that every flexibility > > handle should be critically justified. Would be great to justify > > NodeFilter flexibility. > > > > > Filters based of hostname or ip address. > > Is it a good idea to use IP address for node filtering? IP can be > > changed for a node with persistence, does it mean that not relevant > > data (according to a filter) should be cleared, does it work now? > > > > And there is also one idea (I am not fan of it but still). Can we use > > some kind of scripting for nodes filtering? In that case node filter > > is represented by script string, e.g. javascript. > > > > вт, 6 авг. 2019 г. в 12:22, Alexey Kukushkin >: > > > > > > Pavel, > > > > > > Just a real example you asked for: we have a user attribute "ROLE_DC", > > > which is a comma separated list like "wfe_a, as_a, db_a" (server role > and > > > data center designator) and we have node filters to deploy services and > > > start caches on servers with specific role (like WFE) and sometimes > > > specific role and DC (like WFE_A). The node filter splits the list and > > uses > > > a regular expression to match each segment. > > > > > > If you replace generic node filter with a user attribute filter then we > > > still can achieve what we need by creating 3 user attributes (ROLE_DC, > > > ROLE and DC) but we lose flexibility since now adding a new data center > > > requires updating all cache and service configurations. With regex > > matching > > > we do not need to update the configurations since we still match all > the > > > roles in the new DC. > > > > > > So we would have a solution with user attributes filter but I we lose > > some > > > flexibility. > > > > > > -- > > > Best regards, > > > Alexey > > > > > > > > -- > > Best regards, > > Ivan Pavlukhin > > >
Re: Apache Ignite 2.7.6 (Time, Scope, and Release manager)
Hello! What's the scope for this release? Regards, -- Ilya Kasnacheev чт, 8 авг. 2019 г. в 15:07, Dmitriy Pavlov : > Hi Apache Ignite Developers, > > > > We seem to be on the same page about 2.8 release, but we’ve started new > practice - minor releases, the first release was 2.7.5. Unfortunately, > there is a couple of issues still not fixed in that release, so I would > like to propose to prepare one more minor release for Apache Ignite. > > > > I propose my candidacy to be release manager once again. > > > > I could (of course with help from Peter Ivanov) do some additional efforts > to complete and improve process doc > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process > > > > I propose (most optimistic) dates for release stages: > > Scope Freeze: August 12, 2019 > > Code Freeze: August 15, 2019 > > Voting Date: August 22, 2019 > > Release Date: August 27, 2019 > > > WDYT? > > > If nobody minds, I will create branch 2.7.6 based on 2.7.5 and set up in > the TC Bot during the weekend. > > > > Sincerely, > > Dmitriy Pavlov >
Apache Ignite 2.7.6 (Time, Scope, and Release manager)
Hi Apache Ignite Developers, We seem to be on the same page about 2.8 release, but we’ve started new practice - minor releases, the first release was 2.7.5. Unfortunately, there is a couple of issues still not fixed in that release, so I would like to propose to prepare one more minor release for Apache Ignite. I propose my candidacy to be release manager once again. I could (of course with help from Peter Ivanov) do some additional efforts to complete and improve process doc https://cwiki.apache.org/confluence/display/IGNITE/Release+Process I propose (most optimistic) dates for release stages: Scope Freeze: August 12, 2019 Code Freeze: August 15, 2019 Voting Date: August 22, 2019 Release Date: August 27, 2019 WDYT? If nobody minds, I will create branch 2.7.6 based on 2.7.5 and set up in the TC Bot during the weekend. Sincerely, Dmitriy Pavlov
Re: SecurityTestSuite as a separate test suite at TC
Hello, Ivan! >> Could you please provide more details why do we need to run these tests in forked JVM? Surefite documentation [1] says: If forkCount=0, it's impossible to use the system class loader or a plain old Java classpath; we have to use an isolated class loader. When using isolated class loader will cause compiler error: package org.apache.ignite.lang does not exist We cannot compile the TestIgniteCallable class. 1. https://maven.apache.org/surefire/maven-surefire-plugin/examples/class-loading.html чт, 8 авг. 2019 г. в 09:44, Павлухин Иван : > Denis, > > Could you please provide more details why do we need to run these > tests in forked JVM? > > Still, having separate security suite on TC sounds not bad. > > ср, 7 авг. 2019 г. в 09:35, Vyacheslav Daradur : > > > > Hi Denis. > > > > I think it is fine to extract security tests in a separate build plan on > TC. > > > > BTW, if you are going to write a lot of Sandbox's tests pay attention > > to 'extdata' module and an approach of P2P tests > > (IgniteP2PSelfTestSuite) - this may help you to avoid Maven's > > classloading issues. > > > > On Tue, Aug 6, 2019 at 3:25 PM Denis Garus wrote: > > > > > > Hello Igniters! > > > > > > I made the test DoPrivelegedOnRemoteNodeTest[1] (SecurityTestSuite) > for the > > > task "Sandbox for user-defined code" [2] > > > that uses p2p deploy like the test > > > ServiceHotRedeploymentViaDeploymentSpiTest [3] from > > > IgniteServiceGridTestSuite. > > > That test requires additional Maven command line parameter -P > > > surefire-fork-count-1. > > > The suite Basic 1 contains the SecurityTestSuite and many other test > suites > > > at TeamCity that do not need that additional Maven parameter. > > > I suggest extracting SecurityTestSuite as a separate test suite to > define > > > additional Maven command line parameter for it. > > > > > > WDYT? > > > > > > > > > 1. https://github.com/apache/ignite/pull/6707 > > > 2. https://issues.apache.org/jira/browse/IGNITE-11410 > > > 3. > > > > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/service/ServiceHotRedeploymentViaDeploymentSpiTest.java > > > > > > > > -- > > Best Regards, Vyacheslav D. > > > > -- > Best regards, > Ivan Pavlukhin >
Re: Replacing NodeFilter functionality with label approach
Ivan, > And there is also one idea (I am not fan of it but still). Can we use > some kind of scripting for nodes filtering? In that case node filter > is represented by script string, e.g. javascript. I guess it can lead to the same situation as in Java NodeFilter's. We can't control what happens in a filter in this case. We can consider regex as an option instead of just labels. It's still string and can be validated on correctness during node start. But we still don't have any real examples that require more flexibility than labels have. вт, 6 авг. 2019 г. в 14:46, Павлухин Иван : > Alexey, > > It seems that a problem has a solution with using 2 attributes or 2 > labels. Is not it more clear than using custom code? > > Folks, > > > I don't think we should take "hard to implement" as an argument in this > discussion :) > Did not fully get the point. KISS principle is not true anymore? Or is > this discussion somehow special? I believe that every flexibility > handle should be critically justified. Would be great to justify > NodeFilter flexibility. > > > Filters based of hostname or ip address. > Is it a good idea to use IP address for node filtering? IP can be > changed for a node with persistence, does it mean that not relevant > data (according to a filter) should be cleared, does it work now? > > And there is also one idea (I am not fan of it but still). Can we use > some kind of scripting for nodes filtering? In that case node filter > is represented by script string, e.g. javascript. > > вт, 6 авг. 2019 г. в 12:22, Alexey Kukushkin : > > > > Pavel, > > > > Just a real example you asked for: we have a user attribute "ROLE_DC", > > which is a comma separated list like "wfe_a, as_a, db_a" (server role and > > data center designator) and we have node filters to deploy services and > > start caches on servers with specific role (like WFE) and sometimes > > specific role and DC (like WFE_A). The node filter splits the list and > uses > > a regular expression to match each segment. > > > > If you replace generic node filter with a user attribute filter then we > > still can achieve what we need by creating 3 user attributes (ROLE_DC, > > ROLE and DC) but we lose flexibility since now adding a new data center > > requires updating all cache and service configurations. With regex > matching > > we do not need to update the configurations since we still match all the > > roles in the new DC. > > > > So we would have a solution with user attributes filter but I we lose > some > > flexibility. > > > > -- > > Best regards, > > Alexey > > > > -- > Best regards, > Ivan Pavlukhin >
[jira] [Created] (IGNITE-12051) Update javadoc for the IgniteKernal class
Maxim Muzafarov created IGNITE-12051:
Summary: Update javadoc for the IgniteKernal class
Key: IGNITE-12051
URL: https://issues.apache.org/jira/browse/IGNITE-12051
Project: Ignite
Issue Type: Task
Reporter: Maxim Muzafarov
Assignee: Maxim Muzafarov
Update all ambiguous an empty javadoc comments, such as:
{code}
/** */
/** Periodic starvation check interval. */
private static final long PERIODIC_STARVATION_CHECK_FREQ = 1000 * 30;
/** Long jvm pause detector. */
private LongJVMPauseDetector longJVMPauseDetector;
/** Scheduler. */
private IgniteScheduler scheduler;
/** Stop guard. */
private final AtomicBoolean stopGuard = new AtomicBoolean();
{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (IGNITE-12050) MVCC test suites resurrection
Ivan Pavlukhin created IGNITE-12050: --- Summary: MVCC test suites resurrection Key: IGNITE-12050 URL: https://issues.apache.org/jira/browse/IGNITE-12050 Project: Ignite Issue Type: Bug Components: mvcc Reporter: Ivan Pavlukhin Assignee: Ivan Pavlukhin Need to resurrect MVCC test suites for RunAll. Mute all tests with fail rate more 50% And start use scale factor for MVCC tests to reduce execution time. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
Re: SecurityTestSuite as a separate test suite at TC
Denis, Could you please provide more details why do we need to run these tests in forked JVM? Still, having separate security suite on TC sounds not bad. ср, 7 авг. 2019 г. в 09:35, Vyacheslav Daradur : > > Hi Denis. > > I think it is fine to extract security tests in a separate build plan on TC. > > BTW, if you are going to write a lot of Sandbox's tests pay attention > to 'extdata' module and an approach of P2P tests > (IgniteP2PSelfTestSuite) - this may help you to avoid Maven's > classloading issues. > > On Tue, Aug 6, 2019 at 3:25 PM Denis Garus wrote: > > > > Hello Igniters! > > > > I made the test DoPrivelegedOnRemoteNodeTest[1] (SecurityTestSuite) for the > > task "Sandbox for user-defined code" [2] > > that uses p2p deploy like the test > > ServiceHotRedeploymentViaDeploymentSpiTest [3] from > > IgniteServiceGridTestSuite. > > That test requires additional Maven command line parameter -P > > surefire-fork-count-1. > > The suite Basic 1 contains the SecurityTestSuite and many other test suites > > at TeamCity that do not need that additional Maven parameter. > > I suggest extracting SecurityTestSuite as a separate test suite to define > > additional Maven command line parameter for it. > > > > WDYT? > > > > > > 1. https://github.com/apache/ignite/pull/6707 > > 2. https://issues.apache.org/jira/browse/IGNITE-11410 > > 3. > > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/service/ServiceHotRedeploymentViaDeploymentSpiTest.java > > > > -- > Best Regards, Vyacheslav D. -- Best regards, Ivan Pavlukhin
Re: Partition loss event
Ilya, I am not sure that enabling subset of events will fit all needs. If we would like to do so then it might be good idea to make it clear in API that certain events are enabled by default (e.g. put them into separate class). Also we should be careful with backward compatibility, some kind of events look more like debug stuff, e.g. EVT_CACHE_REBALANCE_OBJECT_LOADED. Is it relevant for historical rebalance? What does it mean for rebalancing whole partition files? How user is going to use it? And a care stil should be put to performance, aforementioned EVT_CACHE_REBALANCE_OBJECT_LOADED can introduce non-negliable impact, cannot it? Also, we can employ some soft means like printing a warning if a listener is registered for disabled event. And a last point about code and javadocs. There is a line "Note that certain events are required for Ignite's internal operations and such events will still be generated". Perhaps we can provide a list of such events in docs or do not expose them to a user listeners. And a bit of mess. IgniteConfiguration#getIncludeEventTypes claims "Note that by default all events in Ignite are disabled". While EventType says "Note that by default all events in Ignite are enabled and therefore generated and stored by whatever event storage SPI is configured". чт, 1 авг. 2019 г. в 17:42, Ilya Kasnacheev : > > Dear fellows! > > I think we have a problem: when events were introduced, we were talking > about high-bandwdith events which may overflow your nodes if you > accidentally turn them on. > > However, now we have a bunch of low-bandwidth events, such as: > EVT_CHECKPOINT_SAVED > EVT_CHECKPOINT_LOADED > EVT_CHECKPOINT_REMOVED > EVT_NODE_JOINED > EVT_NODE_LEFT > EVT_NODE_FAILED > EVT_NODE_SEGMENTED > EVT_CACHE_REBALANCE_STARTED > EVT_CACHE_REBALANCE_STOPPED > EVT_CACHE_REBALANCE_PART_LOADED > EVT_CACHE_REBALANCE_PART_UNLOADED > EVT_CACHE_REBALANCE_OBJECT_LOADED > EVT_CACHE_REBALANCE_OBJECT_UNLOADED > EVT_CACHE_REBALANCE_PART_DATA_LOST > EVT_CACHE_REBALANCE_PART_SUPPLIED > EVT_CACHE_REBALANCE_PART_MISSED > EVT_CLIENT_NODE_DISCONNECTED > EVT_CLIENT_NODE_RECONNECTED > EVT_WAL_SEGMENT_ARCHIVED > EVT_WAL_SEGMENT_COMPACTED > EVT_CLUSTER_ACTIVATED > EVT_CLUSTER_DEACTIVATED > EVT_PAGE_REPLACEMENT_STARTED > > I suggest we enable these events by default, since I fail to see how this > may ever cause problems, but it will definitely decrease confusion > surrounding events. > > WDYT? > > Regards, > -- > Ilya Kasnacheev > > > чт, 1 авг. 2019 г. в 15:18, balazspeterfi : > > > Hi Alexandr, > > > > Thanks, that was the missing part. It would be nice to mention it in the > > docs I guess as it's quite easy to miss it. > > > > Regards, > > Balazs > > > > > > > > -- > > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > > -- Best regards, Ivan Pavlukhin
