Re: [RESULT][VOTE] Ignite Packaging IEP
A wrong vote thread referred? https://lists.apache.org/thread/mrwzck7olzgkt25tzg5co7b6v5so1m93 Best regards, Ivan Pavlukhin пн, 29 авг. 2022 г. в 18:46, Petr Ivanov : > > Hi, Mikhail! > > > Did you mean Apache Ignite 3? > I think we should not forget to add 2 or 3 respectively, 'cause these are > different products whatsoever. > > > Thank you! > > > On 29 Aug 2022, at 17:32, Mikhail Pochatkin wrote: > > > > Hello Igniters, > > > > Switching Apache Ignite build system to Gradle and agreement on all target > > package distribution formats for Apache Ignite has been accepted. > > > > Alexander Polovtcev +1 > > Vyaceslav Koptilin +1 > > Andrey Gura +1 > > > > Vote thread: > > https://lists.apache.org/thread/mrwzck7olzgkt25tzg5co7b6v5so1m93 > > Ignite Packaging IEP Ignite Packaging IEP >
Re: [ANNOUNCE] New PMC member: Taras Ledkov
Congratulations, Taras! Best regards, Ivan Pavlukhin сб, 20 авг. 2022 г. в 00:18, Dmitriy Pavlov : > > Hi Igniters! > > The Project Management Committee (PMC) for Apache Ignite has invited > Taras Ledkov to become a member of the PMC and we are pleased to > announce that he has accepted. > > Taras is a veteran committer, and it is needless to say how long > and successfully he is contributing to the Apache Ignite. > Taras is responsible for a solid bulk of SQL and JDBC code. > > Please join me in congratulating Taras on his new role. > > Best Regards, > Dmitriy Pavlov > on behalf of Apache Ignite PMC
Re: [ANNOUNCE] New PMC member: Vyacheslav Koptilin
Congratulations, Slava! Best regards, Ivan Pavlukhin ср, 17 авг. 2022 г. в 21:11, Roman Puchkovskiy : > > Congratulations, Slava! > > ср, 17 авг. 2022 г. в 22:10, Aleksandr Pakhomov : > > > > Congratulations! Well-deserved. > > > > > > > On 17 Aug 2022, at 17:34, Kseniya Romanova wrote: > > > > > > Hi Igniters! > > > > > > The Project Management Committee (PMC) for Apache Ignite has invited > > > Vyacheslav Koptilin to become a member of the PMC and we are pleased to > > > announce that he has accepted. > > > > > > Vyacheslav is a veteran committer and contributes a lot to the Ignite > > > storage https://github.com/sk0x50. > > > > > > Please join me in congratulating Vyacheslav on his new role. > > > > > > Best Regards, > > > Kseniya Romanova > > > on behalf of Apache Ignite PMC > >
Duplicates [jira] [Created] (IGNITE-17403) SQL "select by key" performance 50-60 times slower than key-value get
Hi, Are these tickets duplicates? Could someone please get rid of them? Reporter is Kirill Gusakov. https://issues.apache.org/jira/browse/IGNITE-17403 https://issues.apache.org/jira/browse/IGNITE-17404 https://issues.apache.org/jira/browse/IGNITE-17405 https://issues.apache.org/jira/browse/IGNITE-17406 Best regards, Ivan Pavlukhin
Re: [VOTE] Add swagger dependency to Ignite 3
+1 Aleksandr, thanks for the explanation. Best regards, Ivan Pavlukhin чт, 26 мая 2022 г. в 21:54, Andrey Gura : > > +1 from me. Open API spec could be useful in the future for > implementing external cluster management tools. > > On Thu, May 26, 2022 at 11:41 AM Aleksandr Pakhomov wrote: > > > > Hi Ivan, > > > > Dependencies that are needed for annotating > > classes are going to be included. From those > > annotations Open API spec is generated by > > maven plugin. That’s it. > > > > If you are asking about swagger ui or any > > web stuff then the answer is no. We are > > not going to include this into production. > > > > > On 26 May 2022, at 07:34, Ivan Pavlukhin wrote: > > > > > > Hi, > > > > > > Are we going to include swagger into production packages? I always > > > thought (I might be mistaken) that swagger should be used during > > > development. Worries are usual: > > > 1. Potential vulnerabilities. > > > 2. Unintentional use of transitive dependencies. > > > > > > Best regards, > > > Ivan Pavlukhin > > > > > > чт, 26 мая 2022 г. в 00:46, Mikhail Pochatkin : > > >> > > >> +1 from me, de facto swagger standard within OpenApi. > > >> > > >> On Mon, May 23, 2022 at 7:57 PM Aleksandr Pakhomov > > >> wrote: > > >> > > >>> Dear community, > > >>> > > >>> Discussion about 3rd party dependencies took place > > >>> and I think it is time to vote if we agreed to include > > >>> swagger dependency to the Ignite 3 or not. > > >>> > > >>> The exact list of dependencies could be fined in IEP-87 [1] > > >>> (swagger-annotations, swagger-core, > > >>> swagger-codegen-maven-plugin) > > >>> > > >>> Micronaut is out of the scope of this voting. I will launch > > >>> a separate one. > > >>> > > >>> The vote is formal, see voting guidelines [2] > > >>> > > >>> +1 - to accept additional dependencies to be included to Java code > > >>> Guidelines [3] > > >>> 0 - don't care either way > > >>> -1 - DO NOT accept (explain why) > > >>> > > >>> [1] > > >>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST#IEP87:OpenAPIsupportforREST-Additionaldependencies > > >>> [2] https://www.apache.org/foundation/voting.html < > > >>> https://www.apache.org/foundation/voting.html> > > >>> [3] > > >>> https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries > >
Re: [VOTE] Add swagger dependency to Ignite 3
Hi, Are we going to include swagger into production packages? I always thought (I might be mistaken) that swagger should be used during development. Worries are usual: 1. Potential vulnerabilities. 2. Unintentional use of transitive dependencies. Best regards, Ivan Pavlukhin чт, 26 мая 2022 г. в 00:46, Mikhail Pochatkin : > > +1 from me, de facto swagger standard within OpenApi. > > On Mon, May 23, 2022 at 7:57 PM Aleksandr Pakhomov wrote: > > > Dear community, > > > > Discussion about 3rd party dependencies took place > > and I think it is time to vote if we agreed to include > > swagger dependency to the Ignite 3 or not. > > > > The exact list of dependencies could be fined in IEP-87 [1] > > (swagger-annotations, swagger-core, > > swagger-codegen-maven-plugin) > > > > Micronaut is out of the scope of this voting. I will launch > > a separate one. > > > > The vote is formal, see voting guidelines [2] > > > > +1 - to accept additional dependencies to be included to Java code > > Guidelines [3] > > 0 - don't care either way > > -1 - DO NOT accept (explain why) > > > > [1] > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST#IEP87:OpenAPIsupportforREST-Additionaldependencies > > [2] https://www.apache.org/foundation/voting.html < > > https://www.apache.org/foundation/voting.html> > > [3] > > https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries
Re: Add to the Contributors group in JIRA
Aleksandr, I added you to a contributors list. Now you can assign tickets to yourself. Best regards, Ivan Pavlukhin вт, 5 апр. 2022 г. в 13:56, Aleksandr Pakhomov : > > Hi Ivan, > > My account name is aleksandr.pakhomov > > Best regards, > Aleksandr Pakhomov > > > On 5 Apr 2022, at 10:53, Ivan Pavlukhin wrote: > > > > Hi Aleksandr, > > > > Welcome to the Apache Ignite Community! > > > > What is your account name in Apache JIRA [1]? > > > > [1] https://issues.apache.org/jira > > > > Best regards, > > Ivan Pavlukhin > > > > вт, 5 апр. 2022 г. в 13:40, Aleksandr Pakhomov : > >> > >> Hello to the apache dev community, > >> > >> I am Pakhomov Aleksandr from Russia and I would like to contribute to the > >> project. Could you please add me to the Contributors group in JIRA? > >> > >> Thank you. >
Re: Add to the Contributors group in JIRA
Hi Aleksandr, Welcome to the Apache Ignite Community! What is your account name in Apache JIRA [1]? [1] https://issues.apache.org/jira Best regards, Ivan Pavlukhin вт, 5 апр. 2022 г. в 13:40, Aleksandr Pakhomov : > > Hello to the apache dev community, > > I am Pakhomov Aleksandr from Russia and I would like to contribute to the > project. Could you please add me to the Contributors group in JIRA? > > Thank you.
IFRA ticket regarding GitHub notifications on dev list
Hi, I observed some GitHub PR notifications on dev list and today and created a ticket https://issues.apache.org/jira/browse/INFRA-23075 Best regards, Ivan Pavlukhin
Re: [ANNOUNCE] Welcome Alexander Lapin as a new committer
Alex, congratulations, well deserved! 2022-03-09 18:18 GMT+03:00, Petr Ivanov : > Congratulations! > >> On 9 Mar 2022, at 17:46, Kseniya Romanova wrote: >> >> Igniters! The Apache Ignite Project Management Committee (PMC) has >> invited >> Alexander Lapin to become a new committer and are happy to announce that >> he >> has >> accepted. >> >> Alexander contributed to the Ignite several important improvements of the >> JDBC thin driver, and tracing framework. Also, he created some valuable >> content about Tracing and Data Rebalance. Now, he is deeply involved in >> developing Apache Ignite 3.0. >> >> Being a committer enables easier contribution to the project since there >> is >> no need to go via the patch submission process. This should enable better >> productivity. >> >> Please join me in welcoming Alexander, and congratulating him on the new >> role in the Apache Ignite Community. >> >> Cheers, >> Kseniya Romanova > > -- Best regards, Ivan Pavlukhin
Re: CVE-2021-42392
AFAIR, H2 console was disabled [1] [1] https://issues.apache.org/jira/browse/IGNITE-11333 2022-01-07 14:25 GMT+03:00, Vishwas Bm : > Is ignite impacted by CVE-2021-42392 ? > > https://jfrog.com/blog/the-jndi-strikes-back-unauthenticated-rce-in-h2-database-console/ > > > Regards, > Vishwas > -- Best regards, Ivan Pavlukhin
Re: [ANNOUNCE] Welcome Vladislav Pyatkov as a new committer
Congrats, Vlad! 2021-12-22 20:48 GMT+03:00, Ivan Daschinsky : > Vlad, congrats! You have definitely deserved it! > > ср, 22 дек. 2021 г. в 20:40, Andrey Mashenkov : > >> Congrats, Vlad! >> >> ср, 22 дек. 2021 г., 20:23 Valentin Kulichenko < >> valentin.kuliche...@gmail.com>: >> >> > The Apache Ignite Project Management Committee (PMC) has invited >> Vladislav >> > Pyatkov to become a new committer and are happy to announce that he has >> > accepted. >> > >> > Vladislav is a veteran of the Apache Ignite project. He joined the >> > community in 2016. >> > He participated in the development of Ignite 1.x, 2.x and he is >> > actively >> > working on a new Apache Ignite 3.0 release. >> > >> > Being a committer enables easier contribution to the project since >> > there >> > is no need to go via the patch submission process. This should enable >> > better productivity. >> > >> > Please join me in welcoming Vlad, and congratulating him on the new >> > role >> > in the Apache Ignite Community. >> > >> > -Val >> > >> > > > -- > Sincerely yours, Ivan Daschinskiy > -- Best regards, Ivan Pavlukhin
Re: [DISCUSSION] @Nullable/@NotNull annotation usage in Ignite 3
Val, > Basically, what I'm suggesting is to accept non-nullable as default Sounds good for me. Then it should be clearly stated in code guidelines and regular static analysis for this should be organized from the beginning. 2021-12-21 20:35 GMT+03:00, Valentin Kulichenko : > Ivan, > > I agree with you. Basically, what I'm suggesting is to accept non-nullable > as default, but if a parameter must be nullable for whatever reason, mark > it with the @Nullable annotation. > > Regarding "a user can put anything there without much thinking": I > understand what you mean, but I believe the annotation is not the problem. > It's nullability itself that causes the ambiguity - null value is always a > special case that means different things for different values. @Nullable at > least sends a signal that nulls need to be considered. If a user doesn't > want to think anyway, there is not much we can do :) (Except trying to make > the variable non-nullable, of course) > > -Val > > On Mon, Dec 20, 2021 at 11:40 PM Ivan Pavlukhin > wrote: > >> Val, >> >> > Therefore, it's crucial to bring the attention of the API's user to >> > such >> parameters. (@Nullable) >> >> This sounds wrong for me. If a method parameter is marked as >> @Nullable, then a user can put anything there without much thinking. >> Opposite happens with @NotNull parameters, with them a user should >> think whether his variable can be null or not. >> >> As for me using nullable variables should be avoided as much as >> possible, but not all developers share this point of view. And >> especially in Ignite codebase it was quite common to use nullable >> variables, for the first time it was very unusual for me. >> >> 2021-12-20 22:06 GMT+03:00, Valentin Kulichenko < >> valentin.kuliche...@gmail.com>: >> > Neither solution is completely bulletproof, and I don't think that's >> > what >> > we are looking for. We need something that provides a reasonable value, >> but >> > also does not clutter the code with too many annotations. >> > >> > I would also keep in mind that annotations are used not only for static >> > analysis, but by developers as well. And from this standpoint, I feel >> > that @Nullable is much more important, because 'null' is a special >> > value >> > which is treated differently in different circumstances. Therefore, >> > it's >> > crucial to bring the attention of the API's user to such parameters. On >> the >> > other hand, non-nullable parameters are inherently safer, so there is >> less >> > need to annotate them. >> > >> > -Val >> > >> > On Mon, Dec 20, 2021 at 7:42 AM Alexander Polovtcev >> > >> > wrote: >> > >> >> Hi, Ivan. >> >> >> >> > it seems not bulletproof >> >> >> >> I completely agree with you. As I wrote in the original message, this >> >> becomes even worse in case of 3-rd party dependencies, because they >> >> may >> >> not >> >> be annotated, which can lead to confusions. But looks like this is not >> >> a >> >> big deal, because these annotations are not intended to cover 100% of >> >> cases >> >> and should not be treated as a guarantee against NPEs. Maybe covering >> >> *some* cases is enough. >> >> >> >> > Perhaps we can insist on not using nulls as much as possible. >> >> >> >> Same here, I agree with you and it correlates with clear API design in >> >> general. However, sometimes nullable parameters and return values are >> >> justified, so how can we formalize that? >> >> >> >> On Sat, Dec 18, 2021 at 10:52 AM Ivan Pavlukhin >> >> wrote: >> >> >> >> > Hi, >> >> > >> >> > While option #2 looks very appealing it seems not bulletproof >> >> > reliable, someone can occasionally miss @Nullable annotation. Option >> >> > #3 seems more practical too me, as missed @NotNull annotations >> >> > cannot >> >> > do much harm. >> >> > >> >> > Also I am thinking about using nullable parameters in general. >> >> > Perhaps >> >> > we can insist on not using nulls as much as possible. With such >> >> > requirement in guidelines option #2 becomes practical. >> >> > >> >> > 2021-12-17 14:47 GMT+03:00, Alexander Polovtcev >> >&g
Re: [DISCUSSION] @Nullable/@NotNull annotation usage in Ignite 3
Val, > Therefore, it's crucial to bring the attention of the API's user to such > parameters. (@Nullable) This sounds wrong for me. If a method parameter is marked as @Nullable, then a user can put anything there without much thinking. Opposite happens with @NotNull parameters, with them a user should think whether his variable can be null or not. As for me using nullable variables should be avoided as much as possible, but not all developers share this point of view. And especially in Ignite codebase it was quite common to use nullable variables, for the first time it was very unusual for me. 2021-12-20 22:06 GMT+03:00, Valentin Kulichenko : > Neither solution is completely bulletproof, and I don't think that's what > we are looking for. We need something that provides a reasonable value, but > also does not clutter the code with too many annotations. > > I would also keep in mind that annotations are used not only for static > analysis, but by developers as well. And from this standpoint, I feel > that @Nullable is much more important, because 'null' is a special value > which is treated differently in different circumstances. Therefore, it's > crucial to bring the attention of the API's user to such parameters. On the > other hand, non-nullable parameters are inherently safer, so there is less > need to annotate them. > > -Val > > On Mon, Dec 20, 2021 at 7:42 AM Alexander Polovtcev > > wrote: > >> Hi, Ivan. >> >> > it seems not bulletproof >> >> I completely agree with you. As I wrote in the original message, this >> becomes even worse in case of 3-rd party dependencies, because they may >> not >> be annotated, which can lead to confusions. But looks like this is not a >> big deal, because these annotations are not intended to cover 100% of >> cases >> and should not be treated as a guarantee against NPEs. Maybe covering >> *some* cases is enough. >> >> > Perhaps we can insist on not using nulls as much as possible. >> >> Same here, I agree with you and it correlates with clear API design in >> general. However, sometimes nullable parameters and return values are >> justified, so how can we formalize that? >> >> On Sat, Dec 18, 2021 at 10:52 AM Ivan Pavlukhin >> wrote: >> >> > Hi, >> > >> > While option #2 looks very appealing it seems not bulletproof >> > reliable, someone can occasionally miss @Nullable annotation. Option >> > #3 seems more practical too me, as missed @NotNull annotations cannot >> > do much harm. >> > >> > Also I am thinking about using nullable parameters in general. Perhaps >> > we can insist on not using nulls as much as possible. With such >> > requirement in guidelines option #2 becomes practical. >> > >> > 2021-12-17 14:47 GMT+03:00, Alexander Polovtcev >> > > >: >> > > Maksim, thank you for the suggestion. >> > > >> > > I've never used NullAway, but after having a quick look I think it >> might >> > be >> > > an overkill, since it is a plugin for the ErrorProne, which is a >> separate >> > > tool. I recall some efforts of introducing ErrorProne to Ignite 3 and >> > they >> > > were not successful. But again, I don't have much experience in that >> > > regard. I wonder if IDEA inspections would be enough, since they are >> easy >> > > to use during development and AFAIK are already run during the TC >> builds. >> > > >> > > Regarding Ignite 2, I don't know if it would be viable to add these >> > > annotations to the existing code (in order to have meaningful >> > > checks), >> > > since the codebase is so large. But nevertheless we can try to adopt >> the >> > > same approach there as well. >> > > >> > > On Fri, Dec 17, 2021 at 12:10 PM Maksim Timonin < >> timoninma...@apache.org >> > > >> > > wrote: >> > > >> > >> Hi! >> > >> >> > >> There is a pretty popular project NullAway [1] that checks code of a >> > >> project in compile-time for possible NPE. AFAIK, it works only with >> the >> > >> "@Nullable" annotation. I think we can try to add this check to >> Ignite2 >> > >> and >> > >> 3. >> > >> >> > >> But I wonder, whether smbd already tried to introduce this check? If >> > not, >> > >> I >> > >> can try, WDYT? >> > >> >> > >> [1] https://github.com/uber/NullAway >> > >> >> > >> On Fri, Dec 17, 2021 at 9:04 AM ткаленко кирилл >> > >> > > >> > >> wrote: >> > >> >> > >> > I'm for the second option. >> > >> > >> > >> >> > > >> > > >> > > -- >> > > With regards, >> > > Aleksandr Polovtcev >> > > >> > >> > >> > -- >> > >> > Best regards, >> > Ivan Pavlukhin >> > >> >> >> -- >> With regards, >> Aleksandr Polovtcev >> > -- Best regards, Ivan Pavlukhin
Re: [DISCUSSION] @Nullable/@NotNull annotation usage in Ignite 3
Hi, While option #2 looks very appealing it seems not bulletproof reliable, someone can occasionally miss @Nullable annotation. Option #3 seems more practical too me, as missed @NotNull annotations cannot do much harm. Also I am thinking about using nullable parameters in general. Perhaps we can insist on not using nulls as much as possible. With such requirement in guidelines option #2 becomes practical. 2021-12-17 14:47 GMT+03:00, Alexander Polovtcev : > Maksim, thank you for the suggestion. > > I've never used NullAway, but after having a quick look I think it might be > an overkill, since it is a plugin for the ErrorProne, which is a separate > tool. I recall some efforts of introducing ErrorProne to Ignite 3 and they > were not successful. But again, I don't have much experience in that > regard. I wonder if IDEA inspections would be enough, since they are easy > to use during development and AFAIK are already run during the TC builds. > > Regarding Ignite 2, I don't know if it would be viable to add these > annotations to the existing code (in order to have meaningful checks), > since the codebase is so large. But nevertheless we can try to adopt the > same approach there as well. > > On Fri, Dec 17, 2021 at 12:10 PM Maksim Timonin > wrote: > >> Hi! >> >> There is a pretty popular project NullAway [1] that checks code of a >> project in compile-time for possible NPE. AFAIK, it works only with the >> "@Nullable" annotation. I think we can try to add this check to Ignite2 >> and >> 3. >> >> But I wonder, whether smbd already tried to introduce this check? If not, >> I >> can try, WDYT? >> >> [1] https://github.com/uber/NullAway >> >> On Fri, Dec 17, 2021 at 9:04 AM ткаленко кирилл >> wrote: >> >> > I'm for the second option. >> > >> > > > -- > With regards, > Aleksandr Polovtcev > -- Best regards, Ivan Pavlukhin
Re: [DISCUSSION] Reject join of nodes with different character encodings
Do encodings in question somehow influence on actual stored data (bytes)? If so, using an implicit platform encoding sounds quite dangerous. Moving data between servers (or perhaps even rebalancing) can lead to bad consequences. Anyways, IMHO an implicit encoding is not good, but sensible default is quite robust. 2021-12-13 23:07 GMT+03:00, Ivan Daschinsky : > Unpaited surrogates are emoji symbols. One should be completely insane to > use emojis in login. > > пн, 13 дек. 2021 г., 21:30 Mikhail Petrov : > >> Ivan, string with unpaired surrogates symbols are serialized and >> deserialized by java UTF-8 decoder successfully but the result does not >> match the initial string. It may result in that if the user's login >> contains these symbols, it will be distorted after deserialization and >> the user will not be able to log in. I understand that it is a quite >> rare case. >> Anyway, the way to solve this problem was introduced here - >> https://issues.apache.org/jira/browse/IGNITE-3098 >> >> Frankly, it is not the topic I would like to discuss now. The main >> question is - should we restrict the join of nodes with different >> encodings or just fix all places where implicit default encoding is used >> and specify the explicit one as Ivan Daschinsky suggested? >> >> From my point of view, it is better to reject nodes with different >> encodings (especially after Ilya Kasnacheev mentioned that we already >> have a warning "Differing character encodings across cluster may lead >> to erratic behavior"). It will help to avoid "erratic behavior", not >> just warn about it. It is important since the problems related to string >> encoding can occur in different components and the cause of them is not >> always obvious. >> >> WDYT? >> >> On 13.12.2021 20:01, Ivan Pavlukhin wrote: >> >> I guess Nikolay is talking about the problem with UTF-8 in case string >> contains unpaired surrogate symbols >> > Folks, give me a clue why it is a problem? Naively it seems to be a >> > good restriction rather than problem. What problems can it cause in >> > practice? >> > >> > 2021-12-13 16:32 GMT+03:00, Ilya Kasnacheev >> > : >> >> Hello! >> >> >> >> We already have a warning about this, see >> IgniteKernal.checkFileEncoding() >> >> >> >> Regards, >> >> -- >> >> Ilya Kasnacheev >> >> >> >> >> >> пн, 13 дек. 2021 г. в 16:26, Ivan Daschinsky : >> >> >> >>>>> But now multiple components >> >>>>> independently serialize strings for their needs and use default >> >>>>> encoding >> >>>>> for this. >> >>>>> For example DirectByteBufferStreamImplV2#writeString, >> >>>>> MetaStorage#writeRaw and so on >> >>> We should fix all of them. >> >>> >> >>>>> BinaryUtils#utf8BytesToStr >> >>> Lets use this everywhere. >> >>> >> >>> As for me, I'm expecting a way more problem with enforcing rule to >> fail, >> >>> rather than enforcing all components to use UTF-8 >> >>> Some weird cases (surrogate pairs) we can (I strongly believe it is >> OK) >> >>> simply do not consider at all. >> >>> >> >>> пн, 13 дек. 2021 г. в 15:15, Nikolay Izhikov : >> >>> >> >>>>> Does Java String support all unicode characters and particularly >> >>>>> does >> >>> it >> >>>> support more characters than UTF-8 >> >>>> >> >>>> It’s not about Java, it’s about UTF-8 standard. >> >>>> >> >>>> Please, take a look at [1] >> >>>> >> >>>>> In November 2003, UTF-8 was restricted by RFC 3629 to match the >> >>>> constraints of the UTF-16 character encoding: explicitly prohibiting >> >>>> code >> >>>> points corresponding to the high and low surrogate characters >> >>>> removed >> >>> more >> >>>> than 3% of the three-byte sequences, and ending at U+10 removed >> >>>> more >> >>>> than 48% of the four-byte sequences and all five- and six-byte >> >>>> sequences. >> >>>> >> >>>> And [2] >> >>>> >> >>>>> The definition of UTF-8 prohibits encodin
Re: [DISCUSSION] Reject join of nodes with different character encodings
> I guess Nikolay is talking about the problem with UTF-8 in case string > contains unpaired surrogate symbols Folks, give me a clue why it is a problem? Naively it seems to be a good restriction rather than problem. What problems can it cause in practice? 2021-12-13 16:32 GMT+03:00, Ilya Kasnacheev : > Hello! > > We already have a warning about this, see IgniteKernal.checkFileEncoding() > > Regards, > -- > Ilya Kasnacheev > > > пн, 13 дек. 2021 г. в 16:26, Ivan Daschinsky : > >> >> But now multiple components >> >> independently serialize strings for their needs and use default >> >> encoding >> >> for this. >> >> For example DirectByteBufferStreamImplV2#writeString, >> >> MetaStorage#writeRaw and so on >> We should fix all of them. >> >> >> BinaryUtils#utf8BytesToStr >> Lets use this everywhere. >> >> As for me, I'm expecting a way more problem with enforcing rule to fail, >> rather than enforcing all components to use UTF-8 >> Some weird cases (surrogate pairs) we can (I strongly believe it is OK) >> simply do not consider at all. >> >> пн, 13 дек. 2021 г. в 15:15, Nikolay Izhikov : >> >> > > Does Java String support all unicode characters and particularly does >> it >> > support more characters than UTF-8 >> > >> > It’s not about Java, it’s about UTF-8 standard. >> > >> > Please, take a look at [1] >> > >> > > In November 2003, UTF-8 was restricted by RFC 3629 to match the >> > constraints of the UTF-16 character encoding: explicitly prohibiting >> > code >> > points corresponding to the high and low surrogate characters removed >> more >> > than 3% of the three-byte sequences, and ending at U+10 removed >> > more >> > than 48% of the four-byte sequences and all five- and six-byte >> > sequences. >> > >> > And [2] >> > >> > > The definition of UTF-8 prohibits encoding character numbers between >> > U+D800 and U+DFFF, which are reserved for use with the UTF-16 encoding >> form >> > (as surrogate pairs) and do not directly represent characters. >> > >> > Actually, we already has some modes to support this restriction of >> > UTF-8. >> > Please, take a look at BinaryUtils#utf8BytesToStr [3] >> > >> > >> > [1] https://en.wikipedia.org/wiki/UTF-8 >> > [2] https://datatracker.ietf.org/doc/html/rfc3629 >> > [3] >> > >> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/binary/BinaryUtils.java#L2387 >> > >> > > 13 дек. 2021 г., в 13:57, Ivan Pavlukhin >> > написал(а): >> > > >> > >> UTF-8 can’t encode all UNICODE characters. >> > > >> > > Nikolay, could you please elaborate? My understanding is that >> > > encoding >> > > we speak about matters for conversion from byte arrays to strings. >> > > Does Java String support all unicode characters and particularly does >> > > it support more characters than UTF-8 (I am not saying here that java >> > > String uses UTF-8)? >> > > >> > > 2021-12-13 12:56 GMT+03:00, Ivan Daschinsky : >> > >> UTF-8 is already a default encoding in our BinaryObject format. >> > >> So >> > I am >> > >> for unification. >> > >> >> > >> пн, 13 дек. 2021 г. в 12:50, Nikolay Izhikov : >> > >> >> > >>> Hello, Ivan. >> > >>> >> > >>> UTF-8 can’t encode all UNICODE characters. >> > >>> >> > >>>> 13 дек. 2021 г., в 12:49, Ivan Daschinsky >> > >>> написал(а): >> > >>>> >> > >>>> Khm, maybe a better variant is to enforce all strings to be >> > >>>> encoded >> > in >> > >>>> UTF-8? >> > >>>> AFAIK multi OS cluster is a quite common case. >> > >>>> >> > >>>> >> > >>>> пн, 13 дек. 2021 г. в 11:36, Mikhail Petrov > >: >> > >>>> >> > >>>>> Igniters, >> > >>>>> >> > >>>>> Recently we faced the problem that if the cluster consists of >> > >>>>> nodes >> > >>>>> running in the JVM with different encodings, many issues arise. >> > >>>>&g
Re: [DISCUSSION] Reject join of nodes with different character encodings
> UTF-8 can’t encode all UNICODE characters. Nikolay, could you please elaborate? My understanding is that encoding we speak about matters for conversion from byte arrays to strings. Does Java String support all unicode characters and particularly does it support more characters than UTF-8 (I am not saying here that java String uses UTF-8)? 2021-12-13 12:56 GMT+03:00, Ivan Daschinsky : > UTF-8 is already a default encoding in our BinaryObject format. So I am > for unification. > > пн, 13 дек. 2021 г. в 12:50, Nikolay Izhikov : > >> Hello, Ivan. >> >> UTF-8 can’t encode all UNICODE characters. >> >> > 13 дек. 2021 г., в 12:49, Ivan Daschinsky >> написал(а): >> > >> > Khm, maybe a better variant is to enforce all strings to be encoded in >> > UTF-8? >> > AFAIK multi OS cluster is a quite common case. >> > >> > >> > пн, 13 дек. 2021 г. в 11:36, Mikhail Petrov : >> > >> >> Igniters, >> >> >> >> Recently we faced the problem that if the cluster consists of nodes >> >> running in the JVM with different encodings, many issues arise. >> >> The root cause of the mentioned issues is components that use >> >> `String#getBytes()` and `new String()`, which relies on >> >> the >> >> system default encoding. Thus, if a string is deserialized on a node >> >> with a different encoding from the one that serialized it, the >> >> deserialized string can be different from the original one. >> >> >> >> For example: >> >> >> >> Serialization/deserialization of string in communication messages may >> >> be >> >> broken for some strings on nodes running in a JVM with a different >> >> encoding as DirectByteBufferStreamImplV2 uses String#getBytes() to >> >> serialize strings - [1] >> >> >> >> Or the IgniteAuthenticationProcessor can compute different security >> >> IDs >> >> for the user on different nodes in this case - [2] >> >> >> >> What do you think, if we solve this problem globally, by rejecting to >> >> join nodes that run on JVMs with different encodings? >> >> >> >> As a result, we will be sure that all cluster nodes have the same >> >> encoding and all related problems will be solved. >> >> >> >> [1] - https://issues.apache.org/jira/browse/IGNITE-16106 >> >> [2] - https://issues.apache.org/jira/browse/IGNITE-16068 >> >> >> >> -- >> >> Mikhail >> >> >> >> >> > >> > -- >> > Sincerely yours, Ivan Daschinskiy >> >> > > -- > Sincerely yours, Ivan Daschinskiy > -- Best regards, Ivan Pavlukhin
Re: [DISCUSSION] Reserve partitions for CacheQueries
Thanks for details! 2021-12-13 11:14 GMT+03:00, Maksim Timonin : > Hi, Ivan! > >> Does IndexQuery has separate codebase? Does it share some code with > ScanQuery > > Yes, IndexQuery mostly shares processing with ScanQuery: > requests/responses, a remote filter. Reducer (merge of query results) on an > initiator node has its own implementation for IndexQuery (MergeSort), but > it's built into existing ScanQuery processing. > > On Mon, Dec 13, 2021 at 11:00 AM Ivan Pavlukhin > wrote: > >> > Then, if there are no objections, the short-term plan is: >> >> Sounds ok for me. >> >> > AFAIR Scan and SQL queries implementations are totally different. Could >> you tell me how Index queries fit there? >> >> I suppose my questions was misleading. Actually I would like to know >> how code is organized today. AFAIR SQL and scan queries has their own >> codebases (messages, merging results from different nodes and etc). >> Does IndexQuery has separate codebase? Does it share some code with >> ScanQuery on a "query" layer (higher than BTree layer)? >> >> 2021-12-10 13:25 GMT+03:00, Maksim Timonin : >> > Hi! >> > >> >> Existing users may already have some logic in place to handle >> > inconsistencies >> > >> > Pavel, I'm not aware of such users but your comment makes sense. So, I' >> OK >> > with adding an option for ScanQuery. Naming of the option is debatable. >> The >> > name "reservePartitions" looks good, but we actually already reserve a >> > partition when it is explicitly specified in ScanQuery. Then, it can be >> > a >> > bit misleading in the case of explicitly setting this param to `false` >> with >> > a specified partition. But we can just mention it in javadocs, that the >> > setting affects only full scan. >> > >> >> AFAIR Scan and SQL queries implementations are totally different. >> >> Could >> > you tell me how Index queries fit there? >> > >> > IndexQuery scans indexes like SQL does with few optimizations of >> traversing >> > BPlusTree, also there are some differences in query processing, also >> > IndexQuery provides sorted results by default. But I don't expect >> > inconsistency in results for SQL / IndexQuery for similar queries. >> > Actually, I should add tests for that and fix failures if any. >> > >> >> In an ideal world it would be great to have only one API >> > >> > I think it's possible to teach SQL to switch to cache queries for some >> > cases, or provide such an opportunity with hints or explicit functions. >> And >> > these queries might be parts of an SQL query plan. Or we can go even >> > deeper, maybe it could be like Apache Spark stages, when we can build >> > our >> > plan with different types of queries, and the same type provides to >> > users the opportunity to run only specific stages: DataFrame.sql() or >> > DataFrame.filter(inline=true/false). >> > >> >> Perhaps IndexQuery should also cover regular entry iteration cases >> > >> > It's possible too. IndexQuery provides an opportunity to scan the PK >> index, >> > then it can start ScanQuery under the hood for some cases. >> > >> > But anyway, to make them run through the single API, we should provide >> the >> > same guarantees. >> > >> > Then, if there are no objections, the short-term plan is: >> > 1. Implement partition reservation for IndexQuery. >> > 2. Add the option for ScanQuery. >> > 3. Make tests that show that IndexQuery / SQL / ScanQuery is >> > replaceable >> > for some types of queries in terms of result consistency. >> > 4. Then discuss again, how we can integrate them together. >> > >> > >> > On Fri, Dec 10, 2021 at 11:41 AM Ivan Pavlukhin >> > wrote: >> > >> >> Actually I brought a point about using SQL queries instead of scan >> >> queries because I worry about inconsistency between different >> >> implementations. AFAIR Scan and SQL queries implementations are >> >> totally different. Could you tell me how Index queries fit there? >> >> >> >> My general ideas are as follows: >> >> 1. In an ideal world it would be great to have only one API (to rule >> >> them all) and implementation to cover all cases (i.e. scan, index, >> >> SQL). Also, I wonder how other vendors tackle this? >> >> 2. New Ind
Re: [DISCUSSION] Reserve partitions for CacheQueries
> Then, if there are no objections, the short-term plan is: Sounds ok for me. > AFAIR Scan and SQL queries implementations are totally different. Could you > tell me how Index queries fit there? I suppose my questions was misleading. Actually I would like to know how code is organized today. AFAIR SQL and scan queries has their own codebases (messages, merging results from different nodes and etc). Does IndexQuery has separate codebase? Does it share some code with ScanQuery on a "query" layer (higher than BTree layer)? 2021-12-10 13:25 GMT+03:00, Maksim Timonin : > Hi! > >> Existing users may already have some logic in place to handle > inconsistencies > > Pavel, I'm not aware of such users but your comment makes sense. So, I' OK > with adding an option for ScanQuery. Naming of the option is debatable. The > name "reservePartitions" looks good, but we actually already reserve a > partition when it is explicitly specified in ScanQuery. Then, it can be a > bit misleading in the case of explicitly setting this param to `false` with > a specified partition. But we can just mention it in javadocs, that the > setting affects only full scan. > >> AFAIR Scan and SQL queries implementations are totally different. Could > you tell me how Index queries fit there? > > IndexQuery scans indexes like SQL does with few optimizations of traversing > BPlusTree, also there are some differences in query processing, also > IndexQuery provides sorted results by default. But I don't expect > inconsistency in results for SQL / IndexQuery for similar queries. > Actually, I should add tests for that and fix failures if any. > >> In an ideal world it would be great to have only one API > > I think it's possible to teach SQL to switch to cache queries for some > cases, or provide such an opportunity with hints or explicit functions. And > these queries might be parts of an SQL query plan. Or we can go even > deeper, maybe it could be like Apache Spark stages, when we can build our > plan with different types of queries, and the same type provides to > users the opportunity to run only specific stages: DataFrame.sql() or > DataFrame.filter(inline=true/false). > >> Perhaps IndexQuery should also cover regular entry iteration cases > > It's possible too. IndexQuery provides an opportunity to scan the PK index, > then it can start ScanQuery under the hood for some cases. > > But anyway, to make them run through the single API, we should provide the > same guarantees. > > Then, if there are no objections, the short-term plan is: > 1. Implement partition reservation for IndexQuery. > 2. Add the option for ScanQuery. > 3. Make tests that show that IndexQuery / SQL / ScanQuery is replaceable > for some types of queries in terms of result consistency. > 4. Then discuss again, how we can integrate them together. > > > On Fri, Dec 10, 2021 at 11:41 AM Ivan Pavlukhin > wrote: > >> Actually I brought a point about using SQL queries instead of scan >> queries because I worry about inconsistency between different >> implementations. AFAIR Scan and SQL queries implementations are >> totally different. Could you tell me how Index queries fit there? >> >> My general ideas are as follows: >> 1. In an ideal world it would be great to have only one API (to rule >> them all) and implementation to cover all cases (i.e. scan, index, >> SQL). Also, I wonder how other vendors tackle this? >> 2. New IndexQuery might not mimic ScanQuery behavior but instead have >> a correct/expected one (including partition reservation). Perhaps >> IndexQuery should also cover regular entry iteration (scan) cases and >> become a new ground for generalized scan/index queries. >> >> 2021-12-09 17:08 GMT+03:00, Pavel Tupitsyn : >> > IndexQuery is experimental, so we can indeed make it reserve partitions >> by >> > default. No objections here. >> > >> > But with ScanQuery, I think it's better to add a "reservePartitions" >> > property so that default behavior is not affected. >> > Existing users may already have some logic in place to handle >> > inconsistencies. So if we enable reservation by default, >> > they'll get an unnecessary performance degradation. >> > >> > On Thu, Dec 9, 2021 at 4:42 PM Maksim Timonin >> > wrote: >> > >> >> Hi, Ivan, Pavel! Thanks for your responses. >> >> >> >> > But is there a real need/benefit from using scan queries over >> primitive >> >> SQL queries today >> >> >> >> In addition to Pavel's response, I'd like to add about IndexQuery. >> >> Thi
Re: [DISCUSSION] Reserve partitions for CacheQueries
Actually I brought a point about using SQL queries instead of scan queries because I worry about inconsistency between different implementations. AFAIR Scan and SQL queries implementations are totally different. Could you tell me how Index queries fit there? My general ideas are as follows: 1. In an ideal world it would be great to have only one API (to rule them all) and implementation to cover all cases (i.e. scan, index, SQL). Also, I wonder how other vendors tackle this? 2. New IndexQuery might not mimic ScanQuery behavior but instead have a correct/expected one (including partition reservation). Perhaps IndexQuery should also cover regular entry iteration (scan) cases and become a new ground for generalized scan/index queries. 2021-12-09 17:08 GMT+03:00, Pavel Tupitsyn : > IndexQuery is experimental, so we can indeed make it reserve partitions by > default. No objections here. > > But with ScanQuery, I think it's better to add a "reservePartitions" > property so that default behavior is not affected. > Existing users may already have some logic in place to handle > inconsistencies. So if we enable reservation by default, > they'll get an unnecessary performance degradation. > > On Thu, Dec 9, 2021 at 4:42 PM Maksim Timonin > wrote: > >> Hi, Ivan, Pavel! Thanks for your responses. >> >> > But is there a real need/benefit from using scan queries over primitive >> SQL queries today >> >> In addition to Pavel's response, I'd like to add about IndexQuery. This >> query is in experimental status, but it shows great performance and beats >> SQL for *some* type of queries. For IndexQuery we can leverage on >> understanding that we use only index and apply some optimizations: skip >> boundaries check (h2 Select.isConditionMet), apply filtering by inline IO >> instead of extracting fields from BinaryObjects, also we skip some H2 >> related stuff. Also it's not implemented yet, but for IndexQuery we can >> access data pages sequentially instead of randomly (by iterating with >> IndexRow cursor by batches), and I expect it may make performance better >> in >> some highload cases. >> >> > it is desirable that successful (but may be strange) scenarios continue >> to work after fix (and not fail e.g. due to some introduced exception) >> >> > Partition reservation should be opt-in if we decide to proceed. >> >> With the approach I proposed we can reach it. Without specified query >> timeout (and cache queries don't have a public API for setting timeout), >> we >> can retry our attempts to reserve partitions while we do not succeed. So, >> users may get some increase in query time execution under unstable >> topology. But in exchange users will get more stable results, and avoid >> exceptions like in the ticket I showed [1]. >> >> We can implement this logic for IndexQuery only (as it's only >> experimental >> since 2.12), and extend it on ScanQuery later after stabilizing with >> IndexQuery. WDYT? >> >> >> [1] https://issues.apache.org/jira/browse/IGNITE-12591 >> >> On Thu, Dec 9, 2021 at 10:39 AM Pavel Tupitsyn >> wrote: >> >> > Agree with Ivan regarding compatibility. >> > Partition reservation should be opt-in if we decide to proceed. >> > >> > > is there a real need/benefit from using >> > > scan queries over primitive SQL queries today >> > >> > SQL requires additional configuration (QueryEntity) and involves memory >> and >> > CPU overhead to maintain the indexes. >> > Especially if the query is used rarely (maybe some maintenance tasks), >> scan >> > is a better option. >> > >> > On Thu, Dec 9, 2021 at 10:12 AM Ivan Pavlukhin >> > wrote: >> > >> > > Hi Maksim, >> > > >> > > Thank you for looking into this. While fixing wrong/surprising >> > > behavior is very important, I also have some concerns, let's say, >> > > from >> > > different angle of view: >> > > 1. From a first glance it seems that similar behavior of scan and SQL >> > > queries is a good idea. But is there a real need/benefit from using >> > > scan queries over primitive SQL queries today (e.g. SELECT * FROM >> > > table1)? >> > > 2. Also we need to carefully think about compatibility. It is >> > > desirable that successful (but may be strange) scenarios continue to >> > > work after fix (and not fail e.g. due to some introduced exception). >> > > >> > > Regarding partition reservation for long-time open cursors
Re: [DISCUSSION] Reserve partitions for CacheQueries
Hi Maksim, Thank you for looking into this. While fixing wrong/surprising behavior is very important, I also have some concerns, let's say, from different angle of view: 1. From a first glance it seems that similar behavior of scan and SQL queries is a good idea. But is there a real need/benefit from using scan queries over primitive SQL queries today (e.g. SELECT * FROM table1)? 2. Also we need to carefully think about compatibility. It is desirable that successful (but may be strange) scenarios continue to work after fix (and not fail e.g. due to some introduced exception). Regarding partition reservation for long-time open cursors I guess some ideas might be found in LAZY mode for SQL queries [1]. [1] https://issues.apache.org/jira/browse/IGNITE-9171 2021-12-08 20:11 GMT+03:00, Maksim Timonin : > Hi, Igniters! > > There is a known issue that ScanQuery on unstable topology returns > incorrect results: duplicates data [1] or fails with an exception [2]. The > reason is ScanQuery doesn't reserve partitions. > > IndexQuery shares the same query processing as ScanQuery, and then it is > also affected by unstable topology. I want to fix it for IndexQuery. > IndexQuery should provide the same stability as SQL queries do - no > occasional failures or data duplication. > > I dived into the SQL query processing and found that we can unify logic for > SQL queries and cache queries (ScanQuery, IndexQuery, TextQuery): > > 1. Currently, cache queries use `GridCacheQueryAdapter#nodes` to find nodes > to run a query. From the other side, SQL processing uses for the same goal > `ReducePartitionMapper#nodesForPartitions`. It looks like those methods > have some slight differences, but in the common, they do the same. So, I > propose to make a single method for both cases. > > 2. SQL processing uses `PartitionReservationManager` for reserve partitions > on the map side. I propose to move it to the ignite-core module and start > using it for the cache queries. > > 3. Implement retries for the cache queries in case we failed to reserve > partitions on the map side. > > Currently, I see a downside of reserving partitions for cache queries: > cache queries are lazy. And the time of partition reservation depends on a > user's application code (how fast a cursor is iterated and closed). AFAIU, > it's not very good to have a partition in reserve too long. Please, correct > me if I'm wrong here. > > But from the other side, Ignite reserves partitions for ScanQuery when a > partition has been specified as a ScanQuery parameter, and Ignite reserves > partitions for SQL with the flag lazy=true. Also: > - IndexQuery: I expect simple queries that will return a relatively small > amount of data. Then partitions wouldn't be reserved too much time. > - the same is for TextQuery - it returns a limited amount of data (due to > the Lucene logic). > - full ScanQuery it's not in use much, AFAIK. So, it's by default a pretty > heavy operation. > > So, I think it's safe to reserve partitions in any case. But there could be > an alternative optimistic approach, smth like that: > 1. Check that topology is stable and waiting while it's not stabilized. > 2. Run a query on a stable cluster. > 3. In cases when a cluster becomes unstable during query execution - try to > reserve partitions at runtime (query should be aware of topology changes) > and fail in case of reservation failure (if a user already fetched some > data). > > I don't like the idea of this optimistic approach because in case a user > got some data, we don't have a better solution than to fail a query in case > of cluster instability and reservation failure. > > Igniters, WDYT? > > [1] https://issues.apache.org/jira/browse/IGNITE-12591 > [2] https://issues.apache.org/jira/browse/IGNITE-16031 > -- Best regards, Ivan Pavlukhin
Re: [DISCUSSION][IEP-80] Breaking changes in Ignite-2.x
t; > > >> >>> > > > >> [1] https://issues.apache.org/jira/browse/IGNITE-12662 >>> > > > >> [2] https://issues.apache.org/jira/browse/IGNITE-14613 >>> > > > >> >>> > > > >> On Fri, 15 Oct 2021 at 15:46, Anton Vinogradov >>> > wrote: >>> > > > >>> >>> > > > >>> +1 >>> > > > >>> >>> > > > >>> On Fri, Oct 15, 2021 at 3:41 PM Nikita Amelchev < >>> > namelc...@apache.org> >>> > > > >>> wrote: >>> > > > >>> >>> > > > >>>> +1 for deprecation in the 2.12 release >>> > > > >>>> >>> > > > >>>> пт, 15 окт. 2021 г. в 15:35, Nikolay Izhikov < >>> nizhi...@apache.org >>> > >: >>> > > > >>>>> >>> > > > >>>>> Hello, Igniters. >>> > > > >>>>> >>> > > > >>>>> I’ve prepared IEP-80 [1] to track breaking changes that >>> > > > >>>>> should >>> > be done >>> > > > >>>> in Ignite 2.x. >>> > > > >>>>> >>> > > > >>>>> We agreed on the following algorithm to deal with breaking >>> > changes: >>> > > > >>>>> >>> > > > >>>>> - Ignite 2.[x] - deprecate the API + notification of the >>> users. >>> > > > >>>>> - Ignite 2.[x+1] - API removal. >>> > > > >>>>> >>> > > > >>>>> >>> > > > >>>>> I propose to start these improvements with the d >>> (depuration & >>> > > > >>>> removal) of the following features: >>> > > > >>>>> >>> > > > >>>>> - LOCAL caches >>> > > > >>>>> - MVCC caches >>> > > > >>>>> - legacy service grid implementation >>> > > > >>>>> - legacy JMX metrics beans. >>> > > > >>>>> >>> > > > >>>>> Deprecation in 2.12 >>> > > > >>>>> Removal in 2.13 >>> > > > >>>>> >>> > > > >>>>> Please, share your opinion. >>> > > > >>>>> >>> > > > >>>>> [1] >>> > > > >>>> >>> > >>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=191334475 >>> > > > >>>> >>> > > > >>>> >>> > > > >>>> >>> > > > >>>> -- >>> > > > >>>> Best wishes, >>> > > > >>>> Amelchev Nikita >>> > > > >>>> >>> > > > > >>> > > > >>> > >>> >>> >>> -- >>> Best Regards, >>> Vyacheslav D. >>> >> >> >> -- >> Sincerely yours, Ivan Daschinskiy >> > > > -- > Sincerely yours, Ivan Daschinskiy > -- Best regards, Ivan Pavlukhin
Re: Stop sending IGNITE Created e-mails to dev@
A last one I received was IGNITE-15872 as well. 2021-11-11 22:43 GMT+03:00, Ilya Kasnacheev : > Hello! > > For some reason, looks like it has stopped working. I did not get any > notification for issues since IGNITE-15872. > > Is it just me? The settings in the JIRA seem alright. > > Regards, > > сб, 1 мая 2021 г. в 12:49, Mikhail Petrov : > >> It works. Thanks a lot! >> >> Sent from my iPhone >> >> On 30 Apr 2021, at 15:20, Ilya Kasnacheev >> wrote: >> >> >> Hello! >> >> I have added you to this role >> >> Regards, >> -- >> Ilya Kasnacheev >> >> >> пт, 30 апр. 2021 г. в 11:55, Mikhail Petrov : >> >>> Hi, Ilya, could you please add me to "Contributors 1" role? Or can I do >>> it myself somehow? >>> >>> On 29.04.2021 18:59, Ilya Kasnacheev wrote: >>> >>> Hello! >>> >>> Pavel, Yurii, I have added you both to the role. >>> >>> Regards, >>> >>> > -- Best regards, Ivan Pavlukhin
Re: Text Queries Support
Hi Maximiliano,
Thank you for pointing this out, rather interesting. Have you tried to
communicate with a hawkore team? I doubt that anyone in Community
knows implementation details of hawkore additions.
2021-10-22 19:58 GMT+03:00, Maximiliano Gazquez :
> Hello everyone!
>
> I wanted to add this to the discussion.
> I've found this project https://github.com/hawkore/ignite-hk which promises
> to solve most of the issues that are being discussed here like pagination,
> sorting and most important, persisting the lucene index.
>
> It does stuff like this to create indexes:
>
> CREATE INDEX PERSON_LUCENE_IDX ON "PUBLIC".PERSON(LUCENE)
> FULLTEXT '{
> ''refresh_seconds'':''60'',
> ''directory_path'':,
> ''ram_buffer_mb'':''10'',
> ''max_cached_mb'':''-1'',
> ''partitioner'':''{"type":"token","partitions":10}'',
> ''optimizer_enabled'':''true'',
> ''optimizer_schedule'':''0 1 * * *'',
> ''version'':''0'',
> ''schema'':''{
> "default_analyzer":"english",
>
> "analyzers":{"my_custom_analyzer":{"type":"snowball","language":"Spanish","stopwords":"el,la,lo,loas,las,a,ante,bajo,cabe,con,contra"}},
> "fields":{
>
> "duration":{"type":"date_range","from":"start_date","to":"stop_date","validated":false,"pattern":"/MM/dd"},
>
> "place":{"type":"geo_point","latitude":"latitude","longitude":"longitude"},
> "date":{"type":"date","validated":true,"pattern":"/MM/dd"},
> "number":{"type":"integer","validated":false,"boost":1.0},
> "gender":{"type":"string","validated":true,"case_sensitive":true},
> "bool":{"type":"boolean","validated":false},
>
> "phrase":{"type":"text","validated":false,"analyzer":"my_custom_analyzer"},
> "name":{"type":"string","validated":false,"case_sensitive":true},
> "animal":{"type":"string","validated":false,"case_sensitive":true},
> "age":{"type":"integer","validated":false,"boost":1.0},
> "food":{"type":"string","validated":false,"case_sensitive":true}
> }
> }''
> }';
>
> And this to use that lucene index from inside SQL:
>
> SELECT * FROM "test".user
> WHERE lucene = '{ query : {
> type : "boolean",
> must : [{type : "wildcard", field : "name",
> value : "J*"},
> {type : "wildcard", field : "food",
> value : "tu*"}]}}';
>
> More examples here
> https://github.com/hawkore/examples-apache-ignite-extensions/tree/master/examples-advanced-ignite-indexing
>
> I don't have anything to do with that company but it would be great to know
> how they implemented this stuff.
>
>
> On Mon, Aug 9, 2021 at 3:00 AM Ivan Pavlukhin wrote:
>
>> Hi Atri,
>>
>> Sorry for a late answer.
>>
>> > I didn't quite understand. Are you proposing that Ignite should not
>> > have
>> FTS capabilities?
>>
>> It seems an option to me. IMHO it is better to have no FTS instead of
>> something like current Ignite TextQueries.
>>
>> 2021-08-03 12:45 GMT+03:00, Atri Sharma :
>> > Hi Ivan,
>> >
>> > I didn't quite understand. Are you proposing that Ignite should not
>> > have FTS capabilities?
>> >
>> > Atri
>> >
>> > On Tue, Aug 3, 2021 at 2:57 PM Ivan Pavlukhin
>> wrote:
>> >>
>> >> Hi Atri,
>> >>
>> >> My main concern is non-maleficence. Every task has several solutions,
>> >> e.g. straightforward ones:
>> >> 1. Do not implement FTS.
>> >> 2. Create own implementation.
>> >>
>> >> Some of the strongest ones live without FTS [1].
>> >>
>> >> [1] https://github.com/cockroachdb/cockroach/issues/7821
>> >>
>> >> 2021-08-02 11:33 GMT+03:00, Atri Sharma :
>> >> > Hi Ivan,
>> >> >
>> >> >
Re: [VOTE] Create separate Jira project and Confluence space for Ignite 3
0 from me. > Voting ends in 72 hours, at 5pm PDT on October 7: I suggest to prolong voting at least till the end of the week. I see no reason to have such limited timeframe. Whole week including a weekend will allow more people to cast votes. 2021-10-05 10:06 GMT+03:00, ткаленко кирилл : > +1 > > 05.10.2021, 02:52, "Valentin Kulichenko" : >> Hello Community, >> >> As discussed in [1], I would like to propose the creation of a separate >> Jira project and Confluence space for Ignite 3. >> >> Ignite 2 and Ignite 3 are developed in parallel in separate repos, so we >> need a clear separation in other tools as well - this will help to >> streamline the development process. Please refer to the discussion for >> more >> details. >> >> [1] >> https://lists.apache.org/thread.html/rdcad3fc64b9f3a848c93089baae2bee1124a97869a94f4a04dd80fdf%40%3Cdev.ignite.apache.org%3E >> >> Voting options: >> >> - +1 - Agree with the suggestion >> - 0 - Don't care much about the suggestion >> - -1 - Disagree with the suggestion >> >> This is a majority vote. >> >> Voting ends in 72 hours, at 5pm PDT on October 7: >> https://www.timeanddate.com/counters/fullscreen.html?mode=a=20211007T17=2021=10=7=17=0=0=224 >> >> -Val > -- Best regards, Ivan Pavlukhin
Re: [DISCUSSION] Separate Jira project and Confluence space for Ignite 3
Sorry, If I missed something in the thread but in case of a separate JIRA project how are users supposed to create e.g. bug tickets? How can we make sure that users will not use a wrong JIRA project often? 2021-10-05 2:50 GMT+03:00, Valentin Kulichenko : > Ivan, > > I'm not pushing, I'm trying to apply the lazy consensus. It soon will be a > whole month since I've started the discussion - more than enough to express > concerns and provide alternative suggestions. Please keep in mind that we > are trying to address a very specific technical problem that influences the > development. "Do nothing" is not really an option here. > > Either way, I will put the initial suggestion for the vote. > > -Val > > On Thu, Sep 30, 2021 at 12:24 AM Ivan Pavlukhin > wrote: > >> Val, >> >> > Let's discuss this until the end of the week. If there is no clear >> picture on option #2 by then, I suggest we go with #1. >> >> For a moment I felt that the proposal is pushed. Let's not do so. The >> subject is very important, years impact I suppose. And the best way >> here is to reach absolute consensus. Without tight timelines so far. >> In case if we fail with consensus we can arrange formal voting. >> >> 2021-09-29 14:34 GMT+03:00, Petr Ivanov : >> > I am watching how Apache Ignite does evolve for over a 3 years already >> and >> > see that such hidden (almost no Open Source Community points could be >> > achieved for refactoring and addressing something that is not directly >> > project's source executable code) issues drown under constant pressure >> > of >> > new features and releases. >> > >> > I have never created issues for Maven build refactoring (for instanced) >> > because I understand that 1) it is almost impossible for current tech >> debt >> > already accumulated and 2) to won't be welcomed by community because of >> > indirect relationship to main project's goals. >> > Considering other parts, please, note [1], [2], [3], [4], [5], [6], >> > [7], >> [8] >> > and many many more issues that have no separate ticket. >> > >> > My point — such technical debt is overwhelming and will be never ever >> > approached. >> > That is one of the reasons why Ignite 3 being built from scratch, >> > having >> in >> > mind all mistakes we've already made and lots of errors we will never >> > do >> > just because there would be no legacy basic for that. >> > >> > >> > >> > [1] https://issues.apache.org/jira/browse/IGNITE-7190 >> > [2] https://issues.apache.org/jira/browse/IGNITE-7326 >> > [3] https://issues.apache.org/jira/browse/IGNITE-7672 >> > [4] https://issues.apache.org/jira/browse/IGNITE-8496 >> > [5] https://issues.apache.org/jira/browse/IGNITE-9866 >> > [6] https://issues.apache.org/jira/browse/IGNITE-10600 >> > [7] https://issues.apache.org/jira/browse/IGNITE-10683 >> > [8] https://issues.apache.org/jira/browse/IGNITE-10696 >> > >> > >> > >> > >> > >> >> On 29 Sep 2021, at 14:14, Nikolay Izhikov wrote: >> >> >> >>> — issues related to Maven build? possible Gradle upgrade? >> >> >> >> I’m not aware of the issues. >> >> Can you, please, send a tickets or description of existing issues? >> >> Anyway, it seems change of build tool can be done at any time we want >> >> >> >>> — issues related to run scripts? >> >>> — issues related to release and delivery processes and scripts? >> >> >> >> I’m not aware of those too. >> >> Can you point to then, please? >> >> >> >>> Are they going to be addressed during Apache Ignite evolution too? >> >> >> >> Yes, from my point of view. >> >> >> >>> 29 сент. 2021 г., в 14:03, Petr Ivanov >> написал(а): >> >>> >> >>> And what about: >> >>> — issues related to Maven build? possible Gradle upgrade? >> >>> — issues related to run scripts? >> >>> — issues related to release and delivery processes and scripts? >> >>> >> >>> Are they going to be addressed during Apache Ignite evolution too? >> >>> >> >>>> On 29 Sep 2021, at 13:47, Nikolay Izhikov >> wrote: >> >>>> >> >>>>> Does you vision of evolutionary improvement involve technical debt >> >>>>> addressing >> &g
Re: [DISCUSSION] Separate Jira project and Confluence space for Ignite 3
cheev >>>>>>> написал(а): >>>>>>> >>>>>>> Hello! >>>>>>> >>>>>>> If we go the second route, we can call the field "Generation". >>>>>>> >>>>>>> Generation: Ignite 2.x >>>>>>> Generation: Ignite 3 >>>>>>> >>>>>>> (no new tickets should ever be filed for Ignite 1.x but if they are, >>>>>>> they >>>>>>> should go to the first Generation) >>>>>>> >>>>>>> Regards. >>>>>>> -- >>>>>>> Ilya Kasnacheev >>>>>>> >>>>>>> >>>>>>> ср, 29 сент. 2021 г. в 00:33, Valentin Kulichenko < >>>>>>> valentin.kuliche...@gmail.com>: >>>>>>> >>>>>>>> As for the original topic, we need to come to a solution. Let me >>>>>>>> summarize >>>>>>>> what we've discussed so far. >>>>>>>> >>>>>>>> -PROBLEM- >>>>>>>> >>>>>>>> Ignite 3 is the next major version of Apache Ignite. It targets the >>>>>>>> same >>>>>>>> use cases and provides a similar set of features as Ignite 2. At the >>>>>>>> same >>>>>>>> time, Ignite 2 and Ignite 3 are *technically* separate projects. >>>>>>>> They are >>>>>>>> developed in different repositories (and therefore are based on >>>>>>>> different >>>>>>>> codebases) and implement different internal architectures. To >>>>>>>> achieve a >>>>>>>> more efficient development process, we need to create a clear >>>>>>>> separation >>>>>>>> between 2.x and 3.x within *development resources* (Jira and >>>>>>>> Confluence). >>>>>>>> >>>>>>>> -POSSIBLE SOLUTIONS- >>>>>>>> >>>>>>>> 1. Create a separate Jira project and Confluence space for Ignite 3 >>>>>>>> (initial suggestion). >>>>>>>> 2. Add a *mandatory* field in Jira to identify whether a ticket >>>>>>>> belongs to >>>>>>>> 2.x or 3.x. >>>>>>>> >>>>>>>> If we go with #2, there are still several things to figure out: >>>>>>>> >>>>>>>> - What is the name of this field? It needs to be intuitive to anyone >>>>>>>> who >>>>>>>> joins the community. >>>>>>>> - We need to make sure that Ignite 3 tickets are not mapped to 2.x >>>>>>>> versions, and vice versa. Can we restrict this in Jira? Or we will >>>>>>>> have >>>>>>>> to >>>>>>>> monitor this manually? >>>>>>>> - What do we do with Confluence? >>>>>>>> >>>>>>>> Nikolay, Ilya, Denis, and others who opposed the initial suggestion: >>>>>>>> if you >>>>>>>> still prefer the second option, could you please address the points >>>>>>>> above? >>>>>>>> I don't think it can be treated as an actual suggestion until we >>>>>>>> cover >>>>>>>> these details. >>>>>>>> >>>>>>>> Let's discuss this until the end of the week. If there is no clear >>>>>>>> picture >>>>>>>> on option #2 by then, I suggest we go with #1. >>>>>>>> >>>>>>>> -Val >>>>>>>> >>>>>>>> On Tue, Sep 28, 2021 at 11:22 PM Valentin Kulichenko < >>>>>>>> valentin.kuliche...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Folks, >>>>>>>>> >>>>>>>>> Versioning is a separate topic. We agreed on the current scheme in >>>>>>>>> March >>>>>>>>> [1]. If someone thinks we need to change it, please create a new >>>>>>>>> thread >>>>>>>> and >>>>>>>>> present your suggestion
Re: [DISCUSSION] Separate Jira project and Confluence space for Ignite 3
I mean that Ignite 2.x will continue to use old scheme and Ignite 3 will be e.g. Ignite 21.1 and so on. 2021-09-27 14:57 GMT+03:00, Petr Ivanov : > How will not they clash if version is based only on date? > >> On 27 Sep 2021, at 14:33, Ivan Pavlukhin wrote: >> >> Today it is quite common to use calendar-based versioning scheme, e.g. >> [1]. We can consider it for Ignite 3. Luckily versions will not clash. >> >> [1] https://www.cockroachlabs.com/docs/releases/index.html >> >> 2021-09-27 10:49 GMT+03:00, Petr Ivanov : >>> That name will definitely confuse Jira users. >>> >>> Let's stick to basic devision by 2.x and 3.x — it seems most intuitive >>> and >>> has lots of examples inside ASF, look at the Tomcat for instance. >>> >>>> On 25 Sep 2021, at 21:05, Saikat Maitra >>>> wrote: >>>> >>>> Hi, >>>> >>>> I like the major version update like Ignite 3.0 but if we were to come >>>> up >>>> with a name my other suggestion would be >>>> >>>> Ignite-kernel >>>> >>>> kernel - for the central or most important part of something >>>> >>>> Also taken references from Compute kernel - a routine compiled for high >>>> throughput accelerators >>>> >>>> https://en.wikipedia.org/wiki/Compute_kernel >>>> >>>> Regards, >>>> Saikat >>>> >>>> >>>> On Sat, Sep 25, 2021 at 3:12 AM Valentin Kulichenko < >>>> valentin.kuliche...@gmail.com> wrote: >>>> >>>>> Kafka and Spark didn't split codebases (at least to my knowledge). >>>>> Separating codebases was the fundamental step, everything else is a >>>>> technicality. >>>>> >>>>> Having said that, I will be OK with your suggestion as I don't really >>>>> see >>>>> a >>>>> difference, although I'm not sure we will be able to come up with a >>>>> name >>>>> that is more intuitive than a separate project :) >>>>> >>>>> Let's see what others think. >>>>> >>>>> -Val >>>>> >>>>> On Sat, Sep 25, 2021 at 12:23 AM Denis Magda >>>>> wrote: >>>>> >>>>>> Moving the discussion back to the dev list. >>>>>> >>>>>> Val, Andrey, for that purpose we can ask INFRA to create a >>>>>> special mandatory field such as "Architecture" with two predefined >>>>> values - >>>>>> "Ignite 2.x" and "Ignite 3.x". Come up with a better name, it needs >>>>>> to >>>>>> be >>>>>> intuitive enough even for users who submit issues. What disturbs me >>>>>> is >>>>> that >>>>>> neither Kafka nor Spark have a different project for the recently >>>>> released >>>>>> versions 3. A different GitHub project is not that disturbing. >>>>>> >>>>>> - >>>>>> Denis >>>>>> >>>>>> >>>>>> On Fri, Sep 24, 2021 at 4:09 AM Valentin Kulichenko < >>>>>> valentin.kuliche...@gmail.com> wrote: >>>>>> >>>>>>> Denis, >>>>>>> >>>>>>> From a purely technical perspective, these are indeed two separate >>>>>>> projects, because they are based on different codebases. The split >>>>> you're >>>>>>> talking about happened a year ago, when we created the repo for >>>>>>> Ignite >>>>> 3. >>>>>>> This significantly differs from the 1.x->2.x transition, as these >>>>>>> two >>>>>>> shared the codebase. >>>>>>> >>>>>>> For the same reason, a bug filed for 2.x can't be just transitioned >>>>>>> to >>>>>>> 3.x. It will either not exist in 3.x in the first place, or will >>>>> require >>>>>> a >>>>>>> completely different fix, which will mean two different tickets. >>>>>>> >>>>>>> That said, I still believe that Ignite 2 and Ignite 3 are just >>>>> different >>>>>>> versions of the same product, because, as you correctly mentioned, >>&
Re: [DISCUSSION] Separate Jira project and Confluence space for Ignite 3
elease cycle. >>>>>>>>>> *** you are here *** >>>>>>>>>> - jira >>>>>>>>>> - confluence >>>>>>>>>> >>>>>>>>>> Should we accept the fact that thing we calling as "Ignite3" is >>>> just >>>>>>> another project? >>>>>>>>>> Can you, please, share your vision on how Ignite and Ignite3 >>> should >>>>>>> coexists? >>>>>>>>>> >>>>>>>>>> вт, 21 сент. 2021 г. в 17:13, Dmitry Pavlov >>> : >>>>>>>>>>> >>>>>>>>>>> Ok, if nobody minds, I'll create spaces a bit later. >>>>>>>>>>> >>>>>>>>>>> I hope it is not too urgent. >>>>>>>>>>> >>>>>>>>>>> Sincerely, >>>>>>>>>>> Dmitriy Pavlov >>>>>>>>>>> >>>>>>>>>>> On 2021/09/21 10:37:42, Valentin Kulichenko < >>>>>>> valentin.kuliche...@gmail.com> wrote: >>>>>>>>>>>> Hi Dmitry, >>>>>>>>>>>> >>>>>>>>>>>> According to Infra, this has to be done through >>>>>>> http://selfserve.apache.org/, >>>>>>>>>>>> but only PMC chairs have access. >>>>>>>>>>>> >>>>>>>>>>>> Could you please assist with the creation of the Jira project >>>> and >>>>>>>>>>>> Confluence space? >>>>>>>>>>>> >>>>>>>>>>>> -Val >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Sep 21, 2021 at 10:46 AM Valentin Kulichenko < >>>>>>>>>>>> valentin.kuliche...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Infra requests created: >>>>>>>>>>>>> >>>>>>>>>>>>> - https://issues.apache.org/jira/browse/INFRA-22349 >>>>>>>>>>>>> - https://issues.apache.org/jira/browse/INFRA-22350 >>>>>>>>>>>>> >>>>>>>>>>>>> -Val >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Sep 20, 2021 at 10:50 AM Petr Ivanov < >>>>>>> mr.wei...@gmail.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> +1 >>>>>>>>>>>>>> >>>>>>>>>>>>>> Since we've agreed that there are two projects (that are >>>>>>> Ignite2 and >>>>>>>>>>>>>> Ignite3), separate development environments seem to be >>>> logical >>>>>>> and natural >>>>>>>>>>>>>> course of things. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 18 Sep 2021, at 12:42, Alexander Polovtcev < >>>>>>> alexpolovt...@gmail.com> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> +1 >>>>>>>>>>>>>>> This is a welcome proposal, because we already have some >>>>>>> pending Ignite >>>>>>>>>>>>>> 3 >>>>>>>>>>>>>>> specific documents, and it is not clear where to put them >>>> at >>>>>>> the moment. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sat, Sep 18, 2021 at 4:22 AM Valentin Kulichenko < >>>>>>>>>>>>>>> valentin.kuliche...@gmail.com> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Igniters, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I think it's clear to all of us that Ignite 2.x and 3.x >>>>>>> will coexist >>>>>>>>>>>>>> for a >>>>>>>>>>>>>>>> while. They are developed in separate Git repos, but we >>>>>>> still >>>>>>>>>>>>>> accumulate >>>>>>>>>>>>>>>> the tickets for both versions in the same Jira project, >>>>>>> which seems to >>>>>>>>>>>>>>>> complicate the ticket management. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> For example, we use the "ignite-3" label for 3.x >>> tickets, >>>>>>> but this >>>>>>>>>>>>>> approach >>>>>>>>>>>>>>>> is fragile. If someone forgets to add the label to a new >>>>>>> ticket, it's >>>>>>>>>>>>>>>> likely to be lost. We need a better separation. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> All the above is true for Wiki as well - we use a single >>>>>>> Confluence >>>>>>>>>>>>>> space. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I suggest creating a new Jira project and a new >>> Confluence >>>>>>> space for >>>>>>>>>>>>>> Ignite >>>>>>>>>>>>>>>> 3 and moving all the relevant tickets and pages there. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Any thoughts or objections? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -Val >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> With regards, >>>>>>>>>>>>>>> Aleksandr Polovtcev >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>> >>>>>> >>>> >>> > > -- Best regards, Ivan Pavlukhin
Re: [DISCUSSION] Remove VS project from C++
+1 for removing VS project 2021-09-15 12:01 GMT+03:00, Nikolay Izhikov : > +1 > >> 15 сент. 2021 г., в 11:57, Pavel Tupitsyn >> написал(а): >> >> -1 >> >> This may become an obstacle for some of the users and I'm not sure how it >> improves anything. >> >>> 3. Sometimes even maintainers forget to add test sources to VS projects >> [1] >> We can add an automatic check for this (in form of a test). >> >> On Wed, Sep 15, 2021 at 10:28 AM Zhenya Stanilovsky >> wrote: >> >>> >>> >>> completely support ! >>> >>>> Igniters! >>>> >>>> Currently we have CMake build system, that works on Windows, Linux and >>>> MacOs flawlessly >>>> >>>> 1. CMake is supported natively in VS 2019 >>>> 2. CMake can generate VS projects for about 20 years flawlessly. >>>> 3. Sometimes even maintainers forget to add test sources to VS projects >>> [1] >>>> 4. Currently on TC we build Ignite C++ on windows and linux flawlessly >>>> using CMake >>>> 5. VS projects are not backward compatible. We have to add manually (or >>>> by >>>> sed or patch) some dependencies in order to build current VS projects >>>> on >>>> newer versions of VS. >>>> >>>> So I suggest simpy to remove VS projects because of reasons I've >>>> written >>>> above. >>>> >>>> WDYT? >>>> >>>> >>>> >>>> [1] -- https://issues.apache.org/jira/browse/IGNITE-15511 >>> >>> >>> >>> > > -- Best regards, Ivan Pavlukhin
Re: Ban Java Streams usage in Ignite 3 code
Does not this trivial strategy work for us? https://wiki.c2.com/?OptimizeLater 2021-09-08 13:52 GMT+03:00, Andrey Gura : > Agree that any additional object creation on a hot path could be a > problem. So hot path should not contain stream API and any other > potentially problem code (e.g. iterator instead of for by index). > > On Wed, Sep 8, 2021 at 1:45 PM Pavel Tupitsyn wrote: >> >> Ok, maybe a total ban is overkill, but right now streams are used even on >> some hot paths like getAllAsync [1]. >> >> Another issue is that Collectors.toList() and other variants don't accept >> capacity, and we end up with unnecessary reallocations of underlying >> arrays. >> >> [1] >> https://github.com/apache/ignite-3/blob/1d7d703ff2b18234b15a9a7b03104fbb73388edf/modules/table/src/main/java/org/apache/ignite/internal/table/KVBinaryViewImpl.java#L83 >> >> On Wed, Sep 8, 2021 at 1:06 PM Konstantin Orlov >> wrote: >> >> > Hi! >> > >> > Agree with Ivan that it’s an overkill. Code readability and >> > maintainability should have >> > the same priority as the performance (with some exceptions of course). >> > >> > BTW the result of your benchmark looks quite strange. The performance >> > penalty on >> > my laptop (Core i7 9750H, 6 cores up to 4.50 GHz) is 25%, not 8 times: >> > >> > Benchmark Mode Cnt Score Error >> > Units >> > JmhIncrementBenchmark.loopSumthrpt 10 32347.819 ± 676.548 >> > ops/ms >> > JmhIncrementBenchmark.streamSum thrpt 10 24459.196 ± 610.152 >> > ops/ms >> > >> > >> > -- >> > Regards, >> > Konstantin Orlov >> > >> > >> > > On 8 Sep 2021, at 12:23, Ivan Bessonov wrote: >> > > >> > > Hello Igniters, >> > > >> > > I object, banning streams is an overkill. I would argue that most of >> > > the >> > > code >> > > is not on hot paths and that allocations in TLAB don't create much >> > pressure >> > > on GC. >> > > >> > > Streams must be used cautiously, developers should know whether they >> > > write hot methods or not. And if methods are not hot, code simplicity >> > must >> > > be >> > > the first priority. I don't want Ignite 3 code to look like Ignite 2 >> > code, >> > > where >> > > people would iterate over Lists using explicit access by indexes, >> > because it >> > > saves them a single Iterator allocation. That's absurd. >> > > >> > > ср, 8 сент. 2021 г. в 11:43, Pavel Tupitsyn : >> > > >> > >> Igniters, >> > >> >> > >> Java streams are known to be slower and cause more GC pressure than >> > >> an >> > >> equivalent loop. >> > >> Below is a simple filter/map/reduce scenario (code [1]): >> > >> >> > >> * Benchmark Mode >> > Cnt >> > >>Score Error Units >> > >> >> > >> * StreamVsLoopBenchmark.loopSum >> > >> thrpt >> > 3 >> > >> 7987.016 ± 293.013 ops/ms >> > >> * StreamVsLoopBenchmark.loopSum:·gc.alloc.rate >> > >> thrpt >> > 3 >> > >> ≈ 10⁻⁴MB/sec >> > >> * StreamVsLoopBenchmark.loopSum:·gc.count >> > >> thrpt >> > 3 >> > >> ≈ 0counts >> > >> >> > >> * StreamVsLoopBenchmark.streamSum >> > >> thrpt >> > 3 >> > >> 1060.244 ± 36.485 ops/ms >> > >> * StreamVsLoopBenchmark.streamSum:·gc.alloc.rate >> > >> thrpt >> > 3 >> > >> 315.819 ± 10.844 MB/sec >> > >> * StreamVsLoopBenchmark.streamSum:·gc.count >> > >> thrpt >> > 3 >> > >> 55.000counts >> > >> >> > >> Loop is several times faster and does not allocate at all. >> > >> >> > >> 1. Performance is one of the most important features of our product. >> > >> 2. Most of our APIs will be on the hot path. >> > >> >> > >> One can argue about performance differences in real-world scenarios, >> > >> but increasing GC pressure just to make the code a little bit nicer >> > >> is >> > >> unacceptable. >> > >> >> > >> I propose to ban streams usage in the codebase (except for the >> > >> tests). >> > >> >> > >> Thoughts, objections? >> > >> >> > >> [1] >> > >> https://gist.github.com/ptupitsyn/5934bbbf8f92ac4937e534af9386da97 >> > >> >> > > >> > > >> > > -- >> > > Sincerely yours, >> > > Ivan Bessonov >> > >> > > -- Best regards, Ivan Pavlukhin
Re: Ban Java Streams usage in Ignite 3 code
-1 for "banning" streams. Fully agree with Ivan B. 2021-09-08 12:23 GMT+03:00, Ivan Bessonov : > Hello Igniters, > > I object, banning streams is an overkill. I would argue that most of the > code > is not on hot paths and that allocations in TLAB don't create much pressure > on GC. > > Streams must be used cautiously, developers should know whether they > write hot methods or not. And if methods are not hot, code simplicity must > be > the first priority. I don't want Ignite 3 code to look like Ignite 2 code, > where > people would iterate over Lists using explicit access by indexes, because > it > saves them a single Iterator allocation. That's absurd. > > ср, 8 сент. 2021 г. в 11:43, Pavel Tupitsyn : > >> Igniters, >> >> Java streams are known to be slower and cause more GC pressure than an >> equivalent loop. >> Below is a simple filter/map/reduce scenario (code [1]): >> >> * Benchmark Mode >> Cnt >> Score Error Units >> >> * StreamVsLoopBenchmark.loopSum thrpt >> 3 >> 7987.016 ± 293.013 ops/ms >> * StreamVsLoopBenchmark.loopSum:·gc.alloc.rate thrpt >> 3 >>≈ 10⁻⁴MB/sec >> * StreamVsLoopBenchmark.loopSum:·gc.count thrpt >> 3 >> ≈ 0counts >> >> * StreamVsLoopBenchmark.streamSum thrpt >> 3 >> 1060.244 ± 36.485 ops/ms >> * StreamVsLoopBenchmark.streamSum:·gc.alloc.ratethrpt >> 3 >> 315.819 ± 10.844 MB/sec >> * StreamVsLoopBenchmark.streamSum:·gc.count thrpt >> 3 >>55.000counts >> >> Loop is several times faster and does not allocate at all. >> >> 1. Performance is one of the most important features of our product. >> 2. Most of our APIs will be on the hot path. >> >> One can argue about performance differences in real-world scenarios, >> but increasing GC pressure just to make the code a little bit nicer is >> unacceptable. >> >> I propose to ban streams usage in the codebase (except for the tests). >> >> Thoughts, objections? >> >> [1] https://gist.github.com/ptupitsyn/5934bbbf8f92ac4937e534af9386da97 >> > > > -- > Sincerely yours, > Ivan Bessonov > -- Best regards, Ivan Pavlukhin
Re: [ANNOUNCE] Welcome Andrei Aleksandrov as a new committer
Andrei, congrats! 2021-08-16 12:04 GMT+03:00, Ilya Kazakov : > Andrei! My congratulations! > > пн, 16 авг. 2021 г. в 16:51, Ilya Kasnacheev : > >> Hello! >> >> The Project Management Committee (PMC) for Apache Ignite >> has invited Andrei Aleksandrov to become a committer and we are pleased >> to announce that he has accepted. >> >> His experience with our processes and guidelines will allow him >> to assist other contributors who are trying to contribute a fix to some >> known issue in existing code, but find themself in need of guidance, >> review >> and then ensuring the code is included. >> >> Regards, >> >> -- >> Ilya Kasnacheev >> > -- Best regards, Ivan Pavlukhin
Re: [DISCUSSION] Send documentation feedback notifications to dev list
1. Unfortunately Bugyard notifications support looks not great. I see a first real feedback item in app [1] but do not see it on dev list. Actually I suppose documentation feedback digest would solve all problems (if delivery works fine) but they do not have such feature. 2. It looks sane to see how it goes first. Unfortunately I have time to follow discussions occasionally, so I started this when I had time. 3. What other feedback services message to dev list? [1] https://app.bugyard.io/public/app/rycqZJDyY/f/6114c25933d82400145d8adc?sort=recent=new,in%20progress,implemented 2021-08-15 20:33 GMT+03:00, Denis Magda : > I would leave everything as is and come back to this question if the volume > mounts. The notifications@ and issues@ are too busy - it will be easy to > miss doc feedback that is shared once in a while. Plus, apart from the > notifications, it's better to have a single dev@ account on this service so > that any community member can log in and handle feedback (we already follow > this practice for other services). > > - > Denis > > > On Monday, August 9, 2021, Ivan Pavlukhin wrote: > >> Igniters, >> >> Recently documentation feedback notifications were set up. And >> currently destination for such notifications is dev list >> dev@ignite.apache.org. It was announced in [1]. A notification message >> itself looks as [2]. >> >> Let's discuss if we all are fine with this notifications on dev list >> as much effort was spent for keeping dev list clean, e.g. [3]. For me >> personally these notifications are some kind of "noice" as I read the >> list on spare time. >> >> Please share your opinion! Formal vote will be started if needed. >> >> [1] >> https://lists.apache.org/thread.html/rf6b0ee140239ae119c97fe4022316b4e2f5c9f06a677bace5033e313%40%3Cdev.ignite.apache.org%3E >> [2] >> https://lists.apache.org/thread.html/rbd7c06f76559f46bc2872484529bc4fa1c503d705b6cc379156f0bb0%40%3Cdev.ignite.apache.org%3E >> [3] >> https://lists.apache.org/thread.html/rbd7c06f76559f46bc2872484529bc4fa1c503d705b6cc379156f0bb0%40%3Cdev.ignite.apache.org%3E >> >> -- >> >> Best regards, >> Ivan Pavlukhin >> > -- Best regards, Ivan Pavlukhin
Re: [DISCUSSION] Send documentation feedback notifications to dev list
Pavel, > PS Ivan, links 2 and 3 are the same. Thanks for noticing. 3 related to clean dev list should be https://lists.apache.org/thread.html/re24faf295e3081eac72276e12419be412423650a21eafe81ac5c7ff1%40%3Cdev.ignite.apache.org%3E 2021-08-09 11:19 GMT+03:00, Anton Vinogradov : > +1 to keeping the dev list clean. > > On Mon, Aug 9, 2021 at 11:00 AM Pavel Tupitsyn > wrote: > >> I agree, let's keep the dev list clean. >> Automated notifications of any kind should not be sent to dev@i.a.o. >> >> PS Ivan, links 2 and 3 are the same. >> >> On Mon, Aug 9, 2021 at 8:54 AM Ivan Pavlukhin >> wrote: >> >> > Igniters, >> > >> > Recently documentation feedback notifications were set up. And >> > currently destination for such notifications is dev list >> > dev@ignite.apache.org. It was announced in [1]. A notification message >> > itself looks as [2]. >> > >> > Let's discuss if we all are fine with this notifications on dev list >> > as much effort was spent for keeping dev list clean, e.g. [3]. For me >> > personally these notifications are some kind of "noice" as I read the >> > list on spare time. >> > >> > Please share your opinion! Formal vote will be started if needed. >> > >> > [1] >> > >> https://lists.apache.org/thread.html/rf6b0ee140239ae119c97fe4022316b4e2f5c9f06a677bace5033e313%40%3Cdev.ignite.apache.org%3E >> > [2] >> > >> https://lists.apache.org/thread.html/rbd7c06f76559f46bc2872484529bc4fa1c503d705b6cc379156f0bb0%40%3Cdev.ignite.apache.org%3E >> > [3] >> > >> https://lists.apache.org/thread.html/rbd7c06f76559f46bc2872484529bc4fa1c503d705b6cc379156f0bb0%40%3Cdev.ignite.apache.org%3E >> > >> > -- >> > >> > Best regards, >> > Ivan Pavlukhin >> > >> > -- Best regards, Ivan Pavlukhin
Re: Text Queries Support
Hi Atri, Sorry for a late answer. > I didn't quite understand. Are you proposing that Ignite should not have FTS > capabilities? It seems an option to me. IMHO it is better to have no FTS instead of something like current Ignite TextQueries. 2021-08-03 12:45 GMT+03:00, Atri Sharma : > Hi Ivan, > > I didn't quite understand. Are you proposing that Ignite should not > have FTS capabilities? > > Atri > > On Tue, Aug 3, 2021 at 2:57 PM Ivan Pavlukhin wrote: >> >> Hi Atri, >> >> My main concern is non-maleficence. Every task has several solutions, >> e.g. straightforward ones: >> 1. Do not implement FTS. >> 2. Create own implementation. >> >> Some of the strongest ones live without FTS [1]. >> >> [1] https://github.com/cockroachdb/cockroach/issues/7821 >> >> 2021-08-02 11:33 GMT+03:00, Atri Sharma : >> > Hi Ivan, >> > >> > Would you like to propose an alternative to Lucene? >> > >> > Atri >> > >> > On Mon, 2 Aug 2021, 13:48 Ivan Pavlukhin, wrote: >> > >> >> Folks, >> >> >> >> Sorry if read the thread not thoroughly enough, but do we consider >> >> Lucene as obviously right choice? In my understanding Ignite history >> >> has shown clearly that "fastest feature implementation" is not usually >> >> the best. And one example of this are text queries. Are not we trying >> >> to do a same mistake again? FTS is a huge feature, I do not believe >> >> there is an easy win for it. >> >> >> >> 2021-07-27 19:18 GMT+03:00, Atri Sharma : >> >> > Andrey, >> >> > >> >> >> Per-partition Lucene index looks simple to implement, but it may >> >> >> require >> >> >> per-partition SQL to make full-text search expressions work >> >> >> correctly >> >> >> within the SQL quiery. >> >> > I think that as long as we follow the map - reduce process that we >> >> > already do for other queries, we should be fine. >> >> > >> >> >> Per-partition SQL index may kill the performance. We already tried >> >> >> to >> >> >> do >> >> >> that in Ignite 2. However, QueryParallelism feature helps to speed >> >> >> up >> >> >> some >> >> >> data-intensive queries, >> >> >> but hits the performance in simple cases, and at some point (e.g. >> >> >> segments >> >> >> > number of CPU) the performance rapidly degrades with the >> >> >> > increasing >> >> >> number of segments. >> >> > >> >> > Yeah, that is always the case, but a global index will be a >> >> > nightmare >> >> > in terms of concurrency and pessimistic concurrency control will >> >> > anyways kill the benefits, coupled with the metadata requirements. >> >> > What were the specific issues with per partition index? >> >> >> >> >> >> AFAIK, Lucene widely used bitmap indices that are easy to merge. >> >> >> Maybe, the map-reduce technique underneath FTS expressions and some >> >> hacks >> >> >> will add a minimal overhead. >> >> > >> >> > Lucene uses many types of indices but the aspect here is that per >> >> > partition Lucene indices can return docIDs and we can merge them in >> >> > reduce phase. So we are abstracted out from specifics of the >> >> > internal >> >> > index being used to serve the query. >> >> > >> >> >> >> >> >> > As illustrated by Ilya, we can use Ignite's WAL records to >> >> >> > rebuild >> >> >> > Lucene indices. The important thing here is to not treat Lucene >> >> >> > indices as source of truth. >> >> >> To use WAL we either should relay Lucene files to our Page memory >> >> >> or >> >> >> be >> >> >> aware of Lucene files structure. >> >> >> The first looks tricky, as we should guarantee a contiguous address >> >> space >> >> >> in Page memory for reflecting Lucene file. Maybe separate managed >> >> >> memory >> >> >> segment with its own rules? >> >> > >> >> > Why not use Lucene's MMappedDirectory and map it to our s
[DISCUSSION] Send documentation feedback notifications to dev list
Igniters, Recently documentation feedback notifications were set up. And currently destination for such notifications is dev list dev@ignite.apache.org. It was announced in [1]. A notification message itself looks as [2]. Let's discuss if we all are fine with this notifications on dev list as much effort was spent for keeping dev list clean, e.g. [3]. For me personally these notifications are some kind of "noice" as I read the list on spare time. Please share your opinion! Formal vote will be started if needed. [1] https://lists.apache.org/thread.html/rf6b0ee140239ae119c97fe4022316b4e2f5c9f06a677bace5033e313%40%3Cdev.ignite.apache.org%3E [2] https://lists.apache.org/thread.html/rbd7c06f76559f46bc2872484529bc4fa1c503d705b6cc379156f0bb0%40%3Cdev.ignite.apache.org%3E [3] https://lists.apache.org/thread.html/rbd7c06f76559f46bc2872484529bc4fa1c503d705b6cc379156f0bb0%40%3Cdev.ignite.apache.org%3E -- Best regards, Ivan Pavlukhin
Re: [DISCUSSION] Documentation feedback button
Is there an easy way to subscribe personally for such notifications? I forsee complaints that the majority of developers are not interested in documentation feedback notifications. 2021-08-04 12:40 GMT+03:00, Denis Magda : > Nikita, > > We created a bugyard account for the Ignite community. I'm suggesting using > a shared account bound to 'dev@ignite.apache.org'. That will allow everyone > on the dev list to receive and process documentation feedback. > https://svn.apache.org/repos/private/pmc/ignite/credentials/bugyard.io > > Mauricio will add the button to the website shortly. I'll let you update > the documentation process. > > -- > Denis > > > - > Denis > > On Tue, Jul 27, 2021 at 5:24 PM Nikita Safonov > wrote: > >> Hello Igniters, >> >> I would like to proceed with implementing the feedback service for the >> Ignite documentation website. >> If nobody objects, let's start the work. >> The details are described here: >> https://issues.apache.org/jira/browse/IGNITE-15198 >> >> With best regards, >> Nikita >> >> ср, 17 мар. 2021 г. в 15:50, Mauricio Stekl : >> >> > Hi, >> > I agree with Nikita, it would be a very simple way of getting feedback >> > to >> > improve the documentation. >> > This tool in particular is quite easy to integrate into the online docs >> > template. >> > >> > Best, >> > Mauricio >> > >> > >> > >> > >> > >> > On Tue, Mar 16, 2021 at 4:46 PM Denis Magda wrote: >> > >> > > Nikita, thanks for starting the conversation. I'll just cast my vote >> for >> > > the bugyard.io for its ability to select and comment on a specific >> > > problematic point on a documentation page (a paragraph, sentence, >> > picture, >> > > code snippet, etc.). It makes it trivial to share feedback and then >> it's >> > > easy to process it on the other side. Those who'd like to experience >> it, >> > > click the "Documentation Feedback" button on the GridGain docs: >> > > https://www.gridgain.com/docs/latest/getting-started/what-is-gridgain >> > > >> > > - >> > > Denis >> > > >> > > >> > > On Tue, Mar 16, 2021 at 3:14 PM Никита Сафонов < >> > vlasovpavel2...@gmail.com> >> > > wrote: >> > > >> > > > Hello Igniters, >> > > > >> > > > I would like to propose an enhancement that can significantly >> > > > improve >> > the >> > > > quality of our documentation. >> > > > >> > > > I suggest adding a feedback button (let’s call it a «Documentation >> > > > feedback» button) to the web site documentation pages so everyone >> could >> > > > leave a comment to what is already published. >> > > > >> > > > The feedback may refer to the Ignite documentation in general or to >> > > > a >> > > > particular section, paragraph, or even term or value. >> > > > >> > > > I do believe that we would harvest dozens and hundreds of ideas, >> > > > suggestions, and observations. >> > > > >> > > > I would also suggest using bugyard.io as a solid service for >> gathering >> > > > feedback (I’m quite familiar with it, it is easy to implement, use, >> and >> > > > maintain). >> > > > >> > > > So folks, what do you think of this? >> > > > >> > > > With best regards, >> > > > Nikita >> > > > >> > > >> > >> > -- Best regards, Ivan Pavlukhin
Re: Apache Ignite 3 Alpha 2 webinar follow up questions
Hi Courtney, Statistics for query planning is a rather complex subject. Unfortunately I am now aware of technical details how is it expected to be implemented in Ignite 3. Folks driving SQL please step in. 2021-07-31 20:18 GMT+03:00, Courtney Robinson : > Hi Ivan, > Atri's description of the query plan being cached is what I was thinking of > with my description. > > I lack the knowledge on how the statistics are maintained to really comment > constructively Atri but my first question about the problem you raise with > statistics would be: > > How/where are the stats maintained and if a query plan is cached based on > some stats, is it not possible to invalidate the cached plan periodically > or based on statistics changes? > > Regards, > Courtney Robinson > Founder and CEO, Hypi > Tel: ++44 208 123 2413 (GMT+0) <https://hypi.io> > > <https://hypi.io> > https://hypi.io > > > On Sat, Jul 31, 2021 at 8:54 AM Atri Sharma wrote: > >> Query caching works on three levels - caching results, caching blocks and >> caching query plans. >> >> Prepared queries work by caching a plan for a query and reusing that plan >> by changing the parameters for the incoming query. So the query remains >> the >> same, but input values keep changing. >> >> The problem with prepared queries is that query execution can go bad very >> fast if the underlying data distribution changes and the cached plan is >> no >> longer optimal for the given statistics. >> >> On Sat, 31 Jul 2021, 12:54 Ivan Pavlukhin, wrote: >> >> > Hi Courtney, >> > >> > Please clarify what do you mean by prepared queries and query caching? >> > Do you mean caching query results? If so, in my mind material views >> > are the best approach here (Ignite 2 does not support them). Do you >> > have other good approaches in your mind? E.g. implemented in other >> > databases. >> > >> > 2021-07-26 21:27 GMT+03:00, Valentin Kulichenko < >> > valentin.kuliche...@gmail.com>: >> > > Hi Courtney, >> > > >> > > Generally speaking, query caching certainly makes sense. As far as I >> > know, >> > > Ignite 2.x actually does that, but most likely there might be room >> > > for >> > > improvement as well. We will look into this. >> > > >> > > As for the SQL API - the answer is yes. The requirement for a dummy >> cache >> > > is an artifact of the current architecture. This is 100% wrong and >> > > will >> > be >> > > changed in 3.0. >> > > >> > > -Val >> > > >> > > On Sun, Jul 25, 2021 at 2:51 PM Courtney Robinson >> > > >> > > wrote: >> > > >> > >> Something else came to mind, are there plans to support prepared >> > queries? >> > >> >> > >> I recall someone saying before that Ignite does internally cache >> queries >> > >> but it's not at all clear if or how it does do that. I assume a >> > >> simple >> > >> hash >> > >> of the query isn't enough. >> > >> >> > >> We generate SQL queries based on user runtime settings and they can >> get >> > >> to >> > >> hundreds of lines long, I imagine this means most of our queries are >> not >> > >> being cached but there are patterns so we could generate and manage >> > >> prepared queries ourselves. >> > >> >> > >> Also, will there be a dedicated API for doing SQL queries rather >> > >> than >> > >> having to pass a SqlFieldsQuery to a cache that has nothing to do >> > >> with >> > >> the >> > >> cache being queried? When I first started with Ignite years ago, >> > >> this >> > was >> > >> beyond confusing for me. I'm trying to run select x from B but I >> > >> pass >> > >> this >> > >> to a cache called DUMMY or whatever arbitrary name... >> > >> >> > >> On Fri, Jul 23, 2021 at 4:05 PM Courtney Robinson < >> > >> courtney.robin...@hypi.io> >> > >> wrote: >> > >> >> > >> > Andrey, >> > >> > Thanks for the response - see my comments inline. >> > >> > >> > >> > >> > >> >> I've gone through the questions and have no the whole picture of >> your >> > &g
Re: Text Queries Support
Hi Atri, My main concern is non-maleficence. Every task has several solutions, e.g. straightforward ones: 1. Do not implement FTS. 2. Create own implementation. Some of the strongest ones live without FTS [1]. [1] https://github.com/cockroachdb/cockroach/issues/7821 2021-08-02 11:33 GMT+03:00, Atri Sharma : > Hi Ivan, > > Would you like to propose an alternative to Lucene? > > Atri > > On Mon, 2 Aug 2021, 13:48 Ivan Pavlukhin, wrote: > >> Folks, >> >> Sorry if read the thread not thoroughly enough, but do we consider >> Lucene as obviously right choice? In my understanding Ignite history >> has shown clearly that "fastest feature implementation" is not usually >> the best. And one example of this are text queries. Are not we trying >> to do a same mistake again? FTS is a huge feature, I do not believe >> there is an easy win for it. >> >> 2021-07-27 19:18 GMT+03:00, Atri Sharma : >> > Andrey, >> > >> >> Per-partition Lucene index looks simple to implement, but it may >> >> require >> >> per-partition SQL to make full-text search expressions work correctly >> >> within the SQL quiery. >> > I think that as long as we follow the map - reduce process that we >> > already do for other queries, we should be fine. >> > >> >> Per-partition SQL index may kill the performance. We already tried to >> >> do >> >> that in Ignite 2. However, QueryParallelism feature helps to speed up >> >> some >> >> data-intensive queries, >> >> but hits the performance in simple cases, and at some point (e.g. >> >> segments >> >> > number of CPU) the performance rapidly degrades with the increasing >> >> number of segments. >> > >> > Yeah, that is always the case, but a global index will be a nightmare >> > in terms of concurrency and pessimistic concurrency control will >> > anyways kill the benefits, coupled with the metadata requirements. >> > What were the specific issues with per partition index? >> >> >> >> AFAIK, Lucene widely used bitmap indices that are easy to merge. >> >> Maybe, the map-reduce technique underneath FTS expressions and some >> hacks >> >> will add a minimal overhead. >> > >> > Lucene uses many types of indices but the aspect here is that per >> > partition Lucene indices can return docIDs and we can merge them in >> > reduce phase. So we are abstracted out from specifics of the internal >> > index being used to serve the query. >> > >> >> >> >> > As illustrated by Ilya, we can use Ignite's WAL records to rebuild >> >> > Lucene indices. The important thing here is to not treat Lucene >> >> > indices as source of truth. >> >> To use WAL we either should relay Lucene files to our Page memory or >> >> be >> >> aware of Lucene files structure. >> >> The first looks tricky, as we should guarantee a contiguous address >> space >> >> in Page memory for reflecting Lucene file. Maybe separate managed >> >> memory >> >> segment with its own rules? >> > >> > Why not use Lucene's MMappedDirectory and map it to our storage >> > classes? >> > >> >> >> >> >> Transactions. >> >> >> * Will we support transactions? >> >> > Lucene has no concept of transactions. >> >> Yes, but we have. >> >> Lucene index may be non-transactional, but users never expect to see >> >> uncommited data. >> >> How does this connect with transactional SQL? >> > We could have the Lucene writes done as a part of transactions and ack >> > back only when it succeeds/fails. WDYT? >> >> >> >> On Tue, Jul 27, 2021 at 1:36 PM Atri Sharma wrote: >> >> >> >> > Sorry, I planned on creating a Wiki page for this, but it makes more >> >> > sense to be replying here. >> >> > >> >> > > * How Lucene index can be split among the nodes? >> >> > >> >> > We can have partition level indices on each node. >> >> > >> >> > > * If we'll have a single index for all partitions on the >> >> > > particular >> >> > > node, >> >> > > then how index records will be aware of partitioning? >> >> > >> >> > Index records dont need to be aware of partitioning -- each Lucene >> >> &
Re: Text Queries Support
t;: >> > > > > > > > > >> > > > > > > > > > Hi, Atri! >> > > > > > > > > > >> > > > > > > > > > You're right, Actually there is a lack of support for >> > > > TextQueries. >> > > > > > > For >> > > > > > > > the >> > > > > > > > > > last ticket I'm doing I see some obvious issues with >> > > > > > > > > > them >> > (no >> > > > page >> > > > > > > size >> > > > > > > > > > support, for example). I'm glad that somebody wants to >> > maintain >> > > > > > this >> > > > > > > > > > functionality. Thanks a lot! >> > > > > > > > > > >> > > > > > > > > > For the MergeSort algorithm there is already a patch >> > > > > > > > > > for >> > that >> > > > [1]. >> > > > > > > It's >> > > > > > > > > > currently on review. This patch introduces an abstract >> > reducer >> > > > for >> > > > > > > > > > CacheQueries with 2 implementations (unordered, >> > merge-sort). >> > > > Then >> > > > > > > > TextQuery >> > > > > > > > > > leverages on MergeSort to order results from multiple >> > nodes by >> > > > > > score. >> > > > > > > > This >> > > > > > > > > > patch also fixes the pageSize issue, I've mentioned >> > > > > > > > > > before. >> > > > Could >> > > > > > you >> > > > > > > > > > please check if it fully matches your idea? Any issues >> > > > > > > > > > or >> > > > comments >> > > > > > > are >> > > > > > > > > > welcome. >> > > > > > > > > > >> > > > > > > > > > I've prepared this ticket, because I need the MergeSort >> > > > algorithm >> > > > > > for >> > > > > > > > the >> > > > > > > > > > new type of queries I'm implementing (IndexQuery, it >> > > > > > > > > > should >> > > > also >> > > > > > > > provide >> > > > > > > > > > ordered results over multiple nodes). Currently I'm not >> > > > planning to >> > > > > > > go >> > > > > > > > > > further with TextQuery, so if you're going to support >> > > > > > > > > > this >> > > > it'll >> > > > > > be a >> > > > > > > > great >> > > > > > > > > > contribution, I think. >> > > > > > > > > > >> > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-14703 >> > > > > > > > > > [2] https://github.com/apache/ignite/pull/9081 >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > On Mon, Jun 21, 2021 at 11:11 AM Atri Sharma < >> > a...@apache.org> >> > > > > > > wrote: >> > > > > > > > > > >> > > > > > > > > > > Hi All, >> > > > > > > > > > > >> > > > > > > > > > > I have been looking into our text queries support and >> > > > > > > > > > > see >> > > > that it >> > > > > > > has >> > > > > > > > > > > limited community support. >> > > > > > > > > > > >> > > > > > > > > > > Therefore, I volunteer to be the maintainer of the >> > module and >> > > > > > work >> > > > > > > on >> > > > > > > > > > > enhancing it further. >> > > > > > > > > > > >> > > > > > > > > > > First goal would be to move to Lucene 8.x, then work >> > > > > > > > > > > on >> > > > sorted >> > > > > > > reduce >> > > > > > > > > > > - merge across nodes. Fundamentally, this is doable >> > > > > > > > > > > since >> > > > Lucene >> > > > > > > > ranks >> > > > > > > > > > > documents according to their score, and documents are >> > > > returned in >> > > > > > > the >> > > > > > > > > > > order of their score. Since the scoring function is >> > > > homogeneous, >> > > > > > > this >> > > > > > > > > > > means that across nodes, we can compare scores and >> > > > > > > > > > > merge >> > > > sort. >> > > > > > > > > > > >> > > > > > > > > > > Please let me know if I can take this up. >> > > > > > > > > > > >> > > > > > > > > > > Atri >> > > > > > > > > > > >> > > > > > > > > > > -- >> > > > > > > > > > > Regards, >> > > > > > > > > > > >> > > > > > > > > > > Atri >> > > > > > > > > > > Apache Concerted >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > -- >> > > > > > > > > >> > > > > > > > > Best regards, >> > > > > > > > > Alexei Scherbakov >> > > > > > > > >> > > > > > > > -- >> > > > > > > > Regards, >> > > > > > > > >> > > > > > > > Atri >> > > > > > > > Apache Concerted >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > >> > > > > -- >> > > > > Best regards, >> > > > > Andrey V. Mashenkov >> > > > >> > > > -- >> > > > Regards, >> > > > >> > > > Atri >> > > > Apache Concerted >> > > > >> > > >> > > >> > > -- >> > > Best regards, >> > > Andrey V. Mashenkov >> > >> > -- >> > Regards, >> > >> > Atri >> > Apache Concerted >> > >> >> >> -- >> Best regards, >> Andrey V. Mashenkov > > -- > Regards, > > Atri > Apache Concerted > -- Best regards, Ivan Pavlukhin
Re: MTCGA site is down
Folks, Am I getting it right that MTCGA bot IP addresses are changed from time to time? If so, can we use something like CNAME [1] to alias mtcga.ignite.apache.org to mtcga.gridgain.com? What do you think? [1] https://en.wikipedia.org/wiki/CNAME_record 2021-07-28 11:13 GMT+03:00, Alex Plehanov : > Petr, thank you! > > It's working now. > > ср, 28 июл. 2021 г. в 11:03, Petr Ivanov : > >> I've created issue in INFRA project about this matter [1]. >> >> Please, stand by, >> >> >> >> [1] https://issues.apache.org/jira/browse/INFRA-22150 >> >> > On 28 Jul 2021, at 09:42, Alex Plehanov >> > wrote: >> > >> > Hello, Igniters! >> > >> > Looks like the MTCGA site hosted on Apache Ignite domain [1] is >> > currently >> > down (at least since yesterday), however, this site hosted on GridGain >> > domain [2] is working. >> > Can anyone have a look what's happening? >> > >> > [1] : mtcga.ignite.apache.org >> > [2] : mtcga.gridgain.com >> >> > -- Best regards, Ivan Pavlukhin
Re: Apache Ignite 3 Alpha 2 webinar follow up questions
t;> > optimisations and have slowly pushed more of our queries to >> >> > Calcite that we >> >> > then push down to Ignite. >> >> > 2. We Index into Solr and use the Solr indices and others to >> >> > fulfill over all queries with Ignite just being one of the >> >> > possible storage >> >> > targets Calcite pushes down to. If we could get to the calcite >> >> > API from an >> >> > Ignite thick client, it would enable us to remove a layer of >> >> > abstraction >> >> > and complexity and make Ignite our primary that we then link >> >> > with Solr and >> >> > others to fulfill queries. >> >> >4. Will the unified storage model enable different versions of >> >> Ignite to >> >> >be in the cluster when persistence is enabled so that rolling >> >> restarts >> >> > can >> >> >be done? >> >> >1. We have to do a strange dance to perform Ignite upgrades >> >> > without >> >> > downtime because pods/nodes will fail to start on version >> mismatch >> >> > and if >> >> > we get that dance wrong, we will corrupt a node's data. It >> >> > will >> >> make >> >> > admin/upgrades far less brittle and error prone if this was >> >> possible. >> >> >5. Will it be possible to provide a custom cache store still and >> will >> >> >these changes enable custom cache stores to be queryable from >> >> > SQL? >> >> >1. Our Ignite usage is wide and complex because we use KV, SQL >> >> > and >> >> other >> >> > APIs. The inconsistency of what can and can't be used from one >> >> API to >> >> > another is a real challenge and has forced us over time to >> >> > stick >> >> > to one API >> >> > and write alternative solutions outside of Ignite. It will >> >> > drastically >> >> > simplify things if any CacheStore (or some new equivalent) >> >> > could >> >> > be plugged >> >> > in and be made accessible to SQL (and in fact all other APIs) >> >> without >> >> > having to load all the data from the underlying CacheStore >> >> > first >> >> > into memory >> >> >6. This question wasn't mine but I was going to ask it as well: >> What >> >> >will happen to the Indexing API since H2 is being removed? >> >> > 1. As I mentioned above, we Index into Solr, in earlier >> >> > versions >> >> of >> >> > our product we used the indexing SPI to index into Lucene on >> >> > the >> >> > Ignite >> >> > nodes but this presented so many challenges we ultimately >> >> > abandoned it and >> >> > replaced it with the current Solr solution. >> >> > 2. Lucene indexing was ideal because it meant we didn't have >> >> > to >> >> > re-invent Solr or Elasticsearch's sharding capabilities, that >> was >> >> > almost >> >> > automatic with Ignite only giving you the data that was meant >> for >> >> the >> >> > current node. >> >> > 3. The Lucene API enabled more flexibility and removed a >> >> > network >> >> > round trip from our queries. >> >> > 4. Given Calcite's ability to support custom SQL functions, >> >> > I'd >> >> love >> >> > to have the ability to define custom functions that Lucene was >> >> > answering >> >> >7. What impact does RAFT now have on conflict resolution, off the >> >> top of >> >> >my head there are two cases >> >> > 1. On startup after a split brain Ignite currently takes an >> >> "exercise >> >> > for the reader" approach and dumps a log along the lines of >> >> > >> >> > >1. BaselineTopology of joining node is not compatible with >> >> > > BaselineTopology in the cluster. >> >> > >1. Branching history of cluster BlT doesn't contain branching >> point >> >> > > hash of joining node BlT. Consider cleaning persistent >> >> > > storage >> >> of >> >> > the node >> >> > > and adding it to the cluster again. >> >> > > >> >> >1. This leaves you with no choice except to take one half and >> >> manually >> >> > copy, write data back over to the other half then destroy the >> bad >> >> > one. >> >> > 2. The second case is conflicts on keys, I >> >> > beleive CacheVersionConflictResolver and manager are used >> >> > by GridCacheMapEntry which just says if use old value do this >> >> > otherwise use >> >> > newVal. Ideally this will be exposed in the new API so that >> >> > one >> >> can >> >> > override this behaviour. The last writer wins approach isn't >> >> always >> >> > ideal >> >> > and the semantics of the domain can mean that what is consider >> >> > "correct" in >> >> > a conflict is not so for a different domain. >> >> >8. This is last on the list but is actually the most important >> >> > for >> us >> >> >right now as it is an impending and growing risk. We allow >> customers >> >> to >> >> >create their own tables on demand. We're already using the same >> cache >> >> > group >> >> >etc for data structures to be re-used but now that we're getting >> >> > to >> >> >thousands of tables/caches our startup times are sometimes >> >> unpredictably >> >> >long - at present it seems to depend on the state of the >> cache/table >> >> > before >> >> >the restart but we're into the order of 5 - 7 mins and steadily >> >> > increasing >> >> >with the growth of tables. Are there any provisions in Ignite 3 >> >> > for >> >> >ensuring startup time isn't proportional to the number of >> >> tables/caches >> >> >available? >> >> > >> >> > >> >> > Those are the key things I can think of at the moment. Val and >> >> > others >> >> I'd >> >> > love to open a conversation around these. >> >> > >> >> > Regards, >> >> > Courtney Robinson >> >> > Founder and CEO, Hypi >> >> > Tel: ++44 208 123 2413 (GMT+0) <https://hypi.io> >> >> > >> >> > <https://hypi.io> >> >> > https://hypi.io >> >> > >> >> >> >> >> >> -- >> >> Best regards, >> >> Andrey V. Mashenkov >> >> >> > >> > -- Best regards, Ivan Pavlukhin
Re: [DISCUSS] Confuse default inspections.
+ for both points.
2021-07-20 9:56 GMT+03:00, Ivan Daschinsky :
> Hi!
>
> Firstly, lets talk about interfaces.
>
> 1. First of all, do we have an automatic inspection for it? AFAIK we don't
> 2. I am for consistency. At least for production code. Nothing worse is
> when someone mixes both approaches.
>
> About a prohibition of curly brackets around one line -- I am strongly
> against this rule, it should be removed. There were a few bugs that this
> rule caused.
>
> вт, 20 июл. 2021 г. в 09:23, Zhenya Stanilovsky >:
>
>>
>> Igniters, i understand that this is very long and fundamental story. but
>> … still want to rise up this discussion, we have 2 very strange
>> inspections:
>> * «public» modifier in interface methods.
>> * Illegal ‘{}’ for one line statement. — i found it harmful.
>> I don`t want to link additional discussion about pos. 2 i think that harm
>> is obvious.
>> I suggest to get rid of them.
>>
>> what do you think ?
>>
>>
>>
>
>
>
> --
> Sincerely yours, Ivan Daschinskiy
>
--
Best regards,
Ivan Pavlukhin
Re: Ignite 3.0 Ignition API, node startup, and thin client startup
her things can be added later. >> > > > > >> > > > > On Fri, Jul 9, 2021 at 3:37 PM Ivan Daschinsky < >> ivanda...@gmail.com> >> > > > > wrote: >> > > > > >> > > > > > > I don't think we need an explicit reactive API in the core >> > library. >> > > > > > >> > > > > > Have you ever thought about why thin client should be in core >> > module? >> > > > Why >> > > > > > we do the same thing as we did in ignite 2.x? In the days of >> cloud >> > > > native >> > > > > > we still think about large uber-jar with everything? >> > > > > > >> > > > > > > Same story with Kotlin, it works with CompletableFuture. >> > > > > > Users don't want to code by theyselves, they want to use ready >> and >> > > > > complete >> > > > > > clients. Please, don't underestimate kotlin, kotlin coroutines >> and >> > > > > reactive >> > > > > > streams. They are all the first class citizens in spring 5 for >> > > > > > 3 >> > > years >> > > > > > >> > > > > > пт, 9 июл. 2021 г., 14:43 Pavel Tupitsyn >> > > > > > : >> > > > > > >> > > > > > > > You forget about reactive api >> > > > > > > >> > > > > > > I don't think we need an explicit reactive API in the core >> > library. >> > > > > > > Observable.fromFuture bridges async to Rx easily: >> > > > > > > >> > > > > > > Observable.fromFuture(client.putAsync(k, v)).flatMap(...) >> > > > > > > >> > > > > > > Same story with Kotlin, it works with CompletableFuture. >> > > > > > > >> > > > > > > On Fri, Jul 9, 2021 at 1:31 PM Ivan Daschinsky < >> > > ivanda...@gmail.com> >> > > > > > > wrote: >> > > > > > > >> > > > > > > > You forget about reactive api :) >> > > > > > > > >> > > > > > > > And whats a problem with discocerability? >> > > > > > > > >> > > > > > > > var syncApi = client.sync(); >> > > > > > > > syncApi.put(k, v); >> > > > > > > > >> > > > > > > > var rxApi = client.reactive(); >> > > > > > > > rxApi.put(k,v).flatMap(res -> ); >> > > > > > > > >> > > > > > > > And sync, async and reactive is not enough, it is good idea >> to >> > > > > support >> > > > > > > > kotlin coroutines also :) >> > > > > > > > >> > > > > > > > пт, 9 июл. 2021 г., 13:26 Pavel Tupitsyn < >> ptupit...@apache.org >> > >: >> > > > > > > > >> > > > > > > > > Ivan D., >> > > > > > > > > >> > > > > > > > > > container of properties >> > > > > > > > > >> > > > > > > > > What is a container of properties? >> > > > > > > > > As a user, I want a simple way to start a client and >> perform >> > > > > > > operations. >> > > > > > > > > >> > > > > > > > > I don't want anything confusing and complicated like >> > > > > > > > > Netty >> > > > > Bootstrap. >> > > > > > > > There >> > > > > > > > > might be a reason for Netty to be this way - it is a >> > low-level >> > > > > > library. >> > > > > > > > But >> > > > > > > > > Ignite is not. >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > > separate facades for sync, async >> > > > > > > > > >> > > > > > > > > Strongly disagree with this idea. It hurts API >> > discoverability. >> > > > > > > > > As a user, in my IDE I type "igniteTable.get" and see a >> list
Re: Ignite 3.0 Ignition API, node startup, and thin client startup
; > > two >> > > > > > > usages of the 'Ignite' API: >> > > > > > > >> > > > > > > 1. Remote, via the binary protocol. >> > > > > > > 2. Local - needed for compute and services. (This is how it >> > works >> > > > > now.) >> > > > > > > >> > > > > > > I believe that the API should be the same, and there should >> > > > > > > be >> a >> > > > > unified >> > > > > > > access point. Ignition seems to be a good candidate for this. >> > > > > > > >> > > > > > > Ignition#start should eventually be removed from the public >> API. >> > It >> > > > is >> > > > > > > currently there only because we don't have the thin client >> > > > > > > yet. >> > > > > > > >> > > > > > > -Val >> > > > > > > >> > > > > > >> On Wed, Jul 7, 2021 at 5:47 AM Pavel Tupitsyn < >> > > ptupit...@apache.org >> > > > > >> > > > > > wrote: >> > > > > > >> >> > > > > > >> Igniters, >> > > > > > >> >> > > > > > >> I have a few questions regarding server node startup and >> > > > > > >> thin >> > > > clients. >> > > > > > >> >> > > > > > >> State of things: >> > > > > > >> - Server nodes will be started with 'ignite run' from CLI >> > > > > > >> [1] >> > > > > > >> - ignite-api module represents our public API >> > > > > > >> - ignite-api has Ignition interface to start server nodes >> > > > > > >> >> > > > > > >> Questions: >> > > > > > >> - What's the idea behind Ignition interface in the public >> > > > > > >> API? >> > Are >> > > > we >> > > > > > going >> > > > > > >> to have an "embedded mode" where servers can be started from >> > > code? I >> > > > > > >> thought this was not planned. >> > > > > > >> - How are users supposed to retrieve an instance of the >> Ignition >> > > > > > interface? >> > > > > > >> - Are there any plans to start thin clients from Ignition >> > > interface, >> > > > > or >> > > > > > >> should we have a separate way of doing this? >> > > > > > >> >> > > > > > >> >> > > > > > >> [1] >> > > > > > >> >> > > > > > >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158873958 >> > > > > > >> >> > > > > > >> > > > > >> > > > >> > > >> > >> > -- Best regards, Ivan Pavlukhin
Re: [DISCUSSION] Code style. Variable abbrevations
Hi Pavel, Thank you for sharing your thoughts! My personal opinion is very similar. But do you have a plan what should we do with the subject? Do you suggest to cleanup whole codebase from abbreviations? Or do you think that it is fine to keep existing code as is and avoid abbreviations for new code? Or something else? > Who can tell me at least 5 contributors with 10+ commits who are not > working/worked in GridGain or SberBank? It's sad to hide behind an open > source community that really does not exist. Actually it seems to me offtopic. But here my opinion is different. It is ok to me that the project is mainly driven by aforementioned companies. And I would rather care more about ease of single contributions. I contributed to other open source projects and did < 10 commits to each. Because in daily work generally we use stable libraries and can allow ourselves to work only with bugs or improvements in open source libraries. And I consider it a way to go in open source. 2021-06-20 14:13 GMT+03:00, Pavel Kovalenko : > I think in this discussion there should be a question - why do we still > need abbreviations for all variables and fields? What problem does it > solve? > Nobody still clearly answered this question. > I saw only 2 arguments to keep abbreviations: > 1) Less codebase symbols. I think it is a weak argument. We're not in the > 80-90s when there were no modern IDEs with hints and auto-completion. > Ideally code should be read just as english text. That really helps to > understand the code if you see it for the first time. Abbreviations > everywhere (except well-known CS terms) don't improve code readability. > 2) We shouldn't break constitency with old codebase written with enforced > abbreviations. Hence the question - why was this rule introduced at project > startup? Who knows that? > I guess most people who decided to introduce this rule left the Ignite > project. > >>> Consistency is what makes it easier to contribute to the project > Yes it is. But what makes it easier is actually clear architecture, simple > and convenient abstractions and well-documented code. Ignite has a big lack > of that. > For example, look at the transactions or PME codebase. It's darkness and no > abbreviations for "consistency" will help to understand that code. > >>> and attract new members > Who can tell me at least 5 contributors with 10+ commits who are not > working/worked in GridGain or SberBank? > It's sad to hide behind an open source community that really does not > exist. > > > пт, 18 июн. 2021 г. в 14:21, Anton Vinogradov : > >> > Agree. I think you can start a discussion and change some abbrevs if >> > you >> wish. >> >> How about keeping the abbreviation map file inside the Ignite repo? >> In this case, you may change the abbreviation by issue/pr covered by a >> green TC check. >> >> On Thu, Jun 17, 2021 at 11:36 AM Nikolay Izhikov >> wrote: >> >> > > No validation in the world prevent me from typing manually "mess" >> > instead of "msg"/«message" >> > >> > Code review will do. >> > >> > > btw "svc" is more common imo >> > >> > Agree. I think you can start a discussion and change some abbrevs if >> > you >> > wish. >> > >> > >> > > 17 июня 2021 г., в 10:23, Ivan Daschinsky >> > написал(а): >> > > >> > > I'm sorry, but the rule is not strict and, moreover, not clear enough >> for >> > > constans. See here [1] >> > > ``` >> > > Type and method names are usually not abbreviated (except for the >> > > well-accepted abbreviations such as EOF, Impl, Fifo, etc.). >> > > >> > > Abbreviation applies to local variables, method parameters, class >> fields >> > > and in **some cases public static fileds** (constants) of the class. >> > > ``` >> > > But there is not any list that contains these cases. I suppose, that >> > > constant naming should not be abbreviated. >> > > As I see, the most cases of violations, described in initial post, >> > > are >> > > about constant's namings. >> > > >> > > I suppose that we should define strict and clear rules about >> > > constants >> > > naming. >> > > >> > > [1] -- >> > https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules >> > > >> > > >> > > чт, 17 июн. 2021 г. в 10:10, Konstantin Orlov : >> > > >> > >> Enforced validation doesn't guara
Re: [DISCUSSION] Code style. Variable abbrevations
Hi Nikolay, Thanks, it is rather interesting. Do we all agree that using different conventions for different code packages does not break "consistency"? Or did I get something wrong? 2021-06-17 7:12 GMT+03:00, Николай Ижиков : > Hello, Ivan. > > We can create checkstyle rule to enforce usage of abbreviations. > Internal/public code differs by package. > > PoC of rule [1] > > [1] https://github.com/apache/ignite/pull/9153 > >> 17 июня 2021 г., в 07:01, Ivan Pavlukhin >> написал(а): >> >> Nikita, >> >> Do you have a plan in your mind how to make Ignite codebase consistent? >> >> AFAIR, we had it intentionally inconsistent for a long time at least >> for one sake: for internal code we used special conventions (e.g. >> abbreviations, shorten getters) and common Java conventions for public >> API and examples (e.g. no abbreviations and usual getters). >> >> 2021-06-16 23:37 GMT+03:00, Nikita Ivanov : >>> Consistency is what makes it easier to contribute to the project and >>> attract new members. Consistency implies strong, well defined and >>> universally enforced rules. Just because we have some individuals who >>> are lazy or inexperienced - it does not mean that the entire project >>> should relax the basic-level engineering discipline. >>> >>> On a personal node - nothing screams "immaturity" louder that a code >>> that uses inconsistent naming, commenting, code style & organization, >>> etc. >>> -- >>> Nikita Ivanov >>> >>>> On Wed, Jun 16, 2021 at 5:56 AM Andrey Gura wrote: >>>> >>>> Hi, >>>> >>>> I understand that this rule seems too strict or may be weird. But >>>> removing the rule could lead to review comments like "use future >>>> instead of fut". So my proposal is to change rule from "required" to >>>> "recommended". >>>> >>>> On Wed, Jun 16, 2021 at 2:49 PM Valentin Kulichenko >>>> wrote: >>>>> >>>>> Konstantin, thanks for chiming in. >>>>> >>>>> That's exactly my concern. Overformalization is generally not a good >>>>> thing. >>>>> Someone used "mess" to abbreviate "message"? That is surely not a good >>>>> coding style, but that's what code reviews are for. I believe that our >>>>> committers are more than capable of identifying such issues. >>>>> >>>>> At the same time, with the current rules, we are *forced* to use >>>>> abbreviations like "locAddrGrpMgr", whether we like it or not. All it >>>>> does >>>>> is makes it harder to contribute to Ignite, without providing any >>>>> clear >>>>> value. >>>>> >>>>> -Val >>>>> >>>>> On Thu, Jun 10, 2021 at 9:46 AM Konstantin Orlov >>>>> wrote: >>>>> >>>>>> +1 for making this optional >>>>>> >>>>>> Common abbreviation rules is good for sure, but sometimes I getting >>>>>> sick >>>>>> of all those locAddrGrpMgr. >>>>>> >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> Konstantin Orlov >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> On 7 Jun 2021, at 14:31, Nikolay Izhikov >>>>>>> wrote: >>>>>>> >>>>>>> Hello, Anton, Alexei. >>>>>>> >>>>>>> Thanks for the feedback. >>>>>>> >>>>>>> Personally, I’m pretty happy current abbreviation rules too. >>>>>>> Let see what we can do to make our codebase even more consistent. >>>>>>> >>>>>>>> 7 июня 2021 г., в 13:23, Alexei Scherbakov < >>>>>> alexey.scherbak...@gmail.com> написал(а): >>>>>>>> >>>>>>>> -1 >>>>>>>> Common abbrevs add quality to the code. >>>>>>>> >>>>>>>> пн, 7 июн. 2021 г. в 12:38, Anton Vinogradov : >>>>>>>> >>>>>>>>> -1 here. >>>>>>>>> We can fix the code and set up the rule. >>>>>>>>> >>>>>>>>> This will help to prevent having a wei
Re: [DISCUSSION] Code style. Variable abbrevations
;> ``` >> > > >>>>>>>> [ERROR] >> > > >>>>>>>> >> > > >>>>> >> > > >>>> >> > > >>> >> > > /Users/sbt-izhikov-nv/work/ignite/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/IgnitePdsWithTtlTest.java:94: >> > > >>>>>>>> Abbrevation should be used for CACHE_NAME_LOCAL_ATOMIC! >> > > >>>>>>>> Please, >> > > use >> > > >>>>> loc, >> > > >>>>>>>> instead of LOCAL [IgniteAbbrevationsRule] >> > > >>>>>>>> [ERROR] >> > > >>>>>>>> >> > > >>>>> >> > > >>>> >> > > >>> >> > > /Users/sbt-izhikov-nv/work/ignite/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/IgnitePdsWithTtlTest.java:97: >> > > >>>>>>>> Abbrevation should be used for CACHE_NAME_LOCAL_TX! Please, >> > > >>>>>>>> use >> > > >>> loc, >> > > >>>>>>>> instead of LOCAL [IgniteAbbrevationsRule] >> > > >>>>>>>> [ERROR] >> > > >>>>>>>> >> > > >>>>> >> > > >>>> >> > > >>> >> > > /Users/sbt-izhikov-nv/work/ignite/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/IgnitePdsWithTtlTest.java:264: >> > > >>>>>>>> Abbrevation should be used for checkpointManager! Please, use >> > > >>>>>>>> mgr, >> > > >>>>> instead >> > > >>>>>>>> of Manager [IgniteAbbrevationsRule] >> > > >>>>>>>> [ERROR] >> > > >>>>>>>> >> > > >>>>> >> > > >>>> >> > > >>> >> > > /Users/sbt-izhikov-nv/work/ignite/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/IgnitePdsPartitionPreloadTest.java:63: >> > > >>>>>>>> Abbrevation should be used for DEFAULT_REGION! Please, use >> > > >>>>>>>> dflt, >> > > >>>>> instead of >> > > >>>>>>>> DEFAULT [IgniteAbbrevationsRule] >> > > >>>>>>>> [ERROR] >> > > >>>>>>>> >> > > >>>>> >> > > >>>> >> > > >>> >> > > /Users/sbt-izhikov-nv/work/ignite/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/IgnitePdsWholeClusterRestartTest.java:47: >> > > >>>>>>>> Abbrevation should be used for ENTRIES_COUNT! Please, use >> > > >>>>>>>> cnt, >> > > >>>> instead >> > > >>>>> of >> > > >>>>>>>> COUNT [IgniteAbbrevationsRule] >> > > >>>>>>>> [ERROR] >> > > >>>>>>>> >> > > >>>>> >> > > >>>> >> > > >>> >> > > /Users/sbt-izhikov-nv/work/ignite/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/IgnitePdsRebalancingOnNotStableTopologyTest.java:49: >> > > >>>>>>>> Abbrevation should be used for CHECKPOINT_FREQUENCY! Please, >> > > >>>>>>>> use >> > > >>>> freq, >> > > >>>>>>>> instead of FREQUENCY [IgniteAbbrevationsRule] >> > > >>>>>>>> [ERROR] >> > > >>>>>>>> >> > > >>>>> >> > > >>>> >> > > >>> >> > > /Users/sbt-izhikov-nv/work/ignite/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/IgnitePdsTransactionsHangTest.java:75: >> > > >>>>>>>> Abbrevation should be used for MAX_KEY_COUNT! Please, use >> > > >>>>>>>> cnt, >> > > >>>> instead >> > > >>>>> of >> > > >>>>>>>> COUNT [IgniteAbbrevationsRule] >> > > >>>>>>>> [ERROR] >> > > >>>>>>>> >> > > >>>>> >> > > >>>> >> > > >>> >> > > /Users/sbt-izhikov-nv/work/ignite/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/IgnitePdsTransactionsHangTest.java:78: >> > > >>>>>>>> Abbrevation should be used for CHECKPOINT_FREQUENCY! Please, >> > > >>>>>>>> use >> > > >>>> freq, >> > > >>>>>>>> instead of FREQUENCY [IgniteAbbrevationsRule] >> > > >>>>>>>> [ERROR] >> > > >>>>>>>> >> > > >>>>> >> > > >>>> >> > > >>> >> > > /Users/sbt-izhikov-nv/work/ignite/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/SlowHistoricalRebalanceSmallHistoryTest.java:57: >> > > >>>>>>>> Abbrevation should be used for SUPPLY_MESSAGE_LATCH! Please, >> > > >>>>>>>> use >> > > >>> msg, >> > > >>>>>>>> instead of MESSAGE [IgniteAbbrevationsRule] >> > > >>>>>>>> [ERROR] >> > > >>>>>>>> >> > > >>>>> >> > > >>>> >> > > >>> >> > > /Users/sbt-izhikov-nv/work/ignite/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/IgniteLogicalRecoveryTest.java:74: >> > > >>>>>>>> Abbrevation should be used for SHARED_GROUP_NAME! Please, use >> > > >>>>>>>> grp, >> > > >>>>> instead >> > > >>>>>>>> of GROUP [IgniteAbbrevationsRule] >> > > >>>>>>>> [ERROR] >> > > >>>>>>>> >> > > >>>>> >> > > >>>> >> > > >>> >> > > /Users/sbt-izhikov-nv/work/ignite/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/IgniteLogicalRecoveryTest.java:200: >> > > >>>>>>>> Abbrevation should be used for cacheLoader! Please, use ldr, >> > > >>> instead >> > > >>>> of >> > > >>>>>>>> Loader [IgniteAbbrevationsRule] >> > > >>>>>>>> ``` >> > > >>>>>>>> >> > > >>>>>>>> [1] >> > > >>>>>>>> >> > > >>>>> >> > > >>>> >> > > >>> >> > > https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules#AbbreviationRules-VariableAbbreviation >> > > >>>>>>>> [2] https://github.com/apache/ignite/pull/9153 >> > > >>>>>>>> >> > > >>>>>>>> >> > > >>>>>>> >> > > >>>>> >> > > >>>>> >> > > >>>> >> > > >>> >> > > >> >> > > >> >> > > >> -- >> > > >> >> > > >> Best regards, >> > > >> Alexei Scherbakov >> > > > >> > > >> > > > -- Best regards, Ivan Pavlukhin
Re: Obsolete classnames.properties
Gmail treated this as spam for me. Bump. 2021-05-18 14:39 GMT+03:00, Данилов Семён : > Hello, Igniters! > > I've recently stumbled across an issue where some tests can't be run locally > due to the absence of some entries in "classnames.properties" file. > For example, TcpRestUnmarshalVulnerabilityTest can't be run locally because > "classnames.properties" lacks > org.apache.ignite.configuration.EncryptionConfiguration and > org.apache.ignite.configuration.PageReplacementMode. On teamcity, however, > such tests pass without any problem, because "classnames.properties" is also > generated by org.apache.ignite.tools.classgen.ClassesGenerator during > process-classes phase in maven build. > > I propose removing this file, as it is generated during the build which > means it's a pointless job keeping it up-to-date in the repository. Also > it's impossible to tell right now if it is or it is not up-to-date (without > executing ClassesGenerator). > > Kind regards, Semyon. > -- Best regards, Ivan Pavlukhin
Re: Greetings
Hi Ilya, Welcome to Apache Ignite Community! I added your account to contributors list in JIRA. Now you can assign ticket to yourself. 2021-05-12 16:29 GMT+03:00, Ilya Korol : > Hello Ignite Community! > > My name isIlya Korol. I want to contribute to Apache Ignite. My JIRA > username*Korol*. > > Thanks! > > -- Best regards, Ivan Pavlukhin
Re: Issue building Ignite 2.10 branch
EN/DependencyResolutionException >> > [ERROR] >> > [ERROR] After correcting the problems, you can resume the build with >> > the >> > command >> > [ERROR] mvn -rf :ignite-jta >> > >> > I found some release notes that indicated the Maven (as of 3.8.1) is no >> > longer supporting HTTP addresses for dependencies. I downgraded to >> > Maven >> > 3.8.0 with no change to the result. >> > >> > Is there an easy work around for this? >> > >> > Thanks, >> > Raymond. >> > >> > >> > -- >> > <http://www.trimble.com/> >> > Raymond Wilson >> > Trimble Distinguished Engineer, Civil Construction Software (CCS) >> > 11 Birmingham Drive | Christchurch, New Zealand >> > raymond_wil...@trimble.com >> > >> > < >> > >> https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch >> > > >> > >> > > > -- > <http://www.trimble.com/> > Raymond Wilson > Trimble Distinguished Engineer, Civil Construction Software (CCS) > 11 Birmingham Drive | Christchurch, New Zealand > raymond_wil...@trimble.com > > <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch> > -- Best regards, Ivan Pavlukhin
Re: Re[2]: [DISCUSSION] MaxLineLength checkstyle rule
+1 for 140 lines compromise. +1 if someone is ready to fix everything to fit 120. BTW, was there any progress with this? 2021-04-15 21:41 GMT+03:00, Maxim Muzafarov : > Folks, > > I've briefly checked the total amount of the max length violations: > > 120 line length - 5540 violations > 130 line length - 1891 violations > 140 line length - 895 violations > 150 line length - 478 violations > > > I think the 140 max line length might be the best option for us. > > On Thu, 15 Apr 2021 at 14:51, Ivan Daschinsky wrote: >> >> But super long lines are a real problem while merging. It is super >> inconvenient. >> 120 chars is a good compromise. >> >> чт, 15 апр. 2021 г. в 14:39, Zhenya Stanilovsky >> > >: >> >> > >> > Python is not so verbose as java ) >> > +1 for 140 >> > >> > >Hi! >> > >Personally, I suppose that 120 chars per line is OK. Moreover, many >> > >codestyles suggests less chars per line. >> > >For example PEP8 recommends 80 (but we use 120 in pyignite and flake8 >> > >codestyle checks it). Google java codestyle insists on 100. >> > > >> > >More than 120 chars is too long as for me and is not convenient for >> > > 3-way >> > >merges. >> > > >> > >чт, 15 апр. 2021 г. в 12:28, Nikolay Izhikov < nizhi...@apache.org >: >> > > >> > >> Hello, Ilya. >> > >> >> > >> Thanks for the feedback. >> > >> >> > >> 140 characters is fine for me. >> > >> >> > >> > 15 апр. 2021 г., в 12:25, Ilya Kasnacheev < >> > >> > ilya.kasnach...@gmail.com >> > > >> > >> написал(а): >> > >> > >> > >> > Hello! >> > >> > >> > >> > Please find attached the distribution of line lengths in the >> > >> > project, >> > in >> > >> the form of (count, line length). >> > >> > >> > >> > I think that we can enforce a hard limit of 140 chars per line. I >> > think >> > >> that having longer lines is excessive and does not benefit >> > >> readability. >> > >> > >> > >> > Having a limit of 150 or 180 does not give us much since there's >> > >> > still >> > >> a long tail which has to be fixed. >> > >> > >> > >> > Regards, >> > >> > -- >> > >> > Ilya Kasnacheev >> > >> > >> > >> > >> > >> > чт, 15 апр. 2021 г. в 11:30, Nikolay Izhikov < nizhi...@apache.org >> > >> > >: >> > >> > Hello, Igniters. >> > >> > >> > >> > Right now, we have a code style rule [1] - the line should fit in >> > >> > 120 >> > >> characters. >> > >> > But, this rule violated in many and many places through code. >> > >> > I have a plan to add a check style rule to force maximum line >> > >> > length. >> > >> > >> > >> > For me, personally, 120 characters a bit old-fashioned >> > >> > restriction. >> > >> > Should we increase the maximum line length to 150 or even 180 >> > characters? >> > >> > >> > >> > [1] >> > https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines >> > >> > >> > >> >> > >> >> > >-- >> > >Sincerely yours, Ivan Daschinskiy >> > >> > >> > >> > >> >> >> >> -- >> Sincerely yours, Ivan Daschinskiy > -- Best regards, Ivan Pavlukhin
Re: Stop sending IGNITE Created e-mails to dev@
Ilya, > You can mention it on the wiki if you know where. It is sad that we intentionally add some magics and secrets to the project. I have not found a good section in wiki for that and added a new dumb section [1]. [1] https://cwiki.apache.org/confluence/display/IGNITE/How+to+subscribe+for+%5BJIRA%5D+%5BCreated%5D+notifications 2021-04-27 20:02 GMT+03:00, Pavel Pereslegin : > Hello Ilya. > > Could you please add me to "Contributors 1" too? > > Thank you. > > вт, 27 апр. 2021 г. в 18:51, Ilya Kasnacheev : >> >> Hello! >> >> I think that role names are per-JIRA global, we can't rename them on a >> per >> project basis. >> >> It exists for very large projects to put more contributors on, we don't >> use >> it so I changed its meaning. You can mention it on the wiki if you know >> where. I'm not sure if it belongs on "how to contribute" because most of >> new contributors don't need new issies feed. >> >> Slava, I have added you to the role also. >> >> Regards, >> -- >> Ilya Kasnacheev >> >> >> вт, 27 апр. 2021 г. в 11:39, Dmitriy Pavlov : >> >> > I guess so. It was always not clear for me why do we have 'contributors >> > 1' >> > role. >> > >> > It might worth to rename role itself to contibutors with notifications. >> > >> > Sincerely, >> > Dmitriy Pavlov >> > >> > пн, 26 апр. 2021 г., 13:01 Ivan Pavlukhin : >> > >> > > Hi Ilya, >> > > >> > > > I have added you to "Contributors 1" role. Everybody in this role >> > > > will >> > > still get those "issue created" e-mail. >> > > >> > > Thank you for that! >> > > >> > > Should not we mention this tweak with "Contributors 1" role somewhere >> > > in advanced how-to-contribute page? >> > > >> > > 2021-04-26 12:28 GMT+03:00, Ilya Kasnacheev >> > > : >> > > > Hello! >> > > > >> > > > I have added you to "Contributors 1" role. Everybody in this role >> > > > will >> > > > still get those "issue created" e-mail. >> > > > >> > > > Feel free in asking me to enlist. >> > > > >> > > > Regards, >> > > > -- >> > > > Ilya Kasnacheev >> > > > >> > > > >> > > > чт, 22 апр. 2021 г. в 18:16, Ivan Pavlukhin : >> > > > >> > > >> > All issues notifications are also sent to >> > > >> > iss...@ignite.apache.org >> > so >> > > >> one can subscribe to this list in order to track the created >> > > >> tickets. >> > > >> >> > > >> Does not sound as useful advice. Issues list [1] looks like real >> > > >> scrapyard, I doubt that it can be usable for anyone in current >> > > >> flavor. >> > > >> Can we send only "Created" notifications there? >> > > >> >> > > >> [1] https://lists.apache.org/list.html?iss...@ignite.apache.org >> > > >> >> > > >> 2021-04-21 18:30 GMT+03:00, Ilya Kasnacheev < >> > ilya.kasnach...@gmail.com >> > > >: >> > > >> > Hello! >> > > >> > >> > > >> > INFRA ticket created: >> > > https://issues.apache.org/jira/browse/INFRA-21762 >> > > >> > >> > > >> > I have asked to keep sending the created issue notifications for >> > > >> > "Contributors 1" role, which is empty at present. So if you wish >> > > >> > to >> > > >> > keep >> > > >> > getting those e-mails, please add yourself to this role or tell >> > > >> > me >> > to >> > > >> > do >> > > >> so >> > > >> > for you. >> > > >> > >> > > >> > Regards, >> > > >> > -- >> > > >> > Ilya Kasnacheev >> > > >> > >> > > >> > >> > > >> > ср, 21 апр. 2021 г. в 17:59, Alexey Goncharuk < >> > > >> alexey.goncha...@gmail.com>: >> > > >> > >> > > >> >> I support the idea. All issues notifications are also sent to >> > >
Re: Stop sending IGNITE Created e-mails to dev@
Hi Ilya, > I have added you to "Contributors 1" role. Everybody in this role will still > get those "issue created" e-mail. Thank you for that! Should not we mention this tweak with "Contributors 1" role somewhere in advanced how-to-contribute page? 2021-04-26 12:28 GMT+03:00, Ilya Kasnacheev : > Hello! > > I have added you to "Contributors 1" role. Everybody in this role will > still get those "issue created" e-mail. > > Feel free in asking me to enlist. > > Regards, > -- > Ilya Kasnacheev > > > чт, 22 апр. 2021 г. в 18:16, Ivan Pavlukhin : > >> > All issues notifications are also sent to iss...@ignite.apache.org so >> one can subscribe to this list in order to track the created tickets. >> >> Does not sound as useful advice. Issues list [1] looks like real >> scrapyard, I doubt that it can be usable for anyone in current flavor. >> Can we send only "Created" notifications there? >> >> [1] https://lists.apache.org/list.html?iss...@ignite.apache.org >> >> 2021-04-21 18:30 GMT+03:00, Ilya Kasnacheev : >> > Hello! >> > >> > INFRA ticket created: https://issues.apache.org/jira/browse/INFRA-21762 >> > >> > I have asked to keep sending the created issue notifications for >> > "Contributors 1" role, which is empty at present. So if you wish to >> > keep >> > getting those e-mails, please add yourself to this role or tell me to >> > do >> so >> > for you. >> > >> > Regards, >> > -- >> > Ilya Kasnacheev >> > >> > >> > ср, 21 апр. 2021 г. в 17:59, Alexey Goncharuk < >> alexey.goncha...@gmail.com>: >> > >> >> I support the idea. All issues notifications are also sent to >> >> iss...@ignite.apache.org so one can subscribe to this list in order to >> >> track the created tickets. The notifications trash the devlist archive >> UI >> >> and make it extremely difficult to navigate. >> >> >> >> вт, 20 апр. 2021 г. в 18:35, Ilya Kasnacheev >> >> > >: >> >> >> >> > Hello, Maxim! >> >> > >> >> > You are free to revert any commit which has led to any new stable >> >> > test >> >> > failure, or new flaky test that was non-flaky before. >> >> > >> >> > Just revert the change and reopen the ticket. >> >> > >> >> > The problem here is that it's very hard to detect on the spot, most >> >> > of >> >> > MTCGA e-mails are false positives and even if they are not, it is >> >> > not >> >> > relevant for most of developers. >> >> > >> >> > WDYT? I'm also still waiting for more input. >> >> > >> >> > Regards, >> >> > -- >> >> > Ilya Kasnacheev >> >> > >> >> > >> >> > ср, 14 апр. 2021 г. в 21:26, Maxim Muzafarov : >> >> > >> >> > > +1 for new JIRA issues >> >> > > -1 for MTCGA notifications >> >> > > >> >> > > Why we should hide errors from the dev-list? Who should take care >> >> > > of >> >> > > issues reported by MTCGA.Bot in this case? >> >> > > We must apply stricter rules for such issues: a commit leading to >> >> > > an >> >> > > error must be reverted. >> >> > > >> >> > > On Wed, 14 Apr 2021 at 20:00, Denis Mekhanikov >> >> > > >> >> > > wrote: >> >> > > > >> >> > > > Huge +1 to this. >> >> > > > >> >> > > > I've already brought up this topic in the past: >> >> > > > >> >> > > >> >> > >> >> >> http://apache-ignite-developers.2346864.n4.nabble.com/Bots-on-dev-list-td34406.html >> >> > > > I hope some day newcomers won't need to set up their email >> >> > > > filters >> >> when >> >> > > > they come to the developers list. >> >> > > > >> >> > > > Denis >> >> > > > >> >> > > > ср, 14 апр. 2021 г. в 18:07, Atri Sharma : >> >> > > > >> >> > > > > +1 to move issues to the issues list. >> >> > > > > >> >> >
Re: Stop sending IGNITE Created e-mails to dev@
Ilya, INFA has apparently accomplished with the ticket [1]. Do we have plans to document somewhere how to subscribe for new ticket notifications? [1] https://issues.apache.org/jira/browse/INFRA-21762 2021-04-22 18:22 GMT+03:00, Dmitriy Pavlov : > +1 for issues > -1 for TC bot. For the latter I siggest improving analysis to avoid odd > emails and leave only relevant > > чт, 22 апр. 2021 г., 18:16 Ivan Pavlukhin : > >> > All issues notifications are also sent to iss...@ignite.apache.org so >> one can subscribe to this list in order to track the created tickets. >> >> Does not sound as useful advice. Issues list [1] looks like real >> scrapyard, I doubt that it can be usable for anyone in current flavor. >> Can we send only "Created" notifications there? >> >> [1] https://lists.apache.org/list.html?iss...@ignite.apache.org >> >> 2021-04-21 18:30 GMT+03:00, Ilya Kasnacheev : >> > Hello! >> > >> > INFRA ticket created: https://issues.apache.org/jira/browse/INFRA-21762 >> > >> > I have asked to keep sending the created issue notifications for >> > "Contributors 1" role, which is empty at present. So if you wish to >> > keep >> > getting those e-mails, please add yourself to this role or tell me to >> > do >> so >> > for you. >> > >> > Regards, >> > -- >> > Ilya Kasnacheev >> > >> > >> > ср, 21 апр. 2021 г. в 17:59, Alexey Goncharuk < >> alexey.goncha...@gmail.com>: >> > >> >> I support the idea. All issues notifications are also sent to >> >> iss...@ignite.apache.org so one can subscribe to this list in order to >> >> track the created tickets. The notifications trash the devlist archive >> UI >> >> and make it extremely difficult to navigate. >> >> >> >> вт, 20 апр. 2021 г. в 18:35, Ilya Kasnacheev >> >> > >: >> >> >> >> > Hello, Maxim! >> >> > >> >> > You are free to revert any commit which has led to any new stable >> >> > test >> >> > failure, or new flaky test that was non-flaky before. >> >> > >> >> > Just revert the change and reopen the ticket. >> >> > >> >> > The problem here is that it's very hard to detect on the spot, most >> >> > of >> >> > MTCGA e-mails are false positives and even if they are not, it is >> >> > not >> >> > relevant for most of developers. >> >> > >> >> > WDYT? I'm also still waiting for more input. >> >> > >> >> > Regards, >> >> > -- >> >> > Ilya Kasnacheev >> >> > >> >> > >> >> > ср, 14 апр. 2021 г. в 21:26, Maxim Muzafarov : >> >> > >> >> > > +1 for new JIRA issues >> >> > > -1 for MTCGA notifications >> >> > > >> >> > > Why we should hide errors from the dev-list? Who should take care >> >> > > of >> >> > > issues reported by MTCGA.Bot in this case? >> >> > > We must apply stricter rules for such issues: a commit leading to >> >> > > an >> >> > > error must be reverted. >> >> > > >> >> > > On Wed, 14 Apr 2021 at 20:00, Denis Mekhanikov >> >> > > >> >> > > wrote: >> >> > > > >> >> > > > Huge +1 to this. >> >> > > > >> >> > > > I've already brought up this topic in the past: >> >> > > > >> >> > > >> >> > >> >> >> http://apache-ignite-developers.2346864.n4.nabble.com/Bots-on-dev-list-td34406.html >> >> > > > I hope some day newcomers won't need to set up their email >> >> > > > filters >> >> when >> >> > > > they come to the developers list. >> >> > > > >> >> > > > Denis >> >> > > > >> >> > > > ср, 14 апр. 2021 г. в 18:07, Atri Sharma : >> >> > > > >> >> > > > > +1 to move issues to the issues list. >> >> > > > > >> >> > > > > For MTCGA, maybe build@? >> >> > > > > >> >> > > > > On Wed, Apr 14, 2021 at 8:35 PM Ilya Kasnacheev >> >> > > > > >>
Re: Stop sending IGNITE Created e-mails to dev@
> All issues notifications are also sent to iss...@ignite.apache.org so one can > subscribe to this list in order to track the created tickets. Does not sound as useful advice. Issues list [1] looks like real scrapyard, I doubt that it can be usable for anyone in current flavor. Can we send only "Created" notifications there? [1] https://lists.apache.org/list.html?iss...@ignite.apache.org 2021-04-21 18:30 GMT+03:00, Ilya Kasnacheev : > Hello! > > INFRA ticket created: https://issues.apache.org/jira/browse/INFRA-21762 > > I have asked to keep sending the created issue notifications for > "Contributors 1" role, which is empty at present. So if you wish to keep > getting those e-mails, please add yourself to this role or tell me to do so > for you. > > Regards, > -- > Ilya Kasnacheev > > > ср, 21 апр. 2021 г. в 17:59, Alexey Goncharuk : > >> I support the idea. All issues notifications are also sent to >> iss...@ignite.apache.org so one can subscribe to this list in order to >> track the created tickets. The notifications trash the devlist archive UI >> and make it extremely difficult to navigate. >> >> вт, 20 апр. 2021 г. в 18:35, Ilya Kasnacheev : >> >> > Hello, Maxim! >> > >> > You are free to revert any commit which has led to any new stable test >> > failure, or new flaky test that was non-flaky before. >> > >> > Just revert the change and reopen the ticket. >> > >> > The problem here is that it's very hard to detect on the spot, most of >> > MTCGA e-mails are false positives and even if they are not, it is not >> > relevant for most of developers. >> > >> > WDYT? I'm also still waiting for more input. >> > >> > Regards, >> > -- >> > Ilya Kasnacheev >> > >> > >> > ср, 14 апр. 2021 г. в 21:26, Maxim Muzafarov : >> > >> > > +1 for new JIRA issues >> > > -1 for MTCGA notifications >> > > >> > > Why we should hide errors from the dev-list? Who should take care of >> > > issues reported by MTCGA.Bot in this case? >> > > We must apply stricter rules for such issues: a commit leading to an >> > > error must be reverted. >> > > >> > > On Wed, 14 Apr 2021 at 20:00, Denis Mekhanikov >> > > >> > > wrote: >> > > > >> > > > Huge +1 to this. >> > > > >> > > > I've already brought up this topic in the past: >> > > > >> > > >> > >> http://apache-ignite-developers.2346864.n4.nabble.com/Bots-on-dev-list-td34406.html >> > > > I hope some day newcomers won't need to set up their email filters >> when >> > > > they come to the developers list. >> > > > >> > > > Denis >> > > > >> > > > ср, 14 апр. 2021 г. в 18:07, Atri Sharma : >> > > > >> > > > > +1 to move issues to the issues list. >> > > > > >> > > > > For MTCGA, maybe build@? >> > > > > >> > > > > On Wed, Apr 14, 2021 at 8:35 PM Ilya Kasnacheev >> > > > > >> > > wrote: >> > > > > > >> > > > > > Hello! >> > > > > > >> > > > > > We have a discussion on how to ensure best engagement in dev@ >> > list, >> > > and >> > > > > it >> > > > > > seems that Issue Created emails from IGNITE project consume a >> > > > > > lot >> > of >> > > > > screen >> > > > > > space, it's hard to spot genuine discussions in >> > > > > > https://lists.apache.org/list.html?dev@ignite.apache.org for >> > > example. >> > > > > > >> > > > > > We already have issues@ mailing list. I propose that we stop >> > > sending any >> > > > > > JIRA emails to dev@. If anyone wishes to get just Created >> emails, >> > > they >> > > > > can >> > > > > > subscribe to these messages in their JIRA account settings. I >> > imagine >> > > > > most >> > > > > > of you already filter these messages out, so you may need to >> adjust >> > > your >> > > > > > filters slightly. >> > > > > > >> > > > > > A distant second is MTCGA messages, which are also >> > > > > > autogenerated >> > and >> > > not >> > > > > > informative for most readers of the channel, since they are at >> best >> > > > > > targeted at a single committer and at worst flaky. >> > > > > > >> > > > > > Where could we move those? What is your opinion here, on both >> > issues? >> > > > > > >> > > > > > Regards, >> > > > > >> > > > > -- >> > > > > Regards, >> > > > > >> > > > > Atri >> > > > > Apache Concerted >> > > > > >> > > >> > >> > -- Best regards, Ivan Pavlukhin
Re: [DISCUSSION] Javadoc style requirements and checks in Ignite-3.
Hi Andrey and Igniters,
Sorry if I misread something but I have totally different opinion in
one aspect. In my mind it is much better in practice to follow strict
rules for public API javadocs but not for internals. I would use
static checks for API packages and disable them for internal ones and
refine code readability during code review. Main motivation here is
that ubiquitous javadocs did not work well in ignite-2 and I believe
it would not in ignite-3.
2021-04-19 13:30 GMT+03:00, Andrey Mashenkov :
> Hi Igniters,
>
> We use JDK Javadoc tool to validate javadocs on TC (Javadoc suite) in
> Ignite 2 and now in Ignite 3.
> A javadoc tool is designed for javadoc site generation that also performs
> some basic checks and markup validation,
> but has nothing to do with javadoc styles.
>
> I've found maven-checkstyle-plugin has modules for javadoc style checking,
> which looks more suited for the issue.
> I've tried to turn on javadoc checks in checkstyle plugin, then run it
> against Ignite-3 main branch and got 1200+ errors.
>
> For Ignite-2 thing may goes worse and numbers can be much higher,
> but let please, start a separate discussion for this if one feels it make
> sense.
>
> Javadoc is part of documentation which a face of product and we MUST keep
> high standards for javadocs as well.
>
> Let's improve this within the ticket [1] regarding the next suggestions:
> 1. Add Javadoc checks to mavan-checkstyle-plugin. I've made a PR for
> Ignite-3 [1] to turn on style checks for javadocs.
>
> 2. Keep the current Javadoc TC suite as is. because it allow detecting
> markup errors regarding current javadoc tool version capabilities.
>
> 3. Fix the Codestyle guidelines to follow higher standards.
> 3.1. Disallow empty javadocs (or with missed description) for member that
> can be used outside the class/package/module by users or other developers.
> I believe all methods/classes/fields that can be used by developers
> (public, protected, package visible) MUST have meaningful description,
> excepts may be tests, well-known constants (e.g. serialVersionUid), and
> private members.
> Actually, it not a big deal to put few words into javadoc even if the
> method looks simple,
> if one feels difficulties expressing a class/method purpose, then most
> likely refactoring is needed.
>
> 3.2. Check all params/throws/returns/generics/deprecates MUST be
> well-documented and goes in order.
>
> 3.4. Suggest to allow optional tags @apiNote, @implSpec, @implNote as
> described in [3],
> to put e.g. expectations/requirements of implementation for developers that
> may be non-obvious.
> The tags values are rendered in separate blocks on javadoc site.
>
> 3.5 However, one-liner javadoc like "{@inheritDoc}" does nothing and can be
> safely omitted. I'd keep it.
>
> 3.6 Add javadoc checks for 'package-info'. Do we want an additional
> requirement to every package has package-info?
>
> Working on the patch I've found it is impossible to have different policies
> in the same config for different scopes: source and test code.
> Thus, we can either exclude tests from style checks at all, which looks
> like not a good idea,
> or have different configs with different policies for source and test code.
>
> Any thoughts?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-14591
> [2] https://github.com/apache/ignite-3/pull/98
> [3] http://openjdk.java.net/jeps/8068562
>
> --
> Best regards,
> Andrey V. Mashenkov
>
--
Best regards,
Ivan Pavlukhin
Re: Stop sending IGNITE Created e-mails to dev@
Hi, Personally I do not like an idea of excluding notifications for created JIRA tickets from dev-list. I used such notifications in following cases: * React to tickets created by Ignite users in my area of responsibility. * Redirect questions created as JIRA tickets to user-list. * Trash spam tickets. Perhaps personally subscribing to "Created" notifications in JIRA is fine (I have not found how), but I am not sure what should be a default behavior. Perhaps we can revisit iss...@ignite.apache.org list content and leave only "Created" notifications there as I doubt that anyone really uses this list today. 2021-04-14 21:26 GMT+03:00, Maxim Muzafarov : > +1 for new JIRA issues > -1 for MTCGA notifications > > Why we should hide errors from the dev-list? Who should take care of > issues reported by MTCGA.Bot in this case? > We must apply stricter rules for such issues: a commit leading to an > error must be reverted. > > On Wed, 14 Apr 2021 at 20:00, Denis Mekhanikov > wrote: >> >> Huge +1 to this. >> >> I've already brought up this topic in the past: >> http://apache-ignite-developers.2346864.n4.nabble.com/Bots-on-dev-list-td34406.html >> I hope some day newcomers won't need to set up their email filters when >> they come to the developers list. >> >> Denis >> >> ср, 14 апр. 2021 г. в 18:07, Atri Sharma : >> >> > +1 to move issues to the issues list. >> > >> > For MTCGA, maybe build@? >> > >> > On Wed, Apr 14, 2021 at 8:35 PM Ilya Kasnacheev >> > wrote: >> > > >> > > Hello! >> > > >> > > We have a discussion on how to ensure best engagement in dev@ list, >> > > and >> > it >> > > seems that Issue Created emails from IGNITE project consume a lot of >> > screen >> > > space, it's hard to spot genuine discussions in >> > > https://lists.apache.org/list.html?dev@ignite.apache.org for example. >> > > >> > > We already have issues@ mailing list. I propose that we stop sending >> > > any >> > > JIRA emails to dev@. If anyone wishes to get just Created emails, >> > > they >> > can >> > > subscribe to these messages in their JIRA account settings. I imagine >> > most >> > > of you already filter these messages out, so you may need to adjust >> > > your >> > > filters slightly. >> > > >> > > A distant second is MTCGA messages, which are also autogenerated and >> > > not >> > > informative for most readers of the channel, since they are at best >> > > targeted at a single committer and at worst flaky. >> > > >> > > Where could we move those? What is your opinion here, on both issues? >> > > >> > > Regards, >> > >> > -- >> > Regards, >> > >> > Atri >> > Apache Concerted >> > > -- Best regards, Ivan Pavlukhin
Re: [ANNOUNCE] Welcome Ivan Daschinsky as a new committer
Ivan, congrats! 2021-04-13 9:41 GMT+03:00, Nikolay Izhikov : > Congrats! Well deserved. > >> 13 апр. 2021 г., в 09:34, Zhenya Stanilovsky >> написал(а): >> >> >> Big deal ! Ivan, ignite it !) >> >> >> >>> The Project Management Committee (PMC) for Apache Ignite has invited >>> Ivan Daschinsky to become a committer and we are pleased to announce >>> that >>> he has accepted. >>> >>> Ivan made a lot of contributions to Apache Ignite. >>> He helped a lot to improve our Python Thin Client fixing a lot of >>> different >>> bugs and contributing major feature such as asyncio support and >>> C-extension >>> which improved performance significantly for many cases. Thanks to Ivan >>> Python >>> Thin client has become much more stable and production-ready. He also >>> introduced the CMake building system for Ignite C++, and has made a >>> number >>> of >>> other important improvements. Besides the code contributions, Ivan is >>> also >>> an >>> active community member. >>> >>> Being a committer enables easier contribution to the project since there >>> is >>> no need to go via the patch submission process. This should enable >>> better >>> productivity. >>> >>> Please join me in welcoming Ivan, and congratulating him on the new role >>> in >>> the Apache Ignite Community. >>> >>> Best Regards, >>> Igor >> >> >> >> > > -- Best regards, Ivan Pavlukhin
Re: [DISCUSSION] IgniteFuture class future in Ignite-3.0.
Hi Andrey, I agree that public API based on CompletableFuture might be very useful. The thing I worry about is that with CompletableFuture used internally we can miss some important capabilities (e.g. mentioned cancellation). ReactiveStreams specification on the other hand defines a lot of interesting and important things and it can serve as guidance for Ignite developers. So, my vision here is that it is very important to have well-defined semantics internally. Introducing public API based on CompletableFuture is fine as well because enough users can be quite happy with it. 2021-04-05 14:49 GMT+03:00, Andrey Mashenkov : > Hi Ivan, > > JDK flow API looks interesting. I think we should use it abroad. > Though, Java 14 is not LTS (long-term support) release. I guess many users > will prefer Java 15, > but actually, we have no agreement about switching to Java 15 which may > require some additional efforts. > For now, we could import the required classes into Ignite module as we've > done with JSR-166 (concurrent collections). > > I think we should use Flow-like API for Queries, Cursors, Compute results > and even Transactions. > E.g. reactive transaction API is must have and it can be looked like: > Transaction.compose(tx -> table1.getAsync(a)).thenApply(tx -> > table2.putAsync()).thenApply(tx -> tx.commit()); > > Agree, CompletableFuture looks totally unusable for cases that supposed > cancellation capabilities, due to broken cancellation semantic [1]. > However, cancellation support for simple table operations doesn't make any > sense and we still can use CompletableFuture for e.g. table operations > unless we found it unusable for Transaction API that is not designed yet. > > I've looked at reactive API as CompletableFuture replacement for simple > cases. > At first glance, it is very complex and may require too much effort on our > user side, and/or hard to understand for the user. > > Maybe it make sense to create an IEP for future replacement with all the > examples for the table, transaction, compute APIs? > WDYT? > > [1] > https://stackoverflow.com/questions/43389894/recursively-cancel-an-allof-completablefuture > > On Sat, Apr 3, 2021 at 1:40 AM Ivan Pavlukhin wrote: > >> Andrey, >> >> As you might know Java has it is own Reactive abstractions since 9 >> [1]. Moreover, an original Reactive Streams library has a bridge with >> Java [2]. >> >> Personally I do not love CompletableFuture because it's API seems >> questionable to me in various aspects, e.g. mentioned cancellation. >> Currently I think that the cleanest way is custom abstractions >> (possibly implementing CompletionStage) at first. And providing >> bridges to other APIs (including CompletableFuture) as a next step. >> Various API integrations can come in form of extensions. >> >> > JDK classes are well-known and reactive frameworks usually have >> converters >> to/from CompletableFuture [1] [2]. >> >> Here I have doubts that it is feasible to achieve it fairly. E.g. once >> again cancellation. While reactive API supports cancellation it will >> not be possible to cancel computation for anything built from >> CompletableFuture. >> >> [1] >> https://docs.oracle.com/en/java/javase/14/docs/api/java.base/java/util/concurrent/Flow.Publisher.html >> [2] >> https://www.reactive-streams.org/reactive-streams-1.0.3-javadoc/org/reactivestreams/FlowAdapters.html >> >> 2021-04-02 12:00 GMT+03:00, Andrey Mashenkov >> : >> > Ivan, >> > thanks for the link. I think, now I've got what you mean. >> > >> > I don't think we want to have any framework as a dependency and rely on >> > their lifecycle, roadmaps goals and >> > bother about compatibility. >> > JDK classes are well-known and reactive frameworks usually have >> converters >> > to/from CompletableFuture [1] [2]. >> > So, users are free to choose any reactive framework. >> > >> > I think will need reactive abstractions in near future for Queries API >> and >> > maybe Transaction API design. >> > These projects are good enough where we can get inspiration. >> > >> > [1] >> > >> https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Mono.html#fromFuture-java.util.concurrent.CompletableFuture- >> > [2] >> > >> https://github.com/ReactiveX/RxJava/blob/3.x/src/main/java/io/reactivex/rxjava3/internal/jdk8/CompletableFromCompletionStage.java >> > >> > On Thu, Apr 1, 2021 at 1:29 PM Ivan Pavlukhin >> wrote: >> > >> >> Andrey, >> >> >> >> > React
Re: [DISCUSSION] IgniteFuture class future in Ignite-3.0.
Andrey, As you might know Java has it is own Reactive abstractions since 9 [1]. Moreover, an original Reactive Streams library has a bridge with Java [2]. Personally I do not love CompletableFuture because it's API seems questionable to me in various aspects, e.g. mentioned cancellation. Currently I think that the cleanest way is custom abstractions (possibly implementing CompletionStage) at first. And providing bridges to other APIs (including CompletableFuture) as a next step. Various API integrations can come in form of extensions. > JDK classes are well-known and reactive frameworks usually have converters to/from CompletableFuture [1] [2]. Here I have doubts that it is feasible to achieve it fairly. E.g. once again cancellation. While reactive API supports cancellation it will not be possible to cancel computation for anything built from CompletableFuture. [1] https://docs.oracle.com/en/java/javase/14/docs/api/java.base/java/util/concurrent/Flow.Publisher.html [2] https://www.reactive-streams.org/reactive-streams-1.0.3-javadoc/org/reactivestreams/FlowAdapters.html 2021-04-02 12:00 GMT+03:00, Andrey Mashenkov : > Ivan, > thanks for the link. I think, now I've got what you mean. > > I don't think we want to have any framework as a dependency and rely on > their lifecycle, roadmaps goals and > bother about compatibility. > JDK classes are well-known and reactive frameworks usually have converters > to/from CompletableFuture [1] [2]. > So, users are free to choose any reactive framework. > > I think will need reactive abstractions in near future for Queries API and > maybe Transaction API design. > These projects are good enough where we can get inspiration. > > [1] > https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Mono.html#fromFuture-java.util.concurrent.CompletableFuture- > [2] > https://github.com/ReactiveX/RxJava/blob/3.x/src/main/java/io/reactivex/rxjava3/internal/jdk8/CompletableFromCompletionStage.java > > On Thu, Apr 1, 2021 at 1:29 PM Ivan Pavlukhin wrote: > >> Andrey, >> >> > Reactive abstractions look more suitable for Queries rather than >> > cache/table async operations and don't offer the power of chaining like >> > CompletableStage. >> >> Could you please elaborate what capabilities do you mean? AFAIK there >> are quite powerful APIs for singular reactive abstractions as well. >> E.g. [1]. >> >> [1] >> https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Mono.html >> >> 2021-04-01 12:33 GMT+03:00, Andrey Mashenkov >> : >> > Val, >> > I just complain JDK CompletableFuture itself is not ideal, but already >> > implemented, well-known and tested. >> > It still a good alternative compared to custom future implementation to >> me. >> > >> > Ok, I feel most of us agree with CompletableFuture as a replacement for >> > custom IgniteFuture. >> > Let's try to use it, but keep in mind that we MUST return a defective >> copy >> > (via copy() or minimalStage()) to user to prevent misusage on the user >> > side. >> > It would be nice if we'll follow the same requirement in our internal >> code, >> > e.g. if a component returns a future that further can be used in other >> > components, especially in custom plugins. >> > >> > Ivan, thanks for the example. >> > Reactive abstractions look more suitable for Queries rather than >> > cache/table async operations and don't offer the power of chaining like >> > CompletableStage. >> > AFAIK, guys who involved in SQL engine development based on Calcite >> already >> > use reactive patterns. >> > >> > >> > On Thu, Apr 1, 2021 at 3:15 AM Ivan Pavlukhin >> wrote: >> > >> >> Folks, >> >> >> >> Regarding Reactive abstractions. While it might look too complex for >> >> simple KV cases it can be quite powerful for queries. Also there are >> >> known solutions for cancellation, backpressure and flow control. It >> >> can greatly simplify things for users familiar with Reactive >> >> programming rather than learning Ignite-specific Query/QueryCursor >> >> API. >> >> >> >> Also it looks like there are well-defined semantics [1]. E.g. >> >> cancellation seems to be defined much better than for >> >> CompletableFuture. >> >> >> >> [1] >> >> https://github.com/reactive-streams/reactive-streams-jvm#specification >> >> >> >> 2021-03-31 21:30 GMT+03:00, Valentin Kulichenko < >> >> valentin.kuliche...
Re: [DISCUSSION] IgniteFuture class future in Ignite-3.0.
Andrey,
> Reactive abstractions look more suitable for Queries rather than
> cache/table async operations and don't offer the power of chaining like
> CompletableStage.
Could you please elaborate what capabilities do you mean? AFAIK there
are quite powerful APIs for singular reactive abstractions as well.
E.g. [1].
[1]
https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Mono.html
2021-04-01 12:33 GMT+03:00, Andrey Mashenkov :
> Val,
> I just complain JDK CompletableFuture itself is not ideal, but already
> implemented, well-known and tested.
> It still a good alternative compared to custom future implementation to me.
>
> Ok, I feel most of us agree with CompletableFuture as a replacement for
> custom IgniteFuture.
> Let's try to use it, but keep in mind that we MUST return a defective copy
> (via copy() or minimalStage()) to user to prevent misusage on the user
> side.
> It would be nice if we'll follow the same requirement in our internal code,
> e.g. if a component returns a future that further can be used in other
> components, especially in custom plugins.
>
> Ivan, thanks for the example.
> Reactive abstractions look more suitable for Queries rather than
> cache/table async operations and don't offer the power of chaining like
> CompletableStage.
> AFAIK, guys who involved in SQL engine development based on Calcite already
> use reactive patterns.
>
>
> On Thu, Apr 1, 2021 at 3:15 AM Ivan Pavlukhin wrote:
>
>> Folks,
>>
>> Regarding Reactive abstractions. While it might look too complex for
>> simple KV cases it can be quite powerful for queries. Also there are
>> known solutions for cancellation, backpressure and flow control. It
>> can greatly simplify things for users familiar with Reactive
>> programming rather than learning Ignite-specific Query/QueryCursor
>> API.
>>
>> Also it looks like there are well-defined semantics [1]. E.g.
>> cancellation seems to be defined much better than for
>> CompletableFuture.
>>
>> [1]
>> https://github.com/reactive-streams/reactive-streams-jvm#specification
>>
>> 2021-03-31 21:30 GMT+03:00, Valentin Kulichenko <
>> valentin.kuliche...@gmail.com>:
>> > Hi Andrey,
>> >
>> > Please see below.
>> >
>> > -Val
>> >
>> > On Wed, Mar 31, 2021 at 7:55 AM Andrey Mashenkov
>> >
>> > wrote:
>> >
>> >> CompletableFuture cancellation will not work as many users expected.
>> >> Javadoc says:
>> >> /* Since (unlike {@link FutureTask}) this class has no direct
>> >> * control over the computation that causes it to be completed,
>> >> * cancellation is treated as just another form of exceptional
>> >> * completion.
>> >> */
>> >>
>> >
>> > Let's not make assumptions about the expectations of the users. That's
>> > exactly why I initially leaned towards a custom interface as well, but
>> > that's a mistake.
>> > Indeed, this contract might look weird to us, because the current
>> > version
>> > of Ignite behaves differently. However, there are much more developers
>> > using CompletableFuture and other standard async frameworks, than
>> > developers using the async functionality of Ignite. Therefore, our
>> > intuitions can easily be wrong.
>> >
>> >
>> >> Completion of a child future doesn't trigger the completion of a
>> >> parent.
>> >> So, we will need to extend the future anyway.
>> >>
>> >
>> > First of all, as Pavel mentioned, you can attach your own listener
>> > before
>> > returning a CompletableFuture to the user. You don't need to extend.
>> > Second of all, there is still a discussion to be had on whether the
>> parent
>> > needs to be completed. I don't actually think it's obvious, and most
>> likely
>> > it's case by case. With CompletableFuture you have enough flexibility
>> > to
>> > control the behavior.
>> >
>> >
>> >>
>> >> Also, cancel() method contract differs from IgniteFuture in 2.0,
>> >> Completable method return true if the future was cancelled,
>> >> but IgniteFuture return true only if it wasn't cancel prior the call.
>> >> Thus, if cancel() was called twice we will have different results for
>> >> CompletableFuture and IgniteFuture,
>> >> that makes CompletableFuture barely usable for our internal purposes.
>> >>
>> >
>> > It doesn't really matter how IgniteFuture in 2.0 be
Re: [DISCUSSION] IgniteFuture class future in Ignite-3.0.
If you look at Ignite-2 code, you may found a number of >> places >> > > > where >> > > > > we >> > > > > > > return e.g. exchange futures or partition release futures. >> > > > > > > > Assume the impact if we will return CompletableFuture >> instead, >> > > > which >> > > > > > can >> > > > > > > be completed in 3-rd party plugin by mistake? >> > > > > > > >> > > > > > > If exchange futures or partition release futures can be >> returned >> > to >> > > > > 3-rd >> > > > > > > party plugin by mistake, it is poor encapsulation. >> > > > > > > And if it will be IgniteFuter rather than CompletedFuture, >> > anyway, >> > > > this >> > > > > > can >> > > > > > > harm. >> > > > > > > >> > > > > > > вт, 30 мар. 2021 г. в 13:14, Andrey Mashenkov < >> > > > > > andrey.mashen...@gmail.com >> > > > > > > >: >> > > > > > > >> > > > > > > > Guys, >> > > > > > > > >> > > > > > > > I want to remember there is one more point to pay attention >> to. >> > > > > > > > Extending Future and CompletableStage is more than just >> > prevents >> > > > > > > unexpected >> > > > > > > > behavior if a user completed the future. >> > > > > > > > >> > > > > > > > First of all, it helps us to write safer code as we won't a >> > > method >> > > > > > > contract >> > > > > > > > exposed such methods as to a user as to a developer. >> > > > > > > > If you look at Ignite-2 code, you may found a number of >> places >> > > > where >> > > > > we >> > > > > > > > return e.g. exchange futures or partition release futures. >> > > > > > > > Assume the impact if we will return CompletableFuture >> instead, >> > > > which >> > > > > > can >> > > > > > > be >> > > > > > > > completed in 3-rd party plugin by mistake? >> > > > > > > > >> > > > > > > > The suggested approach allows us to don't bother if a >> > > > > CompletableFuture >> > > > > > > has >> > > > > > > > to be wrapped or not. >> > > > > > > > >> > > > > > > > >> > > > > > > > On Tue, Mar 30, 2021 at 12:22 PM Alexey Goncharuk < >> > > > > > > > alexey.goncha...@gmail.com> wrote: >> > > > > > > > >> > > > > > > > > Ivan, >> > > > > > > > > >> > > > > > > > > My concern with the concept of a user completing the >> > > > > > > > > future >> > > > > returned >> > > > > > > from >> > > > > > > > > Ignite public API is that it is unclear how to interpret >> this >> > > > > action >> > > > > > > > (this >> > > > > > > > > backs Val's message). >> > > > > > > > > Let's say a user started a compute with fut = >> > > > > compute.runAsync(task); >> > > > > > > and >> > > > > > > > > now calls fut.complete(someVal); Does this mean that >> > > > > > > > > Ignite >> > no >> > > > > longer >> > > > > > > > needs >> > > > > > > > > to execute the task? If the task is currently running, >> > > > > > > > > does >> > it >> > > > need >> > > > > > to >> > > > > > > be >> > > > > > > > > canceled? >> > > > > > > > > >> > > > > > > > > Using CompletableFuture.anyOf() is a good instrument in >> this >> > > case >> > > > > > > because >> > > > > > > > > it makes the 'first fu
Re: [DISCUSSION] IgniteFuture class future in Ignite-3.0.
Val,
There were enough hype around Reactive programming past years. I
remind a lot of talks about RxJava. And I suppose it worth to consider
it. But it requires some time to study modern trends to make a choice.
So far I am not ready to facilitate Reactive API for Ignite 3.
Regarding CompletableFuture.
> The point is that currently a future returned from any of Ignite's async
> operations is supposed to be completed with a value only by Ignite itself,
> not by the user. If we follow the same approach in Ignite 3, returning
> CompletableFuture is surely wrong in my view.
My first thoughts was similar. But later I thought what a user would
like do with returned future. And one of cases I imagined was a case
of alternative result. E.g. a user uses Ignite and another data source
in his application. He wants to use a value arrived faster. He
combines 2 futures like CompletableFuture.anyOf(...). Consequently
even if we prohibit CompletableFuture.complete(...) explicitly then it
will be possible to create a combination that will allow premature
future completion. After all generally CompletableFuture is a
placeholder for async computaion result and if a user wants to
substitute result returned from Ignite why should we disallow him to
do it?
Also I found one more suspicious thing with CompletableFuture. As it
is a concrete class it implements a cancel() method. And as I see the
implementation does not try to cancel underlying computations. Is not
it a problem?
2021-03-29 7:30 GMT+03:00, Valentin Kulichenko :
> Ivan,
>
> It's not really about the "harm", but more about "what should we do if this
> method is called?". Imagine the following code:
>
> CompletableFuture fut = cache.getAsync(key);
> fut.complete("something");
>
> What should happen in this case?
>
> The point is that currently a future returned from any of Ignite's async
> operations is supposed to be completed with a value only by Ignite itself,
> not by the user. If we follow the same approach in Ignite 3, returning
> CompletableFuture is surely wrong in my view.
>
> At the same time, if we take a fundamentally different route with the async
> APIs, this whole discussion might become irrelevant. For example, can you
> elaborate on your thinking around the reactive API? Do you have any
> specifics in mind?
>
> -Val
>
> On Sat, Mar 27, 2021 at 9:18 PM Ivan Pavlukhin wrote:
>
>> > The methods below shouldn't be accessible for user:
>> > complete()
>> > completeExceptionaly()
>>
>> Folks, in case of user-facing API, do you think there is a real harm
>> in allowing a user to manually "complete" a future? I suppose a user
>> employs some post-processing for future results and potentially wants
>> to have control of these results as well. E.g. premature completion in
>> case when a result is no longer needed is possible usage.
>>
>> Also I thinkg it might be a good time to ponder about Future/Promise
>> APIs in general. Why such API is our choice? Can we choose e.g.
>> Reactive API style instead?
>>
>> 2021-03-27 0:33 GMT+03:00, Valentin Kulichenko <
>> valentin.kuliche...@gmail.com>:
>> > Andrey,
>> >
>> > I see. So in a nutshell, you're saying that we want to return a future
>> that
>> > the user's code is not allowed to complete. In this case, I think it's
>> > clear that CompletableFuture is not what we need. We actually need a
>> > NonCompletableFuture :)
>> >
>> > My vote is for the custom interface.
>> >
>> > -Val
>> >
>> > On Fri, Mar 26, 2021 at 2:25 AM Andrey Mashenkov
>> >
>> > wrote:
>> >
>> >> Val,
>> >>
>> >> The methods below shouldn't be accessible for user:
>> >> complete()
>> >> completeExceptionaly()
>> >>
>> >> Returning CompletableFuture we must always make a copy to prevent the
>> >> original future from being completed by mistake.
>> >> I think it will NOT be enough to do that returing the future to the
>> >> end-user, but from every critical module to the outside of the module,
>> >> e.g. to plugins. The impact of disclosing ExchangeFuture,
>> >> PartitionReleaseFuture to plugins may be serious.
>> >>
>> >> IgniteFuture extends Future, CompletionStage which
>> >> implementation
>> >> will just wrap CompletableFuture these issues will be resolved in
>> natural
>> >> way.
>> >> In addition we can force toCompletableFuture() method to return a
>> >> defensive
>> >> copy(), that resolves the last concern.
>>
Re: Significant Items to Tackle
Gmail treated last message as spam. Bumping. 2021-03-28 16:05 GMT+03:00, Данилов Семён : > Hello Atri! > > Would you like to contribute into Ignite Spring integrations? You can take a > look at this particular jira filter: > https://issues.apache.org/jira/browse/IGNITE-9524?jql=project%20%3D%20Ignite%20and%20summary%20~%20%22spring%22%20and%20status%20not%20in%20(Resolved%2C%20Closed) > . > Personally, I think one of the most valuable issues there is > https://issues.apache.org/jira/browse/IGNITE-9524. I have a lot of > experience with Spring, so you can reach out to me if you have any > questions. > > Cheers, Sam. > > 25.03.2021, 23:33, "Atri Sharma" : >> Hi Community, >> >> First off, I want to thank the community for being so welcoming and >> helpful. This is an awesome place to be in. >> >> Now that I have worked on some issues, I would like to get my hands >> deeper >> and with larger issues in the core. Also, some more intermediate tickets >> to >> tackle, my queue has gotten empty :) >> >> Please help and advice. >> >> Regards, >> >> Atri > -- Best regards, Ivan Pavlukhin
Re: [DISCUSSION] IgniteFuture class future in Ignite-3.0.
use it's
>> > >>> standard. But we need to make sure it actually fits our needs and
>> > doesn't
>> > >>> do more harm than good.
>> > >>>
>> > >>> -Val
>> > >>>
>> > >>> On Thu, Mar 25, 2021 at 12:23 PM Alexei Scherbakov <
>> > >>> alexey.scherbak...@gmail.com> wrote:
>> > >>>
>> > >>>> I think both options are fine, but personally lean toward
>> > >>>> CompletableFuture.
>> > >>>>
>> > >>>> чт, 25 мар. 2021 г. в 17:56, Atri Sharma :
>> > >>>>
>> > >>>>> I would suggest using CompletableFuture -- I don't see a need for
>> > >>>>> a
>> > >>>> custom
>> > >>>>> interface that is unique to us.
>> > >>>>>
>> > >>>>> It also allows a lower barrier for new contributors for
>> understanding
>> > >>>>> existing code
>> > >>>>>
>> > >>>>> On Thu, 25 Mar 2021, 20:18 Andrey Mashenkov, <
>> > >>> andrey.mashen...@gmail.com
>> > >>>>>
>> > >>>>> wrote:
>> > >>>>>
>> > >>>>>> Hi Igniters,
>> > >>>>>>
>> > >>>>>> I'd like to start a discussion about replacing our custom
>> > >>> IgniteFuture
>> > >>>>>> class with CompletableFuture - existed JDK class
>> > >>>>>> or rework it's implementation (like some other products done) to
>> > >>>>>> a
>> > >>>>>> composition of CompletionStage and Future interfaces.
>> > >>>>>> or maybe other option if you have any ideas. Do you?
>> > >>>>>>
>> > >>>>>> 1. The first approach pros and cons are
>> > >>>>>> + Well-known JDK class
>> > >>>>>> + Already implemented
>> > >>>>>> - It is a class, not an interface.
>> > >>>>>> - Expose some potentially harmful methods like "complete()".
>> > >>>>>>
>> > >>>>>> On the other side, it has copy() method to create defensive copy
>> > >> and
>> > >>>>>> minimalCompletionStage() to restrict harmful method usage.
>> > >>>>>> Thus, this look like an applicable solution, but we should be
>> > >> careful
>> > >>>>>> exposing internal future to the outside.
>> > >>>>>>
>> > >>>>>> 2. The second approach is to implement our own interface like
>> > >>>>>> the
>> > >>> next
>> > >>>>> one:
>> > >>>>>>
>> > >>>>>> interface IgniteFuture extends CompletableStage, Future
>> > >>>>>> {
>> > >>>>>> }
>> > >>>>>>
>> > >>>>>> Pros and cons are
>> > >>>>>> + Our interfaces/classes contracts will expose an interface
>> > >>>>>> rather
>> > >>> than
>> > >>>>>> concrete implementation.
>> > >>>>>> + All methods are safe.
>> > >>>>>> - Some implementation is required.
>> > >>>>>> - CompletableStage has a method toCompletableFuture() and can be
>> > >>>>> converted
>> > >>>>>> to CompletableFuture. This should be supported.
>> > >>>>>>
>> > >>>>>> However, we still could wrap CompletableFuture and don't bother
>> > >> about
>> > >>>>>> creating a defensive copy.
>> > >>>>>>
>> > >>>>>>
>> > >>>>>> Other project experience:
>> > >>>>>> * Spotify uses CompletableFuture directly [1].
>> > >>>>>> * Redis goes the second approach [2]
>> > >>>>>> * Vertx explicitly extends CompletableFuture [3]. However, they
>> > >> have
>> > >>>>> custom
>> > >>>>>> future classes and a number of helpers that could be replaced
>> > >>>>>> with
>> > >>>>>> CompletableStage. Maybe it is just a legacy.'
>> > >>>>>>
>> > >>>>>> Any thoughts?
>> > >>>>>>
>> > >>>>>> [1]
>> > >>>>>>
>> > >>>>>>
>> > >>>>>
>> > >>>>
>> > >>>
>> > >>
>> >
>> https://spotify.github.io/completable-futures/apidocs/com/spotify/futures/ConcurrencyReducer.html
>> > >>>>>> [2]
>> > >>>>>>
>> > >>>>>>
>> > >>>>>
>> > >>>>
>> > >>>
>> > >>
>> >
>> https://lettuce.io/lettuce-4/release/api/com/lambdaworks/redis/RedisFuture.html
>> > >>>>>> [3]
>> > >>>>>>
>> > >>>>>>
>> > >>>>>
>> > >>>>
>> > >>>
>> > >>
>> >
>> https://javadoc.io/static/org.jspare.vertx/vertx-jspare/1.1.0-M03/org/jspare/vertx/concurrent/VertxCompletableFuture.html
>> > >>>>>> --
>> > >>>>>> Best regards,
>> > >>>>>> Andrey V. Mashenkov
>> > >>>>>>
>> > >>>>>
>> > >>>>
>> > >>>>
>> > >>>> --
>> > >>>>
>> > >>>> Best regards,
>> > >>>> Alexei Scherbakov
>> > >>>>
>> > >>>
>> > >>
>> > >>
>> > >> --
>> > >> Best regards,
>> > >> Alexey
>> > >>
>> >
>> >
>>
>> --
>> Best regards,
>> Andrey V. Mashenkov
>>
>
--
Best regards,
Ivan Pavlukhin
Re: How to Contribute 2021
> >> > > >> > >> > > Do we need this on a page for newcomers? I'd like to mention >> > >> > > that >> > some >> > >> > > of the committers still use the commit script, however, I think >> > >> > > it >> > >> > > will be better to configure the GitHub interaction. >> > >> > > >> > >> > I don't think there's a separate page for committers. If there is, >> > >> please >> > >> > point me to it, and we can remove the section. I don't think we >> > >> > should >> > >> be >> > >> > writing two pages at once, so I decided not to drop any essential >> > >> > information. >> > >> > >> > >> > > Components and their maintainers >> > >> > > >> > >> > > It seems that this list should be updated too. >> > >> > > >> > >> > I would be glad if somebody does it, but I don't have any more >> > >> information >> > >> > to fill there. >> > >> > >> > >> > >> > >> > > > Working on a ticket >> > >> > > I think we should mention the Intellij IDEA checkstyle plugin >> > >> > > and >> > its >> > >> > > configuration (importation of checkstyle.xml to the IDE). >> > >> > > >> > >> > I would be glad if somebody contributes to it, or we may just >> > >> > provide >> > a >> > >> > link to coding guidelines and mention it there. >> > >> > >> > >> > >> > >> > >> > >> > > > GIT workflow >> > >> > > >> > >> > > Do we need it? >> > >> > > >> > >> > I think we do, this workflow is non-trivial and I don't think it >> > >> > is >> > >> > documented anywhere. We can get rid of ASCII art section, though. >> > >> > >> > >> > WDYT? >> > >> > >> > >> > Regards, >> > >> > >> > >> > >> > >> > > >> > >> > > >> > >> > > On Mon, 15 Mar 2021 at 17:25, Ilya Kasnacheev < >> > >> ilya.kasnach...@gmail.com >> > >> > > >> > >> > > wrote: >> > >> > > > >> > >> > > > Hello! >> > >> > > > >> > >> > > > When adding new users to the Contributor role, we usually give >> > them >> > >> a >> > >> > > link >> > >> > > > to "How to Contribute" wiki page. >> > >> > > > >> > >> > > > However, I was feeling that it was in many ways outdated, >> > referring >> > >> to >> > >> > > > outdated development practices and not emphasising TC tests >> > >> > > > and >> > >> MTCGA >> > >> > > bot. >> > >> > > > >> > >> > > > So we took liberty to rewrite this page, meet >> > >> > > > >> > >> > > >> > >> > >> > >> >> > https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute+2021 >> > >> > > > >> > >> > > > We tried to streamline it, make it more friendly to newcomers >> > >> > > > and >> > >> just >> > >> > > > shorter. >> > >> > > > >> > >> > > > Please check it out, share your feelings. >> > >> > > > >> > >> > > > I plan to replace the legacy >> > >> > > > >> > >> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute >> > >> > > with >> > >> > > > this page based on your feedback.. >> > >> > > > >> > >> > > > Regards, >> > >> > > > -- >> > >> > > > Ilya Kasnacheev >> > >> > > >> > >> > >> > >> >> > > >> > > -- Best regards, Ivan Pavlukhin
Re: How to Contribute 2021
Hi Igniters, > I think this guide should be much shorter and simple. As you know we also have CONTRIBUTING.md [1] on GitHub. Perhaps it can be the shortest one for newcomers. [1] https://github.com/apache/ignite/blob/master/CONTRIBUTING.md 2021-03-15 21:56 GMT+03:00, Kseniya Romanova : >> >> > Components and their maintainers >> > It seems that this list should be updated too. >> I would be glad if somebody does it, but I don't have any more >> information >> to fill there. > > > I'll be happy to collect information and update this section. But I also > think this should be a separate page (and howto should have a link on it). > > пн, 15 мар. 2021 г. в 18:25, Pavel Tupitsyn : > >> Ilya, >> >> Thanks for the effort! >> >> I think this guide should be much shorter and simple. >> Right now it is intimidating for newcomers. >> >> What they need is basically >> * Register in Jira, pick a ticket, assign, put In Progress >> * Create a fork, implement >> * Create a PR >> * Ask for review >> >> Maybe we should have a separate, detailed guide for Committers, >> and a simple one for Contributors? >> >> On Mon, Mar 15, 2021 at 6:19 PM Ilya Kasnacheev >> > > >> wrote: >> >> > Hello! >> > >> > Please see inline. >> > >> > пн, 15 мар. 2021 г. в 18:06, Maxim Muzafarov : >> > >> > > Hello, >> > > >> > > >> > > > Ignite employs both Review-Then-Commit processes. >> > > >> > > The Commit-Then-Review (CTR) removed? >> > > >> > I don't see any applications of CTR during the few last years. >> > Streamers >> > were supposed to be CTR but Saikat Maitra still asked for the review of >> > streamers-related commits. >> > >> > > Information for committers >> > > >> > > Do we need this on a page for newcomers? I'd like to mention that >> > > some >> > > of the committers still use the commit script, however, I think it >> > > will be better to configure the GitHub interaction. >> > > >> > I don't think there's a separate page for committers. If there is, >> > please >> > point me to it, and we can remove the section. I don't think we should >> > be >> > writing two pages at once, so I decided not to drop any essential >> > information. >> > >> > > Components and their maintainers >> > > >> > > It seems that this list should be updated too. >> > > >> > I would be glad if somebody does it, but I don't have any more >> information >> > to fill there. >> > >> > >> > > > Working on a ticket >> > > I think we should mention the Intellij IDEA checkstyle plugin and its >> > > configuration (importation of checkstyle.xml to the IDE). >> > > >> > I would be glad if somebody contributes to it, or we may just provide a >> > link to coding guidelines and mention it there. >> > >> > >> > >> > > > GIT workflow >> > > >> > > Do we need it? >> > > >> > I think we do, this workflow is non-trivial and I don't think it is >> > documented anywhere. We can get rid of ASCII art section, though. >> > >> > WDYT? >> > >> > Regards, >> > >> > >> > > >> > > >> > > On Mon, 15 Mar 2021 at 17:25, Ilya Kasnacheev < >> ilya.kasnach...@gmail.com >> > > >> > > wrote: >> > > > >> > > > Hello! >> > > > >> > > > When adding new users to the Contributor role, we usually give them >> > > > a >> > > link >> > > > to "How to Contribute" wiki page. >> > > > >> > > > However, I was feeling that it was in many ways outdated, referring >> to >> > > > outdated development practices and not emphasising TC tests and >> > > > MTCGA >> > > bot. >> > > > >> > > > So we took liberty to rewrite this page, meet >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute+2021 >> > > > >> > > > We tried to streamline it, make it more friendly to newcomers and >> just >> > > > shorter. >> > > > >> > > > Please check it out, share your feelings. >> > > > >> > > > I plan to replace the legacy >> > > > https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute >> > > with >> > > > this page based on your feedback.. >> > > > >> > > > Regards, >> > > > -- >> > > > Ilya Kasnacheev >> > > >> > >> > -- Best regards, Ivan Pavlukhin
Re: [MTCGA]: new failures in builds [5882649] needs to be handled
Igniters, Did not we agree on disabling MVCC suites? Why does TC bot still run them? 2021-02-20 8:16 GMT+03:00, Atri Sharma : > I ran the MVCC Test Suite 7 on branch with my changes a couple of > times with no failures. > > On Sat, Feb 20, 2021 at 8:37 AM wrote: >> >> Hi Igniters, >> >> I've detected some new issue on TeamCity to be handled. You are more than >> welcomed to help. >> >> If your changes can lead to this failure(s): We're grateful that you were >> a volunteer to make the contribution to this project, but things change >> and you may no longer be able to finalize your contribution. >> Could you respond to this email and indicate if you wish to continue and >> fix test failures or step down and some committer may revert you commit. >> >> *New Critical Failure in master-nightly MVCC Cache 7 >> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_MvccCache7?branch=%3Cdefault%3E >> Changes may lead to failure were done by >> - aleksey plekhanov >> https://ci.ignite.apache.org/viewModification.html?modId=917288 >> - atri sharma >> https://ci.ignite.apache.org/viewModification.html?modId=917319 >> - mikhail petrov >> https://ci.ignite.apache.org/viewModification.html?modId=917314 >> >> - Here's a reminder of what contributors were agreed to do >> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute >> - Should you have any questions please contact >> dev@ignite.apache.org >> >> Best Regards, >> Apache Ignite TeamCity Bot >> https://github.com/apache/ignite-teamcity-bot >> Notification generated at 06:06:54 20-02-2021 > > -- > Regards, > > Atri > Apache Concerted > -- Best regards, Ivan Pavlukhin
Re: Disable MVCC test suites
+ 2021-02-06 0:50 GMT+03:00, Maxim Muzafarov : > Folks, > > Since the vote of removing MVCC API [1] completes successfully can we > disable the MVCC suites? > > [1] > http://apache-ignite-developers.2346864.n4.nabble.com/RESULT-VOTE-Removing-MVCC-public-API-tp50705.html > > On Mon, 7 Dec 2020 at 09:38, Ivan Pavlukhin wrote: >> >> Hi Igniters, >> >> > The feature is not production-ready, and I don't think it is used at >> > all. >> >> Unfortunately, we cannot be sure that the feature is not used at all. >> I recall some threads on user list about it. >> >> I understand concerns of those who want to get rid of MVCC. And the >> feature seems to be abandoned today and only increases development >> complexity. So, I think that cleaning up MVCC code is a good idea if >> there is no benefit for the developer community to maintain it today. >> But I suppose such decision should be properly communicated with the >> user community. >> >> 2020-12-02 18:31 GMT+03:00, Alex Plehanov : >> > -1 for disabling test without removing the code. Current tests give us >> > at >> > least "something works" status for the feature available to users, >> > without >> > these tests, we can smoothly move to "totally unusable" status. >> > Complete removal of MVCC can be resource-consuming, but if we want to >> > disable tests at least we should hide the public MVCC API or totally >> > prohibit MVCC usage. Also, it can't be done in 2.x release due to >> > backward >> > compatibility. >> > >> > ср, 2 дек. 2020 г. в 17:28, Nikolay Izhikov : >> > >> >> > Yes, it can be done. However, I don't think that we will get an >> >> agreement on that >> >> >> >> Let’s give it a try and see what happens :) >> >> >> >> >> >> > 2 дек. 2020 г., в 17:23, Вячеслав Коптилин >> >> > >> >> написал(а): >> >> > >> >> >> Can you start the vote? >> >> > Yes, it can be done. However, I don't think that we will get an >> >> > agreement >> >> > on that (I just recall the previous discussion). >> >> > And so, we will not remove the MVCC code; on the other hand, nobody >> >> > will >> >> > support it in the future. We already at this point. This is just my >> >> humble >> >> > opinion. >> >> > >> >> >> It's strange turning off here the whole MVCC tests just because >> >> something >> >> > in the master branch was broken when in the second thread >> >> > On one side, it looks weird, I agree. On the other hand, nobody >> >> > cares >> >> about >> >> > that and wants to fix tests. This is a stalemate, I think. >> >> > >> >> > Thanks, >> >> > S. >> >> > >> >> > ср, 2 дек. 2020 г. в 16:47, Maxim Muzafarov : >> >> > >> >> >> Slava, >> >> >> >> >> >> Can you start the vote? >> >> >> >> >> >> It's strange turning off here the whole MVCC tests just because >> >> >> something in the master branch was broken when in the second thread >> >> >> Community decide to continue MVCC support. Let's start the vote and >> >> >> see what happens. >> >> >> >> >> >> On Wed, 2 Dec 2020 at 16:39, Вячеслав Коптилин < >> >> slava.kopti...@gmail.com> >> >> >> wrote: >> >> >>> >> >> >>>> It will be even worse if our users will face NPE or things like >> >> >>>> that >> >> >> in >> >> >>> the basic MVCC scenarios just because we don’t tests it. >> >> >>> The feature is not production-ready, and I don't think it is used >> >> >>> at >> >> all. >> >> >>> Moreover, MVCC Cache 7, 8, 8, MVCC PDS 1, 2, 4 are already broken >> >> >>> (execution timeouts, flaky test, etc) and I haven't seen anyone >> >> >>> who >> >> would >> >> >>> like to fix this. >> >> >>> Why should we waste every contributor's time? IMHO, MVCC suites >> >> >>> are >> >> >> useless >> >> >>> and everyone just pushes "re-run poss
Re: [Discussion] Kanban board for quick start issues to join the community
Hi, Saikat, will "newbie" label work fine for you? I am curious what benefits of having a board do you see? For example I can think it can be useful if someone would like to see if there is much activity with such tickets. Is it interesting for someone? 2021-01-24 12:20 GMT+03:00, Pavel Tupitsyn : > There are quite a lot of issues with the `newbie` label [1], > we usually recommend them to the new contributors. > > This label is also mentioned on wiki [2] > > [1] > https://issues.apache.org/jira/issues/?jql=project%3Dignite%20and%20labels%3Dnewbie%20and%20status%3Dopen > [2] https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute > > On Sat, Jan 23, 2021 at 8:44 AM Konstantin Boudnik wrote: > >> I guess the same result could be achieved with simple labels, no? >> >> With regards, >>Cos >> >> On 22.01.2021 16:25, Saikat Maitra wrote: >> > Hi, >> > >> > I was looking into our community page to find new open issues that I >> > can >> > contribute to and the easy and moderate tickets, tickets for beginners >> and >> > SQL tasks help me to find open issues to contribute. >> > >> > I am also thinking if we should use a project Kanban board in jira and >> move >> > these issues in the ToDo column so that it is easy to find and >> > contribute >> > to these open issues. >> > >> > I have seen few Apache Project like Beam, Flink already uses Kanban >> > board >> > >> > >> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=BEAM=325 >> > >> > >> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=356=FLINK >> > >> > Please review and let me know your thoughts. >> > >> > Regards >> > Saikat >> > >> > -- Best regards, Ivan Pavlukhin
Re: SHA-512 for Maven deployment
Folks, Were you able to resolve this? 2020-12-28 22:15 GMT+03:00, Valentin Kulichenko : > Hi Ivan, > > Thanks for your response. I've looked into the PGP plugin, and > unfortunately it looks like it only can create signatures, but not > checksums. > > -Val > > On Sun, Dec 27, 2020 at 11:54 PM Ivan Bessonov > wrote: > >> Hi, >> >> I've never done this before, but it seems like we need maven-gpg-plugin >> for >> it [1]. >> >> Algorithm configuration would look like this: >> >> --digest-algo=SHA512 >> >> >> Maybe this will help. >> >> [1] >> >> http://maven.apache.org/plugins-archives/maven-gpg-plugin-LATEST/sign-mojo.html >> >> пн, 28 дек. 2020 г. в 01:25, Valentin Kulichenko < >> valentin.kuliche...@gmail.com>: >> >> > Igniters, >> > >> > I've been preparing the 3.0.0-alpha1 release and got confused about the >> > requirements for checksums in Maven deployments. The Apache instruction >> [1] >> > states that MD5 is deprecated and SHA1 should be avoided in favor of >> > SHA-256 or SHA-512. However, it looks like we are still using the >> MD5/SHA1 >> > combination (at least that's what the staging for 2.9.1 [2] contains). >> > >> > On top of that, I can't find an easy way to switch to another checksum >> > - >> > Maven deploy plugin [3] creates MD5 and SHA1 files automatically and >> > doesn't seem to have any options to tweak this behavior. >> > >> > That said, I have two questions: >> > >> >1. Are we required to use SHA512 or MD5/SHA1 is OK for now? >> >2. Is there a painless way to include SHA512 in addition to >> > MD5/SHA1? >> > >> > Can anyone shed some light on this? >> > >> > [1] https://infra.apache.org/release-signing.html#basic-facts >> > [2] >> > >> > >> https://repository.apache.org/content/repositories/orgapacheignite-1490/org/apache/ignite/ignite-core/2.9.1/ >> > [3] >> https://maven.apache.org/plugins/maven-deploy-plugin/deploy-mojo.html >> > >> > -Val >> > >> >> >> -- >> Sincerely yours, >> Ivan Bessonov >> > -- Best regards, Ivan Pavlukhin
Re: [DISCUSSION] 3.0.0 Alpha release
owse/IGNITE-13914 >> > > > >> https://issues.apache.org/jira/browse/IGNITE-13924 >> > > > >> https://issues.apache.org/jira/browse/IGNITE-13921 >> > > > >> https://issues.apache.org/jira/browse/IGNITE-13922 >> > > > >> https://issues.apache.org/jira/browse/IGNITE-13920 >> > > > >> >> > > > >> >> > > > >> On Sun, Dec 27, 2020 at 2:18 AM Valentin Kulichenko < >> > > > >> valentin.kuliche...@gmail.com> wrote: >> > > > >> >> > > > >> > Folks, >> > > > >> > >> > > > >> > I've created a version in Jira and a Wiki page for the alpha >> > > release: >> > > > >> > >> > > > >> > >> > > > >> >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0.0-alpha1 >> > > > >> > >> > > > >> > Based on the scope described earlier, all code changes are >> > > completed - >> > > > >> only >> > > > >> > documentation is left. Code freeze starts now - only bug fixes >> are >> > > > >> > permitted. >> > > > >> > >> > > > >> > I'm planning to start the vote next week, presumably on >> > > > >> > Tuesday. >> > > > >> > >> > > > >> > -Val >> > > > >> > >> > > > >> > On Tue, Dec 22, 2020 at 9:56 AM Valentin Kulichenko < >> > > > >> > valentin.kuliche...@gmail.com> wrote: >> > > > >> > >> > > > >> > > Ivan, >> > > > >> > > >> > > > >> > > Yes, you are correct - no tables, caches, compute or any >> > > > >> > > other >> > > major >> > > > >> > > features will be available. >> > > > >> > > >> > > > >> > > Alpha releases are generally produced in the middle of the >> > > > development >> > > > >> > > process for internal testing. In my view, that's exactly >> > > > >> > > what >> > the >> > > > >> > upcoming >> > > > >> > > release is, but let me know if you have a better naming in >> mind. >> > > > >> > > >> > > > >> > > -Val >> > > > >> > > >> > > > >> > > On Tue, Dec 22, 2020 at 1:56 AM Ivan Pavlukhin < >> > > vololo...@gmail.com >> > > > > >> > > > >> > > wrote: >> > > > >> > > >> > > > >> > >> Hi, >> > > > >> > >> >> > > > >> > >> Is it right that suggested Alpha will not be able to >> > > > >> > >> perform >> > any >> > > > >> > >> operations with data (tables, caches)? In that case Alpha >> > naming >> > > > >> > >> confuses me. Actually do not know how such kind of demo >> > releases >> > > > >> > >> should be attributed. >> > > > >> > >> >> > > > >> > >> 2020-12-21 20:42 GMT+03:00, Denis Magda >> > > > >> > >> : >> > > > >> > >> > Certainly, the end of Jan is more realistic and >> > > > >> > >> > reasonable. >> > > Let's >> > > > >> talk >> > > > >> > >> this >> > > > >> > >> > out offline and then put a session on the virtual >> > > > >> > >> > meetup's >> > > > >> schedule. >> > > > >> > >> > >> > > > >> > >> > - >> > > > >> > >> > Denis >> > > > >> > >> > >> > > > >> > >> > >> > > > >> > >> > On Sun, Dec 20, 2020 at 4:35 PM Valentin Kulichenko < >> > > > >> > >> > valentin.kuliche...@gmail.com> wrote: >> > &
Re: [ANNOUNCE] Welcome Ivan Bessonov as a new committer
Ivan, congratulations! 2021-01-12 13:36 GMT+03:00, Kseniya Romanova : > Ivan, my congratulations! > > вт, 12 янв. 2021 г. в 13:32, Andrey Gura : > >> Igniters, >> >> The Apache Ignite Project Management Committee (PMC) has invited Ivan >> Bessonov to become a new committer and are happy to announce that he >> has accepted. >> >> Ivan has a lot of contributions to the Apache Ignite project. He >> implemented Distributed Meta Storage feature which is one of key >> features in the project. Also he contributed non trivial >> improvements to such important parts of the project as Communication, >> Discovery, Page Memory and Native Persistence. Another simple, but >> very useful contribution is the widely used @WithSystemProperty >> annotation for Apache Ignite test framework >> >> Being a committer enables easier contribution to the project since there >> is >> no need to go via the patch submission process. This should enable better >> productivity. >> >> Please join me in welcoming Ivan, and congratulating him on the new role >> in >> the Apache Ignite Community. >> >> Best Regards, >> Andrey Gura >> > -- Best regards, Ivan Pavlukhin
Re: [COMMUNITY] Recognizing those who contribute to user support and project awareness
Great news! Thanks and Happy holidays! 2020-12-30 12:10 GMT+03:00, Alexey Zinoviev : > Thanks for recognition! Happy New Year! > > вт, 29 дек. 2020 г. в 21:36, Kseniya Romanova : > >> Hi, Igniters! >> >> >> ASF welcomes all types of contributions, including responses to user >> questions and promotion of projects. And, the GriGain DelRel team knows >> that people spend a lot of their private time on such activities. Our >> code >> and documentation contributors are recognized by being promoted to the >> role >> of " committer." But, unfortunately, we don't have a formal way to >> recognize our user-support and project-promotion contributors. >> >> >> In 2020, more than 40 people contributed their time and experience to >> promote project visibility and encourage adoption by developers: >> >> >> >>- >> >>59 talks to 2400 developers and architects at meetup and conference >>events >>- >> >>60+ blog posts and tutorials >>- >> >>2000+ questions answered on the User list and Stackoverflow >> >> >> Let’s praise some of our most active Ignite supporters in 2020: >> >> >> >>- >> >>In user support, Ilya Kasnacheev is our absolute user support >> champion. >>He alone answered 756 questions on Stackoverflow and the User list. >>- >> >>Val Kulichenko posted 9 articles and presented at 6 meetups. >>- >> >>Denis Magda created 10 demos and 4 tutorials and presented at 14 >> meetups >>and 7 conferences. >> - >> >>Alexey Zinoviev wrote 4 articles about Ignite ML and presented at 1 >>meetup and 1 conference. >>- >> >>Pavel Tupitsyn created 4 articles and helped Ignite users on the >> mailing >>list and Stackoverflow. >>- >> >>Saikat Maitra presented at 2 conferences. >>- >> >>Ivan Pavlukhin helped many new developers to start contributing to >>Ignite. >> >> >> As a sign of our gratitude for their efforts, these contributors will >> receive special New Year gifts. >> >> >> In 2021, we want to track this type of contribution and ensure that the >> contributors are recognized within the community. The Alfa version of our >> tracking system is already functioning inside GridGain, so the next step >> is >> to upgrade the current system and start a discussion with PMC members. >> Those who help to attract and retain users are definitely worthy of >> praise >> and promotion, as the value of their contribution matches the value that >> is >> provided through code and documentation. >> >> >> Keep being awesome and have a Happy New Year! >> >> >> Kseniya >> >> DevRel at GridGain >> > -- Best regards, Ivan Pavlukhin
Re: [DISCUSS] Notifications for ignite-3 repository
Hi Mirza, Actually recent PR has redirected notifications for https://github.com/apache/ignite-python-thin-client Unfortunately there are other repositories. Currently I do not know if it is possible to configure all Ignite repositories at once. 2020-12-28 16:49 GMT+03:00, Mirza Aliev : > I've just realized that those notifications were > from ignite-python-thin-client, probably we should do the same trick with > all ignite repos. > > пн, 28 дек. 2020 г. в 16:44, Мирза Алиев : > >> Seems, that this doesn't help, I'm still receiving notifications after >> Ivan's PR was merged >> >> ср, 23 дек. 2020 г. в 21:55, Pavel Tupitsyn : >> >>> Ivan, thank you for fixing this! >>> >>> On Wed, Dec 23, 2020 at 11:13 AM Ivan Pavlukhin >>> wrote: >>> >>> > Actually a first attempt was not successful. And the reason was "main" >>> > branch name. I asked INFRA folks assistance and as I understood they >>> > configured GitBox HEAD to be "main" branch HEAD. I suppose it did the >>> > trick and we should not see ignite-3 PR notifications on dev-list >>> > anymore. >>> > >>> > 2020-12-22 13:05 GMT+03:00, Ivan Pavlukhin : >>> > > Ticket was merged. Thanks to all participants. >>> > > >>> > > 2020-12-21 10:12 GMT+03:00, Ivan Pavlukhin : >>> > >> Folks, >>> > >> >>> > >> I will merge the PR tomorrow if there is no objections. >>> > >> >>> > >> 2020-12-19 0:56 GMT+03:00, Valentin Kulichenko >>> > >> : >>> > >>> Folks, >>> > >>> >>> > >>> Can someone take a look at this PR? I'm generally OK with the >>> change, >>> > >>> but >>> > >>> I'm not familiar with the mechanisms used here. >>> > >>> >>> > >>> -Val >>> > >>> >>> > >>> On Thu, Dec 17, 2020 at 11:16 PM Ivan Pavlukhin < >>> vololo...@gmail.com> >>> > >>> wrote: >>> > >>> >>> > >>>> Igniters, >>> > >>>> >>> > >>>> I noticed notifications about PRs in ignite-3 repository sent to >>> > >>>> dev-list. I suppose we should configure notifications similar to >>> main >>> > >>>> ignite repository. I prepared a ticket [1] and PR for this. >>> > >>>> Please >>> > >>>> review. >>> > >>>> >>> > >>>> [1] https://issues.apache.org/jira/browse/IGNITE-13875 >>> > >>>> >>> > >>>> -- >>> > >>>> >>> > >>>> Best regards, >>> > >>>> Ivan Pavlukhin >>> > >>>> >>> > >>> >>> > >> >>> > >> >>> > >> -- >>> > >> >>> > >> Best regards, >>> > >> Ivan Pavlukhin >>> > >> >>> > > >>> > > >>> > > -- >>> > > >>> > > Best regards, >>> > > Ivan Pavlukhin >>> > > >>> > >>> > >>> > -- >>> > >>> > Best regards, >>> > Ivan Pavlukhin >>> > >>> >> > -- Best regards, Ivan Pavlukhin
Re: [DISCUSS] Notifications for ignite-3 repository
Actually a first attempt was not successful. And the reason was "main" branch name. I asked INFRA folks assistance and as I understood they configured GitBox HEAD to be "main" branch HEAD. I suppose it did the trick and we should not see ignite-3 PR notifications on dev-list anymore. 2020-12-22 13:05 GMT+03:00, Ivan Pavlukhin : > Ticket was merged. Thanks to all participants. > > 2020-12-21 10:12 GMT+03:00, Ivan Pavlukhin : >> Folks, >> >> I will merge the PR tomorrow if there is no objections. >> >> 2020-12-19 0:56 GMT+03:00, Valentin Kulichenko >> : >>> Folks, >>> >>> Can someone take a look at this PR? I'm generally OK with the change, >>> but >>> I'm not familiar with the mechanisms used here. >>> >>> -Val >>> >>> On Thu, Dec 17, 2020 at 11:16 PM Ivan Pavlukhin >>> wrote: >>> >>>> Igniters, >>>> >>>> I noticed notifications about PRs in ignite-3 repository sent to >>>> dev-list. I suppose we should configure notifications similar to main >>>> ignite repository. I prepared a ticket [1] and PR for this. Please >>>> review. >>>> >>>> [1] https://issues.apache.org/jira/browse/IGNITE-13875 >>>> >>>> -- >>>> >>>> Best regards, >>>> Ivan Pavlukhin >>>> >>> >> >> >> -- >> >> Best regards, >> Ivan Pavlukhin >> > > > -- > > Best regards, > Ivan Pavlukhin > -- Best regards, Ivan Pavlukhin
Re: [DISCUSS] Ignite 3.0 development approach
Pavel, thanks for explanation! 2020-12-22 13:34 GMT+03:00, Pavel Tupitsyn : > Ivan, it is the new GitHub default > > "On Oct. 1, 2020, any new repositories you create will use main as the > default branch, instead of master" [1] > > [1] > https://www.zdnet.com/article/github-to-replace-master-with-main-starting-next-month/ > > On Tue, Dec 22, 2020 at 1:12 PM Ivan Pavlukhin wrote: > >> Also I noticed that ignite-3 repository has "main" but not "master" >> branch. Who can shed light on this? Did not find an explanation in >> this thread. >> >> 2020-12-22 13:09 GMT+03:00, Ivan Pavlukhin : >> > I noticed some free-from commit messages in ignite-3 repository. I >> > think we should use ticket-based workflow and commit messages as >> > usual. >> > >> > [1] https://github.com/apache/ignite-3/commits/main >> > >> > 2020-12-21 10:55 GMT+03:00, Petr Ivanov : >> >> There is no problem to have both in new repository, if skilled >> enthusiast >> >> will take over that job. >> >> >> >> I guess we will stick to Maven for time being but development of >> >> Gradle-based building system can be done in parallel. >> >> I can even add corresponding development build configurations for >> >> TeamCity, >> >> or even introduce some kind of switch — so that we can test old and >> >> new >> >> build approaches and provide seamless transition if we agree on that. >> >> >> >>> On 19 Dec 2020, at 01:00, Valentin Kulichenko >> >>> wrote: >> >>> >> >>> Hi Ivan, >> >>> >> >>> There was a very brief discussion around this. Basically, it looks >> >>> like >> >>> switching from Maven to something else is not going to bring much >> value, >> >>> but at the same time will be quite demanding because the community >> >>> has >> >>> much >> >>> more experience with Maven. However, I would say that it is still >> >>> debatable at this point -- please feel free to share your thoughts on >> >>> this. >> >>> >> >>> -Val >> >>> >> >>> On Thu, Dec 17, 2020 at 10:53 PM Ivan Pavlukhin >> >>> wrote: >> >>> >> >>>> Hi Igniters, >> >>>> >> >>>> Forgive me that I am not reading dev list carefully these days. Was >> >>>> it >> >>>> explicitly decided that Maven should be used as a build system for >> >>>> Ignite 3? As there is a new repository we possibly can update our >> >>>> build tools as well. What do you think? >> >>>> >> >>>> 2020-12-17 22:45 GMT+03:00, Valentin Kulichenko < >> >>>> valentin.kuliche...@gmail.com>: >> >>>>> Hi Dmitriy, >> >>>>> >> >>>>> I don't think there is any reason for concern at this point. The >> >>>> community >> >>>>> agreed on the scope of the changes for 3.0 - it is described on >> >>>>> Wiki. >> >>>>> The >> >>>>> scope is quite big, so it is clear that 2.x and 3.x will have to >> exist >> >>>>> in >> >>>>> parallel for a significant amount of time, so we need a place where >> we >> >>>> can >> >>>>> merge the code for 3.x. Thus, I've created this new repo. We >> >>>>> already >> >>>>> have >> >>>>> multiple IEPs, as well as several contributors who are actively >> >>>>> involved >> >>>> in >> >>>>> development. Some of the first PRs were merged today. >> >>>>> >> >>>>> I didn't hear any objections since the repo was created. >> >>>>> >> >>>>> -Val >> >>>>> >> >>>>> On Wed, Dec 16, 2020 at 7:28 AM Dmitriy Pavlov >> >>>> wrote: >> >>>>> >> >>>>>> Folks, I'm a little bit concerned about the simultaneous >> >>>>>> - existence of the repo https://github.com/apache/ignite-3 and PRs >> >>>>>> for >> >>>>>> that >> >>>>>> repo >> >>>>>> - and a couple of downvotes from PMC mem
Re: [DISCUSS] Ignite 3.0 development approach
Also I noticed that ignite-3 repository has "main" but not "master" branch. Who can shed light on this? Did not find an explanation in this thread. 2020-12-22 13:09 GMT+03:00, Ivan Pavlukhin : > I noticed some free-from commit messages in ignite-3 repository. I > think we should use ticket-based workflow and commit messages as > usual. > > [1] https://github.com/apache/ignite-3/commits/main > > 2020-12-21 10:55 GMT+03:00, Petr Ivanov : >> There is no problem to have both in new repository, if skilled enthusiast >> will take over that job. >> >> I guess we will stick to Maven for time being but development of >> Gradle-based building system can be done in parallel. >> I can even add corresponding development build configurations for >> TeamCity, >> or even introduce some kind of switch — so that we can test old and new >> build approaches and provide seamless transition if we agree on that. >> >>> On 19 Dec 2020, at 01:00, Valentin Kulichenko >>> wrote: >>> >>> Hi Ivan, >>> >>> There was a very brief discussion around this. Basically, it looks like >>> switching from Maven to something else is not going to bring much value, >>> but at the same time will be quite demanding because the community has >>> much >>> more experience with Maven. However, I would say that it is still >>> debatable at this point -- please feel free to share your thoughts on >>> this. >>> >>> -Val >>> >>> On Thu, Dec 17, 2020 at 10:53 PM Ivan Pavlukhin >>> wrote: >>> >>>> Hi Igniters, >>>> >>>> Forgive me that I am not reading dev list carefully these days. Was it >>>> explicitly decided that Maven should be used as a build system for >>>> Ignite 3? As there is a new repository we possibly can update our >>>> build tools as well. What do you think? >>>> >>>> 2020-12-17 22:45 GMT+03:00, Valentin Kulichenko < >>>> valentin.kuliche...@gmail.com>: >>>>> Hi Dmitriy, >>>>> >>>>> I don't think there is any reason for concern at this point. The >>>> community >>>>> agreed on the scope of the changes for 3.0 - it is described on Wiki. >>>>> The >>>>> scope is quite big, so it is clear that 2.x and 3.x will have to exist >>>>> in >>>>> parallel for a significant amount of time, so we need a place where we >>>> can >>>>> merge the code for 3.x. Thus, I've created this new repo. We already >>>>> have >>>>> multiple IEPs, as well as several contributors who are actively >>>>> involved >>>> in >>>>> development. Some of the first PRs were merged today. >>>>> >>>>> I didn't hear any objections since the repo was created. >>>>> >>>>> -Val >>>>> >>>>> On Wed, Dec 16, 2020 at 7:28 AM Dmitriy Pavlov >>>> wrote: >>>>> >>>>>> Folks, I'm a little bit concerned about the simultaneous >>>>>> - existence of the repo https://github.com/apache/ignite-3 and PRs >>>>>> for >>>>>> that >>>>>> repo >>>>>> - and a couple of downvotes from PMC members. >>>>>> >>>>>> Is it all fine here? Was there any vote /discussion where it was >>>>>> discussed >>>>>> and consensus approved? What is the status of the ignite-3 repo? >>>>>> >>>>>> вт, 15 дек. 2020 г. в 17:30, Carbone, Adam >>>>>> >>>> : >>>>>> >>>>>>> I don't believe Java 7 was LTS, and I hope that others will have >>>>>>> upgraded >>>>>>> long before that. If that is the release timeframe for 3.0, then I >>>>>> suppose >>>>>>> that would makes sense, I would still doubt that people would be >>>>>>> going >>>>>>> newer than java 11, just my opinion of what I'm seeing. >>>>>>> >>>>>>> Regards >>>>>>> ~adam >>>>>>> >>>>>>> Adam Carbone | Director of Innovation – Intelligent Platform Team | >>>>>>> Bottomline Technologies >>>>>>> Office: 603-501-6446 | Mobile: 603-570-8418 >>>>>>> www.bottomline.com >>>>>>> &g
Re: [DISCUSS] Ignite 3.0 development approach
I noticed some free-from commit messages in ignite-3 repository. I think we should use ticket-based workflow and commit messages as usual. [1] https://github.com/apache/ignite-3/commits/main 2020-12-21 10:55 GMT+03:00, Petr Ivanov : > There is no problem to have both in new repository, if skilled enthusiast > will take over that job. > > I guess we will stick to Maven for time being but development of > Gradle-based building system can be done in parallel. > I can even add corresponding development build configurations for TeamCity, > or even introduce some kind of switch — so that we can test old and new > build approaches and provide seamless transition if we agree on that. > >> On 19 Dec 2020, at 01:00, Valentin Kulichenko >> wrote: >> >> Hi Ivan, >> >> There was a very brief discussion around this. Basically, it looks like >> switching from Maven to something else is not going to bring much value, >> but at the same time will be quite demanding because the community has >> much >> more experience with Maven. However, I would say that it is still >> debatable at this point -- please feel free to share your thoughts on >> this. >> >> -Val >> >> On Thu, Dec 17, 2020 at 10:53 PM Ivan Pavlukhin >> wrote: >> >>> Hi Igniters, >>> >>> Forgive me that I am not reading dev list carefully these days. Was it >>> explicitly decided that Maven should be used as a build system for >>> Ignite 3? As there is a new repository we possibly can update our >>> build tools as well. What do you think? >>> >>> 2020-12-17 22:45 GMT+03:00, Valentin Kulichenko < >>> valentin.kuliche...@gmail.com>: >>>> Hi Dmitriy, >>>> >>>> I don't think there is any reason for concern at this point. The >>> community >>>> agreed on the scope of the changes for 3.0 - it is described on Wiki. >>>> The >>>> scope is quite big, so it is clear that 2.x and 3.x will have to exist >>>> in >>>> parallel for a significant amount of time, so we need a place where we >>> can >>>> merge the code for 3.x. Thus, I've created this new repo. We already >>>> have >>>> multiple IEPs, as well as several contributors who are actively >>>> involved >>> in >>>> development. Some of the first PRs were merged today. >>>> >>>> I didn't hear any objections since the repo was created. >>>> >>>> -Val >>>> >>>> On Wed, Dec 16, 2020 at 7:28 AM Dmitriy Pavlov >>> wrote: >>>> >>>>> Folks, I'm a little bit concerned about the simultaneous >>>>> - existence of the repo https://github.com/apache/ignite-3 and PRs for >>>>> that >>>>> repo >>>>> - and a couple of downvotes from PMC members. >>>>> >>>>> Is it all fine here? Was there any vote /discussion where it was >>>>> discussed >>>>> and consensus approved? What is the status of the ignite-3 repo? >>>>> >>>>> вт, 15 дек. 2020 г. в 17:30, Carbone, Adam >>>>> >>> : >>>>> >>>>>> I don't believe Java 7 was LTS, and I hope that others will have >>>>>> upgraded >>>>>> long before that. If that is the release timeframe for 3.0, then I >>>>> suppose >>>>>> that would makes sense, I would still doubt that people would be >>>>>> going >>>>>> newer than java 11, just my opinion of what I'm seeing. >>>>>> >>>>>> Regards >>>>>> ~adam >>>>>> >>>>>> Adam Carbone | Director of Innovation – Intelligent Platform Team | >>>>>> Bottomline Technologies >>>>>> Office: 603-501-6446 | Mobile: 603-570-8418 >>>>>> www.bottomline.com >>>>>> >>>>>> >>>>>> >>>>>> On 12/15/20, 4:25 AM, "Ilya Kasnacheev" >>>>>> wrote: >>>>>> >>>>>>Hello! >>>>>> >>>>>>I guess Ignite 3.0 will be ready for production use somewhere in >>>>> 2022, >>>>>>realistically. By that time, Java 8 will be long enough out of >>>>> support >>>>>> so >>>>>>that most companies will actually forbid its use, WRT >>>>>> vulnerabilities >
Re: [DISCUSS] Notifications for ignite-3 repository
Ticket was merged. Thanks to all participants. 2020-12-21 10:12 GMT+03:00, Ivan Pavlukhin : > Folks, > > I will merge the PR tomorrow if there is no objections. > > 2020-12-19 0:56 GMT+03:00, Valentin Kulichenko > : >> Folks, >> >> Can someone take a look at this PR? I'm generally OK with the change, but >> I'm not familiar with the mechanisms used here. >> >> -Val >> >> On Thu, Dec 17, 2020 at 11:16 PM Ivan Pavlukhin >> wrote: >> >>> Igniters, >>> >>> I noticed notifications about PRs in ignite-3 repository sent to >>> dev-list. I suppose we should configure notifications similar to main >>> ignite repository. I prepared a ticket [1] and PR for this. Please >>> review. >>> >>> [1] https://issues.apache.org/jira/browse/IGNITE-13875 >>> >>> -- >>> >>> Best regards, >>> Ivan Pavlukhin >>> >> > > > -- > > Best regards, > Ivan Pavlukhin > -- Best regards, Ivan Pavlukhin
Re: [DISCUSSION] 3.0.0 Alpha release
Hi, Is it right that suggested Alpha will not be able to perform any operations with data (tables, caches)? In that case Alpha naming confuses me. Actually do not know how such kind of demo releases should be attributed. 2020-12-21 20:42 GMT+03:00, Denis Magda : > Certainly, the end of Jan is more realistic and reasonable. Let's talk this > out offline and then put a session on the virtual meetup's schedule. > > - > Denis > > > On Sun, Dec 20, 2020 at 4:35 PM Valentin Kulichenko < > valentin.kuliche...@gmail.com> wrote: > >> Hi Denis, >> >> Sure, that's a great idea! I'm thinking, however, that it makes sense to >> take a pause after the release and let people try it out on their own so >> that the experience is as close to real-life as possible. How about >> closer >> to the end of January? >> >> -Val >> >> On Fri, Dec 18, 2020 at 8:53 PM Denis Magda wrote: >> >> > Hi Val, >> > >> > That's the pace, I'll be happy to play with the alpha and share >> > feedback. >> > Also, what if we arrange a community gathering after the holidays to >> > demonstrate what alpha does and get more details from the involved >> > contributors on what's coming next? >> > >> > As for the release process, yes, that needs to be a formal vote as long >> as >> > you're publishing artifacts to Maven and updating the website with >> > references to the binaries and documentation pages. >> > >> > >> > - >> > Denis >> > >> > >> > On Fri, Dec 18, 2020 at 6:13 PM Valentin Kulichenko < >> > valentin.kuliche...@gmail.com> wrote: >> > >> > > Igniters, >> > > >> > > The repository for Ignite 3.x [1] was created less than a month ago, >> and >> > we >> > > already have several merged pull requests. Many thanks to everyone >> > involved >> > > for the contributions! >> > > >> > > Currently, we have the following functionality available in the main >> > > branch: >> > > >> > >- New configuration framework which is compatible with the HOCON >> > format. >> > >- The “ignite-runner” module, which currently incorporates the >> > >aforementioned configuration framework and REST endpoints to >> interact >> > > with >> > >the framework. >> > >- The first version of the unified CLI tool. >> > > >> > > This set of features does not include any actual Ignite APIs. >> > > However, >> it >> > > clearly demonstrates the basic mechanics of how the product will be >> > > installed and upgraded, how the user will interact with it, etc. I >> would >> > > like to use this to gather feedback from the community in the early >> > stages. >> > > >> > > To make sure the experience is as close to real life as possible, I >> want >> > to >> > > deliver what we have as an alpha release, meaning that all the >> > > required >> > > binaries will be deployed on the website and in Maven. There will >> > > also >> be >> > > some basic documentation describing how to install and use the >> > > product. >> > > This way, anyone will be able to download the product and try it out. >> > > >> > > My main question here -- do we need a formal vote for such a build? >> > > Or >> it >> > > can be treated as a release candidate? >> > > >> > > Any other thoughts? >> > > >> > > [1] https://github.com/apache/ignite-3 >> > > >> > > -Val >> > > >> > >> > -- Best regards, Ivan Pavlukhin
Re: [DISCUSS] Notifications for ignite-3 repository
Folks, I will merge the PR tomorrow if there is no objections. 2020-12-19 0:56 GMT+03:00, Valentin Kulichenko : > Folks, > > Can someone take a look at this PR? I'm generally OK with the change, but > I'm not familiar with the mechanisms used here. > > -Val > > On Thu, Dec 17, 2020 at 11:16 PM Ivan Pavlukhin > wrote: > >> Igniters, >> >> I noticed notifications about PRs in ignite-3 repository sent to >> dev-list. I suppose we should configure notifications similar to main >> ignite repository. I prepared a ticket [1] and PR for this. Please >> review. >> >> [1] https://issues.apache.org/jira/browse/IGNITE-13875 >> >> -- >> >> Best regards, >> Ivan Pavlukhin >> > -- Best regards, Ivan Pavlukhin
[DISCUSS] Notifications for ignite-3 repository
Igniters, I noticed notifications about PRs in ignite-3 repository sent to dev-list. I suppose we should configure notifications similar to main ignite repository. I prepared a ticket [1] and PR for this. Please review. [1] https://issues.apache.org/jira/browse/IGNITE-13875 -- Best regards, Ivan Pavlukhin
[jira] [Created] (IGNITE-13875) Configure notifications for ignite-3 git repository
Ivan Pavlukhin created IGNITE-13875: --- Summary: Configure notifications for ignite-3 git repository Key: IGNITE-13875 URL: https://issues.apache.org/jira/browse/IGNITE-13875 Project: Ignite Issue Type: Task Components: general Reporter: Ivan Pavlukhin Currently notifications for PRs in https://github.com/apache/ignite-3 repository are sent to dev-list. Notifications can be configured similarly to https://github.com/apache/ignite repository. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSS] Ignite 3.0 development approach
t; > > > > for working on these IEPs. I believe this might be >> > frustrating >> > > for the >> > > > > > folks who would like to commit code. >> > > > > > >> > > > > > The scope we agreed on is quite big, and it will surely >> > take >> > > > significant >> > > > > > time to implement all the changes and stabilize them. >> > Therefore, >> > > it's >> > > > > clear >> > > > > > to me that we will have to maintain 2.x and 3.x in >> > parallel for >> > > quite >> > > > > some >> > > > > > time - this needs to be addressed somehow. I'm >> > convinced >> > that >> > > having a >> > > > > > separate repo is the ONLY way to do that, and so far, I >> > haven't >> > > heard >> > > > any >> > > > > > clear alternatives or reasons why we shouldn't do this. >> > > > > > >> > > > > > That said, I'm inclined to proceed with this in the >> > next >> > few >> > > days - I >> > > > > will >> > > > > > create a repo and describe the process (which we, of >> > course, can >> > > > discuss >> > > > > > and modify going forward). Let's, at the very least, >> > try >> > and see >> > > where >> > > > it >> > > > > > leads us. >> > > > > > >> > > > > > If someone has any concrete alternative options on how >> > to >> > we can >> > > > maintain >> > > > > > two major versions in parallel, let's have another >> > voice >> > > discussion >> > > > this >> > > > > > Friday. If we do the meeting, we should set it up with >> > a >> > clear >> > > goal to >> > > > > make >> > > > > > a decision. Please let me know if there is interest in >> > this. >> > > > > > >> > > > > > -Val >> > > > > > >> > > > > > On Mon, Nov 16, 2020 at 6:31 AM Alexey Goncharuk < >> > > > > > alexey.goncha...@gmail.com> wrote: >> > > > > > >> > > > > >> Good, >> > > > > >> >> > > > > >> I think we have an intermediate agreement on the scope >> and >> > > > significance >> > > > > of >> > > > > >> the changes we want to make. I suggest creating >> > separate >> > > discussion >> > > > > >> streams >> > > > > >> and calls for each of the suggested topics so that: >> > > > > >> >> > > > > >>- It is clear for the community what is the >> motivation >> > of the >> > > > stream >> > > > > >>(this includes both functional targets and >> > technical >> > debt >> > > issues >> > > > > >> pointed >> > > > > >>out by Sergey) >> > > > > >>- Who is planning to take an active part in each of >> the >> > > streams >> > > > (i.e. >> > > > > >>the 'design committee', as Sergey suggested) >> > > > > >>- What are the intermediate and final goals for >> > each >> > of the >> > > streams >> > > > > >>- What are the cross-stream interactions and how we >> > > integrate them >> > > > > >>- How each of the streams will be integrated with >> > the >> > current >> > > > > codebase >> > > > > >>based on the above (here is where we will see >> > whether >> > > drop-in or >> > > > > >>incremental approaches make more sense) >> > > > > >> >> > > > > > >> > > > > >> > > > >> > > > >> > > > -- >> > > > Best regards, >> > > > Andrey V. Mashenkov >> > > > >> > > >> > > >> > >> > >> > -- Best regards, Ivan Pavlukhin
Re: Disable MVCC test suites
l take. >> >>>>> I would say that the opinion of the rest of the community is needed >> >> here. >> >>>>> >> >>>>> Anyway, I think test suites can be disabled even today, while the >> >> fate of >> >>>>> the MVCC feature can be (and should be) discussed separately. >> >>>>> What do you think? >> >>>>> >> >>>>> Thanks, >> >>>>> S. >> >>>>> >> >>>>> ср, 2 дек. 2020 г. в 15:38, Nikolay Izhikov : >> >>>>> >> >>>>>> Hello, Slava! >> >>>>>> >> >>>>>> Yes, this topic comes to the top from time to time :) >> >>>>>> >> >>>>>>> . I just want to save the time required for getting TCBot's visa >> >> and TC >> >>>>>> resources. >> >>>>>> >> >>>>>> Why do we need feature in the project that not even tested >> >> regularly? >> >>>>>> >> >>>>>>> 2 дек. 2020 г., в 15:36, Вячеслав Коптилин < >> >> slava.kopti...@gmail.com> >> >>>>>> написал(а): >> >>>>>>> >> >>>>>>> Hello Nikolay, >> >>>>>>> >> >>>>>>>> +1 to vote for complete MVCC removal. >> >>>>>>> It has already been discussed here [1] and, unfortunately, I have >> >> not >> >>>>>> seen >> >>>>>>> an agreement on that. >> >>>>>>> >> >>>>>>> [1] >> >>>>>>> >> >>>>>> >> >>>> >> >> >> http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-td45669.html >> >>>>>>> >> >>>>>>> >> >>>>>>> Thanks, >> >>>>>>> S. >> >>>>>>> >> >>>>>>> ср, 2 дек. 2020 г. в 13:05, Nikolay Izhikov >> >>>>>>> : >> >>>>>>> >> >>>>>>>> +1 to vote for complete MVCC removal. >> >>>>>>>> >> >>>>>>>> MVCC is a great feature but we should implement it as a >> >> first-class >> >>>>>>>> feature and not «something that pretends to be working» >> >>>>>>>> >> >>>>>>>>> 2 дек. 2020 г., в 12:53, Maxim Muzafarov >> >>>>>> написал(а): >> >>>>>>>>> >> >>>>>>>>> Hello Slava, >> >>>>>>>>> >> >>>>>>>>> I think we should vote for MVCC termination of support. If the >> >> vote >> >>>>>>>>> will be successful than remove it from the source code and >> >> disable >> >>>>>>>>> MVCC suites. >> >>>>>>>>> >> >>>>>>>>> Only disabling tests from MVCC sounds not good. >> >>>>>>>>> >> >>>>>>>>> On Wed, 2 Dec 2020 at 12:32, Вячеслав Коптилин < >> >>>>>> slava.kopti...@gmail.com> >> >>>>>>>> wrote: >> >>>>>>>>>> >> >>>>>>>>>> Hello Igniters, >> >>>>>>>>>> >> >>>>>>>>>> It looks like there is no activity related to maintaining or >> >>>>>> developing >> >>>>>>>> the >> >>>>>>>>>> MVCC feature. >> >>>>>>>>>> So, I see no reason to waste TeamCity resources. I propose to >> >>>> disable >> >>>>>>>> the >> >>>>>>>>>> corresponding test suites. >> >>>>>>>>>> This has already been discussed here as well [1]. >> >>>>>>>>>> >> >>>>>>>>>> [1] >> >>>>>>>>>> >> >>>>>>>> >> >>>>>> >> >>>> >> >> >> http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-td45669.html >> >>>>>>>>>> >> >>>>>>>>>> Thanks, >> >>>>>>>>>> S. >> >>>>>>>> >> >>>>>>>> >> >>>>>> >> >>>>>> >> >>>> >> >>>> >> >> >> >> > -- Best regards, Ivan Pavlukhin
Re: [VOTE] Define Apache Ignite as a Distributed Database
+1 2020-11-24 11:33 GMT+03:00, Anton Vinogradov : > +1 > > On Tue, Nov 24, 2020 at 10:24 AM Pavel Tupitsyn > wrote: > >> +1 >> >> On Tue, Nov 24, 2020 at 3:25 AM Saikat Maitra >> wrote: >> >> > +1 >> > >> > On Mon, Nov 23, 2020 at 4:55 PM Valentin Kulichenko < >> > valentin.kuliche...@gmail.com> wrote: >> > >> > > +1 >> > > >> > > On Mon, Nov 23, 2020 at 2:44 PM Denis Magda >> > > wrote: >> > > >> > > > Igniters, >> > > > >> > > > With this vote, I'd like to formally wrap up our discussion on >> defining >> > > > Ignite as a "distributed database" instead of an "in-memory >> computing" >> > > > platform. See the following discussion for the rationale and >> > > > context: >> > > > >> > > > >> > > >> > >> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Renaming-Ignite-s-product-category-td49246.html >> > > > >> > > > If the vote passes, the website will define Ignite as "a >> > > > distributed >> > > > database for in-memory speed at petabyte scale" to underscore our >> > > in-memory >> > > > origins (and the primary reason why the project is selected) but >> > > > not >> > > > diminishing our persistence capabilities. All prominent use cases >> such >> > as >> > > > caching, high-performance computing, etc. will remain visible on >> > > > the >> > > > website. There is nothing wrong if a distributed database is used >> > > > as >> a >> > > > cache or high-performance compute cluster; it's up to an >> > > > application >> > > > developer to decide. >> > > > >> > > > Overall, please cast your vote for defining Ignite as a >> > > > "distributed >> > > > database": >> > > > +1 - support the change >> > > > -1 - disagree with the change, explain why >> > > > 0 - neutral >> > > > >> > > > This is a majority vote that is open for the next 7 days and to be >> > closed >> > > > on Monday, Nov 30th: >> > > > https://www.timeanddate.com/countdown/to?year=2020=11=30 >> > > > >> > > > - >> > > > Denis >> > > > >> > > >> > >> > -- Best regards, Ivan Pavlukhin
Re: Alex Plehanov joins Ignite Project Management Committee
Congratulations Alex! 2020-11-18 11:17 GMT+03:00, Nikolay Izhikov : > Congrats! > >> 18 нояб. 2020 г., в 11:00, Nikita Amelchev >> написал(а): >> >> Alex, >> >> My congratulations! >> >> ср, 18 нояб. 2020 г. в 05:38, Saikat Maitra : >>> >>> Congratulations Alex!!! >>> >>> On Tue, 17 Nov 2020 at 6:26 PM, Denis Magda wrote: >>> >>>> Igniters, >>>> >>>> The Project Management Committee (PMC) for Apache Ignite >>>> has invited Alex to become a PMC member and we are pleased to announce >>>> that >>>> he has accepted. >>>> >>>> Alex joined our community back in 2018 and became one of the key >>>> community >>>> members. >>>> He made numerous contributions >>>> < >>>> https://issues.apache.org/jira/browse/IGNITE-13595?jql=project%20%3D%20Ignite%20and%20assignee%20%3D%20alex_pl%20and%20status%20in%20(Closed%2C%20Resolved) >>>>> >>>> since >>>> then, was driving Ignite 2.9 as a release manager, regularly >>>> participates >>>> in the dev lists discussions pertaining to the future and evolution of >>>> Ignite. >>>> >>>> Join me in congratulating Alex! >>>> >>>> >>>> - >>>> Denis >>>> >> >> >> >> -- >> Best wishes, >> Amelchev Nikita > > -- Best regards, Ivan Pavlukhin
Re: [DISCUSS] Missed (non-suited) tests
Max, Ivan, Using explicit @Ignore and the automated check sounds good to me. If nobody has arguments against it I think we should do it. 2020-10-19 19:30 GMT+03:00, Max Timonin : > Hi Ivan, > > I've checked the ticket you provide. It contains subtasks to uncomment or > to remove some unused tests. It definitely describes some cases I've found. > So what do you think if I uncomment them in suites, add @Ignore annotation > for those tests while the tickets are open? This will help to find out > tests that were forgiven in a recent time. > > Also I believe that this check must be automated. I didn't find a way how > uncomment / unused tests are found in the ticket. If there is no any - I > propose mine PR for this purpose. > > > > On Mon, Oct 19, 2020 at 5:24 PM Ivan Daschinsky > wrote: > >> Ivan, as far as I understand, Max also created verification check for not >> included test and found a few tests, that have never been included in any >> testsuites. >> >> Also, I suppose, that even if we cannot run some tests, these tests >> should >> be ignored using annotation, but not commented. >> >> пн, 19 окт. 2020 г. в 16:33, Ivan Pavlukhin : >> >> > Hi Max, >> > >> > There is an existing effort about "abandoned" tests >> > https://issues.apache.org/jira/browse/IGNITE-9210 >> > >> > 2020-10-19 16:25 GMT+03:00, Max Timonin : >> > > Hi Igniters! >> > > >> > > I made a research into tests that aren't included in any test suite. >> > > As >> > > TeamCity runs tests by suites so there could be tests that never run >> > > on >> > TC. >> > > So I tried implementing a simple check for such tests and include it >> > > in >> > > Ignite's travis config. >> > > >> > > The check runs while "mvn test" command and piggy-backs on the maven >> > > surefire plugin. I replaced the junit provider with a custom one that >> > > checks if a class is a test or a suite (there are some Ignite >> > > specific >> > > stuff), marks tests that are in suites and raises an exception if >> > > there >> > are >> > > non-suited tests. It's implemented as a part of maven command so it >> runs >> > > for every module separately. >> > > >> > > I've prepared draft PR with this check: >> > > https://github.com/apache/ignite/pull/8367 >> > > Travis check report is here: >> > > https://travis-ci.org/github/apache/ignite/jobs/737046387 >> > > As It's a draft, so I skip some maven configuration steps for a >> > > while. >> > Also >> > > I run the check only for the core module. >> > > >> > > But I have some results that want to discuss before continue the >> > > work: >> > > 1. Currently in the core module there are 53 tests that aren't part >> > > of >> > any >> > > test suite. I'm not sure about the reason for every test. So I just >> > > put >> > > below a list of the tests and last contributor to a file that >> > > contains >> a >> > > test. >> > > >> > > 2. Some tests are located in the core module, but suites are in a >> > > different, for example ignite-indexing suite >> > > IgniteCacheQuerySelfTestSuite3 contains >> > > only tests written in the core module, and none from the indexing >> module. >> > > Also there are suites in spring, uri-deploy, zookeeper modules. In my >> PR >> > > I've just copied the test suites to the core module. >> > > >> > > 3. Some test classes are named with the "Abstract" suffix but don't >> have >> > > the corresponding modifier (for example, >> > > IgniteTxTimeoutAbstractTest). >> > So, >> > > I add the modifier for every such file if it's not a part of any >> > > suite. >> > > >> > > What do you think about this check? If Ignite needs it, let's discuss >> > next >> > > things: >> > > 1. Mark tests that should never be in any suite by some reason; >> > > 2. Fix the missed tests; >> > > 3. How to declare suites that contains tests from a different module; >> > > 4. How to check if TC runs all suites. >> > > >> > > List of non-suited tests in the core module: >> > > >> > > maksim.stepac...@gmail.com:
Re: [DISCUSS] Missed (non-suited) tests
t; IgniteCacheAtomicOnheapExpiryPolicyTest > IgniteCacheAtomicLocalOnheapExpiryPolicyTest > GridCacheReplicatedOnheapFullApiSelfTest > GridCacheBinaryObjectsLocalOnheapSelfTest > > oignate...@users.noreply.github.com: > GridCacheTtlManagerEvictionSelfTest > > ira...@apache.org: > CommonPoolStarvationCheckpointTest > > alievmi...@gmail.com: > RemoveAllDeadlockTest > > schugu...@gridgain.com: > FullyConnectedComponentSearcherTest > > sboi...@gridgain.com: > IgniteDataStructuresNoClassOnServerTest > > timonin.ma...@gmail.com: > ReliableChannelTest > ThinClientPartitionAwarenessDiscoveryTest > -- Best regards, Ivan Pavlukhin
Re: Continuous query not transactional ?
Hi Veena, As far as I know, CQ listeners should not be notified about non-committed writes. I suppose that it is possible to reproduce a described scenario, what does it show in practice? Also it is important to know that all participating caches should be TRANSACTIONAL and "update of cache b fails" possibly does not mean that a transaction is rolled back. 2020-10-17 14:28 GMT+03:00, VeenaMithare : > Hello team, > > This is in continuation to these posts on the ignite users > > http://apache-ignite-users.70518.x6.nabble.com/Continuous-query-not-transactional-td34270.html > and > http://apache-ignite-users.70518.x6.nabble.com/Lag-before-records-are-visible-after-transaction-commit-tp33787p33861.html > > Does this mean we could get dirty reads as updates in continuous query ? > i.e. for eg if the code is as below: > 1. Start transaction > 2. update records of cache a > 3. update records of cache b > 4. update records for cache c > 5. commit > > if update of cache a succeeds , but update of cache b fails, will the local > listener for continuous query for 'cache a' get an update ? > > regards, > Veena. > > > > > > -- > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > -- Best regards, Ivan Pavlukhin
Re: [DISCUSS] Remove a Patch-file contribution approach from guides
Folks, I updated the page https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute 2020-10-12 11:47 GMT+03:00, Ilya Kasnacheev : > +1 > > I've never seen this patchway in action nor did I knew about its existence. > > Regards, > -- > Ilya Kasnacheev > > > вс, 11 окт. 2020 г. в 04:43, Saikat Maitra : > >> +1 >> >> Regards, >> Saikat >> >> On Sat, Oct 10, 2020 at 8:22 AM Alexey Zinoviev >> wrote: >> >> > Agree, need to update >> > >> > сб, 10 окт. 2020 г., 15:35 Ivan Pavlukhin : >> > >> > > Folks, >> > > >> > > A new contributor Bojidar tried to used a Patch-file contribution >> > > approach [1]. The approach is stated in our guides [2]. I suppose >> > > that >> > > the approach is obsolete. I several threads seem to confirm it [3, >> > > 4]. >> > > >> > > If nobody objects I will remove a section [2] from the guide. >> > > >> > > [1] >> > > >> > >> http://apache-ignite-developers.2346864.n4.nabble.com/Introduction-td49570.html >> > > [2] >> > > >> > >> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-2.CreateaPatch-file >> > > [3] >> > > >> > >> http://apache-ignite-developers.2346864.n4.nabble.com/ci-ignite-apache-Run-all-for-patch-jira-number-td24861.html >> > > [4] >> > > >> > >> http://apache-ignite-developers.2346864.n4.nabble.com/Disabling-patch-validation-mechanism-on-TC-td5400.html >> > > >> > > -- >> > > >> > > Best regards, >> > > Ivan Pavlukhin >> > > >> > >> > -- Best regards, Ivan Pavlukhin
Re: New Committer: Sergey Stronchinskiy
Sergey, congratulations! With great people Ignite is a great community-driven polyglot project. 2020-10-14 17:43 GMT+03:00, Pavel Tupitsyn : > Igniters, > > The Project Management Committee (PMC) for Apache Ignite > has invited Sergey Stronchinskiy to become a committer > and we are pleased to announce that he has accepted. > > Sergey is one of the most active Ignite.NET contributors and a > long time Ignite enthusiast. > > Being a committer enables easier contribution to the project since there is > no need to go via the patch submission process. This should enable better > productivity. > > Sergey, congratulations and welcome on board! > > Best Regards, > Pavel Tupitsyn > on behalf of Apache Ignite PMC > -- Best regards, Ivan Pavlukhin
[DISCUSS] Remove a Patch-file contribution approach from guides
Folks, A new contributor Bojidar tried to used a Patch-file contribution approach [1]. The approach is stated in our guides [2]. I suppose that the approach is obsolete. I several threads seem to confirm it [3, 4]. If nobody objects I will remove a section [2] from the guide. [1] http://apache-ignite-developers.2346864.n4.nabble.com/Introduction-td49570.html [2] https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-2.CreateaPatch-file [3] http://apache-ignite-developers.2346864.n4.nabble.com/ci-ignite-apache-Run-all-for-patch-jira-number-td24861.html [4] http://apache-ignite-developers.2346864.n4.nabble.com/Disabling-patch-validation-mechanism-on-TC-td5400.html -- Best regards, Ivan Pavlukhin
Re: Introduction
Bojidar, Sincerely I never used an approach with attaching patch files. It would be great if you can open PR. I suppose we should check with the Community whether a patch file approach is outdated and delete the section from the guide if it is. 2020-10-09 18:27 GMT+03:00, Bojidar Marinov : > Hey Ivan, > > According to the "How to contribute" document, submitting patch files is a > valid alternative to github pull requests, refs > https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-2.CreateaPatch-file > . > If that information is outdated, I would happily open a pull request on > github instead. > > BRs, > Bojidar > > On Fri, Oct 9, 2020 at 6:20 PM Ivan Pavlukhin wrote: > >> Bojidar, >> >> While my knowledge can be outdated, but in my time it was required to >> create a pull request in GitHub and launch a TC bot check against that >> PR manually. >> >> 2020-10-09 18:08 GMT+03:00, Bojidar Marinov > >: >> > On Fri, Oct 9, 2020 at 5:15 PM Ivan Pavlukhin >> wrote: >> > >> >> Hi Bojidar, >> >> >> >> Welcome to the Apache Ignite Community! >> >> >> >> Thanks! >> > >> >> I added your account to a contributors list, now you can assign >> >> tickets to yourself. Am I getting it right that your contribution is >> >> in .NET part? >> >> >> >> Indeed, it is a contribution to Ignite.NET. >> > >> > Also it could be helpful to get familiar with a contribution process >> >> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute >> >> >> >> >> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot >> >> >> >> Went over them again and tagged the issue as patch-available. >> >> Hopefully, >> > teamcity would pick it up soon. :) >> > >> >> >> >> 2020-10-08 23:08 GMT+03:00, Божидар Маринов < >> bojidar.marinov...@gmail.com >> >> >: >> >> > Hello Ignite Community! >> >> > >> >> > My name is Bojidar Marinov. I have used Apache Ignite for the past >> >> > few >> >> > months, and I would like to contribute to it starting with issue >> >> > IGNITE-13563, my JIRA username being bojidar-bg. >> >> > I have already submitted a patch, and reviews would be welcome. >> >> > >> >> > BRs, >> >> > Bojidar >> >> > >> >> >> >> >> >> -- >> >> >> >> Best regards, >> >> Ivan Pavlukhin >> >> >> > >> >> >> -- >> >> Best regards, >> Ivan Pavlukhin >> > -- Best regards, Ivan Pavlukhin
Re: Introduction
Hi Bojidar, Welcome to the Apache Ignite Community! I added your account to a contributors list, now you can assign tickets to yourself. Am I getting it right that your contribution is in .NET part? Also it could be helpful to get familiar with a contribution process https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot 2020-10-08 23:08 GMT+03:00, Божидар Маринов : > Hello Ignite Community! > > My name is Bojidar Marinov. I have used Apache Ignite for the past few > months, and I would like to contribute to it starting with issue > IGNITE-13563, my JIRA username being bojidar-bg. > I have already submitted a patch, and reviews would be welcome. > > BRs, > Bojidar > -- Best regards, Ivan Pavlukhin
Re: [MTCGA]: new failures in builds [5643882] needs to be handled
e/ignite/snippets/UsingContinuousQueries.java >> > >> > >> docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/UsingScanQueries.java >> > >> > >> docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/CustomThreadPool.java >> > >> > >> docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/services/MyCounterService.java >> > >> > >> docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/services/MyCounterServiceImpl.java >> > >> > >> docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/services/ServiceExample.java >> > >> > >> docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/DataPartitioning.java >> > >> > >> docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/BackupFilter.java >> > >> > >> docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/PartitionLossPolicyExample.java >> > >> > >> docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/Snapshots.java >> > >> > >> docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/EvictionPolicies.java >> > docs/_docs/code-snippets/nodejs/connecting.js >> > docs/_docs/code-snippets/nodejs/tls.js >> > docs/_docs/code-snippets/nodejs/binary-types.js >> > docs/_docs/code-snippets/nodejs/sql.js >> > docs/_docs/code-snippets/nodejs/conf1.js >> > docs/_docs/code-snippets/nodejs/get-existing-cache.js >> > docs/_docs/code-snippets/nodejs/sql-fields-query.js >> > docs/_docs/code-snippets/nodejs/types-mapping-configuration.js >> > docs/_docs/code-snippets/nodejs/scan-query.js >> > docs/_docs/code-snippets/nodejs/scanquery.js >> > docs/_docs/code-snippets/nodejs/authentication.js >> > docs/_docs/code-snippets/nodejs/configuring-cache-2.js >> > docs/_docs/code-snippets/nodejs/enabling-debug.js >> > docs/_docs/code-snippets/nodejs/key-value.js >> > docs/_docs/code-snippets/nodejs/configuring-cache-1.js >> > docs/_docs/code-snippets/nodejs/initialize.js >> > docs/_docs/code-snippets/nodejs/conf2.js >> > docs/_docs/sql-reference/system-functions.adoc >> > docs/_docs/sql-reference/aggregate-functions.adoc >> > docs/_docs/sql-reference/data-types.adoc >> > docs/_docs/sql-reference/dml.adoc >> > docs/_docs/sql-reference/date-time-functions.adoc >> > docs/_docs/sql-reference/ddl.adoc >> > docs/_docs/sql-reference/numeric-functions.adoc >> > docs/_docs/sql-reference/index.adoc >> > docs/_docs/sql-reference/transactions.adoc >> > docs/_docs/sql-reference/operational-commands.adoc >> > docs/_docs/sql-reference/string-functions.adoc >> > docs/_docs/sql-reference/sql-conformance.adoc >> > docs/_docs/data-streaming.adoc >> > docs/_docs/net-specific/net-standalone-nodes.adoc >> > docs/_docs/net-specific/net-configuration-options.adoc >> > docs/_docs/net-specific/net-platform-interoperability.adoc >> > docs/_docs/net-specific/net-remote-assembly-loading.adoc >> > docs/_docs/net-specific/net-plugins.adoc >> > docs/_docs/net-specific/index.adoc >> > docs/_docs/net-specific/net-troubleshooting.adoc >> > docs/_docs/net-specific/net-logging.adoc >> > docs/_docs/net-specific/asp-net-output-caching.adoc >> > docs/_docs/net-specific/net-linq.adoc >> > docs/_docs/net-specific/net-platform-cache.adoc >> > docs/_docs/net-specific/asp-net-session-state-caching.adoc >> > docs/_docs/net-specific/net-cross-platform-support.adoc >> > docs/_docs/net-specific/net-serialization.adoc >> > docs/_docs/net-specific/net-entity-framework-cache.adoc >> > docs/_docs/net-specific/net-java-services-execution.adoc >> > docs/_docs/net-specific/net-deployment-options.adoc >> > docs/_docs/includes/dotnet-prerequisites.adoc >> > docs/_docs/includes/nodes-and-clustering.adoc >> > docs/_docs/includes/intro-languages.adoc >> > docs/_docs/includes/install-nodejs-npm.adoc >> > docs/_docs/includes/note-on-deactivation.adoc >> > docs/_docs/includes/exampleprojects.adoc >> > docs/_docs/includes/cpp-prerequisites.adoc >> > docs/_docs/includes/cpp-linux-build-prerequisites.adoc >> > docs/_docs/includes/java9.adoc >> > docs/_docs/includes/install-ignite.adoc >> > docs/_docs/includes/prereqs.adoc >> > docs/_docs/includes/starting-node.adoc >> > docs/_docs/includes/install-php-composer.adoc >> > docs/_docs/includes/thick-and-thin-clients.adoc >> > docs/_docs/includes/partition-awareness.adoc >> > docs/_docs/includes/install-python-pip.adoc >> > docs/_docs/persistence/disk-compression.adoc >> > docs/_docs/persistence/external-storage.adoc >> > docs/_docs/persistence/snapshots.adoc >> > docs/_docs/persistence/persistence-tuning.adoc >> > docs/_docs/persistence/swap.adoc >> > docs/_docs/persistence/custom-cache-store.adoc >> > docs/_docs/persistence/native-persistence.adoc >> > docs/_docs/distributed-computing/map-reduce.adoc >> > docs/_docs/distributed-computing/collocated-computations.adoc >> > docs/_docs/distributed-computing/cluster-groups.adoc >> > docs/_docs/distributed-computing/executor-service.adoc >> > docs/_docs/distributed-computing/distributed-computing.adoc >> > docs/_docs/distributed-computing/job-scheduling.adoc >> > docs/_docs/distributed-computing/fault-tolerance.adoc >> > docs/_docs/distributed-computing/load-balancing.adoc >> > docs/_docs/code-deployment/deploying-user-code.adoc >> > docs/_docs/code-deployment/peer-class-loading.adoc >> > >> > docs/_docs/extensions-and-integrations/streaming/rocketmq-streamer.adoc >> > docs/_docs/extensions-and-integrations/streaming/zeromq-streamer.adoc >> > docs/_docs/extensions-and-integrations/streaming/jms-streamer.adoc >> > >> > docs/_docs/extensions-and-integrations/streaming/twitter-streamer.adoc >> > docs/_docs/extensions-and-integrations/streaming/storm-streamer.adoc >> > docs/_docs/extensions-and-integrations/streaming/flume-sink.adoc >> > docs/_docs/extensions-and-integrations/streaming/mqtt-streamer.adoc >> > docs/_docs/extensions-and-integrations/streaming/kafka-streamer.adoc >> > docs/_docs/extensions-and-integrations/streaming/camel-streamer.adoc >> > docs/_docs/extensions-and-integrations/streaming/flink-streamer.adoc >> > docs/_docs/extensions-and-integrations/spring/spring-data.adoc >> > docs/_docs/extensions-and-integrations/spring/spring-caching.adoc >> > docs/_docs/extensions-and-integrations/spring/spring-boot.adoc >> > docs/_docs/extensions-and-integrations/mybatis-l2-cache.adoc >> > docs/_docs/extensions-and-integrations/cassandra/ddl-generator.adoc >> > docs/_docs/extensions-and-integrations/cassandra/usage-examples.adoc >> > docs/_docs/extensions-and-integrations/cassandra/overview.adoc >> > docs/_docs/extensions-and-integrations/cassandra/configuration.adoc >> > >> > >> docs/_docs/extensions-and-integrations/ignite-for-spark/ignite-dataframe.adoc >> > docs/_docs/extensions-and-integrations/ignite-for-spark/overview.adoc >> > >> > >> docs/_docs/extensions-and-integrations/ignite-for-spark/ignitecontext-and-rdd.adoc >> > >> >> docs/_docs/extensions-and-integrations/ignite-for-spark/installation.adoc >> > >> > >> docs/_docs/extensions-and-integrations/ignite-for-spark/troubleshooting.adoc >> > >> docs/_docs/extensions-and-integrations/ignite-for-spark/spark-shell.adoc >> > docs/_docs/extensions-and-integrations/php-pdo.adoc >> > docs/_docs/extensions-and-integrations/hibernate-l2-cache.adoc >> > docs/_docs/key-value-api/with-expiry-policy.adoc >> > docs/_docs/key-value-api/basic-cache-operations.adoc >> > docs/_docs/key-value-api/binary-objects.adoc >> > docs/_docs/key-value-api/using-scan-queries.adoc >> > docs/_docs/key-value-api/transactions.adoc >> > docs/_docs/key-value-api/continuous-queries.adoc >> > docs/_docs/services/services.adoc >> > docs/_docs/memory-configuration/eviction-policies.adoc >> > docs/_docs/memory-configuration/index.adoc >> > docs/_docs/memory-configuration/data-regions.adoc >> > docs/_layouts/toc.html >> > docs/_layouts/default.html >> > docs/_layouts/doc.html >> > >> > * >> > >> > >> > вт, 6 окт. 2020 г. в 13:05, Ivan Pavlukhin : >> > >> > > Denis, folks, >> > > >> > > The mentioned commit breaks our license checks. >> > > >> > > Is someone working on the fix at the moment? If not it might be >> > > simpler to revert the commit and push a fixed version later on. Also >> > > it looks sad that so massive contribution was pushed without bot visa >> > > [1]. >> > > >> > > [1] https://issues.apache.org/jira/browse/IGNITE-7595 >> > > >> > > 2020-10-06 12:11 GMT+03:00, dpavlov.ta...@gmail.com < >> > > dpavlov.ta...@gmail.com>: >> > > > Hi Igniters, >> > > > >> > > > I've detected some new issue on TeamCity to be handled. You are >> > > > more >> > > than >> > > > welcomed to help. >> > > > >> > > > If your changes can lead to this failure(s): We're grateful that >> > > > you >> > > were a >> > > > volunteer to make the contribution to this project, but things >> > > > change >> > and >> > > > you may no longer be able to finalize your contribution. >> > > > Could you respond to this email and indicate if you wish to >> > > > continue >> > and >> > > > fix test failures or step down and some committer may revert you >> > commit. >> > > > >> > > > *New Critical Failure in master [Licenses Headers] >> > > > >> > > >> > >> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_LicensesHeaders?branch=%3Cdefault%3E >> > > > Changes may lead to failure were done by >> > > >- denis magda >> > > > https://ci.ignite.apache.org/viewModification.html?modId=908111 >> > > > >> > > >- Here's a reminder of what contributors were agreed to do >> > > > https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute >> > > >- Should you have any questions please contact >> > > dev@ignite.apache.org >> > > > >> > > > Best Regards, >> > > > Apache Ignite TeamCity Bot >> > > > https://github.com/apache/ignite-teamcity-bot >> > > > Notification generated at 12:11:17 06-10-2020 >> > > > >> > > >> > > >> > > -- >> > > >> > > Best regards, >> > > Ivan Pavlukhin >> > > >> > >> > -- Best regards, Ivan Pavlukhin
