[jira] [Created] (IGNITE-12502) Document ignite-spring-data_2.2 module
Ilya Kasnacheev created IGNITE-12502: Summary: Document ignite-spring-data_2.2 module Key: IGNITE-12502 URL: https://issues.apache.org/jira/browse/IGNITE-12502 Project: Ignite Issue Type: Improvement Components: documentation Reporter: Ilya Kasnacheev After IGNITE-12259 I think there are no API changes, but we should mention that we have such module and what its dependencies are. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
Hello!
I have committed ignite-spring-data_2.2 to ignite-2.8.
By bumping versisons I mean the following:
1.7.*7*
1.6.*4*
1.1.7.*2*
2.6.*5*
2.3.*0*
1.13.*14*.RELEASE
4.3.*18*.RELEASE
2.0.*9*.RELEASE
5.0.*8*.RELEASE
All these libraries have maintenance release (such as our 2.7.*6*) and I
think it would be beneficial to upgrade these dependencies to the latest
maintenance version found in Maven Central.
For example, there is spring.data-2.0 2.0.*14*.RELEASE.
Regards,
--
Ilya Kasnacheev
чт, 26 дек. 2019 г. в 19:32, Denis Magda :
> A huge +1 for adding Spring Data related fixes/improvements. Ilya is right
> that Spring Data related questions sparked last time due to missing support
> of 2.2 version.
>
> Ilya, could you elaborate on what you mean under "bumping the versions"? Do
> you suggest performing a straightforward upgrade of "ignite-spring-data" to
> version 2.2 and introducing "ignite-spring-data-{old-version"} for the
> previous versions? If it's so, I fully agree with the proposal.
>
> -
> Denis
>
>
> On Thu, Dec 26, 2019 at 4:52 AM Ilya Kasnacheev >
> wrote:
>
> > Hello!
> >
> > I propose to add the following ticket to the scope:
> > https://issues.apache.org/jira/browse/IGNITE-12259 (3 commits, be
> careful
> > with release version)
> >
> > Adding tickets to scope surely seems crazy now, but I will provide the
> > following considerations:
> > * This is Spring Data 2.2 integration, which we currently do not have,
> > leading to lots of confused questions on stack overflow and mailing list.
> > Spring Data is important to our public image since many people may learn
> > about out project by starting with Spring Data.
> >
> > * It has zero code impact outside of its own module (just 2 POM file
> > touched and that's all).
> >
> > * The core was ready since early November but, due to gmail quirk, we did
> > not react to it in time.
> >
> > WDYT?
> >
> > Another semi-related question. *Should we bump our dependencies' versions
> > before releasing 2.8?* I talk mainly about spring and hibernate
> > dependencies. We could switch them to their latest maintenance versions
> to
> > avoid shipping default links to outdated packages.
> >
> > I think this is one of things that are very hard to do between releases,
> so
> > I think this dependencies bumping should be a part of a formal
> > release/testing cycle, and then be backported to master.
> >
> > I could volunteer to do that myself, if we agree to merge these version
> > upgrades to ignite-2.8 and then re-test.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > вт, 24 дек. 2019 г. в 13:22, Zhenya Stanilovsky
> > >:
> >
> > >
> > > Igniters, i`l try to compare 2.8 release candidate vs 2.7.6,
> > > last sha 2.8 was build from : 9d114f3137f92aebc2562a
> > > i use yardstick benchmarks, 4 bare machine with: 2x Xeon X5570 96Gb
> > 512GB
> > > SSD 2048GB HDD 10GB/s
> > > 1 for client (driver) and 3 for servers.
> > > this mappings for graphs and real yardstick tests:
> > >
> > > atomic-put: IgnitePutBenchmark
> > > sql-merge-query: IgniteSqlMergeQueryBenchmark
> > > atomic-get: IgniteGetBenchmark
> > > tx-get: IgniteGetTxBenchmark
> > > tx-put: IgnitePutTxBenchmark
> > > atomic-put-all-bs-10: IgnitePutAllBenchmark
> > > tx-put-all-bs-10: IgnitePutAllTxBenchmark
> > >
> > > cacheMode — partitioned
> > > CacheWriteSynchronizationMode.FULL_SYNC
> > > 1 backup
> > >
> > > 1. wal = log_only 2. wal = none 3. persistence disabled.
> > > Thanks Maxim for wiki page [1]
> > >
> > >
> > > [1]
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.8#ApacheIgnite2.8-Benchmarks
> > >
> > > do we need some bisect or other work here ?
> > >
> > > >
> > > >
> > > >--- Forwarded message ---
> > > >From: "Maxim Muzafarov" < mmu...@apache.org >
> > > >To: dev@ignite.apache.org
> > > >Cc:
> > > >Subject: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
> > > >Date: Fri, 20 Sep 2019 14:44:31 +0300
> > > >
> > > >Igniters,
> > > >
> > > >
> > > >It's almost a year has passed since the last major Apache Ignite 2.7
> > > >has been released. We've accumulated a lot of performance improvements
> > > >and a lot of new features which are waiting for their release date.
> > > >Here is my list of the most interesting things from my point since the
> > > >last major release:
> > > >
> > > >Service Grid,
> > > >Monitoring,
> > > >Recovery Read
> > > >BLT auto-adjust,
> > > >PDS compression,
> > > >WAL page compression,
> > > >Thin client: best effort affinity,
> > > >Thin client: transactions support (not yet)
> > > >SQL query history
> > > >SQL statistics
> > > >
> > > >I think we should no longer wait and freeze the master branch anymore
> > > >and prepare the next major release by the end of the year.
> > > >
> > > >
> > > >I propose to discuss Time, Scope of Apache Ignite 2.8 release and also
> > > >I want to propose myself to be the release manager of the planning
>
Re: Visor plugin
> 2. Move all colde from visorcmd to controls.sh > 3. Drop visorcm Hello, Zhenya, Alexey, can you, please, clarify, why you suggest dropping of visorcmd? What are the problems of the visorcmd that can’t be solved? > 27 дек. 2019 г., в 09:26, Zhenya Stanilovsky > написал(а): > > > +1 here. > > >> Hi, All! >> >> I think that the best way is to do the following: >> 1. Move controls.sh to separate module (this will allow us to use any third >> party libs for argument parsing, interactive mode, and other stuff). >> 2. Move all colde from visorcmd to controls.sh >> 3. Drop visorcmd >> >> On Thu, Dec 26, 2019 at 11:29 PM Николай Ижиков < nizhi...@apache.org > >> wrote: >> >>> +1 to keep visor cmd and fix it’s issues. >>> 26 дек. 2019 г., в 19:23, Denis Magda < dma...@apache.org > написал(а): Personally, I see no reason for deprecating Visor CLI and moving all its capabilities to the control.sh script. It's better to merge the script's capabilities into Visor CLI and rework the connectivity/communication protocol of the latter. If to recall the reason for the control.sh >>> creation it was the Visor's daemon-mode way of interaction with the cluster that >>> is cumbersome and complicated. - Denis On Thu, Dec 26, 2019 at 1:49 AM Ilya Kasnacheev < >>> ilya.kasnach...@gmail.com > wrote: > Hello! > > I think that control.sh should be extended instead of Visor CLI, it is a > tool which sees a lot more activity currently. > > Regards, > -- > Ilya Kasnacheev > > > чт, 26 дек. 2019 г. в 12:45, Ivan Pavlukhin < vololo...@gmail.com >: > >> Folks, >> >> Do not we have plans to discontinue Visor? If so, it might be better >> to add a desired functionality to another management API? >> >> вт, 24 дек. 2019 г. в 08:02, Denis Magda < dma...@apache.org >: >>> >>> Forwarding to the dev list. How exactly would you like to expand Visor >> CMD? >>> Please describe your idea and we can mover from that point. >>> >>> Denis >>> >>> On Monday, December 23, 2019, sgtech19 < rajado...@gmail.com > wrote: >>> Hello team, I would like to add a new feature to the existing >> visor commands. Could you give me some direction on how to achieve this if >> its possible? Do we need a visor plugin ? if so,please provide any > example >> of this plugin . Thanks -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >>> >>> >>> -- >>> - >>> Denis >> >> >> >> -- >> Best regards, >> Ivan Pavlukhin >> > >>> >>> >> -- >> Alexey Kuznetsov >> > > > >
Re[2]: Visor plugin
+1 here. >Hi, All! > >I think that the best way is to do the following: >1. Move controls.sh to separate module (this will allow us to use any third >party libs for argument parsing, interactive mode, and other stuff). >2. Move all colde from visorcmd to controls.sh >3. Drop visorcmd > >On Thu, Dec 26, 2019 at 11:29 PM Николай Ижиков < nizhi...@apache.org > wrote: > >> +1 to keep visor cmd and fix it’s issues. >> >> > 26 дек. 2019 г., в 19:23, Denis Magda < dma...@apache.org > написал(а): >> > >> > Personally, I see no reason for deprecating Visor CLI and moving all its >> > capabilities to the control.sh script. It's better to merge the script's >> > capabilities into Visor CLI and rework the connectivity/communication >> > protocol of the latter. If to recall the reason for the control.sh >> creation >> > it was the Visor's daemon-mode way of interaction with the cluster that >> is >> > cumbersome and complicated. >> > >> > - >> > Denis >> > >> > >> > On Thu, Dec 26, 2019 at 1:49 AM Ilya Kasnacheev < >> ilya.kasnach...@gmail.com > >> > wrote: >> > >> >> Hello! >> >> >> >> I think that control.sh should be extended instead of Visor CLI, it is a >> >> tool which sees a lot more activity currently. >> >> >> >> Regards, >> >> -- >> >> Ilya Kasnacheev >> >> >> >> >> >> чт, 26 дек. 2019 г. в 12:45, Ivan Pavlukhin < vololo...@gmail.com >: >> >> >> >>> Folks, >> >>> >> >>> Do not we have plans to discontinue Visor? If so, it might be better >> >>> to add a desired functionality to another management API? >> >>> >> >>> вт, 24 дек. 2019 г. в 08:02, Denis Magda < dma...@apache.org >: >> >> Forwarding to the dev list. How exactly would you like to expand Visor >> >>> CMD? >> Please describe your idea and we can mover from that point. >> >> Denis >> >> On Monday, December 23, 2019, sgtech19 < rajado...@gmail.com > wrote: >> >> > Hello team, >> > I would like to add a new feature to the existing >> >>> visor >> > commands. Could you give me some direction on how to achieve this if >> >>> its >> > possible? Do we need a visor plugin ? if so,please provide any >> >> example >> >>> of >> > this plugin . >> > >> > Thanks >> > >> > >> > >> > -- >> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >> > >> >> >> -- >> - >> Denis >> >>> >> >>> >> >>> >> >>> -- >> >>> Best regards, >> >>> Ivan Pavlukhin >> >>> >> >> >> >> >-- >Alexey Kuznetsov >
Re: Visor plugin
Hi, All! I think that the best way is to do the following: 1. Move controls.sh to separate module (this will allow us to use any third party libs for argument parsing, interactive mode, and other stuff). 2. Move all colde from visorcmd to controls.sh 3. Drop visorcmd On Thu, Dec 26, 2019 at 11:29 PM Николай Ижиков wrote: > +1 to keep visor cmd and fix it’s issues. > > > 26 дек. 2019 г., в 19:23, Denis Magda написал(а): > > > > Personally, I see no reason for deprecating Visor CLI and moving all its > > capabilities to the control.sh script. It's better to merge the script's > > capabilities into Visor CLI and rework the connectivity/communication > > protocol of the latter. If to recall the reason for the control.sh > creation > > it was the Visor's daemon-mode way of interaction with the cluster that > is > > cumbersome and complicated. > > > > - > > Denis > > > > > > On Thu, Dec 26, 2019 at 1:49 AM Ilya Kasnacheev < > ilya.kasnach...@gmail.com> > > wrote: > > > >> Hello! > >> > >> I think that control.sh should be extended instead of Visor CLI, it is a > >> tool which sees a lot more activity currently. > >> > >> Regards, > >> -- > >> Ilya Kasnacheev > >> > >> > >> чт, 26 дек. 2019 г. в 12:45, Ivan Pavlukhin : > >> > >>> Folks, > >>> > >>> Do not we have plans to discontinue Visor? If so, it might be better > >>> to add a desired functionality to another management API? > >>> > >>> вт, 24 дек. 2019 г. в 08:02, Denis Magda : > > Forwarding to the dev list. How exactly would you like to expand Visor > >>> CMD? > Please describe your idea and we can mover from that point. > > Denis > > On Monday, December 23, 2019, sgtech19 wrote: > > > Hello team, > > I would like to add a new feature to the existing > >>> visor > > commands. Could you give me some direction on how to achieve this if > >>> its > > possible? Do we need a visor plugin ? if so,please provide any > >> example > >>> of > > this plugin . > > > > Thanks > > > > > > > > -- > > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > > > > > -- > - > Denis > >>> > >>> > >>> > >>> -- > >>> Best regards, > >>> Ivan Pavlukhin > >>> > >> > > -- Alexey Kuznetsov
SQL dialects supported by Calcite
Igor S., Roman and others who involved in Calcite prototyping, Is it true that Calcite can parse MySQL, Oracle, ANSI-99 and other dialects as suggested by this page? https://calcite.apache.org/apidocs/org/apache/calcite/sql/validate/SqlConformanceEnum.html Do I need to select a dialect globally or can it be set on a per-query level? - Denis
Re: Contribution page & easy/moderate tickets for newcomers
Hello Saikat, Thanks for checking the page and suggesting the improvement. I'll add the link to the page. Also, I'm thinking to go for another round of changes by listing other forms of contributions such as helping to answer users' questions, reviewing changes or updating documentation. Like the way it's described by Spark community: https://spark.apache.org/contributing.html - Denis On Thu, Dec 26, 2019 at 12:18 PM Saikat Maitra wrote: > Hello Denis, > > Thank you for publishing and modifying the Contributions page. Many time > folks have approached me over email, linkedin and offline about how they > can contribute to Apache projects and I share this link and about my > experience. I will continue to share this link going forward. > > I am thinking if it will be ok to add link to the apache getting started > 101 page https://community.apache.org/gettingStarted/101.html in the > Contributions page. > > Regards, > Saikat > > > > > > > > On Sat, Dec 21, 2019 at 7:53 AM Denis Magda wrote: > > > Pavel, Alexey, thanks for checking and updates! > > > > I’ll send this page to a fellow editor for a review once it’s checked by > us > > internally. Let’s wait a bit letting others join the thread. > > > > Denis > > > > On Saturday, December 21, 2019, Alexey Zinoviev > > wrote: > > > > > Cool, filters much more better than previous list of tasks inserted > > > directly in HTML > > > > > > сб, 21 дек. 2019 г. в 11:31, Pavel Tupitsyn : > > > > > > > Hi Denis, > > > > > > > > I've reviewed the list of newbie tickets for .NET, added some more > > there. > > > > > > > > Also I've updated the page a little bit: > > > > > > > > - Changed the wording of the first paragraph: "you need to sign up" > -> > > > > "please sign up" > > > > To be honest, the mood on this whole page feels a bit off to me. > > > > I'm not a native speaker though, maybe we should ask for help here to > > > make > > > > the text more friendly. > > > > > > > > - Fixed the JIRA links > > > > Many of them pointed to a specific issue instead along with the > search > > > > results > > > > > > > > Thanks, > > > > Pavel > > > > > > > > On Sat, Dec 21, 2019 at 12:07 AM Denis Magda > > wrote: > > > > > > > > > Ignite, > > > > > > > > > > I've done a major overhaul of our contribution page that promotes > > > > > involvement in Ignite development: > > > > > https://ignite.apache.org/community/contribute.html > > > > > > > > > > Many sections were simplified, some legacy stuff was removed and > I'll > > > > > appreciate if some of you check the page and advise further > > > improvements. > > > > > > > > > > Apart from that, one of the sections (Pick a Ticket) lists of easy > > and > > > > > moderate tickets > > > > > < > > > > > > > > > https://issues.apache.org/jira/issues/?jql=project%20% > > > 3D%20IGNITE%20AND%20labels%20in%20(newbie)%20and%20status%20%3D%20OPEN > > > > > > > > > > > for > > > > > newcomers. I do believe that the list needs to be reviewed. Could > > each > > > of > > > > > you recall any tickets of easy/moderate complexity that are still > > > pending > > > > > but will be great to implement? Put the "newbie" label for those > and > > > > > mention "component". I've already highlighted some of the > components > > on > > > > the > > > > > webpage (ML, SQL, platforms) and we can expand the list. > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > - > > > > > Denis > > > > > > > > > > > > > > > > > > -- > > - > > Denis > > >
Re: Contribution page & easy/moderate tickets for newcomers
Hello Denis, Thank you for publishing and modifying the Contributions page. Many time folks have approached me over email, linkedin and offline about how they can contribute to Apache projects and I share this link and about my experience. I will continue to share this link going forward. I am thinking if it will be ok to add link to the apache getting started 101 page https://community.apache.org/gettingStarted/101.html in the Contributions page. Regards, Saikat On Sat, Dec 21, 2019 at 7:53 AM Denis Magda wrote: > Pavel, Alexey, thanks for checking and updates! > > I’ll send this page to a fellow editor for a review once it’s checked by us > internally. Let’s wait a bit letting others join the thread. > > Denis > > On Saturday, December 21, 2019, Alexey Zinoviev > wrote: > > > Cool, filters much more better than previous list of tasks inserted > > directly in HTML > > > > сб, 21 дек. 2019 г. в 11:31, Pavel Tupitsyn : > > > > > Hi Denis, > > > > > > I've reviewed the list of newbie tickets for .NET, added some more > there. > > > > > > Also I've updated the page a little bit: > > > > > > - Changed the wording of the first paragraph: "you need to sign up" -> > > > "please sign up" > > > To be honest, the mood on this whole page feels a bit off to me. > > > I'm not a native speaker though, maybe we should ask for help here to > > make > > > the text more friendly. > > > > > > - Fixed the JIRA links > > > Many of them pointed to a specific issue instead along with the search > > > results > > > > > > Thanks, > > > Pavel > > > > > > On Sat, Dec 21, 2019 at 12:07 AM Denis Magda > wrote: > > > > > > > Ignite, > > > > > > > > I've done a major overhaul of our contribution page that promotes > > > > involvement in Ignite development: > > > > https://ignite.apache.org/community/contribute.html > > > > > > > > Many sections were simplified, some legacy stuff was removed and I'll > > > > appreciate if some of you check the page and advise further > > improvements. > > > > > > > > Apart from that, one of the sections (Pick a Ticket) lists of easy > and > > > > moderate tickets > > > > < > > > > > > > https://issues.apache.org/jira/issues/?jql=project%20% > > 3D%20IGNITE%20AND%20labels%20in%20(newbie)%20and%20status%20%3D%20OPEN > > > > > > > > > for > > > > newcomers. I do believe that the list needs to be reviewed. Could > each > > of > > > > you recall any tickets of easy/moderate complexity that are still > > pending > > > > but will be great to implement? Put the "newbie" label for those and > > > > mention "component". I've already highlighted some of the components > on > > > the > > > > webpage (ML, SQL, platforms) and we can expand the list. > > > > > > > > > > > > Thanks! > > > > > > > > - > > > > Denis > > > > > > > > > > > > -- > - > Denis >
Re: [DISCUSS] dependencies and release process for Ignite Extensions
Hello, Please find the PR link below. https://github.com/apache/ignite-extensions/pull/1/files Regards, Saikat On Thu, Dec 26, 2019 at 1:43 PM Saikat Maitra wrote: > Hello > > I have been able to resolve the build issue in teamcity and was able to > pass the build failures. > > > https://ci.ignite.apache.org/viewLog.html?buildId=4883394=IgniteExtensions_Build > > The issue that was causing failure was that I had put ignite.zip as > artifact dependency and it was downloading the artifacts including pom file > inside the build checkout folder of ignite-extensions and as a result the > pom.xml of ignite-extensions was getting overwritten by ignite's pom.xml. > This was resulting in missing flink-ext module error as it was not even > present in the ignite main pom.xml. > > To resolve the issue I just had to download and extract artifact of > ignite.zip in a separate directory. > > If I can get a +1 on this PR then I can go ahead and merge the changes. > > Regards, > Saikat > > > > > > > On Tue, Dec 10, 2019 at 8:26 PM Saikat Maitra > wrote: > >> Hi Ivan, >> >> Thank you for sharing your feedback. I will try to install all modules in >> local agent's repository and run the build. >> >> I will share updates. >> >> Regards, >> Saikat >> >> On Tue, Dec 10, 2019 at 9:56 AM Ivan Pavlukhin >> wrote: >> >>> Saikat, >>> >>> Let me share my understanding how do we run the majority of our jobs >>> executing test suites. Hope it will help. >>> 1. ~Build Apache Ignite~ in a separate run and publish everything as a >>> single artifact ignite.zip (it contains ignite working directory with >>> compiled modules, e.g. modules/core/target and others). >>> 2. Download and extract ignite.zip and run particular test suites >>> against extracted files. >>> >>> In a such setup test suites run the same as you do it on your local >>> machine and it always works with a single multi-module maven project >>> (apache-ignite). So, all dependencies are resolved in the working >>> directory. >>> >>> For a separate IgniteExtensions maven project you need to install all >>> dependencies from downloaded ignite.zip into .m2/repository local to >>> the agent. I suppose we can do it as follows: >>> 1. Download ignite.zip and extract it into "ignite" subfolder. >>> 2. Run a maven command installing all modules into local maven >>> repository. Something like "mvn install" but here we need to make sure >>> that nothing is compiled once again. >>> 3. Build IgniteExtensions. >>> >>> There might be a better way to do the same, I just tried to describe a >>> schema how it can be done. >>> >>> вт, 10 дек. 2019 г. в 06:30, Saikat Maitra : >>> > >>> > Hi Ivan, Ilya >>> > >>> > Thank you for your reply. I have added you as dev in IgniteExtensions >>> > project and you should be able to check the build configurations. >>> > >>> > Yes, when I set artifact dependencies for ~Build Apache Ignite~ >>> similar to >>> > other projects then I receive below error >>> > >>> > [ERROR] [ERROR] Could not find the selected project in the reactor: >>> > :ignite-flink-ext @ >>> > >>> > I do not see the same error when I am running the build in local and >>> > maven package command can still get 2.9.0-SNAPSHOT dependencies from my >>> > local .m2 repository but it is failing in teamcity. >>> > >>> > Regards, >>> > Saikat >>> > >>> > >>> > >>> > >>> > On Mon, Dec 9, 2019 at 5:18 AM Ilya Kasnacheev < >>> ilya.kasnach...@gmail.com> >>> > wrote: >>> > >>> > > Hello! >>> > > >>> > > I think that your build should depend on an Apache Ignite build(s) >>> or just >>> > > use an already released version. >>> > > >>> > > In the same fashion as all our teamcity depend on "Build Apache >>> Ignite" and >>> > > use its artifacts. >>> > > >>> > > Here you want Ignite snapshot but Teamcity does not know where to >>> take it >>> > > from. >>> > > >>> > > Regards, >>> > > -- >>> > > Ilya Kasnacheev >>> > > >>> > > >>> > > пн, 9 дек. 2019 г. в 06:40, Saikat Maitra : >>> > > >>> > > > Hello, >>> > > > >>> > > > I am running into a problem specific to teamcity build for Ignite >>> > > > Extensions project. When I set the dependencies to 2.9.0-SNAPSHOT >>> I am >>> > > > getting an error message during build as below >>> > > > >>> > > > [06:24:12][Step 4/5] Failed to execute goal on project >>> ignite-flink-ext: >>> > > > Could not resolve dependencies for project >>> > > > org.apache.ignite.ext:ignite-flink-ext:jar:1.0.0-SNAPSHOT: The >>> following >>> > > > artifacts could not be resolved: >>> > > > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT, >>> > > > org.apache.ignite:ignite-core:jar:tests:2.9.0-SNAPSHOT, >>> > > > org.apache.ignite:ignite-log4j:jar:2.9.0-SNAPSHOT, >>> > > > org.apache.ignite:ignite-spring:jar:2.9.0-SNAPSHOT: Could not find >>> > > artifact >>> > > > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT in h2database.com >>> ( >>> > > > https://h2database.com/m2-repo) >>> > > > >>> > > > >>> > > > and if set artifact dependencies for ~Build Apache Ignite~ then I >>>
Re: [DISCUSS] Pub/Sub Streamer Implementation
Hi Emmanouil, I have been able to resolve the build failures for missing modules. https://ci.ignite.apache.org/viewLog.html?buildId=4883394=IgniteExtensions_Build Will you please take the latest changes from my PR and run the build for your PR. https://github.com/apache/ignite-extensions/pull/1/files The new build configurations should let your module build also pass. The issue that was causing failure was that I had put ignite.zip as artifact dependency and it was downloading the artifacts including pom file inside the build checkout folder of ignite-extensions and as a result the pom.xml of ignite-extensions was getting overwritten by ignite's pom.xml. This was resulting in missing flink-ext module error as it was not even present in the ignite main pom.xml. To resolve the issue I just had to download and extract artifact of ignite.zip in a separate directory. Regards, Saikat On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras wrote: > Hi Saikat! > > Thank you for your assistance on that! > > Kind regards > > *Emmanouil Gkatziouras* > https://egkatzioura.com/ | > https://www.linkedin.com/in/gkatziourasemmanouil/ > https://github.com/gkatzioura > > > On Sat, 30 Nov 2019 at 23:51, Saikat Maitra > wrote: > > > Hi Emmanouil, > > > > I looked into the build logs and I observed that since I added artifacts > > dependencies to "Build apache Ignite"[1] it was able to pull the required > > dependencies and was able to run the existing tests. It is however not > > identifying new modules like I changed from flink to flink-ext or > pub-sub. > > > > [1] > > > > > https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build > > > > I will look into the issue and debug further. > > > > Regards, > > Saikat > > > > > > On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras < > > gkatzio...@gmail.com> > > wrote: > > > > > Hi Saikat! > > > > > > From the logs[1] it seems that TC cannot find the project. However I > did > > > include the project on the parent pom (modules sections) [2] > > > I tried to reproduce locally unfortunately I am not aware of the full > > > context on TC. It seems as if a different parent pom is being picked > up. > > > > > > Kind regards > > > > > > [1] > > > > > > > > > https://ci.ignite.apache.org/viewLog.html?buildId=4804893=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=1704 > > > [2] > > > > > > > > > https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46 > > > > > > *Emmanouil Gkatziouras* > > > https://egkatzioura.com/ | > > > https://www.linkedin.com/in/gkatziourasemmanouil/ > > > https://github.com/gkatzioura > > > > > > > > > On Sat, 30 Nov 2019 at 22:20, Saikat Maitra > > > wrote: > > > > > > > Hello, > > > > > > > > I have made changes in Ignite Extensions build configurations. > > > > > > > > It is now running the tests as expected. > > > > > > > > > > > > > > > > > > https://ci.ignite.apache.org/viewLog.html?buildId=4804480=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead > > > > > > > > Regards, > > > > Saikat > > > > > > > > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra < > saikat.mai...@gmail.com > > > > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > We do run build with -DskipTests and then configure selective test > > > suites > > > > > to run. > > > > > > > > > > I will check why these test suites are not getting executed. > > > > > > > > > > Regards > > > > > Saikat > > > > > > > > > > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras < > > > > > gkatzio...@gmail.com> wrote: > > > > > > > > > >> Hi all, > > > > >> > > > > >> I did add a test suite and run the build with the corresponding > > > > arguments, > > > > >> however the tests did not run [1]. > > > > >> I checked the Flink's build logs and the same message is displayed > > > there > > > > >> too `No tests to run` [2]. > > > > >> > > > > >> [1] > > > > >> > > > > >> > > > > > > > > > > https://ci.ignite.apache.org/viewLog.html?buildId=4798203=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=248 > > > > >> [2] > > > > >> > > > > >> > > > > > > > > > > https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build=4789199=buildLog=tree=debug=all&_focus=1256 > > > > >> > > > > >> Kind regards > > > > >> *Emmanouil Gkatziouras* > > > > >> https://egkatzioura.com/ | > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/ > > > > >> https://github.com/gkatzioura > > > > >> > > > > >> > > > > >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra < > > saikat.mai...@gmail.com> > > > > >> wrote: > > > > >> > > > > >> > Hi, > > > > >> > > > > > >> > We will need to add test suites for tests to be executed in > build. > > > We > > > > >> pass > > > > >> > them as TestSuites param. I had added Flink sink and source > > > > testsuites. > > > > >> > > > > > >> > Can you please review and confirm. > > > > >> > > > > > >> > Regards > > > > >>
Re: [DISCUSS] dependencies and release process for Ignite Extensions
Hello I have been able to resolve the build issue in teamcity and was able to pass the build failures. https://ci.ignite.apache.org/viewLog.html?buildId=4883394=IgniteExtensions_Build The issue that was causing failure was that I had put ignite.zip as artifact dependency and it was downloading the artifacts including pom file inside the build checkout folder of ignite-extensions and as a result the pom.xml of ignite-extensions was getting overwritten by ignite's pom.xml. This was resulting in missing flink-ext module error as it was not even present in the ignite main pom.xml. To resolve the issue I just had to download and extract artifact of ignite.zip in a separate directory. If I can get a +1 on this PR then I can go ahead and merge the changes. Regards, Saikat On Tue, Dec 10, 2019 at 8:26 PM Saikat Maitra wrote: > Hi Ivan, > > Thank you for sharing your feedback. I will try to install all modules in > local agent's repository and run the build. > > I will share updates. > > Regards, > Saikat > > On Tue, Dec 10, 2019 at 9:56 AM Ivan Pavlukhin > wrote: > >> Saikat, >> >> Let me share my understanding how do we run the majority of our jobs >> executing test suites. Hope it will help. >> 1. ~Build Apache Ignite~ in a separate run and publish everything as a >> single artifact ignite.zip (it contains ignite working directory with >> compiled modules, e.g. modules/core/target and others). >> 2. Download and extract ignite.zip and run particular test suites >> against extracted files. >> >> In a such setup test suites run the same as you do it on your local >> machine and it always works with a single multi-module maven project >> (apache-ignite). So, all dependencies are resolved in the working >> directory. >> >> For a separate IgniteExtensions maven project you need to install all >> dependencies from downloaded ignite.zip into .m2/repository local to >> the agent. I suppose we can do it as follows: >> 1. Download ignite.zip and extract it into "ignite" subfolder. >> 2. Run a maven command installing all modules into local maven >> repository. Something like "mvn install" but here we need to make sure >> that nothing is compiled once again. >> 3. Build IgniteExtensions. >> >> There might be a better way to do the same, I just tried to describe a >> schema how it can be done. >> >> вт, 10 дек. 2019 г. в 06:30, Saikat Maitra : >> > >> > Hi Ivan, Ilya >> > >> > Thank you for your reply. I have added you as dev in IgniteExtensions >> > project and you should be able to check the build configurations. >> > >> > Yes, when I set artifact dependencies for ~Build Apache Ignite~ similar >> to >> > other projects then I receive below error >> > >> > [ERROR] [ERROR] Could not find the selected project in the reactor: >> > :ignite-flink-ext @ >> > >> > I do not see the same error when I am running the build in local and >> > maven package command can still get 2.9.0-SNAPSHOT dependencies from my >> > local .m2 repository but it is failing in teamcity. >> > >> > Regards, >> > Saikat >> > >> > >> > >> > >> > On Mon, Dec 9, 2019 at 5:18 AM Ilya Kasnacheev < >> ilya.kasnach...@gmail.com> >> > wrote: >> > >> > > Hello! >> > > >> > > I think that your build should depend on an Apache Ignite build(s) or >> just >> > > use an already released version. >> > > >> > > In the same fashion as all our teamcity depend on "Build Apache >> Ignite" and >> > > use its artifacts. >> > > >> > > Here you want Ignite snapshot but Teamcity does not know where to >> take it >> > > from. >> > > >> > > Regards, >> > > -- >> > > Ilya Kasnacheev >> > > >> > > >> > > пн, 9 дек. 2019 г. в 06:40, Saikat Maitra : >> > > >> > > > Hello, >> > > > >> > > > I am running into a problem specific to teamcity build for Ignite >> > > > Extensions project. When I set the dependencies to 2.9.0-SNAPSHOT I >> am >> > > > getting an error message during build as below >> > > > >> > > > [06:24:12][Step 4/5] Failed to execute goal on project >> ignite-flink-ext: >> > > > Could not resolve dependencies for project >> > > > org.apache.ignite.ext:ignite-flink-ext:jar:1.0.0-SNAPSHOT: The >> following >> > > > artifacts could not be resolved: >> > > > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT, >> > > > org.apache.ignite:ignite-core:jar:tests:2.9.0-SNAPSHOT, >> > > > org.apache.ignite:ignite-log4j:jar:2.9.0-SNAPSHOT, >> > > > org.apache.ignite:ignite-spring:jar:2.9.0-SNAPSHOT: Could not find >> > > artifact >> > > > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT in h2database.com >> ( >> > > > https://h2database.com/m2-repo) >> > > > >> > > > >> > > > and if set artifact dependencies for ~Build Apache Ignite~ then I >> receive >> > > > below error >> > > > >> > > > [ERROR] [ERROR] Could not find the selected project in the reactor: >> > > > :ignite-flink-ext @ >> > > > >> > > > Build url >> > > > >> > > > >> > > >> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteExtensions_Build_IgniteExtensions=pull%2F1%2Fhead=buildTypeStatusDiv >> >
Re: Issue with replicated cache
Let me loop in the Ignite dev list as long as I've not heard about such an
issue. Personally, don't see any misconfiguration in your Ignite config.
-
Denis
On Thu, Dec 26, 2019 at 10:17 AM Prasad Bhalerao <
prasadbhalerao1...@gmail.com> wrote:
> I used cache.remove(key) method to delete an entry from cache.
>
> Basically I was not getting the consistent result on subsequent API
> calls with the same input.
>
> So I used grid gain console to query the cache. I executed the SQL on
> single node at a time.
> While doing this I found data only on node n1. But same entry was not
> present on nodes n2,n3,n4.
>
> Thanks,
> Prasad
>
>
>
>
> On Thu 26 Dec, 2019, 11:09 PM Denis Magda
>> Hello Prasad,
>>
>> What APIs did you use to remove the entry from the cache and what method
>> did you use to confirm that the entry still exists on some of the nodes?
>>
>> -
>> Denis
>>
>>
>> On Thu, Dec 26, 2019 at 8:54 AM Prasad Bhalerao <
>> prasadbhalerao1...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I am using ignite 2.6.0 version and the time out settings are as follows.
>>>
>>> IgniteConfiguration cfg = new IgniteConfiguration();
>>> cfg.setFailureDetectionTimeout(12);
>>> cfg.setNetworkTimeout(1);
>>> cfg.setClientFailureDetectionTimeout(12);
>>>
>>> I have 4 server nodes (n1,n2,n3,n4) and 6 client nodes. I am using a
>>> replicated cache and cache configuration is as shown below.
>>> As you can see write-through is false, read through is true and write
>>> synchronization mode is FULL_SYNC.
>>>
>>> I got an issue, a network entry was removed from network cache but some
>>> how it was removed from only 3 server nodes (n2,n3,n4). I was able to see
>>> the network entry on node n1 consistently for a day(when it was removed).
>>> So I checked the logs for any errors/warnings but I could not find any.
>>> I did not see any segmentation issue in logs, looked like cluster was in
>>> healthy state.
>>> When I checked the cache after 2 days, I could not find that entry.
>>> Cache was in a state as it was supposed to be. Servers were not stopped
>>> and restarted during this whole time.
>>>
>>> Some how I am not able to reproduce this issue on dev env.
>>>
>>> Is there any way to investigate/debug this issue? Can someone please
>>> advise?
>>>
>>> private CacheConfiguration networkCacheCfg() {
>>> CacheConfiguration networkCacheCfg = new
>>> CacheConfiguration<>(CacheName.NETWORK_CACHE.name());
>>> networkCacheCfg.setAtomicityMode(CacheAtomicityMode.ATOMIC);
>>> networkCacheCfg.setWriteThrough(false);
>>> networkCacheCfg.setReadThrough(true);
>>> networkCacheCfg.setRebalanceMode(CacheRebalanceMode.ASYNC);
>>>
>>> networkCacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
>>> networkCacheCfg.setBackups(this.backupCount);
>>> networkCacheCfg.setCacheMode(CacheMode.REPLICATED);
>>> Factory storeFactory =
>>> FactoryBuilder.factoryOf(NetworkDataCacheLoader.class);
>>> networkCacheCfg.setCacheStoreFactory(storeFactory);
>>> networkCacheCfg.setIndexedTypes(DefaultDataAffinityKey.class,
>>> NetworkData.class);
>>> networkCacheCfg.setSqlIndexMaxInlineSize(65);
>>> RendezvousAffinityFunction affinityFunction = new
>>> RendezvousAffinityFunction();
>>> affinityFunction.setExcludeNeighbors(true);
>>> networkCacheCfg.setAffinity(affinityFunction);
>>> networkCacheCfg.setStatisticsEnabled(true);
>>>
>>> return networkCacheCfg;
>>> }
>>>
>>>
>>>
>>> Thanks,
>>> PRasad
>>>
>>>
Re: Warning from user/dev lists mailing program
Ivan, There is no way to turn this off. Just check with the Nabble interface. Anyway, if anybody unregistered sends a message via Nabble it will arrive to a dev/user list moderator for approval. Dmitry P, I and some other folks receive those emails. - Denis On Tue, Dec 24, 2019 at 10:15 AM Ivan Pavlukhin wrote: > Denis, > > Thank you for advice! > > Also one idea came to mind. As messages sent via nabble portal might > be lost, can we disable sending messages via nabble at all? > > вт, 24 дек. 2019 г. в 20:38, Denis Magda : > > > > Ivan, > > > > Probably, INFRA team can give advice or clear things out. Please try to > > reach them out by opening a ticket in Jira. > > > > On Tuesday, December 24, 2019, Ivan Pavlukhin > wrote: > > > > > I went through all such warnings in my inbox and all they are for > > > messages sent from nabble portal [1]. Currently I have following > > > guesses: > > > 1. Something is wrong with content type. > > > 2. Something is wrong with sender address (via portal). > > > > > > [1] Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > > > > > > вт, 24 дек. 2019 г. в 15:14, Ivan Pavlukhin : > > > > > > > > > > > > > > Another one valuable opinion missed [1] (at least my inbox). > > > > > > > > [1] http://apache-ignite-developers.2346864.n4.nabble. > > > > com/Critical-worker-threads-liveness-checking-drawbacks-tp34783p34978.html > > > > > > > > вт, 24 дек. 2019 г. в 13:48, Ivan Pavlukhin : > > > > > > > > > > Actually it would be great resolve this somehow. I checked rejected > > > > > messages and found one [1] related to really important ticket. It > was > > > > > not delivered to my inbox at all =( > > > > > > > > > > [1] http://apache-ignite-developers.2346864.n4.nabble. > > > com/jira-Created-IGNITE-12259-Create-new-module-for-support- > > > spring-5-2-X-and-spring-data-2-2-X-td43881.html > > > > > > > > > > пн, 23 дек. 2019 г. в 23:27, Dmitriy Pavlov : > > > > > > > > > > > > Hi Ivan, > > > > > > > > > > > > I also receive such emails recurrently, but this program didn't > > > reject my > > > > > > subscription. So I just ignore it. > > > > > > > > > > > > Sincerely, > > > > > > Dmitriy Pavlov > > > > > > > > > > > > пн, 23 дек. 2019 г. в 18:25, Ivan Pavlukhin >: > > > > > > > > > > > > > What is also interesting here is it specific to gmail or other > > > email > > > > > > > providers are affected as well? > > > > > > > > > > > > > > And what bothers me is that some messages seems to be > undelivered > > > to > > > > > > > my inbox. I even followed ezmlm instructions to request > "bouncing" > > > > > > > messages and it worked fine. But it is inconvenient =( > > > > > > > > > > > > > > пн, 23 дек. 2019 г. в 16:33, Alexey Goncharuk < > > > alexey.goncha...@gmail.com > > > > > > > >: > > > > > > > > > > > > > > > > Hi Ivan, > > > > > > > > > > > > > > > > I receive such emails from time to time and I did not manage > to > > > > > > > > understand the conditions when this happens. From a quick > search > > > it > > > > > > > looked > > > > > > > > like it was related to the number of messages being received > per > > > certain > > > > > > > > time interval, but I could not confirm this based on the > number > > > of > > > > > > > messages > > > > > > > > on our lists. > > > > > > > > > > > > > > > > I also would be curios to know is someone solved this. > > > > > > > > > > > > > > > > пн, 23 дек. 2019 г. в 14:45, Ivan Pavlukhin < > vololo...@gmail.com > > > >: > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > From time to time I receive a following warning message > (see > > > quoted > > > > > > > > > below) from a program managing Ignite mailing lists. My > guess > > > is that > > > > > > > > > it is somehow related to gmail (spam) filtering. Does > anyone > > > > > > > > > experience the same? Does anyone know how to fix it? > > > > > > > > > > > > > > > > > > Quoted message > > > > > > > > > Hi! This is the ezmlm program. I'm managing the > > > > > > > > > u...@ignite.apache.org mailing list. > > > > > > > > > > > > > > > > > > > > > > > > > > > Messages to you from the user mailing list seem to > > > > > > > > > have been bouncing. I've attached a copy of the first > bounce > > > > > > > > > message I received. > > > > > > > > > > > > > > > > > > If this message bounces too, I will send you a probe. If > the > > > probe > > > > > > > bounces, > > > > > > > > > I will remove your address from the user mailing list, > > > > > > > > > without further notice. > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Best regards, > > > > > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Best regards, > > > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Best regards, > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > -- > > > > Best regards, > > > > Ivan Pavlukhin > > > > > > > > > > > > -- > > > Best regards, > >
Re: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
A huge +1 for adding Spring Data related fixes/improvements. Ilya is right
that Spring Data related questions sparked last time due to missing support
of 2.2 version.
Ilya, could you elaborate on what you mean under "bumping the versions"? Do
you suggest performing a straightforward upgrade of "ignite-spring-data" to
version 2.2 and introducing "ignite-spring-data-{old-version"} for the
previous versions? If it's so, I fully agree with the proposal.
-
Denis
On Thu, Dec 26, 2019 at 4:52 AM Ilya Kasnacheev
wrote:
> Hello!
>
> I propose to add the following ticket to the scope:
> https://issues.apache.org/jira/browse/IGNITE-12259 (3 commits, be careful
> with release version)
>
> Adding tickets to scope surely seems crazy now, but I will provide the
> following considerations:
> * This is Spring Data 2.2 integration, which we currently do not have,
> leading to lots of confused questions on stack overflow and mailing list.
> Spring Data is important to our public image since many people may learn
> about out project by starting with Spring Data.
>
> * It has zero code impact outside of its own module (just 2 POM file
> touched and that's all).
>
> * The core was ready since early November but, due to gmail quirk, we did
> not react to it in time.
>
> WDYT?
>
> Another semi-related question. *Should we bump our dependencies' versions
> before releasing 2.8?* I talk mainly about spring and hibernate
> dependencies. We could switch them to their latest maintenance versions to
> avoid shipping default links to outdated packages.
>
> I think this is one of things that are very hard to do between releases, so
> I think this dependencies bumping should be a part of a formal
> release/testing cycle, and then be backported to master.
>
> I could volunteer to do that myself, if we agree to merge these version
> upgrades to ignite-2.8 and then re-test.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 24 дек. 2019 г. в 13:22, Zhenya Stanilovsky >:
>
> >
> > Igniters, i`l try to compare 2.8 release candidate vs 2.7.6,
> > last sha 2.8 was build from : 9d114f3137f92aebc2562a
> > i use yardstick benchmarks, 4 bare machine with: 2x Xeon X5570 96Gb
> 512GB
> > SSD 2048GB HDD 10GB/s
> > 1 for client (driver) and 3 for servers.
> > this mappings for graphs and real yardstick tests:
> >
> > atomic-put: IgnitePutBenchmark
> > sql-merge-query: IgniteSqlMergeQueryBenchmark
> > atomic-get: IgniteGetBenchmark
> > tx-get: IgniteGetTxBenchmark
> > tx-put: IgnitePutTxBenchmark
> > atomic-put-all-bs-10: IgnitePutAllBenchmark
> > tx-put-all-bs-10: IgnitePutAllTxBenchmark
> >
> > cacheMode — partitioned
> > CacheWriteSynchronizationMode.FULL_SYNC
> > 1 backup
> >
> > 1. wal = log_only 2. wal = none 3. persistence disabled.
> > Thanks Maxim for wiki page [1]
> >
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.8#ApacheIgnite2.8-Benchmarks
> >
> > do we need some bisect or other work here ?
> >
> > >
> > >
> > >--- Forwarded message ---
> > >From: "Maxim Muzafarov" < mmu...@apache.org >
> > >To: dev@ignite.apache.org
> > >Cc:
> > >Subject: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
> > >Date: Fri, 20 Sep 2019 14:44:31 +0300
> > >
> > >Igniters,
> > >
> > >
> > >It's almost a year has passed since the last major Apache Ignite 2.7
> > >has been released. We've accumulated a lot of performance improvements
> > >and a lot of new features which are waiting for their release date.
> > >Here is my list of the most interesting things from my point since the
> > >last major release:
> > >
> > >Service Grid,
> > >Monitoring,
> > >Recovery Read
> > >BLT auto-adjust,
> > >PDS compression,
> > >WAL page compression,
> > >Thin client: best effort affinity,
> > >Thin client: transactions support (not yet)
> > >SQL query history
> > >SQL statistics
> > >
> > >I think we should no longer wait and freeze the master branch anymore
> > >and prepare the next major release by the end of the year.
> > >
> > >
> > >I propose to discuss Time, Scope of Apache Ignite 2.8 release and also
> > >I want to propose myself to be the release manager of the planning
> > >release.
> > >
> > >Scope Freeze: November 4, 2019
> > >Code Freeze: November 18, 2019
> > >Voting Date: December 10, 2019
> > >Release Date: December 17, 2019
> > >
> > >
> > >WDYT?
> >
> >
> >
> >
>
Re: Visor plugin
+1 to keep visor cmd and fix it’s issues. > 26 дек. 2019 г., в 19:23, Denis Magda написал(а): > > Personally, I see no reason for deprecating Visor CLI and moving all its > capabilities to the control.sh script. It's better to merge the script's > capabilities into Visor CLI and rework the connectivity/communication > protocol of the latter. If to recall the reason for the control.sh creation > it was the Visor's daemon-mode way of interaction with the cluster that is > cumbersome and complicated. > > - > Denis > > > On Thu, Dec 26, 2019 at 1:49 AM Ilya Kasnacheev > wrote: > >> Hello! >> >> I think that control.sh should be extended instead of Visor CLI, it is a >> tool which sees a lot more activity currently. >> >> Regards, >> -- >> Ilya Kasnacheev >> >> >> чт, 26 дек. 2019 г. в 12:45, Ivan Pavlukhin : >> >>> Folks, >>> >>> Do not we have plans to discontinue Visor? If so, it might be better >>> to add a desired functionality to another management API? >>> >>> вт, 24 дек. 2019 г. в 08:02, Denis Magda : Forwarding to the dev list. How exactly would you like to expand Visor >>> CMD? Please describe your idea and we can mover from that point. Denis On Monday, December 23, 2019, sgtech19 wrote: > Hello team, > I would like to add a new feature to the existing >>> visor > commands. Could you give me some direction on how to achieve this if >>> its > possible? Do we need a visor plugin ? if so,please provide any >> example >>> of > this plugin . > > Thanks > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > -- - Denis >>> >>> >>> >>> -- >>> Best regards, >>> Ivan Pavlukhin >>> >>
Re: Visor plugin
Personally, I see no reason for deprecating Visor CLI and moving all its capabilities to the control.sh script. It's better to merge the script's capabilities into Visor CLI and rework the connectivity/communication protocol of the latter. If to recall the reason for the control.sh creation it was the Visor's daemon-mode way of interaction with the cluster that is cumbersome and complicated. - Denis On Thu, Dec 26, 2019 at 1:49 AM Ilya Kasnacheev wrote: > Hello! > > I think that control.sh should be extended instead of Visor CLI, it is a > tool which sees a lot more activity currently. > > Regards, > -- > Ilya Kasnacheev > > > чт, 26 дек. 2019 г. в 12:45, Ivan Pavlukhin : > > > Folks, > > > > Do not we have plans to discontinue Visor? If so, it might be better > > to add a desired functionality to another management API? > > > > вт, 24 дек. 2019 г. в 08:02, Denis Magda : > > > > > > Forwarding to the dev list. How exactly would you like to expand Visor > > CMD? > > > Please describe your idea and we can mover from that point. > > > > > > Denis > > > > > > On Monday, December 23, 2019, sgtech19 wrote: > > > > > > > Hello team, > > > > I would like to add a new feature to the existing > > visor > > > > commands. Could you give me some direction on how to achieve this if > > its > > > > possible? Do we need a visor plugin ? if so,please provide any > example > > of > > > > this plugin . > > > > > > > > Thanks > > > > > > > > > > > > > > > > -- > > > > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > > > > > > > > > > > > > -- > > > - > > > Denis > > > > > > > > -- > > Best regards, > > Ivan Pavlukhin > > >
Re: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
Ilya, I agree with you that there is no risk and spring-data-2.2 can be safely cherry-picked to the ignite-2.8 branch. I'm OK with it. Will you do such merge or I should do it by myself? As for the second part of your email, you are proposing to bump up a minor dependencies version (no API changes) for the whole components mentioned in the parent/pom.xml file, right? From a point of the release view, it seems not a good thing since a scope test of the release becomes too wider. I don't think we will simplify thus the year-long release test scope, so as for me, this sounds not good but I'd like to hear thoughts of other community members on this point. As an alternative, for instance, we can bump minor versions only for those components which have security vulnerabilities. To find such dependencies, I've run some local test with a maven dependency-check-maven [1] an open-source dependency check tool. Here is a brief report (only a few modules tested): spring-core-4.3.18.RELEASE.jar : CVE-2018-15756 [2] h2-1.4.197.jar : CVE-2018-10054, CVE-2018-14335 (discussed also [3]) ignite-shmem-1.0.0.jar : CVE-2017-14614 [1] https://jeremylong.github.io/DependencyCheck/index.html [2] https://nvd.nist.gov/vuln/detail/CVE-2018-15756 [3] https://issues.apache.org/jira/browse/IGNITE-10801 On Thu, 26 Dec 2019 at 15:52, Ilya Kasnacheev wrote: > > Hello! > > I propose to add the following ticket to the scope: > https://issues.apache.org/jira/browse/IGNITE-12259 (3 commits, be careful > with release version) > > Adding tickets to scope surely seems crazy now, but I will provide the > following considerations: > * This is Spring Data 2.2 integration, which we currently do not have, > leading to lots of confused questions on stack overflow and mailing list. > Spring Data is important to our public image since many people may learn > about out project by starting with Spring Data. > > * It has zero code impact outside of its own module (just 2 POM file > touched and that's all). > > * The core was ready since early November but, due to gmail quirk, we did > not react to it in time. > > WDYT? > > Another semi-related question. *Should we bump our dependencies' versions > before releasing 2.8?* I talk mainly about spring and hibernate > dependencies. We could switch them to their latest maintenance versions to > avoid shipping default links to outdated packages. > > I think this is one of things that are very hard to do between releases, so > I think this dependencies bumping should be a part of a formal > release/testing cycle, and then be backported to master. > > I could volunteer to do that myself, if we agree to merge these version > upgrades to ignite-2.8 and then re-test. > > Regards, > -- > Ilya Kasnacheev > > > вт, 24 дек. 2019 г. в 13:22, Zhenya Stanilovsky >: > > > > > Igniters, i`l try to compare 2.8 release candidate vs 2.7.6, > > last sha 2.8 was build from : 9d114f3137f92aebc2562a > > i use yardstick benchmarks, 4 bare machine with: 2x Xeon X5570 96Gb 512GB > > SSD 2048GB HDD 10GB/s > > 1 for client (driver) and 3 for servers. > > this mappings for graphs and real yardstick tests: > > > > atomic-put: IgnitePutBenchmark > > sql-merge-query: IgniteSqlMergeQueryBenchmark > > atomic-get: IgniteGetBenchmark > > tx-get: IgniteGetTxBenchmark > > tx-put: IgnitePutTxBenchmark > > atomic-put-all-bs-10: IgnitePutAllBenchmark > > tx-put-all-bs-10: IgnitePutAllTxBenchmark > > > > cacheMode — partitioned > > CacheWriteSynchronizationMode.FULL_SYNC > > 1 backup > > > > 1. wal = log_only 2. wal = none 3. persistence disabled. > > Thanks Maxim for wiki page [1] > > > > > > [1] > > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.8#ApacheIgnite2.8-Benchmarks > > > > do we need some bisect or other work here ? > > > > > > > > > > >--- Forwarded message --- > > >From: "Maxim Muzafarov" < mmu...@apache.org > > > >To: dev@ignite.apache.org > > >Cc: > > >Subject: Apache Ignite 2.8 RELEASE [Time, Scope, Manager] > > >Date: Fri, 20 Sep 2019 14:44:31 +0300 > > > > > >Igniters, > > > > > > > > >It's almost a year has passed since the last major Apache Ignite 2.7 > > >has been released. We've accumulated a lot of performance improvements > > >and a lot of new features which are waiting for their release date. > > >Here is my list of the most interesting things from my point since the > > >last major release: > > > > > >Service Grid, > > >Monitoring, > > >Recovery Read > > >BLT auto-adjust, > > >PDS compression, > > >WAL page compression, > > >Thin client: best effort affinity, > > >Thin client: transactions support (not yet) > > >SQL query history > > >SQL statistics > > > > > >I think we should no longer wait and freeze the master branch anymore > > >and prepare the next major release by the end of the year. > > > > > > > > >I propose to discuss Time, Scope of Apache Ignite 2.8 release and also > > >I want to propose myself to be the release manager of the planning > > >release. > > > > > >Scope
Re: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
Hello! I propose to add the following ticket to the scope: https://issues.apache.org/jira/browse/IGNITE-12259 (3 commits, be careful with release version) Adding tickets to scope surely seems crazy now, but I will provide the following considerations: * This is Spring Data 2.2 integration, which we currently do not have, leading to lots of confused questions on stack overflow and mailing list. Spring Data is important to our public image since many people may learn about out project by starting with Spring Data. * It has zero code impact outside of its own module (just 2 POM file touched and that's all). * The core was ready since early November but, due to gmail quirk, we did not react to it in time. WDYT? Another semi-related question. *Should we bump our dependencies' versions before releasing 2.8?* I talk mainly about spring and hibernate dependencies. We could switch them to their latest maintenance versions to avoid shipping default links to outdated packages. I think this is one of things that are very hard to do between releases, so I think this dependencies bumping should be a part of a formal release/testing cycle, and then be backported to master. I could volunteer to do that myself, if we agree to merge these version upgrades to ignite-2.8 and then re-test. Regards, -- Ilya Kasnacheev вт, 24 дек. 2019 г. в 13:22, Zhenya Stanilovsky : > > Igniters, i`l try to compare 2.8 release candidate vs 2.7.6, > last sha 2.8 was build from : 9d114f3137f92aebc2562a > i use yardstick benchmarks, 4 bare machine with: 2x Xeon X5570 96Gb 512GB > SSD 2048GB HDD 10GB/s > 1 for client (driver) and 3 for servers. > this mappings for graphs and real yardstick tests: > > atomic-put: IgnitePutBenchmark > sql-merge-query: IgniteSqlMergeQueryBenchmark > atomic-get: IgniteGetBenchmark > tx-get: IgniteGetTxBenchmark > tx-put: IgnitePutTxBenchmark > atomic-put-all-bs-10: IgnitePutAllBenchmark > tx-put-all-bs-10: IgnitePutAllTxBenchmark > > cacheMode — partitioned > CacheWriteSynchronizationMode.FULL_SYNC > 1 backup > > 1. wal = log_only 2. wal = none 3. persistence disabled. > Thanks Maxim for wiki page [1] > > > [1] > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.8#ApacheIgnite2.8-Benchmarks > > do we need some bisect or other work here ? > > > > > > >--- Forwarded message --- > >From: "Maxim Muzafarov" < mmu...@apache.org > > >To: dev@ignite.apache.org > >Cc: > >Subject: Apache Ignite 2.8 RELEASE [Time, Scope, Manager] > >Date: Fri, 20 Sep 2019 14:44:31 +0300 > > > >Igniters, > > > > > >It's almost a year has passed since the last major Apache Ignite 2.7 > >has been released. We've accumulated a lot of performance improvements > >and a lot of new features which are waiting for their release date. > >Here is my list of the most interesting things from my point since the > >last major release: > > > >Service Grid, > >Monitoring, > >Recovery Read > >BLT auto-adjust, > >PDS compression, > >WAL page compression, > >Thin client: best effort affinity, > >Thin client: transactions support (not yet) > >SQL query history > >SQL statistics > > > >I think we should no longer wait and freeze the master branch anymore > >and prepare the next major release by the end of the year. > > > > > >I propose to discuss Time, Scope of Apache Ignite 2.8 release and also > >I want to propose myself to be the release manager of the planning > >release. > > > >Scope Freeze: November 4, 2019 > >Code Freeze: November 18, 2019 > >Voting Date: December 10, 2019 > >Release Date: December 17, 2019 > > > > > >WDYT? > > > >
Re: PME speedup #2, TX recovery delay elimination.
Ivan, The "magic numbers" are always the "magic numbers" :) We must get rid of them to see problems covered by them. >> Was there any >> performance measurements for such multiple left nodes cases? Since this fix made to speedup pme-free switch which prohibits the merges, the answer is "no". BTW, the fix was merged to master. On Thu, Dec 26, 2019 at 2:21 PM Ivan Pavlukhin wrote: > Anton, > > Thank you for your efforts! And sorry for a late reply. > > I am a little bit familiar with tx recovery. I personally like the > idea of removing such "magic" logic from the code. I think a proper > way is either justify and sustain (tests, documentation) some behavior > or get rid of it. > > Regarding a delay before tx recovery. My understanding was that it > might be useful when multiple (client) nodes leaves almost at the same > time (perhaps due to some network connectivity issues). With a delay > recovering multiple failed nodes will be grouped into one recovery > round (+PME). Correct me if my understanding is wrong. Was there any > performance measurements for such multiple left nodes cases? > > вт, 24 дек. 2019 г. в 16:22, Anton Vinogradov : > > > > Rechecked TC two more times. > > Going to merge to master in case no objections here. > > > > On Mon, Dec 23, 2019 at 1:44 PM Anton Vinogradov wrote: > > > > > Igniters, > > > > > > One more PME optimization ready to be reviewed. > > > I found a strange tx recovery delay caused by > IGNITE_TX_SALVAGE_TIMEOUT. > > > I've checked the code and tests and found no reason to delay recovery. > > > > > > So, the issue [1] is ready to be reviewed. > > > > > > Improvement checked with benchmark [2] and fix, obviously, 100 ms > faster > > > :) > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12272 > > > [2] > > > > https://github.com/anton-vinogradov/ignite/commit/f8c27253b0ecfe7381418f505aafe942efe5a0a8 > > > > > > > -- > Best regards, > Ivan Pavlukhin >
[jira] [Created] (IGNITE-12501) The AuthenticationProcessorNodeRestartTest.testConcurrentAuthorize test is flacky.
Amelchev Nikita created IGNITE-12501:
Summary: The
AuthenticationProcessorNodeRestartTest.testConcurrentAuthorize test is flacky.
Key: IGNITE-12501
URL: https://issues.apache.org/jira/browse/IGNITE-12501
Project: Ignite
Issue Type: Bug
Reporter: Amelchev Nikita
Assignee: Amelchev Nikita
Fix For: 2.9
The test is flacky. [TC
history|https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=7463074180495175640=testDetails].
(Success rate 76%)
The reason is that there is not enough check for a message about the node exit:
Now:
{noformat}
private boolean serverDownMessage(String text) {
return text.contains("Failed to send message (node may have left the
grid or "
+ "TCP connection cannot be established due to firewall issues)")
|| text.contains("Failed to send message, node left");
}
{noformat}
But message send can be failed by follow message introduced in IGNITE-7648:
{noformat}
java.lang.AssertionError: Unexpected exception: Failed to send message (node
left topology):
{noformat}
It seems we should to add this message to the check.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12500) IgniteProxy should be injected into non-system types only.
Denis Garus created IGNITE-12500: Summary: IgniteProxy should be injected into non-system types only. Key: IGNITE-12500 URL: https://issues.apache.org/jira/browse/IGNITE-12500 Project: Ignite Issue Type: Improvement Reporter: Denis Garus When the Ignite Sandbox is enabled, the GridResourceProxiedIgniteInjector should inject an IgniteProxy into non-system types only. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12499) Node took a long time to start after kill
Ivan Bessonov created IGNITE-12499: -- Summary: Node took a long time to start after kill Key: IGNITE-12499 URL: https://issues.apache.org/jira/browse/IGNITE-12499 Project: Ignite Issue Type: Bug Reporter: Ivan Bessonov Assignee: Ivan Bessonov Fix For: 2.9 Test scenario: 1) Start 4 node cluster 2) Activate 3) Load 1k rows to each cache 4) Stop node 5) Return it back without index.bin files 6) Wait until start Somehow the first node takes Waiting for topology snapshot: server(s) 4/4, client(s) 0/*, timeout 1166/1800 sec to start. [10:47:21,360][INFO][main][G] Node started : [stage="Configure system pool" (129 ms),stage="Start managers" (440 ms),stage="Configure binary metadata" (86 ms),stage="Start processors" (39341 ms),stage="Start 'GridGain' plugin" (16 ms),s tage="Init and start regions" (210 ms),stage="Restore binary memory" (228224 ms),stage="Restore logical state" (859694 ms),stage="Finish recovery" (8938 ms),stage="Join topology" (6024 ms),stage="Await transition" (16 ms),stage="Await e xchange" (14855 ms),stage="Total time" (1157973 ms)] -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: PME speedup #2, TX recovery delay elimination.
Anton, Thank you for your efforts! And sorry for a late reply. I am a little bit familiar with tx recovery. I personally like the idea of removing such "magic" logic from the code. I think a proper way is either justify and sustain (tests, documentation) some behavior or get rid of it. Regarding a delay before tx recovery. My understanding was that it might be useful when multiple (client) nodes leaves almost at the same time (perhaps due to some network connectivity issues). With a delay recovering multiple failed nodes will be grouped into one recovery round (+PME). Correct me if my understanding is wrong. Was there any performance measurements for such multiple left nodes cases? вт, 24 дек. 2019 г. в 16:22, Anton Vinogradov : > > Rechecked TC two more times. > Going to merge to master in case no objections here. > > On Mon, Dec 23, 2019 at 1:44 PM Anton Vinogradov wrote: > > > Igniters, > > > > One more PME optimization ready to be reviewed. > > I found a strange tx recovery delay caused by IGNITE_TX_SALVAGE_TIMEOUT. > > I've checked the code and tests and found no reason to delay recovery. > > > > So, the issue [1] is ready to be reviewed. > > > > Improvement checked with benchmark [2] and fix, obviously, 100 ms faster > > :) > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12272 > > [2] > > https://github.com/anton-vinogradov/ignite/commit/f8c27253b0ecfe7381418f505aafe942efe5a0a8 > > -- Best regards, Ivan Pavlukhin
Re: Improve logging of Data Region configuration
Hello! Okay, I will mark default region. We already log information about internal memory regions, so removing it will require a lot of consensus. However, I can clearly map them as system regions when printing. Regards, -- Ilya Kasnacheev чт, 26 дек. 2019 г. в 12:10, Ivan Pavlukhin : > Ilya, > > Indeed the matters can be improved. > > Is not it useful to mark what region is default? Also some doubts > about internal memory regions. It is not obvious that we should print > an information about them for every user. If we need to have some > determinism about offheap memory than I can think about logging > amounts for internal needs of total ones (a sum for all regions). > > вт, 24 дек. 2019 г. в 15:38, Ilya Kasnacheev : > > > > Hello! > > > > It came to my attention that we output data regions' configurations twice > > when starting node, but we never output list of data regions (including > > system, etc) that were actually started. > > > > First we have IgniteConfiguration printed (quiet=false): > > 2019-07-24 02:33:33.918[INFO > ][Thread-139][o.a.i.i.IgniteKernal%GridNodeName > > ] IgniteConfiguration [... dfltDataRegConf=DataRegionConfiguration [name= > > mem_plc, maxSize=635655159808, initSize=268435456, swapPath=null, > > pageEvictionMode=DISABLED, evictionThreshold=0.9, emptyPagesPoolSize=100, > > metricsEnabled=true, metricsSubIntervalCount=5, > metricsRateTimeInterval=1000 > > , persistenceEnabled=true, checkpointPageBufSize=17179869184], > storagePath=/ > > ssd/data, checkpointFreq=3, lockWaitTime=1, checkpointThreads=4, > > checkpointWriteOrder=SEQUENTIAL, walHistSize=2147483647, walSegments=10, > > walSegmentSize=1073741824, walPath=/ssd/data/wal, walArchivePath=/sas/ > > wal_archive, metricsEnabled=false, walMode=LOG_ONLY, walTlbSize=131072, > > walBuffSize=5242880, walFlushFreq=2000, walFsyncDelay=1000, > > walRecordIterBuffSize=67108864, alwaysWriteFullPages=false, > fileIOFactory= > > org.apache.ignite.internal.processors.cache.persistence.file. > > AsyncFileIOFactory@3612c49a, metricsSubIntervalCnt=5, > > metricsRateTimeInterval=6, walAutoArchiveAfterInactivity=-1, > > writeThrottlingEnabled=false, walCompactionEnabled=true, > walCompactionLevel= > > 1], ...] > > > > Then we have all configured Data Regions printed per IGNITE-8803 > > (quiet=true): > > [11:30:36] Data Regions Configured: > > [11:30:36] ^-- plcWithMetrics [initSize=256,0 MiB, maxSize=6,3 GiB, > > persistence=false, lazyMemoryAllocation=true] > > [11:30:36] ^-- plcNoMetrics [initSize=256,0 MiB, maxSize=6,3 GiB, > > persistence=false, lazyMemoryAllocation=true] > > > > Then we print number of Data Regions that were initialized as per > > IGNITE-7196, but not regions themselves (quiet=false): > > Configured data regions initialized successfully [total=4] > > > > I propose to keep the first one (IgniteConfiguration), remove the second > > one (Data Regions Configured), and promote the last one to quiet mode > while > > also outputting the regions themselves like this: > > [11:30:36] Data Regions Initialized Successfully: 4 > > [11:30:36] ^-- plcWithMetrics [initSize=256,0 MiB, maxSize=6,3 GiB, > > persistence=true, lazyMemoryAllocation=true] > > [11:30:36] ^-- plcNoMetrics [initSize=256,0 MiB, maxSize=6,3 GiB, > > persistence=true, lazyMemoryAllocation=true] > > [11:30:36] ^-- sysMemPlc [initSize=40,0 MiB, maxSize=100,0 MiB, > > persistence=true, lazyMemoryAllocation=false] > > [11:30:36] ^-- volatileMemPlc [initSize=40,0 MiB, maxSize=100,0 MiB, > > persistence=false, lazyMemoryAllocation=true] > > > > (maybe it will also include information about current usage of region in > > line with IGNITE-9305's "Metrics for local node" > > > > WDYT? > > > > Regards, > > > > -- > Best regards, > Ivan Pavlukhin >
Re: Visor plugin
Hello! I think that control.sh should be extended instead of Visor CLI, it is a tool which sees a lot more activity currently. Regards, -- Ilya Kasnacheev чт, 26 дек. 2019 г. в 12:45, Ivan Pavlukhin : > Folks, > > Do not we have plans to discontinue Visor? If so, it might be better > to add a desired functionality to another management API? > > вт, 24 дек. 2019 г. в 08:02, Denis Magda : > > > > Forwarding to the dev list. How exactly would you like to expand Visor > CMD? > > Please describe your idea and we can mover from that point. > > > > Denis > > > > On Monday, December 23, 2019, sgtech19 wrote: > > > > > Hello team, > > > I would like to add a new feature to the existing > visor > > > commands. Could you give me some direction on how to achieve this if > its > > > possible? Do we need a visor plugin ? if so,please provide any example > of > > > this plugin . > > > > > > Thanks > > > > > > > > > > > > -- > > > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > > > > > > > > > -- > > - > > Denis > > > > -- > Best regards, > Ivan Pavlukhin >
Re: Visor plugin
Folks, Do not we have plans to discontinue Visor? If so, it might be better to add a desired functionality to another management API? вт, 24 дек. 2019 г. в 08:02, Denis Magda : > > Forwarding to the dev list. How exactly would you like to expand Visor CMD? > Please describe your idea and we can mover from that point. > > Denis > > On Monday, December 23, 2019, sgtech19 wrote: > > > Hello team, > > I would like to add a new feature to the existing visor > > commands. Could you give me some direction on how to achieve this if its > > possible? Do we need a visor plugin ? if so,please provide any example of > > this plugin . > > > > Thanks > > > > > > > > -- > > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > > > > > -- > - > Denis -- Best regards, Ivan Pavlukhin
[jira] [Created] (IGNITE-12498) Calcite integration. Create a test suite on TC.
Ivan Pavlukhin created IGNITE-12498: --- Summary: Calcite integration. Create a test suite on TC. Key: IGNITE-12498 URL: https://issues.apache.org/jira/browse/IGNITE-12498 Project: Ignite Issue Type: Bug Components: sql Reporter: Ivan Pavlukhin Assignee: Ivan Pavlukhin A separate TC build job should be created. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12497) PartitionsEvictManager should log all partitions which will be evicted
Kirill Tkalenko created IGNITE-12497: Summary: PartitionsEvictManager should log all partitions which will be evicted Key: IGNITE-12497 URL: https://issues.apache.org/jira/browse/IGNITE-12497 Project: Ignite Issue Type: Improvement Reporter: Kirill Tkalenko Assignee: Kirill Tkalenko Fix For: 2.9 Now we print information about eviction only if it longer then threshold (i.e. progress). And we can't detect in logs, that partition was evicted due to different reasons (rebalance, wrong configuration custom affinity). I think we could print information on info level about each evicted partition before start of eviction. Information about partitions could be aggregated, compacted and printed by timer, but *all evicted partitions must be printed to log anyway.* I would have the following information about each partition: * partitionId * groupId * groupName * reason (eviction, clearing) -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Improve logging of Data Region configuration
Ilya, Indeed the matters can be improved. Is not it useful to mark what region is default? Also some doubts about internal memory regions. It is not obvious that we should print an information about them for every user. If we need to have some determinism about offheap memory than I can think about logging amounts for internal needs of total ones (a sum for all regions). вт, 24 дек. 2019 г. в 15:38, Ilya Kasnacheev : > > Hello! > > It came to my attention that we output data regions' configurations twice > when starting node, but we never output list of data regions (including > system, etc) that were actually started. > > First we have IgniteConfiguration printed (quiet=false): > 2019-07-24 02:33:33.918[INFO ][Thread-139][o.a.i.i.IgniteKernal%GridNodeName > ] IgniteConfiguration [... dfltDataRegConf=DataRegionConfiguration [name= > mem_plc, maxSize=635655159808, initSize=268435456, swapPath=null, > pageEvictionMode=DISABLED, evictionThreshold=0.9, emptyPagesPoolSize=100, > metricsEnabled=true, metricsSubIntervalCount=5, metricsRateTimeInterval=1000 > , persistenceEnabled=true, checkpointPageBufSize=17179869184], storagePath=/ > ssd/data, checkpointFreq=3, lockWaitTime=1, checkpointThreads=4, > checkpointWriteOrder=SEQUENTIAL, walHistSize=2147483647, walSegments=10, > walSegmentSize=1073741824, walPath=/ssd/data/wal, walArchivePath=/sas/ > wal_archive, metricsEnabled=false, walMode=LOG_ONLY, walTlbSize=131072, > walBuffSize=5242880, walFlushFreq=2000, walFsyncDelay=1000, > walRecordIterBuffSize=67108864, alwaysWriteFullPages=false, fileIOFactory= > org.apache.ignite.internal.processors.cache.persistence.file. > AsyncFileIOFactory@3612c49a, metricsSubIntervalCnt=5, > metricsRateTimeInterval=6, walAutoArchiveAfterInactivity=-1, > writeThrottlingEnabled=false, walCompactionEnabled=true, walCompactionLevel= > 1], ...] > > Then we have all configured Data Regions printed per IGNITE-8803 > (quiet=true): > [11:30:36] Data Regions Configured: > [11:30:36] ^-- plcWithMetrics [initSize=256,0 MiB, maxSize=6,3 GiB, > persistence=false, lazyMemoryAllocation=true] > [11:30:36] ^-- plcNoMetrics [initSize=256,0 MiB, maxSize=6,3 GiB, > persistence=false, lazyMemoryAllocation=true] > > Then we print number of Data Regions that were initialized as per > IGNITE-7196, but not regions themselves (quiet=false): > Configured data regions initialized successfully [total=4] > > I propose to keep the first one (IgniteConfiguration), remove the second > one (Data Regions Configured), and promote the last one to quiet mode while > also outputting the regions themselves like this: > [11:30:36] Data Regions Initialized Successfully: 4 > [11:30:36] ^-- plcWithMetrics [initSize=256,0 MiB, maxSize=6,3 GiB, > persistence=true, lazyMemoryAllocation=true] > [11:30:36] ^-- plcNoMetrics [initSize=256,0 MiB, maxSize=6,3 GiB, > persistence=true, lazyMemoryAllocation=true] > [11:30:36] ^-- sysMemPlc [initSize=40,0 MiB, maxSize=100,0 MiB, > persistence=true, lazyMemoryAllocation=false] > [11:30:36] ^-- volatileMemPlc [initSize=40,0 MiB, maxSize=100,0 MiB, > persistence=false, lazyMemoryAllocation=true] > > (maybe it will also include information about current usage of region in > line with IGNITE-9305's "Metrics for local node" > > WDYT? > > Regards, -- Best regards, Ivan Pavlukhin
[jira] [Created] (IGNITE-12496) Index deletion blocks checkpoint for all of its duration, which can cause "Critical system error: system critical thread blocked"
Denis Chudov created IGNITE-12496: - Summary: Index deletion blocks checkpoint for all of its duration, which can cause "Critical system error: system critical thread blocked" Key: IGNITE-12496 URL: https://issues.apache.org/jira/browse/IGNITE-12496 Project: Ignite Issue Type: Bug Reporter: Denis Chudov Assignee: Denis Chudov GridH2Table#removeIndex(Session, Index) acquires checkpoint read lock and releases it only after full completion of deletion process. It happens because H2TreeIndex#destroy requires to be run when checkpoint lock is held. Meanwhile, checkpoint thread stops on Checkpointer#markCheckpointBegin, trying to acquire write lock, and stays locked for all the time of index deletion. The possible fix is that checkpoint read lock is periodically released while index deletion is in progress. To avoid persistence corruption in case of node crush in the middle of the process, we should put index root into some persistent structure like index meta tree and remember it as "pending delete". Then we must delete tree pages from leafs to root, this allows to avoid links to deleted pages. When deletion is complete, tree root can be removed from "pending delete". -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: When Cache Metrics are switched on (statisticsEnabled = true) the empty cache events arrive to the client nodes
Hi Roman,
I suppose that we can resolve the ticket with 2.8 fix version if you
have no objections.
чт, 26 дек. 2019 г. в 10:52, :
>
> Hi Ivan,
> Does it mean that the problem is gone and I should close the JIRA
> IGNITE-12445 ?
>
>
> -Original Message-
> From: Ivan Pavlukhin
> Sent: Monday, December 16, 2019 11:05 PM
> To: dev
> Subject: Re: When Cache Metrics are switched on (statisticsEnabled = true)
> the empty cache events arrive to the client nodes
>
> I also checked the reproducer with current master. It seems that the problem
> is fixed there.
>
> пн, 16 дек. 2019 г. в 19:36, Ilya Kasnacheev :
> >
> > Hello!
> >
> > Is there a chance you are using Zk?
> >
> > I believe it's https://issues.apache.org/jira/browse/IGNITE-6564
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > пт, 13 дек. 2019 г. в 12:24, :
> >
> > > Hi Community,
> > >
> > > I’d like to ask you about the following behavior of Apache Ignite:
> > >
> > >
> > > If we want to react on some PUT or READ cache operations first of
> > > all we need to turn on the appropriate cache events on the server
> > > node and catch those events on the client nodes using remote approach
> > > with two listeners.
> > > It works well until we switch on statisticsEnabled on the server
> > > node, it will lead to the situation when we get empty CacheEvent objects.
> > >
> > > The example that demonstrates this issue is in the attachments. This
> > > example is consists of three nodes: 1 server node with cache and 2
> > > clients. One client is filling the cache and the second one is
> > > listening PUT operations. When we turn on Cache Metrics on the server
> > > node:
> > > cacheConfig.setStatisticsEnabled(true); in EventServerCache.java we
> > > get empty events (Sometimes CacheEvent objects with null fields.
> > > Sometimes there are no events at all)
> > >
> > > My suppose is there is some Exception in
> > > GridCacheEventManager.addEvent() when Cache Metrics is turned on.
> > >
> > > catch (Exception e) {
> > > if
> > > (!cctx.cacheObjectContext().kernalContext().cacheObjects().isBinaryEnabled(cctx.config()))
> > > throw e; if (log.isDebugEnabled())
> > > log.debug("Failed to unmarshall cache object value for the
> > > event
> > > notification: " + e);
> > >
> > > if (!forceKeepBinary)
> > > LT.warn(log, "Failed to unmarshall cache object value for the
> > > event notification " +
> > > "(all further notifications will keep binary object
> > > format).");
> > >
> > > forceKeepBinary = true;
> > >
> > > key0 = cctx.cacheObjectContext().unwrapBinaryIfNeeded(key, true,
> > > false);
> > >
> > > val0 = cctx.cacheObjectContext().unwrapBinaryIfNeeded(newVal,
> > > true, false);
> > >
> > > oldVal0 = cctx.cacheObjectContext().unwrapBinaryIfNeeded(oldVal,
> > > true, false);
> > >
> > > }
> > >
> > > Can public this point in JIRA?
> > >
> > > Best regards,
> > >
> > > T-Systems RUS
> > > Point of Production
> > > Roman Koriakov
> > > Software Developer
> > > Kirova 11, Voronezh, Russia
> > > Tel: + 7 473 200 15 30
> > > E-mail:
> > > roman.koria...@t-systems.com
> > > http://www.t-systems.com > > > > ms.com
> > > >>
> > >
> > > > http://www.t-systems.com > > http://www.t-systems.com%3chttp:/www.t-systems.ru/>>
> > >
> > > >
> > >
> > > >
> > >
> > > >
> > >
> > > > -Original Message-
> > >
> > > > From: Ilya Kasnacheev > > ilya.kasnach...@gmail.com>>
> > >
> > > > Sent: Thursday, December 12, 2019 5:25 PM
> > >
> > > > To: dev mailto:dev@ignite.apache.org>>
> > >
> > > > Subject: Re: joining
> > >
> > > >
> > >
> > > >
> > >
> > > >
> > >
> > > > Hello!
> > >
> > > >
> > >
> > > >
> > >
> > > >
> > >
> > > > I will need an Apache JIRA username to add you to contributors.
> > > > Can you
> > >
> > > > provide it?
> > >
> > > >
> > >
> > > >
> > >
