Re: Let's remove ignite-schedule module and IgniteScheduler interface
Hello! I did not find the exact information but I can confirm that ignite-schedule 1.0.0 may see some downloads, but that's all. Since 1.0.0 we don't publish this artifact so its usage also remains a mystery. Regards, -- Ilya Kasnacheev пн, 23 дек. 2019 г. в 22:37, Denis Magda : > https://repository.apache.org > > At least Ignite PMC has access to data. > > - > Denis > > > On Mon, Dec 23, 2019 at 11:35 AM Ilya Kasnacheev < > ilya.kasnach...@gmail.com> > wrote: > > > Hello! > > > > Can you guide me where these downloads are from? We don't seem to publish > > ignite-schedule to Maven Central since early 1.x. > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > сб, 21 дек. 2019 г. в 03:20, Denis Magda : > > > > > Ilya, good points, then support the idea of the API removal in 3.0. > > > > > > Ivan, downloaded the screenshot to Google Drive: > > > > > > > > > https://drive.google.com/file/d/1N21N7yqCbeZtCNs1sHvJLiJfHF_Hp0wd/view?usp=sharing > > > > > > > > > - > > > Denis > > > > > > > > > On Fri, Dec 20, 2019 at 7:09 AM Ilya Kasnacheev < > > ilya.kasnach...@gmail.com > > > > > > > wrote: > > > > > > > Hello! > > > > > > > > This module has two obvious downsides: > > > > > > > > - It's LGPL. > > > > - It can only schedule locally. > > > > > > > > We could fix 1) by using other implementation, but given 2) this no > > > longer > > > > sounds feasible. If someone wants to use local scheduler, why not > just > > > use > > > > it directly? > > > > > > > > Regards, > > > > -- > > > > Ilya Kasnacheev > > > > > > > > > > > > пт, 20 дек. 2019 г. в 10:26, Ivan Pavlukhin : > > > > > > > > > Denis, > > > > > > > > > > > The API is definitely used with even higher demand for the last > > > months > > > > > (overall the demand is comparable to Ignite Kafka and ML). See > > > > attachment. > > > > > I do not see the attachement. Where can I find it? > > > > > > > > > > чт, 19 дек. 2019 г. в 20:01, Denis Magda : > > > > > > > > > > > > The API is definitely used with even higher demand for the last > > > months > > > > > (overall the demand is comparable to Ignite Kafka and ML). See > > > > attachment. > > > > > > > > > > > > If the module has some problems let's discuss them separately and > > see > > > > > how to approach first. Do we have a list of the issues tracked > > > anywhere? > > > > > > > > > > > > > > > > > > - > > > > > > Denis > > > > > > > > > > > > > > > > > > On Thu, Dec 19, 2019 at 8:52 AM Valentin Kulichenko < > > > > > valentin.kuliche...@gmail.com> wrote: > > > > > >> > > > > > >> Ivan, > > > > > >> > > > > > >> IGFS and Hadoop Accelerator had inherent architectural flaws - > the > > > > vast > > > > > >> majority of users who tried to use these features failed to > > achieve > > > > > >> expected results. And yes, at the same time the interest was > very > > > > high, > > > > > so > > > > > >> we really needed to take action :) > > > > > >> > > > > > >> Scheduler module, on the other hand, works as expected and might > > be > > > > > used by > > > > > >> someone. There is no need to hurry. > > > > > >> > > > > > >> It probably makes sense to deprecate the functionality in 2.8 so > > > that > > > > > users > > > > > >> are aware of upcoming removal. But the removal itself should > > happen > > > in > > > > > the > > > > > >> major release. > > > > > >> > > > > > >> -Val > > > > > >> > > > > > >> On Thu, Dec 19, 2019 at 12:09 AM Ivan Pavlukhin < > > > vololo...@gmail.com> > > > > > wrote: > > > > > >> > > > > > >> > Guys, > > > > > >> > > > > > > >> > Why some of us are so critical regarding the subject? If I > > recall > > > > > >> > correctly we decided to drop IGFS and Hadoop support before > 2.8 > > > > > >> > without much debate. And it was a feature users were > interested > > > in. > > > > I > > > > > >> > never saw an interest to IgniteSchedule. My statistics is > based > > on > > > > our > > > > > >> > User mailing list. > > > > > >> > > > > > > >> > чт, 19 дек. 2019 г. в 11:00, Alexey Kuznetsov < > > > > akuznet...@apache.org > > > > > >: > > > > > >> > > > > > > > >> > > I will vote "+1" for 3.0 > > > > > >> > > > > > > > >> > > On Thu, Dec 19, 2019 at 10:57 AM Anton Vinogradov < > > > a...@apache.org> > > > > > wrote: > > > > > >> > > > > > > > >> > > > My Vote was for 3.0 > > > > > >> > > > > > > > > >> > > > On Thu, Dec 19, 2019 at 10:44 AM Valentin Kulichenko < > > > > > >> > > > valentin.kuliche...@gmail.com> wrote: > > > > > >> > > > > > > > > >> > > > > Is this suggested for 3.0 or 2.8? > > > > > >> > > > > > > > > > >> > > > > I tend to agree with Alexey - API compatibility should > be > > > > > preserved > > > > > >> > > > within > > > > > >> > > > > a major version. I would oppose doing such a change in > > 2.x. > > > > > >> > > > > > > > > > >> > > > > If this is planned for 3.0, then it's a definite +1 from > > me. > > > > > >> > > > > > > > > > >> > > > > -Val > > > > > >> > > > > > > > > > >> > > > > On Wed, Dec 18, 2019 at 11:34 PM Alexey Kuznetsov < > > > > > >> >
Re: Visor plugin
Forwarding to the dev list. How exactly would you like to expand Visor CMD? Please describe your idea and we can mover from that point. Denis On Monday, December 23, 2019, sgtech19 wrote: > Hello team, > I would like to add a new feature to the existing visor > commands. Could you give me some direction on how to achieve this if its > possible? Do we need a visor plugin ? if so,please provide any example of > this plugin . > > Thanks > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > -- - Denis
Re: Thin Clients: Renaming of AffinityAwareness to PartitionAwareness
Pavel, Thanks for taking over! - Denis On Mon, Dec 23, 2019 at 12:21 AM Pavel Tupitsyn wrote: > Since there are no objections, I'm doing the rename: > https://issues.apache.org/jira/browse/IGNITE-12458 > > On Fri, Dec 20, 2019 at 1:12 PM Igor Sapego wrote: > > > +1, sounds reasonable to me. > > > > Best Regards, > > Igor > > > > > > On Fri, Dec 20, 2019 at 10:25 AM Pavel Tupitsyn > > wrote: > > > > > +1, let's rename while we have a chance before 2.8 is released. > > > > > >
Re: Warning from user/dev lists mailing program
Hi Ivan, I also receive such emails recurrently, but this program didn't reject my subscription. So I just ignore it. Sincerely, Dmitriy Pavlov пн, 23 дек. 2019 г. в 18:25, Ivan Pavlukhin : > What is also interesting here is it specific to gmail or other email > providers are affected as well? > > And what bothers me is that some messages seems to be undelivered to > my inbox. I even followed ezmlm instructions to request "bouncing" > messages and it worked fine. But it is inconvenient =( > > пн, 23 дек. 2019 г. в 16:33, Alexey Goncharuk >: > > > > Hi Ivan, > > > > I receive such emails from time to time and I did not manage to > > understand the conditions when this happens. From a quick search it > looked > > like it was related to the number of messages being received per certain > > time interval, but I could not confirm this based on the number of > messages > > on our lists. > > > > I also would be curios to know is someone solved this. > > > > пн, 23 дек. 2019 г. в 14:45, Ivan Pavlukhin : > > > > > Hi, > > > > > > From time to time I receive a following warning message (see quoted > > > below) from a program managing Ignite mailing lists. My guess is that > > > it is somehow related to gmail (spam) filtering. Does anyone > > > experience the same? Does anyone know how to fix it? > > > > > > Quoted message > > > Hi! This is the ezmlm program. I'm managing the > > > u...@ignite.apache.org mailing list. > > > > > > > > > Messages to you from the user mailing list seem to > > > have been bouncing. I've attached a copy of the first bounce > > > message I received. > > > > > > If this message bounces too, I will send you a probe. If the probe > bounces, > > > I will remove your address from the user mailing list, > > > without further notice. > > > > > > -- > > > Best regards, > > > Ivan Pavlukhin > > > > > > > -- > Best regards, > Ivan Pavlukhin >
Re: Let's remove ignite-schedule module and IgniteScheduler interface
https://repository.apache.org At least Ignite PMC has access to data. - Denis On Mon, Dec 23, 2019 at 11:35 AM Ilya Kasnacheev wrote: > Hello! > > Can you guide me where these downloads are from? We don't seem to publish > ignite-schedule to Maven Central since early 1.x. > > Regards, > -- > Ilya Kasnacheev > > > сб, 21 дек. 2019 г. в 03:20, Denis Magda : > > > Ilya, good points, then support the idea of the API removal in 3.0. > > > > Ivan, downloaded the screenshot to Google Drive: > > > > > https://drive.google.com/file/d/1N21N7yqCbeZtCNs1sHvJLiJfHF_Hp0wd/view?usp=sharing > > > > > > - > > Denis > > > > > > On Fri, Dec 20, 2019 at 7:09 AM Ilya Kasnacheev < > ilya.kasnach...@gmail.com > > > > > wrote: > > > > > Hello! > > > > > > This module has two obvious downsides: > > > > > > - It's LGPL. > > > - It can only schedule locally. > > > > > > We could fix 1) by using other implementation, but given 2) this no > > longer > > > sounds feasible. If someone wants to use local scheduler, why not just > > use > > > it directly? > > > > > > Regards, > > > -- > > > Ilya Kasnacheev > > > > > > > > > пт, 20 дек. 2019 г. в 10:26, Ivan Pavlukhin : > > > > > > > Denis, > > > > > > > > > The API is definitely used with even higher demand for the last > > months > > > > (overall the demand is comparable to Ignite Kafka and ML). See > > > attachment. > > > > I do not see the attachement. Where can I find it? > > > > > > > > чт, 19 дек. 2019 г. в 20:01, Denis Magda : > > > > > > > > > > The API is definitely used with even higher demand for the last > > months > > > > (overall the demand is comparable to Ignite Kafka and ML). See > > > attachment. > > > > > > > > > > If the module has some problems let's discuss them separately and > see > > > > how to approach first. Do we have a list of the issues tracked > > anywhere? > > > > > > > > > > > > > > > - > > > > > Denis > > > > > > > > > > > > > > > On Thu, Dec 19, 2019 at 8:52 AM Valentin Kulichenko < > > > > valentin.kuliche...@gmail.com> wrote: > > > > >> > > > > >> Ivan, > > > > >> > > > > >> IGFS and Hadoop Accelerator had inherent architectural flaws - the > > > vast > > > > >> majority of users who tried to use these features failed to > achieve > > > > >> expected results. And yes, at the same time the interest was very > > > high, > > > > so > > > > >> we really needed to take action :) > > > > >> > > > > >> Scheduler module, on the other hand, works as expected and might > be > > > > used by > > > > >> someone. There is no need to hurry. > > > > >> > > > > >> It probably makes sense to deprecate the functionality in 2.8 so > > that > > > > users > > > > >> are aware of upcoming removal. But the removal itself should > happen > > in > > > > the > > > > >> major release. > > > > >> > > > > >> -Val > > > > >> > > > > >> On Thu, Dec 19, 2019 at 12:09 AM Ivan Pavlukhin < > > vololo...@gmail.com> > > > > wrote: > > > > >> > > > > >> > Guys, > > > > >> > > > > > >> > Why some of us are so critical regarding the subject? If I > recall > > > > >> > correctly we decided to drop IGFS and Hadoop support before 2.8 > > > > >> > without much debate. And it was a feature users were interested > > in. > > > I > > > > >> > never saw an interest to IgniteSchedule. My statistics is based > on > > > our > > > > >> > User mailing list. > > > > >> > > > > > >> > чт, 19 дек. 2019 г. в 11:00, Alexey Kuznetsov < > > > akuznet...@apache.org > > > > >: > > > > >> > > > > > > >> > > I will vote "+1" for 3.0 > > > > >> > > > > > > >> > > On Thu, Dec 19, 2019 at 10:57 AM Anton Vinogradov < > > a...@apache.org> > > > > wrote: > > > > >> > > > > > > >> > > > My Vote was for 3.0 > > > > >> > > > > > > > >> > > > On Thu, Dec 19, 2019 at 10:44 AM Valentin Kulichenko < > > > > >> > > > valentin.kuliche...@gmail.com> wrote: > > > > >> > > > > > > > >> > > > > Is this suggested for 3.0 or 2.8? > > > > >> > > > > > > > > >> > > > > I tend to agree with Alexey - API compatibility should be > > > > preserved > > > > >> > > > within > > > > >> > > > > a major version. I would oppose doing such a change in > 2.x. > > > > >> > > > > > > > > >> > > > > If this is planned for 3.0, then it's a definite +1 from > me. > > > > >> > > > > > > > > >> > > > > -Val > > > > >> > > > > > > > > >> > > > > On Wed, Dec 18, 2019 at 11:34 PM Alexey Kuznetsov < > > > > >> > akuznet...@apache.org > > > > >> > > > > > > > > >> > > > > wrote: > > > > >> > > > > > > > > >> > > > > > Hi! > > > > >> > > > > > > > > > >> > > > > > What if some users already using this module? > > > > >> > > > > > What they should do? Rewrite code? > > > > >> > > > > > I do not think it is a good idea. > > > > >> > > > > > > > > > >> > > > > > My "-1" here. > > > > >> > > > > > > > > > >> > > > > > On Thu, Dec 19, 2019 at 8:53 AM Anton Vinogradov < > > > > a...@apache.org> > > > > >> > > > wrote: > > > > >> > > > > > > > > > >> > > > > > > ignite-schedule does not look to be properly located > or > > > > useful. > > > > >> > > > > > > My +1
Re: Discovery-based services deployment guarantees question
Hi, Alexey Please attach a reproducer to the ticket. As far as I remember we have the following behaviour for the proxies: Let's assume you have deployed service from node A, then: * if you invoke service locally from node A - it is guaranteed to service to be deployed and ready to work * if you take a proxy from node A to remote node B right after deploy - there is might be a race between disco-spi (a message which releases deployed service) and comm-spi (remote call works via Compute over comm-spi), but it shouldn't affect end-users because the failed request will be retried in this case On Mon, Dec 23, 2019 at 6:55 PM Alexey Goncharuk wrote: > > Nikolay, > > Yes, I've rechecked, the new service processor is being used. I'll file a > bug shortly. > > пн, 23 дек. 2019 г. в 17:33, Николай Ижиков : > > > Alexey, are you sure, you are testing new service framework? > > > > Is yes - you definitely should file a bug. > > > > > 23 дек. 2019 г., в 17:02, Alexey Goncharuk > > написал(а): > > > > > > Igniters, > > > > > > I have a question based on one of my recent tests debugging. > > > > > > The test is related to Ignite services. I noticed that sometimes a proxy > > > invocation of a newly deployed service fails because the service cannot > > be > > > found. I managed to reduce the test to a simple "start two nodes, deploy > > a > > > service, create a proxy, invoke the proxy" scenario. The proxy invocation > > > fails in about ~80% of runs. > > > > > > As far as I remember, the new discovery-based service deployment was > > > supposed to be synchronous, so not only non-proxy service instances > > should > > > work, but the proxies as well. Was my understanding correct? Should I > > file > > > a bug for the observed behavior? > > > > > > --AG > > > > -- Best Regards, Vyacheslav D.
[jira] [Created] (IGNITE-12489) Error during purges by expiration: Unknown page type
Ruslan Kamashev created IGNITE-12489: Summary: Error during purges by expiration: Unknown page type Key: IGNITE-12489 URL: https://issues.apache.org/jira/browse/IGNITE-12489 Project: Ignite Issue Type: Bug Affects Versions: 2.7 Reporter: Ruslan Kamashev {{*logger*}} {code:java} org.apache.ignite.internal.processors.cache.GridCacheIoManager {code} {{*message*}} {code:java} Failed to process message [senderId=969d56ba-4b46-40cf-886e-ac445cf6a95d, messageType=class o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicUpdateRequest]{code} {{*thread*}} {code:java} sys-stripe-19-#20{code} {{*trace*}} {code:java} java.lang.IllegalStateException: Unknown page type: 1 pageId: 00010303117d at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.io(BPlusTree.java:5058) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$200(BPlusTree.java:90) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.nextPage(BPlusTree.java:5330) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.next(BPlusTree.java:5566) at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2232) at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2157) at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:845) at org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:207) at org.apache.ignite.internal.processors.cache.GridCacheUtils.unwindEvicts(GridCacheUtils.java:888) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessageProcessed(GridCacheIoManager.java:1103) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1076) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101) at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295) at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569) at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197) at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127) at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093) at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:505) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Dec 23, 2019 @ 18:28:28.457 {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Discovery-based services deployment guarantees question
Nikolay, Yes, I've rechecked, the new service processor is being used. I'll file a bug shortly. пн, 23 дек. 2019 г. в 17:33, Николай Ижиков : > Alexey, are you sure, you are testing new service framework? > > Is yes - you definitely should file a bug. > > > 23 дек. 2019 г., в 17:02, Alexey Goncharuk > написал(а): > > > > Igniters, > > > > I have a question based on one of my recent tests debugging. > > > > The test is related to Ignite services. I noticed that sometimes a proxy > > invocation of a newly deployed service fails because the service cannot > be > > found. I managed to reduce the test to a simple "start two nodes, deploy > a > > service, create a proxy, invoke the proxy" scenario. The proxy invocation > > fails in about ~80% of runs. > > > > As far as I remember, the new discovery-based service deployment was > > supposed to be synchronous, so not only non-proxy service instances > should > > work, but the proxies as well. Was my understanding correct? Should I > file > > a bug for the observed behavior? > > > > --AG > >
Re: Warning from user/dev lists mailing program
What is also interesting here is it specific to gmail or other email providers are affected as well? And what bothers me is that some messages seems to be undelivered to my inbox. I even followed ezmlm instructions to request "bouncing" messages and it worked fine. But it is inconvenient =( пн, 23 дек. 2019 г. в 16:33, Alexey Goncharuk : > > Hi Ivan, > > I receive such emails from time to time and I did not manage to > understand the conditions when this happens. From a quick search it looked > like it was related to the number of messages being received per certain > time interval, but I could not confirm this based on the number of messages > on our lists. > > I also would be curios to know is someone solved this. > > пн, 23 дек. 2019 г. в 14:45, Ivan Pavlukhin : > > > Hi, > > > > From time to time I receive a following warning message (see quoted > > below) from a program managing Ignite mailing lists. My guess is that > > it is somehow related to gmail (spam) filtering. Does anyone > > experience the same? Does anyone know how to fix it? > > > > Quoted message > > Hi! This is the ezmlm program. I'm managing the > > u...@ignite.apache.org mailing list. > > > > > > Messages to you from the user mailing list seem to > > have been bouncing. I've attached a copy of the first bounce > > message I received. > > > > If this message bounces too, I will send you a probe. If the probe bounces, > > I will remove your address from the user mailing list, > > without further notice. > > > > -- > > Best regards, > > Ivan Pavlukhin > > -- Best regards, Ivan Pavlukhin
Re: Re[2]: Cache operations performance metrics
Hello! Let me chime in to this discussion. If we are doing any new metrics, please make sure that they are accessible. I would expect that metrics are printed to console from time to time, at least when they deviate from norm. It would also help if they are available as Web Console screen, a system SQL view, or command.sh command - in that order. It would be ideal to start discussion of any new metrics Web-Console-screen-first. Much easier to sell to community. Let me tell about my skin in the game: as you know, I answer a large number of user questions. I ask users for logs often, so I potentially benefit from anything which is printed to logs, and I have zero benefit of something that needs extensive pre-configuration, since users would often abandon their efforts before they set up any comprehensive monitoring framework. So it would really help me as we remove useless messages from logs and add insightful messages there. This also goes for existing monitoring! If you think that we have enough metrics available, please make them more accessible to remove the need for discussion. Regards, -- Ilya Kasnacheev пт, 20 дек. 2019 г. в 16:59, Zhenya Stanilovsky : > > >> Is it become slower or faster? > > > >Very correct question! User saw "cache put time" metric becomes x2 > >bigger. Does it become slower or faster? Or we just put into the cache > >values that 4x bigger in size? Or all time before we put values > >locally and now we put values on remote nodes. Or our operations > >execute in transaction and then time will depend on transaction type, > >actions in transaction and other transaction and actually will nothing > >talk about real cache operation. We have more questions then answers. > > Andrey, i hope i understand your point of view here, but between to have > something and have nothing i choose — something, it sometimes really > helpful. From real life case: i found 1 grid machine with very different io > usage than others, «dig deeper» highlight cache with very different > from other nodes cache put operations and final «dig deeper» help to found > code bug, but to be clear — old mechanism works ok for me here, if new one > would be more useful — why not ? > > >> On the other hand - if `PuTime` increased - then we know for sure, all > operation executing `put` becomes slower. > > > >Of course not :) See above. > > > >On Fri, Dec 20, 2019 at 3:20 PM Николай Ижиков < nizhi...@apache.org > > wrote: > >> > >> > It also will be visible on other metrics > >> > >> How will it be visible? > >> > >> For example, the user saw «checkpoint time» metric becomes x2 bigger. > >> How it relates to business operations? Is it become slower or faster? > >> What does it mean for an application performance? > >> > >> On the other hand - if `PuTime` increased - then we know for sure, all > operation executing `put` becomes slower. > >> > >> *Why* it’s become slower - is the essence of «go deeper» investigation. > >> > >> > 20 дек. 2019 г., в 15:07, Andrey Gura < ag...@apache.org > > написал(а): > >> > > >> >> If a cache has some percent of the relatively slow transaction this > is a trigger to make a deeper investigation. > >> > > >> > It also will be visible on other metrics. So cache operations metrics > >> > still useless because it transitive values. > >> > > >> >>> 1. Measure some important internals (WAL operations, checkpoint > time, etc) because it can talk about real problems. > >> > > >> >> We already implement it. > >> > > >> > I don't talk that it isn't implemented. It is just example of things > >> > that should be measured. All other metrics depends on internals. > >> > > >> >>> 2. Measure business operations in user context, not cache API > operations. > >> > > >> >> Why do you think these approaches should exclude one another? > >> > > >> > Because one of them is useless. > >> > > >> > On Fri, Dec 20, 2019 at 1:43 PM Николай Ижиков < nizhi...@apache.org > > wrote: > >> >> > >> >> Hello, Andrey. > >> >> > >> >>> Where the sense in this value? I explained why this metrics are > relatively useless. > >> >> > >> >> I don’t agree with you. > >> >> I believe they are not useless for a user. > >> >> And I try to explain why I think so. > >> >> > >> >>> But user can't distinguish one transaction from another, so his > knowledge doesn't make sense definitely. > >> >> > >> >> Users shouldn’t distinguish. > >> >> If a cache has some percent of the relatively slow transaction this > is a trigger to make a deeper investigation. > >> >> > >> >>> 1. Measure some important internals (WAL operations, checkpoint > time, etc) because it can talk about real problems. > >> >> > >> >> We already implement it. > >> >> What metrics are missing for internal processes? > >> >> > >> >>> 2. Measure business operations in user context, not cache API > operations. > >> >> > >> >> Why do you think these approaches should exclude one another? > >> >> Users definitely should measure whole business transaction > performance. >
Re: Discovery-based services deployment guarantees question
Alexey, are you sure, you are testing new service framework? Is yes - you definitely should file a bug. > 23 дек. 2019 г., в 17:02, Alexey Goncharuk > написал(а): > > Igniters, > > I have a question based on one of my recent tests debugging. > > The test is related to Ignite services. I noticed that sometimes a proxy > invocation of a newly deployed service fails because the service cannot be > found. I managed to reduce the test to a simple "start two nodes, deploy a > service, create a proxy, invoke the proxy" scenario. The proxy invocation > fails in about ~80% of runs. > > As far as I remember, the new discovery-based service deployment was > supposed to be synchronous, so not only non-proxy service instances should > work, but the proxies as well. Was my understanding correct? Should I file > a bug for the observed behavior? > > --AG
[jira] [Created] (IGNITE-12488) Fix JavaDocs in DistributedMetaStorage
Ivan Bessonov created IGNITE-12488: -- Summary: Fix JavaDocs in DistributedMetaStorage Key: IGNITE-12488 URL: https://issues.apache.org/jira/browse/IGNITE-12488 Project: Ignite Issue Type: Improvement Reporter: Ivan Bessonov Assignee: Ivan Bessonov Fix For: 2.9 Some information is obsolete after https://issues.apache.org/jira/browse/IGNITE-12109 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12487) Inconsistent GridIoManager API for sendToGridTopic(Collection nodes) and sendToGridTopic(UUID nodeId)
Ivan Bessonov created IGNITE-12487: -- Summary: Inconsistent GridIoManager API for sendToGridTopic(Collection nodes) and sendToGridTopic(UUID nodeId) Key: IGNITE-12487 URL: https://issues.apache.org/jira/browse/IGNITE-12487 Project: Ignite Issue Type: Improvement Reporter: Ivan Bessonov Assignee: Ivan Bessonov Fix For: 2.9 Method {{1}}{{ctx.io().sendToGridTopic(Collection nodes, )}} will throw exception "Internal Ignite code should never call the method with local node in a node list." But at the same time {{1}}{{ctx.io().sendToGridTopic(((IgniteEx)ignite).localNode().id(), ...)}} Works without any exception. >From my point of view we should not throw exception. Processing messages in common listener is much more comfortable than writing same code twice, one for remote nodes and one for local. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Discovery-based services deployment guarantees question
Igniters, I have a question based on one of my recent tests debugging. The test is related to Ignite services. I noticed that sometimes a proxy invocation of a newly deployed service fails because the service cannot be found. I managed to reduce the test to a simple "start two nodes, deploy a service, create a proxy, invoke the proxy" scenario. The proxy invocation fails in about ~80% of runs. As far as I remember, the new discovery-based service deployment was supposed to be synchronous, so not only non-proxy service instances should work, but the proxies as well. Was my understanding correct? Should I file a bug for the observed behavior? --AG
[jira] [Created] (IGNITE-12486) Truncation of archived WAL segments doesn't work
Ivan Bessonov created IGNITE-12486: -- Summary: Truncation of archived WAL segments doesn't work Key: IGNITE-12486 URL: https://issues.apache.org/jira/browse/IGNITE-12486 Project: Ignite Issue Type: Bug Reporter: Ivan Bessonov Assignee: Ivan Bessonov Index calculation is wrong in FileWriteAheadLogManager#rollOver. It leads to unexpected and faulty WAL segments truncation and data corruption as a result. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12485) DiscoveryEvent make event message lazy initialization
Ivan Bessonov created IGNITE-12485: -- Summary: DiscoveryEvent make event message lazy initialization Key: IGNITE-12485 URL: https://issues.apache.org/jira/browse/IGNITE-12485 Project: Ignite Issue Type: Improvement Reporter: Ivan Bessonov Assignee: Ivan Bessonov Fix For: 2.9 In GridDiscoveryManager$DiscoveryWorker#recordEvent() we set to each event message: "msg " + clusterNode Invocation toString() on ClusterNode's inheritor could be expensive. I think event message could be lazy generated from event type and cluster node. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Warning from user/dev lists mailing program
Hi Ivan, I receive such emails from time to time and I did not manage to understand the conditions when this happens. From a quick search it looked like it was related to the number of messages being received per certain time interval, but I could not confirm this based on the number of messages on our lists. I also would be curios to know is someone solved this. пн, 23 дек. 2019 г. в 14:45, Ivan Pavlukhin : > Hi, > > From time to time I receive a following warning message (see quoted > below) from a program managing Ignite mailing lists. My guess is that > it is somehow related to gmail (spam) filtering. Does anyone > experience the same? Does anyone know how to fix it? > > Quoted message > Hi! This is the ezmlm program. I'm managing the > u...@ignite.apache.org mailing list. > > > Messages to you from the user mailing list seem to > have been bouncing. I've attached a copy of the first bounce > message I received. > > If this message bounces too, I will send you a probe. If the probe bounces, > I will remove your address from the user mailing list, > without further notice. > > -- > Best regards, > Ivan Pavlukhin >
[jira] [Created] (IGNITE-12484) Fix issues related to client cache stop and SQL metadata retrieval
Taras Ledkov created IGNITE-12484: - Summary: Fix issues related to client cache stop and SQL metadata retrieval Key: IGNITE-12484 URL: https://issues.apache.org/jira/browse/IGNITE-12484 Project: Ignite Issue Type: Bug Components: sql Affects Versions: 2.7 Reporter: Taras Ledkov Assignee: Taras Ledkov Fix For: 2.8 After IGNITE-6173 the problem is appeared. We need to fix it by properly handling client cache stop. Main idea is that SQL schema metadata (tables, indexes, etc.) should not be removed on client cache.close(). But the implementation should be carefully designed as many factors should be taken into account, e.g. cache destroy, persistence, client reconnect and etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Warning from user/dev lists mailing program
Hi, >From time to time I receive a following warning message (see quoted below) from a program managing Ignite mailing lists. My guess is that it is somehow related to gmail (spam) filtering. Does anyone experience the same? Does anyone know how to fix it? Quoted message Hi! This is the ezmlm program. I'm managing the u...@ignite.apache.org mailing list. Messages to you from the user mailing list seem to have been bouncing. I've attached a copy of the first bounce message I received. If this message bounces too, I will send you a probe. If the probe bounces, I will remove your address from the user mailing list, without further notice. -- Best regards, Ivan Pavlukhin
PME speedup #2, TX recovery delay elimination.
Igniters, One more PME optimization ready to be reviewed. I found a strange tx recovery delay caused by IGNITE_TX_SALVAGE_TIMEOUT. I've checked the code and tests and found no reason to delay recovery. So, the issue [1] is ready to be reviewed. Improvement checked with benchmark [2] and fix, obviously, 100 ms faster :) [1] https://issues.apache.org/jira/browse/IGNITE-12272 [2] https://github.com/anton-vinogradov/ignite/commit/f8c27253b0ecfe7381418f505aafe942efe5a0a8
[jira] [Created] (IGNITE-12483) ReflectionFactory is essential thanks to PlatformDotNetSessionLockResult
Ilya Kasnacheev created IGNITE-12483: Summary: ReflectionFactory is essential thanks to PlatformDotNetSessionLockResult Key: IGNITE-12483 URL: https://issues.apache.org/jira/browse/IGNITE-12483 Project: Ignite Issue Type: Bug Components: binary Reporter: Ilya Kasnacheev We currently treat ReflectionFactory as a nice-to-have thing, so we silently ignore failures of its reflection: {code} try { Class refFactoryCls = Class.forName("sun.reflect.ReflectionFactory"); refFac = refFactoryCls.getMethod("getReflectionFactory").invoke(null); ctorFac = refFac.getClass().getMethod("newConstructorForSerialization", Class.class, Constructor.class); } catch (NoSuchMethodException | ClassNotFoundException | IllegalAccessException | InvocationTargetException ignored) { // No-op. } {code} However, it is now essential thanks to the class PlatformDotNetSessionLockResult, which is always registered during note start-up and which does not have empty constructor. So not having access to ReflectionFactory (JBoss will hide it, for example) will lead to the following cryptic exception (courtesy stack overflow): {code} 2019-12-19 09:11:39,355 SEVERE [org.apache.ignite.internal.IgniteKernal] (ServerService Thread Pool -- 81) Got exception while starting (will rollback startup routine).: class org.apache.ignite.binary.BinaryObjectException: Failed to find empty constructor for class: org.apache.ignite.internal.processors.platform.websession.PlatformDotNetSessionLockResult at deployment.StreamsApp.ear//org.apache.ignite.internal.binary.BinaryClassDescriptor.constructor(BinaryClassDescriptor.java:981) at deployment.StreamsApp.ear//org.apache.ignite.internal.binary.BinaryClassDescriptor.(BinaryClassDescriptor.java:267) at deployment.StreamsApp.ear//org.apache.ignite.internal.binary.BinaryContext.registerPredefinedType(BinaryContext.java:1063) at deployment.StreamsApp.ear//org.apache.ignite.internal.binary.BinaryContext.registerPredefinedType(BinaryContext.java:1048) at deployment.StreamsApp.ear//org.apache.ignite.internal.binary.BinaryContext.(BinaryContext.java:350) at deployment.StreamsApp.ear//org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.start(CacheObjectBinaryProcessorImpl.java:208) at deployment.StreamsApp.ear//org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1700) at deployment.StreamsApp.ear//org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1013) at deployment.StreamsApp.ear//org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038) at org.jboss.as.ee@18.0.1.Final//org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:88) {code} My suggestions are the following: - Introduce a warning when ReflectionFactory not found instead of ignoring exception. - Add empty constructor to PlatformDotNetSessionLockResult and make sure no other classes need reflection during start-up. - (optionally) instead, introduce an error when ReflectionFactory not found. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Re[2]: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
Folks, I raised one more blocker for 2.8 [1]. It is about improper results of query execution over REPLICATED cache in presence of MOVING partitions. Fix is already on the way. Will finish with it in couple of days. [1] https://issues.apache.org/jira/browse/IGNITE-12482 пн, 23 дек. 2019 г. в 11:16, Maxim Muzafarov : > > Folks, > > > Just for your information. Here is the list of release blockers issues > at this moment: > > > [1] IGNITE-9184 [Dmitriy Sorokin] Cluster hangs during concurrent node > client and server nodes restart > [2] IGNITE-12458 [Pavel Tupitsyn] Rename Affinity Awareness to > Partition Awareness > [3] IGNITE-12227 [Anton Kalashnikov] Default auto-adjust baseline > enabled flag calculated incorrectly in some cases > [4] IGNITE-12398 [Unassigned] Apache Ignite Cluster(Amazon S3 Based > Discovery) Nodes getting down if we connect Ignite Visor Command Line > Interface > [5] IGNITE-12456 [Unassigned] Cluster Data Store grid gets Corrupted > for Load test > [6] IGNITE-12470 [Sergei Ryzhov] Pme-free switch feature should be > deactivatable > > > [1] https://issues.apache.org/jira/browse/IGNITE-9184 > [2] https://issues.apache.org/jira/browse/IGNITE-12458 > [3] https://issues.apache.org/jira/browse/IGNITE-12227 > [4] https://issues.apache.org/jira/browse/IGNITE-12398 > [5] https://issues.apache.org/jira/browse/IGNITE-12456 > [6] https://issues.apache.org/jira/browse/IGNITE-12470 > > On Mon, 23 Dec 2019 at 08:51, Anton Vinogradov wrote: > > > > >> Shouldn't we target IGNITE-12470 to 2.8? > > Sure. > > > > >> The issue [1] has pinned to 2.8 and raised as the release blocker. > > Thanks. > > > > On Fri, Dec 20, 2019 at 4:42 PM Zhenya Stanilovsky > > wrote: > > > > > > > > Folks, > > > > > > The issue [1] already in master, submit minor fix with performance > > > enhancements. > > > i think it`s not risky to gather it too. > > > thanks! > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12442 > > > > > > >Пятница, 20 декабря 2019, 16:09 +03:00 от Maxim Muzafarov < > > > mmu...@apache.org>: > > > > > > > >Folks, > > > > > > > >The issue [1] has pinned to 2.8 and raised as the release blocker. > > > > > > > >[1] https://issues.apache.org/jira/browse/IGNITE-12470 > > > > > > > >On Fri, 20 Dec 2019 at 16:04, Alexey Goncharuk > > > >< alexey.goncha...@gmail.com > wrote: > > > >> > > > >> Anton, > > > >> > > > >> Shouldn't we target IGNITE-12470 to 2.8? It kind of does not make sense > > > to > > > >> add an ability to disable potentially dangerous change only in the next > > > >> release. > > > >> > > > >> чт, 19 дек. 2019 г. в 16:18, Anton Vinogradov < a...@apache.org >: > > > >> > > > >> > Pavel, > > > >> > Issue created - https://issues.apache.org/jira/browse/IGNITE-12470 > > > >> > > > > >> > Slava, > > > >> > Does it mean we able to perform merge once I'll provide check > > > >> > results? > > > >> > > > > >> > On Thu, Dec 19, 2019 at 4:04 PM Вячеслав Коптилин < > > > >> > slava.kopti...@gmail.com > > > > >> > wrote: > > > >> > > > > >> > > Hi Anton, > > > >> > > > > > >> > > >> It would be nice to cut off a new branch from the release and > > > run all > > > >> > > the > > > >> > > > >> tests with an integrated feature and after that, > > > >> > > > >> if you don’t see new failures and the release engineer agrees > > > with > > > >> > the > > > >> > > > >> result, then and only then this feature can be merged into the > > > >> > > release. > > > >> > > > I fully agree and see no other way to perform this. > > > >> > > > PR already created and testing check scheduled. > > > >> > > > > > > >> > > This is good news, this means that we are on the same page! It was > > > not so > > > >> > > clear that the integration branch has been created and will be used > > > to > > > >> > > check all the failures. > > > >> > > > > > >> > > Thanks, > > > >> > > S. > > > >> > > > > > >> > > чт, 19 дек. 2019 г. в 15:26, Anton Vinogradov < a...@apache.org >: > > > >> > > > > > >> > > > Slava, > > > >> > > > > > > >> > > > >> It would be nice to cut off a new branch from the release and > > > run > > > >> > all > > > >> > > > the > > > >> > > > >> tests with an integrated feature and after that, > > > >> > > > >> if you don’t see new failures and the release engineer agrees > > > with > > > >> > the > > > >> > > > >> result, then and only then this feature can be merged into the > > > >> > > release. > > > >> > > > I fully agree and see no other way to perform this. > > > >> > > > PR already created and testing check scheduled. > > > >> > > > > > > >> > > > But, Maxim's proposal was to perform checks at the master branch > > > >> > > > > > > >> > > > >> 1. Merge the issue to the master branch. > > > >> > > > >> 2. Wait for two-three weeks of running tests. > > > >> > > > >> 3. Check that there are not flaky failures and fix them all > > > >> > otherwise. > > > >> > > > >> 4. Cherry-pick commit to the ignite-2.8 branch. > > > >> > > > > > > >> > > > and that's the point of discussion. > > > >> > > >
[jira] [Created] (IGNITE-12482) SQL: sql returns incorrect results for replicated caches if started on node where rebalance is in progress
Ivan Pavlukhin created IGNITE-12482: --- Summary: SQL: sql returns incorrect results for replicated caches if started on node where rebalance is in progress Key: IGNITE-12482 URL: https://issues.apache.org/jira/browse/IGNITE-12482 Project: Ignite Issue Type: Bug Components: sql Reporter: Ivan Pavlukhin Assignee: Ivan Pavlukhin Fix For: 2.8 To reproduce you should do next: 1)Start 3 data nodes with persistence 2)Load 100_000 entries to SQL cache 3)Stop one node 4)Load another100_000 entries to SQL cache 5)Stop second node 6)Load another100_000 entries to SQL cache 7)Start node one and node2 -> rebalance will be started 8)In parallel start select count query from every data node Results will be like next. SqlSize is 30, Ignite is sqltests.IncorrectSizeDuringRebalanceOfReplicatedCachesTest2 SqlSize is 122684, Ignite is sqltests.IncorrectSizeDuringRebalanceOfReplicatedCachesTest1 SqlSize is 26898, Ignite is sqltests.IncorrectSizeDuringRebalanceOfReplicatedCachesTest0 Cache sizes will be correct in this case: CacheSize is 30, Ignite is sqltests.IncorrectSizeDuringRebalanceOfReplicatedCachesTest2 CacheSize is 30, Ignite is sqltests.IncorrectSizeDuringRebalanceOfReplicatedCachesTest1 CacheSize is 30, Ignite is sqltests.IncorrectSizeDuringRebalanceOfReplicatedCachesTest0 It means that during rebalance customers will be able to get incorrect results for SQL queries during the rebalance process if it will be started from "bad" node. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Thin Clients: Renaming of AffinityAwareness to PartitionAwareness
Since there are no objections, I'm doing the rename: https://issues.apache.org/jira/browse/IGNITE-12458 On Fri, Dec 20, 2019 at 1:12 PM Igor Sapego wrote: > +1, sounds reasonable to me. > > Best Regards, > Igor > > > On Fri, Dec 20, 2019 at 10:25 AM Pavel Tupitsyn > wrote: > > > +1, let's rename while we have a chance before 2.8 is released. > > >
Re: Re[2]: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
Folks, Just for your information. Here is the list of release blockers issues at this moment: [1] IGNITE-9184 [Dmitriy Sorokin] Cluster hangs during concurrent node client and server nodes restart [2] IGNITE-12458 [Pavel Tupitsyn] Rename Affinity Awareness to Partition Awareness [3] IGNITE-12227 [Anton Kalashnikov] Default auto-adjust baseline enabled flag calculated incorrectly in some cases [4] IGNITE-12398 [Unassigned] Apache Ignite Cluster(Amazon S3 Based Discovery) Nodes getting down if we connect Ignite Visor Command Line Interface [5] IGNITE-12456 [Unassigned] Cluster Data Store grid gets Corrupted for Load test [6] IGNITE-12470 [Sergei Ryzhov] Pme-free switch feature should be deactivatable [1] https://issues.apache.org/jira/browse/IGNITE-9184 [2] https://issues.apache.org/jira/browse/IGNITE-12458 [3] https://issues.apache.org/jira/browse/IGNITE-12227 [4] https://issues.apache.org/jira/browse/IGNITE-12398 [5] https://issues.apache.org/jira/browse/IGNITE-12456 [6] https://issues.apache.org/jira/browse/IGNITE-12470 On Mon, 23 Dec 2019 at 08:51, Anton Vinogradov wrote: > > >> Shouldn't we target IGNITE-12470 to 2.8? > Sure. > > >> The issue [1] has pinned to 2.8 and raised as the release blocker. > Thanks. > > On Fri, Dec 20, 2019 at 4:42 PM Zhenya Stanilovsky > wrote: > > > > > Folks, > > > > The issue [1] already in master, submit minor fix with performance > > enhancements. > > i think it`s not risky to gather it too. > > thanks! > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12442 > > > > >Пятница, 20 декабря 2019, 16:09 +03:00 от Maxim Muzafarov < > > mmu...@apache.org>: > > > > > >Folks, > > > > > >The issue [1] has pinned to 2.8 and raised as the release blocker. > > > > > >[1] https://issues.apache.org/jira/browse/IGNITE-12470 > > > > > >On Fri, 20 Dec 2019 at 16:04, Alexey Goncharuk > > >< alexey.goncha...@gmail.com > wrote: > > >> > > >> Anton, > > >> > > >> Shouldn't we target IGNITE-12470 to 2.8? It kind of does not make sense > > to > > >> add an ability to disable potentially dangerous change only in the next > > >> release. > > >> > > >> чт, 19 дек. 2019 г. в 16:18, Anton Vinogradov < a...@apache.org >: > > >> > > >> > Pavel, > > >> > Issue created - https://issues.apache.org/jira/browse/IGNITE-12470 > > >> > > > >> > Slava, > > >> > Does it mean we able to perform merge once I'll provide check results? > > >> > > > >> > On Thu, Dec 19, 2019 at 4:04 PM Вячеслав Коптилин < > > >> > slava.kopti...@gmail.com > > > >> > wrote: > > >> > > > >> > > Hi Anton, > > >> > > > > >> > > >> It would be nice to cut off a new branch from the release and > > run all > > >> > > the > > >> > > > >> tests with an integrated feature and after that, > > >> > > > >> if you don’t see new failures and the release engineer agrees > > with > > >> > the > > >> > > > >> result, then and only then this feature can be merged into the > > >> > > release. > > >> > > > I fully agree and see no other way to perform this. > > >> > > > PR already created and testing check scheduled. > > >> > > > > > >> > > This is good news, this means that we are on the same page! It was > > not so > > >> > > clear that the integration branch has been created and will be used > > to > > >> > > check all the failures. > > >> > > > > >> > > Thanks, > > >> > > S. > > >> > > > > >> > > чт, 19 дек. 2019 г. в 15:26, Anton Vinogradov < a...@apache.org >: > > >> > > > > >> > > > Slava, > > >> > > > > > >> > > > >> It would be nice to cut off a new branch from the release and > > run > > >> > all > > >> > > > the > > >> > > > >> tests with an integrated feature and after that, > > >> > > > >> if you don’t see new failures and the release engineer agrees > > with > > >> > the > > >> > > > >> result, then and only then this feature can be merged into the > > >> > > release. > > >> > > > I fully agree and see no other way to perform this. > > >> > > > PR already created and testing check scheduled. > > >> > > > > > >> > > > But, Maxim's proposal was to perform checks at the master branch > > >> > > > > > >> > > > >> 1. Merge the issue to the master branch. > > >> > > > >> 2. Wait for two-three weeks of running tests. > > >> > > > >> 3. Check that there are not flaky failures and fix them all > > >> > otherwise. > > >> > > > >> 4. Cherry-pick commit to the ignite-2.8 branch. > > >> > > > > > >> > > > and that's the point of discussion. > > >> > > > > > >> > > > On Thu, Dec 19, 2019 at 3:14 PM Вячеслав Коптилин < > > >> > > > slava.kopti...@gmail.com > > > >> > > > wrote: > > >> > > > > > >> > > > > Hi, > > >> > > > > > > >> > > > > > We're releasing release branch, not master... why we're > > checking > > >> > the > > >> > > > > "wrong" branch? :) > > >> > > > > > Performing the release verification we're checking not only > > how > > >> > > > features > > >> > > > > work, but also how they work together. > > >> > > > > Yep, I agree that we should do verification for both branches of > > >> > corse. > > >> > > > > > > >> > > >