[REVIEW REQUEST] IGNITE-11951 Improvements in JdkMarshaller
Hi, I made some small improvements in JdkMarshaller [1]. I will be happy if someone reviews it. Changes are quite simple. [1] https://issues.apache.org/jira/browse/IGNITE-11951 -- Best regards, Ivan Pavlukhin
Re: Place for MPL 2.0/EPL 1.0 source code in ASF ecosystem?
Hen, Ignite community knows that the changes will be available under non-Apache 2.0 license and doesn't see this as a showstopper for contribution. Some community members will be contributing to the H2 fork owned by the community. The question is where can we host this fork. If the fork can be located in ASF-repositories then we'll keep it on Github. Thanks, Denis > From: Hen > Date: Tue, Jul 9, 2019 at 1:02 PM > Subject: Re: Place for MPL 2.0/EPL 1.0 source code in ASF ecosystem? > To: > Cc: dev > > > The modifications to the H2 source would need to be published under > "MPL-2.0", "EPL-1.0", or "(MPL-2.0 OR EPL-1.0)". I don't believe there > would be interest in publishing ASF-community authored source under a > different license than Apache 2.0. > > Hen > > On Tue, Jul 9, 2019 at 12:50 PM Denis Magda wrote: > >> Adding a valid legal mailing list. >> >> - >> Denis >> >> >> On Tue, Jul 9, 2019 at 12:46 PM Denis Magda wrote: >> >>> Apache Ignite's SQL engine is tightly coupled with the H2 database >>> engine [1] that provides basic parsing and query execution capabilities. >>> >>> H2 is licensed under dual MPL 2.0 and EPL 1.0 [2], thus Ignite >>> community uses H2 binary artifacts for the development needs and doesn't >>> have H2 source code in Ignite codebase. >>> >>> However, these days Ignite SQL needs to go beyond H2 capabilities and >>> we're planning to fork H2 and produce H2 binaries from there for Ignite >>> releases. To comply with legal requirements, our community thinks to host >>> the fork in a separate Github repo. Are there any alternatives? Is there a >>> legal way to host the fork in ASF repos? >>> >>> [1] https://www.h2database.com/html/main.html >>> [2] https://www.h2database.com/html/license.html >>> >>> - >>> Denis >>> >> > > -- > Удачи, > Денис Магда >
Re: Lightweight version of partitions map exchange
Hi Nikita, I've checked out your branch, looked through the changes and run IgniteBaselineNodeLeaveExchangeTest. Some thoughts: 1. First of all, there's fundamental issue that backup and primary partitions behave differently: - On primary, updating transaction needs to own exclusive lock (be on top of GridCacheMapEntry#localCandidates queue) on key object for the whole prepare-commit cycle. That's how two-phase commit works in Ignite. - Primary node generates update counter via PartitionTxUpdateCounterImpl#reserve, while backup receives update and just applies it with provided counter. So, if we'll perform PME in non-distributed way, we'll lose happen-before guarantees between updates of transactions mapped on previous topology and ones that are mapped to new topology. This may cause the following issues: - New primary node may start behaving as primary (spawn DHT transaction instances and acquire exclusive locks) but still may receive updates from previous primary. I don't know how to handle these updates correctly as they may conflict with new updates and locks. - New primary node should start generating update counters, but it actually doesn't know last update counter in cluster. If it optimistically will start from last known counter, partition consistency may break in case updates with actual last update counter will arrive (I guess, this issue should be reproduced as LWM > HWM assertion error). 2. According to current state of your test, testBltServerLeaveUnderLoad is called only with PickKeyOption#NO_DATA_ON_LEAVING_NODE (which means backups that are promoted to primaries without global synchronization are not affected by transactional load). However, it still fails with LWM > HWM assertion. I guess, there are another details in new partition counters implementation that require global happen-before between updates of transactions that are mapped to different topology versions. Alex S, backups that are promoted to primaries without global synchronization are not affected by transactional load test still fails with LWM > HWM assertion Do you have any ideas why this may happen? New primary node should start generating update counters, but it actually doesn't know last update counter in cluster. If it optimistically will start from last known counter, partition consistency may break in case updates with actual last update counter will arrive (I guess, this issue should be reproduced as LWM > HWM assertion error). How do you think, does this problem looks solvable? Alex S and Alex G, New primary node may start behaving as primary (spawn DHT transaction instances and acquire exclusive locks) but still may receive updates from previous primary. I don't know how to handle these updates correctly as they may conflict with new updates and locks. How do you think, can we overcome this limitation with our existing implementation of transactions? Best Regards, Ivan Rakov On 01.07.2019 11:13, Nikita Amelchev wrote: Hi, Igniters. I'm working on the implementation of lightweight PME for a baseline node leave case. [1] In my implementation, each node recalculates a new affinity and completes PME locally without distributed communication. This is possible because there are all partitions are distributed according to the baseline topology. And I found two possible blockers to do it without blocking updates: 1. Finalize partitions counter. It seems that we can't correctly collect gaps and process them without completing all txs. See the GridDhtPartitionTopologyImpl#finalizeUpdateCounters method. 2. Apply update counters. We can't correctly set HWM counter if primary left the cluster and sent updates to part of backups. Such updates can be processed later and break guarantee that LWM<=HWM. Is it impossible to leave a baseline node without waiting for all txs completed? 1. https://issues.apache.org/jira/browse/IGNITE-9913 ср, 5 июн. 2019 г. в 12:15, Nikita Amelchev : Maksim, I agree with you that we should implement current issue and do not allow lightweight PME if there are MOVING partitions in the cluster. But now I'm investigating issue about finalizing update counters cause it assumes that finalizing happens on exchange and all cache updates are completed. Here we can wrong process update counters gaps and break recently merged IGNITE-10078. And about phase 2, correct me if I misunderstood you. You suggest do not move primary partitions on rebalancing completing (do not change affinity assignment)? In this case, nodes recently join to cluster will not have primary partitions and won't get a load after rebalancing. чт, 30 мая 2019 г. в 19:55, Maxim Muzafarov : Igniters, I've looked through Nikita's changes and I think for the current issue [1] we should not allow the existence of MOVING partitions in the cluster (it must be stable) to run the lightweight PME on BLT node leave event occurred to achieve truly unlocked operations and here are my thoughts
Re: Lightweight version of partitions map exchange
My bad, I've sent the message accidentally. What I wanted to ask: Alex S, backups that are promoted to primaries without global synchronization are not affected by transactional load test still fails with LWM > HWM assertion Do you have any ideas why this may happen? New primary node should start generating update counters, but it actually doesn't know last update counter in cluster. If it optimistically will start from last known counter, partition consistency may break in case updates with actual last update counter will arrive (I guess, this issue should be reproduced as LWM > HWM assertion error). How do you think, does this problem looks solvable? Alex S and Alex G, New primary node may start behaving as primary (spawn DHT transaction instances and acquire exclusive locks) but still may receive updates from previous primary. I don't know how to handle these updates correctly as they may conflict with new updates and locks. How do you think, can we overcome this limitation with existing transaction implementation? Best Regards, Ivan Rakov On 10.07.2019 2:25, Ivan Rakov wrote: Hi Nikita, I've checked out your branch, looked through the changes and run IgniteBaselineNodeLeaveExchangeTest. Some thoughts: 1. First of all, there's fundamental issue that backup and primary partitions behave differently: - On primary, updating transaction needs to own exclusive lock (be on top of GridCacheMapEntry#localCandidates queue) on key object for the whole prepare-commit cycle. That's how two-phase commit works in Ignite. - Primary node generates update counter via PartitionTxUpdateCounterImpl#reserve, while backup receives update and just applies it with provided counter. So, if we'll perform PME in non-distributed way, we'll lose happen-before guarantees between updates of transactions mapped on previous topology and updates of transactions that are mapped to new topology. This may cause the following issues: - New primary node may start behaving as primary (spawn DHT transaction instances and acquire exclusive locks) but still may receive updates from previous primary. I don't know how to handle these updates correctly as they may conflict with new updates and locks. - New primary node should start generating update counters, but it actually doesn't know last update counter in cluster. If it optimistically will start from last known counter, partition consistency may break in case updates with actual last update counter will arrive (I guess, this issue should be reproduced as LWM > HWM assertion error). 2. According to current state of your test, testBltServerLeaveUnderLoad is called only with PickKeyOption#NO_DATA_ON_LEAVING_NODE (which means backups that are promoted to primaries without global synchronization are not affected by transactional load). However, it still fails with LWM > HWM assertion. I guess, there are another details in new partition counters implementation that require global happen-before between updates of transactions that are mapped to different topology versions. Alex S, backups that are promoted to primaries without global synchronization are not affected by transactional load Best Regards, Ivan Rakov On 01.07.2019 11:13, Nikita Amelchev wrote: Hi, Igniters. I'm working on the implementation of lightweight PME for a baseline node leave case. [1] In my implementation, each node recalculates a new affinity and completes PME locally without distributed communication. This is possible because there are all partitions are distributed according to the baseline topology. And I found two possible blockers to do it without blocking updates: 1. Finalize partitions counter. It seems that we can't correctly collect gaps and process them without completing all txs. See the GridDhtPartitionTopologyImpl#finalizeUpdateCounters method. 2. Apply update counters. We can't correctly set HWM counter if primary left the cluster and sent updates to part of backups. Such updates can be processed later and break guarantee that LWM<=HWM. Is it impossible to leave a baseline node without waiting for all txs completed? 1. https://issues.apache.org/jira/browse/IGNITE-9913 ср, 5 июн. 2019 г. в 12:15, Nikita Amelchev : Maksim, I agree with you that we should implement current issue and do not allow lightweight PME if there are MOVING partitions in the cluster. But now I'm investigating issue about finalizing update counters cause it assumes that finalizing happens on exchange and all cache updates are completed. Here we can wrong process update counters gaps and break recently merged IGNITE-10078. And about phase 2, correct me if I misunderstood you. You suggest do not move primary partitions on rebalancing completing (do not change affinity assignment)? In this case, nodes recently join to cluster will not have primary partitions and won't get a load after rebalancing. чт, 30 мая 2019 г. в 19:55, Maxim Muzafarov : Igniters, I've looked
Re: Lightweight version of partitions map exchange
Hi Nikita, I've checked out your branch, looked through the changes and run IgniteBaselineNodeLeaveExchangeTest. Some thoughts: 1. First of all, there's fundamental issue that backup and primary partitions behave differently: - On primary, updating transaction needs to own exclusive lock (be on top of GridCacheMapEntry#localCandidates queue) on key object for the whole prepare-commit cycle. That's how two-phase commit works in Ignite. - Primary node generates update counter via PartitionTxUpdateCounterImpl#reserve, while backup receives update and just applies it with provided counter. So, if we'll perform PME in non-distributed way, we'll lose happen-before guarantees between updates of transactions mapped on previous topology and updates of transactions that are mapped to new topology. This may cause the following issues: - New primary node may start behaving as primary (spawn DHT transaction instances and acquire exclusive locks) but still may receive updates from previous primary. I don't know how to handle these updates correctly as they may conflict with new updates and locks. - New primary node should start generating update counters, but it actually doesn't know last update counter in cluster. If it optimistically will start from last known counter, partition consistency may break in case updates with actual last update counter will arrive (I guess, this issue should be reproduced as LWM > HWM assertion error). 2. According to current state of your test, testBltServerLeaveUnderLoad is called only with PickKeyOption#NO_DATA_ON_LEAVING_NODE (which means backups that are promoted to primaries without global synchronization are not affected by transactional load). However, it still fails with LWM > HWM assertion. I guess, there are another details in new partition counters implementation that require global happen-before between updates of transactions that are mapped to different topology versions. Alex S, backups that are promoted to primaries without global synchronization are not affected by transactional load Best Regards, Ivan Rakov On 01.07.2019 11:13, Nikita Amelchev wrote: Hi, Igniters. I'm working on the implementation of lightweight PME for a baseline node leave case. [1] In my implementation, each node recalculates a new affinity and completes PME locally without distributed communication. This is possible because there are all partitions are distributed according to the baseline topology. And I found two possible blockers to do it without blocking updates: 1. Finalize partitions counter. It seems that we can't correctly collect gaps and process them without completing all txs. See the GridDhtPartitionTopologyImpl#finalizeUpdateCounters method. 2. Apply update counters. We can't correctly set HWM counter if primary left the cluster and sent updates to part of backups. Such updates can be processed later and break guarantee that LWM<=HWM. Is it impossible to leave a baseline node without waiting for all txs completed? 1. https://issues.apache.org/jira/browse/IGNITE-9913 ср, 5 июн. 2019 г. в 12:15, Nikita Amelchev : Maksim, I agree with you that we should implement current issue and do not allow lightweight PME if there are MOVING partitions in the cluster. But now I'm investigating issue about finalizing update counters cause it assumes that finalizing happens on exchange and all cache updates are completed. Here we can wrong process update counters gaps and break recently merged IGNITE-10078. And about phase 2, correct me if I misunderstood you. You suggest do not move primary partitions on rebalancing completing (do not change affinity assignment)? In this case, nodes recently join to cluster will not have primary partitions and won't get a load after rebalancing. чт, 30 мая 2019 г. в 19:55, Maxim Muzafarov : Igniters, I've looked through Nikita's changes and I think for the current issue [1] we should not allow the existence of MOVING partitions in the cluster (it must be stable) to run the lightweight PME on BLT node leave event occurred to achieve truly unlocked operations and here are my thoughts why. In general, as Nikita mentioned above, the existence of MOVING partitions in the cluster means that the rebalance procedure is currently running. It owns cache partitions locally and sends in the background (with additional timeout) the actual statuses of his local partitions to the coordinator node. So, we will always have a lag between local node partition states and all other cluster nodes partitions states. This lag can be very huge since previous #scheduleResendPartitions() is cancelled when a new cache group rebalance finished. Without the fair partition states synchronization (without full PME) and in case of local affinity recalculation on BLT node leave event, other nodes will mark such partitions LOST in most of the cases, which in fact are present in the cluster and saved on some node under checkpoint. I see that it cannot be
Re: .NET: excessive memory usage by continuous queries
Hi Denis, from SO question and https://issues.apache.org/jira/browse/IGNITE-11970 ticket description it seems to me it is not platform/.NET specific. According to issue description proposed fix is not looking as overly complex. May be ticket reporter can provide a fix and run TC (since there is strict algorithm). As always, there is a risk we will break some test with the proposed solution and we'll need to find another. Sincerely, Dmitriy Pavlov ср, 10 июл. 2019 г. в 00:43, Denis Magda : > Pavel, Igor, Aleksandr, > > > Could you please check this thread: > > > > https://stackoverflow.com/questions/56803904/java-outofmemoryexception-when-using-continuous-query-in-ignite > > Here is a related ticket: > https://issues.apache.org/jira/browse/IGNITE-11970 > > Is it easy to implement this optimization? Can we make it to the next > Ignite release? > - > Denis >
Re: Is Ignite planning to support COMMENT ON statements?
Hi, I'm not aware of such plans, for now. Maybe SQL experts (CCed) can provide some more details. Sincerely, Dmitriy Pavlov вс, 7 июл. 2019 г. в 15:39, liyuj <18624049...@163.com>: > Hi, > > Is Ignite planning to support COMMENT ON statements? > > H2 supports this command. > If Ignite can support this command, it will be helpful to increase the > maintainability of tables. > > >
.NET: excessive memory usage by continuous queries
Pavel, Igor, Aleksandr, Could you please check this thread: https://stackoverflow.com/questions/56803904/java-outofmemoryexception-when-using-continuous-query-in-ignite Here is a related ticket: https://issues.apache.org/jira/browse/IGNITE-11970 Is it easy to implement this optimization? Can we make it to the next Ignite release? - Denis
Re: Who is owning this account
There is the "ignite-dev" user that manages Nubble forums linked to the dev list: http://apache-ignite-developers.2346864.n4.nabble.com/template/NamlServlet.jtp?macro=user_nodes=1 Do you we know a password used for that account? Dmitry, is it in that passwords page available for the PMC group? I'd like to create "ignite-user" account for the user list Nubble and add it to the same private page in our repo. - Denis On Mon, Jul 8, 2019 at 11:54 PM Dmitriy Pavlov wrote: > Which account do you mean? > > I've checked both email and list thread > > https://lists.apache.org/thread.html/9eeec05720f62a6f581ef169bec680f2d2291b02da9ad3d69bd13c76@%3Cdev.ignite.apache.org%3E > and > I can't find mentioning or reply. > > вт, 9 июл. 2019 г. в 02:05, : > > > Who owns this community account? Please repond > > > > _ > > Sent from http://apache-ignite-developers.2346864.n4.nabble.com > > > > >
Re: Place for MPL 2.0/EPL 1.0 source code in ASF ecosystem?
The modifications to the H2 source would need to be published under "MPL-2.0", "EPL-1.0", or "(MPL-2.0 OR EPL-1.0)". I don't believe there would be interest in publishing ASF-community authored source under a different license than Apache 2.0. Hen On Tue, Jul 9, 2019 at 12:50 PM Denis Magda wrote: > Adding a valid legal mailing list. > > - > Denis > > > On Tue, Jul 9, 2019 at 12:46 PM Denis Magda wrote: > >> Apache Ignite's SQL engine is tightly coupled with the H2 database engine >> [1] that provides basic parsing and query execution capabilities. >> >> H2 is licensed under dual MPL 2.0 and EPL 1.0 [2], thus Ignite community >> uses H2 binary artifacts for the development needs and doesn't have H2 >> source code in Ignite codebase. >> >> However, these days Ignite SQL needs to go beyond H2 capabilities and >> we're planning to fork H2 and produce H2 binaries from there for Ignite >> releases. To comply with legal requirements, our community thinks to host >> the fork in a separate Github repo. Are there any alternatives? Is there a >> legal way to host the fork in ASF repos? >> >> [1] https://www.h2database.com/html/main.html >> [2] https://www.h2database.com/html/license.html >> >> - >> Denis >> >
Re: Place for MPL 2.0/EPL 1.0 source code in ASF ecosystem?
Adding a valid legal mailing list. - Denis On Tue, Jul 9, 2019 at 12:46 PM Denis Magda wrote: > Apache Ignite's SQL engine is tightly coupled with the H2 database engine > [1] that provides basic parsing and query execution capabilities. > > H2 is licensed under dual MPL 2.0 and EPL 1.0 [2], thus Ignite community > uses H2 binary artifacts for the development needs and doesn't have H2 > source code in Ignite codebase. > > However, these days Ignite SQL needs to go beyond H2 capabilities and > we're planning to fork H2 and produce H2 binaries from there for Ignite > releases. To comply with legal requirements, our community thinks to host > the fork in a separate Github repo. Are there any alternatives? Is there a > legal way to host the fork in ASF repos? > > [1] https://www.h2database.com/html/main.html > [2] https://www.h2database.com/html/license.html > > - > Denis >
Place for MPL 2.0/EPL 1.0 source code in ASF ecosystem?
Apache Ignite's SQL engine is tightly coupled with the H2 database engine [1] that provides basic parsing and query execution capabilities. H2 is licensed under dual MPL 2.0 and EPL 1.0 [2], thus Ignite community uses H2 binary artifacts for the development needs and doesn't have H2 source code in Ignite codebase. However, these days Ignite SQL needs to go beyond H2 capabilities and we're planning to fork H2 and produce H2 binaries from there for Ignite releases. To comply with legal requirements, our community thinks to host the fork in a separate Github repo. Are there any alternatives? Is there a legal way to host the fork in ASF repos? [1] https://www.h2database.com/html/main.html [2] https://www.h2database.com/html/license.html - Denis
Re: [DISCUSSION][IEP-35] Metrics configuration
Igniters, I made a PR for metrics configuration. Please, review https://github.com/apache/ignite/pull/6676/files В Вт, 09/07/2019 в 12:27 +0300, Nikolay Izhikov пишет: > Hello, Alex. > > OK, Let's go with the simplest solution. > I will provide API and JMX method for metrics configuration shortly. > > В Пн, 08/07/2019 в 18:23 +0300, Alexey Goncharuk пишет: > > Nikolay, > > > > To me a separate metrics configuration file seems to be not very > > user-friendly. First of all, it does not allow to configure the system only > > from Java code. Second, having multiple configuration files seem to be > > quite confusing for end users (judging by the logging configuration > > questions). > > > > Perhaps, we will still end up with the configuration file - but for now I > > would put this aside for a more thorough brainstorm and added the JMX and > > internal API for changing metrics configuration. > > > > пт, 5 июл. 2019 г. в 14:17, Seliverstov Igor : > > > > > Igniters, > > > > > > One more question on topic. > > > > > > Should we preserve metrics configuration on restart? (I think we should) > > > > > > If so, which configuration use after restart? Defined in config file or > > > saved in config storage? (I guess, saved configuration should have a > > > priority) > > > > > > So, how to tell users that any changes in configuration file have no > > > effect on Ignite configuration after first start? > > > > > > I think there are too many open questions and (at least at now) we should > > > provide only JMX API until all of the questions are clarified. > > > > > > Regards, > > > Igor > > > > > > > 4 июля 2019 г., в 19:55, Nikolay Izhikov > > > > > > написал(а): > > > > > > > > Hello, Andrey. > > > > > > > > > 3. I can't imagine that adequate values will be chosen on project > > > > > setup stage. > > > > > > > > Configuration file required in the case we adds new node or replace > > > > > > existing to the cluster. > > > > Use can have parameters similar to Ignite configuration, log > > > > > > configuration files. > > > > > > > > > My proposal is adding API for boundaries configuration to the metrics > > > > > framework and expose it via JMX > > > > > > > > Agree. I think we should have both: > > > > > > > > 1. Configuration file. > > > > 2. JMX API to change bounaries of histogram *and HitRateMetric params*. > > > > > > > > But, if you and other community member are against config file, let's > > > > > > have only JMX. > > > > Seems, JMX will provide required level of configurability for metrics. > > > > > > > > > > > > В Чт, 04/07/2019 в 17:53 +0300, Andrey Gura пишет: > > > > > Igniters, > > > > > > > > > > I rethought the issue and I see some problems: > > > > > > > > > > 1. It seems that in most cases bucket boundaries configuration will be > > > > > problem for user. Absolute values for latency boundaries it is very > > > > > odd choice. > > > > > 2. Also seems that latency for most caches (if we configure cache > > > > > metrics fro example) will be similar. > > > > > 3. I can't imagine that adequate values will be chosen on project > > > > > setup stage. So chosen values should be changed in the future. > > > > > > > > > > Solution with configuration file looks unnatural and creates more > > > > > problems than could solve. > > > > > > > > > > My proposal is adding API for boundaries configuration to the metrics > > > > > framework and expose it via JMX (at this step). It still provides > > > > > configuration possibility but don't force user to do it. > > > > > > > > > > Also we should chose default values for bucket boundaries. And it is > > > > > most complex problem at the moment :) Let's discuss it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 3, 2019 at 4:49 PM Andrey Gura wrote: > > > > > > > > > > > > Nikolai, > > > > > > > > > > > > Metric is disabled if it doesn't allocate any memory and doesn't > > > > > > update any variable because doesn't have any value. Ideally > > > > > > disabling > > > > > > metrics for some cache should be equal to cache stopping. > > > > > > > > > > > > On Fri, Jun 28, 2019 at 1:02 PM Nikolay Izhikov > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > Hello, Alexey. > > > > > > > > > > > > > > Thanks for the feedback! > > > > > > > > > > > > > > > My only concert is that we should have the metrics framework > > > > > > configuration > > > > > > > > as the first-citizen of the framework itself > > > > > > > > > > > > > > Yes. I planned to add `void configure(String param)` method to the > > > > > > metric API. > > > > > > > > > > > > > > > but change the metrics parameters in > > > > > > > > runtime from JMX or command-line, etc. > > > > > > > > > > > > > > I've add requirement of JMX method to the ticket: > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-11927 > > > > > > > > > > > > > > > Another concern is to have an > > > > > > > > ability to
[jira] [Created] (IGNITE-11970) Excessive use of memory in continuous queries
Igor Belyakov created IGNITE-11970: -- Summary: Excessive use of memory in continuous queries Key: IGNITE-11970 URL: https://issues.apache.org/jira/browse/IGNITE-11970 Project: Ignite Issue Type: Bug Reporter: Igor Belyakov When we prepare to send an entry into the continuous query's filter and listener, we store it in an instance of CacheContinuousQueryEventBuffer.Batch. The batch is an array of entries of size IGNITE_CONTINUOUS_QUERY_SERVER_BUFFER_SIZE (default is 1000) that stores the currently received entries (we need it for the case of concurrent updates to make sure that we preserve the order of update counters). The issue is that when we process a part of the array we keep the links to the processed entries until we exhaust the array (after when we finally clear it). Because of that we may store up to 999 garbage objects which can be a lot if the entries are big. Need to clear the entries right after we've processed them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [DISCUSSION][IEP-35] Metrics configuration
Hello, Alex. OK, Let's go with the simplest solution. I will provide API and JMX method for metrics configuration shortly. В Пн, 08/07/2019 в 18:23 +0300, Alexey Goncharuk пишет: > Nikolay, > > To me a separate metrics configuration file seems to be not very > user-friendly. First of all, it does not allow to configure the system only > from Java code. Second, having multiple configuration files seem to be > quite confusing for end users (judging by the logging configuration > questions). > > Perhaps, we will still end up with the configuration file - but for now I > would put this aside for a more thorough brainstorm and added the JMX and > internal API for changing metrics configuration. > > пт, 5 июл. 2019 г. в 14:17, Seliverstov Igor : > > > Igniters, > > > > One more question on topic. > > > > Should we preserve metrics configuration on restart? (I think we should) > > > > If so, which configuration use after restart? Defined in config file or > > saved in config storage? (I guess, saved configuration should have a > > priority) > > > > So, how to tell users that any changes in configuration file have no > > effect on Ignite configuration after first start? > > > > I think there are too many open questions and (at least at now) we should > > provide only JMX API until all of the questions are clarified. > > > > Regards, > > Igor > > > > > 4 июля 2019 г., в 19:55, Nikolay Izhikov > > > > написал(а): > > > > > > Hello, Andrey. > > > > > > > 3. I can't imagine that adequate values will be chosen on project > > > > setup stage. > > > > > > Configuration file required in the case we adds new node or replace > > > > existing to the cluster. > > > Use can have parameters similar to Ignite configuration, log > > > > configuration files. > > > > > > > My proposal is adding API for boundaries configuration to the metrics > > > > framework and expose it via JMX > > > > > > Agree. I think we should have both: > > > > > > 1. Configuration file. > > > 2. JMX API to change bounaries of histogram *and HitRateMetric params*. > > > > > > But, if you and other community member are against config file, let's > > > > have only JMX. > > > Seems, JMX will provide required level of configurability for metrics. > > > > > > > > > В Чт, 04/07/2019 в 17:53 +0300, Andrey Gura пишет: > > > > Igniters, > > > > > > > > I rethought the issue and I see some problems: > > > > > > > > 1. It seems that in most cases bucket boundaries configuration will be > > > > problem for user. Absolute values for latency boundaries it is very > > > > odd choice. > > > > 2. Also seems that latency for most caches (if we configure cache > > > > metrics fro example) will be similar. > > > > 3. I can't imagine that adequate values will be chosen on project > > > > setup stage. So chosen values should be changed in the future. > > > > > > > > Solution with configuration file looks unnatural and creates more > > > > problems than could solve. > > > > > > > > My proposal is adding API for boundaries configuration to the metrics > > > > framework and expose it via JMX (at this step). It still provides > > > > configuration possibility but don't force user to do it. > > > > > > > > Also we should chose default values for bucket boundaries. And it is > > > > most complex problem at the moment :) Let's discuss it. > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 3, 2019 at 4:49 PM Andrey Gura wrote: > > > > > > > > > > Nikolai, > > > > > > > > > > Metric is disabled if it doesn't allocate any memory and doesn't > > > > > update any variable because doesn't have any value. Ideally disabling > > > > > metrics for some cache should be equal to cache stopping. > > > > > > > > > > On Fri, Jun 28, 2019 at 1:02 PM Nikolay Izhikov > > > > wrote: > > > > > > > > > > > > Hello, Alexey. > > > > > > > > > > > > Thanks for the feedback! > > > > > > > > > > > > > My only concert is that we should have the metrics framework > > > > configuration > > > > > > > as the first-citizen of the framework itself > > > > > > > > > > > > Yes. I planned to add `void configure(String param)` method to the > > > > metric API. > > > > > > > > > > > > > but change the metrics parameters in > > > > > > > runtime from JMX or command-line, etc. > > > > > > > > > > > > I've add requirement of JMX method to the ticket: > > > > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-11927 > > > > > > > > > > > > > Another concern is to have an > > > > > > > ability to disable/enable metrics per metrics group/prefix. > > > > > > > > > > > > Yes, we discusss it. > > > > > > But, let's make it clear: > > > > > > > > > > > > *What is disabling metric?* > > > > > > > > > > > > Looks like exporter filter solve this task. > > > > > > > > > > > > В Чт, 27/06/2019 в 16:24 +0300, Alexey Goncharuk пишет: > > > > > > > Nikolay, > > > > > > > > > > > > > > My only concert is that we should have the metrics framework > > > >
[jira] [Created] (IGNITE-11969) Incorrect DefaultConcurrencyLevel value in .net test
Anton Kalashnikov created IGNITE-11969: -- Summary: Incorrect DefaultConcurrencyLevel value in .net test Key: IGNITE-11969 URL: https://issues.apache.org/jira/browse/IGNITE-11969 Project: Ignite Issue Type: Test Reporter: Anton Kalashnikov Assignee: Anton Kalashnikov Incorrect DefaultConcurrencyLevel value in .net test after default configuration in java was changed -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11968) FailureProcessor reports tcp-comm-worker as blocked when resolving communication errors in ZK
Stanislav Lukyanov created IGNITE-11968:
---
Summary: FailureProcessor reports tcp-comm-worker as blocked when
resolving communication errors in ZK
Key: IGNITE-11968
URL: https://issues.apache.org/jira/browse/IGNITE-11968
Project: Ignite
Issue Type: Improvement
Affects Versions: 2.7
Reporter: Stanislav Lukyanov
When multiple clients fail (stopped) at the same time and ZK discovery is used
FailureProcessor treats tcp-comm-worker as blocked with the following trace:
{code}
Thread [name="tcp-comm-worker-#1", id=122, state=WAITING, blockCnt=1465,
waitCnt=4557]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at
o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:178)
at o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141)
at
o.a.i.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.resolveCommunicationError(ZookeeperDiscoveryImpl.java:359)
at
o.a.i.spi.discovery.zk.ZookeeperDiscoverySpi.resolveCommunicationFailure(ZookeeperDiscoverySpi.java:259)
at
o.a.i.i.managers.discovery.GridDiscoveryManager.resolveCommunicationError(GridDiscoveryManager.java:2549)
at
o.a.i.i.managers.GridManagerAdapter$1.resolveCommunicationFailure(GridManagerAdapter.java:626)
at
o.a.i.spi.communication.tcp.TcpCommunicationSpi.processClientCreationError(TcpCommunicationSpi.java:3530)
at
o.a.i.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3483)
at
o.a.i.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:2987)
at
o.a.i.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2870)
at
o.a.i.spi.communication.tcp.TcpCommunicationSpi.access$6000(TcpCommunicationSpi.java:271)
at
o.a.i.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.processDisconnect(TcpCommunicationSpi.java:4489)
at
o.a.i.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.body(TcpCommunicationSpi.java:4294)
at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
at
o.a.i.spi.communication.tcp.TcpCommunicationSpi$5.body(TcpCommunicationSpi.java:2237)
at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Who is owning this account
Which account do you mean? I've checked both email and list thread https://lists.apache.org/thread.html/9eeec05720f62a6f581ef169bec680f2d2291b02da9ad3d69bd13c76@%3Cdev.ignite.apache.org%3E and I can't find mentioning or reply. вт, 9 июл. 2019 г. в 02:05, : > Who owns this community account? Please repond > > _ > Sent from http://apache-ignite-developers.2346864.n4.nabble.com > >
