Re: New Ignite PMC Member: Ilya Kasnacheev
Ilya, my congratulations! вт, 6 авг. 2019 г. в 23:12, Dmitriy Pavlov : > > Ilya, congratulations! > > вт, 6 авг. 2019 г. в 22:58, Denis Magda : > > > The Project Management Committee (PMC) for Apache Ignite > > has invited Ilya Kasnacheev to become a committer and we are pleased > > to announce that he has accepted. > > > > Ilya is one of the most active and valuable Ignite committers who is not > > only contributing source-code changes but also takes an active stance on > > Ignite adoption - he constantly helps Ignite users to design, troubleshoot > > and optimize their Ignite deployments. > > > > Being a PMC member enables assistance with the management and to guide the > > direction of the project. > > > > Please join me in congratulating Ilya with this new role! > > > > > > - > > Denis > > -- Best regards, Ivan Pavlukhin
Re: New Ignite PMC Member: Ilya Kasnacheev
Ilya, congratulations! вт, 6 авг. 2019 г. в 22:58, Denis Magda : > The Project Management Committee (PMC) for Apache Ignite > has invited Ilya Kasnacheev to become a committer and we are pleased > to announce that he has accepted. > > Ilya is one of the most active and valuable Ignite committers who is not > only contributing source-code changes but also takes an active stance on > Ignite adoption - he constantly helps Ignite users to design, troubleshoot > and optimize their Ignite deployments. > > Being a PMC member enables assistance with the management and to guide the > direction of the project. > > Please join me in congratulating Ilya with this new role! > > > - > Denis >
New Ignite PMC Member: Ilya Kasnacheev
The Project Management Committee (PMC) for Apache Ignite has invited Ilya Kasnacheev to become a committer and we are pleased to announce that he has accepted. Ilya is one of the most active and valuable Ignite committers who is not only contributing source-code changes but also takes an active stance on Ignite adoption - he constantly helps Ignite users to design, troubleshoot and optimize their Ignite deployments. Being a PMC member enables assistance with the management and to guide the direction of the project. Please join me in congratulating Ilya with this new role! - Denis
[jira] [Created] (IGNITE-12047) senderNodeId is absent in StatusCheckMessage
Denis Chudov created IGNITE-12047:
-
Summary: senderNodeId is absent in StatusCheckMessage
Key: IGNITE-12047
URL: https://issues.apache.org/jira/browse/IGNITE-12047
Project: Ignite
Issue Type: Bug
Reporter: Denis Chudov
Assignee: Denis Chudov
Fix For: 2.8
TcpDiscoveryCoordinatorFailureTest.testClusterFailedNewCoordinatorInitialized
{code:java}
[2019-07-29
02:42:11,609][ERROR][tcp-disco-sock-reader-[]-#21611%tcp.TcpDiscoveryCoordinatorFailureTest4%][TcpDiscoverySpi]
Failed to initialize connection (this can happen due to short time network
problems and can be ignored if does not affect node discovery)
[sock=Socket[addr=/127.0.0.1,port=44033,localport=47504]]
java.net.SocketTimeoutException: Read timed out at
java.net.SocketInputStream.socketRead0(Native Method) at
java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at
java.net.SocketInputStream.read(SocketInputStream.java:171) at
java.net.SocketInputStream.read(SocketInputStream.java:141) at
java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at
java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at
java.io.BufferedInputStream.read(BufferedInputStream.java:345) at
org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:6464)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:60)
Exception in thread
"disco-pool-#62924%tcp.TcpDiscoveryCoordinatorFailureTest4%" Exception in
thread "disco-pool-#62925%tcp.TcpDiscoveryCoordinatorFailureTest4%"
java.lang.AssertionError at
org.apache.ignite.spi.discovery.tcp.ServerImpl.pingNode(ServerImpl.java:723) at
org.apache.ignite.spi.discovery.tcp.ServerImpl.access$4000(ServerImpl.java:195)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker$8.run(ServerImpl.java:5598)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) java.lang.AssertionError at
org.apache.ignite.spi.discovery.tcp.ServerImpl.pingNode(ServerImpl.java:723) at
org.apache.ignite.spi.discovery.tcp.ServerImpl.access$4000(ServerImpl.java:195)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker$8.run(ServerImpl.java:5598)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (IGNITE-12046) [IEP-35] LongMetric interface contains redundant methods.
Andrey Gura created IGNITE-12046:
Summary: [IEP-35] LongMetric interface contains redundant methods.
Key: IGNITE-12046
URL: https://issues.apache.org/jira/browse/IGNITE-12046
Project: Ignite
Issue Type: Improvement
Reporter: Andrey Gura
Assignee: Andrey Gura
Fix For: 2.8
The following methods on {{LongMetric}} interface a redundant and have the same
functionality:
* {{get}}
* {{longValue}}
Mentioned methods should be removed.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
Re: [DISCUSSION][IEP-35] Metrics configuration
Andrey, It seems that screenshot was rejected, I do not see it. вт, 6 авг. 2019 г. в 15:07, Andrey Gura : > > Good example of proper buckets configuration. Such configuration is > suitable for may cases. See attached screenshot (I hope it will not be > reject by mail system or forum). > > > On Tue, Aug 6, 2019 at 2:45 PM Andrey Gura wrote: > > > > > What do you mean by "exponential bounds"? > > > > Something like this if we talk about latency in ms for example: 5, 10, > > 25, 50, 100, 200, 500, ... > > > > > Thanks, for the feedback, appreciate you ownesty. > > > > Nothing personal. It is just about functionality from user's stand point. > > > > > What is your proposal? > > > How metrics configuration should work? > > > > My proposal is simple: just drop this change. We don't need the > > configuration. Metric owner (developer) defines buckets' bounds for > > each particular case (it could be done uniformly or exponentially, it > > depends on metric and problem definition). > > > > On Mon, Aug 5, 2019 at 6:36 PM Nikolay Izhikov wrote: > > > > > > Hello, Andrey. > > > > > > > Not necessary if we have exponential bounds' values for histograms. > > > > > > What do you mean by "exponential bounds"? > > > > > > > Anyway, in current solution it looks ugly and not usable. > > > > > > Thanks, for the feedback, appreciate you ownesty. > > > > > > > No. But we should admit that this is bad decision and do not include > > > > this change to the code base. > > > > > > What is your proposal? > > > How metrics configuration should work? > > > > > > > Yes. But it still will not give enough accuracy. > > > > > > Enough for what? > > > > > > В Пн, 05/08/2019 в 18:29 +0300, Andrey Gura пишет: > > > > > > - metric configuration is node local (not cluster wide). > > > > > This issue is easy to solve on the user-side and in Ignite core. > > > > > > > > It's imaginary simplicity. The first, you need some additional > > > > automation on user-side in order to configure all nodes of the > > > > cluster. The second, new nodes can join to the cluster and > > > > configuration will be different on new node and on other nodes of the > > > > cluster. This leads to complication whole functionality. Anyway, I > > > > don't like such simplified solution because at the moment it brings > > > > more problems than value. > > > > > > > > > The easiest solution was implemented. > > > > > Do we want to make it more complex right now :)? > > > > > > > > No. But we should admit that this is bad decision and do not include > > > > this change to the code base. > > > > > > > > > The reason it exists in PR - we already have this parameter in > > > > > DataStorageConfiguration#getMetricsSubIntervalCount > > > > > > > > I believe this method should be deprecated and removed in major release. > > > > > > > > > I think the user should be able to configure buckets for histogram > > > > > and rateTimeInterval for hitrate. > > > > > > > > Not necessary if we have exponential bounds' values for histograms. > > > > Anyway, in current solution it looks ugly and not usable. > > > > > > > > > Ignite has dozens of use-cases and deployment modes, seems, > > > > > we can't cover it all with the single predefined > > > > > buckets/rateTimeInterval set. > > > > > > > > Yes. But it still will not give enough accuracy. > > > > > > > > On Mon, Aug 5, 2019 at 5:25 PM Nikolay Izhikov > > > > wrote: > > > > > > > > > > Hello, Andrey. > > > > > > > > > > > - metric configuration is node local (not cluster wide). > > > > > > > > > > This issue is easy to solve on the user-side and in Ignite core. > > > > > > > > > > > - metric configuration doesn't survive node restart. > > > > > > > > > > We decide to go with the simplest solution, for now. > > > > > The easiest solution was implemented. > > > > > Do we want to make it more complex right now :)? > > > > > > > > > > > - User shouldn't configure hit rate metrics at runtime in most > > > > > > cases. > > > > > > > > > > I agree with you - the size of the counters array looks odd as a > > > > > configuration parameter. > > > > > The reason it exists in PR - we already have this parameter in > > > > > DataStorageConfiguration#getMetricsSubIntervalCount > > > > > > > > > > > - May be it is enough for user to have histograms with > > > > > > pre-configured buckets > > > > > > So I think we should drop this change and idea about runtime > > > > > > histrogram and hit rate configuration. > > > > > > > > > > I think the user should be able to configure buckets for histogram > > > > > and rateTimeInterval for hitrate. > > > > > > > > > > Ignite has dozens of use-cases and deployment modes, seems, > > > > > we can't cover it all with the single predefined > > > > > buckets/rateTimeInterval set. > > > > > > > > > > В Пн, 05/08/2019 в 16:59 +0300, Andrey Gura пишет: > > > > > > Igniters, > > > > > > > > > > > > I've took a look to the PR and I want follow up this discussion > > > > > > again. > > > > > > > > > > > >
SecurityTestSuite as a separate test suite at TC
Hello Igniters! I made the test DoPrivelegedOnRemoteNodeTest[1] (SecurityTestSuite) for the task "Sandbox for user-defined code" [2] that uses p2p deploy like the test ServiceHotRedeploymentViaDeploymentSpiTest [3] from IgniteServiceGridTestSuite. That test requires additional Maven command line parameter -P surefire-fork-count-1. The suite Basic 1 contains the SecurityTestSuite and many other test suites at TeamCity that do not need that additional Maven parameter. I suggest extracting SecurityTestSuite as a separate test suite to define additional Maven command line parameter for it. WDYT? 1. https://github.com/apache/ignite/pull/6707 2. https://issues.apache.org/jira/browse/IGNITE-11410 3. https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/service/ServiceHotRedeploymentViaDeploymentSpiTest.java
Re: [DISCUSSION][IEP-35] Metrics configuration
Good example of proper buckets configuration. Such configuration is suitable for may cases. See attached screenshot (I hope it will not be reject by mail system or forum). On Tue, Aug 6, 2019 at 2:45 PM Andrey Gura wrote: > > > What do you mean by "exponential bounds"? > > Something like this if we talk about latency in ms for example: 5, 10, > 25, 50, 100, 200, 500, ... > > > Thanks, for the feedback, appreciate you ownesty. > > Nothing personal. It is just about functionality from user's stand point. > > > What is your proposal? > > How metrics configuration should work? > > My proposal is simple: just drop this change. We don't need the > configuration. Metric owner (developer) defines buckets' bounds for > each particular case (it could be done uniformly or exponentially, it > depends on metric and problem definition). > > On Mon, Aug 5, 2019 at 6:36 PM Nikolay Izhikov wrote: > > > > Hello, Andrey. > > > > > Not necessary if we have exponential bounds' values for histograms. > > > > What do you mean by "exponential bounds"? > > > > > Anyway, in current solution it looks ugly and not usable. > > > > Thanks, for the feedback, appreciate you ownesty. > > > > > No. But we should admit that this is bad decision and do not include this > > > change to the code base. > > > > What is your proposal? > > How metrics configuration should work? > > > > > Yes. But it still will not give enough accuracy. > > > > Enough for what? > > > > В Пн, 05/08/2019 в 18:29 +0300, Andrey Gura пишет: > > > > > - metric configuration is node local (not cluster wide). > > > > This issue is easy to solve on the user-side and in Ignite core. > > > > > > It's imaginary simplicity. The first, you need some additional > > > automation on user-side in order to configure all nodes of the > > > cluster. The second, new nodes can join to the cluster and > > > configuration will be different on new node and on other nodes of the > > > cluster. This leads to complication whole functionality. Anyway, I > > > don't like such simplified solution because at the moment it brings > > > more problems than value. > > > > > > > The easiest solution was implemented. > > > > Do we want to make it more complex right now :)? > > > > > > No. But we should admit that this is bad decision and do not include > > > this change to the code base. > > > > > > > The reason it exists in PR - we already have this parameter in > > > > DataStorageConfiguration#getMetricsSubIntervalCount > > > > > > I believe this method should be deprecated and removed in major release. > > > > > > > I think the user should be able to configure buckets for histogram and > > > > rateTimeInterval for hitrate. > > > > > > Not necessary if we have exponential bounds' values for histograms. > > > Anyway, in current solution it looks ugly and not usable. > > > > > > > Ignite has dozens of use-cases and deployment modes, seems, > > > > we can't cover it all with the single predefined > > > > buckets/rateTimeInterval set. > > > > > > Yes. But it still will not give enough accuracy. > > > > > > On Mon, Aug 5, 2019 at 5:25 PM Nikolay Izhikov > > > wrote: > > > > > > > > Hello, Andrey. > > > > > > > > > - metric configuration is node local (not cluster wide). > > > > > > > > This issue is easy to solve on the user-side and in Ignite core. > > > > > > > > > - metric configuration doesn't survive node restart. > > > > > > > > We decide to go with the simplest solution, for now. > > > > The easiest solution was implemented. > > > > Do we want to make it more complex right now :)? > > > > > > > > > - User shouldn't configure hit rate metrics at runtime in most cases. > > > > > > > > I agree with you - the size of the counters array looks odd as a > > > > configuration parameter. > > > > The reason it exists in PR - we already have this parameter in > > > > DataStorageConfiguration#getMetricsSubIntervalCount > > > > > > > > > - May be it is enough for user to have histograms with pre-configured > > > > > buckets > > > > > So I think we should drop this change and idea about runtime > > > > > histrogram and hit rate configuration. > > > > > > > > I think the user should be able to configure buckets for histogram and > > > > rateTimeInterval for hitrate. > > > > > > > > Ignite has dozens of use-cases and deployment modes, seems, > > > > we can't cover it all with the single predefined > > > > buckets/rateTimeInterval set. > > > > > > > > В Пн, 05/08/2019 в 16:59 +0300, Andrey Gura пишет: > > > > > Igniters, > > > > > > > > > > I've took a look to the PR and I want follow up this discussion again. > > > > > > > > > > Proposed solution has a couple of significant drawbacks: > > > > > > > > > > - metric configuration is node local (not cluster wide). > > > > > - metric configuration doesn't survive node restart. > > > > > > > > > > This drawbacks make configuration complex, annoying and useless in > > > > > most cases. > > > > > > > > > > Moreover, I think that: > > > > > > > > > > -
Re: Replacing NodeFilter functionality with label approach
Alexey, It seems that a problem has a solution with using 2 attributes or 2 labels. Is not it more clear than using custom code? Folks, > I don't think we should take "hard to implement" as an argument in this > discussion :) Did not fully get the point. KISS principle is not true anymore? Or is this discussion somehow special? I believe that every flexibility handle should be critically justified. Would be great to justify NodeFilter flexibility. > Filters based of hostname or ip address. Is it a good idea to use IP address for node filtering? IP can be changed for a node with persistence, does it mean that not relevant data (according to a filter) should be cleared, does it work now? And there is also one idea (I am not fan of it but still). Can we use some kind of scripting for nodes filtering? In that case node filter is represented by script string, e.g. javascript. вт, 6 авг. 2019 г. в 12:22, Alexey Kukushkin : > > Pavel, > > Just a real example you asked for: we have a user attribute "ROLE_DC", > which is a comma separated list like "wfe_a, as_a, db_a" (server role and > data center designator) and we have node filters to deploy services and > start caches on servers with specific role (like WFE) and sometimes > specific role and DC (like WFE_A). The node filter splits the list and uses > a regular expression to match each segment. > > If you replace generic node filter with a user attribute filter then we > still can achieve what we need by creating 3 user attributes (ROLE_DC, > ROLE and DC) but we lose flexibility since now adding a new data center > requires updating all cache and service configurations. With regex matching > we do not need to update the configurations since we still match all the > roles in the new DC. > > So we would have a solution with user attributes filter but I we lose some > flexibility. > > -- > Best regards, > Alexey -- Best regards, Ivan Pavlukhin
Re: [DISCUSSION][IEP-35] Metrics configuration
> What do you mean by "exponential bounds"? Something like this if we talk about latency in ms for example: 5, 10, 25, 50, 100, 200, 500, ... > Thanks, for the feedback, appreciate you ownesty. Nothing personal. It is just about functionality from user's stand point. > What is your proposal? > How metrics configuration should work? My proposal is simple: just drop this change. We don't need the configuration. Metric owner (developer) defines buckets' bounds for each particular case (it could be done uniformly or exponentially, it depends on metric and problem definition). On Mon, Aug 5, 2019 at 6:36 PM Nikolay Izhikov wrote: > > Hello, Andrey. > > > Not necessary if we have exponential bounds' values for histograms. > > What do you mean by "exponential bounds"? > > > Anyway, in current solution it looks ugly and not usable. > > Thanks, for the feedback, appreciate you ownesty. > > > No. But we should admit that this is bad decision and do not include this > > change to the code base. > > What is your proposal? > How metrics configuration should work? > > > Yes. But it still will not give enough accuracy. > > Enough for what? > > В Пн, 05/08/2019 в 18:29 +0300, Andrey Gura пишет: > > > > - metric configuration is node local (not cluster wide). > > > This issue is easy to solve on the user-side and in Ignite core. > > > > It's imaginary simplicity. The first, you need some additional > > automation on user-side in order to configure all nodes of the > > cluster. The second, new nodes can join to the cluster and > > configuration will be different on new node and on other nodes of the > > cluster. This leads to complication whole functionality. Anyway, I > > don't like such simplified solution because at the moment it brings > > more problems than value. > > > > > The easiest solution was implemented. > > > Do we want to make it more complex right now :)? > > > > No. But we should admit that this is bad decision and do not include > > this change to the code base. > > > > > The reason it exists in PR - we already have this parameter in > > > DataStorageConfiguration#getMetricsSubIntervalCount > > > > I believe this method should be deprecated and removed in major release. > > > > > I think the user should be able to configure buckets for histogram and > > > rateTimeInterval for hitrate. > > > > Not necessary if we have exponential bounds' values for histograms. > > Anyway, in current solution it looks ugly and not usable. > > > > > Ignite has dozens of use-cases and deployment modes, seems, > > > we can't cover it all with the single predefined buckets/rateTimeInterval > > > set. > > > > Yes. But it still will not give enough accuracy. > > > > On Mon, Aug 5, 2019 at 5:25 PM Nikolay Izhikov wrote: > > > > > > Hello, Andrey. > > > > > > > - metric configuration is node local (not cluster wide). > > > > > > This issue is easy to solve on the user-side and in Ignite core. > > > > > > > - metric configuration doesn't survive node restart. > > > > > > We decide to go with the simplest solution, for now. > > > The easiest solution was implemented. > > > Do we want to make it more complex right now :)? > > > > > > > - User shouldn't configure hit rate metrics at runtime in most cases. > > > > > > I agree with you - the size of the counters array looks odd as a > > > configuration parameter. > > > The reason it exists in PR - we already have this parameter in > > > DataStorageConfiguration#getMetricsSubIntervalCount > > > > > > > - May be it is enough for user to have histograms with pre-configured > > > > buckets > > > > So I think we should drop this change and idea about runtime histrogram > > > > and hit rate configuration. > > > > > > I think the user should be able to configure buckets for histogram and > > > rateTimeInterval for hitrate. > > > > > > Ignite has dozens of use-cases and deployment modes, seems, > > > we can't cover it all with the single predefined buckets/rateTimeInterval > > > set. > > > > > > В Пн, 05/08/2019 в 16:59 +0300, Andrey Gura пишет: > > > > Igniters, > > > > > > > > I've took a look to the PR and I want follow up this discussion again. > > > > > > > > Proposed solution has a couple of significant drawbacks: > > > > > > > > - metric configuration is node local (not cluster wide). > > > > - metric configuration doesn't survive node restart. > > > > > > > > This drawbacks make configuration complex, annoying and useless in most > > > > cases. > > > > > > > > Moreover, I think that: > > > > > > > > - User shouldn't configure hit rate metrics at runtime in most cases. > > > > Especially HitRateMetric.size because it's just details of > > > > implementation. Purpose of size is plots smoothing and this parameter > > > > could be fixed (e.g. 16 is enough). HitRate metric is just LongMetric > > > > but with additional feature. > > > > - May be it is enough for user to have histograms with pre-configured > > > > buckets. The trick here is properly chosen bounds. It seems
Re: Replacing NodeFilter functionality with label approach
Pavel, Just a real example you asked for: we have a user attribute "ROLE_DC", which is a comma separated list like "wfe_a, as_a, db_a" (server role and data center designator) and we have node filters to deploy services and start caches on servers with specific role (like WFE) and sometimes specific role and DC (like WFE_A). The node filter splits the list and uses a regular expression to match each segment. If you replace generic node filter with a user attribute filter then we still can achieve what we need by creating 3 user attributes (ROLE_DC, ROLE and DC) but we lose flexibility since now adding a new data center requires updating all cache and service configurations. With regex matching we do not need to update the configurations since we still match all the roles in the new DC. So we would have a solution with user attributes filter but I we lose some flexibility. -- Best regards, Alexey
