Re: Hibernate 5.3 merged!
Hi Denis, Thanks for the response. spring is indeed Apache 2.0, see https://github.com/spring-projects/spring-framework/blob/master/src/docs/dist/license.txt Am I missing something? On Fri, Dec 21, 2018 at 22:37 Denis Magda wrote: > Scott, Ilya, > > LGPL license is not compliant with Apache 2.0 and, thus, ASF project can't > include binaries of LGPL libs to ASF releases: > https://www.apache.org/legal/resolved.html#category-x > > Spring doesn't belong to ASF. It has its own rules and policies. > > -- > Denis > > > On Thu, Dec 20, 2018 at 5:04 AM Ilya Kasnacheev > > wrote: > > > Hello! > > > > I don't think we can push it to public repositories in binary form. No > > change in that regard. As for Spring, I honestly have no idea. Maybe > > they're dual license? > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > чт, 20 дек. 2018 г. в 01:32, Scott Feldstein : > > > > > Hi Ilya, that’s great news! > > > > > > What’s the likelihood of pushing the artifacts to a public maven repo > > along > > > with the other ignite artifacts? Are we constrained by the LGPL2 > license? > > > If so, I wonder how spring works around that with their jpa impl? > > > > > > Scott > > > On Wed, Dec 19, 2018 at 09:03 Ilya Kasnacheev < > ilya.kasnach...@gmail.com > > > > > > wrote: > > > > > > > Hello! > > > > > > > > I have just checked, and it looks like hibernate-5.3 module works > > > correctly > > > > with Hibernate 5.4.0.Final! > > > > > > > > It builds without errors and all tests pass. So maybe we can rebrand > it > > > as > > > > hibernate-5.3+ for the duration. > > > > > > > > Regards, > > > > -- > > > > Ilya Kasnacheev > > > > > > > > > > > > ср, 19 дек. 2018 г. в 19:53, Павлухин Иван : > > > > > > > > > Good news! > > > > > > > > > > I also believe that a lot of users will be happy once it is > released. > > > > > Cheers! > > > > > > > > > > By the way, I noticed that the latest Hibernate version is 5.4. It > is > > > > > compatible with 5.3? Should it be somehow addressed by Ignite > > > > > developers? > > > > > ср, 19 дек. 2018 г. в 19:09, Ilya Kasnacheev < > > > ilya.kasnach...@gmail.com > > > > >: > > > > > > > > > > > > Hello! > > > > > > > > > > > > I am proud to announce that I have just merged Hibernate 5.3 > > module, > > > > > > courtesy Scott Feldstein. > > > > > > > > > > > > There were also important fixes which should make TC greener than > > > ever. > > > > > > > > > > > > Since this is a large patch there might be rough edges, please > > notify > > > > me > > > > > if > > > > > > anything went awry, I will try to follow it up. > > > > > > > > > > > > Regards, > > > > > > -- > > > > > > Ilya Kasnacheev > > > > > > > > > > > > > > > > > > > > -- > > > > > Best regards, > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > >
Re: Hibernate 5.3 merged!
Scott, Ilya, LGPL license is not compliant with Apache 2.0 and, thus, ASF project can't include binaries of LGPL libs to ASF releases: https://www.apache.org/legal/resolved.html#category-x Spring doesn't belong to ASF. It has its own rules and policies. -- Denis On Thu, Dec 20, 2018 at 5:04 AM Ilya Kasnacheev wrote: > Hello! > > I don't think we can push it to public repositories in binary form. No > change in that regard. As for Spring, I honestly have no idea. Maybe > they're dual license? > > Regards, > -- > Ilya Kasnacheev > > > чт, 20 дек. 2018 г. в 01:32, Scott Feldstein : > > > Hi Ilya, that’s great news! > > > > What’s the likelihood of pushing the artifacts to a public maven repo > along > > with the other ignite artifacts? Are we constrained by the LGPL2 license? > > If so, I wonder how spring works around that with their jpa impl? > > > > Scott > > On Wed, Dec 19, 2018 at 09:03 Ilya Kasnacheev > > > wrote: > > > > > Hello! > > > > > > I have just checked, and it looks like hibernate-5.3 module works > > correctly > > > with Hibernate 5.4.0.Final! > > > > > > It builds without errors and all tests pass. So maybe we can rebrand it > > as > > > hibernate-5.3+ for the duration. > > > > > > Regards, > > > -- > > > Ilya Kasnacheev > > > > > > > > > ср, 19 дек. 2018 г. в 19:53, Павлухин Иван : > > > > > > > Good news! > > > > > > > > I also believe that a lot of users will be happy once it is released. > > > > Cheers! > > > > > > > > By the way, I noticed that the latest Hibernate version is 5.4. It is > > > > compatible with 5.3? Should it be somehow addressed by Ignite > > > > developers? > > > > ср, 19 дек. 2018 г. в 19:09, Ilya Kasnacheev < > > ilya.kasnach...@gmail.com > > > >: > > > > > > > > > > Hello! > > > > > > > > > > I am proud to announce that I have just merged Hibernate 5.3 > module, > > > > > courtesy Scott Feldstein. > > > > > > > > > > There were also important fixes which should make TC greener than > > ever. > > > > > > > > > > Since this is a large patch there might be rough edges, please > notify > > > me > > > > if > > > > > anything went awry, I will try to follow it up. > > > > > > > > > > Regards, > > > > > -- > > > > > Ilya Kasnacheev > > > > > > > > > > > > > > > > -- > > > > Best regards, > > > > Ivan Pavlukhin > > > > > > > > > >
[jira] [Created] (IGNITE-10795) Create Ignite.NET Dockerfile
Pavel Tupitsyn created IGNITE-10795: --- Summary: Create Ignite.NET Dockerfile Key: IGNITE-10795 URL: https://issues.apache.org/jira/browse/IGNITE-10795 Project: Ignite Issue Type: Improvement Components: platforms Reporter: Pavel Tupitsyn Assignee: Pavel Tupitsyn Make it possible to run Ignite.NET in Docker with a single command. It is possible for Java-only nodes (see docker\apache-ignite\Dockerfile). The following Gist has a POC: https://gist.github.com/ptupitsyn/1cbbdaef1fef7cc4be22addda19cade4 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10794) MVCC: RemoveAll fails with assertion on unstable topology
Andrew Mashenkov created IGNITE-10794:
-
Summary: MVCC: RemoveAll fails with assertion on unstable topology
Key: IGNITE-10794
URL: https://issues.apache.org/jira/browse/IGNITE-10794
Project: Ignite
Issue Type: Bug
Components: mvcc
Reporter: Andrew Mashenkov
Looks like partition can change it's state to MOVING.
Reproducer IgniteCacheEntryProcessorNodeJoinTest.testEntryProcessorNodeLeave
See stacktrace:
{noformat}
java.lang.AssertionError:
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture$Batch.add(GridDhtTxAbstractEnlistFuture.java:1156)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.addToBatch(GridDhtTxAbstractEnlistFuture.java:705)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.processEntry(GridDhtTxAbstractEnlistFuture.java:650)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.continueLoop(GridDhtTxAbstractEnlistFuture.java:533)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.init(GridDhtTxAbstractEnlistFuture.java:362)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxEnlistFuture.enlistLocal(GridNearTxEnlistFuture.java:531)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxEnlistFuture.sendBatch(GridNearTxEnlistFuture.java:426)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxEnlistFuture.sendNextBatches(GridNearTxEnlistFuture.java:173)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxEnlistFuture.map(GridNearTxEnlistFuture.java:149)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxAbstractEnlistFuture.mapOnTopology(GridNearTxAbstractEnlistFuture.java:342)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxAbstractEnlistFuture.init(GridNearTxAbstractEnlistFuture.java:257)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.updateAsync(GridNearTxLocal.java:2074)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.mvccRemoveAllAsync0(GridNearTxLocal.java:1951)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.removeAllAsync0(GridNearTxLocal.java:1670)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.removeAllAsync(GridNearTxLocal.java:550)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Query history statistics API
Vladimir, thanks for your expert opinion. I have some thoughts about 5 point. I tried to find how it works for Oracle and PG: *PG*: keep by default 1000 (can be configured) statements without and discard the least-executed statements. Update statistics is asynchronous process and statistics may have lag. *Oracle*: use shared pool for historical data and can evict records with min time of last execution in case free space at shared pool is not enough for a data which can be related not only historical statistics. So seems also separate asynchronous process (information about it so small). Unfortunately I could not find information about big workload and how it handled for these databases. However We could see that both of vendors use asynchronous statistic processing. I see few variants how we can handle very high workload. First part of variants use asynchronous model with separate thread which should take elements to update stats from a queue: 1) We blocking on overlimited queue and wait when capacity will be enough to put new element. + We have all actual statistics - End of our query execution can be blocked. 2) Discard statistics for ended query in case queue is full. + Very fast for current query - We lose part of statistics. 3) Do full clean of statistic's queue. + Fast and freespace for further elements - We lose big number of statistic elements. Second part of variants use current approach for queryMetrics. When we have some additional capacity for CHM with history + periodical cleanup the Map. In case even the additional space is not enough we can : 1) Discard statistics for ended query 2) Do full clean CHM and discard all gathered information. First part of variants potentially should work faster due to we can update history Map in single thread without contention and put to queue should be faster. What do you think? Which of the variant will be prefer or may be you can suggest another way to handle potential huge workload? Also there is one initial question which stay not clear to me - it is right place for new API. пт, 21 дек. 2018 г. в 13:05, Vladimir Ozerov : > Hi, > > I'd propose the following approach: > 1) Enable history by default. Becuase otherwise users will have to restart > the node to enable it, or we will have to implement dynamic history enable, > which is complex thing. Default value should be relatively small yet > allowing to accommodate typical workloads. E.g. 1000 entries. This should > not put any serious pressure to GC. > 2) Split queries by: schema, query, local flag > 3) Track only growing values: execution count, error count, minimum > duration, maximum duration > 4) Implement ability to clear history - JMX, SQL command, whatever (may be > this is different ticket) > 5) History cleanup might be implemented similarly to current approach: > store everything in CHM. Periodically check it's size. If it is too big - > evict oldest entries. But this should be done with care - under some > workloads new queries will be generated very quickly. In this case we > should either fallback to synchronous evicts, or do not log history at all. > > Thoughts? > > Vladimir. > - > > On Fri, Dec 21, 2018 at 11:22 AM Юрий wrote: > > > Alexey, > > > > Yes, such property to configuration history size will be added. I think > > default value should be 0 and history by default shouldn't be gather at > > all, and can be switched on by property in case when it required. > > > > Currently I planned use the same way to evicting old data as for > > queryMetrics - scheduled task will evict will old data by oldest start > time > > of query. > > > > Will be gathered statistics for only initial clients queries, so internal > > queries will not including. For the same queries we will have one record > in > > history with merged statistics. > > > > All above points just my proposal. Please revert back in case you think > > anything should be implemented by another way. > > > > > > > > > > > > чт, 20 дек. 2018 г. в 18:23, Alexey Kuznetsov : > > > > > Yuriy, > > > > > > I have several questions: > > > > > > Are we going to add some properties to cluster configuration for > history > > > size? > > > > > > And what will be default history size? > > > > > > Will the same queries count as same item of historical data? > > > > > > How we will evict old data that not fit into history? > > > > > > Will we somehow count "reduce" queries? Or only final "map" ones? > > > > > > -- > > > Alexey Kuznetsov > > > > > > > > > -- > > Живи с улыбкой! :D > > > -- Живи с улыбкой! :D
S3 discovery and docker bridge networks
The general problem is a when a node pushes its IP addresses to S3, it has no way to tell whether one of its IP address will be usable by other nodes. On the consumer side, we are unable to determine at runtime which of the addresses will work. So we end up doing discovery using worthless IP addresses, and then need to suffer timeouts to sort this out, slowing down startup. Our fix is to allow configuration of a set of exclusion patterns (regex) on TCPCommunicationSpi, so we could exclude "192.168.*" for example. https://jira.apache.org/jira/browse/IGNITE-10791
[jira] [Created] (IGNITE-10792) [ML] Add seed to test-train filter
Aleksey Zinoviev created IGNITE-10792: - Summary: [ML] Add seed to test-train filter Key: IGNITE-10792 URL: https://issues.apache.org/jira/browse/IGNITE-10792 Project: Ignite Issue Type: Task Components: ml Affects Versions: 2.8 Reporter: Aleksey Zinoviev Assignee: Aleksey Zinoviev Fix For: 2.8 Need to reproduce results from test to test in second Evaluator test -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Volunteer needed: AWS Elastic Load Balancer IP Findersimplemented
Hi, Regarding the name, it's not just about being a good name. I would point to the official AWS documentation [1], [2]. The current ELB does not stand for just the Classic Load balancer. Based on this documentation, the naming is good. I agree with your suggestion about extending TcpDiscoveryApplicationElbIpFinder from the TcpDiscoveryClassicElbIpFinder. I can make the required changes. [1] https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/introduction.html [2] https://docs.aws.amazon.com/elasticloadbalancing/latest/application/introduction.html Regards, Uday On Thu, Dec 20, 2018 at 12:09 AM Stanislav Lukyanov wrote: > Hi, > > Regarding the name - a not-so-good name isn’t always a sufficient > justification for renaming. > Public products such as Ignite have to also take into account convenience > for existing users. > Even in 3.0 when we technically can break compatibility I would avoid > breaking it just to > rename “ELB” to “Classic ELB”. > > Also, I believe the official names Amazon uses for these technologies are > ELB and ALB, > not “classic ELB” and “application ELB”. > > Regarding the code - `extends TcpDiscoveryClassicElbIpFinder` doesn’t > really solve it. > For example, now you have an unused loadBalancerName property in the > TcpDiscoveryApplicationElbIpFinder. > > I guess, to move this forward, let’s just add a new class, > TcpDiscoveryAlbIpFinder. > It doesn’t have to extend the existing TcpDiscoveryElbIpFinder – maybe we > could merge them, but let’s not be > stuck on this any longer. > Also let’s not change the existing TcpDiscoveryElbIpFinder – no need to > rename it or deprecate it. We can thing of the > two IpFinders as of just independent features. > > Thanks, > Stan > > From: uday kale > Sent: 9 октября 2018 г. 19:34 > To: dev@ignite.apache.org > Subject: Re: Volunteer needed: AWS Elastic Load Balancer IP > Findersimplemented > > Hi Stanislav, > > I have removed extended the TcpDiscoveryApplicationElbIpFinder from > TcpDiscoveryClassicElbIpFinder to limi the code duplication. Please share > your feedback. > > Thanks, > Uday > > On Wed, Oct 3, 2018 at 1:12 AM Dmitriy Pavlov > wrote: > > > Hi Uday, > > > > We should keep classes name as is within all minor releases (2.x). We can > > schedule rename only to 3.0. We need to keep both classes and methods > > naming as is to provide compilation guarantee. If someone used existing > > TcpDiscoveryElbIpFinder from 2.6 release than his or her code should > > compile and run well. > > > > I've updated checklist, thank you for pointing to this. > > > > So I suggest keeping existing naming of TcpDiscoveryElbIpFinder as it is > > part of API. > > > > Sincerely, > > Dmitriy Pavlov > > > > вт, 2 окт. 2018 г. в 21:04, uday kale : > > > > > Thanks for the review Stan. > > > I renamed the existing class TcpDiscoveryElbIpFinder since the name is > > not > > > clear regarding the actual load balancer being used. The names > > > TcpDiscoveryClassicElbIpFinder > > > and TcpDiscoveryApplicationElbIpFinder provide a clear distinction. > > > I deprecated the existing class because of review checklist [1] point > 1.a > > > under checklist. It requires methods to be deprecated instead of being > > > renamed. I assumed this will extend to classes too. > > > Regarding the your suggestion for having the tests I agree. It will be > > > helpful to know whether TC can accept properties while initiating a > run. > > > > > > [1] > https://cwiki.apache.org/confluence/display/IGNITE/Review+Checklist > > > > > > Uday > > > > > > On Tue, Oct 2, 2018 at 9:58 PM Stanislav Lukyanov < > > stanlukya...@gmail.com> > > > wrote: > > > > > > > Also, it makes me nervous that we’re lacking any sort of functional > > tests > > > > for AWS-based IP finders (same for GCE, actually). > > > > I understand that it’s hard to include tests like that in regular TC > > runs > > > > because we’d have to include some sort of credentials. > > > > But let’s think of some sort of a middle ground. > > > > > > > > I’m thinking about a functional test suite in the ignite-aws module > > that > > > > would pick up a properties file with AWS credentials. > > > > It wouldn’t be included in any of the TC runs, but someone making > > changes > > > > to ignite-aws would be able to run it locally providing their own > > > > credentials. > > > > They’d then share the results of their testing together with the > usual > > TC > > > > link. > > > > > > > > Stan > > > > > > > > From: Stanislav Lukyanov > > > > Sent: 2 октября 2018 г. 19:08 > > > > To: dev@ignite.apache.org > > > > Subject: RE: Volunteer needed: AWS Elastic Load Balancer IP > > > > Findersimplemented > > > > > > > > Hi, > > > > > > > > I took a look at the code, and I believe the patch needs to be > > enhanced. > > > > > > > > The patch is about adding TcpDiscoveryApplicationElbIpFinder, but it > > also > > > > deprecates the existing TcpDiscoveryElbIpFinder > > > > and replaces it with its copy named
[jira] [Created] (IGNITE-10793) [ML] Create comprehensive example for dataset generators
Alexey Platonov created IGNITE-10793: Summary: [ML] Create comprehensive example for dataset generators Key: IGNITE-10793 URL: https://issues.apache.org/jira/browse/IGNITE-10793 Project: Ignite Issue Type: Sub-task Components: ml Reporter: Alexey Platonov Assignee: Alexey Platonov Fix For: 2.8 We need to create well documented example for generators. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10791) Avoid unusable network during discovery
David Harvey created IGNITE-10791: - Summary: Avoid unusable network during discovery Key: IGNITE-10791 URL: https://issues.apache.org/jira/browse/IGNITE-10791 Project: Ignite Issue Type: Improvement Reporter: David Harvey Problem: In some deployments, there are multiple IP addresses, and S3 discovery tries them in some random order, and times out on the ones that don't work, slowing down discovery unnecessarily. In many such cases, the set of unusable address is known by humans, but is not discoverable at runtime. For example, some IP addresses may be blocked by firewalls. On ECS, the docker bridge network IPs are visible, but are unusable across nodes. http://apache-ignite-users.70518.x6.nabble.com/Avoiding-Docker-Bridge-network-when-using-S3-discovery-td24778.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10790) Control.sh add ability to check tx on deadlock
Sergey Antonov created IGNITE-10790:
---
Summary: Control.sh add ability to check tx on deadlock
Key: IGNITE-10790
URL: https://issues.apache.org/jira/browse/IGNITE-10790
Project: Ignite
Issue Type: Improvement
Reporter: Sergey Antonov
Fix For: 2.8
We should have ability to check selected transaction on deadlock in this moment.
Possible command: {{control.sh --tx detect_deadlock [,xid1,xid2]}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-10789) CacheInterceptor on server node get BinaryObject if put was invoked by ClientCache.
Sergey Antonov created IGNITE-10789:
---
Summary: CacheInterceptor on server node get BinaryObject if put
was invoked by ClientCache.
Key: IGNITE-10789
URL: https://issues.apache.org/jira/browse/IGNITE-10789
Project: Ignite
Issue Type: Bug
Components: cache
Affects Versions: 2.7
Reporter: Sergey Antonov
Assignee: Sergey Antonov
Fix For: 2.8
Cache interceptor on server node expects deserialized object, but gets
BinaryObject in case of put was from thin client.
The exception is following:
{noformat}
[2018-12-21
17:25:08,213][ERROR][client-connector-#53%cache.Test20%][ClientListenerNioListener]
Failed to process client request
[req=o.a.i.i.processors.platform.client.cache.ClientCachePutRequest@72009659]
org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys
(retry update if possible).: [key]
at
org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1313)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:1754)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1107)
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:820)
at
org.apache.ignite.internal.processors.platform.client.cache.ClientCachePutRequest.process(ClientCachePutRequest.java:40)
at
org.apache.ignite.internal.processors.platform.client.ClientRequestHandler.handle(ClientRequestHandler.java:52)
at
org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:174)
at
org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:48)
at
org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279)
at
org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
at
org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:97)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at
org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:70)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: class
org.apache.ignite.internal.processors.cache.CachePartialUpdateCheckedException:
Failed to update keys (retry update if possible).: [key]
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.onPrimaryError(GridNearAtomicAbstractUpdateFuture.java:397)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.onPrimaryResponse(GridNearAtomicSingleUpdateFuture.java:252)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture$1.apply(GridNearAtomicAbstractUpdateFuture.java:304)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture$1.apply(GridNearAtomicAbstractUpdateFuture.java:301)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.map(GridDhtAtomicAbstractUpdateFuture.java:394)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1942)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1671)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:300)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:482)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:442)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:249)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1153)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.put0(GridDhtAtomicCache.java:613)
[jira] [Created] (IGNITE-10788) MVCC: Get operation may hang in some cases
Roman Kondakov created IGNITE-10788:
---
Summary: MVCC: Get operation may hang in some cases
Key: IGNITE-10788
URL: https://issues.apache.org/jira/browse/IGNITE-10788
Project: Ignite
Issue Type: Bug
Components: mvcc
Reporter: Roman Kondakov
Get operation may hang in some cases. Reproducer:
{{IgniteClusterActivateDeactivateTestWithPersistenceAndMemoryReuse.testDeactivateDuringEvictionAndRebalance}}
StackTrace:
{noformat}
Thread
[name="test-runner-#4675%cache.IgniteClusterActivateDeactivateTestWithPersistenceAndMemoryReuse%",
id=5136, state=WAITING, blockCnt=42, waitCnt=382996]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at
o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:178)
at o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141)
at
o.a.i.i.processors.cache.mvcc.MvccUtils.mvccTracker(MvccUtils.java:817)
at
o.a.i.i.processors.cache.distributed.dht.colocated.GridDhtColocatedCache.getAsync(GridDhtColocatedCache.java:244)
at
o.a.i.i.processors.cache.GridCacheAdapter.get0(GridCacheAdapter.java:4725)
at
o.a.i.i.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4706)
at
o.a.i.i.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1450)
at
o.a.i.i.processors.cache.IgniteCacheProxyImpl.get(IgniteCacheProxyImpl.java:927)
at
o.a.i.i.processors.cache.GatewayProtectedCacheProxy.get(GatewayProtectedCacheProxy.java:640)
at
o.a.i.i.processors.cache.IgniteClusterActivateDeactivateTestWithPersistence.testDeactivateDuringEvictionAndRebalance(IgniteClusterActivateDeactivateTestWithPersistence.java:415)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at
o.a.i.testframework.junits.GridAbstractTest$7.run(GridAbstractTest.java:2115)
at java.lang.Thread.run(Thread.java:748)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Code inspection
It seems there is bug in latest 2018.2 TeamCity Bug is filed [1] [1] https://youtrack.jetbrains.com/issue/TW-58504 > On 19 Dec 2018, at 11:31, Petr Ivanov wrote: > > Investigating problem, stand by. > > >> On 18 Dec 2018, at 19:41, Dmitriy Pavlov wrote: >> >> Both patches were applied. Maxim, thank you! >> >> What about 1. An `Unexpected error during build messages processing in >> TeamCity`, what can we do as the next step to fix it? >> >> Sincerely, >> Dmitriy Pavlov >> >> пн, 17 дек. 2018 г. в 18:31, Andrey Mashenkov : >> >>> Maxim, >>> >>> Looks ok. Let's apply IGNITE-10682. >>> >>> All, >>> >>> Also, I'd like to publish idea.logs into artefacts by default. >>> This will give us more details for investigation in future if any failure >>> will occurs. >>> It will costs 1-10 kB. >>> >>> On Mon, Dec 17, 2018 at 3:21 PM Maxim Muzafarov >>> wrote: >>> Dmitry, It seems to me that we have two independent issues here. 1. An `Unexpected error during build messages processing in TeamCity` error message which is related to TC agent configuration. Suppose, server.log will provide us more details about it. I have to access there. 2. A new set of inspection rules was introduced in 2018+ IntelliJ IDEA and they should be disabled in our ignite_inspections_teamcity.xml configuration file. They are not fixed in the Apache Ignite project code yet. I've prepared the issue [1] for it. Please, take a look. Andrey, I've fixed disabled plugins file according to your suggestions. The issue [2] is ready. I've re-run `Excluded [Inspections] Core Debug` suite and the log details show me that now only 3 plugins are enabled: IDEA CORE, Maven Integration, Properties Support. It seems to me that it's correct. [1] https://issues.apache.org/jira/browse/IGNITE-10709 [2] https://issues.apache.org/jira/browse/IGNITE-10682 On Sat, 15 Dec 2018 at 15:22, Dmitriy Pavlov wrote: > > Folks, > > There is a strange error on TC > >>> https://ci.ignite.apache.org/viewLog.html?buildId=2556875=IgniteTests24Java8_InspectionsCore > > It appeared after TC update to the latest version. > > Sincerely, > Dmitry Pavlov > > пт, 14 дек. 2018 г. в 16:09, Andrey Mashenkov < andrey.mashen...@gmail.com>: > >> Maxim, >> >> PR is incomplete. Some plugins should be disabled with different id\name. >> Maven plugin shouldn't be disabled as Idea Inspector use it to use Ignite >> project pom file. >> >> Please, find details in ticket. >> >> >> On Fri, Dec 14, 2018 at 12:00 PM Andrey Mashenkov < >> andrey.mashen...@gmail.com> wrote: >> >>> Maxim, >>> >>> Thanks, I'll check PR and let you know about results. >>> >>> For now, Inspections task execution time looks much better (15-22 min), >>> but fluctuation is still noticeable. >>> >>> On Fri, Dec 14, 2018 at 11:13 AM Maxim Muzafarov < >>> maxmu...@gmail.com > >>> wrote: >>> Andrey, Thanks! I've consulted with the IntelliJ IDEA source code and >>> found how this disabled plugins file should look like. I've created a >>> new issue [1] and prepared PR [2] with the set of disabled plugins (maybe not complete set). I don't have access to change corresponding `~Excluded [Inspections] Core Debug` test suite properties. Can we test this PR? [1] https://issues.apache.org/jira/browse/IGNITE-10682 [2] https://github.com/apache/ignite/pull/5666 On Thu, 13 Dec 2018 at 17:35, Andrey Mashenkov wrote: > > Maxim, > > Idea has a file in config directory >>> ./config/disabled_plugins.txt , >> you can easily find it at you local machine. > Teamcity Inspections runner has an option "Disabled plugins" >>> where disabled_plugins.txt file content can be set. > > So, looks like we can disable useless plugins. > But I'm not expert and can't suggest changes we can safely >>> apply. > > On Thu, Dec 13, 2018 at 4:59 PM Maxim Muzafarov < maxmu...@gmail.com> wrote: >> >> Andrey, >> >> Thank you for solving this issue with GC pauses! I've checked >>> the >> given report. The inspections configuration is correct, but it seems >> to me that we have enabled by default rules of included plugins (for >> instance, KotlinInternalInJava in the report is enabled). >> >> Can you share more details about `disable plugin` option you found? >> >> I see that idea instance starts with the default -Didea.plugins.path >> system property, can we change it so the plugins will
[jira] [Created] (IGNITE-10786) MVCC: Flaky test IgniteClusterActivateDeactivateTestWithPersistence#testDeactivateDuringEvictionAndRebalance
Roman Kondakov created IGNITE-10786:
---
Summary: MVCC: Flaky test
IgniteClusterActivateDeactivateTestWithPersistence#testDeactivateDuringEvictionAndRebalance
Key: IGNITE-10786
URL: https://issues.apache.org/jira/browse/IGNITE-10786
Project: Ignite
Issue Type: Bug
Components: mvcc
Reporter: Roman Kondakov
Test
{{IgniteClusterActivateDeactivateTestWithPersistence#testDeactivateDuringEvictionAndRebalance}}
is flaky when MVCC is enabled. We should investigate it.
{noformat}
[2018-12-21 02:26:51,592][ERROR][main][root] Test failed.
java.lang.AssertionError:
node=cache.IgniteClusterActivateDeactivateTestWithPersistence0, key=12
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertNotNull(Assert.java:621)
at
org.apache.ignite.internal.processors.cache.IgniteClusterActivateDeactivateTestWithPersistence.testDeactivateDuringEvictionAndRebalance(IgniteClusterActivateDeactivateTestWithPersistence.java:425)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at
org.apache.ignite.testframework.junits.GridAbstractTest$7.run(GridAbstractTest.java:2115)
at java.lang.Thread.run(Thread.java:748)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-10787) [TC Bot] Setup TC bot profiling on the server using SJK/stcap and prepare rules for Ignite thread groups
Dmitriy Pavlov created IGNITE-10787: --- Summary: [TC Bot] Setup TC bot profiling on the server using SJK/stcap and prepare rules for Ignite thread groups Key: IGNITE-10787 URL: https://issues.apache.org/jira/browse/IGNITE-10787 Project: Ignite Issue Type: Task Reporter: Dmitriy Pavlov Assignee: Dmitriy Pavlov https://github.com/aragozin/jvm-tools Integrate tool into start scrips, prepare settings for capturing Prepare traces analysis and include into server monitoring page https://mtcga.gridgain.com/monitoring.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10785) MVCC: Grid can hang if transactions is failed to rollback
Roman Kondakov created IGNITE-10785:
---
Summary: MVCC: Grid can hang if transactions is failed to rollback
Key: IGNITE-10785
URL: https://issues.apache.org/jira/browse/IGNITE-10785
Project: Ignite
Issue Type: Bug
Components: mvcc
Reporter: Roman Kondakov
Sometimes grid can hang if transactions is failed to rollback. Reproducer:
{noformat}
[2018-12-14 08:48:13,890][WARN
][sys-stripe-9-#12552%transactions.TxRollbackAsyncWithPersistenceTest2%][GridDhtColocatedCache]
Failed to acquire lock (transaction has been completed):
GridCacheVersion [topVer=156257270, order=1544777270268, nodeOrder=4]
[2018-12-14
08:48:13,893][ERROR][sys-stripe-9-#12552%transactions.TxRollbackAsyncWithPersistenceTest2%][GridDhtColocatedCache]
Failed to rollback the transaction: GridDhtTxLocal
[nearNodeId=b4ff2bcc-dc6a-49c2-a2ab-f26bae6b68c1,
nearFutId=282d09ca761-8ff10d65-a1d6-4f40-9fd7-afb7afa0b25c, nearMiniId=1,
nearFinFutId=382d09ca761-8ff10d65-a1d6-4f40-9fd7-afb7afa0b25c, nearFinMiniId=1,
nearXidVer=GridCacheVersion [topVer=156257270, order=1544777270268,
nodeOrder=4], lb=null, super=GridDhtTxLocalAdapter
[nearOnOriginatingNode=false, nearNodes=KeySetView [], dhtNodes=KeySetView [],
explicitLock=false, super=IgniteTxLocalAdapter [completedBase=null,
sndTransformedVals=false, depEnabled=false, txState=IgniteTxStateImpl
[activeCacheIds=[], recovery=null, mvccEnabled=null, txMap=HashSet []],
mvccWaitTxs=null, qryEnlisted=false, forceSkipCompletedVers=false,
super=IgniteTxAdapter [xidVer=GridCacheVersion [topVer=156257270,
order=1544777270269, nodeOrder=3], writeVer=null, implicit=false, loc=true,
threadId=13949, startTime=1544777293884,
nodeId=cb0ed489-aa1c-4e7a-a196-3bea9e52, startVer=GridCacheVersion
[topVer=156257270, order=1544777270269, nodeOrder=3], endVer=null,
isolation=REPEATABLE_READ, concurrency=PESSIMISTIC, timeout=0,
sysInvalidate=false, sys=false, plc=2, commitVer=GridCacheVersion
[topVer=156257270, order=1544777270269, nodeOrder=3], finalizing=NONE,
invalidParts=null, state=ROLLED_BACK, timedOut=false,
topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0],
mvccSnapshot=MvccSnapshotResponse [futId=82, crdVer=1544777266400, cntr=848,
opCntr=1, txs=[640, 769, 706, 387, 707, 708, 582, 646, 583, 586, 847, 596, 660,
724, 661, 598, 599, 603, 667, 731, 685, 494, 686, 688, 561, 629, 570, 766,
639], cleanupVer=263, tracking=0], parentTx=null, duration=0ms,
onePhaseCommit=false], size=0]]]
class org.apache.ignite.IgniteCheckedException: Failed to finish transaction
[commit=false,
tx=GridDhtTxLocal[xid=df386eba761--0950-4bf6--0003,
xidVersion=GridCacheVersion [topVer=156257270, order=1544777270269,
nodeOrder=3], concurrency=PESSIMISTIC, isolation=REPEATABLE_READ,
state=ROLLED_BACK, invalidate=false, rollbackOnly=true,
nodeId=cb0ed489-aa1c-4e7a-a196-3bea9e52, timeout=0, duration=0]]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.finishTx(GridDhtTxLocal.java:482)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.rollbackDhtLocalAsync(GridDhtTxLocal.java:588)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.rollbackDhtLocal(GridDhtTxLocal.java:563)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.initTxTopologyVersion(GridDhtTransactionalCacheAdapter.java:2178)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearTxEnlistRequest(GridDhtTransactionalCacheAdapter.java:2016)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.access$900(GridDhtTransactionalCacheAdapter.java:112)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$14.apply(GridDhtTransactionalCacheAdapter.java:229)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$14.apply(GridDhtTransactionalCacheAdapter.java:227)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1127)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:592)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:391)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:317)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:108)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:307)
at
[jira] [Created] (IGNITE-10784) SQL: Create a view with list of existing tables
Vladimir Ozerov created IGNITE-10784: Summary: SQL: Create a view with list of existing tables Key: IGNITE-10784 URL: https://issues.apache.org/jira/browse/IGNITE-10784 Project: Ignite Issue Type: Task Components: sql Reporter: Vladimir Ozerov Assignee: Pavel Kuznetsov Fix For: 2.8 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10783) SQL: Ability to optionally disable partition pruning
Vladimir Ozerov created IGNITE-10783: Summary: SQL: Ability to optionally disable partition pruning Key: IGNITE-10783 URL: https://issues.apache.org/jira/browse/IGNITE-10783 Project: Ignite Issue Type: Task Components: sql Reporter: Vladimir Ozerov Fix For: 2.8 Partition pruning is important optimization for SQL queries. But in case partitions are calculated incorrectly, it may lead to query returning less data than expected. Possibility of such scenarios should be mitigated by excessive testing. But it will not give us bug-free guarantee. Let's add special flag which will disable partition pruning for specific user queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10780) control.sh add ability to get tx state change history
Sergey Antonov created IGNITE-10780: --- Summary: control.sh add ability to get tx state change history Key: IGNITE-10780 URL: https://issues.apache.org/jira/browse/IGNITE-10780 Project: Ignite Issue Type: Improvement Reporter: Sergey Antonov Fix For: 2.8 We should have ability to get info about tx state change history. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Service grid redesign
Igniters, Please, let us know if someone is going to do an additional review? We should know can we merge the PR since it has been approved by Nikolay Izhikov and Denis Mekhanikov or we should wait for other community members. On Thu, Dec 20, 2018 at 7:52 PM Vyacheslav Daradur wrote: > > I think I found names which should satisfy me and Denis, and possibly Nikolay > ) > > See the following names (Actual name <- Previously used): > > - ServiceDeploymentManager <- ServicesDeploymentManager > - ServiceDeploymentActions <- ServicesDeploymentActions > - ServiceDeploymentProcessId <- ServicesDeploymentProcessId > - ServiceDeploymentTask <- ServicesDeploymentTask > > - ServiceDeploymentRequest <- ServiceDeploymentChange > - ServiceUndeploymentRequest <- ServiceUndeploymentChange > - ServiceChangeAbstractRequest <- ServiceAbstractChange > > - ServiceSingleNodeDeploymentResult <- ServiceSingleDeploymentsResults > - ServiceSingleNodeDeploymentResultBatch <- ServicesSingleDeploymentsMessage > > - ServiceClusterDeploymentResult <- ServiceFullDeploymentsResults > - ServiceClusterDeploymentResultBatch <- ServicesFullDeploymentsMessage > > - ServiceProcessorCommonDiscoveryData <- ServicesCommonDiscoveryData > - ServiceProcessorJoinNodeDiscoveryData <- ServicesJoinNodeDiscoveryData > > Also, I had a short talk with Alexey Goncharuk about the problem of > nullified custom messages. I changed the implementation to a lock-free > solution which allows us to nullify messages depend on an using > counter. > > In comparison with high priority listener, this allows us to not copy > custom discovery event in service deployment manager and work with the > original object. > > On Thu, Dec 20, 2018 at 8:57 AM Nikolay Izhikov wrote: > > > > Denis, great news! > > > > Alexey, Vova, Yakov, do you want to take a look at this PR? > > > > > > > > В Ср, 19/12/2018 в 18:47 +0300, Denis Mekhanikov пишет: > > > Guys, > > > > > > I finished my code review. The pool request looks good to me. > > > > > > Does anybody else want to look at the changes? > > > There are a few points, that we didn't meet an agreement on, > > > though they don't affect the behaviour in any way: > > > > > >- *Class naming. * See the discussion above. > > >- *Unnecessary task object cleaning. * > > >IMO, ServicesDeploymentTask#clear() method doesn't do anything useful, > > >and it should be removed. > > >By the moment, when this method is called, the task object is removed > > >from all collections anyway, so it's ready for garbage collection. > > >Removing data from it doesn't help anybody. > > >- > > > *Unnecessary tests. *ServiceInfoSelfTest and > > >ServicesDeploymentProcessIdSelfTest look excessive to me. > > >I don't see any point in testing an interface implementation, that only > > >saves some objects and returns them from certain methods. > > >- Interface for events with servicesDeploymentActions() method. > > >Take a look at the discussion: > > > > > > https://github.com/apache/ignite/pull/4434/files/30e69d9a53ce6ea16c4e9d15354e94360caa719d#r239442342 > > > > > > Also solution with *DiscoveryCustomEvent#nullifyingCustomMsgLock* looks > > > clumsy to me. > > > The problem with nullifying of *DiscoveryCustomEvent#customMsg* field can > > > be solved > > > by making *ServiceDiscoveryListener* a high priority listener. > > > > > > Or *DiscoveryCustomEvent#customMessage()* method could be marked > > > synchronized and > > > *GridEventStorageManager#notifyListeners(..)* method could synchronize on > > > the event object. > > > But this solution is the same, it's just a matter of taste. > > > > > > If anybody wants to look the the code of the PR, please consider these > > > points as well. > > > > > > Denis > > > > > > ср, 19 дек. 2018 г. в 17:37, Nikolay Izhikov : > > > > > > > Denis, > > > > > > > > I don't think that differences with your and my naming is huge :) > > > > And, it's definetely a matter of taste. > > > > > > > > If there is no any other issues with PR let's rename and move on! :) > > > > > > > > ср, 19 дек. 2018 г. в 17:32, Vyacheslav Daradur : > > > > > > > > > > We have IgniteServiceProcessor and GridServiceProcessor with > > > > > > singular > > > > > > > > > > "Service" > > > > > > > > > > Maybe we should rename new 'IgniteServiceProcessor' to > > > > > 'IgniteServicesProcessor'? > > > > > > > > > > > And ServiceSingleDeploymentsResults name doesn't make sense to me. > > > > > > "Single deployments" doesn't sound right. > > > > > > > > > > 'Single' means 'single node', maybe we should use one of the > > > > > following: > > > > > - 'ServicesSingleNodeDeploymentsResults' > > > > > - 'ServicesNodeDeploymentsResults' > > > > > - 'ServicesInstanceDeploymentsResults' > > > > > > > > > > On Wed, Dec 19, 2018 at 4:26 PM Denis Mekhanikov > > > > > > > > > > wrote: > > > > > > > > > > > > Slava, > > > > > > I think, it's better to replace word "Change" with "Request". > > > > > > > > > > >
[jira] [Created] (IGNITE-10782) javadoc description for ml.math.exceptions.preprocessing and ml.selection.scoring.evaluator
Stepan Pilschikov created IGNITE-10782: -- Summary: javadoc description for ml.math.exceptions.preprocessing and ml.selection.scoring.evaluator Key: IGNITE-10782 URL: https://issues.apache.org/jira/browse/IGNITE-10782 Project: Ignite Issue Type: Bug Components: documentation, ml Affects Versions: 2.7 Reporter: Stepan Pilschikov Fix For: 2.8 Need to add modules description for - org.apache.ignite.ml.math.exceptions.preprocessing - org.apache.ignite.ml.selection.scoring.evaluator Located in ignite/docs/overview-summary.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10781) Add supportion findNext for H2TreeIndex to use H2 optimization for DISTINCT
Taras Ledkov created IGNITE-10781:
-
Summary: Add supportion findNext for H2TreeIndex to use H2
optimization for DISTINCT
Key: IGNITE-10781
URL: https://issues.apache.org/jira/browse/IGNITE-10781
Project: Ignite
Issue Type: Improvement
Components: sql
Affects Versions: 2.7
Reporter: Taras Ledkov
Now {{H2TreeIndex}} doesn't implement the method {{BaseIndex#findNext}}.
This method is used by H2 engine in the [optimization of the
DISTINCT|https://www.h2database.com/javadoc/org/h2/engine/DbSettings.html#OPTIMIZE_DISTINCT].
To enable this optimization we have to implement the *findNext* method.
Also we have provide selectivity info for the distinct column, because the
optimization is enabled only when column selectivity < 0,2 (and column
selectivity isn't default).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-10779) PagesWriteThrottleSmokeTest.testThrottle is flaky
Nikolai Kulagin created IGNITE-10779:
Summary: PagesWriteThrottleSmokeTest.testThrottle is flaky
Key: IGNITE-10779
URL: https://issues.apache.org/jira/browse/IGNITE-10779
Project: Ignite
Issue Type: Task
Reporter: Nikolai Kulagin
Assignee: Nikolai Kulagin
Fix For: 2.8
Sometimes, at poor checkpoint write speed, put rate degrated to zero for at
least 10 seconds with write throttling enabled. Success rate on TC = 87%. [Test
details|https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=2808794487465215609=testDetails]
{code:java}
junit.framework.AssertionFailedError: Put rate degraded to zero for at least 10
seconds
at junit.framework.Assert.fail(Assert.java:57)
at junit.framework.TestCase.fail(TestCase.java:227)
at
org.apache.ignite.internal.processors.cache.persistence.pagemem.PagesWriteThrottleSmokeTest.testThrottle(PagesWriteThrottleSmokeTest.java:217)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at junit.framework.TestCase.runTest(TestCase.java:176)
at
org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2209)
at
org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:144)
at
org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:2124)
at java.lang.Thread.run(Thread.java:748)
{code}
Test became flaky after IGNITE-10028.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-10778) MVCC: Invoke request may hang sometimes
Roman Kondakov created IGNITE-10778:
---
Summary: MVCC: Invoke request may hang sometimes
Key: IGNITE-10778
URL: https://issues.apache.org/jira/browse/IGNITE-10778
Project: Ignite
Issue Type: Bug
Components: mvcc
Reporter: Roman Kondakov
Invoke request may hang sometimes. Reproducer:
{{GridCacheMultinodeUpdateSelfTest.testInvoke}} with enabled MVCC.
Stacktrace:
{noformat}
Thread [name="invoke-3", id=447, state=WAITING, blockCnt=0, waitCnt=1745]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at
o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:178)
at o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141)
at
o.a.i.i.processors.cache.distributed.near.GridNearTxLocal.rollback(GridNearTxLocal.java:3792)
at
o.a.i.i.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4256)
at
o.a.i.i.processors.cache.GridCacheAdapter.invoke0(GridCacheAdapter.java:2608)
at
o.a.i.i.processors.cache.GridCacheAdapter.invoke(GridCacheAdapter.java:2586)
at
o.a.i.i.processors.cache.IgniteCacheProxyImpl.invoke(IgniteCacheProxyImpl.java:1437)
at
o.a.i.i.processors.cache.GatewayProtectedCacheProxy.invoke(GatewayProtectedCacheProxy.java:1204)
at
o.a.i.i.processors.cache.GridCacheMultinodeUpdateAbstractSelfTest$1.call(GridCacheMultinodeUpdateAbstractSelfTest.java:111)
at
o.a.i.i.processors.cache.GridCacheMultinodeUpdateAbstractSelfTest$1.call(GridCacheMultinodeUpdateAbstractSelfTest.java:100)
at o.a.i.testframework.GridTestThread.run(GridTestThread.java:84)
Thread [name="invoke-2", id=446, state=WAITING, blockCnt=0, waitCnt=1738]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at
o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:178)
at o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141)
at
o.a.i.i.processors.cache.mvcc.MvccUtils.requestSnapshot(MvccUtils.java:841)
at
o.a.i.i.processors.cache.distributed.near.GridNearTxLocal.mvccPutAllAsync0(GridNearTxLocal.java:723)
at
o.a.i.i.processors.cache.distributed.near.GridNearTxLocal.putAsync0(GridNearTxLocal.java:578)
at
o.a.i.i.processors.cache.distributed.near.GridNearTxLocal.invokeAsync(GridNearTxLocal.java:467)
at
o.a.i.i.processors.cache.GridCacheAdapter$25.op(GridCacheAdapter.java:2620)
at
o.a.i.i.processors.cache.GridCacheAdapter$25.op(GridCacheAdapter.java:2608)
at
o.a.i.i.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4244)
at
o.a.i.i.processors.cache.GridCacheAdapter.invoke0(GridCacheAdapter.java:2608)
at
o.a.i.i.processors.cache.GridCacheAdapter.invoke(GridCacheAdapter.java:2586)
at
o.a.i.i.processors.cache.IgniteCacheProxyImpl.invoke(IgniteCacheProxyImpl.java:1437)
at
o.a.i.i.processors.cache.GatewayProtectedCacheProxy.invoke(GatewayProtectedCacheProxy.java:1204)
at
o.a.i.i.processors.cache.GridCacheMultinodeUpdateAbstractSelfTest$1.call(GridCacheMultinodeUpdateAbstractSelfTest.java:111)
at
o.a.i.i.processors.cache.GridCacheMultinodeUpdateAbstractSelfTest$1.call(GridCacheMultinodeUpdateAbstractSelfTest.java:100)
at o.a.i.testframework.GridTestThread.run(GridTestThread.java:84)
Thread [name="invoke-1", id=445, state=WAITING, blockCnt=0, waitCnt=1916]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at
o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:178)
at o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141)
at
o.a.i.i.processors.cache.GridCacheAdapter$25.op(GridCacheAdapter.java:2626)
at
o.a.i.i.processors.cache.GridCacheAdapter$25.op(GridCacheAdapter.java:2608)
at
o.a.i.i.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4244)
at
o.a.i.i.processors.cache.GridCacheAdapter.invoke0(GridCacheAdapter.java:2608)
at
o.a.i.i.processors.cache.GridCacheAdapter.invoke(GridCacheAdapter.java:2586)
at
o.a.i.i.processors.cache.IgniteCacheProxyImpl.invoke(IgniteCacheProxyImpl.java:1437)
at
o.a.i.i.processors.cache.GatewayProtectedCacheProxy.invoke(GatewayProtectedCacheProxy.java:1204)
at
o.a.i.i.processors.cache.GridCacheMultinodeUpdateAbstractSelfTest$1.call(GridCacheMultinodeUpdateAbstractSelfTest.java:111)
at
o.a.i.i.processors.cache.GridCacheMultinodeUpdateAbstractSelfTest$1.call(GridCacheMultinodeUpdateAbstractSelfTest.java:100)
at o.a.i.testframework.GridTestThread.run(GridTestThread.java:84)
{noformat}
--
This message was sent by Atlassian JIRA
Re: Query history statistics API
Hi, I'd propose the following approach: 1) Enable history by default. Becuase otherwise users will have to restart the node to enable it, or we will have to implement dynamic history enable, which is complex thing. Default value should be relatively small yet allowing to accommodate typical workloads. E.g. 1000 entries. This should not put any serious pressure to GC. 2) Split queries by: schema, query, local flag 3) Track only growing values: execution count, error count, minimum duration, maximum duration 4) Implement ability to clear history - JMX, SQL command, whatever (may be this is different ticket) 5) History cleanup might be implemented similarly to current approach: store everything in CHM. Periodically check it's size. If it is too big - evict oldest entries. But this should be done with care - under some workloads new queries will be generated very quickly. In this case we should either fallback to synchronous evicts, or do not log history at all. Thoughts? Vladimir. - On Fri, Dec 21, 2018 at 11:22 AM Юрий wrote: > Alexey, > > Yes, such property to configuration history size will be added. I think > default value should be 0 and history by default shouldn't be gather at > all, and can be switched on by property in case when it required. > > Currently I planned use the same way to evicting old data as for > queryMetrics - scheduled task will evict will old data by oldest start time > of query. > > Will be gathered statistics for only initial clients queries, so internal > queries will not including. For the same queries we will have one record in > history with merged statistics. > > All above points just my proposal. Please revert back in case you think > anything should be implemented by another way. > > > > > > чт, 20 дек. 2018 г. в 18:23, Alexey Kuznetsov : > > > Yuriy, > > > > I have several questions: > > > > Are we going to add some properties to cluster configuration for history > > size? > > > > And what will be default history size? > > > > Will the same queries count as same item of historical data? > > > > How we will evict old data that not fit into history? > > > > Will we somehow count "reduce" queries? Or only final "map" ones? > > > > -- > > Alexey Kuznetsov > > > > > -- > Живи с улыбкой! :D >
[jira] [Created] (IGNITE-10776) migrate from JUnit 3 to 4 config variations test suites
Oleg Ignatenko created IGNITE-10776:
---
Summary: migrate from JUnit 3 to 4 config variations test suites
Key: IGNITE-10776
URL: https://issues.apache.org/jira/browse/IGNITE-10776
Project: Ignite
Issue Type: Sub-task
Affects Versions: 2.8
Reporter: Oleg Ignatenko
This task is to migrate config variations test suites from JUnit 3 to 4.
If needed, refer parent task comments for more details.
Note this might be somewhat tough nut to crack design wise since
{{JUnit4TestAdapter}} plays a special role in config variations tests, see
IGNITE-10739.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Query history statistics API
Alexey, Yes, such property to configuration history size will be added. I think default value should be 0 and history by default shouldn't be gather at all, and can be switched on by property in case when it required. Currently I planned use the same way to evicting old data as for queryMetrics - scheduled task will evict will old data by oldest start time of query. Will be gathered statistics for only initial clients queries, so internal queries will not including. For the same queries we will have one record in history with merged statistics. All above points just my proposal. Please revert back in case you think anything should be implemented by another way. чт, 20 дек. 2018 г. в 18:23, Alexey Kuznetsov : > Yuriy, > > I have several questions: > > Are we going to add some properties to cluster configuration for history > size? > > And what will be default history size? > > Will the same queries count as same item of historical data? > > How we will evict old data that not fit into history? > > Will we somehow count "reduce" queries? Or only final "map" ones? > > -- > Alexey Kuznetsov > -- Живи с улыбкой! :D
[jira] [Created] (IGNITE-10775) migrate from JUnit 3 to 4 test suites of simple dynamic lists that use addTestIfNeeded API
Oleg Ignatenko created IGNITE-10775:
---
Summary: migrate from JUnit 3 to 4 test suites of simple dynamic
lists that use addTestIfNeeded API
Key: IGNITE-10775
URL: https://issues.apache.org/jira/browse/IGNITE-10775
Project: Ignite
Issue Type: Sub-task
Affects Versions: 2.8
Reporter: Oleg Ignatenko
This task is to migrate from JUnit 3 to 4 test suites of simple dynamic lists
that use {{GridTestUtils.addTestIfNeeded}} API.
If needed, refer parent task comments for more details.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-10777) cleanup remainders of JUnit4TestAdapter and other JUnit 3 API involving test suites
Oleg Ignatenko created IGNITE-10777: --- Summary: cleanup remainders of JUnit4TestAdapter and other JUnit 3 API involving test suites Key: IGNITE-10777 URL: https://issues.apache.org/jira/browse/IGNITE-10777 Project: Ignite Issue Type: Sub-task Affects Versions: 2.8 Reporter: Oleg Ignatenko This task is to cleanup remainders of JUnit4TestAdapter and other JUnit 3 API involving test suites that may be missed in prior sub-tasks under the parent task. If needed, refer to parent task comments for more details. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10774) migrate test suites that are fixed lists of test classes from Junit 3 to 4
Oleg Ignatenko created IGNITE-10774:
---
Summary: migrate test suites that are fixed lists of test classes
from Junit 3 to 4
Key: IGNITE-10774
URL: https://issues.apache.org/jira/browse/IGNITE-10774
Project: Ignite
Issue Type: Sub-task
Affects Versions: 2.8
Reporter: Oleg Ignatenko
This task is to migrate test suites that are fixed lists of test classes from
Junit 3 to 4 that can be handled by {{@SuiteClasses}} API.
If needed, find more details in the comments of the parent task.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-10773) migrate examples testsuites from Junit 3 to 4
Oleg Ignatenko created IGNITE-10773: --- Summary: migrate examples testsuites from Junit 3 to 4 Key: IGNITE-10773 URL: https://issues.apache.org/jira/browse/IGNITE-10773 Project: Ignite Issue Type: Sub-task Affects Versions: 2.8 Reporter: Oleg Ignatenko This taks is to migrate examples testsuites from Junit 3 to 4. If needed, find more details in comments of parent task -- This message was sent by Atlassian JIRA (v7.6.3#76005)
