[MTCGA]: new failures in builds [3218097] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. If your changes can lead to this failure(s): We're grateful that you were a volunteer to make the contribution to this project, but things change and you may no longer be able to finalize your contribution. Could you respond to this email and indicate if you wish to continue and fix test failures or step down and some committer may revert you commit. *Recently contributed test failed in master near.IgniteCacheDistributedPartitionQueryNodeRestartsSelfTest https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-4005992156292294495=%3Cdefault%3E=testDetails No changes in the build - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 08:04:17 15-03-2019
Re: Batch updates in Ignite B+ tree.
Folks, I have prepared draft IEP [1] for batch updates in PageMemory [2]. This IEP contains a description of the proposed changes and a plan for their implementation. It does not contain a description for batch insert in B+ Tree, if you are working on it and have any updates it will be good to add them to an appropriate topic [3]. If you are not working on this task I'll do it by myself later. WDYT? Also, I have a draft implementation of batch insertions to the FreeList (without B+ Tree batch updates) and testing results look nice [4]. [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-32+Batch+updates+in+PageMemory [2] https://issues.apache.org/jira/browse/IGNITE-7935 [3] https://cwiki.apache.org/confluence/display/IGNITE/IEP-32+Batch+updates+in+PageMemory#IEP-32BatchupdatesinPageMemory-BatchupdateinB+Tree [4] https://cwiki.apache.org/confluence/display/IGNITE/IEP-32+Batch+updates+in+PageMemory#IEP-32BatchupdatesinPageMemory-Prototypetestingresults ср, 6 мар. 2019 г. в 14:35, Alexey Goncharuk : > Hello Pavel, Vladimir, > > As far as I know, Semyon Boikov and Sergi Vladykin (CCed) are prototyping > this feature. > > Folks, can you comment? > > > ср, 6 мар. 2019 г. в 10:57, Vladimir Ozerov : > > > Hi Pavel, > > > > As far as I know batch tree updates already being developed. Alex, could > > you please elaborate? > > > > On Tue, Mar 5, 2019 at 5:05 PM Pavel Pereslegin > wrote: > > > >> Hi Igniters! > >> > >> I am working on implementing batch updates in PageMemory [1] to > >> improve the performance of preloader, datastreamer and putAll. > >> > >> This task consists of two major related improvements: > >> 1. Batch writing to PageMemory via FreeList - store several values at > >> once to single memory page. > >> 2. Batch updates in BPlusTree (for introducing invokeAll operation). > >> > >> I started to investigate the issue with batch updates in B+ tree, and > >> it seems that the concurrent top-down balancing algorithm (TD) > >> described in this paper [2] may be suitable for batch insertion of > >> keys into Ignite B+ Tree. > >> This algorithm uses a top-down balancing approach and allows to insert > >> a batch of keys belonging to the leaves having the same parent. The > >> negative point of top-down balancing approach is that the parent node > >> is locked when performing insertion/splitting in child nodes. > >> > >> WDYT? Do you know other approaches for implementing batch updates in > >> Ignite B+ Tree? > >> > >> [1] https://issues.apache.org/jira/browse/IGNITE-7935 > >> [2] > >> > https://aaltodoc.aalto.fi/bitstream/handle/123456789/2168/isbn9512258951.pdf > >> > > >
[jira] [Created] (IGNITE-11548) MVCC: MVCC PDS 2 suite became unstable after the get operation mapping fix
Roman Kondakov created IGNITE-11548:
---
Summary: MVCC: MVCC PDS 2 suite became unstable after the get
operation mapping fix
Key: IGNITE-11548
URL: https://issues.apache.org/jira/browse/IGNITE-11548
Project: Ignite
Issue Type: Bug
Components: mvcc
Reporter: Roman Kondakov
Fix For: 2.8
It looks like {{MVCC PDS 2}} suite became unstable after IGNITE-10261 is merged
to master. It should be investigated.
TC run:
[https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_MvccPds2_IgniteTests24Java8=%3Cdefault%3E=buildTypeStatusDiv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-11547) ClassCastException when creating LOCAL cache on client in Data Region which is considered persistent
Ilya Kasnacheev created IGNITE-11547: Summary: ClassCastException when creating LOCAL cache on client in Data Region which is considered persistent Key: IGNITE-11547 URL: https://issues.apache.org/jira/browse/IGNITE-11547 Project: Ignite Issue Type: Bug Components: cache, persistence Affects Versions: 2.7 Reporter: Ilya Kasnacheev If default region in cluster is persistent, then creating a LOCAL cache on client will fail with: org.apache.ignite.internal.pagemem.impl.PageMemoryNoStoreImpl cannot be cast to org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryEx This is unless there is a second non-persistent data region, and cache is created in this non-persistent region. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Consistent ID specification from previous random UUID
Hi, I've prepared demo PR for approach 1, just checking UUID and trying to use node00-UUID. https://github.com/apache/ignite/pull/6266/ Actually for this approach it is really simple, no API change is required. I guess the case is not very often because automatic folders assignment works well in most cases. WDYT? ср, 13 мар. 2019 г. в 09:42, Павлухин Иван : > Alex, > > Why do we need > > - check if consistent ID is set to a string 'nodeXX-UUID'. In this case > > the consistent ID is set to UUID, and the storage folder is chosen > > according to the proper rules. This change has a minimal chance to affect > > current users because it's unlikely that somebody is using auto-generated > > folder naming scheme as consistent ID. > ? > It looks hacky as well. The thing I do not like here is that a > consistentId specified in configuration is not a consistentId used by > a node sometimes. > > Can we go just with > > - either check if consistent ID is an instance of UUID and then take the > > appropriate folder. This approach is straightforward, but may affect > > current users > ? > > And as a last chance a user will have a possibility to rename a directory. > > > вт, 12 мар. 2019 г. в 19:33, Alexey Goncharuk >: > > > > Igniters, > > > > I came across the same issue during development and found no sane > > workaround for this issue. I believe the solution should be as simple as > > possible because we are already adding a warning to let users know that > it > > is good to specify a consistent ID in production deployments. > > > > As for the current solution, I do not like adding a new configuration > > property like 'storageFolder' because it's another way to shoot yourself > in > > the leg (e.g. different nodes have different consistent IDs but > configured > > to have the same storage folder). > > Why can't we: > > - either check if consistent ID is an instance of UUID and then take the > > appropriate folder. This approach is straightforward, but may affect > > current users > > - check if consistent ID is set to a string 'nodeXX-UUID'. In this case > > the consistent ID is set to UUID, and the storage folder is chosen > > according to the proper rules. This change has a minimal chance to affect > > current users because it's unlikely that somebody is using auto-generated > > folder naming scheme as consistent ID. > > > > Thoughts? > > > > вт, 12 мар. 2019 г. в 16:51, Dmitriy Pavlov : > > > > > Hi Igniters, > > > > > > A full description can be found at wiki page > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-SubfoldersGeneration > > > > > > I like the idea of introducing a property like nodeStorageName to > specify > > > exactly name of a directory to use. For example, it could have higher > > > priority then IgniteConfiguration.getConsistentId(). It will impact the > > > mentioned algorithm, but it seems to be easier to understand by users. > > > > > > If nobody else minds, I will try to implement this idea. > > > > > > Sincerely, > > > Dmitriy Pavlov > > > > > > > > > сб, 2 мар. 2019 г. в 08:33, Павлухин Иван : > > > > > > > Dmitriy, > > > > > > > > You wrote > > > > > we need an order to scan and lock random-UUID based folders. > > > > > > > > It would be great if you provide a discussion about that order to > > > > complete the picture. Currently I cannot understand why the order is > > > > important. > > > > > > > > Also, couple of raw thoughts: > > > > 1. Can we extend a directory lookup procedure when consistent id is > > > > specified to check nodeXX-consistentId directories as well? > > > > 2. Can we introduce a property like nodeStorageName to specify exact > > > > name of a directory to use? It looks like a straightforward and > > > > universal workaround. Or is node index better in some sense? Why? > > > > > > > > Please, share your thoughts. > > > > > > > > чт, 28 февр. 2019 г. в 17:43, Dmitriy Pavlov : > > > > > > > > > > Hi Ivan, > > > > > > > > > > Yes, you catch me, I'm a little bit cheating with lazy consensus on > > > code > > > > > modification without providing a PR because I was expecting that > nobody > > > > > comes to discussion. I will prepare PR shortly. And since we anyway > > > have > > > > a > > > > > discussion, I will not apply anything by lazy approval. > > > > > > > > > > - storageNodeIndex without consistent ID will not work. > > > > > cfg.getDataStorageConfiguration().setNodeIdx() will be required > only > > > for > > > > > case we have consistent ID. > > > > > > > > > > Hi Stanislav, > > > > > > > > > > We can't use only consistent ID because > > > > > > > > > > 1) we need an order to scan and lock random-UUID based folders. > Node > > > > index > > > > > provides the order of scan. I can find the corresponding > discussion, > > > but > > > > I > > > > > guess it is not needed. > > > > > 2) we need to separate backward compatible folders from new > random-UUID > > > > >
[jira] [Created] (IGNITE-11546) FileDownloader may be closed earlier than the file is downloaded
Alexey Goncharuk created IGNITE-11546: - Summary: FileDownloader may be closed earlier than the file is downloaded Key: IGNITE-11546 URL: https://issues.apache.org/jira/browse/IGNITE-11546 Project: Ignite Issue Type: Improvement Reporter: Alexey Goncharuk -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11545) Logging baseline auto-adjust
Anton Kalashnikov created IGNITE-11545: -- Summary: Logging baseline auto-adjust Key: IGNITE-11545 URL: https://issues.apache.org/jira/browse/IGNITE-11545 Project: Ignite Issue Type: Bug Reporter: Anton Kalashnikov Assignee: Anton Kalashnikov It needs to add some extra log to baseline auto-adjust process. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11544) Unclear behavior for cache operations using classes different from specified as indexed types
Pavel Vinokurov created IGNITE-11544: Summary: Unclear behavior for cache operations using classes different from specified as indexed types Key: IGNITE-11544 URL: https://issues.apache.org/jira/browse/IGNITE-11544 Project: Ignite Issue Type: Bug Components: cache, sql Affects Versions: 2.7 Reporter: Pavel Vinokurov Attachments: IndexedTypesReproducer.java There are a few cases presented in the attached reproducer where caches are populated by objects of classes different from specified in CacheConfiguration#setIndexedTypes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11543) TransactionOptimisticException on topology change when readThrough is enabled
Denis Mekhanikov created IGNITE-11543:
-
Summary: TransactionOptimisticException on topology change when
readThrough is enabled
Key: IGNITE-11543
URL: https://issues.apache.org/jira/browse/IGNITE-11543
Project: Ignite
Issue Type: Bug
Components: cache
Affects Versions: 2.7
Reporter: Denis Mekhanikov
Attachments:
CacheOptimisticTransactionTopologyChangeReadThroughTest.java
When topology changes during an optimistic serializable transaction on a
replicated cache with {{readThrough}} enabled,
{{TransactionOptimisticException}} may appear.
Cache configuration:
{noformat}
cacheMode: REPLICATED
atomicityMode: TRANSACTIONAL
readThrough: true
{noformat}
Reproducer is attached.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-11542) Fix flacky test testGivenCacheWithPojoValueAndPds_WhenPut_ThenNoHangup.
Andrew Mashenkov created IGNITE-11542: - Summary: Fix flacky test testGivenCacheWithPojoValueAndPds_WhenPut_ThenNoHangup. Key: IGNITE-11542 URL: https://issues.apache.org/jira/browse/IGNITE-11542 Project: Ignite Issue Type: Test Reporter: Andrew Mashenkov IgnitePdsBinarySortObjectFieldsTest.testGivenCacheWithPojoValueAndPds_WhenPut_ThenNoHangup fails sporadically on TC due to 5 sec timeout may be not enough for grid startup. Test checks "put" operation will complete in 5 sec timeout, but grid initialization is included in this timeout with no reason. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: how to get zipkin tracing of REST calls from one microservice to all microservices calls from any task defined in apache ignite
Hi!
As far as I know, Ignite does not support Zipkin trace propagation out of
the box, unlike Spring Cloud. Hence request handler in {{rest-http}} loses
tracing-context-specific headers (see [1]) sent with request and the span
gets broken into two parts. This can be worked around by
replacing {{rest-http}} with Spring Cloud request handler, that in turn
should execute Ignite Compute Task explicitly.
[1]
https://cloud.spring.io/spring-cloud-sleuth/single/spring-cloud-sleuth.html#_propagation
чт, 14 мар. 2019 г. в 09:16, Павлухин Иван :
> Hi,
>
> As far as I remember zipkin defines tracing units in a current thread
> of execution. I cannot say for sure what goes wrong in your case. But
> it might be that traced execution on ser3 side switches from one
> thread to another and you see 2 units as a result.
>
> чт, 14 мар. 2019 г. в 02:09, Aditya Kumar :
> >
> >
> > Hi Team,
> >
> > I was using ignite as dependecy in our application and was able to trace
> end to end trace microservice calls.
> > Then, to let ignite handle our services in compute task, we removed all
> spring-boot dependencies and created task for each service we had in our
> microservice.
> >
> > The issue we are facing is explained using below POC done at our end.
> >
> >
> > We created a sample app where below things have been done and deployed
> in ignite:
> > 1. created beans and interceptors needed to start and track zipkin and
> brave trace
> > 2. craeted a task using org.apache.ignite.compute.ComputeTaskAdapter
> > 3. registered the task in the config file used while starting ignite
> server
> >
> > Below are the list of services(each service is exposing a REST endpoint)
> created to test this scenario:
> > 1. ms1 (spring boot app)
> > 2. ser2 (spring mvc app having a rest endpoint to serve the incoming
> request)
> > 3. ser3 (spring mvc app using Ignite ComputeTaskAdapter to serve the
> incoming request from '/ignite'. used ignite's ignite-rest-http.jar to
> enable '/ignite' endpoint)
> > 4. ms4 (spring boot app)
> > 5. ms5 (spring boot app)
> >
> > Then, there were two scenarios of executions:
> > Case1. ms1 -> ser2 -> ms4 -> ms5 ==> we get single unit of traing in
> zipkin from ms1 to ms5 (i.e. ms1 -> ser2 -> ms4 -> ms5)
> > Case2. ms1 -> ser3 -> ms4 -> ms5 ==> we get two unit of tracing in
> zipkin. One is ms1 -> ser3 and another is ser3 -> ms4 -> ms5
> >
> > I need to get single unit of tracing in zipkin using Case2
> execution(i.e. as we get in Case1)
> >
> > The sample app (ser3) is checked-in at
> https://github.com/aditya2910/adzzz1/tree/master/ignite-rest-task
> >
> > Any help will be appreciated.
> > Please let me know incase you need any more info.
> >
> > Thanks,
> > Aditya
> > This email and any files transmitted with it are confidential,
> proprietary and intended solely for the individual or entity to whom they
> are addressed. If you have received this email in error please delete it
> immediately.
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>
--
Best regards,
Andrey Kuznetsov.
[jira] [Created] (IGNITE-11541) Dynamic columns and indexes can be lost after the cluster restart
Roman Guseinov created IGNITE-11541: --- Summary: Dynamic columns and indexes can be lost after the cluster restart Key: IGNITE-11541 URL: https://issues.apache.org/jira/browse/IGNITE-11541 Project: Ignite Issue Type: Bug Components: cache, persistence, sql Affects Versions: 2.7 Reporter: Roman Guseinov Attachments: StaticCacheAndDdlReproducer.java The case is the following: # Run a server node (persistence enabled. static cache configuration with queryEnities). # Add/drop column/index via DDL. # Restart the node with the same configuration. # The changes are lost... It seems the issue is related to merging cache metadata. There are some workarounds: # Remove/comment cache configuration. After the restart, the metadata will be loaded from the persistence store. This will work for a rolling restart. # Make the same changes in the configuration (update queryEntity). It requires the whole cluster restart. Reproducer is attached. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11540) Node.js thin client: best effort affinity
Alexey Kosenchuk created IGNITE-11540: - Summary: Node.js thin client: best effort affinity Key: IGNITE-11540 URL: https://issues.apache.org/jira/browse/IGNITE-11540 Project: Ignite Issue Type: New Feature Components: thin client Affects Versions: 2.7 Reporter: Alexey Kosenchuk Assignee: Alexey Kosenchuk Implement [IEP-23|https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients] for Node.js thin client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: how to get zipkin tracing of REST calls from one microservice to all microservices calls from any task defined in apache ignite
Hi, As far as I remember zipkin defines tracing units in a current thread of execution. I cannot say for sure what goes wrong in your case. But it might be that traced execution on ser3 side switches from one thread to another and you see 2 units as a result. чт, 14 мар. 2019 г. в 02:09, Aditya Kumar : > > > Hi Team, > > I was using ignite as dependecy in our application and was able to trace end > to end trace microservice calls. > Then, to let ignite handle our services in compute task, we removed all > spring-boot dependencies and created task for each service we had in our > microservice. > > The issue we are facing is explained using below POC done at our end. > > > We created a sample app where below things have been done and deployed in > ignite: > 1. created beans and interceptors needed to start and track zipkin and brave > trace > 2. craeted a task using org.apache.ignite.compute.ComputeTaskAdapter > 3. registered the task in the config file used while starting ignite server > > Below are the list of services(each service is exposing a REST endpoint) > created to test this scenario: > 1. ms1 (spring boot app) > 2. ser2 (spring mvc app having a rest endpoint to serve the incoming request) > 3. ser3 (spring mvc app using Ignite ComputeTaskAdapter to serve the incoming > request from '/ignite'. used ignite's ignite-rest-http.jar to enable > '/ignite' endpoint) > 4. ms4 (spring boot app) > 5. ms5 (spring boot app) > > Then, there were two scenarios of executions: > Case1. ms1 -> ser2 -> ms4 -> ms5 ==> we get single unit of traing in zipkin > from ms1 to ms5 (i.e. ms1 -> ser2 -> ms4 -> ms5) > Case2. ms1 -> ser3 -> ms4 -> ms5 ==> we get two unit of tracing in zipkin. > One is ms1 -> ser3 and another is ser3 -> ms4 -> ms5 > > I need to get single unit of tracing in zipkin using Case2 execution(i.e. as > we get in Case1) > > The sample app (ser3) is checked-in at > https://github.com/aditya2910/adzzz1/tree/master/ignite-rest-task > > Any help will be appreciated. > Please let me know incase you need any more info. > > Thanks, > Aditya > This email and any files transmitted with it are confidential, proprietary > and intended solely for the individual or entity to whom they are addressed. > If you have received this email in error please delete it immediately. -- Best regards, Ivan Pavlukhin
