Re: Non-blocking PME Phase One (Node fail)
Anton, Thanks for effort on reducing PME blocking time. I'm looking at. чт, 5 дек. 2019 г. в 16:14, Anton Vinogradov : > Igniters, > > Solution ready to be reviewed > TeamCity is green: > > https://issues.apache.org/jira/browse/IGNITE-9913?focusedCommentId=16988566=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16988566 > Issue: https://issues.apache.org/jira/browse/IGNITE-9913 > Relevant PR: https://github.com/apache/ignite/pull/7069 > > In brief: Solution allows avoiding PME and running transactions waiting on > node left. > > What done/changed: > Extended Late Affinity Assignment semantic: from "guarantee to switch when > ideal primaries rebalanced" to "guarantee to switch when all expected > owners rebalanced". > This extension provides the ability to perform PME-free switch on baseline > node left when cluster fully rebalanced, avoiding a lot of corner-cases > possible on a partially rebalanced cluster. > > The PME-free switch does not block or cancel or wait for current > operations. > It's just recover failed primaries and once cluster-wide recovery finished > it finishes exchange future (allows new operations). > So in other words, now new-topology operations allowed immediately after > cluster-wide recovery finished (which is fast), not after previous-topology > operations finished + recovery + PME, as it was before. > > Limitations: > Optimization works only when baseline set (alive primaries require no > relocation, recovery required for failed primaries). > > BTW, PME-free code seems to be small and simple, most of the changes > related to the "rebalanced cluster guarantee". > > It will be nice if someone will check the code and tests. > Test coverage proposals are welcome! > > P.s. Here's a benchmark used to check we have performance improvements > listed above > > https://github.com/anton-vinogradov/ignite/blob/426f0de9185ac2938dadb7bd2cbfe488233fe7d6/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/distributed/Bench.java > > P.p.s. We also checked the same case on the real environment and found no > latency degradation. > > On Thu, Sep 19, 2019 at 2:18 PM Anton Vinogradov wrote: > > > Ivan, > > > > The useful profile can be produced only by the reproducible benchmark. > > We're working on such benchmark to analyze duration and measure the > > improvement. > > > > Currently, only the typical overall duration from prod servers is known. > > > > On Thu, Sep 19, 2019 at 12:09 PM Павлухин Иван > > wrote: > > > >> Anton, folks, > >> > >> Out of curiosity. Do we have some measurements of PME execution in an > >> environment similar to some real-world one? In my mind some kind of > >> "profile" showing durations of different PME phases with an indication > >> where we are "blocked" would be ideal. > >> > >> ср, 18 сент. 2019 г. в 13:27, Anton Vinogradov : > >> > > >> > Alexey, > >> > > >> > >> Can you describe the complete protocol changes first > >> > The current goal is to find a better way. > >> > I had at least 5 scenarios discarded because of finding corner cases > >> > (Thanks to Alexey Scherbakov, Aleksei Plekhanov and Nikita Amelchev). > >> > That's why I explained what we able to improve and why I think it > works. > >> > > >> > >> we need to remove this property, not add new logic that relies on > it. > >> > Agree. > >> > > >> > >> How are you going to synchronize this? > >> > Thanks for providing this case, seems it discards #1 + #2.2 case and > >> #2.1 > >> > still possible with some optimizations. > >> > > >> > "Zombie eating transactions" case can be theoretically solved, I > think. > >> > As I said before we may perform "Local switch" in case affinity was > not > >> > changed (except failed mode miss) other cases require regular PME. > >> > In this case, new-primary is an ex-backup and we expect that > old-primary > >> > will try to update new-primary as a backup. > >> > New primary will handle operations as a backup until it notified it's > a > >> > primary now. > >> > Operations from ex-primary will be discarded or remapped once > >> new-primary > >> > notified it became the primary. > >> > > >> > Discarding is a big improvement, > >> > remapping is a huge improvement, > >> > there is no 100% warranty that ex-primary will try to update > >> new-primary as > >> > a backup. > >> > > >> > A lot of corner cases here. > >> > So, seems minimized sync is a better solution. > >> > > >> > Finally, according to your and Alexey Scherbakov's comments, the > better > >> > case is just to improve PME to wait for less, at least now. > >> > Seems, we have to wait for (or cancel, I vote for this case - any > >> > objections?) operations related to the failed primaries and wait for > >> > recovery finish (which is fast). > >> > In case affinity was changed and backup-primary switch (not related to > >> the > >> > failed primaries) required between the owners or even rebalance > >> required, > >> > we should just ignore this and perform only
Re: [Webinar] How to Migrate Your Data Schema to Apache Ignite
Recording for those who couldn't join the webinar: https://www.youtube.com/watch?v=fwMRFA7BWTk Ivan, excellent presentation! - Denis On Tue, Dec 3, 2019 at 4:41 PM Kseniya Romanova wrote: > Hi Humphrey! Checked in chrome and firefox - the link is alive and > registration works. Please try again > https://www.gridgain.com/resources/webinars/how-migrate-your-data-schema-apache-ignite > If you still can not access the page please email me and I'll register you > from our side. > > Regards, > Kseniya Romanova > GridGain Devrel > > > вт, 3 дек. 2019 г., 1:54 Humphrey : > >> Having trouble registering. Link to apply for the webinar not >> active/working. >> >> >> >> -- >> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >> >
Re: [Webinar] Data Streaming Using Apache Flink and Apache Ignite
Thank you Alexey, yes I think all the webinar videos are recorded and shared in youtube. https://www.youtube.com/results?search_query=gridgain+webinar I can share the webinar video once it is available. On Sun, Dec 8, 2019 at 10:28 PM Alexey Zinoviev wrote: > Great to get a video after to watch and share later, very ineresting case > > вс, 8 дек. 2019 г., 20:39 Saikat Maitra : > >> Hi, >> >> I will be presenting in a webinar about Data Streaming Using Apache Flink >> and Apache Ignite on Wednesday, Dec 18th. >> >> >> https://www.gridgain.com/resources/webinars/data-streaming-using-apache-flink-and-apache-ignite >> >> Apache Ignite is a powerful in-memory computing platform. The Apache >> IgniteSink streaming connector enables users to inject Flink data into the >> Ignite cache. Join Saikat Maitra to learn how to build a simple data >> streaming application using Apache Flink and Apache Ignite. This stream >> processing topology will allow data streaming in a distributed, scalable, >> and fault-tolerant manner, which can process data sets consisting of >> virtually unlimited streams of events. >> >> Please feel free to join the webinar. >> >> Regards, >> Saikat >> >
Re: [DISCUSS] dependencies and release process for Ignite Extensions
Hi Ivan, Ilya Thank you for your reply. I have added you as dev in IgniteExtensions project and you should be able to check the build configurations. Yes, when I set artifact dependencies for ~Build Apache Ignite~ similar to other projects then I receive below error [ERROR] [ERROR] Could not find the selected project in the reactor: :ignite-flink-ext @ I do not see the same error when I am running the build in local and maven package command can still get 2.9.0-SNAPSHOT dependencies from my local .m2 repository but it is failing in teamcity. Regards, Saikat On Mon, Dec 9, 2019 at 5:18 AM Ilya Kasnacheev wrote: > Hello! > > I think that your build should depend on an Apache Ignite build(s) or just > use an already released version. > > In the same fashion as all our teamcity depend on "Build Apache Ignite" and > use its artifacts. > > Here you want Ignite snapshot but Teamcity does not know where to take it > from. > > Regards, > -- > Ilya Kasnacheev > > > пн, 9 дек. 2019 г. в 06:40, Saikat Maitra : > > > Hello, > > > > I am running into a problem specific to teamcity build for Ignite > > Extensions project. When I set the dependencies to 2.9.0-SNAPSHOT I am > > getting an error message during build as below > > > > [06:24:12][Step 4/5] Failed to execute goal on project ignite-flink-ext: > > Could not resolve dependencies for project > > org.apache.ignite.ext:ignite-flink-ext:jar:1.0.0-SNAPSHOT: The following > > artifacts could not be resolved: > > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT, > > org.apache.ignite:ignite-core:jar:tests:2.9.0-SNAPSHOT, > > org.apache.ignite:ignite-log4j:jar:2.9.0-SNAPSHOT, > > org.apache.ignite:ignite-spring:jar:2.9.0-SNAPSHOT: Could not find > artifact > > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT in h2database.com ( > > https://h2database.com/m2-repo) > > > > > > and if set artifact dependencies for ~Build Apache Ignite~ then I receive > > below error > > > > [ERROR] [ERROR] Could not find the selected project in the reactor: > > :ignite-flink-ext @ > > > > Build url > > > > > https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteExtensions_Build_IgniteExtensions=pull%2F1%2Fhead=buildTypeStatusDiv > > > > Can you please let me know if you faced similar problem with teamcity > > build? > > > > I can set Ignite Extensions dependencies to released ignite-core > artifacts > > version like 2.7.6 and build works fine. > > > > Regards, > > Saikat > > > > > > > > > > > > On Sat, Nov 30, 2019 at 1:50 PM Saikat Maitra > > wrote: > > > > > Hello Denis, > > > > > > Thank you for your email and sharing your thoughts on the release > > process. > > > I will update the artifact id and dependencies for ignite-extensions > > > accordingly. > > > > > > I had created Ignite-Extensions project as a root level project and in > > > teamcity I was facing issues pulling dependencies for 2.8.0-SNAPSHOT > > > whereas I was able to pull dependencies for ignite-core 2.7.6 from > maven > > > central. I will look into it further why teamcity build was not able to > > > pull snapshot dependencies. > > > > > > I will also create "ignite-core-2.9+" branch for the upcoming release > > > process. > > > > > > Thank you, > > > Saikat > > > > > > > > > > > > > > > On Wed, Nov 27, 2019 at 1:05 PM Denis Magda wrote: > > > > > >> Hi Saikat, > > >> > > >> Thanks for driving this activity forward and raising the question. Let > > me > > >> share my thoughts below and let's see what the broader community > thinks. > > >> > > >> Each extension needs to have its own version unrelated to the core and > > >> Maven's groupId parameter for extension artifacts should be > > >> "org.apache.ignite.ext". For instance, the very first release of Flink > > in > > >> the form of extension should be pulled from Maven this way > > >> > > >> > > >> org.apache.ignite.ext > > >> ignite-flink > > >> 1.0.0 > > >> > > >> > > >> When it comes to the releases, all the extensions need to be verified > > for > > >> an upcoming release and updated if needed (with the version increase > > only > > >> for those updated). Thus, looks like the extensions master needs to be > > >> linked to the latest Ignite core snapshot. Whenever the core will be > > being > > >> prepared and any extensions need to be modified we can take this > > approach: > > >> > > >>- Create a branch of extensions for the upcoming core release. For > > >>instance, "ignite-core-2.9+" branch. That's just the branch name > (and > > >> not > > >>any Maven artifact name) with "+" sign implying that the updated > > >> extensions > > >>will work for Ignite 2.9 and later until we need to update them > again > > >>creating a release branch like "ignite-core-2.14+" > > >>- If only a subset of the extensions was updated then we need to > > >> release > > >>those extensions to Maven. The goal is to avoid the practice of > > >> publishing > > >>Flink or any other extension to Maven for
[jira] [Created] (IGNITE-12429) Rework bytes-based WAL archive size management logic to make historical rebalance more predictable
Ivan Rakov created IGNITE-12429: --- Summary: Rework bytes-based WAL archive size management logic to make historical rebalance more predictable Key: IGNITE-12429 URL: https://issues.apache.org/jira/browse/IGNITE-12429 Project: Ignite Issue Type: Improvement Reporter: Ivan Rakov Since 2.7 DataStorageConfiguration allows to specify size of WAL archive in bytes (see DataStorageConfiguration#maxWalArchiveSize), which is much more trasparent to user. Unfortunately, new logic may be unpredictable when it comes to the historical rebalance. WAL archive is truncated when one of the following conditions occur: 1. Total number of checkpoints in WAL archive is bigger than DataStorageConfiguration#walHistSize 2. Total size of WAL archive is bigger than DataStorageConfiguration#maxWalArchiveSize Independently, in-memory checkpoint history contains only fixed number of last checkpoints (can be changed with IGNITE_PDS_MAX_CHECKPOINT_MEMORY_HISTORY_SIZE, 100 by default). All these particular qualities make it hard for user to cotrol usage of historical rebalance. Imagine the case when user has slight load (WAL gets rotated very slowly) and default checkpoint frequency. After 100 * 3 = 300 minutes, all updates in WAL will be impossible to be received via historical rebalance even if: 1. User has configured large DataStorageConfiguration#maxWalArchiveSize 2. User has configured large DataStorageConfiguration#walHistSize At the same time, setting large IGNITE_PDS_MAX_CHECKPOINT_MEMORY_HISTORY_SIZE will help (only with previous two points combined), but Ignite node heap usage may increase dramatically. I propose to change WAL history management logic in the following way: 1. *Don't* cut WAL archive when number of checkpoint exceeds DataStorageConfiguration#walHistSize. WAL history should be managed only based on DataStorageConfiguration#maxWalArchiveSize. 2. Checkpoint history should contain fixed number of entries, but should cover the whole stored WAL archive (not only its more recent part with IGNITE_PDS_MAX_CHECKPOINT_MEMORY_HISTORY_SIZE last checkpoints). This can be achieved by making checkpoint history sparse: some intermediate checkpoints *may be not present in history*, but fixed number of checkpoints can be positioned either in uniform distribution (trying to keep fixed number of bytes between two neighbour checkpoints) or exponentially (trying to keep fixed ratio between (size of WAL from checkpoint(N-1) to current write pointer) and (size of WAL from checkpoint(N) to current write pointer). -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
Hello! I have found a regression to 2.7 which unfortunately was not caught by our tests: https://issues.apache.org/jira/browse/IGNITE-12428 It would be nice to fix it before releasing 2.8. Regards, -- Ilya Kasnacheev пн, 9 дек. 2019 г. в 14:58, Ivan Pavlukhin : > Maxim, > > > I see the issue [1] is unassigned. Do we have a person who will fix it > > bravely? :) > > Let's wait Saikat as an original ticket contributor. > > > P.S. The issue [1] doesn't seem to be a truly release `blocker`. > > For me it sounds not good to intentionally include such behavior into > product release. > > пн, 9 дек. 2019 г. в 13:26, Maxim Muzafarov : > > > > Ivan, > > > > I see the issue [1] is unassigned. Do we have a person who will fix it > > bravely? :) > > > > If not I think it's better to revert the issue from the 2.8 release > > branch and fix it in a calm manner in the master branch. I suppose > > there will be enough of such changes to perform a next minor release > > (e.g. 2.8.1) with such fixes. > > > > P.S. The issue [1] doesn't seem to be a truly release `blocker`. > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12424 > > > > On Mon, 9 Dec 2019 at 11:39, Ivan Pavlukhin wrote: > > > > > > Saikat, igniters > > > > > > I raised a blocker [1] for 2.8 related to recently implemented default > > > query timeout [2]. Currently we have a buggy behavior for thin JDBC > > > driver when a default timeout is configured. > > > > > > Actually I see 2 options to go: > > > 1. Fix the issue with thin JDBC [1]. > > > 2. Exclude a default query timeout support [2] from 2.8. > > > > > > Please share your thoughts! > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12424 > > > [2] https://issues.apache.org/jira/browse/IGNITE-7285 > > > > > > пт, 6 дек. 2019 г. в 19:55, Maxim Muzafarov : > > > > > > > > Slava, > > > > > > > > > > > > Thanks for noticing. I think we can include both of them. > > > > Do you need any help from my side? > > > > > > > > On Fri, 6 Dec 2019 at 14:02, Вячеслав Коптилин < > slava.kopti...@gmail.com> wrote: > > > > > > > > > > Hello Maxim, > > > > > > > > > > I found two issues that should be included in the upcoming AI 2.8, > I think. > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12409 - Already > done and > > > > > can be cherry-picked into the release branch. > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-12419 - This > one looks > > > > > like a blocker for the release. Pull-request is available. > > > > > > > > > > Thanks, > > > > > S. > > > > > > > > > > вт, 3 дек. 2019 г. в 15:30, Alexei Scherbakov < > alexey.scherbak...@gmail.com > > > > > >: > > > > > > > > > > > No objections. > > > > > > > > > > > > вт, 3 дек. 2019 г. в 15:02, Maxim Muzafarov : > > > > > > > > > > > > > Alexey, > > > > > > > > > > > > > > > > > > > > > Yet another issue [1] with corrupted B+Tree exception on the > master > > > > > > branch. > > > > > > > My suggestion is to revert the IGNITE-11704 [3] issue from the > master > > > > > > > branch and rework the patch. > > > > > > > > > > > > > > Any objections? > > > > > > > > > > > > > > Configuration: > > > > > > > [1] persistence = false > > > > > > > [2] persistence = true > > > > > > > > > > > > > > class > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: > > > > > > > B+Tree is corrupted [pages(groupId, pageId)=[IgniteBiTuple > > > > > > > [val1=1544803905, val2=844420635196573]], msg=Runtime failure > on > > > > > > > cursor iteration] > > > > > > > at > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.corruptedTreeException(BPlusTree.java:5927) > > > > > > > at > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.nextPage(BPlusTree.java:5438) > > > > > > > at > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.next(BPlusTree.java:5661) > > > > > > > at > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl$1.next(IgniteCacheOffheapManagerImpl.java:3020) > > > > > > > at > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition$2.hasNextX(GridDhtLocalPartition.java:1226) > > > > > > > at > > > > > > > > > > > > > > org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45) > > > > > > > at > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.doClear(GridDhtLocalPartition.java:1295) > > > > > > > at > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.clearTombstones(GridDhtLocalPartition.java:1242) > > > > > > > at > > > > > > > > > > > > > >
[jira] [Created] (IGNITE-12428) Cache configuration cacheLoader error after cfgs serialization changes
Ilya Kasnacheev created IGNITE-12428:
Summary: Cache configuration cacheLoader error after cfgs
serialization changes
Key: IGNITE-12428
URL: https://issues.apache.org/jira/browse/IGNITE-12428
Project: Ignite
Issue Type: Bug
Components: cache
Affects Versions: 2.8
Reporter: Ilya Kasnacheev
The attached test will work on 2.7 but will fail on master
{code}
[19:37:44,606][SEVERE][exchange-worker-#43][GridDhtPartitionsExchangeFuture]
Failed to initialize cache(s) (will try to rollback)
[exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1,
minorTopVer=1], discoEvt=DiscoveryCustomEvent
[customMsg=DynamicCacheChangeBatch
[id=18f068bee61-a388ef9b-b1b7-435f-897f-2397f3af9cff, reqs=ArrayList
[DynamicCacheChangeRequest [cacheName=foo-0, hasCfg=true,
nodeId=895176bf-6687-4a05-a7b3-3a1bfa81b728, clientStartOnly=false, stop=false,
destroy=false, disabledAfterStartfalse]], exchangeActions=ExchangeActions
[startCaches=[foo-0], stopCaches=null, startGrps=[foo-0], stopGrps=[],
resetParts=null, stateChangeRequest=null], startCaches=false],
affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1],
super=DiscoveryEvent [evtNode=TcpDiscoveryNode
[id=895176bf-6687-4a05-a7b3-3a1bfa81b728,
consistentId=0:0:0:0:0:0:0:1%lo,127.0.0.1,172.17.0.1,192.168.1.2:47500,
addrs=ArrayList [0:0:0:0:0:0:0:1%lo, 127.0.0.1, 172.17.0.1, 192.168.1.2],
sockAddrs=HashSet [/172.17.0.1:47500, /0:0:0:0:0:0:0:1%lo:47500,
/127.0.0.1:47500, /192.168.1.2:47500], discPort=47500, order=1, intOrder=1,
lastExchangeTime=1575909462941, loc=true, ver=2.7.0#20191209-sha1:,
isClient=false], topVer=1, nodeId8=895176bf, msg=null,
type=DISCOVERY_CUSTOM_EVT, tstamp=1575909464589]], nodeId=895176bf,
evt=DISCOVERY_CUSTOM_EVT],
caches=[o.a.i.i.processors.cache.ExchangeActions$CacheGroupActionData@76180c2b]]
class org.apache.ignite.IgniteCheckedException: Cannot enable read-through
(loader or store is not provided) for cache: foo-0
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.validate(GridCacheProcessor.java:609)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.createCacheContext(GridCacheProcessor.java:1616)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheContext(GridCacheProcessor.java:2398)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:2333)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.lambda$prepareStartCaches$55a0e703$1(GridCacheProcessor.java:2208)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.lambda$prepareStartCachesIfPossible$6(GridCacheProcessor.java:2178)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareStartCaches(GridCacheProcessor.java:2205)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareStartCachesIfPossible(GridCacheProcessor.java:2176)
at
org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.processCacheStartRequests(CacheAffinitySharedManager.java:953)
at
org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:839)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:1270)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:793)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:3031)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2880)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Thread.java:748)
javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException:
Failed to complete exchange process.
at
org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1318)
at
org.apache.ignite.internal.IgniteKernal.createCache(IgniteKernal.java:2980)
at
org.apache.ignite.CacheStoreInitializationTest.createCache(CacheStoreInitializationTest.java:107)
at
org.apache.ignite.CacheStoreInitializationTest.deadlock_shuffledRandomOrder(CacheStoreInitializationTest.java:127)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke
[jira] [Created] (IGNITE-12427) .NET: Ignite does not start under Mono on Linux because of UnmanagedThread
Pavel Tupitsyn created IGNITE-12427:
---
Summary: .NET: Ignite does not start under Mono on Linux because
of UnmanagedThread
Key: IGNITE-12427
URL: https://issues.apache.org/jira/browse/IGNITE-12427
Project: Ignite
Issue Type: Bug
Components: platforms
Affects Versions: 2.8
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
Fix For: 2.9
{code}
System.DllNotFoundException : libcoreclr.so assembly:
type: member:(null)
at (wrapper managed-to-native)
Apache.Ignite.Core.Impl.Unmanaged.UnmanagedThread+NativeMethodsLinux.pthread_key_create(intptr,intptr)
at Apache.Ignite.Core.Impl.Unmanaged.UnmanagedThread.SetThreadExitCallback
(System.IntPtr callbackPtr) [0x00084] in
/home/pavel/w/ignite/modules/platforms/dotnet/Apache.Ignite.Core/Impl/Unmanaged/UnmanagedThread.cs:70
at Apache.Ignite.Core.Impl.Unmanaged.Jni.Jvm..ctor (System.IntPtr jvmPtr)
[0x00041] in
/home/pavel/w/ignite/modules/platforms/dotnet/Apache.Ignite.Core/Impl/Unmanaged/Jni/Jvm.cs:100
at Apache.Ignite.Core.Impl.Unmanaged.Jni.Jvm.GetOrCreate
(System.Collections.Generic.IList`1[T] options) [0x00013] in
/home/pavel/w/ignite/modules/platforms/dotnet/Apache.Ignite.Core/Impl/Unmanaged/Jni/Jvm.cs:149
at Apache.Ignite.Core.Impl.IgniteManager.CreateJvm
(Apache.Ignite.Core.IgniteConfiguration cfg, Apache.Ignite.Core.Log.ILogger
log) [0x00044] in
/home/pavel/w/ignite/modules/platforms/dotnet/Apache.Ignite.Core/Impl/IgniteManager.cs:132
at Apache.Ignite.Core.Impl.IgniteManager.CreateJvmContext
(Apache.Ignite.Core.IgniteConfiguration cfg, Apache.Ignite.Core.Log.ILogger
log) [0x00099] in
/home/pavel/w/ignite/modules/platforms/dotnet/Apache.Ignite.Core/Impl/IgniteManager.cs:83
at Apache.Ignite.Core.Ignition.Start (Apache.Ignite.Core.IgniteConfiguration
cfg) [0x00078] in
/home/pavel/w/ignite/modules/platforms/dotnet/Apache.Ignite.Core/Ignition.cs:251
{code}
Mono is not officially supported by Ignite.NET, but it is useful to be able to
compile full solution on Linux.
See how DllLoader uses Os.IsMono check and a separate NativeMethods class.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12426) [IEP-35] Metric configuration: add node local configuration
Nikolay Izhikov created IGNITE-12426: Summary: [IEP-35] Metric configuration: add node local configuration Key: IGNITE-12426 URL: https://issues.apache.org/jira/browse/IGNITE-12426 Project: Ignite Issue Type: Improvement Affects Versions: 2.7 Reporter: Nikolay Izhikov Fix For: 2.9 We should provide the way to node-local configuration for metrics. Configuration should: * survive node restart. * rewrite cluster-wide configuration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Data consistency essentials explained
Thank you! пн, 9 дек. 2019 г. в 19:08, Alexei Scherbakov : > > Ivan, > > It's true. The article is relevant for upcoming 2.8 release. I've added a > remark on that. > > пн, 9 дек. 2019 г. в 15:44, Ivan Pavlukhin : > > > Alexei, > > > > Am I getting it right that the article is relevant for current master > > and there is no released Ignite versions with described behavior? Is > > it mentioned in the document? Should we do it? What do you think? > > > > вт, 3 дек. 2019 г. в 12:34, Alexei Scherbakov < > > alexey.scherbak...@gmail.com>: > > > > > > Ivan, > > > > > > for me GG docs have more clear information on the subject. > > > As soon as community will improve docs quality link could be changed back > > > to Apache. > > > > > > сб, 30 нояб. 2019 г. в 17:22, Ivan Pavlukhin : > > > > > > > Alexei, > > > > > > > > > Ivan, the link is intentional. > > > > > > > > Could you please elaborate why is it fine to have a such link? The > > > > referred document is out of community control. Do not we have a > > > > similar one in Ignite documentation? Should we discuss it in another > > > > mail thread? > > > > > > > > сб, 30 нояб. 2019 г. в 15:48, Alexei Scherbakov < > > > > alexey.scherbak...@gmail.com>: > > > > > > > > > > Ivan, the link is intentional. > > > > > > > > > > Nikolai, the formatting looks ok to me. Feel free to change it > > however > > > > you > > > > > like. > > > > > > > > > > чт, 28 нояб. 2019 г. в 09:53, Николай Ижиков : > > > > > > > > > > > Hello, Alex. > > > > > > > > > > > > I think we should improve article formatting - code highlight, > > fonts, > > > > etc. > > > > > > > > > > > > For now, it’s very hard to read it. > > > > > > > > > > > > Can you, please, do it? > > > > > > > > > > > > > 28 нояб. 2019 г., в 09:25, Ivan Pavlukhin > > > > > > написал(а): > > > > > > > > > > > > > > Alexei, > > > > > > > > > > > > > > Many thanks for that article! Really nice that now we have a > > document > > > > > > > describing Ignite approach for data consistency. I think it is > > super > > > > > > > important because it is not trivial and very Ignite specific. I > > > > > > > believe lots of Ignite developers will learn something new from > > this > > > > > > > document. > > > > > > > > > > > > > > By the way, I noticed a link to GridGain documentation [1]. Is it > > > > fine? > > > > > > > > > > > > > > [1] > > > > > > > > > > > > https://www.gridgain.com/docs/latest/developers-guide/key-value-api/transactions#pessimistic-transactions > > > > > > > > > > > > > > чт, 28 нояб. 2019 г. в 02:49, Denis Magda : > > > > > > >> > > > > > > >> Alex, > > > > > > >> > > > > > > >> Thanks a lot for putting your knowledge on paper. That's an > > > > invaluable > > > > > > contribution! > > > > > > >> > > > > > > >> Looping in the user community (will be interesting for those > > who use > > > > > > native persistence) and has just spread the word via Ignite twitter > > > > handle: > > > > > > >> https://twitter.com/ApacheIgnite/status/1199837055007612928 > > > > > > >> > > > > > > >> - > > > > > > >> Denis > > > > > > >> > > > > > > >> > > > > > > >> On Wed, Nov 27, 2019 at 12:29 AM Alexei Scherbakov < > > > > > > alexey.scherbak...@gmail.com> wrote: > > > > > > >>> > > > > > > >>> Igniters, > > > > > > >>> > > > > > > >>> As a final action in my almost year long effort in repairing > > data > > > > > > >>> consistency related issues for transactional persistent caches > > I've > > > > > > >>> prepared an article explaining the principles of achieving the > > > > > > consistency > > > > > > >>> between partition copies for these scenarios [1] > > > > > > >>> Comments and suggestions are welcome. > > > > > > >>> I plan to keep the article in actual state if something will > > > > change in > > > > > > this > > > > > > >>> area in the future. > > > > > > >>> Fixes were mostly done under IGNITE-10780 and several follow-up > > > > fixes. > > > > > > >>> > > > > > > >>> There is still work to be done. Currently I'm working on > > similar > > > > issue > > > > > > for > > > > > > >>> atomic persistent caches [2]. I hope to give an update on that > > > > soon. > > > > > > >>> > > > > > > >>> [1] > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/Data+consistency > > > > > > >>> [2] https://issues.apache.org/jira/browse/IGNITE-11797 > > > > > > >>> -- > > > > > > >>> > > > > > > >>> Best regards, > > > > > > >>> Alexei Scherbakov > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Best regards, > > > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Best regards, > > > > > Alexei Scherbakov > > > > > > > > > > > > > > > > -- > > > > Best regards, > > > > Ivan Pavlukhin > > > > > > > > > > > > > -- > > > > > > Best regards, > > > Alexei Scherbakov > > > > > > > > -- > > Best regards, > > Ivan Pavlukhin > > > > > -- > > Best regards, > Alexei Scherbakov -- Best regards, Ivan Pavlukhin
Re: Data consistency essentials explained
Ivan, It's true. The article is relevant for upcoming 2.8 release. I've added a remark on that. пн, 9 дек. 2019 г. в 15:44, Ivan Pavlukhin : > Alexei, > > Am I getting it right that the article is relevant for current master > and there is no released Ignite versions with described behavior? Is > it mentioned in the document? Should we do it? What do you think? > > вт, 3 дек. 2019 г. в 12:34, Alexei Scherbakov < > alexey.scherbak...@gmail.com>: > > > > Ivan, > > > > for me GG docs have more clear information on the subject. > > As soon as community will improve docs quality link could be changed back > > to Apache. > > > > сб, 30 нояб. 2019 г. в 17:22, Ivan Pavlukhin : > > > > > Alexei, > > > > > > > Ivan, the link is intentional. > > > > > > Could you please elaborate why is it fine to have a such link? The > > > referred document is out of community control. Do not we have a > > > similar one in Ignite documentation? Should we discuss it in another > > > mail thread? > > > > > > сб, 30 нояб. 2019 г. в 15:48, Alexei Scherbakov < > > > alexey.scherbak...@gmail.com>: > > > > > > > > Ivan, the link is intentional. > > > > > > > > Nikolai, the formatting looks ok to me. Feel free to change it > however > > > you > > > > like. > > > > > > > > чт, 28 нояб. 2019 г. в 09:53, Николай Ижиков : > > > > > > > > > Hello, Alex. > > > > > > > > > > I think we should improve article formatting - code highlight, > fonts, > > > etc. > > > > > > > > > > For now, it’s very hard to read it. > > > > > > > > > > Can you, please, do it? > > > > > > > > > > > 28 нояб. 2019 г., в 09:25, Ivan Pavlukhin > > > > > написал(а): > > > > > > > > > > > > Alexei, > > > > > > > > > > > > Many thanks for that article! Really nice that now we have a > document > > > > > > describing Ignite approach for data consistency. I think it is > super > > > > > > important because it is not trivial and very Ignite specific. I > > > > > > believe lots of Ignite developers will learn something new from > this > > > > > > document. > > > > > > > > > > > > By the way, I noticed a link to GridGain documentation [1]. Is it > > > fine? > > > > > > > > > > > > [1] > > > > > > > > > https://www.gridgain.com/docs/latest/developers-guide/key-value-api/transactions#pessimistic-transactions > > > > > > > > > > > > чт, 28 нояб. 2019 г. в 02:49, Denis Magda : > > > > > >> > > > > > >> Alex, > > > > > >> > > > > > >> Thanks a lot for putting your knowledge on paper. That's an > > > invaluable > > > > > contribution! > > > > > >> > > > > > >> Looping in the user community (will be interesting for those > who use > > > > > native persistence) and has just spread the word via Ignite twitter > > > handle: > > > > > >> https://twitter.com/ApacheIgnite/status/1199837055007612928 > > > > > >> > > > > > >> - > > > > > >> Denis > > > > > >> > > > > > >> > > > > > >> On Wed, Nov 27, 2019 at 12:29 AM Alexei Scherbakov < > > > > > alexey.scherbak...@gmail.com> wrote: > > > > > >>> > > > > > >>> Igniters, > > > > > >>> > > > > > >>> As a final action in my almost year long effort in repairing > data > > > > > >>> consistency related issues for transactional persistent caches > I've > > > > > >>> prepared an article explaining the principles of achieving the > > > > > consistency > > > > > >>> between partition copies for these scenarios [1] > > > > > >>> Comments and suggestions are welcome. > > > > > >>> I plan to keep the article in actual state if something will > > > change in > > > > > this > > > > > >>> area in the future. > > > > > >>> Fixes were mostly done under IGNITE-10780 and several follow-up > > > fixes. > > > > > >>> > > > > > >>> There is still work to be done. Currently I'm working on > similar > > > issue > > > > > for > > > > > >>> atomic persistent caches [2]. I hope to give an update on that > > > soon. > > > > > >>> > > > > > >>> [1] > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/Data+consistency > > > > > >>> [2] https://issues.apache.org/jira/browse/IGNITE-11797 > > > > > >>> -- > > > > > >>> > > > > > >>> Best regards, > > > > > >>> Alexei Scherbakov > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Best regards, > > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > > > -- > > > > > > > > Best regards, > > > > Alexei Scherbakov > > > > > > > > > > > > -- > > > Best regards, > > > Ivan Pavlukhin > > > > > > > > > -- > > > > Best regards, > > Alexei Scherbakov > > > > -- > Best regards, > Ivan Pavlukhin > -- Best regards, Alexei Scherbakov
Re: Trigger for Ignite 2.8
That's ok, thanks. > On 9 Dec 2019, at 17:49, Maxim Muzafarov wrote: > > Hello, Pert. > > The ignite-2.8 branch already exists on GitHub repository [1]. > > I think this trigger can we removed since the TC.Bot has been > configured [2] to run build nightly. > Can you confirm that such testing the release branch is OK ? > > [1] https://github.com/apache/ignite/tree/ignite-2.8 > [2] https://mtcga.gridgain.com/current.html?branch=ignite-2.8 > > On Mon, 9 Dec 2019 at 17:44, Petr Ivanov wrote: >> >> Maxim, I saw you have deployed trigger for test on Ignite 2.8 branch via >> pull request. >> >> Are there problems with making real branch ignite-2.8 and putting it into >> repository?
Re: Trigger for Ignite 2.8
Hello, Pert. The ignite-2.8 branch already exists on GitHub repository [1]. I think this trigger can we removed since the TC.Bot has been configured [2] to run build nightly. Can you confirm that such testing the release branch is OK ? [1] https://github.com/apache/ignite/tree/ignite-2.8 [2] https://mtcga.gridgain.com/current.html?branch=ignite-2.8 On Mon, 9 Dec 2019 at 17:44, Petr Ivanov wrote: > > Maxim, I saw you have deployed trigger for test on Ignite 2.8 branch via pull > request. > > Are there problems with making real branch ignite-2.8 and putting it into > repository?
Trigger for Ignite 2.8
Maxim, I saw you have deployed trigger for test on Ignite 2.8 branch via pull request. Are there problems with making real branch ignite-2.8 and putting it into repository?
Re: [TeamCIty] Issues with the disc space
Thanks a lot! On Mon, Dec 9, 2019 at 4:26 PM Ivan Pavlukhin wrote: > Folks, > > All agents were cleaned. > > пн, 9 дек. 2019 г. в 09:13, Anton Vinogradov : > > > > aitc-lin03_01 also broken > > > > On Fri, Dec 6, 2019 at 2:49 PM Anton Vinogradov wrote: > > > > > listed agents seem to be broken too: > > > aitc-lin11_02 > > > aitc-lin10_02 > > > aitc-lin05_01 > > > aitc-lin04_04 > > > > > > On Fri, Dec 6, 2019 at 2:22 PM Ivan Pavlukhin > wrote: > > > > > >> aitc-lin08 was handled as well. > > >> > > >> пт, 6 дек. 2019 г. в 10:41, Maxim Muzafarov : > > >> > > > >> > Ivan, > > >> > > > >> > And another one. > > >> > Starting the build on the agent "aitc-lin08_01" > > >> > Failed to download file 'ignite.zip': No space left on device > > >> > > > >> > [1] > > >> > https://ci.ignite.apache.org/viewLog.html?buildId=4815786=IgniteTests24Java8_ThinClientPython=buildLog_IgniteTests24Java8=%3Cdefault%3E&_focus=59 > > >> > > > >> > On Thu, 5 Dec 2019 at 16:57, Maxim Muzafarov > wrote: > > >> > > > > >> > > Ivan, > > >> > > > > >> > > Starting the build on the agent "aitc-lin12_03" > > >> > > Failed to download file 'ignite.zip': No space left on device > > >> > > > > >> > > [1] > > >> > https://ci.ignite.apache.org/viewLog.html?tab=buildLog=tree=debug=all=4814624 > > >> > > > > >> > > On Thu, 5 Dec 2019 at 15:13, Ivan Pavlukhin > > >> wrote: > > >> > > > > > >> > > > Folks, > > >> > > > > > >> > > > Agent aitc-lin07 was cleaned. Please check that everything is > > >> working > > >> > > > fine now. If you observe other agents experiencing same problems > > >> > > > please write in this thread. > > >> > > > > > >> > > > чт, 5 дек. 2019 г. в 14:20, Anton Vinogradov : > > >> > > > > > > >> > > > > Main problem here that TC Bot ignores such failures for > feature > > >> branches > > >> > > > > (since they also happen in master). > > >> > > > > So, you may gain green VISA without any checks. > > >> > > > > > > >> > > > > Everyone, > > >> > > > > Please make sure you VISAs are legal before the merge. > > >> > > > > > > >> > > > > On Thu, Dec 5, 2019 at 2:10 PM Maxim Muzafarov < > mmu...@apache.org> > > >> wrote: > > >> > > > > > > >> > > > > > Igniters, > > >> > > > > > > > >> > > > > > > > >> > > > > > It seems the issue is still actual. Do we have any idea > what's > > >> going on? > > >> > > > > > It's hard to work on the 2.8 release stabilization when we > > >> don't have > > >> > > > > > stable mater branch [1]. > > >> > > > > > > > >> > > > > > > > >> > > > > > [1] > > >> > > > > > > > >> > https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_RunAll_IgniteTests24Java8=%3Cdefault%3E=buildTypeStatusDiv > > >> > > > > > > > >> > > > > > On Tue, 3 Dec 2019 at 20:36, Николай Ижиков < > > >> nizhi...@apache.org> wrote: > > >> > > > > > > > > >> > > > > > > Hello, Igniters. > > >> > > > > > > > > >> > > > > > > Looks like we have an issues with the TC agents disc > space [1] > > >> > > > > > > Can someone take a look? > > >> > > > > > > > > >> > > > > > > [1] > > >> > > > > > > > >> > https://ci.ignite.apache.org/viewLog.html?buildId=4809938=buildResultsDiv=IgniteTests24Java8_PlatformCPPLinuxClang > > >> > > > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > -- > > >> > > > Best regards, > > >> > > > Ivan Pavlukhin > > >> > > >> > > >> > > >> -- > > >> Best regards, > > >> Ivan Pavlukhin > > >> > > > > > > > -- > Best regards, > Ivan Pavlukhin >
Re: [TeamCIty] Issues with the disc space
Folks, All agents were cleaned. пн, 9 дек. 2019 г. в 09:13, Anton Vinogradov : > > aitc-lin03_01 also broken > > On Fri, Dec 6, 2019 at 2:49 PM Anton Vinogradov wrote: > > > listed agents seem to be broken too: > > aitc-lin11_02 > > aitc-lin10_02 > > aitc-lin05_01 > > aitc-lin04_04 > > > > On Fri, Dec 6, 2019 at 2:22 PM Ivan Pavlukhin wrote: > > > >> aitc-lin08 was handled as well. > >> > >> пт, 6 дек. 2019 г. в 10:41, Maxim Muzafarov : > >> > > >> > Ivan, > >> > > >> > And another one. > >> > Starting the build on the agent "aitc-lin08_01" > >> > Failed to download file 'ignite.zip': No space left on device > >> > > >> > [1] > >> https://ci.ignite.apache.org/viewLog.html?buildId=4815786=IgniteTests24Java8_ThinClientPython=buildLog_IgniteTests24Java8=%3Cdefault%3E&_focus=59 > >> > > >> > On Thu, 5 Dec 2019 at 16:57, Maxim Muzafarov wrote: > >> > > > >> > > Ivan, > >> > > > >> > > Starting the build on the agent "aitc-lin12_03" > >> > > Failed to download file 'ignite.zip': No space left on device > >> > > > >> > > [1] > >> https://ci.ignite.apache.org/viewLog.html?tab=buildLog=tree=debug=all=4814624 > >> > > > >> > > On Thu, 5 Dec 2019 at 15:13, Ivan Pavlukhin > >> wrote: > >> > > > > >> > > > Folks, > >> > > > > >> > > > Agent aitc-lin07 was cleaned. Please check that everything is > >> working > >> > > > fine now. If you observe other agents experiencing same problems > >> > > > please write in this thread. > >> > > > > >> > > > чт, 5 дек. 2019 г. в 14:20, Anton Vinogradov : > >> > > > > > >> > > > > Main problem here that TC Bot ignores such failures for feature > >> branches > >> > > > > (since they also happen in master). > >> > > > > So, you may gain green VISA without any checks. > >> > > > > > >> > > > > Everyone, > >> > > > > Please make sure you VISAs are legal before the merge. > >> > > > > > >> > > > > On Thu, Dec 5, 2019 at 2:10 PM Maxim Muzafarov > >> wrote: > >> > > > > > >> > > > > > Igniters, > >> > > > > > > >> > > > > > > >> > > > > > It seems the issue is still actual. Do we have any idea what's > >> going on? > >> > > > > > It's hard to work on the 2.8 release stabilization when we > >> don't have > >> > > > > > stable mater branch [1]. > >> > > > > > > >> > > > > > > >> > > > > > [1] > >> > > > > > > >> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_RunAll_IgniteTests24Java8=%3Cdefault%3E=buildTypeStatusDiv > >> > > > > > > >> > > > > > On Tue, 3 Dec 2019 at 20:36, Николай Ижиков < > >> nizhi...@apache.org> wrote: > >> > > > > > > > >> > > > > > > Hello, Igniters. > >> > > > > > > > >> > > > > > > Looks like we have an issues with the TC agents disc space [1] > >> > > > > > > Can someone take a look? > >> > > > > > > > >> > > > > > > [1] > >> > > > > > > >> https://ci.ignite.apache.org/viewLog.html?buildId=4809938=buildResultsDiv=IgniteTests24Java8_PlatformCPPLinuxClang > >> > > > > > > >> > > > > >> > > > > >> > > > > >> > > > -- > >> > > > Best regards, > >> > > > Ivan Pavlukhin > >> > >> > >> > >> -- > >> Best regards, > >> Ivan Pavlukhin > >> > > -- Best regards, Ivan Pavlukhin
Re: Data consistency essentials explained
Alexei, Am I getting it right that the article is relevant for current master and there is no released Ignite versions with described behavior? Is it mentioned in the document? Should we do it? What do you think? вт, 3 дек. 2019 г. в 12:34, Alexei Scherbakov : > > Ivan, > > for me GG docs have more clear information on the subject. > As soon as community will improve docs quality link could be changed back > to Apache. > > сб, 30 нояб. 2019 г. в 17:22, Ivan Pavlukhin : > > > Alexei, > > > > > Ivan, the link is intentional. > > > > Could you please elaborate why is it fine to have a such link? The > > referred document is out of community control. Do not we have a > > similar one in Ignite documentation? Should we discuss it in another > > mail thread? > > > > сб, 30 нояб. 2019 г. в 15:48, Alexei Scherbakov < > > alexey.scherbak...@gmail.com>: > > > > > > Ivan, the link is intentional. > > > > > > Nikolai, the formatting looks ok to me. Feel free to change it however > > you > > > like. > > > > > > чт, 28 нояб. 2019 г. в 09:53, Николай Ижиков : > > > > > > > Hello, Alex. > > > > > > > > I think we should improve article formatting - code highlight, fonts, > > etc. > > > > > > > > For now, it’s very hard to read it. > > > > > > > > Can you, please, do it? > > > > > > > > > 28 нояб. 2019 г., в 09:25, Ivan Pavlukhin > > > > написал(а): > > > > > > > > > > Alexei, > > > > > > > > > > Many thanks for that article! Really nice that now we have a document > > > > > describing Ignite approach for data consistency. I think it is super > > > > > important because it is not trivial and very Ignite specific. I > > > > > believe lots of Ignite developers will learn something new from this > > > > > document. > > > > > > > > > > By the way, I noticed a link to GridGain documentation [1]. Is it > > fine? > > > > > > > > > > [1] > > > > > > https://www.gridgain.com/docs/latest/developers-guide/key-value-api/transactions#pessimistic-transactions > > > > > > > > > > чт, 28 нояб. 2019 г. в 02:49, Denis Magda : > > > > >> > > > > >> Alex, > > > > >> > > > > >> Thanks a lot for putting your knowledge on paper. That's an > > invaluable > > > > contribution! > > > > >> > > > > >> Looping in the user community (will be interesting for those who use > > > > native persistence) and has just spread the word via Ignite twitter > > handle: > > > > >> https://twitter.com/ApacheIgnite/status/1199837055007612928 > > > > >> > > > > >> - > > > > >> Denis > > > > >> > > > > >> > > > > >> On Wed, Nov 27, 2019 at 12:29 AM Alexei Scherbakov < > > > > alexey.scherbak...@gmail.com> wrote: > > > > >>> > > > > >>> Igniters, > > > > >>> > > > > >>> As a final action in my almost year long effort in repairing data > > > > >>> consistency related issues for transactional persistent caches I've > > > > >>> prepared an article explaining the principles of achieving the > > > > consistency > > > > >>> between partition copies for these scenarios [1] > > > > >>> Comments and suggestions are welcome. > > > > >>> I plan to keep the article in actual state if something will > > change in > > > > this > > > > >>> area in the future. > > > > >>> Fixes were mostly done under IGNITE-10780 and several follow-up > > fixes. > > > > >>> > > > > >>> There is still work to be done. Currently I'm working on similar > > issue > > > > for > > > > >>> atomic persistent caches [2]. I hope to give an update on that > > soon. > > > > >>> > > > > >>> [1] > > > > https://cwiki.apache.org/confluence/display/IGNITE/Data+consistency > > > > >>> [2] https://issues.apache.org/jira/browse/IGNITE-11797 > > > > >>> -- > > > > >>> > > > > >>> Best regards, > > > > >>> Alexei Scherbakov > > > > > > > > > > > > > > > > > > > > -- > > > > > Best regards, > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > -- > > > > > > Best regards, > > > Alexei Scherbakov > > > > > > > > -- > > Best regards, > > Ivan Pavlukhin > > > > > -- > > Best regards, > Alexei Scherbakov -- Best regards, Ivan Pavlukhin
[jira] [Created] (IGNITE-12425) Deadlock on CacheStore.loadAll
Ilya Kasnacheev created IGNITE-12425:
Summary: Deadlock on CacheStore.loadAll
Key: IGNITE-12425
URL: https://issues.apache.org/jira/browse/IGNITE-12425
Project: Ignite
Issue Type: Bug
Components: cache
Affects Versions: 2.7.6
Reporter: Ilya Kasnacheev
Attachments: ignite-dataload-deadlock.zip
Yes, we do have a deadlock in CacheStore.loadAll, as demonstrated by this
reproducer.
The obvious embarassing fix is to replace HashMap with TreeMap, which will of
course not work if keys are not compabable
{code}
diff --git
a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheAdapter.java
b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheAdapter.java
index 3156d6d662d..3947800a908 100644
---
a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheAdapter.java
+++
b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheAdapter.java
@@ -34,6 +34,7 @@ import java.util.List;
import java.util.Map;
import java.util.NoSuchElementException;
import java.util.Set;
+import java.util.TreeMap;
import java.util.UUID;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
@@ -2054,7 +2055,7 @@ public abstract class GridCacheAdapter implements
IgniteInternalCache();
+misses = new TreeMap<>();
misses.put(key, res);
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
Maxim, > I see the issue [1] is unassigned. Do we have a person who will fix it > bravely? :) Let's wait Saikat as an original ticket contributor. > P.S. The issue [1] doesn't seem to be a truly release `blocker`. For me it sounds not good to intentionally include such behavior into product release. пн, 9 дек. 2019 г. в 13:26, Maxim Muzafarov : > > Ivan, > > I see the issue [1] is unassigned. Do we have a person who will fix it > bravely? :) > > If not I think it's better to revert the issue from the 2.8 release > branch and fix it in a calm manner in the master branch. I suppose > there will be enough of such changes to perform a next minor release > (e.g. 2.8.1) with such fixes. > > P.S. The issue [1] doesn't seem to be a truly release `blocker`. > > [1] https://issues.apache.org/jira/browse/IGNITE-12424 > > On Mon, 9 Dec 2019 at 11:39, Ivan Pavlukhin wrote: > > > > Saikat, igniters > > > > I raised a blocker [1] for 2.8 related to recently implemented default > > query timeout [2]. Currently we have a buggy behavior for thin JDBC > > driver when a default timeout is configured. > > > > Actually I see 2 options to go: > > 1. Fix the issue with thin JDBC [1]. > > 2. Exclude a default query timeout support [2] from 2.8. > > > > Please share your thoughts! > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12424 > > [2] https://issues.apache.org/jira/browse/IGNITE-7285 > > > > пт, 6 дек. 2019 г. в 19:55, Maxim Muzafarov : > > > > > > Slava, > > > > > > > > > Thanks for noticing. I think we can include both of them. > > > Do you need any help from my side? > > > > > > On Fri, 6 Dec 2019 at 14:02, Вячеслав Коптилин > > > wrote: > > > > > > > > Hello Maxim, > > > > > > > > I found two issues that should be included in the upcoming AI 2.8, I > > > > think. > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12409 - Already done > > > > and > > > > can be cherry-picked into the release branch. > > > > [2] https://issues.apache.org/jira/browse/IGNITE-12419 - This one looks > > > > like a blocker for the release. Pull-request is available. > > > > > > > > Thanks, > > > > S. > > > > > > > > вт, 3 дек. 2019 г. в 15:30, Alexei Scherbakov > > > > > > > >: > > > > > > > > > No objections. > > > > > > > > > > вт, 3 дек. 2019 г. в 15:02, Maxim Muzafarov : > > > > > > > > > > > Alexey, > > > > > > > > > > > > > > > > > > Yet another issue [1] with corrupted B+Tree exception on the master > > > > > branch. > > > > > > My suggestion is to revert the IGNITE-11704 [3] issue from the > > > > > > master > > > > > > branch and rework the patch. > > > > > > > > > > > > Any objections? > > > > > > > > > > > > Configuration: > > > > > > [1] persistence = false > > > > > > [2] persistence = true > > > > > > > > > > > > class > > > > > > > > > > > org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: > > > > > > B+Tree is corrupted [pages(groupId, pageId)=[IgniteBiTuple > > > > > > [val1=1544803905, val2=844420635196573]], msg=Runtime failure on > > > > > > cursor iteration] > > > > > > at > > > > > > > > > > > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.corruptedTreeException(BPlusTree.java:5927) > > > > > > at > > > > > > > > > > > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.nextPage(BPlusTree.java:5438) > > > > > > at > > > > > > > > > > > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.next(BPlusTree.java:5661) > > > > > > at > > > > > > > > > > > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl$1.next(IgniteCacheOffheapManagerImpl.java:3020) > > > > > > at > > > > > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition$2.hasNextX(GridDhtLocalPartition.java:1226) > > > > > > at > > > > > > > > > > > org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45) > > > > > > at > > > > > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.doClear(GridDhtLocalPartition.java:1295) > > > > > > at > > > > > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.clearTombstones(GridDhtLocalPartition.java:1242) > > > > > > at > > > > > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.topology.PartitionsEvictManager$ClearTombstonesTask.run0(PartitionsEvictManager.java:673) > > > > > > at > > > > > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.topology.PartitionsEvictManager$AbstractEvictionTask.run(PartitionsEvictManager.java:587) > > > > > > at > > > > > > > > > > > org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7061) > > > > > > at > > > > > > > > > > > org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:827) > > > > > > at
Re: Joining node validation failure event.
Mikhail, I looked into PR [1] and left a couple of minor comments on GitHub. But actually a real thing where I do not have enough expertise is a notification firing strategy. From the first glance looks fine, but there might be concerns I missed. [1] https://github.com/apache/ignite/pull/7057 ср, 4 дек. 2019 г. в 23:02, Mikhail Petrov : > > Ivan, Andrey, Nikolay, > > Thanks for your answers! > > Igniters, if there are no objections, could anyone take a look at > implementation [1] of described above event, please? > > [1] https://github.com/apache/ignite/pull/7057 > > Regards, > Mikhail. > > On 04.12.2019 21:54, Ivan Pavlukhin wrote: > > Mikhail, > > > > Yes I am fine with the approach. > > > > ср, 4 дек. 2019 г. в 20:30, Mikhail Petrov : > >> Ivan, > >> > >> Am I right, that current approach to solving this problem looks good for > >> you? > >> > >> Regards, > >> Mikhail. > >> > >> On 03.12.2019 15:12, Ivan Pavlukhin wrote: > >>> Mikhail, > >>> > >>> Yep, I see IgniteNodeValidationResult in new event in PR [1]. > >>> > Discovery events such as (join/left/failed) are connected with a > topology version change. In my case that not happens and may be > misleading. That's why the new event type was chosen. > >>> I did not mean that one of those events should be used. I meant that > >>> it sounds natural to me to have an additional "unsuccessful node join" > >>> event (like is done in PR[1]). > >>> > >>> https://github.com/apache/ignite/pull/7057/files > >>> > >>> вт, 3 дек. 2019 г. в 14:32, Mikhail Petrov : > Nikolay, Ivan, > > I understood the possible problem. It seems that it can be solved using > EventStorageSpi which starts before DiscoveryManager. > > As for me, ClusterNode is enough. It contains all info about joining > node including its attributes. > > Discovery events such as (join/left/failed) are connected with a > topology version change. In my case that not happens and may be > misleading. That's why the new event type was chosen. > > The cause of the failure is also presented in the event. > > Regards, > Mikhail. > > On 03.12.2019 13:19, Николай Ижиков wrote: > > Exception(s) from component(s) that don’t want node joined. > > > >> 3 дек. 2019 г., в 12:39, Ivan Pavlukhin > >> написал(а): > >> > >> How that reason will look like? Actually, I mostly thinking about > >> general API here. What I would like to avoid is exposing something not > >> general but needed only for a particular extension (plugin). > >> > >> вт, 3 дек. 2019 г. в 12:31, Николай Ижиков : > >>> I think we also should provide the reason why join failed. > >>> > 3 дек. 2019 г., в 12:22, Ivan Pavlukhin > написал(а): > > Mikhail, > > So, I suppose there should be ordering guarantees that listener is > registered before first validation failure can occur. Hope > GridComponent#onKernalStart is the right place. Is it enough to pass > only problematic node id (or ClusterNode) with an event? Actually > such > event seems to fit naturally node join/left/failed events. > > вт, 3 дек. 2019 г. в 10:38, Mikhail Petrov : > > Hi Ivan. > > > > No other lifecycle events are needed in my case. > > > > We can register a listener in the security component's > > GridComponent#onKernalStart method and listen locally to every > > failed > > joining attempt. Am I missing something? > > > > Regards, > > Mikhail. > > > > On 03.12.2019 8:48, Ivan Pavlukhin wrote: > >> Mikhail, > >> > >> Do you need some ordering guarantees among node lifecycle events > >> and > >> listener notifications. For example, I can imagine that it is good > >> to > >> notify security component about every node failed validation. How > >> can > >> we achieve it with events (I assume dynamic listener registration)? > >> > >> пн, 2 дек. 2019 г. в 18:09, Mikhail Petrov : > >>> Hi, Andrey. > >>> > >>> It doesn't influence on authentication or authorization process. > >>> There > >>> is a security requirement that demands to notify some outer > >>> security > >>> subsystems in a specific way when joining node validation failed > >>> in any > >>> Ignite component (e.g. GridCacheProcessor) not only in > >>> IgniteSecurityProcessor. So PluginProvider#validateNewNode is not > >>> acceptable for me. > >>> > >>> Regards, > >>> Mikhail. > >>> > >>> On 02.12.2019 16:35, Andrey Gura wrote: > Mikhail, > > I don't understand how node validation on join affects
Re: [DISCUSS] dependencies and release process for Ignite Extensions
Hello! I think that your build should depend on an Apache Ignite build(s) or just use an already released version. In the same fashion as all our teamcity depend on "Build Apache Ignite" and use its artifacts. Here you want Ignite snapshot but Teamcity does not know where to take it from. Regards, -- Ilya Kasnacheev пн, 9 дек. 2019 г. в 06:40, Saikat Maitra : > Hello, > > I am running into a problem specific to teamcity build for Ignite > Extensions project. When I set the dependencies to 2.9.0-SNAPSHOT I am > getting an error message during build as below > > [06:24:12][Step 4/5] Failed to execute goal on project ignite-flink-ext: > Could not resolve dependencies for project > org.apache.ignite.ext:ignite-flink-ext:jar:1.0.0-SNAPSHOT: The following > artifacts could not be resolved: > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT, > org.apache.ignite:ignite-core:jar:tests:2.9.0-SNAPSHOT, > org.apache.ignite:ignite-log4j:jar:2.9.0-SNAPSHOT, > org.apache.ignite:ignite-spring:jar:2.9.0-SNAPSHOT: Could not find artifact > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT in h2database.com ( > https://h2database.com/m2-repo) > > > and if set artifact dependencies for ~Build Apache Ignite~ then I receive > below error > > [ERROR] [ERROR] Could not find the selected project in the reactor: > :ignite-flink-ext @ > > Build url > > https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteExtensions_Build_IgniteExtensions=pull%2F1%2Fhead=buildTypeStatusDiv > > Can you please let me know if you faced similar problem with teamcity > build? > > I can set Ignite Extensions dependencies to released ignite-core artifacts > version like 2.7.6 and build works fine. > > Regards, > Saikat > > > > > > On Sat, Nov 30, 2019 at 1:50 PM Saikat Maitra > wrote: > > > Hello Denis, > > > > Thank you for your email and sharing your thoughts on the release > process. > > I will update the artifact id and dependencies for ignite-extensions > > accordingly. > > > > I had created Ignite-Extensions project as a root level project and in > > teamcity I was facing issues pulling dependencies for 2.8.0-SNAPSHOT > > whereas I was able to pull dependencies for ignite-core 2.7.6 from maven > > central. I will look into it further why teamcity build was not able to > > pull snapshot dependencies. > > > > I will also create "ignite-core-2.9+" branch for the upcoming release > > process. > > > > Thank you, > > Saikat > > > > > > > > > > On Wed, Nov 27, 2019 at 1:05 PM Denis Magda wrote: > > > >> Hi Saikat, > >> > >> Thanks for driving this activity forward and raising the question. Let > me > >> share my thoughts below and let's see what the broader community thinks. > >> > >> Each extension needs to have its own version unrelated to the core and > >> Maven's groupId parameter for extension artifacts should be > >> "org.apache.ignite.ext". For instance, the very first release of Flink > in > >> the form of extension should be pulled from Maven this way > >> > >> > >> org.apache.ignite.ext > >> ignite-flink > >> 1.0.0 > >> > >> > >> When it comes to the releases, all the extensions need to be verified > for > >> an upcoming release and updated if needed (with the version increase > only > >> for those updated). Thus, looks like the extensions master needs to be > >> linked to the latest Ignite core snapshot. Whenever the core will be > being > >> prepared and any extensions need to be modified we can take this > approach: > >> > >>- Create a branch of extensions for the upcoming core release. For > >>instance, "ignite-core-2.9+" branch. That's just the branch name (and > >> not > >>any Maven artifact name) with "+" sign implying that the updated > >> extensions > >>will work for Ignite 2.9 and later until we need to update them again > >>creating a release branch like "ignite-core-2.14+" > >>- If only a subset of the extensions was updated then we need to > >> release > >>those extensions to Maven. The goal is to avoid the practice of > >> publishing > >>Flink or any other extension to Maven for every core release if there > >> are > >>no changes. > >>- As for a ZIP archive, we should prepare the archive for a download > >>with the name like "ignite-core-2.9+" > >> > >> > >> - > >> Denis > >> > >> > >> On Tue, Nov 26, 2019 at 9:03 PM Saikat Maitra > >> wrote: > >> > >> > Hello, > >> > > >> > I wanted to connect and discuss on the release process for > >> > ignite-extensions. As of today all our integrations since released > >> together > >> > were able to run build based on latest snapshot for example the > current > >> > build depends on 2.8.0-SNAPSHOT. If we are making ignite-extensions as > >> > separate project with different release cycle then it make sense to > have > >> > dependencies on core modules based on released artifact for example > the > >> > dependency for ignite-core would be 2.7.6 > >> > > >> > Please review and share your thoughts.
Re: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
Ivan, I see the issue [1] is unassigned. Do we have a person who will fix it bravely? :) If not I think it's better to revert the issue from the 2.8 release branch and fix it in a calm manner in the master branch. I suppose there will be enough of such changes to perform a next minor release (e.g. 2.8.1) with such fixes. P.S. The issue [1] doesn't seem to be a truly release `blocker`. [1] https://issues.apache.org/jira/browse/IGNITE-12424 On Mon, 9 Dec 2019 at 11:39, Ivan Pavlukhin wrote: > > Saikat, igniters > > I raised a blocker [1] for 2.8 related to recently implemented default > query timeout [2]. Currently we have a buggy behavior for thin JDBC > driver when a default timeout is configured. > > Actually I see 2 options to go: > 1. Fix the issue with thin JDBC [1]. > 2. Exclude a default query timeout support [2] from 2.8. > > Please share your thoughts! > > [1] https://issues.apache.org/jira/browse/IGNITE-12424 > [2] https://issues.apache.org/jira/browse/IGNITE-7285 > > пт, 6 дек. 2019 г. в 19:55, Maxim Muzafarov : > > > > Slava, > > > > > > Thanks for noticing. I think we can include both of them. > > Do you need any help from my side? > > > > On Fri, 6 Dec 2019 at 14:02, Вячеслав Коптилин > > wrote: > > > > > > Hello Maxim, > > > > > > I found two issues that should be included in the upcoming AI 2.8, I > > > think. > > > [1] https://issues.apache.org/jira/browse/IGNITE-12409 - Already done and > > > can be cherry-picked into the release branch. > > > [2] https://issues.apache.org/jira/browse/IGNITE-12419 - This one looks > > > like a blocker for the release. Pull-request is available. > > > > > > Thanks, > > > S. > > > > > > вт, 3 дек. 2019 г. в 15:30, Alexei Scherbakov > > > > > >: > > > > > > > No objections. > > > > > > > > вт, 3 дек. 2019 г. в 15:02, Maxim Muzafarov : > > > > > > > > > Alexey, > > > > > > > > > > > > > > > Yet another issue [1] with corrupted B+Tree exception on the master > > > > branch. > > > > > My suggestion is to revert the IGNITE-11704 [3] issue from the master > > > > > branch and rework the patch. > > > > > > > > > > Any objections? > > > > > > > > > > Configuration: > > > > > [1] persistence = false > > > > > [2] persistence = true > > > > > > > > > > class > > > > > > > > > org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: > > > > > B+Tree is corrupted [pages(groupId, pageId)=[IgniteBiTuple > > > > > [val1=1544803905, val2=844420635196573]], msg=Runtime failure on > > > > > cursor iteration] > > > > > at > > > > > > > > > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.corruptedTreeException(BPlusTree.java:5927) > > > > > at > > > > > > > > > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.nextPage(BPlusTree.java:5438) > > > > > at > > > > > > > > > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.next(BPlusTree.java:5661) > > > > > at > > > > > > > > > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl$1.next(IgniteCacheOffheapManagerImpl.java:3020) > > > > > at > > > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition$2.hasNextX(GridDhtLocalPartition.java:1226) > > > > > at > > > > > > > > > org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45) > > > > > at > > > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.doClear(GridDhtLocalPartition.java:1295) > > > > > at > > > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.clearTombstones(GridDhtLocalPartition.java:1242) > > > > > at > > > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.topology.PartitionsEvictManager$ClearTombstonesTask.run0(PartitionsEvictManager.java:673) > > > > > at > > > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.topology.PartitionsEvictManager$AbstractEvictionTask.run(PartitionsEvictManager.java:587) > > > > > at > > > > > > > > > org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7061) > > > > > at > > > > > > > > > org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:827) > > > > > at > > > > > > > > > org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) > > > > > at > > > > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > > > > at > > > > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > > > > at java.lang.Thread.run(Thread.java:748) > > > > > > > > > > Caused by: java.lang.AssertionError: Key is not ready: > > > > > CacheDataRowAdapter [key=null, val=null, expireTime=-1, ver=null, > > > > > cacheId=0, link=0001003c77d8] > > > > > at > > > > > > > > >
Ignite meetup at Google Headquarters on Dec 18th
Igniters, This month we united with Silicon Valley Java User Group and we'll be repeating In-Memory Computing Essentials for Software Engineers session at Google HQ. Come over to meet in person, a lot of coding and practical examples. RSVP using our local group below (and attend it to not miss future gatherings): https://www.meetup.com/Bay-Area-In-Memory-Computing/events/267050035/ - Denis
Re: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
Saikat, igniters I raised a blocker [1] for 2.8 related to recently implemented default query timeout [2]. Currently we have a buggy behavior for thin JDBC driver when a default timeout is configured. Actually I see 2 options to go: 1. Fix the issue with thin JDBC [1]. 2. Exclude a default query timeout support [2] from 2.8. Please share your thoughts! [1] https://issues.apache.org/jira/browse/IGNITE-12424 [2] https://issues.apache.org/jira/browse/IGNITE-7285 пт, 6 дек. 2019 г. в 19:55, Maxim Muzafarov : > > Slava, > > > Thanks for noticing. I think we can include both of them. > Do you need any help from my side? > > On Fri, 6 Dec 2019 at 14:02, Вячеслав Коптилин > wrote: > > > > Hello Maxim, > > > > I found two issues that should be included in the upcoming AI 2.8, I think. > > [1] https://issues.apache.org/jira/browse/IGNITE-12409 - Already done and > > can be cherry-picked into the release branch. > > [2] https://issues.apache.org/jira/browse/IGNITE-12419 - This one looks > > like a blocker for the release. Pull-request is available. > > > > Thanks, > > S. > > > > вт, 3 дек. 2019 г. в 15:30, Alexei Scherbakov > >: > > > > > No objections. > > > > > > вт, 3 дек. 2019 г. в 15:02, Maxim Muzafarov : > > > > > > > Alexey, > > > > > > > > > > > > Yet another issue [1] with corrupted B+Tree exception on the master > > > branch. > > > > My suggestion is to revert the IGNITE-11704 [3] issue from the master > > > > branch and rework the patch. > > > > > > > > Any objections? > > > > > > > > Configuration: > > > > [1] persistence = false > > > > [2] persistence = true > > > > > > > > class > > > > > > > org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: > > > > B+Tree is corrupted [pages(groupId, pageId)=[IgniteBiTuple > > > > [val1=1544803905, val2=844420635196573]], msg=Runtime failure on > > > > cursor iteration] > > > > at > > > > > > > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.corruptedTreeException(BPlusTree.java:5927) > > > > at > > > > > > > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.nextPage(BPlusTree.java:5438) > > > > at > > > > > > > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.next(BPlusTree.java:5661) > > > > at > > > > > > > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl$1.next(IgniteCacheOffheapManagerImpl.java:3020) > > > > at > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition$2.hasNextX(GridDhtLocalPartition.java:1226) > > > > at > > > > > > > org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45) > > > > at > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.doClear(GridDhtLocalPartition.java:1295) > > > > at > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.clearTombstones(GridDhtLocalPartition.java:1242) > > > > at > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.topology.PartitionsEvictManager$ClearTombstonesTask.run0(PartitionsEvictManager.java:673) > > > > at > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.topology.PartitionsEvictManager$AbstractEvictionTask.run(PartitionsEvictManager.java:587) > > > > at > > > > > > > org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7061) > > > > at > > > > > > > org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:827) > > > > at > > > > > > > org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) > > > > at > > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > > > at > > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > > > at java.lang.Thread.run(Thread.java:748) > > > > > > > > Caused by: java.lang.AssertionError: Key is not ready: > > > > CacheDataRowAdapter [key=null, val=null, expireTime=-1, ver=null, > > > > cacheId=0, link=0001003c77d8] > > > > at > > > > > > > org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.key(CacheDataRowAdapter.java:837) > > > > at > > > > > > > org.apache.ignite.internal.processors.cache.tree.CacheDataTree.compare(CacheDataTree.java:382) > > > > at > > > > > > > org.apache.ignite.internal.processors.cache.tree.CacheDataTree.compare(CacheDataTree.java:63) > > > > at > > > > > > > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.compare(BPlusTree.java:5200) > > > > at > > > > > > > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.findLowerBound(BPlusTree.java:5317) > > > > at > > > > > > >
[jira] [Created] (IGNITE-12424) Fix default query timeout behavior for thin JDBC
Ivan Pavlukhin created IGNITE-12424: --- Summary: Fix default query timeout behavior for thin JDBC Key: IGNITE-12424 URL: https://issues.apache.org/jira/browse/IGNITE-12424 Project: Ignite Issue Type: Bug Components: sql Reporter: Ivan Pavlukhin Fix For: 2.8 After IGNITE-7285 there appeared a buggy behavior for thin JDBC driver. Thin JDBC handles explicit query timeout on a client side. Default query timeout is tracked on a server side. As a server is not aware of explicit client timeout it is not possible to override a default timeout with longer explicit timeout (effectively a query will be cancelled after a default timeout expiration). The expected behavior is that an explicit query timeout always overrides a default one. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSS] dependencies and release process for Ignite Extensions
Saikat, I cannot access a TC project using your link. It says: "You do not have enough permissions to access project with internal id: project28" Could you please give an access or describe shortly how does a build pipeline look like? пн, 9 дек. 2019 г. в 06:40, Saikat Maitra : > > Hello, > > I am running into a problem specific to teamcity build for Ignite > Extensions project. When I set the dependencies to 2.9.0-SNAPSHOT I am > getting an error message during build as below > > [06:24:12][Step 4/5] Failed to execute goal on project ignite-flink-ext: > Could not resolve dependencies for project > org.apache.ignite.ext:ignite-flink-ext:jar:1.0.0-SNAPSHOT: The following > artifacts could not be resolved: > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT, > org.apache.ignite:ignite-core:jar:tests:2.9.0-SNAPSHOT, > org.apache.ignite:ignite-log4j:jar:2.9.0-SNAPSHOT, > org.apache.ignite:ignite-spring:jar:2.9.0-SNAPSHOT: Could not find artifact > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT in h2database.com ( > https://h2database.com/m2-repo) > > > and if set artifact dependencies for ~Build Apache Ignite~ then I receive > below error > > [ERROR] [ERROR] Could not find the selected project in the reactor: > :ignite-flink-ext @ > > Build url > https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteExtensions_Build_IgniteExtensions=pull%2F1%2Fhead=buildTypeStatusDiv > > Can you please let me know if you faced similar problem with teamcity build? > > I can set Ignite Extensions dependencies to released ignite-core artifacts > version like 2.7.6 and build works fine. > > Regards, > Saikat > > > > > > On Sat, Nov 30, 2019 at 1:50 PM Saikat Maitra > wrote: > > > Hello Denis, > > > > Thank you for your email and sharing your thoughts on the release process. > > I will update the artifact id and dependencies for ignite-extensions > > accordingly. > > > > I had created Ignite-Extensions project as a root level project and in > > teamcity I was facing issues pulling dependencies for 2.8.0-SNAPSHOT > > whereas I was able to pull dependencies for ignite-core 2.7.6 from maven > > central. I will look into it further why teamcity build was not able to > > pull snapshot dependencies. > > > > I will also create "ignite-core-2.9+" branch for the upcoming release > > process. > > > > Thank you, > > Saikat > > > > > > > > > > On Wed, Nov 27, 2019 at 1:05 PM Denis Magda wrote: > > > >> Hi Saikat, > >> > >> Thanks for driving this activity forward and raising the question. Let me > >> share my thoughts below and let's see what the broader community thinks. > >> > >> Each extension needs to have its own version unrelated to the core and > >> Maven's groupId parameter for extension artifacts should be > >> "org.apache.ignite.ext". For instance, the very first release of Flink in > >> the form of extension should be pulled from Maven this way > >> > >> > >> org.apache.ignite.ext > >> ignite-flink > >> 1.0.0 > >> > >> > >> When it comes to the releases, all the extensions need to be verified for > >> an upcoming release and updated if needed (with the version increase only > >> for those updated). Thus, looks like the extensions master needs to be > >> linked to the latest Ignite core snapshot. Whenever the core will be being > >> prepared and any extensions need to be modified we can take this approach: > >> > >>- Create a branch of extensions for the upcoming core release. For > >>instance, "ignite-core-2.9+" branch. That's just the branch name (and > >> not > >>any Maven artifact name) with "+" sign implying that the updated > >> extensions > >>will work for Ignite 2.9 and later until we need to update them again > >>creating a release branch like "ignite-core-2.14+" > >>- If only a subset of the extensions was updated then we need to > >> release > >>those extensions to Maven. The goal is to avoid the practice of > >> publishing > >>Flink or any other extension to Maven for every core release if there > >> are > >>no changes. > >>- As for a ZIP archive, we should prepare the archive for a download > >>with the name like "ignite-core-2.9+" > >> > >> > >> - > >> Denis > >> > >> > >> On Tue, Nov 26, 2019 at 9:03 PM Saikat Maitra > >> wrote: > >> > >> > Hello, > >> > > >> > I wanted to connect and discuss on the release process for > >> > ignite-extensions. As of today all our integrations since released > >> together > >> > were able to run build based on latest snapshot for example the current > >> > build depends on 2.8.0-SNAPSHOT. If we are making ignite-extensions as > >> > separate project with different release cycle then it make sense to have > >> > dependencies on core modules based on released artifact for example the > >> > dependency for ignite-core would be 2.7.6 > >> > > >> > Please review and share your thoughts. > >> > > >> > PR https://github.com/apache/ignite-extensions/pull/1 > >> > > >> > Regards > >> > Saikat > >> >
