[jira] [Created] (IGNITE-7698) Page read during replacement should be outside of segment write lock

Alexey Goncharuk created IGNITE-7698: Summary: Page read during replacement should be outside of segment write lock Key: IGNITE-7698 URL: https://issues.apache.org/jira/browse/IGNITE-7698

[jira] [Created] (IGNITE-7697) Update maven-javadoc-plugin version

Peter Ivanov created IGNITE-7697: Summary: Update maven-javadoc-plugin version Key: IGNITE-7697 URL: https://issues.apache.org/jira/browse/IGNITE-7697 Project: Ignite Issue Type: Improvement

[jira] [Created] (IGNITE-7696) Deadlock at GridDhtAtomicCache.lockEntries called through GridDhtAtomicCache.updateAllAsyncInternal

Sadayuki Furuhashi created IGNITE-7696: -- Summary: Deadlock at GridDhtAtomicCache.lockEntries called through GridDhtAtomicCache.updateAllAsyncInternal Key: IGNITE-7696 URL:

Re: API to enlist running user tasks

Dmitriy I thought about those type of tasks that require custom user code to be executed inside Ignite. Your suggestion make sense for me also. Let's include SQL queries. Should we include ScanQuery, TextQuery to this API? В Вт, 13/02/2018 в 13:27 -0800, Dmitriy Setrakyan пишет: > Nikolay, how

Re: API to enlist running user tasks

Hello, Denis. > Or is it just a separate task? Yes, it is a separate task. I propose to have a different method for each type of long running user task: * continuous queries * compute tasks * compute jobs В Вт, 13/02/2018 в 17:23 -0800, Denis Magda пишет: > Sounds like a right addition to our

[GitHub] ignite pull request #3519: IGNITE-7693: Printing out session ids on joining ...

GitHub user shroman opened a pull request: https://github.com/apache/ignite/pull/3519 IGNITE-7693: Printing out session ids on joining via ZookeeperDiscove… …rySpi. You can merge this pull request into a Git repository by running: $ git pull

Re: Write ahead log and early eviction of new elements

My guess before the entry (element) gets evicted it will be synced to a respective partition file on disk forcibly so that you can grab it from there later. Ignite persistence experts please confirm my understanding. — Denis > On Feb 13, 2018, at 1:28 PM, Raymond Wilson

Re: API to enlist running user tasks

Sounds like a right addition to our metrics APIs. However, I’m not sure how this part is related to continuous queries: > * List of running compute tasks for node: >* node ID task started from. >* task class name. >* other task parameters. >* creation timestamp (?) >*

[GitHub] ignite pull request #3517: IGNITE-7588 Deprecate CacheLocalStore annotation

Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3517 ---

Write ahead log and early eviction of new elements

I have a scenario I would like to validate when using Ignite Persistence. I understand when I add an element to a cache that element is serialized, placed into the local memory for the cache on that server and then placed into the WAL pending checkpointing (merging into the persistence store).

Re: WAL Archive Issue

I see, it seems subgoal 'gain predictable size' can be achieved with following options: - https://issues.apache.org/jira/browse/IGNITE-6552 implementation (in variant of '...WAL history size in time units and maximum size in GBytes', - here we probably should change description or create 2nd

Re: WAL Archive Issue

As far as I understand, the idea is WAL archive with predictable size ("N checkpoints" is not predictable size), which can be safely removed (e.g. if free disk space is urgently needed) without losing crash recovery. No-archiver mode makes sense as well - it should be faster than current mode

Re: WAL Archive Issue

Hi, I didn't get the point why it may be required to separate WAL work, WAL uncheckpointed archive (some work outside segment rotation) and checkpointed archive (which is better to be compressed using Ignite new feature - WAL compressor). Please consider new no-archiver mode implemented recently.

[GitHub] ignite pull request #3518: IGNITE-7690 Move shared memory suite to Ignite Ba...

GitHub user glukos opened a pull request: https://github.com/apache/ignite/pull/3518 IGNITE-7690 Move shared memory suite to Ignite Basic 2 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-7690

Re: [SparkDataFrame] Query Optimization. Prototype

Nikolay, Non-collocated joins should be used only if there is no way to collocate. Please read here for more info: https://apacheignite-sql.readme.io/docs/ distributed-joins As for limitations, I think Vladimir is talking more about syntax related stuff, i.e. what we do or don't support from SQL

Re: WAL Archive Issue

I think, I got the point now. There's no need to copy files from "temp" to "archive" dir - we can just move them, which is a constant-time operation. Makes sense. Change is quite complex (we need to synchronize all movings thoroughly to avoid ruining existing WAL read iterators), but

Re: [SparkDataFrame] Query Optimization. Prototype

Valentin, > Looks like this is because you enabled non-collocated joins But non-collocated joins is only way to be sure that join returns correct results. So in my case it's OK to enable them. Am I right? > do we have this documented somewhere? I'm asked that in previuous mail. Vladimir

Re: WAL Archive Issue

Yakov, This will work. However, I expect performance degradation with this change. Disk storage has a limited number of I/O operations per second on hardware level. List of already existing disk I/O activities (writing to WAL work dir, copying from WAL work dir to WAL archive dir, writing

Re: Should we annotate @CacheLocalStore as @Depricated?

Valentin, thank you for replying. The task [1][2] is ready for review. Please have a look. [1] https://issues.apache.org/jira/browse/IGNITE-5097 [2] https://github.com/apache/ignite/pull/3517/files On Tue, Feb 13, 2018 at 9:23 PM, Valentin Kulichenko wrote: >

[GitHub] ignite pull request #3517: IGNITE-7588 Deprecate CacheLocalStore annotation

GitHub user daradurvs opened a pull request: https://github.com/apache/ignite/pull/3517 IGNITE-7588 Deprecate CacheLocalStore annotation You can merge this pull request into a Git repository by running: $ git pull https://github.com/daradurvs/ignite ignite-7588 Alternatively

Re: WAL Archive Issue

Ivan, I do not want to create new files. As far as I know, now we copy segments to archive dir before they get checkpointed. What I suggest is to copy them to a temp dir under wal directory and then move to archive. In my understanding at the time we copy the files to a temp folder all changes to

Re: WAL Archive Issue

Yakov, I see the only one problem with your suggestion - number of "uncheckpointed" segments is potentially unlimited. Right now we have limited number (10) of file segments with immutable names in WAL "work" directory. We have to keep this approach due to known bug in XFS - fsync time is

Re: Should we annotate @CacheLocalStore as @Depricated?

Vyacheslav, These are test classes, there is not reason to put deprecation on them. We need to deprecate anything that is part of public API (in this case I believe it's only this annotation, nothing else). -Val On Tue, Feb 13, 2018 at 7:09 AM, Vyacheslav Daradur wrote: >

Re: WAL Archive Issue

I meant we still will be copying segment once and then will be moving it to archive which should not affect file system much. Thoughts? --Yakov 2018-02-13 21:19 GMT+03:00 Yakov Zhdanov : > Alex, > > I remember we had some confusing behavior for WAL archive when archived >

WAL Archive Issue

Alex, I remember we had some confusing behavior for WAL archive when archived segments were required for successful recovery. Is issue still present? If yes, what if we copy "uncheckpointed" segments to a directory under wal directory and then move the segments to archive after checkpoint? Will

Re: [SparkDataFrame] Query Optimization. Prototype

Nikolay, Looks like this is because you enabled non-collocated joins. I was not aware of this limitation though, do we have this documented somewhere? -Val On Tue, Feb 13, 2018 at 8:21 AM, Nikolay Izhikov wrote: > Val, > > Source code check:

Re: Ignite Teamcity email notifications

Hi Igniters, Thanks to Peter Ivanov and all involved into this activity. We've enabled first email notification rule for a subset of the per-commit tests, Ignite Run All Basic (suites: license & basic & compute and PDS unit test; branch = master). Email is sent only if failed build contains your

[GitHub] ignite pull request #3516: IGNITE-7695: Enable Ignite Update Notifier tests

Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3516 ---

Re: Looks like a bug in ServerImpl.joinTopology()

Alex, you can alter ServerImpl and insert a latch or thread.sleep(xxx) anywhere you like to show the incorrect behavior you describe. --Yakov

[GitHub] ignite pull request #3516: IGNITE-7695: Enable Ignite Update Notifier tests

GitHub user dspavlov opened a pull request: https://github.com/apache/ignite/pull/3516 IGNITE-7695: Enable Ignite Update Notifier tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-7695

[jira] [Created] (IGNITE-7695) Enable Ignite Update Notifier tests

Dmitriy Pavlov created IGNITE-7695: -- Summary: Enable Ignite Update Notifier tests Key: IGNITE-7695 URL: https://issues.apache.org/jira/browse/IGNITE-7695 Project: Ignite Issue Type: Task

[GitHub] ignite pull request #3515: IGNITE-7640: make DiscoveryDataClusterState immut...

GitHub user SharplEr opened a pull request: https://github.com/apache/ignite/pull/3515 IGNITE-7640: make DiscoveryDataClusterState immutable for CI testing You can merge this pull request into a Git repository by running: $ git pull https://github.com/SharplEr/ignite

[jira] [Created] (IGNITE-7694) testActiveClientReconnectToInactiveCluster hangs because of an assertion

Alexey Goncharuk created IGNITE-7694: Summary: testActiveClientReconnectToInactiveCluster hangs because of an assertion Key: IGNITE-7694 URL: https://issues.apache.org/jira/browse/IGNITE-7694

Re: [SparkDataFrame] Query Optimization. Prototype

Val, Source code check: https://github.com/apache/ignite/blob/master/modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/GridH2CollocationModel.java#L382 Stack trace: javax.cache.CacheException: Failed to prepare distributed join query: join condition does not

[jira] [Created] (IGNITE-7693) New node joining via ZookeeperDiscoverySpi should print out its ZooKeeper sessionId

Sergey Chugunov created IGNITE-7693: --- Summary: New node joining via ZookeeperDiscoverySpi should print out its ZooKeeper sessionId Key: IGNITE-7693 URL: https://issues.apache.org/jira/browse/IGNITE-7693

Re: [SparkDataFrame] Query Optimization. Prototype

Nikolay, This doesn't make sense to me. Not having an index should not cause query to fail. What is the exception? -Val On Tue, Feb 13, 2018 at 8:07 AM, Nikolay Izhikov wrote: > Hello, Valentin. > > > When you're talking about join optimization, what exactly are you >

Re: [SparkDataFrame] Query Optimization. Prototype

Sounds good. Let me know when you feel it's ready and I'll take a look. -Val On Tue, Feb 13, 2018 at 7:56 AM, Nikolay Izhikov wrote: > Hello, Valentin. > > > Since other parts of data frames integration are already merged, > > I think it's a good time to resurrect this

Re: [SparkDataFrame] Query Optimization. Prototype

Hello, Valentin. > Since other parts of data frames integration are already merged, > I think it's a good time to resurrect this thread? > Does it make sense to review it right now? > Or you want to make some more changes? I've already merged PR [1] with current master. So you can review it,

[jira] [Created] (IGNITE-7692) affinityCall and affinityRun may execute code on backup partitions

Alexey Goncharuk created IGNITE-7692: Summary: affinityCall and affinityRun may execute code on backup partitions Key: IGNITE-7692 URL: https://issues.apache.org/jira/browse/IGNITE-7692 Project:

Re: [SparkDataFrame] Query Optimization. Prototype

Hello, Vladimir. I've created ticket https://issues.apache.org/jira/browse/IGNITE-7691 В Пн, 12/02/2018 в 11:20 +0300, Vladimir Ozerov пишет: > Hi Nikolay, > > I am not sure if ticket for DECIMAL column metadata exists. If you haven't > find one under "sql" component, please feel free to

[jira] [Created] (IGNITE-7691) Provide info about DECIMAL column scale and precision

Nikolay Izhikov created IGNITE-7691: --- Summary: Provide info about DECIMAL column scale and precision Key: IGNITE-7691 URL: https://issues.apache.org/jira/browse/IGNITE-7691 Project: Ignite

Re: IGNITE-5714 Context switching for pessimistic transactions

Hello, Alexey. Could you please, write little more about your implementation 1. Design of implementation. 2. Advantages of you implementation in compare with other ways? 3. Transactions performance penalties/improvements? В Вт, 13/02/2018 в 14:17 +, ALEKSEY KUZNETSOV пишет: > Hi,

[jira] [Created] (IGNITE-7690) Move shared memory suite (IpcSharedMemoryCrashDetectionSelfTest) to Ignite Basic 2

Dmitriy Pavlov created IGNITE-7690: -- Summary: Move shared memory suite (IpcSharedMemoryCrashDetectionSelfTest) to Ignite Basic 2 Key: IGNITE-7690 URL: https://issues.apache.org/jira/browse/IGNITE-7690

[GitHub] ignite pull request #3514: IGNITE-7481 suspended tx timeout rollback fix

GitHub user voipp opened a pull request: https://github.com/apache/ignite/pull/3514 IGNITE-7481 suspended tx timeout rollback fix You can merge this pull request into a Git repository by running: $ git pull https://github.com/voipp/ignite IGNITE-7481 Alternatively you can

Re: Page Locking vs Entry-level Locking

Hi John, Entry level locks still exist. I've described locks coming from page memory, page memory and its locks is lower-level abstraction than entries level. Segment is part of region consist from number of pages. 1 region - * segments, 1 segment - * pages. Segment lock is performed when set

API to enlist running user tasks

Hello, Igniters. We have some requests from users [1] to have ability to get list of all running continuous queries. I propose to implement such ability. To implement it we have to extend our JMX beans to provide the following information: * List of continuous queries for cache: *

[jira] [Created] (IGNITE-7689) IgnitePdsBinaryMetadataOnClusterRestartTest flaky fails on TC

Alexey Goncharuk created IGNITE-7689: Summary: IgnitePdsBinaryMetadataOnClusterRestartTest flaky fails on TC Key: IGNITE-7689 URL: https://issues.apache.org/jira/browse/IGNITE-7689 Project:

[jira] [Created] (IGNITE-7688) DDL does not working properly on sql queries.

Muratcan TUKSAL created IGNITE-7688: --- Summary: DDL does not working properly on sql queries. Key: IGNITE-7688 URL: https://issues.apache.org/jira/browse/IGNITE-7688 Project: Ignite Issue

IGNITE-5714 Context switching for pessimistic transactions

Hi, Igniters! Currently we have context switching implemented for optimistic transactions [1]. Goal of the current ticket is to support transaction suspend()\resume() operations for pessimistic transactions. The essential problem with them lies in *IgniteTxAdapter#threadId*. Thread id is

Re: Optimistic Locking and the Prepare Phase

Hi Vladimir, Your answer is what is depicted in the graphics and makes perfect sense to me. I guess what I'm confused about is what a "prepare" phase means and what "*In optimistic transactions, locks are acquired on primary nodes during the "prepare" phase* " means. My understanding of a

Re: Orphaned, duplicate, and main-class tests!

Anton, >Tests should be attached to appropriate suites This I can do > and muted if necessary, Issues should be created on each mute. This is roughly a week of work. I can't spare that right now. I doubt anyone can. Can we approach this by smaller steps? -- Ilya Kasnacheev 2018-02-06 19:55

[jira] [Created] (IGNITE-7687) SQL SELECT doesn't update TTL for Touched/AccessedExpiryPolicy

Stanislav Lukyanov created IGNITE-7687: -- Summary: SQL SELECT doesn't update TTL for Touched/AccessedExpiryPolicy Key: IGNITE-7687 URL: https://issues.apache.org/jira/browse/IGNITE-7687 Project:

Re: Page Locking vs Entry-level Locking

Hi Pavlov, Thanks for your explanation. However, I still have these questions: 1. If locking is per page-level and there are no entry-level locks, then why does the documentation here talk about having entry-level transaction locks? https://apacheignite.readme.io/docs/transactions

Looks like a bug in ServerImpl.joinTopology()

Hello. I saw such code in `ServerImpl.joinTopology()` locNode.order(1); locNode.internalOrder(1); spi.gridStartTime = U.currentTimeMillis(); locNode.visible(true); ring.clear(); ring.topologyVersion(1); And it looks like a bug because the `locNode` is contained inside the `ring`

[jira] [Created] (IGNITE-7686) PDS Direct IO failue: IgnitePdsEvictionTest.testPageEviction

Dmitriy Pavlov created IGNITE-7686: -- Summary: PDS Direct IO failue: IgnitePdsEvictionTest.testPageEviction Key: IGNITE-7686 URL: https://issues.apache.org/jira/browse/IGNITE-7686 Project: Ignite

[jira] [Created] (IGNITE-7685) Incorrect AllocationRate counting

Anton Vinogradov created IGNITE-7685: Summary: Incorrect AllocationRate counting Key: IGNITE-7685 URL: https://issues.apache.org/jira/browse/IGNITE-7685 Project: Ignite Issue Type: Task

[jira] [Created] (IGNITE-7684) Ignore IGNITE_USE_ASYNC_FILE_IO_FACTORY in FileWriteAheadLogManager

Alexander Belyak created IGNITE-7684: Summary: Ignore IGNITE_USE_ASYNC_FILE_IO_FACTORY in FileWriteAheadLogManager Key: IGNITE-7684 URL: https://issues.apache.org/jira/browse/IGNITE-7684 Project:

Re: Make Teamcity Green Again

Forgot to share link to actual investigations list https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=investigations Please feel free to create and assign investigations if you have assumption about causes of failure. вт, 13 февр. 2018 г. в 1:34, Dmitriy Setrakyan

IGNITE-7409 Rework exception handling in suspend()\resume() methods

Hi, Igntrs! When user misuses transactions, one could get assertion error. Currently, when we start optimistic transaction, do some work within it and try to resume it(which is incorrect behavior) assertion error is thrown. It looks weird. On the contrary more descriptive exception should be

Re: Page Locking vs Entry-level Locking

Hi John, 1. No, content modification require lock holding to page to provide consitency in multithreaded environment. 2. Page is locked for read before reading its content, and unlocked after. Same for lock for write for writting. 1 writer or N readers allowed for page. On unlock write lock,

[GitHub] ignite pull request #3498: IGNITE-3111 .NET: Configure SSL without Spring

Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3498 ---

[jira] [Created] (IGNITE-7683) ContinuousQueryWithTransformer needs to be documented

Nikolay Izhikov created IGNITE-7683: --- Summary: ContinuousQueryWithTransformer needs to be documented Key: IGNITE-7683 URL: https://issues.apache.org/jira/browse/IGNITE-7683 Project: Ignite

[GitHub] ignite pull request #3513: Ignite-2.4.2-p4

GitHub user DmitriyGovorukhin opened a pull request: https://github.com/apache/ignite/pull/3513 Ignite-2.4.2-p4 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-2.4.2-p4 Alternatively you can review

Re: Optimistic Locking and the Prepare Phase

Hi John, 1) In PESSIMISTIC mode locks are obtained either on first update (READ_COMMITTED) or even read (REPEATABLE_READ). I.e. they obtained before prepare phase and are held for the duration of transaction. In OPTIMISTIC mode locks are obtained only after you call IgniteTransaction.commit(). 2)

[jira] [Created] (IGNITE-7682) LocalSize cache functions on C++

Roman Bastanov created IGNITE-7682: -- Summary: LocalSize cache functions on C++ Key: IGNITE-7682 URL: https://issues.apache.org/jira/browse/IGNITE-7682 Project: Ignite Issue Type: Bug