Re: Reconsider default WAL mode: we need something between LOG_ONLY and FSYNC

Same question. It would be very difficult to explain these two modes to users. We should do our best to fix LOG_ONLY first. Without these guarantees there is no reason to keep LOG_ONLY at all, user could simply use BACKGROUND with high flush frequency. This is precisely how Cassandra works. p.1 -

Re: Reconsider default WAL mode: we need something between LOG_ONLY and FSYNC

Folks, I do not expect any performance degradation here for high load becase we already do fsync on rollover. So extra fsyncs will be almost free. We should do this fsync without holding CP lock , of course. (see also point 3: 3) We do perform fsync on rollover (switch of current WAL segment) in

Re: Reconsider default WAL mode: we need something between LOG_ONLY and FSYNC

Vladimir, Unlike BACKGROUND, LOG_ONLY provides strict write guarantees unless power loss has happened. Seems like we need to measure performance difference to decide whether do we need separate WAL mode. If it will be invisible, we'll just fix these bugs without introducing new mode; if it

Re: Reconsider default WAL mode: we need something between LOG_ONLY and FSYNC

It really depends on hardware and workload pattern. I expect that LOG_ONLY_SAFE will be either equal to LOG_ONLY or a few percent slower. We'll answer this question for sure after implementation of three fixes and benchmarking. Let's first of all get understanding whether extra durability

Re: Reconsider default WAL mode: we need something between LOG_ONLY and FSYNC

On Fri, Mar 16, 2018 at 12:55 AM, Ivan Rakov wrote: > Vladimir, > > Unlike BACKGROUND, LOG_ONLY provides strict write guarantees unless power > loss has happened. > Seems like we need to measure performance difference to decide whether do > we need separate WAL mode. If it

Re: Reconsider default WAL mode: we need something between LOG_ONLY and FSYNC

Ticket to track changes: https://issues.apache.org/jira/browse/IGNITE-7754 Best Regards, Ivan Rakov On 16.03.2018 10:58, Dmitriy Setrakyan wrote: On Fri, Mar 16, 2018 at 12:55 AM, Ivan Rakov wrote: Vladimir, Unlike BACKGROUND, LOG_ONLY provides strict write

Re: Timeline for support of compute functions by thin clients

> for what Literally no one wants to have JVM in their process and additional dependencies :) As much APIs as possible should be available in thin mode, that is the point. This thread is started by our user, after all :) On Thu, Mar 15, 2018 at 10:25 PM, Denis Magda wrote: >

Re: Timeline for support of compute functions by thin clients

Denis, >From client perspective any compute task is also request - response. This doesn't distinguish compute from any other API anyhow. There are no problem to add closures, tasks, services, etc.. What is really difficult is components requiring non-trivial thread interaction and complex request

[jira] [Created] (IGNITE-7971) SQL: make sure CREATE INDEX doesn't corrupt the cluster

Vladimir Ozerov created IGNITE-7971: --- Summary: SQL: make sure CREATE INDEX doesn't corrupt the cluster Key: IGNITE-7971 URL: https://issues.apache.org/jira/browse/IGNITE-7971 Project: Ignite

[jira] [Created] (IGNITE-7972) NPE in TTL manager.

Andrew Mashenkov created IGNITE-7972: Summary: NPE in TTL manager. Key: IGNITE-7972 URL: https://issues.apache.org/jira/browse/IGNITE-7972 Project: Ignite Issue Type: Bug

[GitHub] ignite pull request #3645: IGNITE-7964 rmvId is stored to MetaStorage metapa...

GitHub user sergey-chugunov-1985 opened a pull request: https://github.com/apache/ignite/pull/3645 IGNITE-7964 rmvId is stored to MetaStorage metapage during operations You can merge this pull request into a Git repository by running: $ git pull

Re: Test testJobIdCollision to use multiple JVMs [IGNITE-4706]

Dmitry, Maxim, Thanks for bringing this up. I reviewed all context about [1], it looks like the test is still valid but is of low priority; I reflected it in jira ticket itself. Also rewriting the test to multi-JVM fashion isn't an easy task, to me it is much better to spend this time working

[GitHub] ignite pull request #3651: IGNITE-7963 Opportunistically flush DataStreamer ...

GitHub user alamar opened a pull request: https://github.com/apache/ignite/pull/3651 IGNITE-7963 Opportunistically flush DataStreamer instead of endless w… …ait. You can merge this pull request into a Git repository by running: $ git pull

Re: Partition eviction failed, this can cause grid hang. (Caused by: java.lang.IllegalStateException: Failed to get page IO instance (page content is corrupted))

Hi Dmitry. Thanks for you attention to this issue. I changed repository to jcenter and set Ignite version to 2.4. Unfortunately the reproducer starts with the same error message in the log (see attached). I cannot say whether behavior of the whole cluster will change on 2.4, I mean if the

Re: (Partition Map) Exchange at wiki

Hi Dmitriy, That's a great article. Future and current contributors/committer who will be dealing with the part of the system, described by you, will be thankful for that page. Please clarify one thing for me. In your rebalancing example, you are saying that the full-map exchange won't happen

[jira] [Created] (IGNITE-7978) Recovery from WAL may result in JVM crash

Denis Mekhanikov created IGNITE-7978: Summary: Recovery from WAL may result in JVM crash Key: IGNITE-7978 URL: https://issues.apache.org/jira/browse/IGNITE-7978 Project: Ignite Issue

[GitHub] ignite pull request #3592: IGNITE-7862: flatten plugin updated to version 1....

Github user nizhikov closed the pull request at: https://github.com/apache/ignite/pull/3592 ---

Re: Maven. Issues with flatten plugin

Hello, guys. We finally updated flatten plugin in master. Petr Ivanov, Alex Volkov - thank you very much! В Пт, 02/03/2018 в 16:45 +0300, Petr Ivanov пишет: > Updated all maven definitions I’ve found in templates of test project. > Please, try once more. > > > > > On 2 Mar 2018, at 16:36,

Re: New test failures in .NET

I did this TC verification before merge. Have no idea why I missed it. пт, 16 мар. 2018 г. в 16:06, Pavel Tupitsyn : > Hi, > > The problem is introduced by our freshman committer Nikolay Izhikov in > "IGNITE-7756: IgniteUuid added to predefined types" [1] > > IgniteUuid

Re: New test failures in .NET

Suite passes now, https://ci.ignite.apache.org/viewLog.html?buildId=1140071=buildResultsDiv=IgniteTests24Java8_IgnitePlatformNet Pavel, thank you! пт, 16 мар. 2018 г. в 16:13, Nikolay Izhikov : > Hello, Pavel. > > Sorry, my bad. > Thanks for fixing. > > Will double check

Re: Test testJobIdCollision to use multiple JVMs [IGNITE-4706]

Hi Sergey, Thank you for stepping in. There is fresh test failures scope reflected in JIRA

[GitHub] ignite pull request #3645: IGNITE-7964 rmvId is stored to MetaStorage metapa...

Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3645 ---

Re: [ANNOUNCE] Apache Ignite 2.4.0 Released: Machine Learning GA and Spark DataFrames

What is the project about? Why should I be interested in it? [rhetorical questions] The Announce emails are sent to people not on the developer or user lists. Most will have no idea what the project is about. So the e-mails should contain at least brief details of what the product does, and some

Re: [ANNOUNCE] Apache Ignite 2.4.0 Released: Machine Learning GA and Spark DataFrames

All our previous announcements were formatted precisely the way you suggest. However, I haven't fount that template effective. Personally, I archive an email immediately if see it's written the standard way and I know nothing about the product. That's why I decided to experiment targeting those

Re: New test failures in .NET

Hello, Pavel. Sorry, my bad. Thanks for fixing. Will double check tests result before next commit. В Пт, 16/03/2018 в 16:06 +0300, Pavel Tupitsyn пишет: > Hi, > > The problem is introduced by our freshman committer Nikolay Izhikov in > "IGNITE-7756: IgniteUuid added to predefined types" [1] >

[GitHub] ignite pull request #3627: IGNITE-7932: Add example for Linear SVM with Iris...

Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3627 ---

Re: IEP-14: Ignite failures handling (Discussion)

Hi! Thank you all for your opinions and ideas! While reading the thread I made two important conclusions: 1. Proposed API should be changed because possible actions enumeration is bad idea. More clean and simple design should allow user provide failure handler implementation with custom logic

[jira] [Created] (IGNITE-7975) SQL TX: allow batch inserts

Vladimir Ozerov created IGNITE-7975: --- Summary: SQL TX: allow batch inserts Key: IGNITE-7975 URL: https://issues.apache.org/jira/browse/IGNITE-7975 Project: Ignite Issue Type: Task

[GitHub] ignite pull request #3505: Ignite 2.3.3

Github user alamar closed the pull request at: https://github.com/apache/ignite/pull/3505 ---

[GitHub] ignite pull request #3406: Ignite 2.1.11

Github user alamar closed the pull request at: https://github.com/apache/ignite/pull/3406 ---

[GitHub] ignite pull request #3452: Ignite 2.3.2.b1

Github user alamar closed the pull request at: https://github.com/apache/ignite/pull/3452 ---

Re: MTCGA: IGNITE-7791 and GridDhtPartitionsSingleMessage

Hi Maxim, I didn't know answer, so I decided to provide at least general intro information. I hope it would be useful for you and for newcomers. https://cwiki.apache.org/confluence/display/IGNITE/%28Partition+Map%29+Exchange+-+under+the+hood Sincerely, Dmitriy Pavlov вт, 13 мар. 2018 г. в

[GitHub] ignite pull request #3648: IGNITE-7863: Spark dependencies cleaned up.

GitHub user nizhikov opened a pull request: https://github.com/apache/ignite/pull/3648 IGNITE-7863: Spark dependencies cleaned up. You can merge this pull request into a Git repository by running: $ git pull https://github.com/nizhikov/ignite IGNITE-7863 Alternatively you

Re: New test failures in .NET

Hi, The problem is introduced by our freshman committer Nikolay Izhikov in "IGNITE-7756: IgniteUuid added to predefined types" [1] IgniteUuid type id has changed on Java side, but not in .NET. I have pushed the fix to master branch. I can only ask everyone again to have some respect for your

[GitHub] ignite pull request #3602: IGNITE-7881 Tests for using TreeMap or TreeSet as...

Github user alamar closed the pull request at: https://github.com/apache/ignite/pull/3602 ---

Re: Ignite-7640 Refactor DiscoveryDataClusterState to be immutable (Done)

Hi, It seems run all here was outdated, triggered one more run https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8_IgniteTests24Java8=pull%2F3515%2Fhead пн, 26 февр. 2018 г. в 13:22, Александр Меньшиков : > Hi to all. > > I have done issue ignite-7640.

[GitHub] ignite pull request #3561: Ignite 1.7 master

Github user alamar closed the pull request at: https://github.com/apache/ignite/pull/3561 ---

[GitHub] ignite pull request #3564: Ignite gg 13518

Github user alamar closed the pull request at: https://github.com/apache/ignite/pull/3564 ---

Re: Partition eviction failed, this can cause grid hang. (Caused by: java.lang.IllegalStateException: Failed to get page IO instance (page content is corrupted))

Hi Arseny, I've observed in reproducer ignite_version=2.3.0 Could you check if it is reproducible in our freshest release 2.4.0. I'm not sure about ticket number, but it is quite possible issue is already fixed. Sincerely, Dmitriy Pavlov чт, 15 мар. 2018 г. в 19:34, Dmitry Pavlov

[GitHub] ignite pull request #3649: IGNITE-7962 Avoid swallowing unexpected exception...

GitHub user alamar opened a pull request: https://github.com/apache/ignite/pull/3649 IGNITE-7962 Avoid swallowing unexpected exceptions. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-7962

Re: Test testJobIdCollision to use multiple JVMs [IGNITE-4706]

Hi Sergey, Is this issue still actual for you? Sincerely, Dmitriy Pavlov пн, 26 февр. 2018 г. в 13:40, Maxim Muzafarov : > Hi all, > > I'm triyng to clarify for myseft issue [1] of rewriting this test case to > use multiple JVMs. I'm trying to reproduce it using steps

(Partition Map) Exchange at wiki

Hi Igniters, According coming questions here at dev.list, I decided to arrange my entries about (Partition Map) Exchange as wiki article. Result is third page in the series 'under the hood': https://cwiki.apache.org/confluence/display/IGNITE/%28Partition+Map%29+Exchange+-+under+the+hood I would

[jira] [Created] (IGNITE-7976) [Test failed] IgnitePersistentStoreCacheGroupsTest.testClusterRestartCachesWithH2Indexes fails on TC

Aleksey Plekhanov created IGNITE-7976: - Summary: [Test failed] IgnitePersistentStoreCacheGroupsTest.testClusterRestartCachesWithH2Indexes fails on TC Key: IGNITE-7976 URL:

[GitHub] ignite pull request #3650: IGNITE-7976 Normalize query entites when dynamic ...

GitHub user alex-plekhanov opened a pull request: https://github.com/apache/ignite/pull/3650 IGNITE-7976 Normalize query entites when dynamic start cache by store… …d cache data You can merge this pull request into a Git repository by running: $ git pull

[GitHub] ignite pull request #3563: Ignite gg 13518

Github user alamar closed the pull request at: https://github.com/apache/ignite/pull/3563 ---

Re: IgniteSet implementation: changes required

Dmitry, your way allows to reuse existing {{Ignite.set()}} API to create both set flavors. We can adopt it unless somebody in the community objects. Personally, I like {{IgniteCache.asSet()}} approach proposed by Vladimir O. more, since it emphasizes the difference between sets being created, but

[jira] [Created] (IGNITE-7977) Download page mirror choice is buried

Sebb created IGNITE-7977: Summary: Download page mirror choice is buried Key: IGNITE-7977 URL: https://issues.apache.org/jira/browse/IGNITE-7977 Project: Ignite Issue Type: Improvement

Re: Nodes which started in separate JVM couldn't stop properly (in tests)

Hello, Guys. I'm reviewed changes and it looks good to me. There is a simple reproducer for a bug in test framework, see below. It fails in master and works in branch. I'm planning to merge the fix [1] if Run All will be OK. Please, write to me if you have any objections. [1]

[GitHub] ignite pull request #3647: Ignite 2.3.4

GitHub user devozerov opened a pull request: https://github.com/apache/ignite/pull/3647 Ignite 2.3.4 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-2.3.4 Alternatively you can review and apply

Re: Maven. Issues with flatten plugin

Folks, thank you! I hope now we can now avoid transitive dependencies enlisting in each module. It will remove odd work from test development. пт, 16 мар. 2018 г. в 15:48, Nikolay Izhikov : > Hello, guys. > > We finally updated flatten plugin in master. > > Petr Ivanov,

[jira] [Created] (IGNITE-7974) Authentication: SQL command to show users

Vladimir Ozerov created IGNITE-7974: --- Summary: Authentication: SQL command to show users Key: IGNITE-7974 URL: https://issues.apache.org/jira/browse/IGNITE-7974 Project: Ignite Issue Type:

Re: Abandoned Patch Available JIRA tickets

Hi Igniters, I want to raise up this thread again. If your ticket seems to be stuck in review process, please write. I will try to help. Can't promise it would be fast, but I hope together we can find a solution for each particular case. Sincerely, Dmitriy Pavlov чт, 8 февр. 2018 г. в 2:08,

Re: [ANNOUNCE] Apache Ignite 2.4.0 Released: Machine Learning GA and Spark DataFrames

Perhaps have a look at the announce mails sent by Httpd and Tomcat. Even though these projects are better known than most, they still provide a short summary of what they do. I don't think they read as though written by robots... On 16 March 2018 at 17:02, Denis Magda wrote:

Re: Deploying 2.4 artifacts to maven repo

Hmm, MvnRepository has not picked up the changes yet. It's strange. Filed a ticket: https://issues.apache.org/jira/browse/INFRA-16198 -- Denis On Wed, Mar 14, 2018 at 5:34 AM, aaksenov wrote: > it was uploaded to maven central no 05.03 and it seems it was most

Re: Partition eviction failed, this can cause grid hang. (Caused by: java.lang.IllegalStateException: Failed to get page IO instance (page content is corrupted))

Hi, We also got exact same error. Ours is setup without kubernetes. We are using ignite data streamer to put data into caches. After streaming aroung 500k records streamer failed with exception mentioned in original email. Thanks, Gaurav On 16-Mar-2018 4:44 PM, "Arseny Kovalchuk"

Re: [ANNOUNCE] Apache Ignite 2.4.0 Released: Machine Learning GA and Spark DataFrames

Thanks for the pointers, will check them up. Have a good weekend, Denis On Fri, Mar 16, 2018 at 11:16 AM, sebb wrote: > Perhaps have a look at the announce mails sent by Httpd and Tomcat. > > Even though these projects are better known than most, they still > provide a short

Re: [ANNOUNCE] Apache Ignite 2.4.0 Released: Machine Learning GA and Spark DataFrames

Denis, The brief pitch we provide on the home page should be good enough, no? Apache Ignite™ is a memory-centric distributed database, caching, and > processing platform for > transactional, analytical, and streaming workloads, delivering in-memory > speeds at petabyte scale D. On Fri, Mar

Re: [ANNOUNCE] Apache Ignite 2.4.0 Released: Machine Learning GA and Spark DataFrames

Absolutely. The concern here was that I didn't provide the necessary description in general. -- Denis On Fri, Mar 16, 2018 at 3:51 PM, Dmitriy Setrakyan wrote: > Denis, > > The brief pitch we provide on the home page should be good enough, no? > > Apache Ignite™ is a

Re: IgniteSet implementation: changes required

On Fri, Mar 16, 2018 at 7:39 AM, Andrey Kuznetsov wrote: > Dmitry, your way allows to reuse existing {{Ignite.set()}} API to create > both set flavors. We can adopt it unless somebody in the community objects. > Personally, I like {{IgniteCache.asSet()}} approach proposed by

Re: IEP-14: Ignite failures handling (Discussion)

Thanks Andrey! I have added a few comments to the IEP-14 page. D. On Fri, Mar 16, 2018 at 6:44 AM, Andrey Gura wrote: > Hi! > > Thank you all for your opinions and ideas! > > While reading the thread I made two important conclusions: > > 1. Proposed API should be changed

Re: IgniteSet implementation: changes required

Thanks, Dmitry. I agree ultimately, DS API uniformity is a weighty reason. 2018-03-17 3:54 GMT+03:00 Dmitriy Setrakyan : > On Fri, Mar 16, 2018 at 7:39 AM, Andrey Kuznetsov > wrote: > > > Dmitry, your way allows to reuse existing {{Ignite.set()}} API to

[GitHub] ignite pull request #3634: IGNITE-7879: Don't push down expressions with agg...

Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3634 ---

[jira] [Created] (IGNITE-7973) TX SQL: plain INSERT should not be broadcasted to all data nodes

Vladimir Ozerov created IGNITE-7973: --- Summary: TX SQL: plain INSERT should not be broadcasted to all data nodes Key: IGNITE-7973 URL: https://issues.apache.org/jira/browse/IGNITE-7973 Project:

New test failures in .NET

Hi, There are 31 test failures in .NET tests https://ci.ignite.apache.org/viewLog.html?buildId=1137460=buildResultsDiv=IgniteTests24Java8_IgnitePlatformNet Unfortunately it continues to reproduce. Igniters, who can advice how to fix it? Was there any changes in .NET tests/new tests

[GitHub] ignite pull request #3646: IGNITE-7931

GitHub user 1vanan opened a pull request: https://github.com/apache/ignite/pull/3646 IGNITE-7931 You can merge this pull request into a Git repository by running: $ git pull https://github.com/1vanan/ignite ignite-7931 Alternatively you can review and apply these changes as