[jira] [Created] (IGNITE-10668) Web Console: Prepare sample NGINX config with SSL and ciphers support

2018-12-13 Thread Alexey Kuznetsov (JIRA)
Alexey Kuznetsov created IGNITE-10668: - Summary: Web Console: Prepare sample NGINX config with SSL and ciphers support Key: IGNITE-10668 URL: https://issues.apache.org/jira/browse/IGNITE-10668

Re: Use of marshaller at node startup routine (need advice)

2018-12-13 Thread Vyacheslav Daradur
In terms of providing consistency of services' API, where the interface `Service` extend `Serializable` end-user should make his implementation to be serializable properly. >From this point of view, `JdkMarshaller` can be used as the common marshaller for service's instance, this allows reduce

Re: Clean up of our PRs and IEPs before 2019

2018-12-13 Thread Petr Ivanov
> On 11 Dec 2018, at 10:10, Nikolay Izhikov wrote: > > Hello, Ivan. > > Personally, I keep my PR's clear. > So, I don't have dozens of opened PR. > > But, I don't support Dmitriy proposal for several reasons: > > 1. We introduce some new, not required, level of bureaucracy. > From my

[jira] [Created] (IGNITE-10669) NPE in freelist.PagesList.findTailIndex

2018-12-13 Thread ARomantsov (JIRA)
ARomantsov created IGNITE-10669: --- Summary: NPE in freelist.PagesList.findTailIndex Key: IGNITE-10669 URL: https://issues.apache.org/jira/browse/IGNITE-10669 Project: Ignite Issue Type: Bug

Re: CONTRIBUTING.md first version

2018-12-13 Thread Petr Ivanov
There is an error in Build project [1] section. How can it be fixed? Ticket to IGNITE Jira? [1] https://github.com/apache/ignite/blob/master/CONTRIBUTING.md#building-project > On 12 Dec 2018, at 20:21, Dmitriy Pavlov wrote: > > Hi Igniters, > > Just to inform, the very first version of the

[jira] [Created] (IGNITE-10670) investigate why Cassandra modules don't have testsuite(s)

2018-12-13 Thread Oleg Ignatenko (JIRA)
Oleg Ignatenko created IGNITE-10670: --- Summary: investigate why Cassandra modules don't have testsuite(s) Key: IGNITE-10670 URL: https://issues.apache.org/jira/browse/IGNITE-10670 Project: Ignite

Continuous queries and duplicates

2018-12-13 Thread Piotr Romański
Hi, as suggested by Ilya here: http://apache-ignite-users.70518.x6.nabble.com/Continuous-queries-and-duplicates-td25314.html I'm resending it to the developers list. >From that thread we know that there might be duplicates between initial query results and listener entries received as part of

Re: Code inspection

2018-12-13 Thread Andrey Mashenkov
Fixed memory issues with increasing heap size and forcing G1GC. Do we need all these plugins loaded for inspections? I've found a 'disable plugin' option in TC Inspections build configuration, but it is unclear how to disable plugin correctly. Can someone take over this? > 46 plugins initialized

Re: [MTCGA]: new failures in builds [2446283, 2368367] needs to be handled

2018-12-13 Thread Dmitriy Pavlov
Hi All, the Bot detected some stable failure in the past because failure history in the latest Bot release is longer, so this is not a recent failure. But anyway both tests need to be fixed as they are flaky. вт, 11 дек. 2018 г. в 23:34, : > Hi Igniters, > > I've detected some new issue on

Re: [MTCGA]: new failures in builds [2372233] needs to be handled

2018-12-13 Thread Dmitriy Pavlov
Mentioned failure is not recent, but occurred in the past. Hopefully, new services implementation will fix the test. чт, 13 дек. 2018 г. в 04:11, : > Hi Igniters, > > I've detected some new issue on TeamCity to be handled. You are more than > welcomed to help. > > If your changes can lead to

Re: CONTRIBUTING.md first version

2018-12-13 Thread Dmitriy Pavlov
I'll fix. There is an ignored files list. чт, 13 дек. 2018 г. в 12:02, Petr Ivanov : > There is an error in Build project [1] section. > How can it be fixed? Ticket to IGNITE Jira? > > > [1] > https://github.com/apache/ignite/blob/master/CONTRIBUTING.md#building-project > > > On 12 Dec 2018, at

[ML] Metric calculation for classification models

2018-12-13 Thread Yuriy Babak
Igniters, Alexey I want to discuss the ticket 10371 [1], currently, we calculate 4 numbers (true positive, true negative, false positive, false negative) for each "point metric" like accuracy, recall, f-score and precision for each label. So for the full score we need calculates those 4 numbers

[jira] [Created] (IGNITE-10671) Double initialization of segmentAware and FileArchiver lead to race breaking file compression.

2018-12-13 Thread Pavel Voronkin (JIRA)
Pavel Voronkin created IGNITE-10671: --- Summary: Double initialization of segmentAware and FileArchiver lead to race breaking file compression. Key: IGNITE-10671 URL:

Re: [MTCGA]: new failures in builds [2481986] needs to be handled

2018-12-13 Thread Dmitriy Pavlov
Flaky timeout. Probably already fixed пн, 10 дек. 2018 г. в 04:47, : > Hi Igniters, > > I've detected some new issue on TeamCity to be handled. You are more than > welcomed to help. > > If your changes can lead to this failure(s): We're grateful that you were > a volunteer to make the

Re: [MTCGA]: new failures in builds [2375823] needs to be handled

2018-12-13 Thread Dmitriy Pavlov
TC Bot detected some stable failure in the past. The mentioned build is outside of TC storage now. Failure history in the latest Bot release is longer, so this is not a recent failure. But anyway, the test needs to be fixed as it is flaky. ср, 12 дек. 2018 г. в 00:11, : > Hi Igniters, > >

Re: [MTCGA]: new failures in builds [2364053] needs to be handled

2018-12-13 Thread Dmitriy Pavlov
The failure occurred a long time ago, it is outside of TC history now. The test is still flaky. чт, 13 дек. 2018 г. в 07:56, : > Hi Igniters, > > I've detected some new issue on TeamCity to be handled. You are more than > welcomed to help. > > If your changes can lead to this failure(s): We're

Re: Code inspection

2018-12-13 Thread Andrey Mashenkov
Guys, I've just creates a copy of Inspections TC build task with GC logs turned on to check if there is any issues and found Inspections task spent too much time in STW due to long Full GC pauses. I've tried to increase Xmx up to 4Gb and use G1GC got 2+ times better execution time down to ~15

Re: Code inspection

2018-12-13 Thread Dmitriy Pavlov
Sure, let's apply. I hope all TC agents may handle 4G heap. чт, 13 дек. 2018 г. в 12:54, Andrey Mashenkov : > Guys, > > I've just creates a copy of Inspections TC build task with GC logs turned > on to check if there is any issues > and found Inspections task spent too much time in STW due to

MVCC: [MTCGA]: new failures in builds [2512587] needs to be handled

2018-12-13 Thread Dmitriy Pavlov
It is really recently broken test: https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=7945125576049372508=%3Cdefault%3E=testDetails Now it fails stable. Igor S, Vladimir O. please chime in. вт, 11 дек. 2018 г. в 09:17, : > Hi Igniters, > > I've detected some new issue on

Re: [MTCGA]: new failures in builds [2421292] needs to be handled

2018-12-13 Thread Dmitriy Pavlov
This failure bot complains was occurred in November. It is not a recent failure. But the test is still flaky. ср, 12 дек. 2018 г. в 03:26, : > Hi Igniters, > > I've detected some new issue on TeamCity to be handled. You are more than > welcomed to help. > > If your changes can lead to this

Re: Code inspection

2018-12-13 Thread Dmitriy Pavlov
Maxim M, do you know if we can disable inspections by wildcard? E.g. Android* ? чт, 13 дек. 2018 г. в 14:59, Andrey Mashenkov : > Fixed memory issues with increasing heap size and forcing G1GC. > > Do we need all these plugins loaded for inspections? > I've found a 'disable plugin' option in TC

Re: [MTCGA]: new failures in builds [2527852] needs to be handled

2018-12-13 Thread Dmitriy Pavlov
https://issues.apache.org/jira/browse/IGNITE-10245 ср, 12 дек. 2018 г. в 17:11, : > Hi Igniters, > > I've detected some new issue on TeamCity to be handled. You are more than > welcomed to help. > > If your changes can lead to this failure(s): We're grateful that you were > a volunteer to make

[jira] [Created] (IGNITE-10672) Changing walSegments property leads to fallen node

2018-12-13 Thread Dmitry Sherstobitov (JIRA)
Dmitry Sherstobitov created IGNITE-10672: Summary: Changing walSegments property leads to fallen node Key: IGNITE-10672 URL: https://issues.apache.org/jira/browse/IGNITE-10672 Project: Ignite

Re: Page eviction for in-memory mode with enabled MVCC

2018-12-13 Thread Andrey Mashenkov
Roman, We have a ticket to improve repeatable_read mode [1] via caching entries locally. This should make mvcc transaction repeatable_read semantic similar to non-mvcc Txs and allow us to implement eviction in correct way. Another way is to introduce mvcc shared (read) entry locks and evict only

Re: [Fwd: Returned post for annou...@apache.org]

2018-12-13 Thread vveider
ASC / SHA512 verification: 1. Download signing key: wget -c https://www.apache.org/dist/ignite/KEYS 2. Import key: gpg --import KEYS 3. Download release with signature and sha512 hash sum: wget -c http://apache.org/dist/ignite/2.7.0/apache-ignite-2.7.0-src.zip

[jira] [Created] (IGNITE-10674) Remove MARSH_CLASS_NAME=BinaryMarshaller from tests

2018-12-13 Thread Ilya Kasnacheev (JIRA)
Ilya Kasnacheev created IGNITE-10674: Summary: Remove MARSH_CLASS_NAME=BinaryMarshaller from tests Key: IGNITE-10674 URL: https://issues.apache.org/jira/browse/IGNITE-10674 Project: Ignite

Re: Page eviction for in-memory mode with enabled MVCC

2018-12-13 Thread Vladimir Ozerov
Andrey, We will not be able to cache the whole data set locally, as it potentially lead to OOME. We will have this only as an option and only for non-SQL updates. Thus, similar semantics is not possible. On Thu, Dec 13, 2018 at 4:56 PM Andrey Mashenkov wrote: > Roman, > > We have a ticket to

Re: Code inspection

2018-12-13 Thread Andrey Mashenkov
Maxim, It looks like we can't make logs more verbose due to possible bug, I've create a ticket in Jetbrains Jira [1]. We can just publish idea logs in artefacts as suggested in this manual [2]. For now, Inspections logs looks like this one [3]. Also, would you please to take a look at inspection

Page eviction for in-memory mode with enabled MVCC

2018-12-13 Thread Roman Kondakov
Hi igniters, I need your advice with the following issue. When in-memory cache reaches it's memory limit, some data may be purged to avoid OOM error. This process is described in [1]. For MVCC caches this eviction may break repeatable read semantics. For example, if transaction reads key

Re: Page eviction for in-memory mode with enabled MVCC

2018-12-13 Thread Ilya Kasnacheev
Hello! Is it possible to 'touch' entries read by MVCC transactions to ensure that they are considered recent and therefore are almost never targeted by eviction? This is 1) with benefits. Regards, -- Ilya Kasnacheev чт, 13 дек. 2018 г. в 16:22, Roman Kondakov : > Hi igniters, > > I need

[jira] [Created] (IGNITE-10673) Prepare instructions for ASC and SHA512 verification

2018-12-13 Thread Peter Ivanov (JIRA)
Peter Ivanov created IGNITE-10673: - Summary: Prepare instructions for ASC and SHA512 verification Key: IGNITE-10673 URL: https://issues.apache.org/jira/browse/IGNITE-10673 Project: Ignite

Re: [ML] Metric calculation for classification models

2018-12-13 Thread Alexey Zinoviev
So, I agree that we should avoid ineffective metrics calculations. I think that in 2.8 release we should have 1. BinaryClassificationMetric with all metrics from Wikipedia 2. Metric interface with 1 or two implementations in example folder or in metric package like roc auc and accuracy

Re: Page eviction for in-memory mode with enabled MVCC

2018-12-13 Thread Andrey Mashenkov
Vladimir, Agree, I've just forget about queries :) On Thu, Dec 13, 2018 at 5:28 PM Vladimir Ozerov wrote: > Andrey, > > We will not be able to cache the whole data set locally, as it potentially > lead to OOME. We will have this only as an option and only for non-SQL > updates. Thus, similar

Re: Page eviction for in-memory mode with enabled MVCC

2018-12-13 Thread Vladimir Ozerov
Roman, I would start with the fact that eviction can never be consistent unless it utilizes atomic broadcast protocol, which is not the case for Ignite. In Ignite entries on node are evicted independently. So you may easily get into situation like this: 1) Start a cache with 1 backup and

Re: Code inspection

2018-12-13 Thread Maxim Muzafarov
Andrey, Thank you for solving this issue with GC pauses! I've checked the given report. The inspections configuration is correct, but it seems to me that we have enabled by default rules of included plugins (for instance, KotlinInternalInJava in the report is enabled). Can you share more details

Re: [ML] Metric calculation for classification models

2018-12-13 Thread Yuriy Babak
Dmitriy, Sure, all changes in ML module will be described on readme.io site with next release (2.8). Best regards, Yuriy Babak чт, 13 дек. 2018 г. в 17:21, Dmitriy Pavlov : > Folks, I sometimes hear complains related to metrics and its clearness for > end-users. > > Would you add a couple of

Re: Code inspection

2018-12-13 Thread Andrey Mashenkov
Maxim, Idea has a file in config directory ./config/disabled_plugins.txt , you can easily find it at you local machine. Teamcity Inspections runner has an option "Disabled plugins" where disabled_plugins.txt file content can be set. So, looks like we can disable useless plugins. But I'm not

Re: [MTCGA]: new failures in builds [2372233] needs to be handled

2018-12-13 Thread Vyacheslav Daradur
Yes, it is expected this test become not flaky once Service Grid phase 1 have been merged: https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=5756858704963167219=%3Cdefault%3E=testDetails_IgniteTests24Java8=pull%2F4434%2Fhead On Thu, Dec 13, 2018 at 3:22 PM Dmitriy Pavlov

Re: [ML] Metric calculation for classification models

2018-12-13 Thread Алексей Платонов
You can compute just TP (true-positive), FP, TN and FN counters and use them to evaluate Recall, Precision, Accuracy, ect. If you want to specify class for Pr evaluation, then you can compute Pr for first label as TP/(TP+FP) and for second label as TN/(TN+FN) for example. After it we can unite all

Re: [ML] Metric calculation for classification models

2018-12-13 Thread Dmitriy Pavlov
Folks, I sometimes hear complains related to metrics and its clearness for end-users. Would you add a couple of words related to each value to wiki/readme.io? чт, 13 дек. 2018 г. в 17:13, Alexey Zinoviev : > So, I agree that we should avoid ineffective metrics calculations. > I think that in

[jira] [Created] (IGNITE-10675) Refactor Release Candidate check build

2018-12-13 Thread Peter Ivanov (JIRA)
Peter Ivanov created IGNITE-10675: - Summary: Refactor Release Candidate check build Key: IGNITE-10675 URL: https://issues.apache.org/jira/browse/IGNITE-10675 Project: Ignite Issue Type:

Re: [ML] Metric calculation for classification models

2018-12-13 Thread Alexey Zinoviev
Please, have a look at new version in my PR where I've implemented the approach that was listed above https://github.com/apache/ignite/pull/5612 чт, 13 дек. 2018 г. в 17:21, Dmitriy Pavlov : > Folks, I sometimes hear complains related to metrics and its clearness for > end-users. > > Would you

[jira] [Created] (IGNITE-10676) Binary: optionally do not check types in metadata

2018-12-13 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-10676: Summary: Binary: optionally do not check types in metadata Key: IGNITE-10676 URL: https://issues.apache.org/jira/browse/IGNITE-10676 Project: Ignite

Re: Page eviction for in-memory mode with enabled MVCC

2018-12-13 Thread Roman Kondakov
Hi Ilya! Touching entries seem to me not a possible solution here. During what the period entry should be considered as recent? I think until all transactions which have touched the given entry are terminated. If all transactions which have read the particular entry are terminated - it can

Re: Page eviction for in-memory mode with enabled MVCC

2018-12-13 Thread Roman Kondakov
Vladimir, correct me please if i misunderstood your thought. So, if eviction is not about a consistency at all, we may evict keys in any way because broken repeatable read semantics is not the biggest problem here. But we should add some notes about it to user documentation. Right? -- Kind

Re: Page eviction for in-memory mode with enabled MVCC

2018-12-13 Thread Vladimir Ozerov
No, I mean that we should think about what kind of guarantees it possible. My proposal was to prevent evictions of locked entries. This way we can say users: "if you want true REPEATABLE_READ when evictions are enabled, then make sure to lock entries on every access". This effectively means that

Re: Continuous queries and duplicates

2018-12-13 Thread Denis Magda
Hello Piotr, That's a known problem and I thought a JIRA ticket already exists. However, failed to locate it. The ticket for the improvement should be created as a result of this conversation. Speaking of an initial query type, I would differentiate from ScanQueries and SqlQueries. For the

Re: Page eviction for in-memory mode with enabled MVCC

2018-12-13 Thread Roman Kondakov
Vladimir, we do not lock entries on backups when MVCC is enabled and therefore we don't avoid entry eviction from backup by locking. So, your first scenario with primary stop is still relevant. -- Kind Regards Roman Kondakov On 13.12.2018 22:14, Vladimir Ozerov wrote: No, I mean that we

Re: Continuous queries and duplicates

2018-12-13 Thread Vladimir Ozerov
[1] http://apache-ignite-developers.2346864.n4.nabble.com/Continuous-queries-and-MVCC-td33972.html On Thu, Dec 13, 2018 at 11:38 PM Vladimir Ozerov wrote: > Denis, > > Not really. They are used to ensure that ordering of notifications is > consistent with ordering of updates, so that when a key

Re: Continuous queries and duplicates

2018-12-13 Thread Vladimir Ozerov
Partition counter is internal implemenattion detail, which has no sensible meaning to end users. It should not be exposed through public API. On Thu, Dec 13, 2018 at 10:14 PM Denis Magda wrote: > Hello Piotr, > > That's a known problem and I thought a JIRA ticket already exists. However, >

Re: Continuous queries and duplicates

2018-12-13 Thread Denis Magda
Vladimir, The partition counter is supposed to be used internally to solve the duplication issue. Does it sound like a right approach then? What would be an approach for SQL queries? Not sure the partition counter is applicable. -- Denis On Thu, Dec 13, 2018 at 11:16 AM Vladimir Ozerov wrote:

Re: Page eviction for in-memory mode with enabled MVCC

2018-12-13 Thread Vladimir Ozerov
It's hard to believe that entries are not locked on backups, because we wrtite data right away. Even if it so, it should be very easy to fix - just do not evict and entry if it was created or deleted by currently active transaction. On Thu, Dec 13, 2018 at 10:28 PM Roman Kondakov wrote: >

[jira] [Created] (IGNITE-10678) Shell script unification

2018-12-13 Thread Sergey Kozlov (JIRA)
Sergey Kozlov created IGNITE-10678: -- Summary: Shell script unification Key: IGNITE-10678 URL: https://issues.apache.org/jira/browse/IGNITE-10678 Project: Ignite Issue Type: Improvement

Re: [Fwd: Returned post for annou...@apache.org]

2018-12-13 Thread Prachi Garg
For point 3, I cannot find where dist.apache.org is used...Can you please point me to the right place? On Thu, Dec 13, 2018 at 6:07 AM Peter Ivanov wrote: > ASC / SHA512 verification: > > 1. Download signing key: wget -c https://www.apache.org/dist/ignite/KEYS > 2. Import key: gpg --import

[jira] [Created] (IGNITE-10681) PME benchmarks become unstable at high number of partitions per cache.

2018-12-13 Thread Pavel Voronkin (JIRA)
Pavel Voronkin created IGNITE-10681: --- Summary: PME benchmarks become unstable at high number of partitions per cache. Key: IGNITE-10681 URL: https://issues.apache.org/jira/browse/IGNITE-10681

Test case failure from IgniteBasicTestSuite

2018-12-13 Thread Namrata Bhave
Hi, I am working on building Apache Ignite v2.7.0 on s390x architecture. The build was successful, however facing some test failures. One of the failures is: [ERROR] testAllocationRateMultiThreaded(org.apache.ignite.internal.processors.database.DataRegionMetricsSelfTest) Time elapsed: 17.204

Re: Service grid redesign

2018-12-13 Thread Nikolay Izhikov
Hello, Denis. Great news. > *1. Testing of the cache-based implementation of the service grid.* > I think, we should make a test suite, that will test the old implementation> > until we> remove it from the project. Aggree. Let's do it. > *2. DynamicServiceChangeRequest.* > I think, this class

[jira] [Created] (IGNITE-10682) Disable unnecessary loaded plugins for the Inspection test suite

2018-12-13 Thread Maxim Muzafarov (JIRA)
Maxim Muzafarov created IGNITE-10682: Summary: Disable unnecessary loaded plugins for the Inspection test suite Key: IGNITE-10682 URL: https://issues.apache.org/jira/browse/IGNITE-10682 Project:

Re: Continuous queries and duplicates

2018-12-13 Thread Vladimir Ozerov
Denis, Not really. They are used to ensure that ordering of notifications is consistent with ordering of updates, so that when a key K is updated to V1, then V2, then V3, you never observe V1 -> V3 -> V2. It also solves duplicate notification problem in case of node failures, when the same update

Re: Service grid redesign

2018-12-13 Thread Denis Mekhanikov
Guys, I've been looking through the PR by Vyacheslav for past few weeks. Slava, great job! You've done an impressive amount of work. I posted my comments to the PR and had a few calls with Slava. I am close to finishing my review. There are some points, that I'd like to settle in this discussion

[jira] [Created] (IGNITE-10677) [TC Bot] Include build failure on metric & exit code into critical failures

2018-12-13 Thread Dmitriy Pavlov (JIRA)
Dmitriy Pavlov created IGNITE-10677: --- Summary: [TC Bot] Include build failure on metric & exit code into critical failures Key: IGNITE-10677 URL: https://issues.apache.org/jira/browse/IGNITE-10677

Re: CONTRIBUTING.md first version

2018-12-13 Thread Alexey Kuznetsov
Hi, >From my experience it is better when all needed stuff available on GitHub. It is very comfortable when thinks like, "read me", "change log", "how to build", "how to contribute" and so on available on GitHub. No need to open some external links. -- Alexey Kuznetsov

[jira] [Created] (IGNITE-10679) Add more debug info for 'Affinity changes' PME stage.

2018-12-13 Thread Pavel Voronkin (JIRA)
Pavel Voronkin created IGNITE-10679: --- Summary: Add more debug info for 'Affinity changes' PME stage. Key: IGNITE-10679 URL: https://issues.apache.org/jira/browse/IGNITE-10679 Project: Ignite

[jira] [Created] (IGNITE-10680) Add the ability to use started kernel context in standalone WAL iterator

2018-12-13 Thread Aleksey Plekhanov (JIRA)
Aleksey Plekhanov created IGNITE-10680: -- Summary: Add the ability to use started kernel context in standalone WAL iterator Key: IGNITE-10680 URL: https://issues.apache.org/jira/browse/IGNITE-10680

Re: [DISCUSSION] Relocation of Apache git repositories from git-wip-us.apache.org to GitBox

2018-12-13 Thread Peter Ivanov
There will be no problem with TC, as we use GitHub as main VCS root, using ASF only for release (which I will reconfigure during release builds refactoring and optimisations for AI 2.8). > On 11 Dec 2018, at 19:25, Alexey Goncharuk wrote: > > Given that there is no option to stay on the old

Re: CONTRIBUTING.md first version

2018-12-13 Thread Denis Magda
The Github page looks like a shorter version of the main wiki page: https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute Not sure we want to maintain both. If the wiki source is sluggish and overloaded then I would rework it instead moving secondary stuff to sub-pages. Think that