[jira] [Created] (IGNITE-7040) Web console: Invalid user table height

[Vasiliy Sisko (JIRA)]

Vasiliy Sisko created IGNITE-7040:
-

 Summary: Web console: Invalid user table height
 Key: IGNITE-7040
 URL: https://issues.apache.org/jira/browse/IGNITE-7040
 Project: Ignite
  Issue Type: Bug
Reporter: Vasiliy Sisko
Assignee: Dmitriy Shabalin


# Filter user list (f.e. to 2 rows)
# Change period of showed activity metrics.

Height of table changed to maximum available but show only filtered rows.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (IGNITE-6878) Naive Bayes classifier for ML module

[Eugeniu Semenciuc (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eugeniu Semenciuc reassigned IGNITE-6878:
-

Assignee: Eugeniu Semenciuc

> Naive Bayes classifier for ML module
> 
>
> Key: IGNITE-6878
> URL: https://issues.apache.org/jira/browse/IGNITE-6878
> Project: Ignite
>  Issue Type: New Feature
>  Components: ml
>Reporter: Yury Babak
>Assignee: Eugeniu Semenciuc
>
> Naive Bayes classifiers are a family of simple probabilistic classifiers 
> based on applying Bayes' theorem with strong (naive) independence assumptions 
> between the features.
> So we want to add this algorithm to Apache Ignite ML module.
> Ideally, implementation should support both multinomial naive Bayes and 
> Bernoulli naive Bayes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7039) SQL: local query should pin affected partitions

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-7039:
---

 Summary: SQL: local query should pin affected partitions
 Key: IGNITE-7039
 URL: https://issues.apache.org/jira/browse/IGNITE-7039
 Project: Ignite
  Issue Type: Bug
  Components: sql
Reporter: Vladimir Ozerov


When distributed query is executed, we pin cache partitions for particular 
topology version on map nodes [1]. However, it seems that we do no do that for 
local queries. This is a bug because partition with required data could be 
evicted from local node at any time, leading to incorrect results.

[1] 
https://github.com/apache/ignite/blob/ignite-2.3/modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/twostep/GridMapQueryExecutor.java#L288



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (IGNITE-7038) SQL: nested tables support

[Vladimir Ozerov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-7038:

Labels: iep-10  (was: )

> SQL: nested tables support
> --
>
> Key: IGNITE-7038
> URL: https://issues.apache.org/jira/browse/IGNITE-7038
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Reporter: Vladimir Ozerov
>  Labels: iep-10
>
> Many commercial databases support a kind of nested tables which is 
> essentially a parent-child relation with special storage format. With this 
> approach child data can be located efficiently without joins. 
> Syntax example:
> {code}
> CREATE TYPE address_t AS OBJECT (
>cityVARCHAR2(20),
>street  VARCHAR2(30)
> );
> CREATE TYPE address_tab IS TABLE OF address_t;
> CREATE TABLE customers (
>custid  NUMBER,
>address address_tab )
> NESTED TABLE address STORE AS customer_addresses;
> INSERT INTO customers VALUES (
> 1,
> address_tab(
> address_t('Redwood Shores', '101 First'),
> address_t('Mill Valley', '123 Maple')
> )
> );
> {code}
> Several storage formats should be considered. First, data can be embedded to 
> parent data row directly or through forward reference to a chain of dedicated 
> blocks (similar to LOB data types). This is how conventional RDBMS systems 
> work. 
> Second, children rows could be stored in the same PK index as parent row. 
> This is how Spanner works. In this case parent and child rows are different 
> rows, but stored in the same data structures. This allows for sequential 
> access to both parent and child data in case of joins, which could be 
> extremely valuable in OLAP cases.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7038) SQL: nested tables support

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-7038:
---

 Summary: SQL: nested tables support
 Key: IGNITE-7038
 URL: https://issues.apache.org/jira/browse/IGNITE-7038
 Project: Ignite
  Issue Type: Bug
  Components: sql
Reporter: Vladimir Ozerov


Many commercial databases support a kind of nested tables which is essentially 
a parent-child relation with special storage format. With this approach child 
data can be located efficiently without joins. 

Syntax example:
{code}
CREATE TYPE address_t AS OBJECT (
   cityVARCHAR2(20),
   street  VARCHAR2(30)
);

CREATE TYPE address_tab IS TABLE OF address_t;

CREATE TABLE customers (
   custid  NUMBER,
   address address_tab )
NESTED TABLE address STORE AS customer_addresses;

INSERT INTO customers VALUES (
1,
address_tab(
address_t('Redwood Shores', '101 First'),
address_t('Mill Valley', '123 Maple')
)
);
{code}

Several storage formats should be considered. First, data can be embedded to 
parent data row directly or through forward reference to a chain of dedicated 
blocks (similar to LOB data types). This is how conventional RDBMS systems 
work. 

Second, children rows could be stored in the same PK index as parent row. This 
is how Spanner works. In this case parent and child rows are different rows, 
but stored in the same data structures. This allows for sequential access to 
both parent and child data in case of joins, which could be extremely valuable 
in OLAP cases.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (IGNITE-584) Need to make sure that scan query returns consistent results on topology changes

[Stanilovsky Evgeny (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267336#comment-16267336
 ] 

Stanilovsky Evgeny edited comment on IGNITE-584 at 11/28/17 7:05 AM:
-

we need to set _keepBinary=false_ on _onEntryUpdated_ for proper unwrap non 
_Externalizable_ class _CollocatedSetItemKey_ and further 
CacheDataStructuresManager check: _if (key0 instanceof SetItemKey)_ check, 
otherwise we need to implement _CollocatedSetItemKey_  as _Externalizable_ too 
and replace from registerPredefinedType.


was (Author: zstan):
we need to set _keepBinary=false_ on _onEntryUpdated_ for proper unwrap non 
_Externalizable_ class _CollocatedSetItemKey_ and further _if (key0 instanceof 
SetItemKey)_ check, otherwise we need to implement _CollocatedSetItemKey_  as 
_Externalizable_ too and replace from registerPredefinedType.

> Need to make sure that scan query returns consistent results on topology 
> changes
> 
>
> Key: IGNITE-584
> URL: https://issues.apache.org/jira/browse/IGNITE-584
> Project: Ignite
>  Issue Type: Sub-task
>  Components: data structures
>Affects Versions: 1.9, 2.0, 2.1
>Reporter: Artem Shutak
>Assignee: Semen Boikov
>  Labels: MakeTeamcityGreenAgain, Muted_test
> Fix For: 2.4
>
> Attachments: tc1.png
>
>
> Consistent results on topology changes was implemented for sql queries, but 
> looks like it still does not work for scan queries.
> This affects 'cache set' tests since set uses scan query for set iteration 
> (to be unmuted on TC): 
> GridCacheSetAbstractSelfTest testNodeJoinsAndLeaves and 
> testNodeJoinsAndLeavesCollocated; 
> Also see todos here GridCacheSetFailoverAbstractSelfTest



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (IGNITE-584) Need to make sure that scan query returns consistent results on topology changes

[Stanilovsky Evgeny (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stanilovsky Evgeny updated IGNITE-584:
--
Attachment: tc1.png

https://ci.ignite.apache.org/viewLog.html?buildId=965587=buildResultsDiv=Ignite20Tests_IgniteDataStructures
 
TC seems ok

> Need to make sure that scan query returns consistent results on topology 
> changes
> 
>
> Key: IGNITE-584
> URL: https://issues.apache.org/jira/browse/IGNITE-584
> Project: Ignite
>  Issue Type: Sub-task
>  Components: data structures
>Affects Versions: 1.9, 2.0, 2.1
>Reporter: Artem Shutak
>Assignee: Semen Boikov
>  Labels: MakeTeamcityGreenAgain, Muted_test
> Fix For: 2.4
>
> Attachments: tc1.png
>
>
> Consistent results on topology changes was implemented for sql queries, but 
> looks like it still does not work for scan queries.
> This affects 'cache set' tests since set uses scan query for set iteration 
> (to be unmuted on TC): 
> GridCacheSetAbstractSelfTest testNodeJoinsAndLeaves and 
> testNodeJoinsAndLeavesCollocated; 
> Also see todos here GridCacheSetFailoverAbstractSelfTest



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (IGNITE-6989) .NET: Thin client: Group operation codes by purpose

[Pavel Tupitsyn (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn resolved IGNITE-6989.

Resolution: Fixed

> .NET: Thin client: Group operation codes by purpose
> ---
>
> Key: IGNITE-6989
> URL: https://issues.apache.org/jira/browse/IGNITE-6989
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms, thin client
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .NET
> Fix For: 2.4
>
>
> Currently operation codes are in random order. Let's group things by purpose: 
> cache operations, metadata, queries, etc.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-7013) .NET: Ignite does not start on macOS

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268207#comment-16268207
 ] 

Pavel Tupitsyn commented on IGNITE-7013:


Thin client tests failed on macOs: setting socket buffer size to 0 throws an 
error there. Fixed that by skipping socket setting update when defaults are set.

> .NET: Ignite does not start on macOS
> 
>
> Key: IGNITE-7013
> URL: https://issues.apache.org/jira/browse/IGNITE-7013
> Project: Ignite
>  Issue Type: Bug
>  Components: platforms
>Affects Versions: 2.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .NET, xplat
> Fix For: 2.4
>
>
> Looks like dlopen code is incorrect for macOS:
> {code}
> Unhandled Exception: System.DllNotFoundException: Unable to load DLL 
> 'libcoreclr.so': The specified module or one of its dependencies could not be 
> found.
>  (Exception from HRESULT: 0x8007007E)
>at 
> Apache.Ignite.Core.Impl.Unmanaged.DllLoader.NativeMethodsCore.dlopen(String 
> filename, Int32 flags)
>at Apache.Ignite.Core.Impl.Unmanaged.DllLoader.Load(String dllPath)
>at Apache.Ignite.Core.Impl.IgniteUtils.LoadDll(String filePath, String 
> simpleName)
>at Apache.Ignite.Core.Impl.IgniteUtils.LoadJvmDll(String configJvmDllPath, 
> ILogger log)
>at Apache.Ignite.Core.Impl.IgniteUtils.LoadDlls(String configJvmDllPath, 
> ILogger log)
>at Apache.Ignite.Core.Ignition.Start(IgniteConfiguration cfg)
>at ignite_nuget_test.Program.Main(String[] args) in 
> /Users/vveider/Development/VCS/Git/ignite-dotnetcore-demo/Program.cs:line 17
> {code}
> Steps to reproduce:
> {code}
> git clone https://github.com/ptupitsyn/ignite-dotnetcore-demo.git
> cd ignite-dotnetcore-demo
> dotnet run
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-5960) Ignite Continuous Query (Queries 3): CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic is flaky

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268184#comment-16268184
 ] 

Sunny Chan commented on IGNITE-5960:


Pull request: https://github.com/apache/ignite/pull/3100

> Ignite Continuous Query (Queries 3): 
> CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic
>  is flaky
> -
>
> Key: IGNITE-5960
> URL: https://issues.apache.org/jira/browse/IGNITE-5960
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Sergey Chugunov
>Assignee: Alexey Kuznetsov
>  Labels: MakeTeamcityGreenAgain, test-failure
> Fix For: 2.4
>
>
> According to [TC 
> history|http://ci.ignite.apache.org/project.html?projectId=Ignite20Tests=6546112007182082024=testDetails_Ignite20Tests=%3Cdefault%3E]
>  test is flaky.
> It is possible to reproduce it locally, sample run shows 9 failed tests out 
> of 30 overall executed.
> Test fails with jUnit assertion check: 
> {noformat}
> junit.framework.AssertionFailedError: 
> Expected :1
> Actual   :0
>  
>   at junit.framework.Assert.fail(Assert.java:57)
>   at junit.framework.Assert.failNotEquals(Assert.java:329)
>   at junit.framework.Assert.assertEquals(Assert.java:78)
>   at junit.framework.Assert.assertEquals(Assert.java:234)
>   at junit.framework.Assert.assertEquals(Assert.java:241)
>   at junit.framework.TestCase.assertEquals(TestCase.java:409)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.concurrentUpdatesAndQueryStart(CacheContinuousQueryConcurrentPartitionUpdateTest.java:385)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.testConcurrentUpdatesAndQueryStartTx(CacheContinuousQueryConcurrentPartitionUpdateTest.java:245)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at junit.framework.TestCase.runTest(TestCase.java:176)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2000)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:132)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1915)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-5960) Ignite Continuous Query (Queries 3): CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic is flaky

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268181#comment-16268181
 ] 

Sunny Chan commented on IGNITE-5960:


[~akuznetsov] I have tried your patch but I can still reproduce the issue on my 
machine, using the unit test above and our own application.



> Ignite Continuous Query (Queries 3): 
> CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic
>  is flaky
> -
>
> Key: IGNITE-5960
> URL: https://issues.apache.org/jira/browse/IGNITE-5960
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Sergey Chugunov
>Assignee: Alexey Kuznetsov
>  Labels: MakeTeamcityGreenAgain, test-failure
> Fix For: 2.4
>
>
> According to [TC 
> history|http://ci.ignite.apache.org/project.html?projectId=Ignite20Tests=6546112007182082024=testDetails_Ignite20Tests=%3Cdefault%3E]
>  test is flaky.
> It is possible to reproduce it locally, sample run shows 9 failed tests out 
> of 30 overall executed.
> Test fails with jUnit assertion check: 
> {noformat}
> junit.framework.AssertionFailedError: 
> Expected :1
> Actual   :0
>  
>   at junit.framework.Assert.fail(Assert.java:57)
>   at junit.framework.Assert.failNotEquals(Assert.java:329)
>   at junit.framework.Assert.assertEquals(Assert.java:78)
>   at junit.framework.Assert.assertEquals(Assert.java:234)
>   at junit.framework.Assert.assertEquals(Assert.java:241)
>   at junit.framework.TestCase.assertEquals(TestCase.java:409)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.concurrentUpdatesAndQueryStart(CacheContinuousQueryConcurrentPartitionUpdateTest.java:385)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.testConcurrentUpdatesAndQueryStartTx(CacheContinuousQueryConcurrentPartitionUpdateTest.java:245)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at junit.framework.TestCase.runTest(TestCase.java:176)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2000)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:132)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1915)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-5490) Implement replacement for obsolete CacheMetrics#getOffHeapAllocatedSize

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-5490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268179#comment-16268179
 ] 

ASF GitHub Bot commented on IGNITE-5490:


GitHub user sunnychanwork opened a pull request:

https://github.com/apache/ignite/pull/3100

IGNITE-5490 Ignite Continuous Query might not send update request

During continuous query setup, there are potential updates in flight that 
would potentially have a situation like this:

T1 updates E1, lsnrs!=null, obtain update counter 1
T2 updates E2, lsnrs==null, obtain update counter 2
T3 updates E3, lsnrs!=null, obtain update counter 3

Notice that as E1 E2 and E3 are different there are no locks and they can 
proceed in parallel. As a result, the sequence of updates being sent to 
CQManager will be 1,3 with 2 missing and it will wait for update 2 forever 
which will never come.

To fix this I propose to use a ReadWrite lock to ensure that the Map Entry 
update will complete before setting up new continuous query.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sunnychanwork/ignite IGNITE-5960

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3100.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3100


commit 3cff5cf0ebc282c9300613e4d6726df5dd56ab60
Author: Sunny Chan, CLSA 
Date:   2017-11-28T05:52:14Z

IGNITE-5490 use a ReadWrite lock to ensure that the Map Entry update
will complete before setting up new continuous query




> Implement replacement for obsolete CacheMetrics#getOffHeapAllocatedSize
> ---
>
> Key: IGNITE-5490
> URL: https://issues.apache.org/jira/browse/IGNITE-5490
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.0
>Reporter: Ivan Rakov
>
> With new 2.0 architecture, many caches can share one memory policy. Memory 
> metrics allows to measure memory usage (loaded pages) for the whole policy. 
> However, there's also a need to measure how much memory (or pages) is used by 
> each cache.
> Before 2.0 such information was accessible with 
> CacheMetrics#getOffHeapAllocatedSize, but current implemetation returns 0.
> We should either implement it or provide alternative metrics (e. g. 
> approximate number of loaded pages per cache). Please note that if 
> persistence is *disabled*, precise number of loaded pages per cache is not 
> defined - one page can contain entries of different caches.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (IGNITE-6614) .NET: DotNetCore examples

[Pavel Tupitsyn (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-6614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn resolved IGNITE-6614.

Resolution: Duplicate

Duplicates IGNITE-7002

> .NET: DotNetCore examples
> -
>
> Key: IGNITE-6614
> URL: https://issues.apache.org/jira/browse/IGNITE-6614
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms
>Affects Versions: 2.3
>Reporter: Pavel Tupitsyn
>Priority: Minor
>  Labels: .NET, xplat
> Fix For: 2.4
>
>
> .NET Thin Client works under .NET Core. Create separate examples project for 
> that.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (IGNITE-6613) .NET: Thin client: Create dotnet core unit test project, run on Linux

[Pavel Tupitsyn (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-6613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn resolved IGNITE-6613.

Resolution: Fixed

Done as part of IGNITE-2662

> .NET: Thin client: Create dotnet core unit test project, run on Linux
> -
>
> Key: IGNITE-6613
> URL: https://issues.apache.org/jira/browse/IGNITE-6613
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms, thin client
>Affects Versions: 2.3
>Reporter: Pavel Tupitsyn
>Priority: Minor
>  Labels: .NET, xplat
> Fix For: 2.4
>
>
> .NET Thin Client actually works on .NET Core and on Linux. Let's add unit 
> tests for that and run them on Linux machines to avoid regression.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Closed] (IGNITE-7020) Web Console: add column resizer to pinned columns

[Alexey Kuznetsov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Kuznetsov closed IGNITE-7020.


> Web Console: add column resizer to pinned columns
> -
>
> Key: IGNITE-7020
> URL: https://issues.apache.org/jira/browse/IGNITE-7020
> Project: Ignite
>  Issue Type: Improvement
>  Components: wizards
>Reporter: Dmitriy Shabalin
>Assignee: Alexey Kuznetsov
> Fix For: 2.4
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7037) Web console: Wrong activities metrics

[Vasiliy Sisko (JIRA)]

Vasiliy Sisko created IGNITE-7037:
-

 Summary: Web console: Wrong activities metrics
 Key: IGNITE-7037
 URL: https://issues.apache.org/jira/browse/IGNITE-7037
 Project: Ignite
  Issue Type: Bug
Reporter: Vasiliy Sisko


Count of activity metrics in grouped columns is not calculated from detail 
columns.
Configuration's activities columns show always 0.
Showed data in table is not equal to data from activity dialog.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7036) Web console: Wrong export of grouped users

[Vasiliy Sisko (JIRA)]

Vasiliy Sisko created IGNITE-7036:
-

 Summary: Web console: Wrong export of grouped users
 Key: IGNITE-7036
 URL: https://issues.apache.org/jira/browse/IGNITE-7036
 Project: Ignite
  Issue Type: Bug
Reporter: Vasiliy Sisko
Assignee: Dmitriy Shabalin


On export of grouped users when group is collapsed in table only header row is 
exported.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7035) Web console: New user redirected on nonexistent page

[Vasiliy Sisko (JIRA)]

Vasiliy Sisko created IGNITE-7035:
-

 Summary: Web console: New user redirected on nonexistent page
 Key: IGNITE-7035
 URL: https://issues.apache.org/jira/browse/IGNITE-7035
 Project: Ignite
  Issue Type: Bug
Reporter: Vasiliy Sisko


New user redirected on login to /configuration page and then redirected to 
/configuration/basic.
Should be redirected directly to /configuration/basic.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7034) Web console: Wrong notebooks on become this user

[Vasiliy Sisko (JIRA)]

Vasiliy Sisko created IGNITE-7034:
-

 Summary: Web console: Wrong notebooks on become this user
 Key: IGNITE-7034
 URL: https://issues.apache.org/jira/browse/IGNITE-7034
 Project: Ignite
  Issue Type: Bug
Reporter: Vasiliy Sisko


On become this user first time showed lint of admin notebooks. That notebooks 
is not available.
On refresh show notebooks of showed user.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7033) Web console: Increase width of columns on admin page

[Vasiliy Sisko (JIRA)]

Vasiliy Sisko created IGNITE-7033:
-

 Summary: Web console: Increase width of columns on admin page
 Key: IGNITE-7033
 URL: https://issues.apache.org/jira/browse/IGNITE-7033
 Project: Ignite
  Issue Type: Bug
 Environment: "Last activity" and "email" columns are too narrow
Reporter: Vasiliy Sisko






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (IGNITE-7032) Web console: Links in activiy details.

[Vasiliy Sisko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vasiliy Sisko updated IGNITE-7032:
--
Description: In activity details dialog of admin page should be showed page 
title instead of its link.  (was: In activity details dialog should be showed 
page title instead of its link.)

> Web console: Links in activiy details.
> --
>
> Key: IGNITE-7032
> URL: https://issues.apache.org/jira/browse/IGNITE-7032
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vasiliy Sisko
>
> In activity details dialog of admin page should be showed page title instead 
> of its link.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7032) Web console: Links in activiy details.

[Vasiliy Sisko (JIRA)]

Vasiliy Sisko created IGNITE-7032:
-

 Summary: Web console: Links in activiy details.
 Key: IGNITE-7032
 URL: https://issues.apache.org/jira/browse/IGNITE-7032
 Project: Ignite
  Issue Type: Bug
Reporter: Vasiliy Sisko


In activity details dialog should be showed page title instead of its link.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7031) Web console: Error on cancellation of comfirm dialog

[Vasiliy Sisko (JIRA)]

Vasiliy Sisko created IGNITE-7031:
-

 Summary: Web console: Error on cancellation of comfirm dialog
 Key: IGNITE-7031
 URL: https://issues.apache.org/jira/browse/IGNITE-7031
 Project: Ignite
  Issue Type: Bug
Reporter: Vasiliy Sisko


On cancellation of confirm dialog error message showed in log of browser.
F.e. Clone dialog or remove all dialog.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7030) Web console: (add) link not create new linked item

[Vasiliy Sisko (JIRA)]

Vasiliy Sisko created IGNITE-7030:
-

 Summary: Web console: (add) link not create new linked item
 Key: IGNITE-7030
 URL: https://issues.apache.org/jira/browse/IGNITE-7030
 Project: Ignite
  Issue Type: Bug
Reporter: Vasiliy Sisko


On click by (add) link on configuration pages opened page with list of objects 
instead of creation of new with link to current.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7029) Add an ability to provide multiple connection addresses for thin JDBC driver

[Valentin Kulichenko (JIRA)]

Valentin Kulichenko created IGNITE-7029:
---

 Summary: Add an ability to provide multiple connection addresses 
for thin JDBC driver
 Key: IGNITE-7029
 URL: https://issues.apache.org/jira/browse/IGNITE-7029
 Project: Ignite
  Issue Type: Improvement
  Components: jdbc
Affects Versions: 2.3
Reporter: Valentin Kulichenko


Currently we allow only to provide one address when connecting via thin JDBC 
driver. This has to issues:
* If node driver is connected to goes down, the driver stops working.
* Driver has to always go though the same node - this is a bottleneck.

As a simple solution we can allow to provide multiple addresses, like MySQL 
does for example: 
https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-usagenotes-j2ee-concepts-managing-load-balanced-connections.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7028) Memcached does not set type flags for response

[Mikhail Cherkasov (JIRA)]

Mikhail Cherkasov created IGNITE-7028:
-

 Summary: Memcached does not set type flags for response
 Key: IGNITE-7028
 URL: https://issues.apache.org/jira/browse/IGNITE-7028
 Project: Ignite
  Issue Type: Bug
  Components: rest
Affects Versions: 2.3
Reporter: Mikhail Cherkasov
 Fix For: 2.4


Memcached does not set type flags for response:
http://apache-ignite-users.70518.x6.nabble.com/Memcached-doesn-t-store-flags-td18403.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (IGNITE-7027) Single primary index instead of mulitple per-partition indexes

[Vladimir Ozerov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-7027:

Description: 
Currently we have per-partition primary index. This gives us easy and effective 
rebalance/recovery capabilities and efficient lookup in key-value mode. 

However, this doesn't work well for SQL case. We cannot use this index for 
range scans. Neither we can use it for PK lookups (it is possible to implement, 
but will be less then optimal due to necessity to build the whole key object).

The following change is suggested as optional storage mode:
1) Single index data structure for all partitions
2) Only single key type is allowed (i.e. no mess in the cache and no cache 
groups)
3) Additional SQL PK index will not be needed in this case

Advantage:
- No overhead on the second PK index

Disadvantage:
- Less efficient rebalance and recovery

  was:
Currently we have per-partition primary index. This gives us easy and effective 
rebalance/recovery capabilities and efficient lookup in key-value mode. 

However, this doesn't work well for SQL case. We cannot use this index for 
range scans. Neither we can use it for PK lookups (it is possible to implement, 
but will be less then optimal due to necessity to build the whole key object).

The following change is suggested as optional storage mode:
1) Single index data structure for all partitions
2) Only single key type is allowed (i.e. no mess in the cache and no cache 
groups)
3) Additional SQL PK index will not be needed in this case


> Single primary index instead of mulitple per-partition indexes
> --
>
> Key: IGNITE-7027
> URL: https://issues.apache.org/jira/browse/IGNITE-7027
> Project: Ignite
>  Issue Type: Task
>  Components: cache, sql
>Reporter: Vladimir Ozerov
>  Labels: iep-10
>
> Currently we have per-partition primary index. This gives us easy and 
> effective rebalance/recovery capabilities and efficient lookup in key-value 
> mode. 
> However, this doesn't work well for SQL case. We cannot use this index for 
> range scans. Neither we can use it for PK lookups (it is possible to 
> implement, but will be less then optimal due to necessity to build the whole 
> key object).
> The following change is suggested as optional storage mode:
> 1) Single index data structure for all partitions
> 2) Only single key type is allowed (i.e. no mess in the cache and no cache 
> groups)
> 3) Additional SQL PK index will not be needed in this case
> Advantage:
> - No overhead on the second PK index
> Disadvantage:
> - Less efficient rebalance and recovery



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (IGNITE-7027) Single primary index instead of mulitple per-partition indexes

[Vladimir Ozerov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-7027:

Component/s: sql

> Single primary index instead of mulitple per-partition indexes
> --
>
> Key: IGNITE-7027
> URL: https://issues.apache.org/jira/browse/IGNITE-7027
> Project: Ignite
>  Issue Type: Task
>  Components: cache, sql
>Reporter: Vladimir Ozerov
>  Labels: iep-10
>
> Currently we have per-partition primary index. This gives us easy and 
> effective rebalance/recovery capabilities and efficient lookup in key-value 
> mode. 
> However, this doesn't work well for SQL case. We cannot use this index for 
> range scans. Neither we can use it for PK lookups (it is possible to 
> implement, but will be less then optimal due to necessity to build the whole 
> key object).
> The following change is suggested as optional storage mode:
> 1) Single index data structure for all partitions
> 2) Only single key type is allowed (i.e. no mess in the cache and no cache 
> groups)
> 3) Additional SQL PK index will not be needed in this case



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7027) Single primary index instead of mulitple per-partition indexes

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-7027:
---

 Summary: Single primary index instead of mulitple per-partition 
indexes
 Key: IGNITE-7027
 URL: https://issues.apache.org/jira/browse/IGNITE-7027
 Project: Ignite
  Issue Type: Task
  Components: cache
Reporter: Vladimir Ozerov


Currently we have per-partition primary index. This gives us easy and effective 
rebalance/recovery capabilities and efficient lookup in key-value mode. 

However, this doesn't work well for SQL case. We cannot use this index for 
range scans. Neither we can use it for PK lookups (it is possible to implement, 
but will be less then optimal due to necessity to build the whole key object).

The following change is suggested as optional storage mode:
1) Single index data structure for all partitions
2) Only single key type is allowed (i.e. no mess in the cache and no cache 
groups)
3) Additional SQL PK index will not be needed in this case



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (IGNITE-7026) Index-organized data storage format

[Vladimir Ozerov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-7026:

Labels: iep-10  (was: iep-10 sql)

> Index-organized data storage format
> ---
>
> Key: IGNITE-7026
> URL: https://issues.apache.org/jira/browse/IGNITE-7026
> Project: Ignite
>  Issue Type: Task
>  Components: cache, sql
>Reporter: Vladimir Ozerov
>  Labels: iep-10
>
> In SQL *index-organized* table is a type of table format where rows are 
> stored as leafs of a primary key index (sometimes called "clustered index"). 
> In this format data within a single page is sorted in accordance with PK 
> index. All leaves are always sorted as well. 
> Another table format is *heap*. Data is put into arbitrary page with enough 
> space. Free space is tracked using either free-lists or allocation maps. 
> Primary key index is organized in the same way as secondary index - leaf 
> pages contain a kind of row pointer. This is how Ignite currently works. 
> This ticket is aimed to implement index-organized storage format, which will 
> give us the following advantages:
> 1) Fast scans over PK index due to decreased number of page reads and page 
> locks, which is especially important for JOINs and OLAP cases;
> 2) Faster inserts in OLTP workloads due to less number of page updates.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (IGNITE-7027) Single primary index instead of mulitple per-partition indexes

[Vladimir Ozerov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-7027:

Labels: iep-10  (was: )

> Single primary index instead of mulitple per-partition indexes
> --
>
> Key: IGNITE-7027
> URL: https://issues.apache.org/jira/browse/IGNITE-7027
> Project: Ignite
>  Issue Type: Task
>  Components: cache, sql
>Reporter: Vladimir Ozerov
>  Labels: iep-10
>
> Currently we have per-partition primary index. This gives us easy and 
> effective rebalance/recovery capabilities and efficient lookup in key-value 
> mode. 
> However, this doesn't work well for SQL case. We cannot use this index for 
> range scans. Neither we can use it for PK lookups (it is possible to 
> implement, but will be less then optimal due to necessity to build the whole 
> key object).
> The following change is suggested as optional storage mode:
> 1) Single index data structure for all partitions
> 2) Only single key type is allowed (i.e. no mess in the cache and no cache 
> groups)
> 3) Additional SQL PK index will not be needed in this case



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (IGNITE-7026) Index-organized data storage format

[Vladimir Ozerov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-7026:

Labels: iep-10 sql  (was: )

> Index-organized data storage format
> ---
>
> Key: IGNITE-7026
> URL: https://issues.apache.org/jira/browse/IGNITE-7026
> Project: Ignite
>  Issue Type: Task
>  Components: cache, sql
>Reporter: Vladimir Ozerov
>  Labels: iep-10, sql
>
> In SQL *index-organized* table is a type of table format where rows are 
> stored as leafs of a primary key index (sometimes called "clustered index"). 
> In this format data within a single page is sorted in accordance with PK 
> index. All leaves are always sorted as well. 
> Another table format is *heap*. Data is put into arbitrary page with enough 
> space. Free space is tracked using either free-lists or allocation maps. 
> Primary key index is organized in the same way as secondary index - leaf 
> pages contain a kind of row pointer. This is how Ignite currently works. 
> This ticket is aimed to implement index-organized storage format, which will 
> give us the following advantages:
> 1) Fast scans over PK index due to decreased number of page reads and page 
> locks, which is especially important for JOINs and OLAP cases;
> 2) Faster inserts in OLTP workloads due to less number of page updates.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7026) Index-organized data storage format

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-7026:
---

 Summary: Index-organized data storage format
 Key: IGNITE-7026
 URL: https://issues.apache.org/jira/browse/IGNITE-7026
 Project: Ignite
  Issue Type: Task
  Components: cache, sql
Reporter: Vladimir Ozerov


In SQL *index-organized* table is a type of table format where rows are stored 
as leafs of a primary key index (sometimes called "clustered index"). In this 
format data within a single page is sorted in accordance with PK index. All 
leaves are always sorted as well. 

Another table format is *heap*. Data is put into arbitrary page with enough 
space. Free space is tracked using either free-lists or allocation maps. 
Primary key index is organized in the same way as secondary index - leaf pages 
contain a kind of row pointer. This is how Ignite currently works. 

This ticket is aimed to implement index-organized storage format, which will 
give us the following advantages:
1) Fast scans over PK index due to decreased number of page reads and page 
locks, which is especially important for JOINs and OLAP cases;
2) Faster inserts in OLTP workloads due to less number of page updates.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (IGNITE-6934) SQL: evaluate performance of onheap row caching

[Vladimir Ozerov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-6934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov resolved IGNITE-6934.
-
Resolution: Won't Fix

No go for now. Correct long term solution will be focused on minimizing 
deserialization during SQL execution.

> SQL: evaluate performance of onheap row caching
> ---
>
> Key: IGNITE-6934
> URL: https://issues.apache.org/jira/browse/IGNITE-6934
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Reporter: Vladimir Ozerov
>  Labels: performance
>
> Ignite has so-called "on heap cache" feature. When cache entry is accessed, 
> we copy it from offheap to heap and put it into a temporal concurrent hash 
> map ([1], [2]), where it resides during usage. When operation is finished, 
> entry is evicted. This is default behavior which keeps GC pressure low even 
> for large in-memory data sets.
> The downside is that we loose time on copying from offheap to heap. To 
> mitigate this problem user can enable on-heap cache through 
> {{IgniteCache.onheapCacheEnabled}}. In this mode entry will not be evicted 
> from on-heap map, so it can be reused between different operations without 
> additional copying. Eviction rules are managed through eviction policy. 
> Unfortunately, SQL cannot use this optimization. As a result if key or value 
> is large enough, we loose a lot of time on memory copying. And we cannot use 
> current on-heap cache directly, we in SQL operate on row links, rather than 
> on keys. So to apply this optimization to SQL we should either create 
> additional row cache, or hack existing cache somehow.
> As a first step I propose to evaluate the impact with quick and dirty 
> solution:
> 1) Just add another map from link to K-V pair in the same cache, putting all 
> concurrency issues aside.
> 2) Use this cache from SQL engine.
> 3) Measure the impact. 
> [1] {{org.apache.ignite.internal.processors.cache.GridCacheConcurrentMapImpl}}
> [2] 
> {{org.apache.ignite.internal.processors.cache.GridCacheConcurrentMap.CacheMapHolder}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Closed] (IGNITE-6934) SQL: evaluate performance of onheap row caching

[Vladimir Ozerov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-6934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov closed IGNITE-6934.
---

> SQL: evaluate performance of onheap row caching
> ---
>
> Key: IGNITE-6934
> URL: https://issues.apache.org/jira/browse/IGNITE-6934
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Reporter: Vladimir Ozerov
>  Labels: performance
>
> Ignite has so-called "on heap cache" feature. When cache entry is accessed, 
> we copy it from offheap to heap and put it into a temporal concurrent hash 
> map ([1], [2]), where it resides during usage. When operation is finished, 
> entry is evicted. This is default behavior which keeps GC pressure low even 
> for large in-memory data sets.
> The downside is that we loose time on copying from offheap to heap. To 
> mitigate this problem user can enable on-heap cache through 
> {{IgniteCache.onheapCacheEnabled}}. In this mode entry will not be evicted 
> from on-heap map, so it can be reused between different operations without 
> additional copying. Eviction rules are managed through eviction policy. 
> Unfortunately, SQL cannot use this optimization. As a result if key or value 
> is large enough, we loose a lot of time on memory copying. And we cannot use 
> current on-heap cache directly, we in SQL operate on row links, rather than 
> on keys. So to apply this optimization to SQL we should either create 
> additional row cache, or hack existing cache somehow.
> As a first step I propose to evaluate the impact with quick and dirty 
> solution:
> 1) Just add another map from link to K-V pair in the same cache, putting all 
> concurrency issues aside.
> 2) Use this cache from SQL engine.
> 3) Measure the impact. 
> [1] {{org.apache.ignite.internal.processors.cache.GridCacheConcurrentMapImpl}}
> [2] 
> {{org.apache.ignite.internal.processors.cache.GridCacheConcurrentMap.CacheMapHolder}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Closed] (IGNITE-6057) SQL: Full scan should be performed through data pages bypassing primary index

[Vladimir Ozerov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-6057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov closed IGNITE-6057.
---

> SQL: Full scan should be performed through data pages bypassing primary index
> -
>
> Key: IGNITE-6057
> URL: https://issues.apache.org/jira/browse/IGNITE-6057
> Project: Ignite
>  Issue Type: Task
>  Components: persistence, sql
>Affects Versions: 2.1
>Reporter: Vladimir Ozerov
>  Labels: performance
>
> Currently both SQL full scan and {{CREATE INDEX}} commands iterate through 
> primary index to get all existing values. Consider that we have 10 entries 
> per data page on average. In this case we will have to read the same data 
> page 10 times when reaching relevant keys in different parts of index tree. 
> This could be very inefficient on certain workloads.
> We should iterate over data pages directly instead. This way a page with 10 
> entries will be accessed only once. However, we should take cache groups in 
> count - if there are too many entries from other logical caches, this 
> approach could make situation even worse, unless we have a mechanism to skip 
> unnecessary entries (or the whole pages!) efficiently.
> Probably we should develop a cost-based model, which will take in count the 
> following statistics:
> 1) Average entry size. The longer the entry, the lesser the benefit. 
> Especially if overflow pages are used frequently. 
> 2) Cache groups. Ideally, we should estimate number of entries from all 
> logical caches. The more entries from other caches, the lesser the benefit.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (IGNITE-6057) SQL: Full scan should be performed through data pages bypassing primary index

[Vladimir Ozerov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-6057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov resolved IGNITE-6057.
-
Resolution: Won't Fix

This should be handled with clustered index in general case.

> SQL: Full scan should be performed through data pages bypassing primary index
> -
>
> Key: IGNITE-6057
> URL: https://issues.apache.org/jira/browse/IGNITE-6057
> Project: Ignite
>  Issue Type: Task
>  Components: persistence, sql
>Affects Versions: 2.1
>Reporter: Vladimir Ozerov
>  Labels: performance
>
> Currently both SQL full scan and {{CREATE INDEX}} commands iterate through 
> primary index to get all existing values. Consider that we have 10 entries 
> per data page on average. In this case we will have to read the same data 
> page 10 times when reaching relevant keys in different parts of index tree. 
> This could be very inefficient on certain workloads.
> We should iterate over data pages directly instead. This way a page with 10 
> entries will be accessed only once. However, we should take cache groups in 
> count - if there are too many entries from other logical caches, this 
> approach could make situation even worse, unless we have a mechanism to skip 
> unnecessary entries (or the whole pages!) efficiently.
> Probably we should develop a cost-based model, which will take in count the 
> following statistics:
> 1) Average entry size. The longer the entry, the lesser the benefit. 
> Especially if overflow pages are used frequently. 
> 2) Cache groups. Ideally, we should estimate number of entries from all 
> logical caches. The more entries from other caches, the lesser the benefit.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-584) Need to make sure that scan query returns consistent results on topology changes

[Stanilovsky Evgeny (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267336#comment-16267336
 ] 

Stanilovsky Evgeny commented on IGNITE-584:
---

we need to set _keepBinary=false_ on _onEntryUpdated_ for proper unwrap non 
_Externalizable_ class _CollocatedSetItemKey_ and further _if (key0 instanceof 
SetItemKey)_ check, otherwise we need to implement _CollocatedSetItemKey_  as 
_Externalizable_ too and replace from registerPredefinedType.

> Need to make sure that scan query returns consistent results on topology 
> changes
> 
>
> Key: IGNITE-584
> URL: https://issues.apache.org/jira/browse/IGNITE-584
> Project: Ignite
>  Issue Type: Sub-task
>  Components: data structures
>Affects Versions: 1.9, 2.0, 2.1
>Reporter: Artem Shutak
>Assignee: Semen Boikov
>  Labels: MakeTeamcityGreenAgain, Muted_test
> Fix For: 2.4
>
>
> Consistent results on topology changes was implemented for sql queries, but 
> looks like it still does not work for scan queries.
> This affects 'cache set' tests since set uses scan query for set iteration 
> (to be unmuted on TC): 
> GridCacheSetAbstractSelfTest testNodeJoinsAndLeaves and 
> testNodeJoinsAndLeavesCollocated; 
> Also see todos here GridCacheSetFailoverAbstractSelfTest



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (IGNITE-6846) Add metrics for entry processor invocations

[Alexey Kuznetsov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264488#comment-16264488
 ] 

Alexey Kuznetsov edited comment on IGNITE-6846 at 11/27/17 7:14 PM:


1)When entry is updated, both 'invoke update' and get\put\remove metrics can be 
updated.
2)When entry processor is invoked, but cache entry isn't affected, then invoke 
metric isn't incremented.
3)'invoke update' metrics incremented only when cache entry value is changed(or 
removed) by entry processor(not just when entry processor is invoked).
4)If entry processor process(...) is called once, and multiple entries(perhaps 
on different nodes) are affected, then local invoke metric is incremented only 
once, cluster global metric is incremented once, local metrics on other nodes 
are zero.
5)Read-only 'invoke' metric is incremented if processor invoke operation is 
called without executing entry.setValue(...) (even calling entry.getValue() can 
be omit), processor can return non-null value.Also, cache might be empty while 
processor invocation.
6)'invoke removal' metric is incremented only if old cache value exists and 
removal succeedes.
7)The following metrics are going to be added : total number of invokes, number 
of invoke updates, number of invoke removals, number of read-only invokes, 
min\max\avg number of invocations(all invocations).


was (Author: alexey kuznetsov):
1)When entry is updated, both 'invoke update' and get\put\remove metrics can be 
updated.
2)When entry processor is invoked, but cache entry isn't affected, then invoke 
metric isn't incremented.
3)'invoke update' metrics incremented only when cache entry value is changed(or 
removed) by entry processor(not just when entry processor is invoked).
4)If entry processor process(...) is called once, and multiple entries(perhaps 
on different nodes) are affected, then local invoke metric is incremented only 
once, cluster global metric is incremented once, local metrics on other nodes 
are zero.
5)Read-only 'invoke' metric is incremented if processor invoke operation is 
called without executing entry.setValue(...) (even calling entry.getValue() can 
be omit), processor can return non-null value.Also, cache might be empty while 
processor invocation.
6)The following metrics are going to be added : total number of invokes, number 
of invoke updates, number of invoke removals, number of read-only invokes, 
min\max\avg number of invocations(all invocations).

> Add metrics for entry processor invocations
> ---
>
> Key: IGNITE-6846
> URL: https://issues.apache.org/jira/browse/IGNITE-6846
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.3
>Reporter: Valentin Kulichenko
>Assignee: Alexey Kuznetsov
>Priority: Critical
>  Labels: iep-6
> Fix For: 2.4
>
>
> {{CacheMetrics}} object has multiple metrics for various cache operations 
> like {{get}}, {{put}} and {{remove}}, but nothing for 
> {{invoke}}/{{EntryProcessor}}. It makes sense to add such metrics, for 
> example:
> * Total number of `invoke` operations executed.
> * Number of `invoke` operations that included updates.
> * Number of read-only `invoke` operations.
> * Min/max/avg execution time.
> * ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (IGNITE-6846) Add metrics for entry processor invocations

[Alexey Kuznetsov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264488#comment-16264488
 ] 

Alexey Kuznetsov edited comment on IGNITE-6846 at 11/27/17 6:20 PM:


1)When entry is updated, both 'invoke update' and get\put\remove metrics can be 
updated.
2)When entry processor is invoked, but cache entry isn't affected, then invoke 
metric isn't incremented.
3)'invoke update' metrics incremented only when cache entry value is changed(or 
removed) by entry processor(not just when entry processor is invoked).
4)If entry processor process(...) is called once, and multiple entries(perhaps 
on different nodes) are affected, then local invoke metric is incremented only 
once, cluster global metric is incremented once, local metrics on other nodes 
are zero.
5)Read-only 'invoke' metric is incremented if processor invoke operation is 
called without executing entry.setValue(...) (even calling entry.getValue() can 
be omit), processor can return non-null value.Also, cache might be empty while 
processor invocation.
6)The following metrics are going to be added : total number of invokes, number 
of invoke updates, number of invoke removals, number of read-only invokes, 
min\max\avg number of invocations(all invocations).


was (Author: alexey kuznetsov):
1)When entry is updated, both 'invoke update' and get\put\remove metrics can be 
updated.
2)When entry processor is invoked, but cache entry isn't affected, then invoke 
metric isn't incremented.
3)'invoke update' metrics incremented only when cache entry value is changed(or 
removed) by entry processor(not just when entry processor is invoked).
4)If entry processor process(...) is called once, and multiple entries(perhaps 
on different nodes) are affected, then local invoke metric is incremented only 
once, cluster global metric is incremented once, local metrics on other nodes 
are zero.
5)Read-only 'invoke' metric is incremented if processor invoke operation is 
called without executing entry.setValue(...) (even calling entry.getValue() can 
be omit), processor can return non-null value.Also, cache might be empty while 
processor invocation.
6)The following metrics are going to be added : total number of invokes, number 
of invoke updates(including removal), number of read-only invokes, min\max\avg 
number of invocations(all invocations).

> Add metrics for entry processor invocations
> ---
>
> Key: IGNITE-6846
> URL: https://issues.apache.org/jira/browse/IGNITE-6846
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.3
>Reporter: Valentin Kulichenko
>Assignee: Alexey Kuznetsov
>Priority: Critical
>  Labels: iep-6
> Fix For: 2.4
>
>
> {{CacheMetrics}} object has multiple metrics for various cache operations 
> like {{get}}, {{put}} and {{remove}}, but nothing for 
> {{invoke}}/{{EntryProcessor}}. It makes sense to add such metrics, for 
> example:
> * Total number of `invoke` operations executed.
> * Number of `invoke` operations that included updates.
> * Number of read-only `invoke` operations.
> * Min/max/avg execution time.
> * ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-5960) Ignite Continuous Query (Queries 3): CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic is flaky

[Alexey Kuznetsov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267062#comment-16267062
 ] 

Alexey Kuznetsov commented on IGNITE-5960:
--

[~sunnychanclsa] I have already created [Pull 
request|https://github.com/apache/ignite/pull/2728]
This PR must fix your issue.

> Ignite Continuous Query (Queries 3): 
> CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic
>  is flaky
> -
>
> Key: IGNITE-5960
> URL: https://issues.apache.org/jira/browse/IGNITE-5960
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Sergey Chugunov
>Assignee: Alexey Kuznetsov
>  Labels: MakeTeamcityGreenAgain, test-failure
> Fix For: 2.4
>
>
> According to [TC 
> history|http://ci.ignite.apache.org/project.html?projectId=Ignite20Tests=6546112007182082024=testDetails_Ignite20Tests=%3Cdefault%3E]
>  test is flaky.
> It is possible to reproduce it locally, sample run shows 9 failed tests out 
> of 30 overall executed.
> Test fails with jUnit assertion check: 
> {noformat}
> junit.framework.AssertionFailedError: 
> Expected :1
> Actual   :0
>  
>   at junit.framework.Assert.fail(Assert.java:57)
>   at junit.framework.Assert.failNotEquals(Assert.java:329)
>   at junit.framework.Assert.assertEquals(Assert.java:78)
>   at junit.framework.Assert.assertEquals(Assert.java:234)
>   at junit.framework.Assert.assertEquals(Assert.java:241)
>   at junit.framework.TestCase.assertEquals(TestCase.java:409)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.concurrentUpdatesAndQueryStart(CacheContinuousQueryConcurrentPartitionUpdateTest.java:385)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.testConcurrentUpdatesAndQueryStartTx(CacheContinuousQueryConcurrentPartitionUpdateTest.java:245)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at junit.framework.TestCase.runTest(TestCase.java:176)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2000)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:132)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1915)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-6929) Preserve mvcc versions on index rebuild

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267058#comment-16267058
 ] 

ASF GitHub Bot commented on IGNITE-6929:


Github user alexpaschenko closed the pull request at:

https://github.com/apache/ignite/pull/3090


> Preserve mvcc versions on index rebuild
> ---
>
> Key: IGNITE-6929
> URL: https://issues.apache.org/jira/browse/IGNITE-6929
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Reporter: Igor Seliverstov
>Assignee: Alexander Paschenko
>
> Currently we may loose next available version for an index row on index 
> rebuild 
> ({{org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.CacheDataStoreImpl#updateIndexes}})



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-6992) Ignite MR problem with accessing hdfs with enabled Kerberos

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267020#comment-16267020
 ] 

ASF GitHub Bot commented on IGNITE-6992:


GitHub user ezhuravl opened a pull request:

https://github.com/apache/ignite/pull/3097

IGNITE-6992 FIX Ignite MR problem with accessing hdfs with enabled Ke…

…rberos

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6992

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3097.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3097


commit 683233bfbb72c35f0ecdcc5703d7d8654ffcc72f
Author: ezhuravl 
Date:   2017-11-27T16:34:32Z

IGNITE-6992 FIX Ignite MR problem with accessing hdfs with enabled Kerberos




> Ignite MR problem with accessing hdfs with enabled Kerberos
> ---
>
> Key: IGNITE-6992
> URL: https://issues.apache.org/jira/browse/IGNITE-6992
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.0, 2.1, 2.3
>Reporter: Evgenii Zhuravlev
>Assignee: Evgenii Zhuravlev
>
> {code:java}
> class org.apache.ignite.IgniteCheckedException: SIMPLE authentication is not 
> enabled.  Available:[TOKEN, KERBEROS]
>   at 
> org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2JobResourceManager.prepareJobEnvironment(HadoopV2JobResourceManager.java:169)
>   at 
> org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2Job.initialize(HadoopV2Job.java:328)
>   at 
> org.apache.ignite.internal.processors.hadoop.jobtracker.HadoopJobTracker.job(HadoopJobTracker.java:1141)
>   at 
> org.apache.ignite.internal.processors.hadoop.jobtracker.HadoopJobTracker.submit(HadoopJobTracker.java:318)
>   at 
> org.apache.ignite.internal.processors.hadoop.HadoopProcessor.submit(HadoopProcessor.java:173)
>   at 
> org.apache.ignite.internal.processors.hadoop.HadoopImpl.submit(HadoopImpl.java:69)
>   at 
> org.apache.ignite.internal.processors.hadoop.proto.HadoopProtocolSubmitJobTask.run(HadoopProtocolSubmitJobTask.java:50)
>   at 
> org.apache.ignite.internal.processors.hadoop.proto.HadoopProtocolSubmitJobTask.run(HadoopProtocolSubmitJobTask.java:33)
>   at 
> org.apache.ignite.internal.processors.hadoop.proto.HadoopProtocolTaskAdapter$Job.execute(HadoopProtocolTaskAdapter.java:101)
>   at 
> org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:566)
>   at 
> org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6629)
>   at 
> org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:560)
>   at 
> org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:489)
>   at 
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
>   at 
> org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1181)
>   at 
> org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1908)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1562)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1190)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1097)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.security.AccessControlException: SIMPLE 
> authentication is not enabled.  Available:[TOKEN, KERBEROS]
>   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>   at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>   at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>   at 
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
>   at 
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
>   at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2110)
>   at 
> 

[jira] [Commented] (IGNITE-6872) Linear regression should implement Model API

[Oleg Ignatenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267012#comment-16267012
 ] 

Oleg Ignatenko commented on IGNITE-6872:


[~chief] - I drafted implementation with Trainer API in branch 
[ignite-6872-2|https://github.com/gridgain/apache-ignite/tree/ignite-6872-2], 
would appreciate if you take a look.

> Linear regression should implement Model API
> 
>
> Key: IGNITE-6872
> URL: https://issues.apache.org/jira/browse/IGNITE-6872
> Project: Ignite
>  Issue Type: Task
>  Components: ml
>Reporter: Oleg Ignatenko
>Assignee: Oleg Ignatenko
> Fix For: 2.4
>
>
> When linear regression was originally implemented per IGNITE-5012 we had no 
> Model API.
> Now that this API is available (merged into master with IGNITE-5218) lin 
> regression needs to adapt to implement it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (IGNITE-532) Implement IgniteAkkaStreamer to stream data from Akka actors.

[Maksim Kozlov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maksim Kozlov reassigned IGNITE-532:


Assignee: (was: Maksim Kozlov)

> Implement IgniteAkkaStreamer to stream data from Akka actors.
> -
>
> Key: IGNITE-532
> URL: https://issues.apache.org/jira/browse/IGNITE-532
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Akka|http://akka.io/] for more information. Given that Akka is a Scala 
> frameworks, this streamer should be available in Scala.
> We should create {{IgniteAkkaStreamer}} which will consume messages from Akka 
> Actors and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> * Convert data from Akka to Ignite using an optional pluggable converter. If 
> not provided, then we should have some default mechanism.
> * Specify the cache name for the Ignite cache to load data into.
> * Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-4910) Add copyOnRead property for near cache.

[Maksim Kozlov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266990#comment-16266990
 ] 

Maksim Kozlov commented on IGNITE-4910:
---

[~vkulichenko], at the moment I'm not working on this ticket.

> Add copyOnRead property for near cache.
> ---
>
> Key: IGNITE-4910
> URL: https://issues.apache.org/jira/browse/IGNITE-4910
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.1
>Reporter: Maksim Kozlov
>
> Add {{setCopyOnRead}} to {{NearCacheConfiguration}} class.
> Continuation of the task from the ticket 
> [https://issues.apache.org/jira/browse/IGNITE-2558].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (IGNITE-3878) Add isPrimary() and isBackup() methods on CacheQUeryEntryEvent

[Maksim Kozlov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maksim Kozlov reassigned IGNITE-3878:
-

Assignee: (was: Maksim Kozlov)

> Add isPrimary() and isBackup() methods on CacheQUeryEntryEvent
> --
>
> Key: IGNITE-3878
> URL: https://issues.apache.org/jira/browse/IGNITE-3878
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 1.7
>Reporter: Nikolay Tikhonov
>Priority: Minor
>
> In some cases useful know where (on primary or backup nodes) was invoked  a 
> continuous query filter.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (IGNITE-4910) Add copyOnRead property for near cache.

[Maksim Kozlov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-4910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maksim Kozlov reassigned IGNITE-4910:
-

Assignee: (was: Maksim Kozlov)

> Add copyOnRead property for near cache.
> ---
>
> Key: IGNITE-4910
> URL: https://issues.apache.org/jira/browse/IGNITE-4910
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.1
>Reporter: Maksim Kozlov
>
> Add {{setCopyOnRead}} to {{NearCacheConfiguration}} class.
> Continuation of the task from the ticket 
> [https://issues.apache.org/jira/browse/IGNITE-2558].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (IGNITE-7025) Implement different strategies to fill missed data in LabeledDataset during loading from file

[Aleksey Zinoviev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Zinoviev updated IGNITE-7025:
-
Description: 
For example, it can be four strategies 

* Fill missed value with zero or empty string or default value for categorical 
features = ZERO
* Fill missed value with mean on column/Requires an additional time to 
calculate = MEAN
* Fill missed value with mode on column. Requires an additional time to 
calculate = MODE
* Deletes observation with missed values. Transforms dataset and changes 
indexing = DELETE



  was:
For example, it can be four strategies 

public enum FillMissingValueWith {
   
* /**
 * Fill missed value with zero or empty string or default value for 
categorical features
 */
ZERO,
/**
 * Fill missed value with mean on column
 * Requires an additional time to calculate
 */
MEAN,
/**
 * Fill missed value with mode on column
 * Requires an additional time to calculate
 */
MODE,
/**
 * Deletes observation with missed values
 * Transforms dataset and changes indexing
 */
DELETE
}



> Implement different strategies to fill missed data in LabeledDataset during 
> loading from file
> -
>
> Key: IGNITE-7025
> URL: https://issues.apache.org/jira/browse/IGNITE-7025
> Project: Ignite
>  Issue Type: Task
>  Components: ml
>Reporter: Aleksey Zinoviev
>Assignee: Aleksey Zinoviev
>Priority: Minor
>
> For example, it can be four strategies 
> * Fill missed value with zero or empty string or default value for 
> categorical features = ZERO
> * Fill missed value with mean on column/Requires an additional time to 
> calculate = MEAN
> * Fill missed value with mode on column. Requires an additional time to 
> calculate = MODE
> * Deletes observation with missed values. Transforms dataset and changes 
> indexing = DELETE



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (IGNITE-7025) Implement different strategies to fill missed data in LabeledDataset during loading from file

[Aleksey Zinoviev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Zinoviev updated IGNITE-7025:
-
Description: 
For example, it can be four strategies 

public enum FillMissingValueWith {
   
* /**
 * Fill missed value with zero or empty string or default value for 
categorical features
 */
ZERO,
/**
 * Fill missed value with mean on column
 * Requires an additional time to calculate
 */
MEAN,
/**
 * Fill missed value with mode on column
 * Requires an additional time to calculate
 */
MODE,
/**
 * Deletes observation with missed values
 * Transforms dataset and changes indexing
 */
DELETE
}


  was:
For example, it can be four strategies 

public enum FillMissingValueWith {
/**
 * Fill missed value with zero or empty string or default value for 
categorical features
 */
ZERO,
/**
 * Fill missed value with mean on column
 * Requires an additional time to calculate
 */
MEAN,
/**
 * Fill missed value with mode on column
 * Requires an additional time to calculate
 */
MODE,
/**
 * Deletes observation with missed values
 * Transforms dataset and changes indexing
 */
DELETE
}



> Implement different strategies to fill missed data in LabeledDataset during 
> loading from file
> -
>
> Key: IGNITE-7025
> URL: https://issues.apache.org/jira/browse/IGNITE-7025
> Project: Ignite
>  Issue Type: Task
>  Components: ml
>Reporter: Aleksey Zinoviev
>Assignee: Aleksey Zinoviev
>Priority: Minor
>
> For example, it can be four strategies 
> public enum FillMissingValueWith {
>
> * /**
>  * Fill missed value with zero or empty string or default value for 
> categorical features
>  */
> ZERO,
> /**
>  * Fill missed value with mean on column
>  * Requires an additional time to calculate
>  */
> MEAN,
> /**
>  * Fill missed value with mode on column
>  * Requires an additional time to calculate
>  */
> MODE,
> /**
>  * Deletes observation with missed values
>  * Transforms dataset and changes indexing
>  */
> DELETE
> }



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7025) Implement different strategies to fill missed data in LabeledDataset during loading from file

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-7025:


 Summary: Implement different strategies to fill missed data in 
LabeledDataset during loading from file
 Key: IGNITE-7025
 URL: https://issues.apache.org/jira/browse/IGNITE-7025
 Project: Ignite
  Issue Type: Task
  Components: ml
Reporter: Aleksey Zinoviev
Assignee: Aleksey Zinoviev
Priority: Minor


For example, it can be four strategies 

public enum FillMissingValueWith {
/**
 * Fill missed value with zero or empty string or default value for 
categorical features
 */
ZERO,
/**
 * Fill missed value with mean on column
 * Requires an additional time to calculate
 */
MEAN,
/**
 * Fill missed value with mode on column
 * Requires an additional time to calculate
 */
MODE,
/**
 * Deletes observation with missed values
 * Transforms dataset and changes indexing
 */
DELETE
}




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-7013) .NET: Ignite does not start on macOS

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266948#comment-16266948
 ] 

Pavel Tupitsyn commented on IGNITE-7013:


Finally was able to start Ignite.NET on macOS Sierra:
* libjvm.dylib support added
* {{DllImport}} refactored to {{dlopen}} + {{dlsym}} calls

Will add separate JAVA_HOME resolving code by looking into 
{{/Library/Java/JavaVirtualMachines/}} folder.

> .NET: Ignite does not start on macOS
> 
>
> Key: IGNITE-7013
> URL: https://issues.apache.org/jira/browse/IGNITE-7013
> Project: Ignite
>  Issue Type: Bug
>  Components: platforms
>Affects Versions: 2.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .NET, xplat
> Fix For: 2.4
>
>
> Looks like dlopen code is incorrect for macOS:
> {code}
> Unhandled Exception: System.DllNotFoundException: Unable to load DLL 
> 'libcoreclr.so': The specified module or one of its dependencies could not be 
> found.
>  (Exception from HRESULT: 0x8007007E)
>at 
> Apache.Ignite.Core.Impl.Unmanaged.DllLoader.NativeMethodsCore.dlopen(String 
> filename, Int32 flags)
>at Apache.Ignite.Core.Impl.Unmanaged.DllLoader.Load(String dllPath)
>at Apache.Ignite.Core.Impl.IgniteUtils.LoadDll(String filePath, String 
> simpleName)
>at Apache.Ignite.Core.Impl.IgniteUtils.LoadJvmDll(String configJvmDllPath, 
> ILogger log)
>at Apache.Ignite.Core.Impl.IgniteUtils.LoadDlls(String configJvmDllPath, 
> ILogger log)
>at Apache.Ignite.Core.Ignition.Start(IgniteConfiguration cfg)
>at ignite_nuget_test.Program.Main(String[] args) in 
> /Users/vveider/Development/VCS/Git/ignite-dotnetcore-demo/Program.cs:line 17
> {code}
> Steps to reproduce:
> {code}
> git clone https://github.com/ptupitsyn/ignite-dotnetcore-demo.git
> cd ignite-dotnetcore-demo
> dotnet run
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (IGNITE-7016) Avoid fsync on WAL segment header write in non-default mode

[Alexey Goncharuk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Goncharuk reassigned IGNITE-7016:


Assignee: Alexey Goncharuk

> Avoid fsync on WAL segment header write in non-default mode
> ---
>
> Key: IGNITE-7016
> URL: https://issues.apache.org/jira/browse/IGNITE-7016
> Project: Ignite
>  Issue Type: Task
>Affects Versions: 2.3
>Reporter: Alexey Goncharuk
>Assignee: Alexey Goncharuk
> Fix For: 2.4
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (IGNITE-7000) .NET: Document .NET Core support and running on Linux / macOs

[Pavel Tupitsyn (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn updated IGNITE-7000:
---
Description: 
Create a documentation page about running Ignite.NET under .NET Core and/or 
Linux. Things to mention:
* Exceptions and delegates can't be serialized under .NET Core: 
https://github.com/dotnet/corefx/issues/24424
* Build warning - suppress {{NU1701}} with 
{{NU1701}}
* Java home, dll paths, etc
* Publish and deploy (copy libs folder to {{dotnet publish}} dir)
* macOs: document {{No Java runtime present, requesting install.}} issue 
https://bugs.openjdk.java.net/browse/JDK-7131356, 
https://github.com/s-u/rJava/issues/37

  was:
Create a documentation page about running Ignite.NET under .NET Core and/or 
Linux. Things to mention:
* Exceptions and delegates can't be serialized under .NET Core: 
https://github.com/dotnet/corefx/issues/24424
* Build warning - suppress {{NU1701}} with 
{{NU1701}}
* Java home, dll paths, etc
* Publish and deploy (copy libs folder to {{dotnet publish}} dir)


> .NET: Document .NET Core support and running on Linux / macOs
> -
>
> Key: IGNITE-7000
> URL: https://issues.apache.org/jira/browse/IGNITE-7000
> Project: Ignite
>  Issue Type: Task
>  Components: documentation, platforms
>Reporter: Pavel Tupitsyn
>Priority: Minor
>  Labels: .NET, xplat
> Fix For: 2.4
>
>
> Create a documentation page about running Ignite.NET under .NET Core and/or 
> Linux. Things to mention:
> * Exceptions and delegates can't be serialized under .NET Core: 
> https://github.com/dotnet/corefx/issues/24424
> * Build warning - suppress {{NU1701}} with 
> {{NU1701}}
> * Java home, dll paths, etc
> * Publish and deploy (copy libs folder to {{dotnet publish}} dir)
> * macOs: document {{No Java runtime present, requesting install.}} issue 
> https://bugs.openjdk.java.net/browse/JDK-7131356, 
> https://github.com/s-u/rJava/issues/37



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (IGNITE-7000) .NET: Document .NET Core support and running on Linux / macOs

[Pavel Tupitsyn (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn updated IGNITE-7000:
---
Summary: .NET: Document .NET Core support and running on Linux / macOs  
(was: .NET: Document .NET Core support and running on Linux)

> .NET: Document .NET Core support and running on Linux / macOs
> -
>
> Key: IGNITE-7000
> URL: https://issues.apache.org/jira/browse/IGNITE-7000
> Project: Ignite
>  Issue Type: Task
>  Components: documentation, platforms
>Reporter: Pavel Tupitsyn
>Priority: Minor
>  Labels: .NET, xplat
> Fix For: 2.4
>
>
> Create a documentation page about running Ignite.NET under .NET Core and/or 
> Linux. Things to mention:
> * Exceptions and delegates can't be serialized under .NET Core: 
> https://github.com/dotnet/corefx/issues/24424
> * Build warning - suppress {{NU1701}} with 
> {{NU1701}}
> * Java home, dll paths, etc
> * Publish and deploy (copy libs folder to {{dotnet publish}} dir)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-7013) .NET: Ignite does not start on macOS

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266924#comment-16266924
 ] 

Pavel Tupitsyn commented on IGNITE-7013:


Very "fun" bug: https://bugs.openjdk.java.net/browse/JDK-7131356
Users may have to update Java to certain version or install Java 6.

> .NET: Ignite does not start on macOS
> 
>
> Key: IGNITE-7013
> URL: https://issues.apache.org/jira/browse/IGNITE-7013
> Project: Ignite
>  Issue Type: Bug
>  Components: platforms
>Affects Versions: 2.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .NET, xplat
> Fix For: 2.4
>
>
> Looks like dlopen code is incorrect for macOS:
> {code}
> Unhandled Exception: System.DllNotFoundException: Unable to load DLL 
> 'libcoreclr.so': The specified module or one of its dependencies could not be 
> found.
>  (Exception from HRESULT: 0x8007007E)
>at 
> Apache.Ignite.Core.Impl.Unmanaged.DllLoader.NativeMethodsCore.dlopen(String 
> filename, Int32 flags)
>at Apache.Ignite.Core.Impl.Unmanaged.DllLoader.Load(String dllPath)
>at Apache.Ignite.Core.Impl.IgniteUtils.LoadDll(String filePath, String 
> simpleName)
>at Apache.Ignite.Core.Impl.IgniteUtils.LoadJvmDll(String configJvmDllPath, 
> ILogger log)
>at Apache.Ignite.Core.Impl.IgniteUtils.LoadDlls(String configJvmDllPath, 
> ILogger log)
>at Apache.Ignite.Core.Ignition.Start(IgniteConfiguration cfg)
>at ignite_nuget_test.Program.Main(String[] args) in 
> /Users/vveider/Development/VCS/Git/ignite-dotnetcore-demo/Program.cs:line 17
> {code}
> Steps to reproduce:
> {code}
> git clone https://github.com/ptupitsyn/ignite-dotnetcore-demo.git
> cd ignite-dotnetcore-demo
> dotnet run
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7024) Introduce some kind of network compression

[Amelchev Nikita (JIRA)]

Amelchev Nikita created IGNITE-7024:
---

 Summary: Introduce some kind of network compression
 Key: IGNITE-7024
 URL: https://issues.apache.org/jira/browse/IGNITE-7024
 Project: Ignite
  Issue Type: New Feature
Reporter: Amelchev Nikita
Assignee: Amelchev Nikita


Introduce some kind of pluggable compression at network level

The main idea is using in-line compression and writes encoded bytes in
network channel by bytes array buffer. It allows us avoiding expensive
memory allocation.

A solution may be implemented at TcpCommunicationSpi level.
For example, introduce Compressor interface which will allow us to describe our 
compression strategy, for example, exclude some small messages, choose 
compression algorithm and other…



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-6406) SQL: CREATE INDEX should fill index from multiple threads

[Roman Kondakov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266892#comment-16266892
 ] 

Roman Kondakov commented on IGNITE-6406:


[~vozerov] please review, tests are ok.

> SQL: CREATE INDEX should fill index from multiple threads
> -
>
> Key: IGNITE-6406
> URL: https://issues.apache.org/jira/browse/IGNITE-6406
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 2.1
>Reporter: Vladimir Ozerov
>Assignee: Roman Kondakov
>  Labels: iep-1, performance
> Fix For: 2.4
>
>
> Currently when {{CREATE INDEX}} command is executed, we fill new index from a 
> single thread. Low CPU, long create time. We can trade off CPU vs time by 
> adding more threads. 
> Probably number of threads should be specified in {{CREATE INDEX}} command as 
> a hint.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (IGNITE-4192) SQL TX: JDBC driver support

[Vladimir Ozerov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266845#comment-16266845
 ] 

Vladimir Ozerov edited comment on IGNITE-4192 at 11/27/17 2:06 PM:
---

Design considerations:
1) We should support common syntax forms:
{{START TRANSACTION}}
{{BEGIN | WORK | TRANSACTION}}
{{COMMIT | TRANSACTION}}
{{ROLLBACK | TRANSACTION}}
2) Nested transactions should be prohibited
3) Proper behavior in case of {{autoCommit=false}}
4) Throw exception if MVCC is disabled on the server side


was (Author: vozerov):
Design considerations:
1) We should support common syntax forms:
{{START TRANSACTION}}
{{BEGIN | WORK | TRANSACTION}}
{{COMMIT | TRANSACTION}}
{{ROLLBACK | TRANSACTION}}
2) Nested transactions should be prohibited
3) Proper behavior in case of {{autoCommit=false}}

> SQL TX: JDBC driver support
> ---
>
> Key: IGNITE-4192
> URL: https://issues.apache.org/jira/browse/IGNITE-4192
> Project: Ignite
>  Issue Type: Task
>  Components: jdbc
>Reporter: Denis Magda
>  Labels: iep-3
> Fix For: 2.4
>
>
> To support execution of DML and SELECT statements inside of a transaction 
> started from JDBC driver side.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (IGNITE-4192) SQL TX: JDBC driver support

[Vladimir Ozerov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266845#comment-16266845
 ] 

Vladimir Ozerov edited comment on IGNITE-4192 at 11/27/17 2:05 PM:
---

Design considerations:
1) We should support common syntax forms:
{{START TRANSACTION}}
{{BEGIN | WORK | TRANSACTION}}
{{COMMIT | TRANSACTION}}
{{ROLLBACK | TRANSACTION}}
2) Nested transactions should be prohibited
3) Proper behavior in case of {{autoCommit=false}}


was (Author: vozerov):
Design considerations:
1) We should support common syntax forms:
{{START TRANSACTION}}
{{BEGIN | WORK | TRANSACTION}}
{{COMMIT | TRANSACTION}}
{{ROLLBACK | TRANSACTION}}
2) Nested transactions should be prohibited

> SQL TX: JDBC driver support
> ---
>
> Key: IGNITE-4192
> URL: https://issues.apache.org/jira/browse/IGNITE-4192
> Project: Ignite
>  Issue Type: Task
>  Components: jdbc
>Reporter: Denis Magda
>  Labels: iep-3
> Fix For: 2.4
>
>
> To support execution of DML and SELECT statements inside of a transaction 
> started from JDBC driver side.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (IGNITE-4192) SQL TX: JDBC driver support

[Vladimir Ozerov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266845#comment-16266845
 ] 

Vladimir Ozerov edited comment on IGNITE-4192 at 11/27/17 2:04 PM:
---

Design considerations:
1) We should support common syntax forms:
{{START TRANSACTION}}
{{BEGIN | WORK | TRANSACTION}}
{{COMMIT | TRANSACTION}}
{{ROLLBACK | TRANSACTION}}
2) Nested transactions should be prohibited


was (Author: vozerov):
Design considerations:
1) We should support common syntax forms:
{{START TRANSACTION}}
{{{BEGIN | WORK | TRANSACTION}}
{{COMMIT | TRANSACTION}}
{{ROLLBACK | TRANSACTION}}
2) Nested transactions should be prohibited

> SQL TX: JDBC driver support
> ---
>
> Key: IGNITE-4192
> URL: https://issues.apache.org/jira/browse/IGNITE-4192
> Project: Ignite
>  Issue Type: Task
>  Components: jdbc
>Reporter: Denis Magda
>  Labels: iep-3
> Fix For: 2.4
>
>
> To support execution of DML and SELECT statements inside of a transaction 
> started from JDBC driver side.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-4192) SQL TX: JDBC driver support

[Vladimir Ozerov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266845#comment-16266845
 ] 

Vladimir Ozerov commented on IGNITE-4192:
-

Design considerations:
1) We should support common syntax forms:
{{START TRANSACTION}}
{{{BEGIN | WORK | TRANSACTION}}
{{COMMIT | TRANSACTION}}
{{ROLLBACK | TRANSACTION}}
2) Nested transactions should be prohibited

> SQL TX: JDBC driver support
> ---
>
> Key: IGNITE-4192
> URL: https://issues.apache.org/jira/browse/IGNITE-4192
> Project: Ignite
>  Issue Type: Task
>  Components: jdbc
>Reporter: Denis Magda
>  Labels: iep-3
> Fix For: 2.4
>
>
> To support execution of DML and SELECT statements inside of a transaction 
> started from JDBC driver side.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (IGNITE-7023) In visorcmd I cannot list certain cache contents located on certain node.

[Galinger Vladimir (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Galinger Vladimir updated IGNITE-7023:
--
Attachment: bug_screenshot.jpg

attached is a screenshot that demonstrates the command and it's erroneous output

> In visorcmd I cannot list certain cache contents located on certain node.
> -
>
> Key: IGNITE-7023
> URL: https://issues.apache.org/jira/browse/IGNITE-7023
> Project: Ignite
>  Issue Type: Bug
>  Components: visor
>Affects Versions: 2.3
>Reporter: Galinger Vladimir
> Attachments: bug_screenshot.jpg
>
>
> When testing affinity collocation example, I cannot see what keys of certain 
> *partitioned* cache are located on certain node.
> When I run something like this:
> {noformat}
> cache -scan -c=CacheQueryExampleOrganizations -id8=5F52C8E4
> {noformat}
> it still outputs all entries from the cache, not regarding the partitioning.
> The issue is simple to reproduce, I slightly modified the 
> org.apache.ignite.examples.datagrid.CacheQueryExample, adding 3 more 
> organizations and starting additional 4 nodes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7023) In visorcmd I cannot list certain cache contents located on certain node.

[Galinger Vladimir (JIRA)]

Galinger Vladimir created IGNITE-7023:
-

 Summary: In visorcmd I cannot list certain cache contents located 
on certain node.
 Key: IGNITE-7023
 URL: https://issues.apache.org/jira/browse/IGNITE-7023
 Project: Ignite
  Issue Type: Bug
  Components: visor
Affects Versions: 2.3
Reporter: Galinger Vladimir


When testing affinity collocation example, I cannot see what keys of certain 
*partitioned* cache are located on certain node.

When I run something like this:

{noformat}
cache -scan -c=CacheQueryExampleOrganizations -id8=5F52C8E4
{noformat}

it still outputs all entries from the cache, not regarding the partitioning.

The issue is simple to reproduce, I slightly modified the 
org.apache.ignite.examples.datagrid.CacheQueryExample, adding 3 more 
organizations and starting additional 4 nodes.





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7022) Use QuadTree for kNN performance

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-7022:


 Summary: Use QuadTree for kNN performance
 Key: IGNITE-7022
 URL: https://issues.apache.org/jira/browse/IGNITE-7022
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Aleksey Zinoviev
Assignee: Aleksey Zinoviev
Priority: Minor


Now, kNN implementation is not too fast. Its performance could be increased 
with [https://en.wikipedia.org/wiki/Quadtree]

Also, benchmarks should be provided too



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (IGNITE-6962) Reduce ExchangeHistory memory consumption

[Alexei Scherbakov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexei Scherbakov reassigned IGNITE-6962:
-

Assignee: Alexei Scherbakov

> Reduce ExchangeHistory memory consumption
> -
>
> Key: IGNITE-6962
> URL: https://issues.apache.org/jira/browse/IGNITE-6962
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Alexander Belyak
>Assignee: Alexei Scherbakov
> Attachments: histo18175_s1.txt, leak.jpg
>
>
> GridDhtPartitionExchangeManager$ExhcangeFutureSet store huge message 
> GridDhtPartitionsFullMessage with IgniteDhtPartitionCountersMap2 for each 
> cache group with two long[partCount]. If we have big grid (100+ nodes) with 
> large amount of cacheGroups and partitions in 
> CachePartitionFullCountersMap(long[] initialUpdCntrs; long[] updCntrs;)
> *<2 

[jira] [Commented] (IGNITE-7013) .NET: Ignite does not start on macOS

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266734#comment-16266734
 ] 

Pavel Tupitsyn commented on IGNITE-7013:


{{LoadLibrary}} + {{DllImport}} combo does not work on macOs (where 
{{LoadLibrary}} is called with a full path, and {{DllImport}} uses simple name).

We have to switch to {{dlopen}} + {{dlsym}} and manual delegate creation.

> .NET: Ignite does not start on macOS
> 
>
> Key: IGNITE-7013
> URL: https://issues.apache.org/jira/browse/IGNITE-7013
> Project: Ignite
>  Issue Type: Bug
>  Components: platforms
>Affects Versions: 2.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .NET, xplat
> Fix For: 2.4
>
>
> Looks like dlopen code is incorrect for macOS:
> {code}
> Unhandled Exception: System.DllNotFoundException: Unable to load DLL 
> 'libcoreclr.so': The specified module or one of its dependencies could not be 
> found.
>  (Exception from HRESULT: 0x8007007E)
>at 
> Apache.Ignite.Core.Impl.Unmanaged.DllLoader.NativeMethodsCore.dlopen(String 
> filename, Int32 flags)
>at Apache.Ignite.Core.Impl.Unmanaged.DllLoader.Load(String dllPath)
>at Apache.Ignite.Core.Impl.IgniteUtils.LoadDll(String filePath, String 
> simpleName)
>at Apache.Ignite.Core.Impl.IgniteUtils.LoadJvmDll(String configJvmDllPath, 
> ILogger log)
>at Apache.Ignite.Core.Impl.IgniteUtils.LoadDlls(String configJvmDllPath, 
> ILogger log)
>at Apache.Ignite.Core.Ignition.Start(IgniteConfiguration cfg)
>at ignite_nuget_test.Program.Main(String[] args) in 
> /Users/vveider/Development/VCS/Git/ignite-dotnetcore-demo/Program.cs:line 17
> {code}
> Steps to reproduce:
> {code}
> git clone https://github.com/ptupitsyn/ignite-dotnetcore-demo.git
> cd ignite-dotnetcore-demo
> dotnet run
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (IGNITE-6846) Add metrics for entry processor invocations

[Alexey Kuznetsov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264488#comment-16264488
 ] 

Alexey Kuznetsov edited comment on IGNITE-6846 at 11/27/17 11:57 AM:
-

1)When entry is updated, both 'invoke update' and get\put\remove metrics can be 
updated.
2)When entry processor is invoked, but cache entry isn't affected, then invoke 
metric isn't incremented.
3)'invoke update' metrics incremented only when cache entry value is changed(or 
removed) by entry processor(not just when entry processor is invoked).
4)If entry processor process(...) is called once, and multiple entries(perhaps 
on different nodes) are affected, then local invoke metric is incremented only 
once, cluster global metric is incremented once, local metrics on other nodes 
are zero.
5)Read-only 'invoke' metric is incremented if processor invoke operation is 
called without executing entry.setValue(...) (even calling entry.getValue() can 
be omit), processor can return non-null value.Also, cache might be empty while 
processor invocation.
6)The following metrics are going to be added : total number of invokes, number 
of invoke updates(including removal), number of read-only invokes, min\max\avg 
number of invocations(all invocations).


was (Author: alexey kuznetsov):
1)When entry is updated, both 'invoke update' and get\put\remove metrics can be 
updated.
2)When entry processor is invoked, but cache entry isn't affected, then invoke 
metric isn't incremented.
3)'invoke update' metrics incremented only when cache entry value is changed(or 
removed) by entry processor(not just when entry processor is invoked).
4)If entry processor process(...) is called once, and multiple entries(perhaps 
on different nodes) are affected, then local invoke metric is incremented only 
once, cluster global metric is incremented once, local metrics on other nodes 
are zero.
5)Read-only 'invoke' metric is incremented if processor invoke operation is 
called without executing entry.setValue(...) (even calling entry.getValue() can 
be omit), processor can return non-null value.Also, cache might be empty while 
processor invocation.

> Add metrics for entry processor invocations
> ---
>
> Key: IGNITE-6846
> URL: https://issues.apache.org/jira/browse/IGNITE-6846
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.3
>Reporter: Valentin Kulichenko
>Assignee: Alexey Kuznetsov
>Priority: Critical
>  Labels: iep-6
> Fix For: 2.4
>
>
> {{CacheMetrics}} object has multiple metrics for various cache operations 
> like {{get}}, {{put}} and {{remove}}, but nothing for 
> {{invoke}}/{{EntryProcessor}}. It makes sense to add such metrics, for 
> example:
> * Total number of `invoke` operations executed.
> * Number of `invoke` operations that included updates.
> * Number of read-only `invoke` operations.
> * Min/max/avg execution time.
> * ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (IGNITE-6846) Add metrics for entry processor invocations

[Alexey Kuznetsov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264488#comment-16264488
 ] 

Alexey Kuznetsov edited comment on IGNITE-6846 at 11/27/17 11:54 AM:
-

1)When entry is updated, both 'invoke update' and get\put\remove metrics can be 
updated.
2)When entry processor is invoked, but cache entry isn't affected, then invoke 
metric isn't incremented.
3)'invoke update' metrics incremented only when cache entry value is changed(or 
removed) by entry processor(not just when entry processor is invoked).
4)If entry processor process(...) is called once, and multiple entries(perhaps 
on different nodes) are affected, then local invoke metric is incremented only 
once, cluster global metric is incremented once, local metrics on other nodes 
are zero.
5)Read-only 'invoke' metric is incremented if processor invoke operation is 
called without executing entry.setValue(...) (even calling entry.getValue() can 
be omit), processor can return non-null value.Also, cache might be empty while 
processor invocation.


was (Author: alexey kuznetsov):
1)When entry is updated, both 'invoke update' and get\put\remove metrics can be 
updated.
2)When entry processor is invoked, but cache entry isn't affected, then invoke 
metric isn't incremented.
3)'invoke update' metrics incremented only when cache entry value is changed(or 
removed) by entry processor(not just when entry processor is invoked).
4)If entry processor process(...) is called once, and multiple entries(perhaps 
on different nodes) are affected, then local invoke metric is incremented only 
once, cluster global metric is incremented once.
5)Read-only 'invoke' operation is processor invocation without calling 
entry.setValue(...) (even entry.getValue() can not be called), processor can 
return non-null value.Also, cache might be empty while processor invocation.

> Add metrics for entry processor invocations
> ---
>
> Key: IGNITE-6846
> URL: https://issues.apache.org/jira/browse/IGNITE-6846
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.3
>Reporter: Valentin Kulichenko
>Assignee: Alexey Kuznetsov
>Priority: Critical
>  Labels: iep-6
> Fix For: 2.4
>
>
> {{CacheMetrics}} object has multiple metrics for various cache operations 
> like {{get}}, {{put}} and {{remove}}, but nothing for 
> {{invoke}}/{{EntryProcessor}}. It makes sense to add such metrics, for 
> example:
> * Total number of `invoke` operations executed.
> * Number of `invoke` operations that included updates.
> * Number of read-only `invoke` operations.
> * Min/max/avg execution time.
> * ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Issue Comment Deleted] (IGNITE-6846) Add metrics for entry processor invocations

[Alexey Kuznetsov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Kuznetsov updated IGNITE-6846:
-
Comment: was deleted

(was: [~vkulichenko] Should we create separate metric for 'invoke removal' ? Or 
it shold be counted by 'invoke update' metric?)

> Add metrics for entry processor invocations
> ---
>
> Key: IGNITE-6846
> URL: https://issues.apache.org/jira/browse/IGNITE-6846
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.3
>Reporter: Valentin Kulichenko
>Assignee: Alexey Kuznetsov
>Priority: Critical
>  Labels: iep-6
> Fix For: 2.4
>
>
> {{CacheMetrics}} object has multiple metrics for various cache operations 
> like {{get}}, {{put}} and {{remove}}, but nothing for 
> {{invoke}}/{{EntryProcessor}}. It makes sense to add such metrics, for 
> example:
> * Total number of `invoke` operations executed.
> * Number of `invoke` operations that included updates.
> * Number of read-only `invoke` operations.
> * Min/max/avg execution time.
> * ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-7001) Refactor dynamic indexes tests to use SQL API

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1620#comment-1620
 ] 

ASF GitHub Bot commented on IGNITE-7001:


GitHub user alexpaschenko opened a pull request:

https://github.com/apache/ignite/pull/3096

IGNITE-7001



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7001

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3096.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3096


commit f40600c691462190c59bf1ac161673863e1ea952
Author: Alexander Paschenko 
Date:   2017-11-24T15:27:48Z

IGNITE-7001 Dynamic index tests refactoring.

commit 68010a36c1365d72a82e8cc3e75d51c3ed5089ee
Author: Alexander Paschenko 
Date:   2017-11-27T11:14:51Z

IGNITE-7001 Continued.

commit 8a5eb38945669a3c0d6e02c0d602ee87f23cc617
Author: Alexander Paschenko 
Date:   2017-11-27T11:22:07Z

Merge remote-tracking branch 'apache/master' into ignite-7001




> Refactor dynamic indexes tests to use SQL API
> -
>
> Key: IGNITE-7001
> URL: https://issues.apache.org/jira/browse/IGNITE-7001
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Reporter: Alexander Paschenko
>Assignee: Alexander Paschenko
> Fix For: 2.4
>
>
> Tests for dynamic indexes should be refactored like those in IGNITE-6416 to 
> use SQL API instead of relying on internal structures.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-6113) Partition eviction prevents exchange from completion

[Alexey Goncharuk (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266650#comment-16266650
 ] 

Alexey Goncharuk commented on IGNITE-6113:
--

The issue with the code is that we wait for partition rent in the assign() 
method. This blocks exchange thread from progress until the partition is 
evicted.

In my understanding, we already have all the machinery to fix this issue:
1) Move partition to MOVING state, but mark it as required to be evicted
2) In demander, do not send first demand message until we iterated over the 
full partition and attempted to clean it. After the iteration is done, even if 
the partition is not empty, we can safely start rebalancing. Also, during the 
clear, we can skip versions that arrived after the rebalancing has started 
(need to figure out if this can be implemented)
3) After iteration is finished, start rebalancing for this particular partition.

Also, note the suspicious code in the GridDhtPreloader#assing() method:
{code}
if (histSupplier != null) {
if (part.state() != MOVING) {
if (log.isDebugEnabled())
log.debug("Skipping partition assignment (state is 
not MOVING): " + part);

continue; // For.
}
{code}
Most likely, we need the same logic here, even if we have a history supplier.

> Partition eviction prevents exchange from completion
> 
>
> Key: IGNITE-6113
> URL: https://issues.apache.org/jira/browse/IGNITE-6113
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Vladislav Pyatkov
>
> I has waited for 3 hours for completion without any success.
> exchange-worker is blocked.
> {noformat}
> "exchange-worker-#92%DPL_GRID%grid554.ca.sbrf.ru%" #173 prio=5 os_prio=0 
> tid=0x7f0835c2e000 nid=0xb907 runnable [0x7e74ab1d]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for  <0x7efee630a7c0> (a 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtLocalPartition$1)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at 
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:189)
> at 
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:139)
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.assign(GridDhtPreloader.java:340)
> at 
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1801)
> at 
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at java.lang.Thread.run(Thread.java:748)
>Locked ownable synchronizers:
> - None
> {noformat}
> {noformat}
> "sys-#124%DPL_GRID%grid554.ca.sbrf.ru%" #278 prio=5 os_prio=0 
> tid=0x7e731c02d000 nid=0xbf4d runnable [0x7e734e7f7000]
>java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
> at sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:60)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
> at sun.nio.ch.IOUtil.write(IOUtil.java:51)
> at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:211)
> - locked <0x7f056161bf88> (a java.lang.Object)
> at 
> org.gridgain.grid.cache.db.wal.FileWriteAheadLogManager$FileWriteHandle.writeBuffer(FileWriteAheadLogManager.java:1829)
> at 
> org.gridgain.grid.cache.db.wal.FileWriteAheadLogManager$FileWriteHandle.flush(FileWriteAheadLogManager.java:1572)
> at 
> org.gridgain.grid.cache.db.wal.FileWriteAheadLogManager$FileWriteHandle.addRecord(FileWriteAheadLogManager.java:1421)
> at 
> org.gridgain.grid.cache.db.wal.FileWriteAheadLogManager$FileWriteHandle.access$800(FileWriteAheadLogManager.java:1331)
> at 
> org.gridgain.grid.cache.db.wal.FileWriteAheadLogManager.log(FileWriteAheadLogManager.java:339)
> at 
> org.gridgain.grid.internal.processors.cache.database.pagemem.PageMemoryImpl.beforeReleaseWrite(PageMemoryImpl.java:1287)
> at 
> org.gridgain.grid.internal.processors.cache.database.pagemem.PageMemoryImpl.writeUnlockPage(PageMemoryImpl.java:1142)
> at 
> org.gridgain.grid.internal.processors.cache.database.pagemem.PageImpl.releaseWrite(PageImpl.java:167)
> 

[jira] [Assigned] (IGNITE-4809) Cache.getAll can return partially commited results.

[Dmitry Karachentsev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Karachentsev reassigned IGNITE-4809:
---

Assignee: (was: Dmitry Karachentsev)

> Cache.getAll can return partially commited results.
> ---
>
> Key: IGNITE-4809
> URL: https://issues.apache.org/jira/browse/IGNITE-4809
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 1.7
>Reporter: Andrew Mashenkov
> Fix For: 2.4
>
> Attachments: DirtyReads2.java
>
>
> 1. Create tansactional cache with Long values and fill it with zeroes.
> 2. Start thread that would increment all values in single transaction.
> 3. Start thread that would check all values are same in single transaction.
> Second thread will see partial commits, but shouldn't.
> Seems, it is related to OPTIMISTIC concurrency level only.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (IGNITE-7020) Web Console: add column resizer to pinned columns

[Dmitriy Shabalin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitriy Shabalin reassigned IGNITE-7020:


Assignee: Alexey Kuznetsov  (was: Dmitriy Shabalin)

> Web Console: add column resizer to pinned columns
> -
>
> Key: IGNITE-7020
> URL: https://issues.apache.org/jira/browse/IGNITE-7020
> Project: Ignite
>  Issue Type: Improvement
>  Components: wizards
>Reporter: Dmitriy Shabalin
>Assignee: Alexey Kuznetsov
> Fix For: 2.4
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (IGNITE-4809) Cache.getAll can return partially commited results.

[Dmitry Karachentsev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Karachentsev reassigned IGNITE-4809:
---

Assignee: Dmitry Karachentsev

> Cache.getAll can return partially commited results.
> ---
>
> Key: IGNITE-4809
> URL: https://issues.apache.org/jira/browse/IGNITE-4809
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 1.7
>Reporter: Andrew Mashenkov
>Assignee: Dmitry Karachentsev
> Fix For: 2.4
>
> Attachments: DirtyReads2.java
>
>
> 1. Create tansactional cache with Long values and fill it with zeroes.
> 2. Start thread that would increment all values in single transaction.
> 3. Start thread that would check all values are same in single transaction.
> Second thread will see partial commits, but shouldn't.
> Seems, it is related to OPTIMISTIC concurrency level only.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-6702) Get rid of values deserialization in H2TreeIndex.getRowCount

[Kirill Shirokov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266624#comment-16266624
 ] 

Kirill Shirokov commented on IGNITE-6702:
-

Thanks for review. Some questions about the style in this product:

1.2) This way you end up with the following:

T1 var1 = getX();

T2 var2 = getY(var1);

T3 var3 = getZ(var2);

Instead of:

T3 var3 = getZ(getY(getX());

or:

T2 var2 = getY(getX());

T3 var3 = getZ(var2);

Is the first style is recommended for this product?

1.3) Then it looks like we have a bit different view on the logic of putting 
these empty lines. I'm trying to understand your one:

Why the lines at BPlusTreeSelfTest.java:1653:

long treeSize = tree.size(rowDumper);
long curPutVal = curPutKey.get();

are semantically similiar and don't have a blank line in between, while the 
following ones at BPlusTree.cpp:1987:

readUnlock(curPageId, curPage, curPageAddr);

curPageAddr = 0; // Set to zero to avoid double unlocking in finalizer

...are different and have a separating line?

Also I'm trying to avoid style where

we

have 

an empty 

line

between

every

two

statements.

(while not having blank lines at all is also quite embarassing).

3) We have around 722 TODOs already in the codebase -- should TODOs also be 
removed when I work on or touch a method that contains a TODO?

4) What kind of binary compatibility problems they introduced? So, should I 
avoid lambdas (when we move to Java 8) / anon. classes in the whole product 
(including the tests) or only in the parts that are involved in serialization?
 
Thank you!

> Get rid of values deserialization in H2TreeIndex.getRowCount
> 
>
> Key: IGNITE-6702
> URL: https://issues.apache.org/jira/browse/IGNITE-6702
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Reporter: Semen Boikov
>Assignee: Kirill Shirokov
>  Labels: performance
> Fix For: 2.4
>
>
> It seems H2TreeIndex.getRowCount is called for 'select count(*) from x' 
> queries, now it is implemented via BPlusTree.find(x, x) method which 
> deserializes all values. Actually values are not needed there, need implement 
> method on BPlusTree computing size without deserialization.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7021) IgniteOOM is not propogated to client in case of implicit transaction

[Mikhail Cherkasov (JIRA)]

Mikhail Cherkasov created IGNITE-7021:
-

 Summary: IgniteOOM is not propogated to client  in case of 
implicit transaction
 Key: IGNITE-7021
 URL: https://issues.apache.org/jira/browse/IGNITE-7021
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.3
Reporter: Mikhail Cherkasov
Priority: Critical
 Fix For: 2.4


it's related to https://issues.apache.org/jira/browse/IGNITE-7019
when transaction fails due IgniteOOM,  ignite tries to rollback transaction and 
it fails too, because can't add free pages to free list due a new IgniteOOM:

[2017-11-27 
12:47:37,539][ERROR][sys-stripe-2-#4%cache.IgniteOutOfMemoryPropagationTest0%][GridNearTxLocal]
 Heuristic transaction failure.
at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:835)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.localFinish(GridDhtTxLocalAdapter.java:774)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.localFinish(GridDhtTxLocal.java:555)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.finishTx(GridDhtTxLocal.java:441)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.commitDhtLocalAsync(GridDhtTxLocal.java:489)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.commitAsync(GridDhtTxLocal.java:498)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.onDone(GridDhtTxPrepareFuture.java:727)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.onDone(GridDhtTxPrepareFuture.java:104)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:451)
at 
org.apache.ignite.internal.util.future.GridCompoundFuture.checkComplete(GridCompoundFuture.java:285)
at 
org.apache.ignite.internal.util.future.GridCompoundFuture.markInitialized(GridCompoundFuture.java:276)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare0(GridDhtTxPrepareFuture.java:1246)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.mapIfLocked(GridDhtTxPrepareFuture.java:666)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare(GridDhtTxPrepareFuture.java:1040)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.prepareAsync(GridDhtTxLocal.java:398)
at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.prepareNearTx(IgniteTxHandler.java:519)
at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxPrepareRequest0(IgniteTxHandler.java:150)
at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxPrepareRequest(IgniteTxHandler.java:135)
at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$000(IgniteTxHandler.java:97)
at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$1.apply(IgniteTxHandler.java:177)
at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$1.apply(IgniteTxHandler.java:175)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1060)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:378)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:304)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:99)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:293)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
at 
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090)
at 
org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499)
at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.IgniteException: Runtime failure on search 
row: org.apache.ignite.internal.processors.cache.tree.SearchRow@2b17e5c8

[jira] [Updated] (IGNITE-7020) Web Console: add column resizer to pinned columns

[Alexey Kuznetsov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Kuznetsov updated IGNITE-7020:
-
Fix Version/s: (was: 3.0)
   2.4

> Web Console: add column resizer to pinned columns
> -
>
> Key: IGNITE-7020
> URL: https://issues.apache.org/jira/browse/IGNITE-7020
> Project: Ignite
>  Issue Type: Improvement
>  Components: wizards
>Reporter: Dmitriy Shabalin
>Assignee: Dmitriy Shabalin
> Fix For: 2.4
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-2766) Cache instance is closed when client disconnects

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266611#comment-16266611
 ] 

ASF GitHub Bot commented on IGNITE-2766:


Github user alamar closed the pull request at:

https://github.com/apache/ignite/pull/2816


> Cache instance is closed when client disconnects
> 
>
> Key: IGNITE-2766
> URL: https://issues.apache.org/jira/browse/IGNITE-2766
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 1.5.0.final
>Reporter: Valentin Kulichenko
>Assignee: Ilya Kasnacheev
>
> In case client disconnects and reconnects after network timeout (i.e., with a 
> new ID), all cache instances acquired by this client are closed and are not 
> functional. User has to create a special logic to handle this case.
> Cache proxy should be able to handle this automatically.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-2766) Cache instance is closed when client disconnects

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266612#comment-16266612
 ] 

ASF GitHub Bot commented on IGNITE-2766:


Github user alamar closed the pull request at:

https://github.com/apache/ignite/pull/2870


> Cache instance is closed when client disconnects
> 
>
> Key: IGNITE-2766
> URL: https://issues.apache.org/jira/browse/IGNITE-2766
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 1.5.0.final
>Reporter: Valentin Kulichenko
>Assignee: Ilya Kasnacheev
>
> In case client disconnects and reconnects after network timeout (i.e., with a 
> new ID), all cache instances acquired by this client are closed and are not 
> functional. User has to create a special logic to handle this case.
> Cache proxy should be able to handle this automatically.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-2766) Cache instance is closed when client disconnects

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266610#comment-16266610
 ] 

ASF GitHub Bot commented on IGNITE-2766:


Github user alamar closed the pull request at:

https://github.com/apache/ignite/pull/3077


> Cache instance is closed when client disconnects
> 
>
> Key: IGNITE-2766
> URL: https://issues.apache.org/jira/browse/IGNITE-2766
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 1.5.0.final
>Reporter: Valentin Kulichenko
>Assignee: Ilya Kasnacheev
>
> In case client disconnects and reconnects after network timeout (i.e., with a 
> new ID), all cache instances acquired by this client are closed and are not 
> functional. User has to create a special logic to handle this case.
> Cache proxy should be able to handle this automatically.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-2766) Cache instance is closed when client disconnects

[Nikolay Tikhonov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266608#comment-16266608
 ] 

Nikolay Tikhonov commented on IGNITE-2766:
--

[~ilyak],
I've checked and merged your test improvment. Thank you for your contribution!

> Cache instance is closed when client disconnects
> 
>
> Key: IGNITE-2766
> URL: https://issues.apache.org/jira/browse/IGNITE-2766
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 1.5.0.final
>Reporter: Valentin Kulichenko
>Assignee: Ilya Kasnacheev
>
> In case client disconnects and reconnects after network timeout (i.e., with a 
> new ID), all cache instances acquired by this client are closed and are not 
> functional. User has to create a special logic to handle this case.
> Cache proxy should be able to handle this automatically.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7020) Web Console: add column resizer to pinned columns

[Dmitriy Shabalin (JIRA)]

Dmitriy Shabalin created IGNITE-7020:


 Summary: Web Console: add column resizer to pinned columns
 Key: IGNITE-7020
 URL: https://issues.apache.org/jira/browse/IGNITE-7020
 Project: Ignite
  Issue Type: Improvement
  Components: wizards
Reporter: Dmitriy Shabalin
Assignee: Dmitriy Shabalin
 Fix For: 3.0






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-6989) .NET: Thin client: Group operation codes by purpose

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266592#comment-16266592
 ] 

Pavel Tupitsyn commented on IGNITE-6989:


Wiki updated: 
https://cwiki.apache.org/confluence/display/IGNITE/IEP-9+Thin+Client+Protocol

> .NET: Thin client: Group operation codes by purpose
> ---
>
> Key: IGNITE-6989
> URL: https://issues.apache.org/jira/browse/IGNITE-6989
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms, thin client
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .NET
> Fix For: 2.4
>
>
> Currently operation codes are in random order. Let's group things by purpose: 
> cache operations, metadata, queries, etc.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-6989) .NET: Thin client: Group operation codes by purpose

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266582#comment-16266582
 ] 

Pavel Tupitsyn commented on IGNITE-6989:


Merged to master: {{150449ca0068335461d977ba00ee425ace907f29}}.

> .NET: Thin client: Group operation codes by purpose
> ---
>
> Key: IGNITE-6989
> URL: https://issues.apache.org/jira/browse/IGNITE-6989
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms, thin client
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .NET
> Fix For: 2.4
>
>
> Currently operation codes are in random order. Let's group things by purpose: 
> cache operations, metadata, queries, etc.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7019) Cluster can not survive after IgniteOOM

[Mikhail Cherkasov (JIRA)]

Mikhail Cherkasov created IGNITE-7019:
-

 Summary: Cluster can not survive after IgniteOOM
 Key: IGNITE-7019
 URL: https://issues.apache.org/jira/browse/IGNITE-7019
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.3
Reporter: Mikhail Cherkasov
Priority: Critical
 Fix For: 2.4


even if we have full sync mode and transactional cache we can't add new nodes 
if there  was IgniteOOM, after adding new nodes and re-balancing, old nodes 
can't evict partitions:

[2017-11-17 20:02:24,588][ERROR][sys-#65%DR1%][GridDhtPreloader] Partition 
eviction failed, this can cause grid hang.
class org.apache.ignite.internal.mem.IgniteOutOfMemoryException: Not enough 
memory allocated [policyName=100MB_Region_Eviction, size=104.9 MB]
Consider increasing memory policy size, enabling evictions, adding more nodes 
to the cluster, reducing number of backups or reducing model size.
at 
org.apache.ignite.internal.pagemem.impl.PageMemoryNoStoreImpl.allocatePage(PageMemoryNoStoreImpl.java:294)
at 
org.apache.ignite.internal.processors.cache.persistence.DataStructure.allocatePageNoReuse(DataStructure.java:117)
at 
org.apache.ignite.internal.processors.cache.persistence.DataStructure.allocatePage(DataStructure.java:105)
at 
org.apache.ignite.internal.processors.cache.persistence.freelist.PagesList.addStripe(PagesList.java:413)
at 
org.apache.ignite.internal.processors.cache.persistence.freelist.PagesList.getPageForPut(PagesList.java:528)
at 
org.apache.ignite.internal.processors.cache.persistence.freelist.PagesList.put(PagesList.java:617)
at 
org.apache.ignite.internal.processors.cache.persistence.freelist.FreeListImpl.addForRecycle(FreeListImpl.java:582)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Remove.reuseFreePages(BPlusTree.java:3847)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Remove.releaseAll(BPlusTree.java:4106)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Remove.access$6900(BPlusTree.java:3166)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doRemove(BPlusTree.java:1782)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.remove(BPlusTree.java:1567)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.remove(IgniteCacheOffheapManagerImpl.java:1387)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.remove(IgniteCacheOffheapManagerImpl.java:374)
at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.removeValue(GridCacheMapEntry.java:3233)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheEntry.clearInternal(GridDhtCacheEntry.java:588)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtLocalPartition.clearAll(GridDhtLocalPartition.java:892)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtLocalPartition.tryEvict(GridDhtLocalPartition.java:750)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader$3.call(GridDhtPreloader.java:593)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader$3.call(GridDhtPreloader.java:580)
at 
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6639)
at 
org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:967)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-6977) Wrong initial BitSet size in GridPartitionStateMap

[Alexander Belyak (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266538#comment-16266538
 ] 

Alexander Belyak commented on IGNITE-6977:
--

https://github.com/apache/ignite/pull/3082

> Wrong initial BitSet size in GridPartitionStateMap
> --
>
> Key: IGNITE-6977
> URL: https://issues.apache.org/jira/browse/IGNITE-6977
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: 2.1
>Reporter: Alexander Belyak
>Assignee: Alexander Belyak
>Priority: Minor
>
> In constructor of org.apache.ignite.internal.utilGridPartitionStateMap(int 
> parts) {
> states = new BitSet(parts);
> }
> we initialize BitSet with part bit, but use private static final int BITS for 
> each partition state. As result long[] in BitSet get difficult predictable 
> size (depends of access order it can be exact as needed or almost twice 
> bigger with at least one additional array copying)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-6828) Confusing messages "SLF4J: Failed to load class" at Ignite start

[Alexey Popov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266508#comment-16266508
 ] 

Alexey Popov commented on IGNITE-6828:
--

[~ntikho...@apache.org], please review

> Confusing messages "SLF4J: Failed to load class" at Ignite start
> 
>
> Key: IGNITE-6828
> URL: https://issues.apache.org/jira/browse/IGNITE-6828
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: 2.1
>Reporter: Alexey Popov
>Assignee: Alexey Popov
>Priority: Trivial
> Fix For: 2.4
>
>
> How to reproduce:
> 1. build Ignite
> 2. go to examples\
> 3. run: mvn exec:java 
> -Dexec.mainClass="org.apache.ignite.examples.ExampleNodeStartup"
> you will see the following confusing output:
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further 
> details.
> SLF4J dependency comes from org.springframework.data:spring-data-commons:
> [INFO] +- org.apache.ignite:ignite-spring-data:jar:2.3.0-SNAPSHOT:compile
> [INFO] |  +- (org.apache.ignite:ignite-core:jar:2.3.0-SNAPSHOT:compile - 
> omitted for duplicate)
> [INFO] |  +- (org.apache.ignite:ignite-indexing:jar:2.3.0-SNAPSHOT:compile - 
> omitted for duplicate)
> [INFO] |  +- 
> org.springframework.data:spring-data-commons:jar:1.13.1.RELEASE:compile
> [INFO] |  |  +- (org.springframework:spring-core:jar:4.3.7.RELEASE:compile - 
> omitted for duplicate)
> [INFO] |  |  +- (org.springframework:spring-beans:jar:4.3.7.RELEASE:compile - 
> omitted for duplicate)
> [INFO] |  |  +- org.slf4j:slf4j-api:jar:1.7.24:compile
> [INFO] |  |  \- org.slf4j:jcl-over-slf4j:jar:1.7.24:runtime
> [INFO] |  | \- (org.slf4j:slf4j-api:jar:1.7.24:runtime - omitted for 
> duplicate)
> [INFO] |  \- (org.apache.ignite:ignite-spring:jar:2.3.0-SNAPSHOT:compile - 
> omitted for duplicate)
> We should remove this dependency because it is confusing and does not affects 
> to Ignite logging functionality
> Dev-list:
> http://apache-ignite-developers.2346864.n4.nabble.com/Confusing-slf4j-error-messages-td24334.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (IGNITE-6828) Confusing messages "SLF4J: Failed to load class" at Ignite start

[Alexey Popov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-6828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Popov updated IGNITE-6828:
-
Description: 
How to reproduce:
1. build Ignite
2. go to examples\
3. run: mvn exec:java 
-Dexec.mainClass="org.apache.ignite.examples.ExampleNodeStartup"

you will see the following confusing output:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further 
details.

SLF4J dependency comes from org.springframework.data:spring-data-commons:

[INFO] +- org.apache.ignite:ignite-spring-data:jar:2.3.0-SNAPSHOT:compile
[INFO] |  +- (org.apache.ignite:ignite-core:jar:2.3.0-SNAPSHOT:compile - 
omitted for duplicate)
[INFO] |  +- (org.apache.ignite:ignite-indexing:jar:2.3.0-SNAPSHOT:compile - 
omitted for duplicate)
[INFO] |  +- 
org.springframework.data:spring-data-commons:jar:1.13.1.RELEASE:compile
[INFO] |  |  +- (org.springframework:spring-core:jar:4.3.7.RELEASE:compile - 
omitted for duplicate)
[INFO] |  |  +- (org.springframework:spring-beans:jar:4.3.7.RELEASE:compile - 
omitted for duplicate)
[INFO] |  |  +- org.slf4j:slf4j-api:jar:1.7.24:compile
[INFO] |  |  \- org.slf4j:jcl-over-slf4j:jar:1.7.24:runtime
[INFO] |  | \- (org.slf4j:slf4j-api:jar:1.7.24:runtime - omitted for 
duplicate)
[INFO] |  \- (org.apache.ignite:ignite-spring:jar:2.3.0-SNAPSHOT:compile - 
omitted for duplicate)

We should remove this dependency because it is confusing and does not affects 
to Ignite logging functionality

Dev-list:
http://apache-ignite-developers.2346864.n4.nabble.com/Confusing-slf4j-error-messages-td24334.html


  was:
How to reproduce:
1. build Ignite
2. go to examples\
3. run: mvn exec:java 
-Dexec.mainClass="org.apache.ignite.examples.ExampleNodeStartup"

you will see the following confusing output:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further 
details.

SLF4J dependency comes from org.springframework.data:spring-data-commons:

[INFO] +- org.apache.ignite:ignite-spring-data:jar:2.3.0-SNAPSHOT:compile
[INFO] |  +- (org.apache.ignite:ignite-core:jar:2.3.0-SNAPSHOT:compile - 
omitted for duplicate)
[INFO] |  +- (org.apache.ignite:ignite-indexing:jar:2.3.0-SNAPSHOT:compile - 
omitted for duplicate)
[INFO] |  +- 
org.springframework.data:spring-data-commons:jar:1.13.1.RELEASE:compile
[INFO] |  |  +- (org.springframework:spring-core:jar:4.3.7.RELEASE:compile - 
omitted for duplicate)
[INFO] |  |  +- (org.springframework:spring-beans:jar:4.3.7.RELEASE:compile - 
omitted for duplicate)
[INFO] |  |  +- org.slf4j:slf4j-api:jar:1.7.24:compile
[INFO] |  |  \- org.slf4j:jcl-over-slf4j:jar:1.7.24:runtime
[INFO] |  | \- (org.slf4j:slf4j-api:jar:1.7.24:runtime - omitted for 
duplicate)
[INFO] |  \- (org.apache.ignite:ignite-spring:jar:2.3.0-SNAPSHOT:compile - 
omitted for duplicate)

We should remove this dependency because it is confusing and does not affects 
to Ignite logging functionality



> Confusing messages "SLF4J: Failed to load class" at Ignite start
> 
>
> Key: IGNITE-6828
> URL: https://issues.apache.org/jira/browse/IGNITE-6828
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: 2.1
>Reporter: Alexey Popov
>Assignee: Alexey Popov
>Priority: Trivial
> Fix For: 2.4
>
>
> How to reproduce:
> 1. build Ignite
> 2. go to examples\
> 3. run: mvn exec:java 
> -Dexec.mainClass="org.apache.ignite.examples.ExampleNodeStartup"
> you will see the following confusing output:
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further 
> details.
> SLF4J dependency comes from org.springframework.data:spring-data-commons:
> [INFO] +- org.apache.ignite:ignite-spring-data:jar:2.3.0-SNAPSHOT:compile
> [INFO] |  +- (org.apache.ignite:ignite-core:jar:2.3.0-SNAPSHOT:compile - 
> omitted for duplicate)
> [INFO] |  +- (org.apache.ignite:ignite-indexing:jar:2.3.0-SNAPSHOT:compile - 
> omitted for duplicate)
> [INFO] |  +- 
> org.springframework.data:spring-data-commons:jar:1.13.1.RELEASE:compile
> [INFO] |  |  +- (org.springframework:spring-core:jar:4.3.7.RELEASE:compile - 
> omitted for duplicate)
> [INFO] |  |  +- (org.springframework:spring-beans:jar:4.3.7.RELEASE:compile - 
> omitted for duplicate)
> [INFO] |  |  +- org.slf4j:slf4j-api:jar:1.7.24:compile
> [INFO] |  |  \- org.slf4j:jcl-over-slf4j:jar:1.7.24:runtime
> [INFO] |  | \- (org.slf4j:slf4j-api:jar:1.7.24:runtime - omitted for 
> 

[jira] [Updated] (IGNITE-6022) SQL: add native batch execution support for DML statements

[Vladimir Ozerov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-6022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-6022:

Labels: performance  (was: iep-1 performance)

> SQL: add native batch execution support for DML statements
> --
>
> Key: IGNITE-6022
> URL: https://issues.apache.org/jira/browse/IGNITE-6022
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 2.1
>Reporter: Vladimir Ozerov
>  Labels: performance
> Fix For: 2.4
>
>
> We have batch execution support for JDBC and ODBC drivers. This decreases 
> number of network hops. However, we do not have any batch execution support 
> on the server side. It means that for batch of N similar statements, every 
> statement will go through the whole execution chain - parsing, splitting, 
> communication with servers. And while parsing and splitting might be avoided 
> with help of statement cache, the most heavy part - network communication - 
> is still there.
> We need to investigate how to optimize the flow for batch updates. Possible 
> improvements:
> 1) Execute statements with certain degree of parallelism;
> 2) Send several query execution requests to the server at once;
> 3) Ensure that caches are used properly for batching - we should not parse 
> the same request multiple times.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-6971) Ignite Logger type & logging file config indication

[Alexey Popov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266505#comment-16266505
 ] 

Alexey Popov commented on IGNITE-6971:
--

TC: 
https://ci.ignite.apache.org/project.html?projectId=Ignite20Tests_Ignite20Tests=pull%2F3095%2Fhead

> Ignite Logger type & logging file config indication
> ---
>
> Key: IGNITE-6971
> URL: https://issues.apache.org/jira/browse/IGNITE-6971
> Project: Ignite
>  Issue Type: Improvement
>  Components: general
>Affects Versions: 2.1
>Reporter: Alexey Popov
>Assignee: Alexey Popov
>Priority: Minor
> Fix For: 2.4
>
>
> Please see 
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Logger-amp-logging-file-config-output-td24435.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-6971) Ignite Logger type & logging file config indication

[Alexey Popov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266506#comment-16266506
 ] 

Alexey Popov commented on IGNITE-6971:
--

[~ntikho...@apache.org], please review

> Ignite Logger type & logging file config indication
> ---
>
> Key: IGNITE-6971
> URL: https://issues.apache.org/jira/browse/IGNITE-6971
> Project: Ignite
>  Issue Type: Improvement
>  Components: general
>Affects Versions: 2.1
>Reporter: Alexey Popov
>Assignee: Alexey Popov
>Priority: Minor
> Fix For: 2.4
>
>
> Please see 
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Logger-amp-logging-file-config-output-td24435.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-6971) Ignite Logger type & logging file config indication

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266500#comment-16266500
 ] 

ASF GitHub Bot commented on IGNITE-6971:


GitHub user apopovgg opened a pull request:

https://github.com/apache/ignite/pull/3095

IGNITE-6971 Ignite Logger type & logging file config indication



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6971

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3095.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3095


commit b244a39935d4b0d0e2c215dce32780c33e6afafa
Author: apopov 
Date:   2017-11-27T08:18:04Z

IGNITE-6971 Ignite Logger type & logging file config indication




> Ignite Logger type & logging file config indication
> ---
>
> Key: IGNITE-6971
> URL: https://issues.apache.org/jira/browse/IGNITE-6971
> Project: Ignite
>  Issue Type: Improvement
>  Components: general
>Affects Versions: 2.1
>Reporter: Alexey Popov
>Assignee: Alexey Popov
>Priority: Minor
> Fix For: 2.4
>
>
> Please see 
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Logger-amp-logging-file-config-output-td24435.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-6971) Ignite Logger type & logging file config indication

[Alexey Popov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266498#comment-16266498
 ] 

Alexey Popov commented on IGNITE-6971:
--

Changes:
1. Now Logger type and its configuration parameters are displayed at console 
via {{GridLoggerProxy.getLoggerInfo()}}
{noformat}
[11:19:01] Quiet mode.
[11:19:01]   ^-- Logging to file 
'C:\incubator-ignite\work\log\ignite-80943058.log'
[11:19:01]   ^-- Logging by 'Log4JLogger 
[cfg=/C:/incubator-ignite/config/ignite-log4j.xml, quiet=true]'
[11:19:01]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false or 
"-v" to ignite.{sh|bat}{noformat}

2. All Loggers were revised and missed {{IgniteLogger.toString()}} were added
3. Existent test cases were updated
4. Minor Logger errors were fixed


> Ignite Logger type & logging file config indication
> ---
>
> Key: IGNITE-6971
> URL: https://issues.apache.org/jira/browse/IGNITE-6971
> Project: Ignite
>  Issue Type: Improvement
>  Components: general
>Affects Versions: 2.1
>Reporter: Alexey Popov
>Assignee: Alexey Popov
>Priority: Minor
> Fix For: 2.4
>
>
> Please see 
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Logger-amp-logging-file-config-output-td24435.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (IGNITE-6971) Ignite Logger type & logging file config indication

[Alexey Popov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Popov reassigned IGNITE-6971:


Assignee: Alexey Popov

> Ignite Logger type & logging file config indication
> ---
>
> Key: IGNITE-6971
> URL: https://issues.apache.org/jira/browse/IGNITE-6971
> Project: Ignite
>  Issue Type: Improvement
>  Components: general
>Affects Versions: 2.1
>Reporter: Alexey Popov
>Assignee: Alexey Popov
>Priority: Minor
>
> Please see 
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Logger-amp-logging-file-config-output-td24435.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (IGNITE-7015) SQL: index should be updated only when relevant values changed

[Vladimir Ozerov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov reassigned IGNITE-7015:
---

Assignee: Roman Kondakov

> SQL: index should be updated only when relevant values changed
> --
>
> Key: IGNITE-7015
> URL: https://issues.apache.org/jira/browse/IGNITE-7015
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Reporter: Vladimir Ozerov
>Assignee: Roman Kondakov
>  Labels: iep-1, performance
> Fix For: 2.4
>
>
> See {{GridH2Table.update}} method. Whenever value is updated, we propagate it 
> to all indexes. Consider the following case:
> 1) Old row is not null, so this is "update", not "create".
> 2) Link hasn't changed
> 3) Indexed fields haven't changed
> If all conditions are met, we can skip index update completely, as state 
> before and after will be the same. This is especially important when 
> persistence is enabled because currently we generate unnecessary dirty pages 
> what increases IO pressure.
> Suggested fix:
> 1) Iterate over index columns, skipping key and affinity columns (as they are 
> guaranteed to be the same);
> 2) Compare relevant index columns of both old and new rows
> 3) If all columns are equal, do nothing.
> Fields should be read through {{GridH2KeyValueRowOnheap#getValue}}, because 
> in this case we will re-use value cache transparently.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (IGNITE-6930) Optionally to do not write free list updates to WAL

[Vladimir Ozerov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov reassigned IGNITE-6930:
---

Assignee: (was: Taras Ledkov)

> Optionally to do not write free list updates to WAL
> ---
>
> Key: IGNITE-6930
> URL: https://issues.apache.org/jira/browse/IGNITE-6930
> Project: Ignite
>  Issue Type: Task
>  Components: cache
>Reporter: Vladimir Ozerov
>  Labels: IEP-8, iep-1
> Fix For: 2.4
>
>
> When cache entry is created, we need to write update the free list. When 
> entry is updated, we need to update free list(s) several times. Currently 
> free list is persistent structure, so every update to it must be logged to be 
> able to recover after crash. This may incur significant overhead, especially 
> for small entries.
> E.g. this is how WAL for a single update looks like. "D" - updates with real 
> data, "F" - free-list management:
> {code}
>  1. [D] DataRecord [writeEntries=[UnwrapDataEntry[k = key, v = [ BinaryObject 
> [idHash=2053299190, hash=1986931360, typeId=-1580729813]], super = [DataEntry 
> [cacheId=94416770, op=UPDATE, writeVer=GridCacheVersion [topVer=122147562, 
> order=1510667560607, nodeOrder=1], partId=0, partCnt=4, super=WALRecord 
> [size=0, chainSize=0, pos=null, type=DATA_RECORD]]
>  2. [F] PagesListRemovePageRecord [rmvdPageId=00010005, 
> pageId=00010006, grpId=94416770, super=PageDeltaRecord 
> [grpId=94416770, pageId=00010006, super=WALRecord [size=37, 
> chainSize=0, pos=null, type=PAGES_LIST_REMOVE_PAGE]]]
>  3. [D] DataPageInsertRecord [super=PageDeltaRecord [grpId=94416770, 
> pageId=00010005, super=WALRecord [size=129, chainSize=0, pos=null, 
> type=DATA_PAGE_INSERT_RECORD]]]
>  4. [F] PagesListAddPageRecord [dataPageId=00010005, 
> super=PageDeltaRecord [grpId=94416770, pageId=00010008, 
> super=WALRecord [size=37, chainSize=0, pos=null, type=PAGES_LIST_ADD_PAGE]]]
>  5. [F] DataPageSetFreeListPageRecord [freeListPage=281474976710664, 
> super=PageDeltaRecord [grpId=94416770, pageId=00010005, 
> super=WALRecord [size=37, chainSize=0, pos=null, 
> type=DATA_PAGE_SET_FREE_LIST_PAGE]]]
>  6. [D] ReplaceRecord [io=DataLeafIO[ver=1], idx=0, super=PageDeltaRecord 
> [grpId=94416770, pageId=00010004, super=WALRecord [size=47, 
> chainSize=0, pos=null, type=BTREE_PAGE_REPLACE]]]
>  7. [F] DataPageRemoveRecord [itemId=0, super=PageDeltaRecord 
> [grpId=94416770, pageId=00010005, super=WALRecord [size=30, 
> chainSize=0, pos=null, type=DATA_PAGE_REMOVE_RECORD]]]
>  8. [F] PagesListRemovePageRecord [rmvdPageId=00010005, 
> pageId=00010008, grpId=94416770, super=PageDeltaRecord 
> [grpId=94416770, pageId=00010008, super=WALRecord [size=37, 
> chainSize=0, pos=null, type=PAGES_LIST_REMOVE_PAGE]]]
>  9. [F] DataPageSetFreeListPageRecord [freeListPage=0, super=PageDeltaRecord 
> [grpId=94416770, pageId=00010005, super=WALRecord [size=37, 
> chainSize=0, pos=null, type=DATA_PAGE_SET_FREE_LIST_PAGE]]]
> 10. [F] PagesListAddPageRecord [dataPageId=00010005, 
> super=PageDeltaRecord [grpId=94416770, pageId=00010006, 
> super=WALRecord [size=37, chainSize=0, pos=null, type=PAGES_LIST_ADD_PAGE]]]
> 11. [F] DataPageSetFreeListPageRecord [freeListPage=281474976710662, 
> super=PageDeltaRecord [grpId=94416770, pageId=00010005, 
> super=WALRecord [size=37, chainSize=0, pos=null, 
> type=DATA_PAGE_SET_FREE_LIST_PAGE]]]
> {code}
> If you sum all space required for operation (size in p.3 is shown incorrectly 
> here), you will see that data update required ~300 bytes, so do free list 
> update! 
> *Proposed solution*
> 1) Optionally do not write free list updates to WAL
> 2) In case of node restart we start with empty free lists, so data inserts 
> will have to allocate new pages
> 3) When old data page is read, add it to the free list
> 4) Start a background thread which will iterate over all old data pages and 
> re-create the free list, so that eventually all data pages are tracked.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (IGNITE-7016) Avoid fsync on WAL segment header write in non-default mode

[Alexey Goncharuk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Goncharuk updated IGNITE-7016:
-
Summary: Avoid fsync on WAL segment header write in non-default mode  (was: 
Avoid fsync on WAL rollover in non-default mode)

> Avoid fsync on WAL segment header write in non-default mode
> ---
>
> Key: IGNITE-7016
> URL: https://issues.apache.org/jira/browse/IGNITE-7016
> Project: Ignite
>  Issue Type: Task
>Affects Versions: 2.3
>Reporter: Alexey Goncharuk
> Fix For: 2.4
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)