[jira] [Commented] (IGNITE-6872) Linear regression should implement Model API

2017-11-10 Thread Oleg Ignatenko (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248247#comment-16248247
 ] 

Oleg Ignatenko commented on IGNITE-6872:


[~chief] I created this ticked based on our recent discussion - please take a 
look and make corrections if I missed something

> Linear regression should implement Model API
> 
>
> Key: IGNITE-6872
> URL: https://issues.apache.org/jira/browse/IGNITE-6872
> Project: Ignite
>  Issue Type: Task
>  Security Level: Public(Viewable by anyone) 
>  Components: ml
>Reporter: Oleg Ignatenko
>Assignee: Oleg Ignatenko
> Fix For: 2.4
>
>
> When linear regression was originally implemented per IGNITE-5012 we had no 
> Model API.
> Now that this API is available (merged into master with IGNITE-5218) lin 
> regression needs to adapt to implement it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (IGNITE-6872) Linear regression should implement Model API

2017-11-10 Thread Oleg Ignatenko (JIRA)
Oleg Ignatenko created IGNITE-6872:
--

 Summary: Linear regression should implement Model API
 Key: IGNITE-6872
 URL: https://issues.apache.org/jira/browse/IGNITE-6872
 Project: Ignite
  Issue Type: Task
  Security Level: Public (Viewable by anyone)
  Components: ml
Reporter: Oleg Ignatenko
Assignee: Oleg Ignatenko
 Fix For: 2.4


When linear regression was originally implemented per IGNITE-5012 we had no 
Model API.

Now that this API is available (merged into master with IGNITE-5218) lin 
regression needs to adapt to implement it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-6495) performance measurement of decision trees algorithms

2017-11-10 Thread Oleg Ignatenko (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Ignatenko updated IGNITE-6495:
---
Attachment: IGNITE-6495-draft.zip

drafted a benchmark (attached - [^IGNITE-6495-draft.zip], to be integrated 
after main ml benchmarks code in IGNITE-6123 is merged into master)

> performance measurement of decision trees algorithms
> 
>
> Key: IGNITE-6495
> URL: https://issues.apache.org/jira/browse/IGNITE-6495
> Project: Ignite
>  Issue Type: Improvement
>  Components: ml
>Reporter: Oleg Ignatenko
>Assignee: Oleg Ignatenko
> Attachments: IGNITE-6495-draft.zip
>
>
> We want to start tracking our performance to avoid performance degradation. 
> Also we need some performance comparison with other ml libs.
> This task is "extracted" from IGNITE-6123 because implementation of decision 
> trees is not yet integrated into master (IGNITE-5218).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-5651) Add long query timeout property

2017-11-10 Thread Alper Tekinalp (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alper Tekinalp updated IGNITE-5651:
---
Description: It would be nice to allow users setting long execution timeout 
when sending a query - by doing so, user won't start getting dull long 
execution warnings which may be well expected. Instead, they will set their own 
acceptable timeout and will start seeing warnings only when they want to.

> Add long query timeout property
> ---
>
> Key: IGNITE-5651
> URL: https://issues.apache.org/jira/browse/IGNITE-5651
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 2.0
>Reporter: Alexander Paschenko
>
> It would be nice to allow users setting long execution timeout when sending a 
> query - by doing so, user won't start getting dull long execution warnings 
> which may be well expected. Instead, they will set their own acceptable 
> timeout and will start seeing warnings only when they want to.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-5651) Add long query timeout property

2017-11-10 Thread Alper Tekinalp (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alper Tekinalp updated IGNITE-5651:
---
Environment: (was: It would be nice to allow users setting long 
execution timeout when sending a query - by doing so, user won't start getting 
dull long execution warnings which may be well expected. Instead, they will set 
their own acceptable timeout and will start seeing warnings only when they want 
to.)

> Add long query timeout property
> ---
>
> Key: IGNITE-5651
> URL: https://issues.apache.org/jira/browse/IGNITE-5651
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 2.0
>Reporter: Alexander Paschenko
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (IGNITE-5940) Datastreamer does not propagate OOME

2017-11-10 Thread Mikhail Cherkasov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-5940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Cherkasov resolved IGNITE-5940.
---
Resolution: Duplicate

this one is fixed by IGNITE-6654

>  Datastreamer does not propagate OOME
> -
>
> Key: IGNITE-5940
> URL: https://issues.apache.org/jira/browse/IGNITE-5940
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Mikhail Cherkasov
>Assignee: Mikhail Cherkasov
>
> DataStreamer throws exception as it's closed if OOM occurs on server node.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-1025) Need to print out warning if IP finder has a lot of addresses on Windows

2017-11-10 Thread Dmitriy Setrakyan (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitriy Setrakyan updated IGNITE-1025:
--
Fix Version/s: 2.4

> Need to print out warning if IP finder has a lot of addresses on Windows
> 
>
> Key: IGNITE-1025
> URL: https://issues.apache.org/jira/browse/IGNITE-1025
> Project: Ignite
>  Issue Type: Improvement
>  Components: general
>Affects Versions: sprint-4
>Reporter: Valentin Kulichenko
>Assignee: Ivan Fedotov
>Priority: Minor
>  Labels: Usability, newbie
> Fix For: 2.4
>
>
> Windows OS has a known issue: when we try to connect to unavailable address, 
> we have to wait for socket timeout (Linux, for example, will throw an 
> exception immediately in this case).
> Because of this issue node startup process can take significant amount of 
> time if there is a long list of addresses in IP finder. And now it looks like 
> a node simply hangs for a while (sometimes several minutes).
> We should print the warning in this case to tell the user that startup can 
> take time and make suggestions on how to avoid it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-6854) Enabling Persistent Memory for Ignite

2017-11-10 Thread Mulugeta Mammo (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mulugeta Mammo updated IGNITE-6854:
---
Description: 
Ignite, when persistence mode is enabled, stores data and indexes on disk. To 
minimize the latency of disks, several tuning options can be applied. Setting 
the page size of a memory region to match the page size of the underlying 
storage, using a separate disk for the WAL, and using production-level SSDs are 
just a few of them [ 
https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning
 ]. 
 
A persistent memory store with low latency and high capacity offers a viable 
alternative to disks. In light of this, we are proposing to make use of our Low 
Level Persistent Library (LLPL), https://github.com/pmem/pcj/tree/master/LLPL, 
to offer a persistent memory storage for Ignite. 
 
At this point, we envision two distinct implementation options:

# Data and indexes will continue to be stored in the off-heap memory but the 
disk will be replaced by a persistent memory. Since persistence memory in this 
option is not a file system, the logic currently offered by WAL file and the 
partition files would have to be implemented from scratch.
# In this option, we eliminate the current check-point process and the WAL 
file. We will use a memory region defined by LLPL to store data and indexes. 
There will be no off-heap memory. DRAM will be exclusively used to store hot 
cache entries just like the on-heap cache is in the current implementation. 
  
 
In both cases, there are more details and subtleties that have to handled – 
e.g. the atomic and transactional guarantees offered. More clarifications will 
be given as we go along.


  was:
Ignite, when persistence mode is enabled, stores data and indexes on disk. To 
minimize the latency of disks, several tuning options can be applied. Setting 
the page size of a memory region to match the page size of the underlying 
storage, using a separate disk for the WAL, and using production-level SSDs are 
just a few of them [ 
https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning
 ]. 
 
A persistent memory store with low latency and high capacity offers a viable 
alternative to disks. In light of this, we are proposing to make use of our Low 
Level Persistent Library (LLPL), https://github.com/pmem/pcj/tree/master/LLPL, 
to offer a persistent memory storage for Ignite. 
 
At this point, we envision two distinct implementation options:

# Data and indexes will continue to be stored in the off-heap memory but the 
disk will be replaced by a persistent memory. Since persistence memory in this 
option is not a file system, the logic currently offered by WAL file and the 
partition files would have to be implemented from scratch.
# In this option, we eliminate the current check-point process and the WAL 
file. We will use a memory region defined by LLPL to store data and indexes. 
There will be no off-heap memory. DRAM will be exclusively used to store hot 
cache entries just like the on-heap cache is in the current implementation. 
  
 
In both cases, there are more details and subtleties that have to handled – 
e.g. the atomic and transactional guarantees offered. More clarifications will 
be offered as we go along.



> Enabling Persistent Memory for Ignite
> -
>
> Key: IGNITE-6854
> URL: https://issues.apache.org/jira/browse/IGNITE-6854
> Project: Ignite
>  Issue Type: New Feature
>  Security Level: Public(Viewable by anyone) 
>Affects Versions: 2.3
>Reporter: Mulugeta Mammo
> Fix For: 2.4
>
>
> Ignite, when persistence mode is enabled, stores data and indexes on disk. To 
> minimize the latency of disks, several tuning options can be applied. Setting 
> the page size of a memory region to match the page size of the underlying 
> storage, using a separate disk for the WAL, and using production-level SSDs 
> are just a few of them [ 
> https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning
>  ]. 
>  
> A persistent memory store with low latency and high capacity offers a viable 
> alternative to disks. In light of this, we are proposing to make use of our 
> Low Level Persistent Library (LLPL), 
> https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory 
> storage for Ignite. 
>  
> At this point, we envision two distinct implementation options:
> # Data and indexes will continue to be stored in the off-heap memory but the 
> disk will be replaced by a persistent memory. Since persistence memory in 
> this option is not a file system, the logic currently offered by WAL file and 
> the partition files would have to be implemented from scratch.
> # In this option, we eliminate the current 

[jira] [Commented] (IGNITE-5218) Decision trees

2017-11-10 Thread Dmitriy Pavlov (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-5218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247787#comment-16247787
 ] 

Dmitriy Pavlov commented on IGNITE-5218:


Hi, could you please fix license headers?
https://ci.ignite.apache.org/viewType.html?buildTypeId=Ignite20Tests_RatJavadoc_Ignite20Tests=%3Cdefault%3E=buildTypeStatusDiv

> Decision trees
> --
>
> Key: IGNITE-5218
> URL: https://issues.apache.org/jira/browse/IGNITE-5218
> Project: Ignite
>  Issue Type: New Feature
>  Components: ml
>Reporter: Yury Babak
>Assignee: Artem Malykh
>
> We want to implement Decision trees for Ignite ML because it's really common 
> one for ML.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-6483) Entries count in caches' MX Beans not immediately available

2017-11-10 Thread Ilya Kasnacheev (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Kasnacheev updated IGNITE-6483:

Fix Version/s: (was: 2.3)
   None

> Entries count in caches' MX Beans not immediately available
> ---
>
> Key: IGNITE-6483
> URL: https://issues.apache.org/jira/browse/IGNITE-6483
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Kasnacheev
> Fix For: None
>
> Attachments: IgniteCacheMxBeansTest.java
>
>
> When querying properties like
> {code}
> cache.mxBean().getOffHeapPrimaryEntriesCount()
> {code}
> The value is often 0 (zero) initially, even if cache is not empty and located 
> in offheap.
> Local MX Bean returns correct value.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (IGNITE-6483) Entries count in caches' MX Beans not immediately available

2017-11-10 Thread Ilya Kasnacheev (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Kasnacheev resolved IGNITE-6483.
-
   Resolution: Won't Fix
Fix Version/s: 2.3

> Entries count in caches' MX Beans not immediately available
> ---
>
> Key: IGNITE-6483
> URL: https://issues.apache.org/jira/browse/IGNITE-6483
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Kasnacheev
> Fix For: 2.3
>
> Attachments: IgniteCacheMxBeansTest.java
>
>
> When querying properties like
> {code}
> cache.mxBean().getOffHeapPrimaryEntriesCount()
> {code}
> The value is often 0 (zero) initially, even if cache is not empty and located 
> in offheap.
> Local MX Bean returns correct value.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Reopened] (IGNITE-6483) Entries count in caches' MX Beans not immediately available

2017-11-10 Thread Ilya Kasnacheev (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Kasnacheev reopened IGNITE-6483:
-

> Entries count in caches' MX Beans not immediately available
> ---
>
> Key: IGNITE-6483
> URL: https://issues.apache.org/jira/browse/IGNITE-6483
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Kasnacheev
> Attachments: IgniteCacheMxBeansTest.java
>
>
> When querying properties like
> {code}
> cache.mxBean().getOffHeapPrimaryEntriesCount()
> {code}
> The value is often 0 (zero) initially, even if cache is not empty and located 
> in offheap.
> Local MX Bean returns correct value.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6752) JDBC thin: connection property refactoring

2017-11-10 Thread Sergey Kalashnikov (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247761#comment-16247761
 ] 

Sergey Kalashnikov commented on IGNITE-6752:


[~tledkov-gridgain], Looks good to me.

> JDBC thin: connection property refactoring
> --
>
> Key: IGNITE-6752
> URL: https://issues.apache.org/jira/browse/IGNITE-6752
> Project: Ignite
>  Issue Type: Task
>  Security Level: Public(Viewable by anyone) 
>  Components: jdbc
>Affects Versions: 2.2
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
> Fix For: 2.4
>
>
> The issues IGNITE-6140 and IGNITE-6625 have to connection properties 
> refactoring.
> Otherwise the logic to work with connection properties is separated between 
> several classes.
> Also, SSL implementation for JDB client adds many new properties.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-1267) JobStealingCollisionSpi never sends jobs to a node that joined after task was executed

2017-11-10 Thread Andrew Mashenkov (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247751#comment-16247751
 ] 

Andrew Mashenkov commented on IGNITE-1267:
--

Update PR against fresh master.
TC looks fine.


> JobStealingCollisionSpi never sends jobs to a node that joined after task was 
> executed
> --
>
> Key: IGNITE-1267
> URL: https://issues.apache.org/jira/browse/IGNITE-1267
> Project: Ignite
>  Issue Type: Bug
>  Components: compute
>Affects Versions: 1.1.4
>Reporter: Valentin Kulichenko
>Assignee: Andrew Mashenkov
>  Labels: user-request
>
> Corresponding user thread (contains detailed description of the scenario that 
> doesn't work): 
> http://apache-ignite-users.70518.x6.nabble.com/Dynamic-ComputeTask-distribution-with-new-nodes-td997.html
> Essentially, {{JobStealingCollisionSpi}} always skips jobs that are not in 
> task topology (see line 713). Task topology is static and created when task 
> is executed, so newly joined node can't steal jobs. I think it should be able 
> to do this if it satisfies initial cluster group predicate.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6272) .NET: Propagate multiple services deployment

2017-11-10 Thread Alexey Popov (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247744#comment-16247744
 ] 

Alexey Popov commented on IGNITE-6272:
--

TC 
https://ci.ignite.apache.org/project.html?projectId=Ignite20Tests_Ignite20Tests=pull%2F2813%2Fhead

> .NET: Propagate multiple services deployment
> 
>
> Key: IGNITE-6272
> URL: https://issues.apache.org/jira/browse/IGNITE-6272
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms
>Reporter: Denis Mekhanikov
>Assignee: Alexey Popov
>  Labels: .NET, newbie
> Fix For: 2.4
>
>
> Multiple services deployment support should be propagated to .NET:
> * {{IgniteServices.deployAll}}
> * {{IgniteServices.deployAllAsync}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6565) Use long type for size and keySize in cache metrics

2017-11-10 Thread Alexander Menshikov (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247735#comment-16247735
 ] 

Alexander Menshikov commented on IGNITE-6565:
-

I have taken a look at the task, and want to ask some questions:

1) How better cast long to int?

Now in GridDhtCacheAdapter used simple casting "(int)size". But such code can 
return confusing values. For example (int)(2*(Integer.MAX_VALUE+1L)+10L) == 10.

I think the best option to return Integer.MAX_VALUE at overflow. In such case, 
old client applications will get the closest value to the right answer. New 
clients anyway have to use long version.

2) getSize() vs getKeySize()

As I said in previous comment getSize() return number of non-null values, but 
getKeySize() return number of all entrys (because key can't be null). Javadoc 
say so. But CacheMetricsImpl#getKeySize() just calls 
CacheMetricsImpl#getSize(). Looks like a bug.

By the way, CacheMetricsImpl#getSize() calls cache#size() and there isn't any 
cache.keySize() like methods.

> Use long type for size and keySize in cache metrics
> ---
>
> Key: IGNITE-6565
> URL: https://issues.apache.org/jira/browse/IGNITE-6565
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.2
>Reporter: Ilya Kasnacheev
>Assignee: Alexander Menshikov
>
> Currently it's int so for large caches there's no way to convey correct value.
> Should introduce getSizeLong() and getKeySizeLong()
> Also introduce the same in .Net and make sure that compatibility not broken 
> when passing OP_LOCAL_METRICS and OP_GLOBAL_METRICS.
> BTW do we need keySize at all? What's it for?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6835) ODBC driver should handle ungraceful tcp disconnects

2017-11-10 Thread Sergey Kalashnikov (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247701#comment-16247701
 ] 

Sergey Kalashnikov commented on IGNITE-6835:


[~isapego]
Here are my comments:

1. Duplicate {{LOG_MSG()}} in linux variant of {{SocketClient::Connect()}}

2. {{SocketClient::TrySetOptions()}}
It would be great to have additional {{LOG_MSG}} output with 
{{errno}}/{{WSAGetLastError()}} in case of {{setsockopt}} failures.

3. Windows {{SocketClient::TrySetOption()}}

{{struct tcp_keepalive settings;}}
I would initialize it with zero (struct tcp_keepalive settings = {0};) before 
filling individual fields.



> ODBC driver should handle ungraceful tcp disconnects
> 
>
> Key: IGNITE-6835
> URL: https://issues.apache.org/jira/browse/IGNITE-6835
> Project: Ignite
>  Issue Type: Bug
>  Security Level: Public(Viewable by anyone) 
>  Components: odbc
>Affects Versions: 2.1
>Reporter: Alexey Popov
>Assignee: Igor Sapego
>  Labels: odbc
> Fix For: 2.4
>
>
> It is found that ungraceful TCP disconnect makes ODBC driver stuck at socket 
> recv().
> Ungraceful TCP disconnect could be caused:
> 1. Network failure (or new firewall rules)
> 2. Remote party shutdown (Half Closed Connection)
> So, the proposal is:
> setup socket  options: 
> 1) SO_KEEPALIVE enabled
> 2) TCP_KEEPIDLE to 60 sec. It is 2 hour by default
> 3) TCP_KEEPINTVL to 5 (\?) sec. It is 1 sec at Win and 75 sec at Linux by 
> default.
> 4) send/receive buffers to some greater value (8k by default)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6005) [Test failed] GridCachePartitionedDataStructuresFailoverSelfTest.testSemaphoreNonFailoverSafe

2017-11-10 Thread Andrey Gura (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247697#comment-16247697
 ] 

Andrey Gura commented on IGNITE-6005:
-

[~NIzhikov] It should work. But I don't understand purpose of {{isInterrupted}} 
flag. It will be initialized only once when closure will be invoked and 
probably it's value will be always {{false}}. It seems that it should be get 
new value in catck block which handles {{InterruptedException}}.

> [Test failed] 
> GridCachePartitionedDataStructuresFailoverSelfTest.testSemaphoreNonFailoverSafe
> -
>
> Key: IGNITE-6005
> URL: https://issues.apache.org/jira/browse/IGNITE-6005
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Eduard Shangareev
>Assignee: Nikolay Izhikov
>  Labels: MakeTeamcityGreenAgain
> Fix For: 2.4
>
>
> Example of fail
> https://ci.ignite.apache.org/viewLog.html?buildId=762788=buildResultsDiv=Ignite20Tests_IgniteDataStrucutures
> Typical problem is 
> {code}
> org.apache.ignite.IgniteInterruptedException: Failed to wait for asynchronous 
> operation permit (thread got interrupted).
> at 
> org.apache.ignite.internal.util.IgniteUtils$3.apply(IgniteUtils.java:805)
> at 
> org.apache.ignite.internal.util.IgniteUtils$3.apply(IgniteUtils.java:803)
> at 
> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:961)
> at 
> org.apache.ignite.internal.processors.datastructures.GridCacheSemaphoreImpl.close(GridCacheSemaphoreImpl.java:1026)
> at 
> org.apache.ignite.internal.processors.cache.datastructures.GridCacheAbstractDataStructuresFailoverSelfTest.testSemaphoreNonFailoverSafe(GridCacheAbstractDataStructuresFailoverSelfTest.java:458)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at 
> org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2000)
> at 
> org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:132)
> at 
> org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1915)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.InterruptedException: null
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1301)
> at java.util.concurrent.Semaphore.acquire(Semaphore.java:317)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.asyncOpAcquire(GridCacheAdapter.java:4314)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.asyncOp(GridCacheAdapter.java:4177)
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedCache.getAsync(GridDhtColocatedCache.java:196)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.get0(GridCacheAdapter.java:4509)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4490)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1324)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.get(GridCacheProxyImpl.java:329)
> at 
> org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$5.applyx(DataStructuresProcessor.java:635)
> at 
> org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.retryTopologySafe(DataStructuresProcessor.java:1519)
> at 
> org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.removeDataStructure(DataStructuresProcessor.java:629)
> at 
> org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.removeSemaphore(DataStructuresProcessor.java:1188)
> at 
> org.apache.ignite.internal.processors.datastructures.GridCacheSemaphoreImpl.close(GridCacheSemaphoreImpl.java:1023)
> at 
> org.apache.ignite.internal.processors.cache.datastructures.GridCacheAbstractDataStructuresFailoverSelfTest.testSemaphoreNonFailoverSafe(GridCacheAbstractDataStructuresFailoverSelfTest.java:458)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at 

[jira] [Updated] (IGNITE-6871) Implement new JMX metrics for partitions map monitoring

2017-11-10 Thread Anton Vinogradov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anton Vinogradov updated IGNITE-6871:
-
Labels: iep-6 jmx  (was: jmx)

> Implement new JMX metrics for partitions map monitoring
> ---
>
> Key: IGNITE-6871
> URL: https://issues.apache.org/jira/browse/IGNITE-6871
> Project: Ignite
>  Issue Type: New Feature
>  Security Level: Public(Viewable by anyone) 
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>  Labels: iep-6, jmx
>
> These additional metrics should be implemented to monitor partitions map per 
> each cache group:
> * Total primary partitions count located on the current node
> * Total backup partitions count located on the current node
> * Min/max partition backups left in the cluster for cache group
> * Maybe some methods to show partitions map/partition distribution statistics 
> in the cluster



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-6867) Implement new JMX metrics for topology monitoring

2017-11-10 Thread Anton Vinogradov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anton Vinogradov updated IGNITE-6867:
-
Labels: iep-6 jmx  (was: jmx)

> Implement new JMX metrics for topology monitoring
> -
>
> Key: IGNITE-6867
> URL: https://issues.apache.org/jira/browse/IGNITE-6867
> Project: Ignite
>  Issue Type: New Feature
>  Security Level: Public(Viewable by anyone) 
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>  Labels: iep-6, jmx
>
> These additional metrics and methods should be implemented:
> * Current topology version
> * Total server nodes count
> * Total client nodes count
> * Method to count nodes filtered by some node attribute
> * Method to count nodes grouped by some node attribute
>  
> There is already a ticket to implement first 2 metrics from this list 
> (IGNITE-6844)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-6870) Implement new JMX metric for cache topology validation monitoring

2017-11-10 Thread Anton Vinogradov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anton Vinogradov updated IGNITE-6870:
-
Labels: iep-6 jmx  (was: jmx)

> Implement new JMX metric for cache topology validation monitoring
> -
>
> Key: IGNITE-6870
> URL: https://issues.apache.org/jira/browse/IGNITE-6870
> Project: Ignite
>  Issue Type: New Feature
>  Security Level: Public(Viewable by anyone) 
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>  Labels: iep-6, jmx
>
> There is no way now to determine from outside the grid that cache is in 
> read-only state after TopologyValidator's "validate" method returns "false" 
> for topology. 
> Implement new JMX metric in CacheMetricsMXBean to show cache's topology 
> validation status.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-6869) Implement new JMX metric for jobs monitoring

2017-11-10 Thread Anton Vinogradov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anton Vinogradov updated IGNITE-6869:
-
Labels: iep-6 jmx  (was: jmx)

> Implement new JMX metric for jobs monitoring
> 
>
> Key: IGNITE-6869
> URL: https://issues.apache.org/jira/browse/IGNITE-6869
> Project: Ignite
>  Issue Type: New Feature
>  Security Level: Public(Viewable by anyone) 
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>  Labels: iep-6, jmx
>
> There are now some metrics in ClusterLocalNodeMetricsMXBean for monitoring 
> job's execution statistics (min/max/avg execution time). But these metrics 
> gathered since node started and can't be used to calculate average execution 
> time between probes.
> To resolve this problem new metric "Total jobs execution time" should be 
> implemented.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-6868) Implement new JMX metrics for TcpCommunicationSpi monitoring

2017-11-10 Thread Anton Vinogradov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anton Vinogradov updated IGNITE-6868:
-
Labels: iep-6 jmx  (was: jmx)

> Implement new JMX metrics for TcpCommunicationSpi monitoring
> 
>
> Key: IGNITE-6868
> URL: https://issues.apache.org/jira/browse/IGNITE-6868
> Project: Ignite
>  Issue Type: New Feature
>  Security Level: Public(Viewable by anyone) 
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>  Labels: iep-6, jmx
>
> These additional metrics should be implemented in TcpCommunicationSpi MBean:
> * Received messages count grouped by message type
> * Received messages count grouped by sender node
> * Sent messages count grouped by message type
> * Sent messages count grouped by receiver node



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6839) Ignite Compatibility: flaky test testFoldersReuseCompatibility_2_1 & 2_2

2017-11-10 Thread Dmitriy Pavlov (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247679#comment-16247679
 ] 

Dmitriy Pavlov commented on IGNITE-6839:


Hi [~daradurvs], [~avinogradov] pointed me to the reason of this problem:
{noformat}
Ignite ver. 2.2.0#20171010-sha1:f0c64e7288ef94c84984a5e7a70f7038a539c6cd
{noformat}

it is not problem in framework, but some local issue: Ignite having newer code 
is deployed as 2.2.0 version.

To fix this problem it will be required to implement maven-local cleanup step 
before compatiblity testing.

> Ignite Compatibility: flaky test testFoldersReuseCompatibility_2_1 & 2_2
> 
>
> Key: IGNITE-6839
> URL: https://issues.apache.org/jira/browse/IGNITE-6839
> Project: Ignite
>  Issue Type: Bug
>  Security Level: Public(Viewable by anyone) 
>  Components: persistence
>Affects Versions: 2.3
>Reporter: Dmitriy Pavlov
>Assignee: Dmitriy Pavlov
>  Labels: MakeTeamcityGreenAgain
> Fix For: 2.4
>
>
> Ignite Compatibility: 2 test are flaky:
>   
> org.apache.ignite.compatibility.testsuites.IgniteCompatibilityBasicTestSuite: 
> org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.testFoldersReuseCompatibility_2_1
>  (fails: 2/55): 
> https://ci.ignite.apache.org/project.html?projectId=Ignite20Tests=-1418165996957466785=%3Cdefault%3E=testDetails
> https://ci.ignite.apache.org/project.html?projectId=Ignite20Tests=1638023358531502921=testDetails_Ignite20Tests=%3Cdefault%3E
> {noformat}
> junit.framework.AssertionFailedError: Directory [binary_meta, 
> 135628fa_5763_46a1_88ff_8c0c8e51fc4e] is expected to exist 
> [/data/teamcity/work/820be461cd64b574/work/binary_meta/135628fa_5763_46a1_88ff_8c0c8e51fc4e]
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.assertTrue(Assert.java:22)
> at junit.framework.TestCase.assertTrue(TestCase.java:192)
> at 
> org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.assertDirectoryExist(FoldersReuseCompatibilityTest.java:220)
> at 
> org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.assertPdsDirsDefaultExist(FoldersReuseCompatibilityTest.java:193)
> at 
> org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.runFoldersReuse(FoldersReuseCompatibilityTest.java:108)
> at 
> org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.testFoldersReuseCompatibility_2_1(FoldersReuseCompatibilityTest.java:86)
> {noformat}
> Test probaly indicates a bug in 
> https://issues.apache.org/jira/browse/IGNITE-6285



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (IGNITE-6871) Implement new JMX metrics for partitions map monitoring

2017-11-10 Thread Aleksey Plekhanov (JIRA)
Aleksey Plekhanov created IGNITE-6871:
-

 Summary: Implement new JMX metrics for partitions map monitoring
 Key: IGNITE-6871
 URL: https://issues.apache.org/jira/browse/IGNITE-6871
 Project: Ignite
  Issue Type: New Feature
  Security Level: Public (Viewable by anyone)
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov


These additional metrics should be implemented to monitor partitions map per 
each cache group:
* Total primary partitions count located on the current node
* Total backup partitions count located on the current node
* Min/max partition backups left in the cluster for cache group
* Maybe some methods to show partitions map/partition distribution statistics 
in the cluster



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6839) Ignite Compatibility: flaky test testFoldersReuseCompatibility_2_1 & 2_2

2017-11-10 Thread Dmitriy Pavlov (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247682#comment-16247682
 ] 

Dmitriy Pavlov commented on IGNITE-6839:


[~avinogradov], thank you!

> Ignite Compatibility: flaky test testFoldersReuseCompatibility_2_1 & 2_2
> 
>
> Key: IGNITE-6839
> URL: https://issues.apache.org/jira/browse/IGNITE-6839
> Project: Ignite
>  Issue Type: Bug
>  Security Level: Public(Viewable by anyone) 
>  Components: persistence
>Affects Versions: 2.3
>Reporter: Dmitriy Pavlov
>Assignee: Dmitriy Pavlov
>  Labels: MakeTeamcityGreenAgain
> Fix For: 2.4
>
>
> Ignite Compatibility: 2 test are flaky:
>   
> org.apache.ignite.compatibility.testsuites.IgniteCompatibilityBasicTestSuite: 
> org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.testFoldersReuseCompatibility_2_1
>  (fails: 2/55): 
> https://ci.ignite.apache.org/project.html?projectId=Ignite20Tests=-1418165996957466785=%3Cdefault%3E=testDetails
> https://ci.ignite.apache.org/project.html?projectId=Ignite20Tests=1638023358531502921=testDetails_Ignite20Tests=%3Cdefault%3E
> {noformat}
> junit.framework.AssertionFailedError: Directory [binary_meta, 
> 135628fa_5763_46a1_88ff_8c0c8e51fc4e] is expected to exist 
> [/data/teamcity/work/820be461cd64b574/work/binary_meta/135628fa_5763_46a1_88ff_8c0c8e51fc4e]
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.assertTrue(Assert.java:22)
> at junit.framework.TestCase.assertTrue(TestCase.java:192)
> at 
> org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.assertDirectoryExist(FoldersReuseCompatibilityTest.java:220)
> at 
> org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.assertPdsDirsDefaultExist(FoldersReuseCompatibilityTest.java:193)
> at 
> org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.runFoldersReuse(FoldersReuseCompatibilityTest.java:108)
> at 
> org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.testFoldersReuseCompatibility_2_1(FoldersReuseCompatibilityTest.java:86)
> {noformat}
> Test probaly indicates a bug in 
> https://issues.apache.org/jira/browse/IGNITE-6285



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (IGNITE-6870) Implement new JMX metric for cache topology validation monitoring

2017-11-10 Thread Aleksey Plekhanov (JIRA)
Aleksey Plekhanov created IGNITE-6870:
-

 Summary: Implement new JMX metric for cache topology validation 
monitoring
 Key: IGNITE-6870
 URL: https://issues.apache.org/jira/browse/IGNITE-6870
 Project: Ignite
  Issue Type: New Feature
  Security Level: Public (Viewable by anyone)
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov


There is no way now to determine from outside the grid that cache is in 
read-only state after TopologyValidator's "validate" method returns "false" for 
topology. 
Implement new JMX metric in CacheMetricsMXBean to show cache's topology 
validation status.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (IGNITE-6869) Implement new JMX metric for jobs monitoring

2017-11-10 Thread Aleksey Plekhanov (JIRA)
Aleksey Plekhanov created IGNITE-6869:
-

 Summary: Implement new JMX metric for jobs monitoring
 Key: IGNITE-6869
 URL: https://issues.apache.org/jira/browse/IGNITE-6869
 Project: Ignite
  Issue Type: New Feature
  Security Level: Public (Viewable by anyone)
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov


There are now some metrics in ClusterLocalNodeMetricsMXBean for monitoring 
job's execution statistics (min/max/avg execution time). But these metrics 
gathered since node started and can't be used to calculate average execution 
time between probes.
To resolve this problem new metric "Total jobs execution time" should be 
implemented.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (IGNITE-6868) Implement new JMX metrics for TcpCommunicationSpi monitoring

2017-11-10 Thread Aleksey Plekhanov (JIRA)
Aleksey Plekhanov created IGNITE-6868:
-

 Summary: Implement new JMX metrics for TcpCommunicationSpi 
monitoring
 Key: IGNITE-6868
 URL: https://issues.apache.org/jira/browse/IGNITE-6868
 Project: Ignite
  Issue Type: New Feature
  Security Level: Public (Viewable by anyone)
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov


These additional metrics should be implemented in TcpCommunicationSpi MBean:
* Received messages count grouped by message type
* Received messages count grouped by sender node
* Sent messages count grouped by message type
* Sent messages count grouped by receiver node



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (IGNITE-6867) Implement new JMX metrics for topology monitoring

2017-11-10 Thread Aleksey Plekhanov (JIRA)
Aleksey Plekhanov created IGNITE-6867:
-

 Summary: Implement new JMX metrics for topology monitoring
 Key: IGNITE-6867
 URL: https://issues.apache.org/jira/browse/IGNITE-6867
 Project: Ignite
  Issue Type: New Feature
  Security Level: Public (Viewable by anyone)
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov


These additional metrics and methods should be implemented:
* Current topology version
* Total server nodes count
* Total client nodes count
* Method to count nodes filtered by some node attribute
* Method to count nodes grouped by some node attribute
 
There is already a ticket to implement first 2 metrics from this list 
(IGNITE-6844)




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-602) [Test] GridToStringBuilder is vulnerable for StackOverflowError caused by infinite recursion

2017-11-10 Thread Andrey Gura (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247648#comment-16247648
 ] 

Andrey Gura commented on IGNITE-602:


[~SomeFire], I've fixed some minor issues and ran TC again. Waiting for results.

> [Test] GridToStringBuilder is vulnerable for StackOverflowError caused by 
> infinite recursion
> 
>
> Key: IGNITE-602
> URL: https://issues.apache.org/jira/browse/IGNITE-602
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Reporter: Artem Shutak
>Assignee: Ryabov Dmitrii
>  Labels: MakeTeamcityGreenAgain, Muted_test
> Fix For: 2.4
>
>
> See test 
> org.gridgain.grid.util.tostring.GridToStringBuilderSelfTest#_testToStringCheckAdvancedRecursionPrevention
>  and related TODO in same source file.
> Also take a look at 
> http://stackoverflow.com/questions/11300203/most-efficient-way-to-prevent-an-infinite-recursion-in-tostring
> Test should be unmuted on TC after fix.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-5218) Decision trees

2017-11-10 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-5218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247613#comment-16247613
 ] 

ASF GitHub Bot commented on IGNITE-5218:


Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2936


> Decision trees
> --
>
> Key: IGNITE-5218
> URL: https://issues.apache.org/jira/browse/IGNITE-5218
> Project: Ignite
>  Issue Type: New Feature
>  Components: ml
>Reporter: Yury Babak
>Assignee: Artem Malykh
>
> We want to implement Decision trees for Ignite ML because it's really common 
> one for ML.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6649) Add EvictionPolicy factory support in IgniteConfiguration.

2017-11-10 Thread Andrey Gura (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247578#comment-16247578
 ] 

Andrey Gura commented on IGNITE-6649:
-

LGTM. Merged to master branch.

> Add EvictionPolicy factory support in IgniteConfiguration.
> --
>
> Key: IGNITE-6649
> URL: https://issues.apache.org/jira/browse/IGNITE-6649
> Project: Ignite
>  Issue Type: Bug
>  Security Level: Public(Viewable by anyone) 
>  Components: cache
>Reporter: Andrew Mashenkov
>Assignee: Andrew Mashenkov
> Fix For: 2.4
>
> Attachments: EvictionPolicyTest.java
>
>
> For now the only way to set EvictionPolicy to IgniteConfiguration is to use 
> EvictionPolicy instance. 
> That looks error prone as user can easily share instance between caches or 
> cache reincarnations and got unexpected results.
> E.g. it can cause an AssertionError if EvictionPolicy is reused.
> Steps to reproduce.
> 1. Create CacheConfiguration object that will be reused.
> 2. Create and fill a cache.
> 3. Destroy cache and create cache again with same CacheConfiguration object.
> 4. One of next put can fails with stacktrace below.
> The error is throws when EvictionPolicy tries to evict entries from cache 
> that has just been destroyed.
> Also, EvictionPolicy object can be implicitly holds by some user objects 
> (together with IgniteConfiguration) that can cause memory leak.
> java.lang.AssertionError
>   at 
> org.apache.ignite.internal.processors.cache.CacheEvictableEntryImpl.evict(CacheEvictableEntryImpl.java:71)
>   at 
> org.apache.ignite.cache.eviction.lru.LruEvictionPolicy.shrink0(LruEvictionPolicy.java:275)
>   at 
> org.apache.ignite.cache.eviction.lru.LruEvictionPolicy.shrink(LruEvictionPolicy.java:250)
>   at 
> org.apache.ignite.cache.eviction.lru.LruEvictionPolicy.onEntryAccessed(LruEvictionPolicy.java:161)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheEvictionManager.notifyPolicy(GridCacheEvictionManager.java:1393)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheEvictionManager.touch(GridCacheEvictionManager.java:825)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.unlockEntries(GridDhtAtomicCache.java:3058)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1952)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1730)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.mapSingle(GridNearAtomicAbstractUpdateFuture.java:264)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:494)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:436)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:209)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1245)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.put0(GridDhtAtomicCache.java:680)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2328)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2305)
>   at 
> org.apache.ignite.internal.processors.cache.IgniteCacheProxy.put(IgniteCacheProxy.java:1379)
>  
> UPD: See discussion here [1].
> [1] 
> http://apache-ignite-developers.2346864.n4.nabble.com/CacheConfiguration-reusage-issues-with-EvictionPolicy-enabled-td23437.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (IGNITE-6866) Allocate offheap on client

2017-11-10 Thread Alexander Belyak (JIRA)
Alexander Belyak created IGNITE-6866:


 Summary: Allocate offheap on client
 Key: IGNITE-6866
 URL: https://issues.apache.org/jira/browse/IGNITE-6866
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
  Components: general
Affects Versions: 2.1
Reporter: Alexander Belyak


Often client use the same config file as a server and ignite start offheap 
memory for client too... but never use it. How it happens:
1) Default memory configuration for server is creating in 
IgnitionEx.initializeConfiguration() method:
if (!myCfg.isClientMode() && myCfg.getMemoryConfiguration() == null) 
so if ignite configuration already contains memoryConfiguration - it stay there
2) In IgniteCacheDatabaseSharedManager.anActivate method do nothing only:
if (cctx.kernalContext().clientNode() && 
cctx.kernalContext().config().getMemoryConfiguration() == null)
return;
So if ignite configuration contains memory configuration - it will be 
allocated. Why its not good:
1) Memory allocation spend virtual memory (OS didn't really allocate memory 
before first access to it) and if overcommit_memory strategy is set to 
OVERCOMMIT_NEVER - it can block start client node (maybe first or second one) 
in same host (see: /proc/sys/vm/overcommit_memory and 
/proc/sys/vm/overcommit_ratio)
2) In IgniteKernal.checkPhysicalRam() we use maxSize of offheap memory and log 
warning about memory overusage
Good news only one - often in memory configuration really big only maxSize, but 
initialSize is just about 256Mb so each client really allocate not so many RAM.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6836) ODBC: Add support for SQL_ATTR_QUERY_TIMEOUT

2017-11-10 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247522#comment-16247522
 ] 

ASF GitHub Bot commented on IGNITE-6836:


GitHub user isapego opened a pull request:

https://github.com/apache/ignite/pull/3015

 IGNITE-6836: Implemented query timeout.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6836

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3015.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3015






> ODBC: Add support for SQL_ATTR_QUERY_TIMEOUT
> 
>
> Key: IGNITE-6836
> URL: https://issues.apache.org/jira/browse/IGNITE-6836
> Project: Ignite
>  Issue Type: Improvement
>  Security Level: Public(Viewable by anyone) 
>  Components: odbc
>Affects Versions: 2.1
>Reporter: Alexey Popov
>Assignee: Igor Sapego
> Fix For: 2.4
>
>
> It would be great if we can support {{SQL_ATTR_QUERY_TIMEOUT}} at ODBC.
> That gives a flexibility to end-user code to handle long-running/timeouted 
> queries.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (IGNITE-6865) Wrong map query build using

2017-11-10 Thread Alin Andrei Corodescu (JIRA)
Alin Andrei Corodescu created IGNITE-6865:
-

 Summary: Wrong map query build using
 Key: IGNITE-6865
 URL: https://issues.apache.org/jira/browse/IGNITE-6865
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
  Components: sql
Affects Versions: 2.3
Reporter: Alin Andrei Corodescu






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (IGNITE-6864) Apply refactorings to decision trees code.

2017-11-10 Thread Artem Malykh (JIRA)
Artem Malykh created IGNITE-6864:


 Summary: Apply refactorings to decision trees code.
 Key: IGNITE-6864
 URL: https://issues.apache.org/jira/browse/IGNITE-6864
 Project: Ignite
  Issue Type: Task
  Security Level: Public (Viewable by anyone)
Reporter: Artem Malykh


* Think about changes signatures in MNISTUtils, maybe we should use collections 
or arrays instead of streams.
* Add tests for CacheUtils new methods used in ColumnDecisionTreeTrainer
* Think about OpenBitSet insead of SparseBitSet and BitSet
* Consider making decision tree internal structure classes



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-6863) visorcmd: 'cache -a' shows nodes that are not matched by nodeFilter

2017-11-10 Thread Denis Mekhanikov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denis Mekhanikov updated IGNITE-6863:
-
Attachment: node-filter.zip

> visorcmd: 'cache -a' shows nodes that are not matched by nodeFilter
> ---
>
> Key: IGNITE-6863
> URL: https://issues.apache.org/jira/browse/IGNITE-6863
> Project: Ignite
>  Issue Type: Bug
>  Security Level: Public(Viewable by anyone) 
>  Components: visor
>Affects Versions: 2.3
>Reporter: Denis Mekhanikov
> Attachments: node-filter.zip
>
>
> 'cache -a' command shows all nodes in the detailed summary, even those, which 
> are not matched by a configured node filter.
> Please find attached archive with the reproducing example project.
> LOCAL cache also has this issue.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (IGNITE-6863) visorcmd: 'cache -a' shows nodes that are not matched by nodeFilter

2017-11-10 Thread Denis Mekhanikov (JIRA)
Denis Mekhanikov created IGNITE-6863:


 Summary: visorcmd: 'cache -a' shows nodes that are not matched by 
nodeFilter
 Key: IGNITE-6863
 URL: https://issues.apache.org/jira/browse/IGNITE-6863
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
  Components: visor
Affects Versions: 2.3
Reporter: Denis Mekhanikov


'cache -a' command shows all nodes in the detailed summary, even those, which 
are not matched by a configured node filter.

Please find attached archive with the reproducing example project.

LOCAL cache also has this issue.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6702) Get rid of values deserialization in H2TreeIndex.getRowCount

2017-11-10 Thread Semen Boikov (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247444#comment-16247444
 ] 

Semen Boikov commented on IGNITE-6702:
--

Please take a look at BPlusTree.size comment: !!! For debug only! May produce 
wrong results on concurrent access.


> Get rid of values deserialization in H2TreeIndex.getRowCount
> 
>
> Key: IGNITE-6702
> URL: https://issues.apache.org/jira/browse/IGNITE-6702
> Project: Ignite
>  Issue Type: Improvement
>  Security Level: Public(Viewable by anyone) 
>  Components: sql
>Reporter: Semen Boikov
>Assignee: Kirill Shirokov
>  Labels: performance
> Fix For: 2.4
>
>
> It seems H2TreeIndex.getRowCount is called for 'select count(*) from x' 
> queries, now it is implemented via BPlusTree.find(x, x) method which 
> deserializes all values. Actually values are not needed there, need implement 
> method on BPlusTree computing size without deserialization.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (IGNITE-6702) Get rid of values deserialization in H2TreeIndex.getRowCount

2017-11-10 Thread Kirill Shirokov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kirill Shirokov reassigned IGNITE-6702:
---

Assignee: Kirill Shirokov

> Get rid of values deserialization in H2TreeIndex.getRowCount
> 
>
> Key: IGNITE-6702
> URL: https://issues.apache.org/jira/browse/IGNITE-6702
> Project: Ignite
>  Issue Type: Improvement
>  Security Level: Public(Viewable by anyone) 
>  Components: sql
>Reporter: Semen Boikov
>Assignee: Kirill Shirokov
>  Labels: performance
> Fix For: 2.4
>
>
> It seems H2TreeIndex.getRowCount is called for 'select count(*) from x' 
> queries, now it is implemented via BPlusTree.find(x, x) method which 
> deserializes all values. Actually values are not needed there, need implement 
> method on BPlusTree computing size without deserialization.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6738) Support mvcc filter for local sql queries

2017-11-10 Thread Vladimir Ozerov (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247438#comment-16247438
 ] 

Vladimir Ozerov commented on IGNITE-6738:
-

[~gvvinblade], my comments:
1) IgniteH2Indexing#mvccTracker - {{GridSqlQueryParser.query}} call is illegal 
here, as it will fail for non-SELECT requests.
2) Request may contain temp tables which are not instance of {{GridH2Table}}.
3) We need tests for this feature.

> Support mvcc filter for local sql queries
> -
>
> Key: IGNITE-6738
> URL: https://issues.apache.org/jira/browse/IGNITE-6738
> Project: Ignite
>  Issue Type: Sub-task
>  Components: cache
>Reporter: Semen Boikov
>Assignee: Igor Seliverstov
>
> Need receive/release mvcc version for queries with setLocal flag set, there 
> are already some tests in CacheMvccSqlQueriesTest.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6406) SQL: CREATE INDEX should fill index from multiple threads

2017-11-10 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247433#comment-16247433
 ] 

ASF GitHub Bot commented on IGNITE-6406:


GitHub user dolphin1414 opened a pull request:

https://github.com/apache/ignite/pull/3014

IGNITE-6406: SQL: CREATE INDEX should fill index from multiple threads.

A parallel index creation and an SQL keyword PARALLEL parsing implemented.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6406

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3014.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3014


commit 97069d5af3b54a008bd32db01064bfdd90c3f5bd
Author: rkondakov 
Date:   2017-11-10T12:13:29Z

IGNITE-6406: Parallel index creation implementated

(cherry picked from commit d34f16f0ecd35ea55efb0b2e5f94734c56c9cca3)




> SQL: CREATE INDEX should fill index from multiple threads
> -
>
> Key: IGNITE-6406
> URL: https://issues.apache.org/jira/browse/IGNITE-6406
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 2.1
>Reporter: Vladimir Ozerov
>Assignee: Roman Kondakov
>  Labels: iep-1, performance
>
> Currently when {{CREATE INDEX}} command is executed, we fill new index from a 
> single thread. Low CPU, long create time. We can trade off CPU vs time by 
> adding more threads. 
> Probably number of threads should be specified in {{CREATE INDEX}} command as 
> a hint.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6850) SQL: integrate index inline size to CREATE INDEX syntax

2017-11-10 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247428#comment-16247428
 ] 

ASF GitHub Bot commented on IGNITE-6850:


GitHub user gg-shq opened a pull request:

https://github.com/apache/ignite/pull/3013

IGNITE-6850  SQL: integrate index inline size to CREATE INDEX syntax



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite IGNITE-6850

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3013.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3013


commit be1d297b145322a3d9a90d49e1817a27fc56be24
Author: gg-shq 
Date:   2017-11-10T11:16:25Z

IGNITE-6850: Added INLINE_SIZE option to internal SQL CREATE INDEX statement

commit 8f4b677a69036aad486249c507166e5189eca979
Author: gg-shq 
Date:   2017-11-10T11:21:46Z

Merge branch 'master' into IGNITE-6850

commit b0e91486d98d4ae56498f11098a4b57f83e9a69a
Author: gg-shq 
Date:   2017-11-10T11:23:01Z

IGNITE-8650: Resolved merge conflicts in SqlCreateIndexCommand.java




> SQL: integrate index inline size to CREATE INDEX syntax
> ---
>
> Key: IGNITE-6850
> URL: https://issues.apache.org/jira/browse/IGNITE-6850
> Project: Ignite
>  Issue Type: Task
>  Security Level: Public(Viewable by anyone) 
>  Components: sql
>Reporter: Vladimir Ozerov
>Assignee: Kirill Shirokov
> Fix For: 2.4
>
>
> Index value inline is important optimization which used to minimize amount of 
> data page reads when doing index lookup (see {{InlineIndexHelper}}). 
> Currently the only way to set it is {{QueryIndex.inlineSize}} property, so it 
> cannot be set from SQL command. We need to integrate it to our SQL syntax 
> (see {{SqlCreateIndexCommand}}) and make sure it is propagated properly.
> Sample syntax:
> {code}
> CREATE INDEX idx ON tbl(field) INLINE_SIZE 20;
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6818) In case of incoming communication connection ping the old one if it's alive

2017-11-10 Thread Dmitry Karachentsev (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247422#comment-16247422
 ] 

Dmitry Karachentsev commented on IGNITE-6818:
-

[~sboikov] please review.

Thanks!

> In case of incoming communication connection ping the old one if it's alive
> ---
>
> Key: IGNITE-6818
> URL: https://issues.apache.org/jira/browse/IGNITE-6818
> Project: Ignite
>  Issue Type: Bug
>  Security Level: Public(Viewable by anyone) 
>Affects Versions: 2.3
>Reporter: Dmitry Karachentsev
>Assignee: Dmitry Karachentsev
>Priority: Critical
> Fix For: 2.4
>
>
> Assume the following scenario:
> 1. Client opens connection to the server.
> 2. Server checks that it is a first connection to that node and accepts it.
> 3. By some reason firewall starts rejecting client messages with TCP reset 
> flag set.
> 4. Client closes connection, but server doesn't know about it.
> 5. Client tries connect again.
> 6. Server rejects new connection, because it already has connection to that 
> node.
> Possible fix: on step 6 server must check old connection if it's alive by 
> sending some communication message and check response. If old connection is 
> dead - close it and accept new one.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6686) Document SQL Data Types

2017-11-10 Thread Igor Sapego (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247418#comment-16247418
 ] 

Igor Sapego commented on IGNITE-6686:
-

[~vozerov], I thought, we map {{DATE}} to {{java.util.Date}}?

> Document SQL Data Types
> ---
>
> Key: IGNITE-6686
> URL: https://issues.apache.org/jira/browse/IGNITE-6686
> Project: Ignite
>  Issue Type: Task
>  Security Level: Public(Viewable by anyone) 
>  Components: documentation
>Reporter: Denis Magda
>Assignee: Vladimir Ozerov
> Fix For: 2.3
>
>
> The data types page should include all SQL types supported and how they 
> mapped to a language or driver specific type. Presently the types are 
> represented in a tabular format:
> https://apacheignite-sql.readme.io/v2.3/docs/data-types
> Guys, please help to fill out langage/drivers specific columns:
> # [~ptupitsyn] - .NET data types.
> # [~isapego] - C++ and ODBC data type.
> # [~al.psc] - JDBC and JAVA data types.
> [~vozerov], please suggest what to do with these few types below. They 
> supported by H2 but I'm not sure how Ignite deals with them:
> http://www.h2database.com/html/datatypes.html#identity_type
> http://www.h2database.com/html/datatypes.html#binary_type
> http://www.h2database.com/html/datatypes.html#other_type
> http://www.h2database.com/html/datatypes.html#varchar_ignorecase_type
> http://www.h2database.com/html/datatypes.html#blob_type
> http://www.h2database.com/html/datatypes.html#clob_type
> http://www.h2database.com/html/datatypes.html#array_type
> http://www.h2database.com/html/datatypes.html#enum_type



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (IGNITE-6802) NullPointerException when WAL store and WAL archive paths are same

2017-11-10 Thread Ryabov Dmitrii (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryabov Dmitrii reassigned IGNITE-6802:
--

Assignee: Ryabov Dmitrii

> NullPointerException when WAL store and WAL archive paths are same 
> ---
>
> Key: IGNITE-6802
> URL: https://issues.apache.org/jira/browse/IGNITE-6802
> Project: Ignite
>  Issue Type: Bug
>  Security Level: Public(Viewable by anyone) 
>Affects Versions: 2.2
>Reporter: Alexey Kukushkin
>Assignee: Ryabov Dmitrii
>  Labels: newbie, usability
> Fix For: 2.4
>
>
> Configuring WAL store and WAL archive paths to be the same results in 
> NullPointerException. We should display a meaningful error about the 
> misconfiguration instead.
> See 
> http://apache-ignite-users.70518.x6.nabble.com/NullPointerException-in-FileWriteAheadLogManager-java-1313-tc17860.html
> The thread includes the reproducer code and stack trace. I was able to 
> reproduce the issue with today's (31-Oct-2017) Apache master.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (IGNITE-6862) SparseDistributedMatrixStorage cache config possibly allows read of old state of matrix

2017-11-10 Thread Artem Malykh (JIRA)
Artem Malykh created IGNITE-6862:


 Summary: SparseDistributedMatrixStorage cache config possibly 
allows  read of old state of matrix
 Key: IGNITE-6862
 URL: https://issues.apache.org/jira/browse/IGNITE-6862
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
Reporter: Artem Malykh
 Fix For: 2.4


Because synchronization mode is PRIMARY_SYNC, but by default we have 
readFromBackup=true, we can read old state of matrix from backups. It seem to 
be an issue.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (IGNITE-6699) Optimize client-side data streamer performance

2017-11-10 Thread Ivan Fedotov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Fedotov reassigned IGNITE-6699:


Assignee: Ivan Fedotov

> Optimize client-side data streamer performance
> --
>
> Key: IGNITE-6699
> URL: https://issues.apache.org/jira/browse/IGNITE-6699
> Project: Ignite
>  Issue Type: Task
>  Security Level: Public(Viewable by anyone) 
>  Components: streaming
>Affects Versions: 2.3
>Reporter: Vladimir Ozerov
>Assignee: Ivan Fedotov
>  Labels: iep-1, performance
> Fix For: 2.4
>
>
> Currently if a user has several server nodes and a single client node with 
> single thread pushing data to streamer, he will not be able to load data at 
> maximum speed. On the other hand, if he start several data loading threads, 
> throughput will increase. 
> One of root causes of this is bad data streamer design. Method 
> {{IgniteDataStreamer.addData(K, V)}} returns new feature for every operation, 
> this is too fine grained approach. Also it generates a lot of garbage and 
> causes contention on streamer internals. 
> Proposed implementation flow:
> 1) Compare performance of {{addData(K, V)}} vs {{addData(Collection)}} 
> methods from one thread in distributed environment. The latter should show 
> considerably higher throughput.
> 2) Users should receive per-batch features, rather than per-key. 
> 3) Try caching thread data in some collection until it is large enough to 
> avoid contention and unnecessary allocations.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (IGNITE-6860) Lack of context information upon serializing and marshalling (writeObject and writeFields)

2017-11-10 Thread Stanilovsky Evgeny (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stanilovsky Evgeny reassigned IGNITE-6860:
--

Assignee: Stanilovsky Evgeny

> Lack of context information upon serializing and marshalling (writeObject and 
> writeFields)
> --
>
> Key: IGNITE-6860
> URL: https://issues.apache.org/jira/browse/IGNITE-6860
> Project: Ignite
>  Issue Type: Bug
>  Security Level: Public(Viewable by anyone) 
>  Components: general
>Affects Versions: 2.3
>Reporter: Alexandr Kuramshin
>Assignee: Stanilovsky Evgeny
> Fix For: 2.4
>
>
> Having the stack trace
> {noformat}
> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to marshal 
> object with optimized marshaller: 
> [org.apache.logging.log4j.core.config.AppenderControl@302e61a8]
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:186)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectField(BinaryWriterExImpl.java:1160)
>   at 
> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write(BinaryFieldAccessor.java:663)
>   at 
> org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:793)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:206)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectField(BinaryWriterExImpl.java:1160)
>   at 
> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write(BinaryFieldAccessor.java:663)
>   at 
> org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:793)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:206)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectField(BinaryWriterExImpl.java:1160)
>   at 
> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write(BinaryFieldAccessor.java:663)
>   at 
> org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:793)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:206)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectField(BinaryWriterExImpl.java:1160)
>   at 
> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write(BinaryFieldAccessor.java:663)
>   at 
> org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:793)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:206)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496)
>   at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectField(BinaryWriterExImpl.java:1160)
>   at 
> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write(BinaryFieldAccessor.java:663)
>   at 
> org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:793)
>   at 
> 

[jira] [Assigned] (IGNITE-6406) SQL: CREATE INDEX should fill index from multiple threads

2017-11-10 Thread Roman Kondakov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Kondakov reassigned IGNITE-6406:
--

Assignee: Roman Kondakov

> SQL: CREATE INDEX should fill index from multiple threads
> -
>
> Key: IGNITE-6406
> URL: https://issues.apache.org/jira/browse/IGNITE-6406
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 2.1
>Reporter: Vladimir Ozerov
>Assignee: Roman Kondakov
>  Labels: iep-1, performance
>
> Currently when {{CREATE INDEX}} command is executed, we fill new index from a 
> single thread. Low CPU, long create time. We can trade off CPU vs time by 
> adding more threads. 
> Probably number of threads should be specified in {{CREATE INDEX}} command as 
> a hint.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-6861) SQL parser: support CREATE TABLE and DROP TABLE commands

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-6861:

Description: Partial implementation is ready (see PR). Need to make sure 
that the full syntax is supported.

> SQL parser: support CREATE TABLE and DROP TABLE commands
> 
>
> Key: IGNITE-6861
> URL: https://issues.apache.org/jira/browse/IGNITE-6861
> Project: Ignite
>  Issue Type: Task
>  Security Level: Public(Viewable by anyone) 
>  Components: sql
>Reporter: Vladimir Ozerov
> Fix For: 2.4
>
>
> Partial implementation is ready (see PR). Need to make sure that the full 
> syntax is supported.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (IGNITE-6769) SQL: prototype for hand-made parser

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov resolved IGNITE-6769.
-
Resolution: Fixed

Prototype is ready and is merged to master as a part of IGNITE-6840.

> SQL: prototype for hand-made parser
> ---
>
> Key: IGNITE-6769
> URL: https://issues.apache.org/jira/browse/IGNITE-6769
> Project: Ignite
>  Issue Type: Task
>  Security Level: Public(Viewable by anyone) 
>  Components: sql
>Reporter: Vladimir Ozerov
>Assignee: Vladimir Ozerov
> Fix For: 2.4
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Closed] (IGNITE-6769) SQL: prototype for hand-made parser

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov closed IGNITE-6769.
---

> SQL: prototype for hand-made parser
> ---
>
> Key: IGNITE-6769
> URL: https://issues.apache.org/jira/browse/IGNITE-6769
> Project: Ignite
>  Issue Type: Task
>  Security Level: Public(Viewable by anyone) 
>  Components: sql
>Reporter: Vladimir Ozerov
>Assignee: Vladimir Ozerov
> Fix For: 2.4
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6769) SQL: prototype for hand-made parser

2017-11-10 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247383#comment-16247383
 ] 

ASF GitHub Bot commented on IGNITE-6769:


Github user devozerov closed the pull request at:

https://github.com/apache/ignite/pull/2940


> SQL: prototype for hand-made parser
> ---
>
> Key: IGNITE-6769
> URL: https://issues.apache.org/jira/browse/IGNITE-6769
> Project: Ignite
>  Issue Type: Task
>  Security Level: Public(Viewable by anyone) 
>  Components: sql
>Reporter: Vladimir Ozerov
>Assignee: Vladimir Ozerov
> Fix For: 2.4
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6861) SQL parser: support CREATE TABLE and DROP TABLE commands

2017-11-10 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247384#comment-16247384
 ] 

ASF GitHub Bot commented on IGNITE-6861:


GitHub user devozerov opened a pull request:

https://github.com/apache/ignite/pull/3012

IGNITE-6861



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6861

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3012.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3012


commit 6ab3e44ff82e47c8904d34e1ddf048598f7e80d1
Author: devozerov 
Date:   2017-10-19T20:48:34Z

Simple lexer implementation.

commit adbb1106cadd94bb0b1ca6faed4271739f9ac72c
Author: devozerov 
Date:   2017-10-27T06:19:39Z

Merge branch 'master' into vozerov-sql-parse

commit 16e932244a756f4057ec274e94b29bdbf0132521
Author: devozerov 
Date:   2017-10-27T06:24:50Z

Merge branch 'master' into vozerov-sql-parse

commit 4f36f62cf05c30b23b14db9482b33a6ebc5d3675
Author: devozerov 
Date:   2017-10-27T06:42:24Z

Skeleton.

commit f3fa23fd695604015697ee1777212a37efab
Author: devozerov 
Date:   2017-10-27T06:53:25Z

Token types.

commit 0832c0262d45282e3c451c7dcc95ae465ef51b2a
Author: devozerov 
Date:   2017-10-27T07:07:06Z

WIP.

commit 27586e19083298501bb35b6c1be4255911d0d2ec
Author: devozerov 
Date:   2017-10-27T07:09:18Z

Minors.

commit 03abe1e8104c06ca18fb218385aba20bbaf27961
Author: devozerov 
Date:   2017-10-27T07:24:44Z

WIP.

commit a7378a8c91391e13c42c56833ae562d54512
Author: devozerov 
Date:   2017-10-27T07:46:06Z

WIP.

commit 6b31cdc46e362c401152f97a118c1ac801e86743
Author: devozerov 
Date:   2017-10-27T07:58:40Z

WIP.

commit c8b9063d3a99be0a70099da73a8e1b935275373d
Author: devozerov 
Date:   2017-10-27T08:10:57Z

WIP on DROP INDEX.

commit 8784139e5a375a78286ff560bc65c73c71ef5c1c
Author: devozerov 
Date:   2017-10-27T08:22:16Z

DROP INDEX done.

commit e02c09d6faebf314bfb3b9a7133838e34524c095
Author: devozerov 
Date:   2017-10-27T09:10:49Z

DROP INDEX.

commit e9a904489052e71cc76bc4a5d9c07c866f560724
Author: devozerov 
Date:   2017-10-27T09:17:58Z

Fixes.

commit 8f67b2def35ce8df727406e3e97a03a404e8d3af
Author: devozerov 
Date:   2017-10-27T09:49:53Z

Finished CREATE INDEX.

commit 4de8b28201a7eb028c78990b593784d05a3190ba
Author: devozerov 
Date:   2017-10-27T09:53:37Z

Finished CREATE INDEX.

commit 4522d8c2395f4410e0df16a5e72b84eb32426e61
Author: devozerov 
Date:   2017-10-27T09:56:13Z

Minors.

commit 1732a3f52d4ba0b1c65598cf7601c08d069ad49b
Author: devozerov 
Date:   2017-10-27T13:15:32Z

Merge branch 'master' into ignite-6769

commit c987773daec2d03c42e5476036fdf60ef0b1090d
Author: devozerov 
Date:   2017-10-27T13:18:23Z

Minors.

commit 2d40e5fcbf6077362eaf2cf67952011ed514efee
Author: devozerov 
Date:   2017-10-27T13:31:19Z

Look ahead support.

commit 52f3dd62c0c02618a7eece916a9b6c36ea0e60bd
Author: devozerov 
Date:   2017-10-27T14:07:02Z

Simplifying.

commit 2a118f74ea75c4ef917e2412e3e1e4a4cdffdac4
Author: devozerov 
Date:   2017-10-27T14:08:24Z

Minors.

commit 58208e4aa7b6fe049fda649e12f08d1da7e93b6a
Author: devozerov 
Date:   2017-10-27T14:23:24Z

Refactoring.

commit c0281f6fa294523a14254dc3bbd67f054ad57dba
Author: devozerov 
Date:   2017-10-27T14:29:06Z

Refactoring.

commit a04c1e86cdee61aacf3fa385b68b3ccdc88a20b5
Author: devozerov 
Date:   2017-10-27T14:46:17Z

INDEX TODOs.

commit ec9df63abc3acab51bc6352b3d6c359d8b8cc67f
Author: devozerov 
Date:   2017-10-30T09:17:57Z

Merge branch 'master' into ignite-6769

commit 3ed74247504858154b396394f5f522cc5e8e610d
Author: devozerov 
Date:   2017-10-30T09:26:22Z

Spatial index processing.

commit 96772985a0a0137ee403a4ea17e58f3ac833c526
Author: devozerov 
Date:   2017-10-30T09:38:42Z

Started work on CREATE TABLE.

commit 547dad042019ae6674a448adde8f689d5f5add1a
Author: devozerov 
Date:   2017-10-30T12:51:47Z

WIP on CREATE TABLE.

commit a44c0a6ab44218f95dac1b3654da9c03baea013f
Author: devozerov 
Date:   

[jira] [Created] (IGNITE-6861) SQL parser: support CREATE TABLE and DROP TABLE commands

2017-11-10 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-6861:
---

 Summary: SQL parser: support CREATE TABLE and DROP TABLE commands
 Key: IGNITE-6861
 URL: https://issues.apache.org/jira/browse/IGNITE-6861
 Project: Ignite
  Issue Type: Task
  Security Level: Public (Viewable by anyone)
  Components: sql
Reporter: Vladimir Ozerov
 Fix For: 2.4






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (IGNITE-6386) Introduction of distributed neural networks.

2017-11-10 Thread Yury Babak (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yury Babak reassigned IGNITE-6386:
--

Assignee: Yury Babak

> Introduction of distributed neural networks.
> 
>
> Key: IGNITE-6386
> URL: https://issues.apache.org/jira/browse/IGNITE-6386
> Project: Ignite
>  Issue Type: New Feature
>  Components: ml
>Reporter: Yury Babak
>Assignee: Yury Babak
>
> We want to have deep learning algorithms and for this we need to implement 
> neural network over Apache Ignite. Currently we think about using 
> [dl4j|https://deeplearning4j.org] as backend but in this case we cannot  
> train one model over multiple nodes efficiently. Also we will think about 
> integration/connector with dl4j.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (IGNITE-6860) Lack of context information upon serializing and marshalling (writeObject and writeFields)

2017-11-10 Thread Alexandr Kuramshin (JIRA)
Alexandr Kuramshin created IGNITE-6860:
--

 Summary: Lack of context information upon serializing and 
marshalling (writeObject and writeFields)
 Key: IGNITE-6860
 URL: https://issues.apache.org/jira/browse/IGNITE-6860
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
  Components: general
Affects Versions: 2.3
Reporter: Alexandr Kuramshin
 Fix For: 2.4


Having the stack trace

{noformat}
Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to marshal 
object with optimized marshaller: 
[org.apache.logging.log4j.core.config.AppenderControl@302e61a8]
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:186)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectField(BinaryWriterExImpl.java:1160)
at 
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write(BinaryFieldAccessor.java:663)
at 
org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:793)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:206)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectField(BinaryWriterExImpl.java:1160)
at 
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write(BinaryFieldAccessor.java:663)
at 
org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:793)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:206)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectField(BinaryWriterExImpl.java:1160)
at 
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write(BinaryFieldAccessor.java:663)
at 
org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:793)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:206)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectField(BinaryWriterExImpl.java:1160)
at 
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write(BinaryFieldAccessor.java:663)
at 
org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:793)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:206)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectField(BinaryWriterExImpl.java:1160)
at 
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write(BinaryFieldAccessor.java:663)
at 
org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:793)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:206)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496)
at 

[jira] [Commented] (IGNITE-6812) Readme.md: Update link to TC build and remove gitter

2017-11-10 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247347#comment-16247347
 ] 

ASF GitHub Bot commented on IGNITE-6812:


Github user dspavlov closed the pull request at:

https://github.com/apache/ignite/pull/2967


> Readme.md: Update link to TC build and remove gitter
> 
>
> Key: IGNITE-6812
> URL: https://issues.apache.org/jira/browse/IGNITE-6812
> Project: Ignite
>  Issue Type: Bug
>  Security Level: Public(Viewable by anyone) 
>  Components: documentation
>Reporter: Dmitriy Pavlov
>Assignee: Dmitriy Pavlov
>Priority: Minor
> Fix For: 2.4
>
>
> Related discussion 
> http://apache-ignite-developers.2346864.n4.nabble.com/Set-gitter-aside-td23690.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6841) ODBC: Add new version for multiple result set functionality

2017-11-10 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247309#comment-16247309
 ] 

ASF GitHub Bot commented on IGNITE-6841:


Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2995


> ODBC: Add new version for multiple result set functionality
> ---
>
> Key: IGNITE-6841
> URL: https://issues.apache.org/jira/browse/IGNITE-6841
> Project: Ignite
>  Issue Type: Bug
>  Security Level: Public(Viewable by anyone) 
>  Components: odbc
>Affects Versions: 2.3
>Reporter: Igor Sapego
>Assignee: Igor Sapego
>  Labels: odbc
> Fix For: 2.4
>
>
> Changes made in IGNITE-6357 changed ODBC protocol, but protocol version was 
> not increased. Need to fix it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-6669) CacheStoreSessionListener#onSessionStart() #onSessionEnd() methods are called by GridCacheStoreManagerAdapter even if a store operation should not be performed.

2017-11-10 Thread Vyacheslav Koptilin (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin updated IGNITE-6669:

Fix Version/s: 2.4

> CacheStoreSessionListener#onSessionStart() #onSessionEnd() methods are called 
> by GridCacheStoreManagerAdapter even if a store operation should not be 
> performed.
> 
>
> Key: IGNITE-6669
> URL: https://issues.apache.org/jira/browse/IGNITE-6669
> Project: Ignite
>  Issue Type: Bug
>  Security Level: Public(Viewable by anyone) 
>  Components: cache
>Affects Versions: 1.8
>Reporter: Vyacheslav Koptilin
>Assignee: Vyacheslav Koptilin
> Fix For: 2.4
>
> Attachments: Reproducer.java
>
>
> In the case of transactional cache, which is configured using {{CacheStore}} 
> and {{CacheJdbcStoreSessionListener}}, every update triggers 
> {{CacheStoreSessionListener # onSessionStart()}} that results in creating the 
> connection to an underlying database, even if read-through and write-through 
> modes are disabled.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6702) Get rid of values deserialization in H2TreeIndex.getRowCount

2017-11-10 Thread Vladimir Ozerov (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247270#comment-16247270
 ] 

Vladimir Ozerov commented on IGNITE-6702:
-

The optimization is not applicable in general case due to backups and upcoming 
MVCC feature. However, we can do the following:
1) If cache is {{LOCAL}}/{{REPLICATED}}/{{PARTITIIONED(backups=0}} and MVCC is 
disabled - just call {{BPlusTree.size()}}
2) Otherwise - let's implement new cursor type which will count rows based on 
some predicate (see last argument in {{BPlusTree#find(L, L, 
java.lang.Object)}}) possibly avoiding deserialization.

> Get rid of values deserialization in H2TreeIndex.getRowCount
> 
>
> Key: IGNITE-6702
> URL: https://issues.apache.org/jira/browse/IGNITE-6702
> Project: Ignite
>  Issue Type: Improvement
>  Security Level: Public(Viewable by anyone) 
>  Components: sql
>Reporter: Semen Boikov
>  Labels: performance
> Fix For: 2.4
>
>
> It seems H2TreeIndex.getRowCount is called for 'select count(*) from x' 
> queries, now it is implemented via BPlusTree.find(x, x) method which 
> deserializes all values. Actually values are not needed there, need implement 
> method on BPlusTree computing size without deserialization.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-6633) Repair basic SQL functionality

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-6633:

Fix Version/s: (was: 2.4)

> Repair basic SQL functionality
> --
>
> Key: IGNITE-6633
> URL: https://issues.apache.org/jira/browse/IGNITE-6633
> Project: Ignite
>  Issue Type: Improvement
>  Security Level: Public(Viewable by anyone) 
>  Components: sql
>Reporter: Alexei Scherbakov
>
> For a long time SQL engine has known limitation (H2 related) [1]
> This is huge usability issue, because proposed workaround requires query 
> rewriting and is difficult to implement in some cases, e.g. when some kind of 
> query builder is used.
> I suggest to fix it at last.
> [1] 
> https://apacheignite.readme.io/v2.1/docs/sql-performance-and-debugging#sql-performance-and-usability-considerations



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Closed] (IGNITE-6633) Repair basic SQL functionality

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov closed IGNITE-6633.
---

> Repair basic SQL functionality
> --
>
> Key: IGNITE-6633
> URL: https://issues.apache.org/jira/browse/IGNITE-6633
> Project: Ignite
>  Issue Type: Improvement
>  Security Level: Public(Viewable by anyone) 
>  Components: sql
>Reporter: Alexei Scherbakov
>
> For a long time SQL engine has known limitation (H2 related) [1]
> This is huge usability issue, because proposed workaround requires query 
> rewriting and is difficult to implement in some cases, e.g. when some kind of 
> query builder is used.
> I suggest to fix it at last.
> [1] 
> https://apacheignite.readme.io/v2.1/docs/sql-performance-and-debugging#sql-performance-and-usability-considerations



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (IGNITE-6633) Repair basic SQL functionality

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov resolved IGNITE-6633.
-
Resolution: Duplicate

> Repair basic SQL functionality
> --
>
> Key: IGNITE-6633
> URL: https://issues.apache.org/jira/browse/IGNITE-6633
> Project: Ignite
>  Issue Type: Improvement
>  Security Level: Public(Viewable by anyone) 
>  Components: sql
>Reporter: Alexei Scherbakov
> Fix For: 2.4
>
>
> For a long time SQL engine has known limitation (H2 related) [1]
> This is huge usability issue, because proposed workaround requires query 
> rewriting and is difficult to implement in some cases, e.g. when some kind of 
> query builder is used.
> I suggest to fix it at last.
> [1] 
> https://apacheignite.readme.io/v2.1/docs/sql-performance-and-debugging#sql-performance-and-usability-considerations



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-5482) Implement basic caching of query results

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-5482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-5482:

Labels: performance  (was: )

> Implement basic caching of query results
> 
>
> Key: IGNITE-5482
> URL: https://issues.apache.org/jira/browse/IGNITE-5482
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 2.0
>Reporter: Alexander Paschenko
>  Labels: performance
>
> Sergi suggested that we reuse results of the same queries running 
> simultaneously - i.e. if a query is being executed with the same arguments 
> and flags again and again, there's no need to do actual querying of data, 
> instead we can really run query once while other simultaneous runs will wait 
> for those results.
> This strategy will be implemented on MAP stage of distributed query.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6752) JDBC thin: connection property refactoring

2017-11-10 Thread Taras Ledkov (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247258#comment-16247258
 ] 

Taras Ledkov commented on IGNITE-6752:
--

[~skalashnikov], please review the changes.

> JDBC thin: connection property refactoring
> --
>
> Key: IGNITE-6752
> URL: https://issues.apache.org/jira/browse/IGNITE-6752
> Project: Ignite
>  Issue Type: Task
>  Security Level: Public(Viewable by anyone) 
>  Components: jdbc
>Affects Versions: 2.2
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
> Fix For: 2.4
>
>
> The issues IGNITE-6140 and IGNITE-6625 have to connection properties 
> refactoring.
> Otherwise the logic to work with connection properties is separated between 
> several classes.
> Also, SSL implementation for JDB client adds many new properties.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Closed] (IGNITE-4957) First join query execution in client mode takes too long when there are many caches on remote nodes

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov closed IGNITE-4957.
---

> First join query execution in client mode takes too long when there are many 
> caches on remote nodes
> ---
>
> Key: IGNITE-4957
> URL: https://issues.apache.org/jira/browse/IGNITE-4957
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 1.9
>Reporter: Alexandr Fedotov
>  Labels: performance
> Attachments: first_run_from_client.png
>
>
> When there are many caches deployed on server nodes and a query containing 
> joins is executed on a client for the first time then it takes much time to 
> complete compared to the following executions.
> If caches aren't enabled locally then the first query tries to load all the 
> missing caches by calling
> {{org.apache.ignite.internal.processors.cache.GridCacheProcessor#createMissingCaches}}
> {{createMissingCaches}} internally sends a request per each registered but 
> not enabled cache.
> Performance could be improved by performing a batch request.
> VisualVM results for the first join query run are below
> !first_run_from_client.png!



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6839) Ignite Compatibility: flaky test testFoldersReuseCompatibility_2_1 & 2_2

2017-11-10 Thread Dmitriy Pavlov (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247256#comment-16247256
 ] 

Dmitriy Pavlov commented on IGNITE-6839:


Hi [~daradurvs], did you have a chance to look through my comments above? Would 
you please assign issue to yourself if it is problem in framework code?

> Ignite Compatibility: flaky test testFoldersReuseCompatibility_2_1 & 2_2
> 
>
> Key: IGNITE-6839
> URL: https://issues.apache.org/jira/browse/IGNITE-6839
> Project: Ignite
>  Issue Type: Bug
>  Security Level: Public(Viewable by anyone) 
>  Components: persistence
>Affects Versions: 2.3
>Reporter: Dmitriy Pavlov
>Assignee: Dmitriy Pavlov
>  Labels: MakeTeamcityGreenAgain
> Fix For: 2.4
>
>
> Ignite Compatibility: 2 test are flaky:
>   
> org.apache.ignite.compatibility.testsuites.IgniteCompatibilityBasicTestSuite: 
> org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.testFoldersReuseCompatibility_2_1
>  (fails: 2/55): 
> https://ci.ignite.apache.org/project.html?projectId=Ignite20Tests=-1418165996957466785=%3Cdefault%3E=testDetails
> https://ci.ignite.apache.org/project.html?projectId=Ignite20Tests=1638023358531502921=testDetails_Ignite20Tests=%3Cdefault%3E
> {noformat}
> junit.framework.AssertionFailedError: Directory [binary_meta, 
> 135628fa_5763_46a1_88ff_8c0c8e51fc4e] is expected to exist 
> [/data/teamcity/work/820be461cd64b574/work/binary_meta/135628fa_5763_46a1_88ff_8c0c8e51fc4e]
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.assertTrue(Assert.java:22)
> at junit.framework.TestCase.assertTrue(TestCase.java:192)
> at 
> org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.assertDirectoryExist(FoldersReuseCompatibilityTest.java:220)
> at 
> org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.assertPdsDirsDefaultExist(FoldersReuseCompatibilityTest.java:193)
> at 
> org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.runFoldersReuse(FoldersReuseCompatibilityTest.java:108)
> at 
> org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.testFoldersReuseCompatibility_2_1(FoldersReuseCompatibilityTest.java:86)
> {noformat}
> Test probaly indicates a bug in 
> https://issues.apache.org/jira/browse/IGNITE-6285



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (IGNITE-6858) Wait for exchange inside GridReduceQueryExecutor.query which never finishes due to opened transaction

2017-11-10 Thread Alexei Scherbakov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexei Scherbakov reassigned IGNITE-6858:
-

Assignee: Alexei Scherbakov  (was: Vladimir Ozerov)

> Wait for exchange inside GridReduceQueryExecutor.query which never finishes 
> due to opened transaction
> -
>
> Key: IGNITE-6858
> URL: https://issues.apache.org/jira/browse/IGNITE-6858
> Project: Ignite
>  Issue Type: Bug
>  Security Level: Public(Viewable by anyone) 
>  Components: sql
>Affects Versions: 2.3
>Reporter: Alexandr Kuramshin
>Assignee: Alexei Scherbakov
> Fix For: 2.4
>
>
> Infinite waiting in loop
> {noformat}
> for (int attempt = 0;; attempt++) {
> if (attempt != 0) {
> try {
> Thread.sleep(attempt * 10); // Wait for exchange.
> }
> catch (InterruptedException e) {
> Thread.currentThread().interrupt();
> throw new CacheException("Query was interrupted.", e);
> }
> }
> {noformat}
> because of exchange will wait for partition eviction with opened transaction 
> in a related thread
> {noformat}
> at java.lang.Thread.sleep(Native Method)
> at 
> o.a.i.i.processors.query.h2.twostep.GridReduceQueryExecutor.query(GridReduceQueryExecutor.java:546)
> at 
> o.a.i.i.processors.query.h2.IgniteH2Indexing$8.iterator(IgniteH2Indexing.java:1236)
> at 
> o.a.i.i.processors.cache.QueryCursorImpl.iterator(QueryCursorImpl.java:95)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (IGNITE-4957) First join query execution in client mode takes too long when there are many caches on remote nodes

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov resolved IGNITE-4957.
-
Resolution: Duplicate

> First join query execution in client mode takes too long when there are many 
> caches on remote nodes
> ---
>
> Key: IGNITE-4957
> URL: https://issues.apache.org/jira/browse/IGNITE-4957
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 1.9
>Reporter: Alexandr Fedotov
>  Labels: performance
> Attachments: first_run_from_client.png
>
>
> When there are many caches deployed on server nodes and a query containing 
> joins is executed on a client for the first time then it takes much time to 
> complete compared to the following executions.
> If caches aren't enabled locally then the first query tries to load all the 
> missing caches by calling
> {{org.apache.ignite.internal.processors.cache.GridCacheProcessor#createMissingCaches}}
> {{createMissingCaches}} internally sends a request per each registered but 
> not enabled cache.
> Performance could be improved by performing a batch request.
> VisualVM results for the first join query run are below
> !first_run_from_client.png!



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (IGNITE-6061) Increase default index inline size

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov resolved IGNITE-6061.
-
Resolution: Won't Fix

Not relevant.

> Increase default index inline size
> --
>
> Key: IGNITE-6061
> URL: https://issues.apache.org/jira/browse/IGNITE-6061
> Project: Ignite
>  Issue Type: Task
>  Components: persistence, sql
>Affects Versions: 2.1
>Reporter: Vladimir Ozerov
>  Labels: performance
>
> Currently default value of maximum inline size is set to 10 bytes. Looks like 
> this is too small for string keys (1 byte overhead, 4 bytes length, 5 bytes 
> for chars - am I right?). I would propose to increase it to greater value, 
> say, 32.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Closed] (IGNITE-6061) Increase default index inline size

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov closed IGNITE-6061.
---

> Increase default index inline size
> --
>
> Key: IGNITE-6061
> URL: https://issues.apache.org/jira/browse/IGNITE-6061
> Project: Ignite
>  Issue Type: Task
>  Components: persistence, sql
>Affects Versions: 2.1
>Reporter: Vladimir Ozerov
>  Labels: performance
>
> Currently default value of maximum inline size is set to 10 bytes. Looks like 
> this is too small for string keys (1 byte overhead, 4 bytes length, 5 bytes 
> for chars - am I right?). I would propose to increase it to greater value, 
> say, 32.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (IGNITE-5362) Improve test coverage of SQL schema-related changes

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-5362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov resolved IGNITE-5362.
-
Resolution: Won't Fix

Not relevant any more.

> Improve test coverage of SQL schema-related changes
> ---
>
> Key: IGNITE-5362
> URL: https://issues.apache.org/jira/browse/IGNITE-5362
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Reporter: Vladimir Ozerov
>  Labels: important
>
> Use {{SqlPublicSchemaSelfTest}} as a base. What to test:
> 1) Type name conflicts
> 2) Index and table name conflicts
> 3) Cleanup on cache destroy 
> 4) Multinode tests
> 5) Different cache modes
> 6) Client and server nodes



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Closed] (IGNITE-5362) Improve test coverage of SQL schema-related changes

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-5362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov closed IGNITE-5362.
---

> Improve test coverage of SQL schema-related changes
> ---
>
> Key: IGNITE-5362
> URL: https://issues.apache.org/jira/browse/IGNITE-5362
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Reporter: Vladimir Ozerov
>  Labels: important
>
> Use {{SqlPublicSchemaSelfTest}} as a base. What to test:
> 1) Type name conflicts
> 2) Index and table name conflicts
> 3) Cleanup on cache destroy 
> 4) Multinode tests
> 5) Different cache modes
> 6) Client and server nodes



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (IGNITE-5331) Investigate performance implications of SQL schema refactoring

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov resolved IGNITE-5331.
-
Resolution: Won't Fix

Not relevant any more: no performance degradation were observed.

> Investigate performance implications of SQL schema refactoring
> --
>
> Key: IGNITE-5331
> URL: https://issues.apache.org/jira/browse/IGNITE-5331
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Reporter: Vladimir Ozerov
>  Labels: performance
>
> SQL is now decoupled from concrete cache. It means tha:
> 1) Special cache-agnostic implementation {{CacheQueryObjectValueContext}} is 
> passed to binary objects, with {{copyOnGet}} always returning true.
> 2) {{GridH2ValueCacheObject.getObject(boolean)}} is now called with {{true}} 
> argument more oftner (see usages and Git history).
> All in all it means that more object copying could occur than before. We need 
> to understand whether performance is affected. 
> One important thing to consider is immutability of binary object. That is, 
> once created, {{BinaryObject}} never changes. It means that is 
> {{BinaryMarshaller}} is enabled, we can always avoid copying safely.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Closed] (IGNITE-5331) Investigate performance implications of SQL schema refactoring

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov closed IGNITE-5331.
---

> Investigate performance implications of SQL schema refactoring
> --
>
> Key: IGNITE-5331
> URL: https://issues.apache.org/jira/browse/IGNITE-5331
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Reporter: Vladimir Ozerov
>  Labels: performance
>
> SQL is now decoupled from concrete cache. It means tha:
> 1) Special cache-agnostic implementation {{CacheQueryObjectValueContext}} is 
> passed to binary objects, with {{copyOnGet}} always returning true.
> 2) {{GridH2ValueCacheObject.getObject(boolean)}} is now called with {{true}} 
> argument more oftner (see usages and Git history).
> All in all it means that more object copying could occur than before. We need 
> to understand whether performance is affected. 
> One important thing to consider is immutability of binary object. That is, 
> once created, {{BinaryObject}} never changes. It means that is 
> {{BinaryMarshaller}} is enabled, we can always avoid copying safely.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (IGNITE-6565) Use long type for size and keySize in cache metrics

2017-11-10 Thread Alexander Menshikov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Menshikov reassigned IGNITE-6565:
---

Assignee: Alexander Menshikov

> Use long type for size and keySize in cache metrics
> ---
>
> Key: IGNITE-6565
> URL: https://issues.apache.org/jira/browse/IGNITE-6565
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.2
>Reporter: Ilya Kasnacheev
>Assignee: Alexander Menshikov
>
> Currently it's int so for large caches there's no way to convey correct value.
> Should introduce getSizeLong() and getKeySizeLong()
> Also introduce the same in .Net and make sure that compatibility not broken 
> when passing OP_LOCAL_METRICS and OP_GLOBAL_METRICS.
> BTW do we need keySize at all? What's it for?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Closed] (IGNITE-4994) Restore GridH2TableSelfTest

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-4994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov closed IGNITE-4994.
---

> Restore GridH2TableSelfTest
> ---
>
> Key: IGNITE-4994
> URL: https://issues.apache.org/jira/browse/IGNITE-4994
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Reporter: Vladimir Ozerov
>
> Test is broken after recent refactoring due to DDL feature. Need to restore 
> it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (IGNITE-4994) Restore GridH2TableSelfTest

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-4994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov resolved IGNITE-4994.
-
Resolution: Won't Fix

Test is removed from the product.

> Restore GridH2TableSelfTest
> ---
>
> Key: IGNITE-4994
> URL: https://issues.apache.org/jira/browse/IGNITE-4994
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Reporter: Vladimir Ozerov
>
> Test is broken after recent refactoring due to DDL feature. Need to restore 
> it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Closed] (IGNITE-4660) Add DML capabilities to legacy JDBC driver

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-4660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov closed IGNITE-4660.
---

> Add DML capabilities to legacy JDBC driver
> --
>
> Key: IGNITE-4660
> URL: https://issues.apache.org/jira/browse/IGNITE-4660
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Reporter: Alexander Paschenko
>Priority: Minor
>
> Legacy Ignite JDBC driver lacks DML capabilities, but it turns out that there 
> still are plenty of its users who need DML too, so we should de-deprecate it 
> and enable updating operations in it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-6085) SQL: JOIN with multiple conditions is extremely slow

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-6085:

Fix Version/s: 2.4

> SQL: JOIN with multiple conditions is extremely slow
> 
>
> Key: IGNITE-6085
> URL: https://issues.apache.org/jira/browse/IGNITE-6085
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 2.1
>Reporter: Vladimir Ozerov
>  Labels: performance
> Fix For: 2.4
>
>
> Consider the following query:
> {code}
> SELECT ... FROM A a
> INNER JOIN B b ON b.id = a.foreign_id1 OR b.id = a.foreign_id2
> {code}
> In this case H2 cannot use indexes on {{foreign_id1}} or {{foreign_id2}} 
> columns and query execution takes extraordinary time. Known workaround for a 
> problem is to apply multiple JOINs, e.g.:
> {code}
> SELECT ... FROM A a
> LEFT OUTER JOIN B b1 ON b1.id = a.foreign_id1 
> LEFT OUTER JOIN B b2 ON b2.id = a.foreign_id2
> WHERE b1.id IS NOT NULL AND b2.id IS NOT NULL
> {code}
> On a single real-world scenario it improved exeution time by a factor of 500 
> (from 4s to 80ms).
> Something is terribly wrong here. Probably, H2 cannot perform necessary query 
> re-write, or cannot understand how to use index. Let's find a way to fix that.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (IGNITE-4660) Add DML capabilities to legacy JDBC driver

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-4660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov resolved IGNITE-4660.
-
Resolution: Won't Fix

Driver is deprecated, hence fix is not needed.

> Add DML capabilities to legacy JDBC driver
> --
>
> Key: IGNITE-4660
> URL: https://issues.apache.org/jira/browse/IGNITE-4660
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Reporter: Alexander Paschenko
>Priority: Minor
>
> Legacy Ignite JDBC driver lacks DML capabilities, but it turns out that there 
> still are plenty of its users who need DML too, so we should de-deprecate it 
> and enable updating operations in it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6085) SQL: JOIN with multiple conditions is extremely slow

2017-11-10 Thread Vladimir Ozerov (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247236#comment-16247236
 ] 

Vladimir Ozerov commented on IGNITE-6085:
-

See IGNITE-4150. Probably similar fix ti H2 could be applied.

> SQL: JOIN with multiple conditions is extremely slow
> 
>
> Key: IGNITE-6085
> URL: https://issues.apache.org/jira/browse/IGNITE-6085
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 2.1
>Reporter: Vladimir Ozerov
>  Labels: performance
> Fix For: 2.4
>
>
> Consider the following query:
> {code}
> SELECT ... FROM A a
> INNER JOIN B b ON b.id = a.foreign_id1 OR b.id = a.foreign_id2
> {code}
> In this case H2 cannot use indexes on {{foreign_id1}} or {{foreign_id2}} 
> columns and query execution takes extraordinary time. Known workaround for a 
> problem is to apply multiple JOINs, e.g.:
> {code}
> SELECT ... FROM A a
> LEFT OUTER JOIN B b1 ON b1.id = a.foreign_id1 
> LEFT OUTER JOIN B b2 ON b2.id = a.foreign_id2
> WHERE b1.id IS NOT NULL AND b2.id IS NOT NULL
> {code}
> On a single real-world scenario it improved exeution time by a factor of 500 
> (from 4s to 80ms).
> Something is terribly wrong here. Probably, H2 cannot perform necessary query 
> re-write, or cannot understand how to use index. Let's find a way to fix that.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-4150) B-Tree index cannot be used efficiently with IN clause.

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-4150:

Fix Version/s: 2.4

> B-Tree index cannot be used efficiently with IN clause.
> ---
>
> Key: IGNITE-4150
> URL: https://issues.apache.org/jira/browse/IGNITE-4150
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 1.7
>Reporter: Vladimir Ozerov
>  Labels: performance
> Fix For: 2.4
>
>
> Consider the following query:
> {code}
> SELECT * FROM table
> WHERE a = ? AND b IN (?, ?)
> {code}
> If there is an index {{(a, b)}}, it will not be used properly: only column 
> {{a}} will be used. This will leads to multiple unnecessary comparisons.
> Most obvious way to fix that - use temporary table and {{JOIN}}. However, 
> this approach doesn't work well when there are multiple {{IN}}'s. 
> Proper solution would be to hack deeper into H2.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-4150) B-Tree index cannot be used efficiently with IN clause.

2017-11-10 Thread Vladimir Ozerov (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247233#comment-16247233
 ] 

Vladimir Ozerov commented on IGNITE-4150:
-

Solution to this ticket: update to latest H2 version when it is available 
(greater than 1.4.196).

> B-Tree index cannot be used efficiently with IN clause.
> ---
>
> Key: IGNITE-4150
> URL: https://issues.apache.org/jira/browse/IGNITE-4150
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 1.7
>Reporter: Vladimir Ozerov
>  Labels: performance
> Fix For: 2.4
>
>
> Consider the following query:
> {code}
> SELECT * FROM table
> WHERE a = ? AND b IN (?, ?)
> {code}
> If there is an index {{(a, b)}}, it will not be used properly: only column 
> {{a}} will be used. This will leads to multiple unnecessary comparisons.
> Most obvious way to fix that - use temporary table and {{JOIN}}. However, 
> this approach doesn't work well when there are multiple {{IN}}'s. 
> Proper solution would be to hack deeper into H2.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-4150) B-Tree index cannot be used efficiently with IN clause.

2017-11-10 Thread Vladimir Ozerov (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247231#comment-16247231
 ] 

Vladimir Ozerov commented on IGNITE-4150:
-

Fixed in 
https://github.com/h2database/h2database/commit/7ec1dbee0ad6a34906ec172afa5645b983db3b3f

> B-Tree index cannot be used efficiently with IN clause.
> ---
>
> Key: IGNITE-4150
> URL: https://issues.apache.org/jira/browse/IGNITE-4150
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 1.7
>Reporter: Vladimir Ozerov
>  Labels: performance
> Fix For: 2.4
>
>
> Consider the following query:
> {code}
> SELECT * FROM table
> WHERE a = ? AND b IN (?, ?)
> {code}
> If there is an index {{(a, b)}}, it will not be used properly: only column 
> {{a}} will be used. This will leads to multiple unnecessary comparisons.
> Most obvious way to fix that - use temporary table and {{JOIN}}. However, 
> this approach doesn't work well when there are multiple {{IN}}'s. 
> Proper solution would be to hack deeper into H2.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (IGNITE-6859) Web console: does not work in IE11

2017-11-10 Thread Ilya Borisov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Borisov reassigned IGNITE-6859:


Assignee: Andrey Novikov  (was: Ilya Borisov)

[~anovikov], please review.

> Web console: does not work in IE11
> --
>
> Key: IGNITE-6859
> URL: https://issues.apache.org/jira/browse/IGNITE-6859
> Project: Ignite
>  Issue Type: Bug
>  Security Level: Public(Viewable by anyone) 
>  Components: wizards
> Environment: WIndows 10, IE11
>Reporter: Ilya Borisov
>Assignee: Andrey Novikov
> Attachments: screenshot-1.png
>
>
> *What happens:*
> Web console does not work in IE11 at all, it gets stuck at loading screen 
> indefinitely and logs an error into console.
> *Expected behavior:*
> Web console should work in IE11.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-3869) Reduce number of temporary objects produced by H2

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-3869:

Labels:   (was: important)

> Reduce number of temporary objects produced by H2
> -
>
> Key: IGNITE-3869
> URL: https://issues.apache.org/jira/browse/IGNITE-3869
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 1.4
>Reporter: Denis Magda
>
> Presently during the execution of a query H2 generates significant number of 
> temporal objects (kind of wrappers) that eventually exhaust the heap and 
> trigger long GC pauses. 
> Need to revisit present implementation improving Ignite SQL engine and/or H2.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Closed] (IGNITE-3869) Reduce number of temporary objects produced by H2

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov closed IGNITE-3869.
---

> Reduce number of temporary objects produced by H2
> -
>
> Key: IGNITE-3869
> URL: https://issues.apache.org/jira/browse/IGNITE-3869
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 1.4
>Reporter: Denis Magda
>
> Presently during the execution of a query H2 generates significant number of 
> temporal objects (kind of wrappers) that eventually exhaust the heap and 
> trigger long GC pauses. 
> Need to revisit present implementation improving Ignite SQL engine and/or H2.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (IGNITE-3869) Reduce number of temporary objects produced by H2

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov resolved IGNITE-3869.
-
Resolution: Won't Fix

Not an issue at the moment. Most objects are short-lived, except of ones passed 
to IO messages. I tried to remove them from messages, but no performance 
changes were observed. Also we do not observe user complaints about GC due to 
H2. 

> Reduce number of temporary objects produced by H2
> -
>
> Key: IGNITE-3869
> URL: https://issues.apache.org/jira/browse/IGNITE-3869
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 1.4
>Reporter: Denis Magda
>
> Presently during the execution of a query H2 generates significant number of 
> temporal objects (kind of wrappers) that eventually exhaust the heap and 
> trigger long GC pauses. 
> Need to revisit present implementation improving Ignite SQL engine and/or H2.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (IGNITE-6859) Web console: does not work in IE11

2017-11-10 Thread Ilya Borisov (JIRA)
Ilya Borisov created IGNITE-6859:


 Summary: Web console: does not work in IE11
 Key: IGNITE-6859
 URL: https://issues.apache.org/jira/browse/IGNITE-6859
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
  Components: wizards
 Environment: WIndows 10, IE11
Reporter: Ilya Borisov
Assignee: Ilya Borisov


*What happens:*
Web console does not work in IE11 at all, it gets stuck at loading screen 
indefinitely and logs an error into console.

*Expected behavior:*
Web console should work in IE11.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-6859) Web console: does not work in IE11

2017-11-10 Thread Ilya Borisov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Borisov updated IGNITE-6859:
-
Attachment: screenshot-1.png

> Web console: does not work in IE11
> --
>
> Key: IGNITE-6859
> URL: https://issues.apache.org/jira/browse/IGNITE-6859
> Project: Ignite
>  Issue Type: Bug
>  Security Level: Public(Viewable by anyone) 
>  Components: wizards
> Environment: WIndows 10, IE11
>Reporter: Ilya Borisov
>Assignee: Ilya Borisov
> Attachments: screenshot-1.png
>
>
> *What happens:*
> Web console does not work in IE11 at all, it gets stuck at loading screen 
> indefinitely and logs an error into console.
> *Expected behavior:*
> Web console should work in IE11.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (IGNITE-6852) Kill ignite process in case of multithreaded reads cause segmentation fault

2017-11-10 Thread Dmitriy Pavlov (JIRA)

[ 
https://issues.apache.org/jira/browse/IGNITE-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247180#comment-16247180
 ] 

Dmitriy Pavlov commented on IGNITE-6852:


[~agoncharuk], should we have some guard and error/assetion here instead of 
crash?

> Kill ignite process in case of multithreaded reads cause segmentation fault
> ---
>
> Key: IGNITE-6852
> URL: https://issues.apache.org/jira/browse/IGNITE-6852
> Project: Ignite
>  Issue Type: Bug
>  Security Level: Public(Viewable by anyone) 
>  Components: persistence
>Affects Versions: 2.3
>Reporter: Dmitriy Pavlov
> Fix For: 2.4
>
> Attachments: hs_err_pid1462.log, hs_err_pid18664.log
>
>
> process is killed by PID, but running Ignite reads.
> Kill for Ignite 2.3 causes segmentation fault.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-5948) DDL: UNIQUE constraint support for CREATE TABLE.

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-5948:

Description: 
This is an umbrella ticket intended to aggregate all the activities related to 
{{UNIQUE}} constraint support for {{CREATE TABLE}} commands.

{{CREATE TABLE Persons (PersonID int, Age int, Salary int, UNIQUE (PersonID));}}

Ignite must prevent setting PersonID to non-unique value.

The feature has to be supported for:
ODBC and JDBC drivers.
Native APIs (Java, .NET, C++)

  was:
This is an umbrella ticket intended to aggregate all the activities related to 
{{UNIQUE}} constraint support for {{CREATE TABLE }}commands.

{{CREATE TABLE Persons (PersonID int, Age int, Salary int, UNIQUE (PersonID));}}

Ignite must prevent setting PersonID to non-unique value.

The feature has to be supported for:
ODBC and JDBC drivers.
Native APIs (Java, .NET, C++)


> DDL: UNIQUE constraint support for CREATE TABLE.
> 
>
> Key: IGNITE-5948
> URL: https://issues.apache.org/jira/browse/IGNITE-5948
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Reporter: Andrew Mashenkov
>
> This is an umbrella ticket intended to aggregate all the activities related 
> to {{UNIQUE}} constraint support for {{CREATE TABLE}} commands.
> {{CREATE TABLE Persons (PersonID int, Age int, Salary int, UNIQUE 
> (PersonID));}}
> Ignite must prevent setting PersonID to non-unique value.
> The feature has to be supported for:
> ODBC and JDBC drivers.
> Native APIs (Java, .NET, C++)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-6022) SQL: add native batch execution support for DML statements

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-6022:

Labels: iep-1 performance  (was: performance)

> SQL: add native batch execution support for DML statements
> --
>
> Key: IGNITE-6022
> URL: https://issues.apache.org/jira/browse/IGNITE-6022
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 2.1
>Reporter: Vladimir Ozerov
>  Labels: iep-1, performance
> Fix For: 2.4
>
>
> We have batch execution support for JDBC and ODBC drivers. This decreases 
> number of network hops. However, we do not have any batch execution support 
> on the server side. It means that for batch of N similar statements, every 
> statement will go through the whole execution chain - parsing, splitting, 
> communication with servers. And while parsing and splitting might be avoided 
> with help of statement cache, the most heavy part - network communication - 
> is still there.
> We need to investigate how to optimize the flow for batch updates. Possible 
> improvements:
> 1) Execute statements with certain degree of parallelism;
> 2) Send several query execution requests to the server at once;
> 3) Ensure that caches are used properly for batching - we should not parse 
> the same request multiple times.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (IGNITE-6022) SQL: add native batch execution support for DML statements

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-6022:

Fix Version/s: 2.4

> SQL: add native batch execution support for DML statements
> --
>
> Key: IGNITE-6022
> URL: https://issues.apache.org/jira/browse/IGNITE-6022
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 2.1
>Reporter: Vladimir Ozerov
>  Labels: iep-1, performance
> Fix For: 2.4
>
>
> We have batch execution support for JDBC and ODBC drivers. This decreases 
> number of network hops. However, we do not have any batch execution support 
> on the server side. It means that for batch of N similar statements, every 
> statement will go through the whole execution chain - parsing, splitting, 
> communication with servers. And while parsing and splitting might be avoided 
> with help of statement cache, the most heavy part - network communication - 
> is still there.
> We need to investigate how to optimize the flow for batch updates. Possible 
> improvements:
> 1) Execute statements with certain degree of parallelism;
> 2) Send several query execution requests to the server at once;
> 3) Ensure that caches are used properly for batching - we should not parse 
> the same request multiple times.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Closed] (IGNITE-6093) SQL: Add hints support

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov closed IGNITE-6093.
---

> SQL: Add hints support
> --
>
> Key: IGNITE-6093
> URL: https://issues.apache.org/jira/browse/IGNITE-6093
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 2.1
>Reporter: Vladimir Ozerov
>
> We already have a lot of query hints. Unfortunately, none of them are 
> supported in the query itself, so we have to set them through 
> {{SqlFieldsQuery}} setters or connection string parameters (for ODBC and 
> JDBC). 
> We need to learn H2 how to work with hints, and then apply it to our "Apache 
> Ignite" mode.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (IGNITE-6093) SQL: Add hints support

2017-11-10 Thread Vladimir Ozerov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-6093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov resolved IGNITE-6093.
-
Resolution: Won't Fix

Will be fixed as a part of our own parser development.

> SQL: Add hints support
> --
>
> Key: IGNITE-6093
> URL: https://issues.apache.org/jira/browse/IGNITE-6093
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Affects Versions: 2.1
>Reporter: Vladimir Ozerov
>
> We already have a lot of query hints. Unfortunately, none of them are 
> supported in the query itself, so we have to set them through 
> {{SqlFieldsQuery}} setters or connection string parameters (for ODBC and 
> JDBC). 
> We need to learn H2 how to work with hints, and then apply it to our "Apache 
> Ignite" mode.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


  1   2   >