Re: New thin JDBC driver

[Denis Magda]

> 1) Let's have separate connection string "jdbc:ignite:*thin*://" to avoid
> any confusion and interference with old drivers.

I would use “jdbc:ignite://“ for the newest driver not forcing to use “thin” in 
the connection string. I think it’s fine to break the compatibility there since 
“jdbc:ignite://“ is used by REST based driver.

—
Denis

> On May 24, 2017, at 5:51 AM, Vladimir Ozerov  wrote:
> 
> Igniters,
> 
> Currently we have two JDBC drivers:
> v1 - old thin driver, deprecated, works over GridClient
> v2 - fat driver, works over Ignite started in "client" mode.
> 
> Both of them have the same connection string "jdbc:ignite://"
> 
> Now we are working on new thin driver. It will use almost the same protocol
> as current ODBC driver, and will require only single port to be opened.
> Also this driver will not be coupled to particular cache. Now I am thinking
> on how to expose it to through public API. My considerations:
> 
> 1) Let's have separate connection string "jdbc:ignite:*thin*://" to avoid
> any confusion and interference with old drivers.
> 
> 2) Let's rename (actually deprecate + duplicate) OdbcConfiguration to
> SqlListenerConfiguration. This is where users will define port and other
> server-side parameters. It will be configurable through
> IgniteConfiguration.sqlListenerConfiguration.
> I think "listener" is a good term here, as it is also used in conventional
> RDBMS, such as Oracle.
> 
> 3) We need to decide whether to start listener service by default or not.
> Tough question. On the one hand, it is convenient if any Ignite node will
> be able to serve SQL requests with no additional configuration. On the
> other hand, it consumes resources and threads (GridNioServer), and normally
> users will have limited set of nodes which will serve user requests. For
> this reason I would not start it by default in the first place.
> 
> Please share your thoughts, especially about p.2 since I am in great doubts
> about it.
> 
> Vladimir.

Re: IgfsPerBlockLruEvictionPolicy does not work as expected any more, what shall we do with it?

[Denis Magda]

Hi Ivan,

I’m for this approach 

> 2) leave it as is, but explain in javadocs that it only works for on-heap
> layer, that does not in fact evict  blocks from the underlying offheap
> layer.

because it should be feasible to enable on-heap caching for IGFS, right? Using 
the memory policies. So, I would reimplement the tests with the on-heap caching 
enabling and checking up that data is pushed out of the heap.

—
Denis

> On May 24, 2017, at 9:57 AM, Ivan V.  wrote:
> 
> Hi, colleagues,
> 
> as Ignite caches moved to paged offheap memory , the
> IgfsPerBlockLruEvictionPolicy does not seem to work as expected any more,
> because
> 1) interface org.apache.ignite.cache.eviction.EvictionPolicy now only
> defines "eviction from on-heap", not a real eviction, because each on-heap
> cache is now accompanied with underlying off-heap cache. It can become
> "real eviction" only for "on-heap-only" mode caches, once they get
> implemented.
> 2) for off-heap eviction an entire page is evicted, not a specific k-v
> pair, and LRU policy is not exactly LRU any more (see
> org.apache.ignite.configuration.DataPageEvictionMode#RANDOM_LRU). So, it
> appears to be impossible to re-implement this policy for the off-heap layer.
> 
> Thus, now IgfsPerBlockLruEvictionPolicy is not quite valid, and some of
> corresponding tests fail
> (org.apache.ignite.internal.processors.igfs.IgfsCachePerBlockLruEvictionPolicySelfTest#testDataSizeEviction,
> org.apache.ignite.internal.processors.igfs.IgfsCachePerBlockLruEvictionPolicySelfTest#testBlockCountEviction)
> 
> So, the options I see are:
> 1) deprecate/remove IgfsPerBlockLruEvictionPolicy ;
> 2) leave it as is, but explain in javadocs that it only works for on-heap
> layer, that does not in fact evict  blocks from the underlying offheap
> layer.
> 
> Please share your opinions.

Re: IGNITE-5124 is ready for review

[Denis Magda]

Sergey,

Looks good to me. Please ask Alex G. to review the implementation part. I’ve 
taken look at the interfaces only.

—
Denis

> On May 24, 2017, at 9:04 AM, Sergey Chugunov  
> wrote:
> 
> Denis,
> 
> Thanks for your comments, I've addressed them and pushed changes.
> 
> Could you please review one more time?
> 
> Thanks,
> Sergey.
> 
> On Wed, May 24, 2017 at 12:20 AM, Denis Magda  wrote:
> 
>> Sergey,
>> 
>> Thanks, reviewed and left the notes in the upsource.
>> 
>> —
>> Denis
>> 
>>> On May 23, 2017, at 1:44 AM, Sergey Chugunov 
>> wrote:
>>> 
>>> Hello Igniters,
>>> 
>>> I implemented UX improvements for MemoryMetrics and added/clarified some
>>> documentation to public API.
>>> 
>>> Please find changes under pull request here
>>> , for discussions feel free
>> to
>>> leave comments in upsource review
>>> .
>>> 
>>> Thanks,
>>> Sergey.
>> 
>> 

Re: IGNITE-532 ready for review

[Denis Magda]

Got you. Alex K., could you review the integration then? You are an experienced 
Scala developer as I know.

—
Denis

> On May 24, 2017, at 12:37 AM, Roman Shtykh  wrote:
> 
> Denis,
> Lately I don't use Scala. Probably someone with more Scala experience will be 
> a better reviewer.
> Roman
> 
> 
> 
> 
> 
> 
>On Wednesday, May 24, 2017 1:03 AM, Denis Magda  wrote:
> 
> 
> Roman,
> 
> Would you mind reviewing the contribution?
> 
> —
> Denis
> 
>> On May 23, 2017, at 8:38 AM, Kozlov Maxim  wrote:
>> 
>> Hi guys!
>> 
>> Review please - https://github.com/apache/ignite/pull/1998 
>> 
>> https://issues.apache.org/jira/browse/IGNITE-532 
>> 
>> 
>> I'm add Akka Stream and Akka Actor integration, please see the solution. 
>> There is a question that the guru scala would like to ask. First 
>> implementation actor stream created a trait StreamAdapterTrait 
>> (IgniteAkkaActorStreamer.scala), because "extends Actor with 
>> StreamAdapter[T, K, V]" as "with" means that the class is using a Trait via 
>> mixin. Second implementation actor stream IgniteAkkaActorStreamerJava.scala 
>> "extends StreamAdapter[T, K, V] with Actor" use default abstract class 
>> StreamAdapter. Which option is better?
>> 
>> --
>> Best Regards,
>> Max K.
>> 
>> 
>> 
>> 
> 
> 

Re: [VOTE] Accept Contribution of Ignite Persistent Store

[Raúl Kripalani]

>
> Raul, I think everyone got confused on the process. At this point, the code
> has not been merged to master, and is sitting in a separate branch. I would
> recommend that we proceed with the vote. If the vote is declined, then we
> will toss the branch.


Just for the record. I don't want to complicate matters further, but IIRC,
the instant the code starts living in the ASF repo (which it already does),
it forms part of the project's codebase. And that means that the PMC and
the committers are responsible for it.

Hence the vote to accept responsibility on something that the PMC is
already responsible for, is somewhat questionable.

On the other hand, the IP clearance indicates what's being donated is the
tree of a particular commit from an external repo [0]. But it seems that
work was done after that commit in the external branch. Is it possible that
the code that was merged was different than that specific commit? Not sure
if this is relevant, as long as all files have the ASF license grant
header, but now is the time to ensure correct due diligence. Better to
ask...

Cos, or anybody?

[0]
https://github.com/agoncharuk/ignite/commit/1cce2fcc960c5098cc684b6138ed306daf5dd5e4


Raúl.

On Tue, May 23, 2017 at 5:04 PM, Dmitriy Setrakyan 
wrote:

> +1
>
> Raul, I think everyone got confused on the process. At this point, the code
> has not been merged to master, and is sitting in a separate branch. I would
> recommend that we proceed with the vote. If the vote is declined, then we
> will toss the branch.
>
> I believe that once this feature is fully released, which will make Apache
> Ignite into an SQL database and In-Memory data grid at the same time, the
> community and user base of Apache Ignite will get a healthy boost.
>
> D.
>
> On Tue, May 23, 2017 at 2:53 AM, Raúl Kripalani 
> wrote:
>
> > I'm not sure what the vote is for, if the code has already been merged to
> > the ASF repo — even if on a branch.
> >
> > I do accept the donation but I think the code should have remained
> outside
> > the official codebase until the VOTE passed.
> >
> > Not sure what the rest think.
> >
> > On 22 May 2017 22:55, "Denis Magda"  wrote:
> >
> > > Igniters,
> > >
> > > This branch (https://github.com/apache/ignite/tree/ignite-5267) adds a
> > > distributed and transactional Persistent Store to Apache Ignite
> project.
> > > The store seamlessly integrates with Apache Ignite 2.0 page memory
> > > architecture. One of the main advantages of the store is that Apache
> > Ignite
> > > becomes fully operational from disk (SSD or Flash) without any need to
> > > preload the data in memory. Plus, with full SQL support already
> available
> > > in Apache Ignite, this feature will allow Apache Ignite serve as a
> > > distributed SQL database, both in memory or on disk, while continuing
> to
> > > support all the existing functionality on the current API.
> > > More information here:
> > > - Persistent Store Overview: https://cwiki.apache.org/
> > > confluence/display/IGNITE/Persistent+Store+Overview
> > > - Persistent Store Internal Design: https://cwiki.apache.org/
> > > confluence/display/IGNITE/Persistent+Store+Internal+Design
> > > The Persistent Store was developed by GridGain outside of Apache
> > community
> > > because it was requested by one of GridGain’s customers. Presently,
> > > GridGain looks forward to donating the Persistent Store to ASF and
> given
> > > the size of the contribution, it is prudent to follow Apache's IP
> > clearance
> > > process.
> > > The SGA has been submitted and acknowledged by ASF Secretary. The IP
> > > clearance form can be found here: http://incubator.apache.org/
> > > ip-clearance/persistent-distributed-store-ignite.html
> > > This vote is to discover if the Apache Ignite PMC and community are in
> > > favour of accepting this contribution.
> > > This vote will be open for at least 72 hours:
> > > [ ] +1, accept contribution of the Persistent Store into the project
> > > [ ] 0, no opinion
> > > [ ] -1, reject contribution because...
> > >
> > > Regards,
> > > Denis
> > >
> > >
> >
>

Re: Geo spatial index

[Dmitriy Setrakyan]

Sergi,

While we are figuring this out, what happens to the GeoSpatial
functionality in the mean time? Is it going to work at all? If not, should
we throw some sort of exception?

D.

On Wed, May 24, 2017 at 1:44 AM, Sergi Vladykin 
wrote:

> Though this may require some changes in BPlusTree. Let me think.
>
> Sergi
>
> 2017-05-24 8:58 GMT+03:00 Sergi Vladykin :
>
> > It must not be too hard to implement kd-tree over b+tree [1]. Depending
> on
> > level we have to compare either X or Y coordinate.
> >
> > I think we will even have a performance boost for spatial indexes after
> > this change.
> >
> > [1] https://en.wikipedia.org/wiki/K-d_tree
> >
> > Sergi
> >
> > 2017-05-23 18:59 GMT+03:00 Denis Magda :
> >
> >> +1
> >>
> >> This looks natural considering that we switched to the new memory
> >> architecture. Sergi, how difficult is to support this?
> >>
> >> —
> >> Denis
> >>
> >> > On May 23, 2017, at 4:25 AM, Sergi Vladykin  >
> >> wrote:
> >> >
> >> > Guys,
> >> >
> >> > Looks like we have to move our geospatial indexes to the new approach
> >> with
> >> > BPlusTree. Right now it stores data in Java heap. This is especially
> >> > important because we are going to have a persistence layer donated by
> >> > GridGain and obviously geo spatial indexes will not work with it at
> all.
> >> >
> >> > Sergi
> >>
> >>
> >
>

Re: New thin JDBC driver

[Dmitriy Setrakyan]

Vladimir,

In general, all good ideas. However, I am concerned about having SQL
embedded into the server side configuration. I think we need to combine
JDBC, ODBC, REST, and Thin Client into the same approach and have the same
configuration for all of them.

What do you think?

D.

On Wed, May 24, 2017 at 5:51 AM, Vladimir Ozerov 
wrote:

> Igniters,
>
> Currently we have two JDBC drivers:
> v1 - old thin driver, deprecated, works over GridClient
> v2 - fat driver, works over Ignite started in "client" mode.
>
> Both of them have the same connection string "jdbc:ignite://"
>
> Now we are working on new thin driver. It will use almost the same protocol
> as current ODBC driver, and will require only single port to be opened.
> Also this driver will not be coupled to particular cache. Now I am thinking
> on how to expose it to through public API. My considerations:
>
> 1) Let's have separate connection string "jdbc:ignite:*thin*://" to avoid
> any confusion and interference with old drivers.
>
> 2) Let's rename (actually deprecate + duplicate) OdbcConfiguration to
> SqlListenerConfiguration. This is where users will define port and other
> server-side parameters. It will be configurable through
> IgniteConfiguration.sqlListenerConfiguration.
> I think "listener" is a good term here, as it is also used in conventional
> RDBMS, such as Oracle.
>
> 3) We need to decide whether to start listener service by default or not.
> Tough question. On the one hand, it is convenient if any Ignite node will
> be able to serve SQL requests with no additional configuration. On the
> other hand, it consumes resources and threads (GridNioServer), and normally
> users will have limited set of nodes which will serve user requests. For
> this reason I would not start it by default in the first place.
>
> Please share your thoughts, especially about p.2 since I am in great doubts
> about it.
>
> Vladimir.
>

[GitHub] ignite pull request #2002: Ignite 5267

[iveselovskiy]

GitHub user iveselovskiy opened a pull request:

https://github.com/apache/ignite/pull/2002

Ignite 5267



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/ignite ignite-5267

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2002.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2002


commit be881725b39926f62df437c2805ac9203fea5737
Author: Ilya Lantukh 
Date:   2016-10-11T16:19:20Z

gg-11595 : WIP.

commit e6d82d8da50b9af9607a914e00da3937d839862a
Author: Ilya Lantukh 
Date:   2016-10-12T16:20:50Z

gg-11595 : WIP.

commit cb9c18ccfc937b4a880eddb5df5d0389de9c7bee
Author: Ilya Lantukh 
Date:   2016-10-13T12:54:39Z

gg-11595 : WIP.

commit ed45f2238361e5bbbe907fdeb4b7a3d7b2b051dd
Author: Ilya Lantukh 
Date:   2016-10-27T13:02:28Z

gg-11595 : Support for restore with concurrent cache operations.

commit e95c68b77cbd3ad30d9f6d0b332137fb26000a41
Author: Ilya Lantukh 
Date:   2016-10-27T13:35:46Z

gg-11595 : Minors.

commit 7a44f1ebccf4fe0f26f54e070bbacbf24fbee3d7
Author: Ilya Lantukh 
Date:   2016-12-16T15:56:24Z

gg-11701 : WIP.

commit 3e2b28075ab5d00dce7fadf4769967f7a0ee2ee8
Author: Ilya Lantukh 
Date:   2016-12-19T12:25:11Z

Merge branches 'ignite-gg-11701' and 'ignite-gg-8.0.2.ea1' of 
https://github.com/gridgain/apache-ignite into ignite-gg-11701

commit cc568f9f6823de04f0beeca45ada4160a58307e4
Author: Ilya Lantukh 
Date:   2016-12-21T13:11:36Z

gg-11701 : WIP

commit e93f8d43a92dd8bde12c4677b5fcc6b0c2de6ce6
Author: Ilya Lantukh 
Date:   2016-12-27T10:45:29Z

Merge branches 'ignite-gg-11701' and 'ignite-gg-8.0.2.ea1' of 
https://github.com/gridgain/apache-ignite into ignite-gg-11701

commit 721f255eb2de1d8207e35328e2dec6514c22500d
Author: Ilya Lantukh 
Date:   2016-12-28T15:58:38Z

gg-11701 : Fixed preloading.

commit bed897dc01960dcbb7219ad948973e0f27bfa564
Author: Ilya Lantukh 
Date:   2016-12-28T16:31:51Z

gg-11701 : Fixed force keys request for single get.

commit 07535d92cd37cc949188434eb7ded2fc8d2e0647
Author: Ilya Lantukh 
Date:   2016-12-28T16:48:15Z

gg-11701 : Added check to multiple get.

commit ff9aa8898445c2a30e80fbc94f668eb5cd29a7d8
Author: Ilya Lantukh 
Date:   2016-12-29T13:04:49Z

Merge branches 'ignite-gg-11701' and 'ignite-gg-8.0.2.ea2' of 
https://github.com/gridgain/apache-ignite into ignite-gg-11701

commit 6cbe8a5a8ea5e4dacb8c5a859cb15aea5b3f38af
Author: Ilya Lantukh 
Date:   2017-01-09T15:22:15Z

gg-11701 : Fixed partition update counters comparison.

commit 3c276c9e092b6f6b4baf0b550ad89cb85ab9c42f
Author: Ilya Lantukh 
Date:   2017-01-10T10:25:09Z

gg-11701 : Implicit lateAffinityAssignment mode when persistence is enabled.

commit b3f7ae3bf903a0a6357163029ed310647877c8fc
Author: Ilya Lantukh 
Date:   2017-01-10T11:25:24Z

gg-11701 : Replaced redundant checks with assertions.

commit 2e55ddb600819dbf4684c0e97cc71a733167a4ce
Author: Ilya Lantukh 
Date:   2017-01-11T15:05:25Z

gg-11701 : Merge with 8.0.2.ea2

commit ecead988090e6a65ffdbb4098252ea26287fe36e
Author: Ilya Lantukh 
Date:   2017-01-11T15:09:25Z

gg-11701 : Merge with 8.0.2.ea2

commit adc0422592a18a7c2635e45185ef17852cd41952
Author: Glukos 
Date:   2017-01-13T17:23:06Z

GG-11595:
Remote exception wrapped on proxy level

commit 4505066481d4fff601a14d344dc8059f5dbe73de
Author: Ivan Rakov 
Date:   2017-01-13T21:32:08Z

GG-11595:
+ serialVersionUid

commit ba847555c86c00a288362e1cfad8c4c30883975c
Author: Ivan Rakov 
Date:   2017-01-16T15:08:35Z

Merge branch 'ignite-gg-8.0.2.ea2#' into ignite-gg-11595

# Conflicts:
#   modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/DynamicCacheChangeRequest.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheProcessor.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheProxy.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/atomic/GridDhtAtomicCache.java
#   
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/IgniteDynamicCacheStartSelfTest.java

commit 9e6732822061319df1e934361cfaf96e06f91573
Author: Alexey Goncharuk 

[jira] [Created] (IGNITE-5289) SQL: forbid WITH queries

[Alexander Paschenko (JIRA)]

Alexander Paschenko created IGNITE-5289:
---

 Summary: SQL: forbid WITH queries
 Key: IGNITE-5289
 URL: https://issues.apache.org/jira/browse/IGNITE-5289
 Project: Ignite
  Issue Type: Task
  Components: sql
Reporter: Alexander Paschenko
Assignee: Alexander Paschenko


Recursive queries starting with WITH keyword must be explicitly forbidden.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5288) Inconsistency of committed and the max memory numbers should not cause stopping cluster

[Dmitriy Pavlov (JIRA)]

Dmitriy Pavlov created IGNITE-5288:
--

 Summary: Inconsistency of committed and the max memory numbers 
should not cause stopping cluster
 Key: IGNITE-5288
 URL: https://issues.apache.org/jira/browse/IGNITE-5288
 Project: Ignite
  Issue Type: Improvement
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov


When using Ignite 1.7 together with Excelsior JET Ahead-Of-Time Compiler we get 
this error on various occasions:
 
java.lang.IllegalArgumentException: committed = 5274103808 should be < max = 
5274095616
at java.lang.management.MemoryUsage.(Unknown Source)
at sun.management.MemoryImpl.(Unknown Source)
at sun.management.MemoryImpl.getHeapMemoryUsage(Unknown Source)
at 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$6.getHeapMemoryCommitted(GridDiscoveryManager.java:778)
at 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$7.metrics(GridDiscoveryManager.java:878)
at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processHeartbeatMessage(ServerImpl.java:4651)
at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2325)
at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2135)
at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6123)
at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
 
The committed and the max memory numbers are not consistent and 
getHeapMemoryUsage() throws a IllegalArgumentEx about it.
 
 GridDiscoveryManager can get such error and not kill the whole process because 
of it.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Negative non-heap memory maximum

[Alexey Goncharuk]

I think it makes sense to switch non-heap memory metrics to new page memory
semantics, this should show a clean picture in the node output and will
also protect us from ambiguous -1 output.

2017-05-24 18:45 GMT+03:00 Dmitry Pavlov :

> Igniters,
>
>
>
> On my jdk 1.8.0_131 there is negative amount of maximum non heap memory
> returned by following code:
>
> ManagementFactory.getMemoryMXBean().getNonHeapMemoryUsage().getMax());
>
> returns -1.
>
>
>
> And this value is used in ignite metrics in
> GridLocalMetrics.getNonHeapMemoryMaximum() which becomes negative.
>
>
>
> I have an idea to take this value from memory policy as sum of max sizes of
> all memory policies, from MemoryPolicyConfiguration.getMaxSize()
>
>
>
> Would it be correct change to avoid negative values in metrics? Do you have
> any comments on this? Or is it better way to use commited memory
> getCommitted() to calculate free non heap space percent in metrics?
>
> Or is there any other way to obtain Max Non Heap memory size?
>
>
>
> Best Regards,
>
> Dmitriy Pavlov
>

Re: IgfsPerBlockLruEvictionPolicy does not work as expected any more, what shall we do with it?

[Ivan Rakov]

As for me, there's no reason to store IGFS blocks in cache with on-heap 
layer enabled.
I vote for the first option: deprecate/remove 
IgfsPerBlockLruEvictionPolicy (along with tests), recommend users to use 
page-based eviction if they need to evict blocks from primary IGFS.


--
Best Regards,
Ivan Rakov

On 24.05.2017 19:57, Ivan V. wrote:

Hi, colleagues,

as Ignite caches moved to paged offheap memory , the
IgfsPerBlockLruEvictionPolicy does not seem to work as expected any more,
because
1) interface org.apache.ignite.cache.eviction.EvictionPolicy now only
defines "eviction from on-heap", not a real eviction, because each on-heap
cache is now accompanied with underlying off-heap cache. It can become
"real eviction" only for "on-heap-only" mode caches, once they get
  implemented.
2) for off-heap eviction an entire page is evicted, not a specific k-v
pair, and LRU policy is not exactly LRU any more (see
org.apache.ignite.configuration.DataPageEvictionMode#RANDOM_LRU). So, it
appears to be impossible to re-implement this policy for the off-heap layer.

Thus, now IgfsPerBlockLruEvictionPolicy is not quite valid, and some of
corresponding tests fail
(org.apache.ignite.internal.processors.igfs.IgfsCachePerBlockLruEvictionPolicySelfTest#testDataSizeEviction,
  
org.apache.ignite.internal.processors.igfs.IgfsCachePerBlockLruEvictionPolicySelfTest#testBlockCountEviction)

So, the options I see are:
1) deprecate/remove IgfsPerBlockLruEvictionPolicy ;
2) leave it as is, but explain in javadocs that it only works for on-heap
layer, that does not in fact evict  blocks from the underlying offheap
layer.

Please share your opinions.

[GitHub] ignite pull request #2001: ignite-5283 Transaction recovery works incorrectl...

[agura]

GitHub user agura opened a pull request:

https://github.com/apache/ignite/pull/2001

ignite-5283 Transaction recovery works incorrectly with cache store and 
writeThrough enabled



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/agura/incubator-ignite ignite-5283

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2001.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2001


commit c6c198f38a012b37a2f1b1d5c7a58e1e6ec965c5
Author: vozerov-gridgain 
Date:   2016-10-24T08:25:49Z

Merge commit 'f9a0676f' into ignite-1.7.3

commit 958f16bb0ae0ebf5307029b9fafc2cd6f0df55c5
Author: vozerov-gridgain 
Date:   2016-10-24T08:26:35Z

IGNITE-4041: Ported pool processor to 1.7.3 version.

commit 09db00d0447ee8862388d4ed444f83a4f432b079
Author: vozerov-gridgain 
Date:   2016-10-24T08:29:22Z

Merge commit '359a392f' into ignite-1.7.3

commit 4dd735ffa01966b400afba2403ed609f54117884
Author: vozerov-gridgain 
Date:   2016-10-24T08:33:41Z

Merge commit '60d27547' into ignite-1.7.3

commit 2c4c7a4b0539aac35f91e0e9cc61f2e74a088d83
Author: vozerov-gridgain 
Date:   2016-10-24T08:50:29Z

Merge commit '2ab094e0' into ignite-1.7.3

commit b9bb1f6bc02cb3054b1e9b9bfbd34b886db77624
Author: vozerov-gridgain 
Date:   2016-10-24T08:50:51Z

Merge commit '2ab094e0' into ignite-1.7.3.

commit bab33016dd7ba99e288f2eaed089a57f867f8af2
Author: Pavel Tupitsyn 
Date:   2016-10-24T09:12:33Z

merge eaf8ae24: IGNITE-4034 Get rid of specialized methods in platform 
targets

commit f97170b723e8ced39c7acae19fadb0039e7cafb7
Author: Pavel Tupitsyn 
Date:   2016-10-24T09:21:00Z

merge eaf8ae24: IGNITE-4034 Get rid of specialized methods in platform 
targets

commit e73125d96c2a8d1f5a53a5fc3d3ed58db1dc7713
Author: Pavel Tupitsyn 
Date:   2016-10-24T09:50:22Z

merge eaf8ae24: IGNITE-4034 Get rid of specialized methods in platform 
targets

commit 6d69701038eeefaff2291a909d63d85e1ab9a376
Author: Pavel Tupitsyn 
Date:   2016-10-24T10:44:47Z

Merge commit '46dc9fc1' into ignite-1.7.3 (IGNITE-4034 Get rid of 
specialized methods in platform targets)

commit 96243fbc410226002e98f15d73aeb96ecec23a2b
Author: Pavel Tupitsyn 
Date:   2016-10-24T10:53:21Z

.NET: Fix ComputeApiTestFullFooter.TestForDaemons

commit 7a2f9ab516e8bc855c04d6ae947de05837f485f7
Author: vozerov-gridgain 
Date:   2016-10-24T10:54:46Z

Merge commit '7adfbcf1' into ignite-1.7.3

commit 2eee94c1ce141d02a4877a1f501c99eb16b0930f
Author: vozerov-gridgain 
Date:   2016-10-24T10:55:49Z

Merge remote-tracking branch 'upstream/ignite-1.7.3' into ignite-1.7.3

commit 9fc3b5095b825b9ca00b45c18f8072f998dd2adf
Author: Pavel Tupitsyn 
Date:   2016-08-02T15:37:52Z

IGNITE-3497 .NET: Improve IgniteConfigurationSection.xsd: add missing 
properties, enums, docs.

# Conflicts:
#   
modules/platforms/dotnet/Apache.Ignite.Core.Tests/IgniteConfigurationSerializerTest.cs

commit 04ac900a06b60fd58b7f5aa71c5371632fe03ede
Author: Pavel Tupitsyn 
Date:   2016-10-24T11:45:32Z

.NET: Fix TestCyrillicChars

commit f664751d2f5fda214ffa9df4f1375da4acd2f200
Author: isapego 
Date:   2016-10-24T11:50:50Z

Merge-related fix for C++.

commit 255b3a3e199b19757a511d4340f184a9de7afddc
Author: Pavel Tupitsyn 
Date:   2016-10-24T11:52:21Z

.NET: Fix ServicesTest.TestDuckTyping

commit ec12a9db2265180f96be72e2217e60ced856164e
Author: vozerov-gridgain 
Date:   2016-10-24T14:52:36Z

Minor fix for flags passed to GridCacheMapEntry.initialValue from data 
streamer isolated updater.

commit 02dd07a58277b357991c1f74a7dbdfdd9de2a2cc
Author: Pavel Tupitsyn 
Date:   2016-10-25T09:34:35Z

IGNITE-4074 Refactor async (*future) operations in PlatformTarget

commit 2a90fcaf8e46a829306ca92e226d984111b3aefe
Author: Pavel Tupitsyn 
Date:   2016-10-25T12:15:55Z

IGNITE-4028 .NET: Get rid of OP_META in PlatformAbstractTarget

commit bf266e971579cad0f40d233090c93f3acec06d0a
Author: vozerov-gridgain 
Date:   2016-10-17T08:26:12Z

IGNITE-4054: Hadoop: added map-reduce plan debug output.

commit 1e21d643aaf4d78e36fccaaeaa29cd3c49c90405
Author: sboikov 
Date:   2016-10-17T09:28:31Z

Fixed indexing test in according to changes from #80abd1b.

commit 51cef7c740828ffcc40f411a66e630421040a2d2
Author: Igor Sapego 
Date:   

Re: Minimize amount of inner classes, predicates, tuples, etc

[Dmitry Pavlov]

Hi Vladimir,

Do you have any idea how this rules may be automatically controlled by, for
example, code inspections and/or TeamCity?

I do beleive if rule is controlled by some automatic way (not only by
argreement) it has a chance to become true practice in real life.

If this excellent idea remains only agreement on dev list, it may be
forgotten in the future.

Sincerely,
Dmitriy Pavlov

вт, 16 мая 2017 г. в 13:01, Vladimir Ozerov :

> Igniters,
>
> Ignite is known to be complex product. Partially this complexity comes from
> ... complex problems we are trying to resolve. But partially it comes from
> how we write our code. I noticed several anti-patterns which makes our code
> hard to manage:
>
> 1) Inner classes (both static and non-static)
> 2) Usage of generic tuples
> 3) Usage of many classes from our "lang"package - closures, predicates,
> etc..
>
> When combined these anti-patterns makes our classes huge, leaky in terms of
> abstractions, hard to follow and refactor. I would like to propose to
> restrict usages of these constructs as much as possible in non-test code.
> Classes should be top-level by default, unless "inner" semantics are
> absolutely necessary or saves a lot of code. Tuples and closures should be
> replaced with concrete classes, specific to your case.
>
> Thoughts?
>
> Vladimir.
>

IgfsPerBlockLruEvictionPolicy does not work as expected any more, what shall we do with it?

[Ivan V.]

Hi, colleagues,

as Ignite caches moved to paged offheap memory , the
IgfsPerBlockLruEvictionPolicy does not seem to work as expected any more,
because
1) interface org.apache.ignite.cache.eviction.EvictionPolicy now only
defines "eviction from on-heap", not a real eviction, because each on-heap
cache is now accompanied with underlying off-heap cache. It can become
"real eviction" only for "on-heap-only" mode caches, once they get
 implemented.
2) for off-heap eviction an entire page is evicted, not a specific k-v
pair, and LRU policy is not exactly LRU any more (see
org.apache.ignite.configuration.DataPageEvictionMode#RANDOM_LRU). So, it
appears to be impossible to re-implement this policy for the off-heap layer.

Thus, now IgfsPerBlockLruEvictionPolicy is not quite valid, and some of
corresponding tests fail
(org.apache.ignite.internal.processors.igfs.IgfsCachePerBlockLruEvictionPolicySelfTest#testDataSizeEviction,
 
org.apache.ignite.internal.processors.igfs.IgfsCachePerBlockLruEvictionPolicySelfTest#testBlockCountEviction)

So, the options I see are:
1) deprecate/remove IgfsPerBlockLruEvictionPolicy ;
2) leave it as is, but explain in javadocs that it only works for on-heap
layer, that does not in fact evict  blocks from the underlying offheap
layer.

Please share your opinions.

[GitHub] ignite pull request #2000: IGNITE-3355: Implemented Compute::Call() for C++

[isapego]

GitHub user isapego opened a pull request:

https://github.com/apache/ignite/pull/2000

IGNITE-3355: Implemented Compute::Call() for C++



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-3355

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2000.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2000


commit 5e2848c2cefe00d7e1d9009cb320ff2910052ed9
Author: Igor Sapego 
Date:   2017-04-28T16:17:26Z

IGNITE-3355: Implemented Compute and ComputeImpl stubs

commit c3b62a74376efbe3da5bcc6fa76dd2b31b8eec6e
Author: Igor Sapego 
Date:   2017-04-28T17:43:49Z

IGNITE-3355: Implemented retrieval of the Compute for the projection.

commit 944a2b490b0caae9fe44279245083052dde48f67
Author: Igor Sapego 
Date:   2017-05-19T10:59:38Z

IGNITE-3355: Added stubs

commit 27ae62473b70a5a4a54d2a017a2701802bcbdce2
Author: Igor Sapego 
Date:   2017-05-19T12:52:25Z

IGNITE-3355: Added promise to task.

commit de639b778eb019e1316b7d627a1bc6ae8f6ba012
Author: Igor Sapego 
Date:   2017-05-19T13:46:09Z

IGNITE-3355: Implemented cancel for Futures.

commit 3f9f37c664656205197caac2ea8d378505cb34ed
Author: Igor Sapego 
Date:   2017-05-19T17:58:17Z

IGNITE-3355: Added test. Debug.

commit 5d46e4530755e53dce1ed4ca9a8731ab0379b3f1
Author: Igor Sapego 
Date:   2017-05-22T17:51:37Z

IGNITE-3355: Implemented all neccessary classes.

commit e588311069ea25bbc2a461fd681c301635198518
Author: Igor Sapego 
Date:   2017-05-22T18:24:19Z

IGNITE-3355: Implemented for local case.

commit 33477d5b1006ceb5f7d507b41dc51d2637482987
Author: Igor Sapego 
Date:   2017-05-23T12:36:53Z

IGNITE-3355: Added test for async call and for remote call.

commit 918b5543f7e86389d250129d19ca933b072d02f5
Author: Igor Sapego 
Date:   2017-05-23T13:14:33Z

IGNITE-3355: Added tests with exceptions

commit e8f12c53af865566a7a15511752fcdb0874ab43f
Author: Igor Sapego 
Date:   2017-05-23T17:26:15Z

IGNITE-3355: Implemented for remote case

commit 3a0132325a524521e7fefe237a2e35abc3429f9c
Author: Igor Sapego 
Date:   2017-05-23T18:18:35Z

IGNITE-3355: Implemented BinaryType

commit ae2152ea0fd77b402d174116a2d2c8ec31b2b64a
Author: Igor Sapego 
Date:   2017-05-24T12:44:43Z

IGNITE-3355: Re-factored internal classes

commit ff60463ce81587b7081b25f880e1fdd7873e5028
Author: Igor Sapego 
Date:   2017-05-24T16:29:10Z

IGNITE-3355: Changed automake files




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: IGNITE-5124 is ready for review

[Sergey Chugunov]

Denis,

Thanks for your comments, I've addressed them and pushed changes.

Could you please review one more time?

Thanks,
Sergey.

On Wed, May 24, 2017 at 12:20 AM, Denis Magda  wrote:

> Sergey,
>
> Thanks, reviewed and left the notes in the upsource.
>
> —
> Denis
>
> > On May 23, 2017, at 1:44 AM, Sergey Chugunov 
> wrote:
> >
> > Hello Igniters,
> >
> > I implemented UX improvements for MemoryMetrics and added/clarified some
> > documentation to public API.
> >
> > Please find changes under pull request here
> > , for discussions feel free
> to
> > leave comments in upsource review
> > .
> >
> > Thanks,
> > Sergey.
>
>

Negative non-heap memory maximum

[Dmitry Pavlov]

Igniters,



On my jdk 1.8.0_131 there is negative amount of maximum non heap memory
returned by following code:

ManagementFactory.getMemoryMXBean().getNonHeapMemoryUsage().getMax());

returns -1.



And this value is used in ignite metrics in
GridLocalMetrics.getNonHeapMemoryMaximum() which becomes negative.



I have an idea to take this value from memory policy as sum of max sizes of
all memory policies, from MemoryPolicyConfiguration.getMaxSize()



Would it be correct change to avoid negative values in metrics? Do you have
any comments on this? Or is it better way to use commited memory
getCommitted() to calculate free non heap space percent in metrics?

Or is there any other way to obtain Max Non Heap memory size?



Best Regards,

Dmitriy Pavlov

Re: [VOTE] Accept Contribution of Ignite Persistent Store

[Anton Vinogradov]

+1

On Wed, May 24, 2017 at 3:35 PM, Igor Sapego  wrote:

> +1
>
> Best Regards,
> Igor
>
> On Tue, May 23, 2017 at 7:04 PM, Dmitriy Setrakyan 
> wrote:
>
> > +1
> >
> > Raul, I think everyone got confused on the process. At this point, the
> code
> > has not been merged to master, and is sitting in a separate branch. I
> would
> > recommend that we proceed with the vote. If the vote is declined, then we
> > will toss the branch.
> >
> > I believe that once this feature is fully released, which will make
> Apache
> > Ignite into an SQL database and In-Memory data grid at the same time, the
> > community and user base of Apache Ignite will get a healthy boost.
> >
> > D.
> >
> > On Tue, May 23, 2017 at 2:53 AM, Raúl Kripalani 
> > wrote:
> >
> > > I'm not sure what the vote is for, if the code has already been merged
> to
> > > the ASF repo — even if on a branch.
> > >
> > > I do accept the donation but I think the code should have remained
> > outside
> > > the official codebase until the VOTE passed.
> > >
> > > Not sure what the rest think.
> > >
> > > On 22 May 2017 22:55, "Denis Magda"  wrote:
> > >
> > > > Igniters,
> > > >
> > > > This branch (https://github.com/apache/ignite/tree/ignite-5267)
> adds a
> > > > distributed and transactional Persistent Store to Apache Ignite
> > project.
> > > > The store seamlessly integrates with Apache Ignite 2.0 page memory
> > > > architecture. One of the main advantages of the store is that Apache
> > > Ignite
> > > > becomes fully operational from disk (SSD or Flash) without any need
> to
> > > > preload the data in memory. Plus, with full SQL support already
> > available
> > > > in Apache Ignite, this feature will allow Apache Ignite serve as a
> > > > distributed SQL database, both in memory or on disk, while continuing
> > to
> > > > support all the existing functionality on the current API.
> > > > More information here:
> > > > - Persistent Store Overview: https://cwiki.apache.org/
> > > > confluence/display/IGNITE/Persistent+Store+Overview
> > > > - Persistent Store Internal Design: https://cwiki.apache.org/
> > > > confluence/display/IGNITE/Persistent+Store+Internal+Design
> > > > The Persistent Store was developed by GridGain outside of Apache
> > > community
> > > > because it was requested by one of GridGain’s customers. Presently,
> > > > GridGain looks forward to donating the Persistent Store to ASF and
> > given
> > > > the size of the contribution, it is prudent to follow Apache's IP
> > > clearance
> > > > process.
> > > > The SGA has been submitted and acknowledged by ASF Secretary. The IP
> > > > clearance form can be found here: http://incubator.apache.org/
> > > > ip-clearance/persistent-distributed-store-ignite.html
> > > > This vote is to discover if the Apache Ignite PMC and community are
> in
> > > > favour of accepting this contribution.
> > > > This vote will be open for at least 72 hours:
> > > > [ ] +1, accept contribution of the Persistent Store into the project
> > > > [ ] 0, no opinion
> > > > [ ] -1, reject contribution because...
> > > >
> > > > Regards,
> > > > Denis
> > > >
> > > >
> > >
> >
>

New thin JDBC driver

[Vladimir Ozerov]

Igniters,

Currently we have two JDBC drivers:
v1 - old thin driver, deprecated, works over GridClient
v2 - fat driver, works over Ignite started in "client" mode.

Both of them have the same connection string "jdbc:ignite://"

Now we are working on new thin driver. It will use almost the same protocol
as current ODBC driver, and will require only single port to be opened.
Also this driver will not be coupled to particular cache. Now I am thinking
on how to expose it to through public API. My considerations:

1) Let's have separate connection string "jdbc:ignite:*thin*://" to avoid
any confusion and interference with old drivers.

2) Let's rename (actually deprecate + duplicate) OdbcConfiguration to
SqlListenerConfiguration. This is where users will define port and other
server-side parameters. It will be configurable through
IgniteConfiguration.sqlListenerConfiguration.
I think "listener" is a good term here, as it is also used in conventional
RDBMS, such as Oracle.

3) We need to decide whether to start listener service by default or not.
Tough question. On the one hand, it is convenient if any Ignite node will
be able to serve SQL requests with no additional configuration. On the
other hand, it consumes resources and threads (GridNioServer), and normally
users will have limited set of nodes which will serve user requests. For
this reason I would not start it by default in the first place.

Please share your thoughts, especially about p.2 since I am in great doubts
about it.

Vladimir.

Re: [VOTE] Accept Contribution of Ignite Persistent Store

[Igor Sapego]

+1

Best Regards,
Igor

On Tue, May 23, 2017 at 7:04 PM, Dmitriy Setrakyan 
wrote:

> +1
>
> Raul, I think everyone got confused on the process. At this point, the code
> has not been merged to master, and is sitting in a separate branch. I would
> recommend that we proceed with the vote. If the vote is declined, then we
> will toss the branch.
>
> I believe that once this feature is fully released, which will make Apache
> Ignite into an SQL database and In-Memory data grid at the same time, the
> community and user base of Apache Ignite will get a healthy boost.
>
> D.
>
> On Tue, May 23, 2017 at 2:53 AM, Raúl Kripalani 
> wrote:
>
> > I'm not sure what the vote is for, if the code has already been merged to
> > the ASF repo — even if on a branch.
> >
> > I do accept the donation but I think the code should have remained
> outside
> > the official codebase until the VOTE passed.
> >
> > Not sure what the rest think.
> >
> > On 22 May 2017 22:55, "Denis Magda"  wrote:
> >
> > > Igniters,
> > >
> > > This branch (https://github.com/apache/ignite/tree/ignite-5267) adds a
> > > distributed and transactional Persistent Store to Apache Ignite
> project.
> > > The store seamlessly integrates with Apache Ignite 2.0 page memory
> > > architecture. One of the main advantages of the store is that Apache
> > Ignite
> > > becomes fully operational from disk (SSD or Flash) without any need to
> > > preload the data in memory. Plus, with full SQL support already
> available
> > > in Apache Ignite, this feature will allow Apache Ignite serve as a
> > > distributed SQL database, both in memory or on disk, while continuing
> to
> > > support all the existing functionality on the current API.
> > > More information here:
> > > - Persistent Store Overview: https://cwiki.apache.org/
> > > confluence/display/IGNITE/Persistent+Store+Overview
> > > - Persistent Store Internal Design: https://cwiki.apache.org/
> > > confluence/display/IGNITE/Persistent+Store+Internal+Design
> > > The Persistent Store was developed by GridGain outside of Apache
> > community
> > > because it was requested by one of GridGain’s customers. Presently,
> > > GridGain looks forward to donating the Persistent Store to ASF and
> given
> > > the size of the contribution, it is prudent to follow Apache's IP
> > clearance
> > > process.
> > > The SGA has been submitted and acknowledged by ASF Secretary. The IP
> > > clearance form can be found here: http://incubator.apache.org/
> > > ip-clearance/persistent-distributed-store-ignite.html
> > > This vote is to discover if the Apache Ignite PMC and community are in
> > > favour of accepting this contribution.
> > > This vote will be open for at least 72 hours:
> > > [ ] +1, accept contribution of the Persistent Store into the project
> > > [ ] 0, no opinion
> > > [ ] -1, reject contribution because...
> > >
> > > Regards,
> > > Denis
> > >
> > >
> >
>

[GitHub] ignite pull request #1999: IGNITE-5284

[devozerov]

Github user devozerov closed the pull request at:

https://github.com/apache/ignite/pull/1999


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Remove/deprecate CacheConfiguration.sqlEscapeAll property

[Vladimir Ozerov]

I found the solution - no changes to public API, just rework how we treat
this flag internally.

On Tue, May 23, 2017 at 10:15 PM, Vladimir Ozerov 
wrote:

> Igniters,
>
> We have a property "CacheConfiguration.sqlEscapeAll". When enabled, all
> database objects belonging to the cache will be case-sensitive and allow
> special characters. Two problems here:
>
> 1) It doesn't allow me to decouple schemas from cache easily since I it
> force me to refer to cache context everywhere, exactly what I am trying to
> avoid.
>
> 2) It doesn't work with DDL at all. E.g. how should I process "MyIndex" if
> cache has "sqlEscapeAll=false"?
> CREATE INDEX "MyIndex" ON TABLE MY_TABLE;
>
> It appears, that we should do the following:
> 1) Remove/deprecate this property;
> 2) Add "escaped" property to QueryEntity and QueryIndex, so that decision
> can be made on per-object basis, rather than on per-cache.
> 3) In DDL we should decide whether object name is case-sensitive based on
> quotes around it's name:
>
> CREATE INDEX MyIndex;   // Insensitive
> CREATE INDEX "MyIndex"; // Sensitive
>
> E.g. this is precisely how Oracle and probably other databases work.
>
> Thoughts?
>

[jira] [Created] (IGNITE-5287) SQL: CacheConfiguration.sqlEscapeAll must affect only predefined objects

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-5287:
---

 Summary: SQL: CacheConfiguration.sqlEscapeAll must affect only 
predefined objects
 Key: IGNITE-5287
 URL: https://issues.apache.org/jira/browse/IGNITE-5287
 Project: Ignite
  Issue Type: Bug
  Components: cache, sql
Reporter: Vladimir Ozerov
Assignee: Alexander Paschenko
 Fix For: 2.1


Mentioned flag should work as follows:
1) If set, all object names (schema, QueryEntity (table, fields and aliases), 
QueryIndex (name and fields)) should be left as is.
2) Otherwise we should convert them to uppercase, and replace class delimiters 
(".", "$") with underscore. 

This is it. This property should not be used anywhere else. Allmost all usages 
of {{IgniteH2Indexing.escapeName}} should go away.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Geo spatial index

[Sergi Vladykin]

Though this may require some changes in BPlusTree. Let me think.

Sergi

2017-05-24 8:58 GMT+03:00 Sergi Vladykin :

> It must not be too hard to implement kd-tree over b+tree [1]. Depending on
> level we have to compare either X or Y coordinate.
>
> I think we will even have a performance boost for spatial indexes after
> this change.
>
> [1] https://en.wikipedia.org/wiki/K-d_tree
>
> Sergi
>
> 2017-05-23 18:59 GMT+03:00 Denis Magda :
>
>> +1
>>
>> This looks natural considering that we switched to the new memory
>> architecture. Sergi, how difficult is to support this?
>>
>> —
>> Denis
>>
>> > On May 23, 2017, at 4:25 AM, Sergi Vladykin 
>> wrote:
>> >
>> > Guys,
>> >
>> > Looks like we have to move our geospatial indexes to the new approach
>> with
>> > BPlusTree. Right now it stores data in Java heap. This is especially
>> > important because we are going to have a persistence layer donated by
>> > GridGain and obviously geo spatial indexes will not work with it at all.
>> >
>> > Sergi
>>
>>
>

[GitHub] ignite pull request #1920: IGNITE-5121: REST API call with empty cache name ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1920


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: IGNITE-532 ready for review

[Roman Shtykh]

Denis,
Lately I don't use Scala. Probably someone with more Scala experience will be a 
better reviewer.
Roman






On Wednesday, May 24, 2017 1:03 AM, Denis Magda  wrote:
 

 Roman,

Would you mind reviewing the contribution?

—
Denis

> On May 23, 2017, at 8:38 AM, Kozlov Maxim  wrote:
> 
> Hi guys!
> 
> Review please - https://github.com/apache/ignite/pull/1998 
> 
> https://issues.apache.org/jira/browse/IGNITE-532 
> 
> 
> I'm add Akka Stream and Akka Actor integration, please see the solution. 
> There is a question that the guru scala would like to ask. First 
> implementation actor stream created a trait StreamAdapterTrait 
> (IgniteAkkaActorStreamer.scala), because "extends Actor with StreamAdapter[T, 
> K, V]" as "with" means that the class is using a Trait via mixin. Second 
> implementation actor stream IgniteAkkaActorStreamerJava.scala "extends 
> StreamAdapter[T, K, V] with Actor" use default abstract class StreamAdapter. 
> Which option is better?
> 
> --
> Best Regards,
> Max K.
> 
> 
> 
> 


   

[jira] [Created] (IGNITE-5286) Reconsider deferredDelete implementation

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-5286:


 Summary: Reconsider deferredDelete implementation
 Key: IGNITE-5286
 URL: https://issues.apache.org/jira/browse/IGNITE-5286
 Project: Ignite
  Issue Type: Improvement
  Components: cache
Affects Versions: 2.0
Reporter: Alexey Goncharuk
 Fix For: 2.2


In current implementation entries marked for 'deferredDelete' should be stored 
on heap (otherwise information about remove is lost). This potentially can be 
an issue if there are a lot of removes.
(note: in current 'deferredDelete' implementation in Ignite there is a bug - 
https://issues.apache.org/jira/browse/IGNITE-3299).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Data compression in Ignite 2.0

[Vyacheslav Daradur]

Guys, any thoughts?

2017-05-16 13:40 GMT+03:00 Vyacheslav Daradur :

> Hi guys,
>
> I've prepared the PR to show my idea.
> https://github.com/apache/ignite/pull/1951/files
>
> About querying - I've just copied existing tests and have annotated the
> testing data.
> https://github.com/apache/ignite/pull/1951/files#diff-
> c19a9df4058141d059bb577e75244764
>
> It means fields which will be marked by @BinaryCompression will be
> compressed at marshalling via BinaryMarshaller.
>
> This solution has no effect on existing data or project architecture.
>
> I'll be glad to see your thougths.
>
>
> 2017-05-15 19:18 GMT+03:00 Vyacheslav Daradur :
>
>> Dmitriy,
>>
>> I have ready prototype. I want to show it.
>> It is always easier to discuss on example.
>>
>> 2017-05-15 19:02 GMT+03:00 Dmitriy Setrakyan :
>>
>>> Vyacheslav,
>>>
>>> I think it is a bit premature to provide a PR without getting a community
>>> consensus on the dev list. Please allow some time for the community to
>>> respond.
>>>
>>> D.
>>>
>>> On Mon, May 15, 2017 at 6:36 AM, Vyacheslav Daradur >> >
>>> wrote:
>>>
>>> > I created the ticket: https://issues.apache.org/jira
>>> /browse/IGNITE-5226
>>> >
>>> > I'll prepare a PR with described solution in couple of days.
>>> >
>>> > 2017-05-15 15:05 GMT+03:00 Vyacheslav Daradur :
>>> >
>>> > > Hi, Igniters!
>>> > >
>>> > > Apache 2.0 is released.
>>> > >
>>> > > Let's continue the discussion about a compression design.
>>> > >
>>> > > At the moment, I found only one solution which is compatible with
>>> > querying
>>> > > and indexing, this is per-objects-field compression.
>>> > > Per-fields compression means that metadata (a header) of an object
>>> won't
>>> > > be compressed, only serialized values of an object fields (in bytes
>>> array
>>> > > form) will be compressed.
>>> > >
>>> > > This solution have some contentious issues:
>>> > > - small values, like primitives and short arrays - there isn't sense
>>> to
>>> > > compress them;
>>> > > - there is no possible to use compression with java-predefined types;
>>> > >
>>> > > We can provide an annotation, @IgniteCompression - for example,
>>> which can
>>> > > be used by users for marking fields to compress.
>>> > >
>>> > > Any thoughts?
>>> > >
>>> > > Maybe someone already have ready design?
>>> > >
>>> > > 2017-04-10 11:06 GMT+03:00 Vyacheslav Daradur :
>>> > >
>>> > >> Alexey,
>>> > >>
>>> > >> Yes, I've read it.
>>> > >>
>>> > >> Ok, let's discuss about public API design.
>>> > >>
>>> > >> I think we need to add some a configure entity to
>>> CacheConfiguration,
>>> > >> which will contain the Compressor interface implementation and some
>>> > usefull
>>> > >> parameters.
>>> > >> Or maybe to provide a BinaryMarshaller decorator, which will be
>>> compress
>>> > >> data after marshalling.
>>> > >>
>>> > >>
>>> > >> 2017-04-10 10:40 GMT+03:00 Alexey Kuznetsov >> >:
>>> > >>
>>> > >>> Vyacheslav,
>>> > >>>
>>> > >>> Did you read initial discussion [1] about compression?
>>> > >>> As far as I remember we agreed to add only some "top-level" API in
>>> > order
>>> > >>> to
>>> > >>> provide a way for
>>> > >>> Ignite users to inject some sort of custom compression.
>>> > >>>
>>> > >>>
>>> > >>> [1]
>>> > >>> http://apache-ignite-developers.2346864.n4.nabble.com/Data-c
>>> > >>> ompression-in-Ignite-2-0-td10099.html
>>> > >>>
>>> > >>> On Mon, Apr 10, 2017 at 2:19 PM, daradurvs 
>>> > wrote:
>>> > >>>
>>> > >>> > Hi Igniters!
>>> > >>> >
>>> > >>> > I am interested in this task.
>>> > >>> > Provide some kind of pluggable compression SPI support
>>> > >>> > 
>>> > >>> >
>>> > >>> > I developed a solution on BinaryMarshaller-level, but reviewer
>>> has
>>> > >>> rejected
>>> > >>> > it.
>>> > >>> >
>>> > >>> > Let's continue discussion of task goals and solution design.
>>> > >>> > As I understood that, the main goal of this task is to store
>>> data in
>>> > >>> > compressed form.
>>> > >>> > This is what I need from Ignite as its user. Compression provides
>>> > >>> economy
>>> > >>> > on
>>> > >>> > servers.
>>> > >>> > We can store more data on same servers at the cost of increasing
>>> CPU
>>> > >>> > utilization.
>>> > >>> >
>>> > >>> > I'm researching a possibility of implementation of compression
>>> at the
>>> > >>> > cache-level.
>>> > >>> >
>>> > >>> > Any thoughts?
>>> > >>> >
>>> > >>> > --
>>> > >>> > Best regards,
>>> > >>> > Vyacheslav
>>> > >>> >
>>> > >>> >
>>> > >>> >
>>> > >>> >
>>> > >>> > --
>>> > >>> > View this message in context: http://apache-ignite-
>>> > >>> > developers.2346864.n4.nabble.com/Data-compression-in-
>>> > >>> > Ignite-2-0-tp10099p16317.html
>>> > >>> > Sent from the Apache Ignite Developers mailing list archive at
>>> > >>> Nabble.com.
>>> > >>> >
>>> > >>>
>>> > >>>
>>> > >>>
>>> > >>>