[jira] [Created] (IGNITE-5387) Web console: Agent should report version of internal demo nodes

[Vasiliy Sisko (JIRA)]

Vasiliy Sisko created IGNITE-5387:
-

 Summary: Web console: Agent should report version of internal demo 
nodes
 Key: IGNITE-5387
 URL: https://issues.apache.org/jira/browse/IGNITE-5387
 Project: Ignite
  Issue Type: Bug
  Components: wizards
Affects Versions: 2.0
Reporter: Vasiliy Sisko
Assignee: Andrey Novikov
 Fix For: 2.1






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: nested SQL sub-queries with non-collocated joins

[Denis Magda]

Here is a “known limitations” section on readme.io:
https://apacheignite.readme.io/docs/sql-queries#section-known-limitations

Feel free to update it with the limitation discussed.

—
Denis

> On Jun 1, 2017, at 1:19 PM, Dmitriy Setrakyan  wrote:
> 
> On Thu, Jun 1, 2017 at 1:07 PM, Sergi Vladykin 
> wrote:
> 
>> If you don't see an exception then it must be supported. This is the whole
>> point of this exception, right?
>> 
> 
> Exception is just to enforce the constraint. We still must clearly document
> what is supported.

Re: nested SQL sub-queries with non-collocated joins

[Dmitriy Setrakyan]

On Thu, Jun 1, 2017 at 1:07 PM, Sergi Vladykin 
wrote:

> If you don't see an exception then it must be supported. This is the whole
> point of this exception, right?
>

Exception is just to enforce the constraint. We still must clearly document
what is supported.

Re: nested SQL sub-queries with non-collocated joins

[Sergi Vladykin]

If you don't see an exception then it must be supported. This is the whole
point of this exception, right?

Sergi

2017-06-01 22:50 GMT+03:00 Dmitriy Setrakyan :

> On Thu, Jun 1, 2017 at 12:32 PM, Sergi Vladykin 
> wrote:
>
> > I guess it must work the following way:
> >
> > If distributed joins are enabled we can try to prove that the subquery is
> > collocated, if we can't then try to rewrite it, if we can't, then throw
> an
> > exception.
> >
> > Still this can not be done 100% correct, probably we have to have some
> flag
> > which allows to disable this subquery rewriting.
> >
>
> Sergi, but how do you explain to users what is supported and what is not?
>

Re: nested SQL sub-queries with non-collocated joins

[Dmitriy Setrakyan]

On Thu, Jun 1, 2017 at 12:32 PM, Sergi Vladykin 
wrote:

> I guess it must work the following way:
>
> If distributed joins are enabled we can try to prove that the subquery is
> collocated, if we can't then try to rewrite it, if we can't, then throw an
> exception.
>
> Still this can not be done 100% correct, probably we have to have some flag
> which allows to disable this subquery rewriting.
>

Sergi, but how do you explain to users what is supported and what is not?

Re: nested SQL sub-queries with non-collocated joins

[Sergi Vladykin]

I guess it must work the following way:

If distributed joins are enabled we can try to prove that the subquery is
collocated, if we can't then try to rewrite it, if we can't, then throw an
exception.

Still this can not be done 100% correct, probably we have to have some flag
which allows to disable this subquery rewriting.

Sergi

2017-06-01 21:33 GMT+03:00 Dmitriy Setrakyan :

> Sergi,
>
> I am OK with any improvement here, but we need to be able to clearly state
> to a user what is supported and what is not. If we cannot clearly describe
> it, I would rather not support it at all and throw an exception.
>
> Is this going to be possible with your solution?
>
> D.
>
> On Thu, Jun 1, 2017 at 2:51 AM, Sergi Vladykin 
> wrote:
>
> > The approach you are suggesting will be very complex for current
> > implementation. Also most probably very inefficient.
> >
> > Actually I was thinking about another but similar approach: in many cases
> > we can rewrite a subquery in WHERE clause into JOIN subquery.
> >
> > Like the following:
> >
> > SELECT x.* FROM x WHERE x.a = (SELECT MAX(y.a) FROM y WHERE y.b = x.b)
> >
> >  ===>
> >
> > SELECT x.* FROM x, (SELECT MAX(y.a), y.b FROM y GROUP BY y.b) z WHERE
> x.b =
> > z.b
> >
> > There are still problems here:
> >
> > 1. We will not be able to rewrite all the queries.
> > 2. We should not rewrite queries like this by default because this will
> > have a noticeable performance penalty for correctly collocated
> subqueries.
> > Probably we will need some flag for that.
> >
> > Sergi
> >
> > 2017-05-31 21:26 GMT+03:00 Dmitriy Setrakyan :
> >
> > > Igniters (specifically Sergi),
> > >
> > > It has come to my attention today that nested sub-select statements,
> when
> > > used in combination with non-collocated joins do not work properly in
> > > Ignite.
> > >
> > > So a query like this, where A, B, and C are all stored in Partitioned
> > > caches and are **not** collocated at all, will not work.
> > >
> > >
> > > > *select * from A, B where a.id  = b.a_id and
> b.somefield
> > in
> > > > (select somefield from C where c.zipcode = ?)*
> > >
> > >
> > > The main reason it is not supported right now is because, in the
> absence
> > of
> > > collocation, such query may create N^N complexity and it was decided
> that
> > > it is best not supporting it at all.
> > >
> > > However, I am not sure why N^N complexity is required. Why not support
> it
> > > as follows?
> > >
> > >1. execute the nested subquery and store the result in a temporary
> > >Replicated table.
> > >2. execute the original query and use the temporary Replicated table
> > >instead of the sub-query.
> > >
> > > Sergi, given that you are the author of the code, can you provide some
> > > insight here?
> > >
> > > Thanks,
> > > D.
> > >
> >
>

Re: Geo spatial index

[Sergi Vladykin]

No, feel free to create one.

Sergi

2017-06-01 21:00 GMT+03:00 Denis Magda :

> Good catch. I found the ticket that’s aim is to integrate the full-text
> search indexes with the virtual page memory architecture.
> https://issues.apache.org/jira/browse/IGNITE-5371 <
> https://issues.apache.org/jira/browse/IGNITE-5371>
>
> Sergi, do we have a similar one for the geo-spatial?
>
> —
> Denis
>
> > On Jun 1, 2017, at 2:12 AM, Alexey Goncharuk 
> wrote:
> >
> > Same think stays for the full-text indexes which are currently stored in
> > Lucene.
> >
> > 2017-05-24 21:56 GMT+03:00 Dmitriy Setrakyan :
> >
> >> Sergi,
> >>
> >> While we are figuring this out, what happens to the GeoSpatial
> >> functionality in the mean time? Is it going to work at all? If not,
> should
> >> we throw some sort of exception?
> >>
> >> D.
> >>
> >> On Wed, May 24, 2017 at 1:44 AM, Sergi Vladykin <
> sergi.vlady...@gmail.com>
> >> wrote:
> >>
> >>> Though this may require some changes in BPlusTree. Let me think.
> >>>
> >>> Sergi
> >>>
> >>> 2017-05-24 8:58 GMT+03:00 Sergi Vladykin :
> >>>
>  It must not be too hard to implement kd-tree over b+tree [1].
> Depending
> >>> on
>  level we have to compare either X or Y coordinate.
> 
>  I think we will even have a performance boost for spatial indexes
> after
>  this change.
> 
>  [1] https://en.wikipedia.org/wiki/K-d_tree
> 
>  Sergi
> 
>  2017-05-23 18:59 GMT+03:00 Denis Magda :
> 
> > +1
> >
> > This looks natural considering that we switched to the new memory
> > architecture. Sergi, how difficult is to support this?
> >
> > —
> > Denis
> >
> >> On May 23, 2017, at 4:25 AM, Sergi Vladykin <
> >> sergi.vlady...@gmail.com
> 
> > wrote:
> >>
> >> Guys,
> >>
> >> Looks like we have to move our geospatial indexes to the new
> >> approach
> > with
> >> BPlusTree. Right now it stores data in Java heap. This is especially
> >> important because we are going to have a persistence layer donated
> >> by
> >> GridGain and obviously geo spatial indexes will not work with it at
> >>> all.
> >>
> >> Sergi
> >
> >
> 
> >>>
> >>
>
>

Re: nested SQL sub-queries with non-collocated joins

[Dmitriy Setrakyan]

Sergi,

I am OK with any improvement here, but we need to be able to clearly state
to a user what is supported and what is not. If we cannot clearly describe
it, I would rather not support it at all and throw an exception.

Is this going to be possible with your solution?

D.

On Thu, Jun 1, 2017 at 2:51 AM, Sergi Vladykin 
wrote:

> The approach you are suggesting will be very complex for current
> implementation. Also most probably very inefficient.
>
> Actually I was thinking about another but similar approach: in many cases
> we can rewrite a subquery in WHERE clause into JOIN subquery.
>
> Like the following:
>
> SELECT x.* FROM x WHERE x.a = (SELECT MAX(y.a) FROM y WHERE y.b = x.b)
>
>  ===>
>
> SELECT x.* FROM x, (SELECT MAX(y.a), y.b FROM y GROUP BY y.b) z WHERE x.b =
> z.b
>
> There are still problems here:
>
> 1. We will not be able to rewrite all the queries.
> 2. We should not rewrite queries like this by default because this will
> have a noticeable performance penalty for correctly collocated subqueries.
> Probably we will need some flag for that.
>
> Sergi
>
> 2017-05-31 21:26 GMT+03:00 Dmitriy Setrakyan :
>
> > Igniters (specifically Sergi),
> >
> > It has come to my attention today that nested sub-select statements, when
> > used in combination with non-collocated joins do not work properly in
> > Ignite.
> >
> > So a query like this, where A, B, and C are all stored in Partitioned
> > caches and are **not** collocated at all, will not work.
> >
> >
> > > *select * from A, B where a.id  = b.a_id and b.somefield
> in
> > > (select somefield from C where c.zipcode = ?)*
> >
> >
> > The main reason it is not supported right now is because, in the absence
> of
> > collocation, such query may create N^N complexity and it was decided that
> > it is best not supporting it at all.
> >
> > However, I am not sure why N^N complexity is required. Why not support it
> > as follows?
> >
> >1. execute the nested subquery and store the result in a temporary
> >Replicated table.
> >2. execute the original query and use the temporary Replicated table
> >instead of the sub-query.
> >
> > Sergi, given that you are the author of the code, can you provide some
> > insight here?
> >
> > Thanks,
> > D.
> >
>

Re: Persistent Distributed Store Metrics

[Denis Magda]

Sergey, thanks,

You get a green light from my side. Please go ahead and bring this to life 
unless anyone else has any comments.

—
Denis

> On Jun 1, 2017, at 4:18 AM, Sergey Chugunov  wrote:
> 
> Guys,
> 
> I created a ticket [1] summing up results of our discussion. Please review
> and leave comments if something is still unclear there.
> 
> Thanks,
> Sergey
> 
> [1] https://issues.apache.org/jira/browse/IGNITE-5375
> 
> On Wed, May 31, 2017 at 11:56 PM, Dmitriy Setrakyan 
> wrote:
> 
>> Denis, I do agree, the concept of virtual memory may work. I will start
>> another discussion thread on that.
>> 
>> We should extensively javadoc all these interfaces. The current one-liner
>> javadoc documentation is not enough.
>> 
>> Also, I would like to make a small correction to the MemoryMetrics
>> interface, see below...
>> 
>> 
>> On Wed, May 31, 2017 at 1:35 PM, Denis Magda  wrote:
>> 
>>> Guys,
>>> If to assume that the *page memory* is a sort of *virtual* memory that
>>> maps a page to RAM or disk then MemoryMetrics interface makes sense if we
>>> think of it as {Virtual}MemoryMetrics. It’s late to rename the interface
>>> but this design flaw can be handled with a proper documentation.
>>> At all, let’s gather all the page memory related metrics (both RAM and
>>> disk) in MemoryMetrics interface. Here is it’s final version:
>>> public interface MemoryMetrics {
>>>public String getName();
>>> 
>>>public long getTotalAllocatedPages();
>> 
>> 
>> I think this method currently returns only the pages allocated in memory.
>> To behave correctly, it now should return the total pages allocated within
>> the virtual memory, which should be equal to the total pages on disk.
>> 
>> 
>>>public long getPagesOnDisk();
>>> 
>> 
>> This method should be removed as it represents the same value as
>> "getTotalAllocatedPages()" method. Instead, we should add the following
>> method:
>> 
>> // Returns the number of pages allocated in physical off-heap memory.
>> getPhysicalMemoryPages();
>> 
>> 
>>> 
>>>public long getDirtyPages();
>>> 
>>>public float getAllocationRate();
>>> 
>>>public float getEvictionRate();
>>> 
>>>public float getPagesFillFactor();
>>> 
>>>public double getPagesMissRate();
>>> 
>>>public float getLargeEntriesPagesPercentage();
>>> }
>>> I would rename getAllocationRate and getEvitionRate to
>>> getPagesAllocationRate and getEvictionRates respectively but then we need
>>> to deprecate the existing methods which is not good.
>>> 
>>> As for the persistence metrics related to WAL and checkpointing let’s
>>> store them here:
>>> 
>>> public interface PersistentStoreMetrics {
>>>public double getWalLoggingRate();
>>> 
>>>public int getWalArchiveSegments();
>>> 
>>>public double getWalFsyncTime();
>>> 
>>>public double getCheckpointingTime();
>>> 
>>>public double getCheckpointingFsyncTime();
>>> 
>>>public long getCheckpointingTotalPagesNumber();
>>> 
>>>public long[] getCheckpointingPagesByTypeNumber();
>>> 
>>>public long getCheckpointingCopiedOnWritePagesNumber();
>>> }
>>> 
>>> Comments on the name of non released methods are welcomed.
>>> 
>>> —
>>> Denis
>>> 
 On May 30, 2017, at 2:59 PM, Dmitriy Setrakyan 
>>> wrote:
 
 On Tue, May 30, 2017 at 2:46 PM, Denis Magda 
>> wrote:
 
> I don’t have any extra metrics in mind for now. All I want to say that
> presently the Store metrics interface aggregates WAL and checkpointing
> metrics and in the future we might add additional Store related
>> metrics
> there (that has nothing to do with WAL). So, this is why
>>> IgniteWalMetrics
> doesn’t sound like a generic interface.
> 
 
 I think it is getting more and more confusing. I, for instance, do not
 understand where memory metrics end and storage begins, neither I think
>>> it
 is important to separate memory policy related metrics from the WAL or
 Checkpoint related metrics.
 
 How about we put them all together and have clear method names?
 
 
> 
> —
> Denis
> 
>> On May 30, 2017, at 2:39 PM, Dmitriy Setrakyan <
>> dsetrak...@apache.org>
> wrote:
>> 
>> On Tue, May 30, 2017 at 1:22 PM, Denis Magda 
>>> wrote:
>> 
>>> I’m afraid that in the future we might need to add non WAL related
> metrics
>>> under this interface. This is why it might make sense to keep the
>> name
>>> generic.
>>> 
>> 
>> Hm... I am not sure we are going to have any other components in
>>> addition
>> to WAL and Store here. What do you have in mind?
> 
> 
>>> 
>>> 
>> 

Re: Geo spatial index

[Denis Magda]

Good catch. I found the ticket that’s aim is to integrate the full-text search 
indexes with the virtual page memory architecture.
https://issues.apache.org/jira/browse/IGNITE-5371 


Sergi, do we have a similar one for the geo-spatial?

—
Denis
 
> On Jun 1, 2017, at 2:12 AM, Alexey Goncharuk  
> wrote:
> 
> Same think stays for the full-text indexes which are currently stored in
> Lucene.
> 
> 2017-05-24 21:56 GMT+03:00 Dmitriy Setrakyan :
> 
>> Sergi,
>> 
>> While we are figuring this out, what happens to the GeoSpatial
>> functionality in the mean time? Is it going to work at all? If not, should
>> we throw some sort of exception?
>> 
>> D.
>> 
>> On Wed, May 24, 2017 at 1:44 AM, Sergi Vladykin 
>> wrote:
>> 
>>> Though this may require some changes in BPlusTree. Let me think.
>>> 
>>> Sergi
>>> 
>>> 2017-05-24 8:58 GMT+03:00 Sergi Vladykin :
>>> 
 It must not be too hard to implement kd-tree over b+tree [1]. Depending
>>> on
 level we have to compare either X or Y coordinate.
 
 I think we will even have a performance boost for spatial indexes after
 this change.
 
 [1] https://en.wikipedia.org/wiki/K-d_tree
 
 Sergi
 
 2017-05-23 18:59 GMT+03:00 Denis Magda :
 
> +1
> 
> This looks natural considering that we switched to the new memory
> architecture. Sergi, how difficult is to support this?
> 
> —
> Denis
> 
>> On May 23, 2017, at 4:25 AM, Sergi Vladykin <
>> sergi.vlady...@gmail.com
 
> wrote:
>> 
>> Guys,
>> 
>> Looks like we have to move our geospatial indexes to the new
>> approach
> with
>> BPlusTree. Right now it stores data in Java heap. This is especially
>> important because we are going to have a persistence layer donated
>> by
>> GridGain and obviously geo spatial indexes will not work with it at
>>> all.
>> 
>> Sergi
> 
> 
 
>>> 
>> 

[GitHub] ignite pull request #2064: IGNITE-5355 Create task with release tools

[achetaev]

GitHub user achetaev opened a pull request:

https://github.com/apache/ignite/pull/2064

IGNITE-5355 Create task with release tools



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5355

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2064.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2064


commit a6ad620cece5be02da4f8394a501f7c7fd76f165
Author: Aleksey Chetaev 
Date:   2017-06-01T15:00:21Z

IGNITE-5355: Added new class to ignite-tools for generate release report

commit 1aeee2062d48324712bc51c1b6cde9969d58cacc
Author: Aleksey Chetaev 
Date:   2017-06-01T15:17:10Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-5355

commit 42876c924e7ccbd84b3883acae21d069aa971b34
Author: Aleksey Chetaev 
Date:   2017-06-01T17:23:43Z

IGNITE-5355 Add needed dependencies to tools/pom.xml




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #2062: IGNITE-5355 Create task with release tools

[achetaev]

Github user achetaev closed the pull request at:

https://github.com/apache/ignite/pull/2062


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

[npordash]

I wanted to provide an update on where I am at with this as I'm still
exploring different options.

For the time being I've taken the path of using a delegating ClassLoader in
IgniteConfiguration as was previously suggested; however, with the
difference being that whenever a service is deployed/undeployed I end up
replacing the ClassLoader with a new instance that has the service's
ClassLoader added/removed. The replacement is necessary so that unused/old
classes can be reclaimed by the garbage collector and that new versions can
be deployed in the future.

Overall I think this is a more comprehensive approach since it also allows
execution constructs like CacheEntryProcessor implementations provided by
the service to be used across the grid without enabling p2p classloading
(assuming the service is deployed to every node).

There are a few concerns/issues with this approach:

1) There are still isolation concerns with class versions across different
services, but as long as the services don't have dependencies between each
other or have overlapping class usage in a cache or execution context then
it's a non-issue.
2) Using OnheapCacheEnabled in conjunction with service provided domain
classes is impossible since once the service is reloaded and we get a new
classloader all calls to deserialize will fail.
3) This approach only works if nothing caches the result of
IgniteConfiguration.getClassLoader, which isn't obvious since the docs
related to classloader behavior in Ignite is pretty sparse (or at least I
could not find it). I don't think anything does when I check that method
usage, but I'm not 100% sure about that.



--
View this message in context: 
http://apache-ignite-developers.2346864.n4.nabble.com/Re-BinaryObjectImpl-deserializeValue-with-specific-ClassLoader-tp17126p18358.html
Sent from the Apache Ignite Developers mailing list archive at Nabble.com.

Re: Apache Ignite 2.1 scope

[Alexander Paschenko]

IGNITE-5327 Create predefined cache templates for CREATE TABLE command
- minor comments left, ETA is Friday.

IGNITE-5380 Validate cache QueryEntities in discovery thread - in
progress, the meat of code is written, but need to add lots of tests.
ETA is Friday.

IGNITE-5188 Support AFFINITY KEY keyword for CREATE TABLE command - in
progress, made few first small steps, ETA is Friday.

Rest is closed/patch available, ignite-4994 has been moved to 2.2.

- Alex

2017-06-01 19:03 GMT+03:00 Sergey Chugunov :
>1. IGNITE-5386 Inactive mode must be forced on starting up grid with
>persistence is enabled
>It is important improvement to fix critical bug IGNITE-5363.
>Working on it, ETA - tomorrow.
>2. IGNITE-5375 New PersistentStoreMetrics, MemoryMetrics interface
>improvements
>A lot of discussions were on this topic, ticket created only today and
>requires several days to implement.
>
>
>
> On Thu, Jun 1, 2017 at 6:56 PM, Taras Ledkov  wrote:
>
>> Folks,
>>
>> IGNITE-4922 JDBC Driver: renew thin client based solution:
>>
>> On 2.1 the functionality of the new thin client JDBC driver will be
>> between deprecated Ignite thin JDBC and Ignite JDBCv2.
>> 1. The most functions of SQL query (include DML) are implemented and ready
>> for review;
>> 2. The most functions of JDBC metadata are implemented and ready for
>> review;
>> 3. Transactions, batching, streaming, blobs, scrollable / writable cursors
>> will not be supported in 2.1.
>>
>>
>>
>> On 01.06.2017 18:43, Vladimir Ozerov wrote:
>>
>>> Folks,
>>>
>>> We are almost reached proposed feature-complete date (June 2), Could you
>>> please share current status of your major features?
>>>
>>> On Tue, May 16, 2017 at 3:51 AM, Dmitriy Setrakyan >> >
>>> wrote:
>>>
>>> Looks a little tight. Let's hope we can make it.

 On Mon, May 15, 2017 at 1:29 PM, Denis Magda  wrote:

 Well, let me propose the following milestones for 2.1 release then.
>
> Code freeze: June 2nd.
> Final QA and benchmarking: June 5 - June 8
> Voting: ~ June 9
> Release: ~ June 13
>
> Also I heard H2 has to be released once again to support Ignite’s CREATE
> table command. Think that we should talk to H2 folks to make it happen
> in
> June 22nd - June 2nd time frame.
>
> —
> Denis
>
> On May 11, 2017, at 2:26 AM, Pavel Tupitsyn 
>>
> wrote:
>
>> As for .NET, I would propose to concentrate on peer deployment
>>
> (IGNITE-2492)
>
>> and related stuff, like IGNITE-1894 .NET: Delegate support in the API
>>
> via

> extension methods.
>>
>> SQL Dependency does not look important to me, we can reschedule it for
>> later versions.
>>
>> On Thu, May 11, 2017 at 12:01 PM, Dmitriy Setrakyan <
>>
> dsetrak...@apache.org>
>
>> wrote:
>>
>> Vyacheslav, I think it is worth the research, but you should always
>>>
>> keep

> data querying and indexing in mind. For example, I don't see how
>>>
>> by-page

> compression will solve it.
>>>
>>> On Thu, May 11, 2017 at 1:52 AM, Vyacheslav Daradur <
>>>
>> daradu...@gmail.com>
>
>> wrote:
>>>
>>> Dmitriy,

 I'm researching a best way for this future.

 At the moment I found only one way (querying and indexing

>>> compatible),

> this
>>>
 is per-objects-field compression.

 But there is a good proffit only for long strings or fields with

>>> large

> objects.

 Maybe it makes sense just to introduce compression for string fileds.

 I'm researching the new page-memory architecture as applied to

>>> by-page

> compression.

 2017-05-11 11:30 GMT+03:00 Dmitriy Setrakyan >> :
>
>> On Thu, May 11, 2017 at 12:44 AM, Vyacheslav Daradur <
>
 daradu...@gmail.com>

> wrote:
>
> Denis,
>>
>> The described roadmap looks great!
>>
>> Additional, I vote for introducing an ability (OOTB) to store
>>
> objects

> in

> a
>
>> cache in a compressed form.
>> This will allow to store more data at the cost of incriasing of CPU
>> utilization.
>>
>> One of the problems with compression is indexing and querying. How
>
 do

> we
>>>
 index the data if it is compressed?
>
>
>
>> 2017-05-11 4:23 GMT+03:00 Denis Magda :
>>
>> Igniters,
>>>
>>> Let me start a discussion around the scope for 2.1 release.
>>>
>>> In my 

Re: Persistent Store Stabilization for release

[Alexey Goncharuk]

Guys,

I merged recent changes from the master branch (SQL-related changes) to the
integration branch, we need another TC run to see if the merge broke
something.

2017-06-01 19:14 GMT+03:00 Dmitriy Setrakyan :

> BTW, can anyone explain to me why do we keep the new persistence code in a
> separate module instead of merging it to core?
>
> On Thu, Jun 1, 2017 at 4:32 AM, Sergey Chugunov  >
> wrote:
>
> > Hello Denis,
> >
> > There are three suites [1] [2] [3] configured on TeamCity with bunch of
> > tests for PDS functionality.
> >
> > As you can see most of tests are passing now (only 6 tests are
> > failing, for *partitionLoss
> > *tests problem was almost sorted out), it looks like that this
> > functionality is pretty stable.
> >
> >
> > [1] Ignite PDS 1
> >  > Ignite20Tests_IgnitePds1_Ignite20Tests=ignite-
> > 5267=buildTypeStatusDiv>
> > [2] Ignite PDS 2
> >  > Ignite20Tests_IgnitePds2_Ignite20Tests=ignite-
> > 5267=buildTypeStatusDiv>
> > [3] Ignite OutOfMemory
> >  > IgnitePdsOutOfMemory_Ignite20Tests=ignite-5267=
> > buildTypeStatusDiv>
> >
> > On Thu, Jun 1, 2017 at 1:24 AM, Dmitriy Setrakyan  >
> > wrote:
> >
> > > Thanks Denis. I also would like to encourage to post all the
> > stabilization
> > > updates for the donated branch here. This will help the community to
> get
> > > familiar with the new code and functionality.
> > >
> > > In addition, everyone in the community should feel free to peruse
> through
> > > the code in this branch and ask any questions that may come up:
> > >
> > > https://github.com/apache/ignite/tree/ignite-5267/modules/pds
> > >
> > > D.
> > >
> > >
> > > On Wed, May 31, 2017 at 2:37 PM, Denis Magda 
> wrote:
> > >
> > > > Igniters,
> > > >
> > > > The Persistent Store donation was accepted by the community and now
> we
> > > > should think of its readiness for a public release.
> > > >
> > > > I know that many of you are involved it the donation stabilization.
> > Could
> > > > you share the current status of the feature? How far is it from the
> > > > releasable state?
> > > >
> > > > —
> > > > Denis
> > >
> >
>

Re: Persistent Store Stabilization for release

[Dmitriy Setrakyan]

BTW, can anyone explain to me why do we keep the new persistence code in a
separate module instead of merging it to core?

On Thu, Jun 1, 2017 at 4:32 AM, Sergey Chugunov 
wrote:

> Hello Denis,
>
> There are three suites [1] [2] [3] configured on TeamCity with bunch of
> tests for PDS functionality.
>
> As you can see most of tests are passing now (only 6 tests are
> failing, for *partitionLoss
> *tests problem was almost sorted out), it looks like that this
> functionality is pretty stable.
>
>
> [1] Ignite PDS 1
>  Ignite20Tests_IgnitePds1_Ignite20Tests=ignite-
> 5267=buildTypeStatusDiv>
> [2] Ignite PDS 2
>  Ignite20Tests_IgnitePds2_Ignite20Tests=ignite-
> 5267=buildTypeStatusDiv>
> [3] Ignite OutOfMemory
>  IgnitePdsOutOfMemory_Ignite20Tests=ignite-5267=
> buildTypeStatusDiv>
>
> On Thu, Jun 1, 2017 at 1:24 AM, Dmitriy Setrakyan 
> wrote:
>
> > Thanks Denis. I also would like to encourage to post all the
> stabilization
> > updates for the donated branch here. This will help the community to get
> > familiar with the new code and functionality.
> >
> > In addition, everyone in the community should feel free to peruse through
> > the code in this branch and ask any questions that may come up:
> >
> > https://github.com/apache/ignite/tree/ignite-5267/modules/pds
> >
> > D.
> >
> >
> > On Wed, May 31, 2017 at 2:37 PM, Denis Magda  wrote:
> >
> > > Igniters,
> > >
> > > The Persistent Store donation was accepted by the community and now we
> > > should think of its readiness for a public release.
> > >
> > > I know that many of you are involved it the donation stabilization.
> Could
> > > you share the current status of the feature? How far is it from the
> > > releasable state?
> > >
> > > —
> > > Denis
> >
>

Re: Apache Ignite 2.1 scope

[Sergey Chugunov]

   1. IGNITE-5386 Inactive mode must be forced on starting up grid with
   persistence is enabled
   It is important improvement to fix critical bug IGNITE-5363.
   Working on it, ETA - tomorrow.
   2. IGNITE-5375 New PersistentStoreMetrics, MemoryMetrics interface
   improvements
   A lot of discussions were on this topic, ticket created only today and
   requires several days to implement.



On Thu, Jun 1, 2017 at 6:56 PM, Taras Ledkov  wrote:

> Folks,
>
> IGNITE-4922 JDBC Driver: renew thin client based solution:
>
> On 2.1 the functionality of the new thin client JDBC driver will be
> between deprecated Ignite thin JDBC and Ignite JDBCv2.
> 1. The most functions of SQL query (include DML) are implemented and ready
> for review;
> 2. The most functions of JDBC metadata are implemented and ready for
> review;
> 3. Transactions, batching, streaming, blobs, scrollable / writable cursors
> will not be supported in 2.1.
>
>
>
> On 01.06.2017 18:43, Vladimir Ozerov wrote:
>
>> Folks,
>>
>> We are almost reached proposed feature-complete date (June 2), Could you
>> please share current status of your major features?
>>
>> On Tue, May 16, 2017 at 3:51 AM, Dmitriy Setrakyan > >
>> wrote:
>>
>> Looks a little tight. Let's hope we can make it.
>>>
>>> On Mon, May 15, 2017 at 1:29 PM, Denis Magda  wrote:
>>>
>>> Well, let me propose the following milestones for 2.1 release then.

 Code freeze: June 2nd.
 Final QA and benchmarking: June 5 - June 8
 Voting: ~ June 9
 Release: ~ June 13

 Also I heard H2 has to be released once again to support Ignite’s CREATE
 table command. Think that we should talk to H2 folks to make it happen
 in
 June 22nd - June 2nd time frame.

 —
 Denis

 On May 11, 2017, at 2:26 AM, Pavel Tupitsyn 
>
 wrote:

> As for .NET, I would propose to concentrate on peer deployment
>
 (IGNITE-2492)

> and related stuff, like IGNITE-1894 .NET: Delegate support in the API
>
 via
>>>
 extension methods.
>
> SQL Dependency does not look important to me, we can reschedule it for
> later versions.
>
> On Thu, May 11, 2017 at 12:01 PM, Dmitriy Setrakyan <
>
 dsetrak...@apache.org>

> wrote:
>
> Vyacheslav, I think it is worth the research, but you should always
>>
> keep
>>>
 data querying and indexing in mind. For example, I don't see how
>>
> by-page
>>>
 compression will solve it.
>>
>> On Thu, May 11, 2017 at 1:52 AM, Vyacheslav Daradur <
>>
> daradu...@gmail.com>

> wrote:
>>
>> Dmitriy,
>>>
>>> I'm researching a best way for this future.
>>>
>>> At the moment I found only one way (querying and indexing
>>>
>> compatible),
>>>
 this
>>
>>> is per-objects-field compression.
>>>
>>> But there is a good proffit only for long strings or fields with
>>>
>> large
>>>
 objects.
>>>
>>> Maybe it makes sense just to introduce compression for string fileds.
>>>
>>> I'm researching the new page-memory architecture as applied to
>>>
>> by-page
>>>
 compression.
>>>
>>> 2017-05-11 11:30 GMT+03:00 Dmitriy Setrakyan >>
>> :

> On Thu, May 11, 2017 at 12:44 AM, Vyacheslav Daradur <

>>> daradu...@gmail.com>
>>>
 wrote:

 Denis,
>
> The described roadmap looks great!
>
> Additional, I vote for introducing an ability (OOTB) to store
>
 objects
>>>
 in
>>>
 a

> cache in a compressed form.
> This will allow to store more data at the cost of incriasing of CPU
> utilization.
>
> One of the problems with compression is indexing and querying. How

>>> do
>>>
 we
>>
>>> index the data if it is compressed?



> 2017-05-11 4:23 GMT+03:00 Denis Magda :
>
> Igniters,
>>
>> Let me start a discussion around the scope for 2.1 release.
>>
>> In my vision the main direction of our ongoing efforts should be
>> implementing in life a use case of Ignite as a transactional
>>
> distributed

> SQL database and HTAP platform. The current use cases (database
>>
> cache,
>>>
 data
>
>> grid, micro services platform, etc.) will be supported as usual,
>>
> no
>>>
 changes
>
>> on that frontier.
>>
>> Keeping this in mind, the roadmap needs to include essential SQL
>>
> related

> features as well as disk based capabilities, MVCC support,
>>
> advanced
>>>
 DDL
>>>

Re: Apache Ignite 2.1 scope

[Igor Sapego]

Compute for C++ - https://issues.apache.org/jira/browse/IGNITE-3355 -
merged to master.

Best Regards,
Igor

On Thu, Jun 1, 2017 at 6:56 PM, Taras Ledkov  wrote:

> Folks,
>
> IGNITE-4922 JDBC Driver: renew thin client based solution:
>
> On 2.1 the functionality of the new thin client JDBC driver will be
> between deprecated Ignite thin JDBC and Ignite JDBCv2.
> 1. The most functions of SQL query (include DML) are implemented and ready
> for review;
> 2. The most functions of JDBC metadata are implemented and ready for
> review;
> 3. Transactions, batching, streaming, blobs, scrollable / writable cursors
> will not be supported in 2.1.
>
>
>
> On 01.06.2017 18:43, Vladimir Ozerov wrote:
>
>> Folks,
>>
>> We are almost reached proposed feature-complete date (June 2), Could you
>> please share current status of your major features?
>>
>> On Tue, May 16, 2017 at 3:51 AM, Dmitriy Setrakyan > >
>> wrote:
>>
>> Looks a little tight. Let's hope we can make it.
>>>
>>> On Mon, May 15, 2017 at 1:29 PM, Denis Magda  wrote:
>>>
>>> Well, let me propose the following milestones for 2.1 release then.

 Code freeze: June 2nd.
 Final QA and benchmarking: June 5 - June 8
 Voting: ~ June 9
 Release: ~ June 13

 Also I heard H2 has to be released once again to support Ignite’s CREATE
 table command. Think that we should talk to H2 folks to make it happen
 in
 June 22nd - June 2nd time frame.

 —
 Denis

 On May 11, 2017, at 2:26 AM, Pavel Tupitsyn 
>
 wrote:

> As for .NET, I would propose to concentrate on peer deployment
>
 (IGNITE-2492)

> and related stuff, like IGNITE-1894 .NET: Delegate support in the API
>
 via
>>>
 extension methods.
>
> SQL Dependency does not look important to me, we can reschedule it for
> later versions.
>
> On Thu, May 11, 2017 at 12:01 PM, Dmitriy Setrakyan <
>
 dsetrak...@apache.org>

> wrote:
>
> Vyacheslav, I think it is worth the research, but you should always
>>
> keep
>>>
 data querying and indexing in mind. For example, I don't see how
>>
> by-page
>>>
 compression will solve it.
>>
>> On Thu, May 11, 2017 at 1:52 AM, Vyacheslav Daradur <
>>
> daradu...@gmail.com>

> wrote:
>>
>> Dmitriy,
>>>
>>> I'm researching a best way for this future.
>>>
>>> At the moment I found only one way (querying and indexing
>>>
>> compatible),
>>>
 this
>>
>>> is per-objects-field compression.
>>>
>>> But there is a good proffit only for long strings or fields with
>>>
>> large
>>>
 objects.
>>>
>>> Maybe it makes sense just to introduce compression for string fileds.
>>>
>>> I'm researching the new page-memory architecture as applied to
>>>
>> by-page
>>>
 compression.
>>>
>>> 2017-05-11 11:30 GMT+03:00 Dmitriy Setrakyan >>
>> :

> On Thu, May 11, 2017 at 12:44 AM, Vyacheslav Daradur <

>>> daradu...@gmail.com>
>>>
 wrote:

 Denis,
>
> The described roadmap looks great!
>
> Additional, I vote for introducing an ability (OOTB) to store
>
 objects
>>>
 in
>>>
 a

> cache in a compressed form.
> This will allow to store more data at the cost of incriasing of CPU
> utilization.
>
> One of the problems with compression is indexing and querying. How

>>> do
>>>
 we
>>
>>> index the data if it is compressed?



> 2017-05-11 4:23 GMT+03:00 Denis Magda :
>
> Igniters,
>>
>> Let me start a discussion around the scope for 2.1 release.
>>
>> In my vision the main direction of our ongoing efforts should be
>> implementing in life a use case of Ignite as a transactional
>>
> distributed

> SQL database and HTAP platform. The current use cases (database
>>
> cache,
>>>
 data
>
>> grid, micro services platform, etc.) will be supported as usual,
>>
> no
>>>
 changes
>
>> on that frontier.
>>
>> Keeping this in mind, the roadmap needs to include essential SQL
>>
> related

> features as well as disk based capabilities, MVCC support,
>>
> advanced
>>>
 DDL
>>>
 implementation and so on so forth. This is for Ignite as a SQL
>>
> database.

> Next, Machine Learning will be a great addition to Ignite as an
>>
> HTAP
>>
>>> platform offering. This is why we should keep investing our time

[GitHub] ignite pull request #2063: IGNITE-5356 .NET: Fix runtime dependency loading ...

[ptupitsyn]

GitHub user ptupitsyn opened a pull request:

https://github.com/apache/ignite/pull/2063

IGNITE-5356 .NET: Fix runtime dependency loading with peer deployment



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ptupitsyn/ignite ignite-5356

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2063.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2063


commit 2af4ded9a1bc911c525c64a7370c1df410c87024
Author: Pavel Tupitsyn 
Date:   2017-05-31T10:26:17Z

IGNITE-5356 .NET: Peer deployment does not load runtime dependencies

commit aaac50cc8311531d9a5658423be8474e04c0dd1a
Author: Pavel Tupitsyn 
Date:   2017-05-31T12:41:40Z

wip

commit 2d6c79bbee15b7fb572f160ef7ebfd39b45bcb97
Author: Pavel Tupitsyn 
Date:   2017-06-01T13:52:33Z

Merge branch 'master' into ignite-5356

commit 27de98ae040444c388e8ac2e6c1c5d11b13eb70f
Author: Pavel Tupitsyn 
Date:   2017-06-01T14:34:28Z

added naive fix

commit 2a84f17b5542f9dc2da405fc4f5257e0264aafe9
Author: Pavel Tupitsyn 
Date:   2017-06-01T14:37:09Z

Fix exception mapping

commit f4aeb412c1807075b41c49d8fb56e9c2e03b1530
Author: Pavel Tupitsyn 
Date:   2017-06-01T14:44:29Z

wip

commit c30641ebd1d47a559fd44f735e705b8ba5f5f69c
Author: Pavel Tupitsyn 
Date:   2017-06-01T14:52:38Z

wip

commit b69ebcafe9cf5f4f597c7268edde82ba46b783ff
Author: Pavel Tupitsyn 
Date:   2017-06-01T15:38:58Z

Load referenced assemblies on demand

commit 4f5d8b0a1084f9683187bd3022c23d33a97530bd
Author: Pavel Tupitsyn 
Date:   2017-06-01T15:40:53Z

cleanup

commit 11f30e3d145a13801de24987b17934189398c384
Author: Pavel Tupitsyn 
Date:   2017-06-01T15:43:48Z

wip

commit 6f0a114b7868fbc83c6a6c7901f401e936135b57
Author: Pavel Tupitsyn 
Date:   2017-06-01T15:57:26Z

cleanup




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Cassandra and Ignite C++ Issue

[Igor Sapego]

Take a look at initial message.

Best Regards,
Igor

On Thu, Jun 1, 2017 at 6:45 PM, Igor Rudyak  wrote:

> Could you share your POJO key/value classes and persistence descriptor?
>
> Igor
>
> On Jun 1, 2017 8:30 AM, "Igor Sapego"  wrote:
>
> > User's comment:
> >
> > I have my own cassandra table with 1 partiton key, 3 clustering keys,
> > and I expect the value not to be stored in one column as a blob.
> >
> > Is there any way I can achieve this?
> > I have used POJO and there is an issue like the key is only getting
> > inserted into the cassandra table
> >
> > Best Regards,
> > Igor
> >
> > On Tue, May 30, 2017 at 12:33 PM, Igor Sapego 
> > wrote:
> >
> > > Thanks.
> > >
> > > Best Regards,
> > > Igor
> > >
> > > On Tue, May 30, 2017 at 11:35 AM, Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com> wrote:
> > >
> > >> Igor,
> > >>
> > >> There is another SO question about this, I already responded:
> > >> https://stackoverflow.com/questions/44254079/ignite-with-
> > >> cassandra-integration
> > >>
> > >> -Val
> > >>
> > >> On Tue, May 30, 2017 at 10:15 AM, Igor Sapego 
> > >> wrote:
> > >>
> > >> > User's answer:
> > >> > I have changed my POJO to BLOB.
> > >> >
> > >> > Current persistence.xml file look like
> > >> >
> > >> > 
> > >> >> >> > column="assetid"/>
> > >> >> >> > column="asset_desc"/>
> > >> > 
> > >> >
> > >> > now it's produce an error like
> > >> >
> > >> > "Caused by: com.datastax.driver.core.exceptions.
> > CodecNotFoundException:
> > >> > Codec not found for requested operation: [varchar <->
> > >> > java.nio.HeapByteBuffer] "
> > >> >
> > >> > Best Regards,
> > >> > Igor
> > >> >
> > >> > On Fri, May 26, 2017 at 2:37 PM, Igor Sapego 
> > >> wrote:
> > >> >
> > >> > > Cross-posted Igor's answer to SO
> > >> > >
> > >> > > Best Regards,
> > >> > > Igor
> > >> > >
> > >> > > On Fri, May 26, 2017 at 8:58 AM, Igor Rudyak 
> > >> wrote:
> > >> > >
> > >> > >> I assume that's because you are using this:
> > >> > >>
> > >> > >> 
> > >> > >>
> > >> > >>
> > >> > >> Current implementation supports only BLOB serialization for
> binary
> > >> > >> objects. We already have a ticket for this:
> > https://issues.apache.or
> > >> > >> g/jira/browse/IGNITE-5270
> > >> > >>
> > >> > >> Igor
> > >> > >>
> > >> > >>
> > >> > >>
> > >> > >> On Thu, May 25, 2017 at 8:19 PM, Denis Magda <
> dma...@gridgain.com>
> > >> > wrote:
> > >> > >>
> > >> > >>> Igor R., Igor S.,
> > >> > >>>
> > >> > >>> Please take a look at this issue reported on SO:
> > >> > >>> https://stackoverflow.com/questions/44178514/ignite-c-client
> > >> > >>> -for-cassandra-integration
> > >> > >>>
> > >> > >>> --
> > >> > >>> Denis
> > >> > >>>
> > >> > >>
> > >> > >>
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

Re: Apache Ignite 2.1 scope

[Taras Ledkov]

Folks,

IGNITE-4922 JDBC Driver: renew thin client based solution:

On 2.1 the functionality of the new thin client JDBC driver will be 
between deprecated Ignite thin JDBC and Ignite JDBCv2.
1. The most functions of SQL query (include DML) are implemented and 
ready for review;

2. The most functions of JDBC metadata are implemented and ready for review;
3. Transactions, batching, streaming, blobs, scrollable / writable 
cursors will not be supported in 2.1.



On 01.06.2017 18:43, Vladimir Ozerov wrote:

Folks,

We are almost reached proposed feature-complete date (June 2), Could you
please share current status of your major features?

On Tue, May 16, 2017 at 3:51 AM, Dmitriy Setrakyan 
wrote:


Looks a little tight. Let's hope we can make it.

On Mon, May 15, 2017 at 1:29 PM, Denis Magda  wrote:


Well, let me propose the following milestones for 2.1 release then.

Code freeze: June 2nd.
Final QA and benchmarking: June 5 - June 8
Voting: ~ June 9
Release: ~ June 13

Also I heard H2 has to be released once again to support Ignite’s CREATE
table command. Think that we should talk to H2 folks to make it happen in
June 22nd - June 2nd time frame.

—
Denis


On May 11, 2017, at 2:26 AM, Pavel Tupitsyn 

wrote:

As for .NET, I would propose to concentrate on peer deployment

(IGNITE-2492)

and related stuff, like IGNITE-1894 .NET: Delegate support in the API

via

extension methods.

SQL Dependency does not look important to me, we can reschedule it for
later versions.

On Thu, May 11, 2017 at 12:01 PM, Dmitriy Setrakyan <

dsetrak...@apache.org>

wrote:


Vyacheslav, I think it is worth the research, but you should always

keep

data querying and indexing in mind. For example, I don't see how

by-page

compression will solve it.

On Thu, May 11, 2017 at 1:52 AM, Vyacheslav Daradur <

daradu...@gmail.com>

wrote:


Dmitriy,

I'm researching a best way for this future.

At the moment I found only one way (querying and indexing

compatible),

this

is per-objects-field compression.

But there is a good proffit only for long strings or fields with

large

objects.

Maybe it makes sense just to introduce compression for string fileds.

I'm researching the new page-memory architecture as applied to

by-page

compression.

2017-05-11 11:30 GMT+03:00 Dmitriy Setrakyan 

wrote:


Denis,

The described roadmap looks great!

Additional, I vote for introducing an ability (OOTB) to store

objects

in

a

cache in a compressed form.
This will allow to store more data at the cost of incriasing of CPU
utilization.


One of the problems with compression is indexing and querying. How

do

we

index the data if it is compressed?




2017-05-11 4:23 GMT+03:00 Denis Magda :


Igniters,

Let me start a discussion around the scope for 2.1 release.

In my vision the main direction of our ongoing efforts should be
implementing in life a use case of Ignite as a transactional

distributed

SQL database and HTAP platform. The current use cases (database

cache,

data

grid, micro services platform, etc.) will be supported as usual,

no

changes

on that frontier.

Keeping this in mind, the roadmap needs to include essential SQL

related

features as well as disk based capabilities, MVCC support,

advanced

DDL

implementation and so on so forth. This is for Ignite as a SQL

database.

Next, Machine Learning will be a great addition to Ignite as an

HTAP

platform offering. This is why we should keep investing our time

and

resources in that recently released component.

Having this said, I see the scope for 2.1 release this way:

1. Distributed Persistent Store - if the donation is accepted by

ASF.

The

decision is to be done in separate discussion. W/o the store
   Ignite can only be used as In-Memory SQL database.

2. SQL Grid:
- CREATE & DROP table commands: https://issues.apache.org/
jira/browse/IGNITE-4651
- Renewed JDBC driver: https://issues.apache.org/
jira/browse/IGNITE-4922
- Collocation based routing of SQL queries:

https://issues.apache.org/

jira/browse/IGNITE-4510,
   https://issues.apache.org/jira/browse/IGNITE-4509
-

3. .NET:
- Peer-class loading: https://issues.apache.org/
jira/browse/IGNITE-2492
- SQLDependency: https://issues.apache.org/

jira/browse/IGNITE-2657

4. C++:
- Compute Grid: https://issues.apache.org/

jira/browse/IGNITE-3574

5. ML Grid:
- Linear regression algorithms: https://issues.apache.org/
jira/browse/IGNITE-5012
- K-means clustering: https://issues.apache.org/
jira/browse/IGNITE-5113


Please join the thread and share your thoughts, ideas and

concerns.

—
Denis







--
Best Regards, Vyacheslav




--
Best Regards, Vyacheslav





--
Taras Ledkov
Mail-To: tled...@gridgain.com

Re: Apache Ignite 2.1 scope

[Pavel Tupitsyn]

.NET:
* IGNITE-2492 Peer assembly loading - merged
* IGNITE-1894 Delegate support in the API via extension methods - postponed
to 2.2 (see comments)
* IGNITE-4904 DML Delete via LINQ - merged

On Thu, Jun 1, 2017 at 6:43 PM, Vladimir Ozerov 
wrote:

> Folks,
>
> We are almost reached proposed feature-complete date (June 2), Could you
> please share current status of your major features?
>
> On Tue, May 16, 2017 at 3:51 AM, Dmitriy Setrakyan 
> wrote:
>
> > Looks a little tight. Let's hope we can make it.
> >
> > On Mon, May 15, 2017 at 1:29 PM, Denis Magda  wrote:
> >
> > > Well, let me propose the following milestones for 2.1 release then.
> > >
> > > Code freeze: June 2nd.
> > > Final QA and benchmarking: June 5 - June 8
> > > Voting: ~ June 9
> > > Release: ~ June 13
> > >
> > > Also I heard H2 has to be released once again to support Ignite’s
> CREATE
> > > table command. Think that we should talk to H2 folks to make it happen
> in
> > > June 22nd - June 2nd time frame.
> > >
> > > —
> > > Denis
> > >
> > > > On May 11, 2017, at 2:26 AM, Pavel Tupitsyn 
> > > wrote:
> > > >
> > > > As for .NET, I would propose to concentrate on peer deployment
> > > (IGNITE-2492)
> > > > and related stuff, like IGNITE-1894 .NET: Delegate support in the API
> > via
> > > > extension methods.
> > > >
> > > > SQL Dependency does not look important to me, we can reschedule it
> for
> > > > later versions.
> > > >
> > > > On Thu, May 11, 2017 at 12:01 PM, Dmitriy Setrakyan <
> > > dsetrak...@apache.org>
> > > > wrote:
> > > >
> > > >> Vyacheslav, I think it is worth the research, but you should always
> > keep
> > > >> data querying and indexing in mind. For example, I don't see how
> > by-page
> > > >> compression will solve it.
> > > >>
> > > >> On Thu, May 11, 2017 at 1:52 AM, Vyacheslav Daradur <
> > > daradu...@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> Dmitriy,
> > > >>>
> > > >>> I'm researching a best way for this future.
> > > >>>
> > > >>> At the moment I found only one way (querying and indexing
> > compatible),
> > > >> this
> > > >>> is per-objects-field compression.
> > > >>>
> > > >>> But there is a good proffit only for long strings or fields with
> > large
> > > >>> objects.
> > > >>>
> > > >>> Maybe it makes sense just to introduce compression for string
> fileds.
> > > >>>
> > > >>> I'm researching the new page-memory architecture as applied to
> > by-page
> > > >>> compression.
> > > >>>
> > > >>> 2017-05-11 11:30 GMT+03:00 Dmitriy Setrakyan <
> dsetrak...@apache.org
> > >:
> > > >>>
> > >  On Thu, May 11, 2017 at 12:44 AM, Vyacheslav Daradur <
> > > >>> daradu...@gmail.com>
> > >  wrote:
> > > 
> > > > Denis,
> > > >
> > > > The described roadmap looks great!
> > > >
> > > > Additional, I vote for introducing an ability (OOTB) to store
> > objects
> > > >>> in
> > >  a
> > > > cache in a compressed form.
> > > > This will allow to store more data at the cost of incriasing of
> CPU
> > > > utilization.
> > > >
> > > 
> > >  One of the problems with compression is indexing and querying. How
> > do
> > > >> we
> > >  index the data if it is compressed?
> > > 
> > > 
> > > >
> > > >
> > > > 2017-05-11 4:23 GMT+03:00 Denis Magda :
> > > >
> > > >> Igniters,
> > > >>
> > > >> Let me start a discussion around the scope for 2.1 release.
> > > >>
> > > >> In my vision the main direction of our ongoing efforts should be
> > > >> implementing in life a use case of Ignite as a transactional
> > >  distributed
> > > >> SQL database and HTAP platform. The current use cases (database
> > > >>> cache,
> > > > data
> > > >> grid, micro services platform, etc.) will be supported as usual,
> > no
> > > > changes
> > > >> on that frontier.
> > > >>
> > > >> Keeping this in mind, the roadmap needs to include essential SQL
> > >  related
> > > >> features as well as disk based capabilities, MVCC support,
> > advanced
> > > >>> DDL
> > > >> implementation and so on so forth. This is for Ignite as a SQL
> > >  database.
> > > >>
> > > >> Next, Machine Learning will be a great addition to Ignite as an
> > > >> HTAP
> > > >> platform offering. This is why we should keep investing our time
> > > >> and
> > > >> resources in that recently released component.
> > > >>
> > > >> Having this said, I see the scope for 2.1 release this way:
> > > >>
> > > >> 1. Distributed Persistent Store - if the donation is accepted by
> > > >> ASF.
> > >  The
> > > >> decision is to be done in separate discussion. W/o the store
> > > >>   Ignite can only be used as In-Memory SQL database.
> > > >>
> > > >> 2. SQL Grid:
> > > >>- CREATE & DROP table commands: https://issues.apache.org/
> > > >> jira/browse/IGNITE-4651
> > > 

Re: Cassandra and Ignite C++ Issue

[Igor Rudyak]

Could you share your POJO key/value classes and persistence descriptor?

Igor

On Jun 1, 2017 8:30 AM, "Igor Sapego"  wrote:

> User's comment:
>
> I have my own cassandra table with 1 partiton key, 3 clustering keys,
> and I expect the value not to be stored in one column as a blob.
>
> Is there any way I can achieve this?
> I have used POJO and there is an issue like the key is only getting
> inserted into the cassandra table
>
> Best Regards,
> Igor
>
> On Tue, May 30, 2017 at 12:33 PM, Igor Sapego 
> wrote:
>
> > Thanks.
> >
> > Best Regards,
> > Igor
> >
> > On Tue, May 30, 2017 at 11:35 AM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> >> Igor,
> >>
> >> There is another SO question about this, I already responded:
> >> https://stackoverflow.com/questions/44254079/ignite-with-
> >> cassandra-integration
> >>
> >> -Val
> >>
> >> On Tue, May 30, 2017 at 10:15 AM, Igor Sapego 
> >> wrote:
> >>
> >> > User's answer:
> >> > I have changed my POJO to BLOB.
> >> >
> >> > Current persistence.xml file look like
> >> >
> >> > 
> >> >>> > column="assetid"/>
> >> >>> > column="asset_desc"/>
> >> > 
> >> >
> >> > now it's produce an error like
> >> >
> >> > "Caused by: com.datastax.driver.core.exceptions.
> CodecNotFoundException:
> >> > Codec not found for requested operation: [varchar <->
> >> > java.nio.HeapByteBuffer] "
> >> >
> >> > Best Regards,
> >> > Igor
> >> >
> >> > On Fri, May 26, 2017 at 2:37 PM, Igor Sapego 
> >> wrote:
> >> >
> >> > > Cross-posted Igor's answer to SO
> >> > >
> >> > > Best Regards,
> >> > > Igor
> >> > >
> >> > > On Fri, May 26, 2017 at 8:58 AM, Igor Rudyak 
> >> wrote:
> >> > >
> >> > >> I assume that's because you are using this:
> >> > >>
> >> > >> 
> >> > >>
> >> > >>
> >> > >> Current implementation supports only BLOB serialization for binary
> >> > >> objects. We already have a ticket for this:
> https://issues.apache.or
> >> > >> g/jira/browse/IGNITE-5270
> >> > >>
> >> > >> Igor
> >> > >>
> >> > >>
> >> > >>
> >> > >> On Thu, May 25, 2017 at 8:19 PM, Denis Magda 
> >> > wrote:
> >> > >>
> >> > >>> Igor R., Igor S.,
> >> > >>>
> >> > >>> Please take a look at this issue reported on SO:
> >> > >>> https://stackoverflow.com/questions/44178514/ignite-c-client
> >> > >>> -for-cassandra-integration
> >> > >>>
> >> > >>> --
> >> > >>> Denis
> >> > >>>
> >> > >>
> >> > >>
> >> > >
> >> >
> >>
> >
> >
>

Re: Apache Ignite 2.1 scope

[Vladimir Ozerov]

Folks,

We are almost reached proposed feature-complete date (June 2), Could you
please share current status of your major features?

On Tue, May 16, 2017 at 3:51 AM, Dmitriy Setrakyan 
wrote:

> Looks a little tight. Let's hope we can make it.
>
> On Mon, May 15, 2017 at 1:29 PM, Denis Magda  wrote:
>
> > Well, let me propose the following milestones for 2.1 release then.
> >
> > Code freeze: June 2nd.
> > Final QA and benchmarking: June 5 - June 8
> > Voting: ~ June 9
> > Release: ~ June 13
> >
> > Also I heard H2 has to be released once again to support Ignite’s CREATE
> > table command. Think that we should talk to H2 folks to make it happen in
> > June 22nd - June 2nd time frame.
> >
> > —
> > Denis
> >
> > > On May 11, 2017, at 2:26 AM, Pavel Tupitsyn 
> > wrote:
> > >
> > > As for .NET, I would propose to concentrate on peer deployment
> > (IGNITE-2492)
> > > and related stuff, like IGNITE-1894 .NET: Delegate support in the API
> via
> > > extension methods.
> > >
> > > SQL Dependency does not look important to me, we can reschedule it for
> > > later versions.
> > >
> > > On Thu, May 11, 2017 at 12:01 PM, Dmitriy Setrakyan <
> > dsetrak...@apache.org>
> > > wrote:
> > >
> > >> Vyacheslav, I think it is worth the research, but you should always
> keep
> > >> data querying and indexing in mind. For example, I don't see how
> by-page
> > >> compression will solve it.
> > >>
> > >> On Thu, May 11, 2017 at 1:52 AM, Vyacheslav Daradur <
> > daradu...@gmail.com>
> > >> wrote:
> > >>
> > >>> Dmitriy,
> > >>>
> > >>> I'm researching a best way for this future.
> > >>>
> > >>> At the moment I found only one way (querying and indexing
> compatible),
> > >> this
> > >>> is per-objects-field compression.
> > >>>
> > >>> But there is a good proffit only for long strings or fields with
> large
> > >>> objects.
> > >>>
> > >>> Maybe it makes sense just to introduce compression for string fileds.
> > >>>
> > >>> I'm researching the new page-memory architecture as applied to
> by-page
> > >>> compression.
> > >>>
> > >>> 2017-05-11 11:30 GMT+03:00 Dmitriy Setrakyan  >:
> > >>>
> >  On Thu, May 11, 2017 at 12:44 AM, Vyacheslav Daradur <
> > >>> daradu...@gmail.com>
> >  wrote:
> > 
> > > Denis,
> > >
> > > The described roadmap looks great!
> > >
> > > Additional, I vote for introducing an ability (OOTB) to store
> objects
> > >>> in
> >  a
> > > cache in a compressed form.
> > > This will allow to store more data at the cost of incriasing of CPU
> > > utilization.
> > >
> > 
> >  One of the problems with compression is indexing and querying. How
> do
> > >> we
> >  index the data if it is compressed?
> > 
> > 
> > >
> > >
> > > 2017-05-11 4:23 GMT+03:00 Denis Magda :
> > >
> > >> Igniters,
> > >>
> > >> Let me start a discussion around the scope for 2.1 release.
> > >>
> > >> In my vision the main direction of our ongoing efforts should be
> > >> implementing in life a use case of Ignite as a transactional
> >  distributed
> > >> SQL database and HTAP platform. The current use cases (database
> > >>> cache,
> > > data
> > >> grid, micro services platform, etc.) will be supported as usual,
> no
> > > changes
> > >> on that frontier.
> > >>
> > >> Keeping this in mind, the roadmap needs to include essential SQL
> >  related
> > >> features as well as disk based capabilities, MVCC support,
> advanced
> > >>> DDL
> > >> implementation and so on so forth. This is for Ignite as a SQL
> >  database.
> > >>
> > >> Next, Machine Learning will be a great addition to Ignite as an
> > >> HTAP
> > >> platform offering. This is why we should keep investing our time
> > >> and
> > >> resources in that recently released component.
> > >>
> > >> Having this said, I see the scope for 2.1 release this way:
> > >>
> > >> 1. Distributed Persistent Store - if the donation is accepted by
> > >> ASF.
> >  The
> > >> decision is to be done in separate discussion. W/o the store
> > >>   Ignite can only be used as In-Memory SQL database.
> > >>
> > >> 2. SQL Grid:
> > >>- CREATE & DROP table commands: https://issues.apache.org/
> > >> jira/browse/IGNITE-4651
> > >>- Renewed JDBC driver: https://issues.apache.org/
> > >> jira/browse/IGNITE-4922
> > >>- Collocation based routing of SQL queries:
> > > https://issues.apache.org/
> > >> jira/browse/IGNITE-4510,
> > >>   https://issues.apache.org/jira/browse/IGNITE-4509
> > >>-
> > >>
> > >> 3. .NET:
> > >>- Peer-class loading: https://issues.apache.org/
> > >> jira/browse/IGNITE-2492
> > >>- SQLDependency: https://issues.apache.org/
> > >>> jira/browse/IGNITE-2657
> > >>
> > >> 4. C++:
> > 

Re: Cassandra and Ignite C++ Issue

[Igor Sapego]

User's comment:

I have my own cassandra table with 1 partiton key, 3 clustering keys,
and I expect the value not to be stored in one column as a blob.

Is there any way I can achieve this?
I have used POJO and there is an issue like the key is only getting
inserted into the cassandra table

Best Regards,
Igor

On Tue, May 30, 2017 at 12:33 PM, Igor Sapego  wrote:

> Thanks.
>
> Best Regards,
> Igor
>
> On Tue, May 30, 2017 at 11:35 AM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
>> Igor,
>>
>> There is another SO question about this, I already responded:
>> https://stackoverflow.com/questions/44254079/ignite-with-
>> cassandra-integration
>>
>> -Val
>>
>> On Tue, May 30, 2017 at 10:15 AM, Igor Sapego 
>> wrote:
>>
>> > User's answer:
>> > I have changed my POJO to BLOB.
>> >
>> > Current persistence.xml file look like
>> >
>> > 
>> >   > > column="assetid"/>
>> >   > > column="asset_desc"/>
>> > 
>> >
>> > now it's produce an error like
>> >
>> > "Caused by: com.datastax.driver.core.exceptions.CodecNotFoundException:
>> > Codec not found for requested operation: [varchar <->
>> > java.nio.HeapByteBuffer] "
>> >
>> > Best Regards,
>> > Igor
>> >
>> > On Fri, May 26, 2017 at 2:37 PM, Igor Sapego 
>> wrote:
>> >
>> > > Cross-posted Igor's answer to SO
>> > >
>> > > Best Regards,
>> > > Igor
>> > >
>> > > On Fri, May 26, 2017 at 8:58 AM, Igor Rudyak 
>> wrote:
>> > >
>> > >> I assume that's because you are using this:
>> > >>
>> > >> 
>> > >>
>> > >>
>> > >> Current implementation supports only BLOB serialization for binary
>> > >> objects. We already have a ticket for this: https://issues.apache.or
>> > >> g/jira/browse/IGNITE-5270
>> > >>
>> > >> Igor
>> > >>
>> > >>
>> > >>
>> > >> On Thu, May 25, 2017 at 8:19 PM, Denis Magda 
>> > wrote:
>> > >>
>> > >>> Igor R., Igor S.,
>> > >>>
>> > >>> Please take a look at this issue reported on SO:
>> > >>> https://stackoverflow.com/questions/44178514/ignite-c-client
>> > >>> -for-cassandra-integration
>> > >>>
>> > >>> --
>> > >>> Denis
>> > >>>
>> > >>
>> > >>
>> > >
>> >
>>
>
>

Configuration inconsistency IGNITE-896

[ALEKSEY KUZNETSOV]

Will start working on the ticket if nobody mind
-- 

*Best Regards,*

*Kuznetsov Aleksey*

[GitHub] ignite pull request #2062: IGNITE-5355 Create task with release tools

[achetaev]

GitHub user achetaev opened a pull request:

https://github.com/apache/ignite/pull/2062

IGNITE-5355 Create task with release tools



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5355

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2062.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2062


commit a6ad620cece5be02da4f8394a501f7c7fd76f165
Author: Aleksey Chetaev 
Date:   2017-06-01T15:00:21Z

IGNITE-5355: Added new class to ignite-tools for generate release report

commit 1aeee2062d48324712bc51c1b6cde9969d58cacc
Author: Aleksey Chetaev 
Date:   2017-06-01T15:17:10Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-5355




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #2061: IGNITE-4887 Support for starting transaction in a...

[voipp]

GitHub user voipp opened a pull request:

https://github.com/apache/ignite/pull/2061

IGNITE-4887 Support for starting transaction in another thread



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/voipp/ignite IGNITE-4887

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2061.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2061


commit 3dc81c6797ab05b07f41c1fe016248b14cbbbe89
Author: voipp 
Date:   2017-05-25T15:56:45Z

IGNITE-4887 Support for starting transaction in another thread




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #2060: Ignite-3562-lucele-5.5.2

[AMashenkov]

GitHub user AMashenkov opened a pull request:

https://github.com/apache/ignite/pull/2060

Ignite-3562-lucele-5.5.2

for test purposes

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-3562-5.5.2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2060.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2060


commit f41b8322ffd704c30df5e4b6727b9b29522be3fc
Author: Andrey V. Mashenkov 
Date:   2017-05-22T16:31:40Z

Lucene dependency upgraded to version 4.10.4

commit 175a75f80896533d93cffe4961ec428079af7019
Author: Andrey V. Mashenkov 
Date:   2017-05-22T17:31:39Z

Styles

commit 3391bc70a41fe69d9fab51f81a0e6b29d7685e60
Author: Andrey V. Mashenkov 
Date:   2017-05-23T12:54:37Z

Minors.

commit 356a5e41fb2b648091df33400388bf4e813d0288
Author: Andrey V. Mashenkov 
Date:   2017-05-23T13:56:19Z

Minors

commit 2fea0334874c06f420b8bcf1062ce7a4d7009a4f
Author: AMRepo 
Date:   2017-05-31T10:29:49Z

Lucene dependency upgraded to 5.5.2.

(cherry picked from commit e6eae62)




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: contention on DataStructure creation/removing

[Ilya Lantukh]

Alexey,

I think, that from user's perspective data structures look more like a
library that works on top of caches, not something completely separate and
independent. Most properties from CacheConfiguration make sense for Atomics
and Collections (for example, properties related to rebalancing). We can
always validate cache config in methods that create data structure and
throw exception if it is incorrect.

But simply adding groupName to AtomicConfiguration is fine too.

On Thu, Jun 1, 2017 at 5:16 PM, Yakov Zhdanov  wrote:

> Agree with Alex.
>
> --Yakov
>
> 2017-06-01 17:10 GMT+03:00 Alexey Goncharuk :
>
>> I do not like this change - we intentionally separated a few properties
>> in AtomicConfiguration that make sense for Atomics, there is not need to
>> get back to cache configuration again. In my understanding, we only need to
>> add groupName to Atomics and Collection configuration.
>>
>> Thoughts?
>>
>> 2017-06-01 16:32 GMT+03:00 Ilya Lantukh :
>>
>>> This is how I see API to create data structures in user-defined caches:
>>> https://github.com/apache/ignite/pull/2058
>>>
>>> On Thu, Jun 1, 2017 at 2:56 PM, Ilya Lantukh 
>>> wrote:
>>>
 Hi all,

 There are other problems with current data structures implementation,
 which are related to new persistence mechanics. For example, take a look at
 this ticket: https://issues.apache.org/jira/browse/IGNITE-5324
 I think the best approach is to store data structures not in special
 system cache, but in user defined ones. All API methods to access data
 structures will have cacheName parameter, and unique identifier will be a
 pair (cacheName, dsName). In this case we won't need a single place to
 store all data structure metadata.
 What do you think?


 On Thu, Jun 1, 2017 at 1:32 PM, Mikhail Cherkasov <
 mcherka...@gridgain.com> wrote:

> On Thu, Jun 1, 2017 at 1:29 AM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> wrote:
>
> > Won't it be confusing from a user stand point to have multiple data
> > structures with the same name?
> >
>
> I won't change this, optionally I can allow to have the same name for
> different data structures' types,
> but it's better to keep single namespace for all data structures as
> it's
> implemented now.
>
>
> > Also, what is the performance impact of this
> > change?
> >
>
> I'm working on a benchmark.
>
>
> --
> Thanks,
> Mikhail.
>



 --
 Best regards,
 Ilya

>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Ilya
>>>
>>
>>
>


-- 
Best regards,
Ilya

[jira] [Created] (IGNITE-5386) Inactive mode must be forced on starting up grid with persistence enabled

[Sergey Chugunov (JIRA)]

Sergey Chugunov created IGNITE-5386:
---

 Summary: Inactive mode must be forced on starting up grid with 
persistence enabled
 Key: IGNITE-5386
 URL: https://issues.apache.org/jira/browse/IGNITE-5386
 Project: Ignite
  Issue Type: Task
  Components: persistence
Reporter: Sergey Chugunov
Assignee: Sergey Chugunov
 Fix For: 2.1


h2. Notes
There is an issue in grid startup documented in IGNITE-5363.

In a nutshell it is caused by double initialization of database memory and disk 
structures when grid started in inactive mode gets activated.

To prevent this grid with persistence enabled must be started in the following 
order
# Inactive mode must be forced if persistence is enabled (if node is starting 
up in active mode and persistence is enabled, an exception must be thrown).
# During starting the node database is left uninitialized.
# When user triggers activating the cluster, database memory and disk 
structures are initialized on all nodes.

h2. Acceptance Criteria
# Inactive mode is forced on grid startup when persistence is enabled.
# New node join scenario is properly covered.
# All persistence-related tests are passing, no new test failures are added.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5385) Get rid of discovery custom message on exchange completion

[Yakov Zhdanov (JIRA)]

Yakov Zhdanov created IGNITE-5385:
-

 Summary: Get rid of discovery custom message on exchange completion
 Key: IGNITE-5385
 URL: https://issues.apache.org/jira/browse/IGNITE-5385
 Project: Ignite
  Issue Type: Improvement
  Components: cache
Affects Versions: 2.0
Reporter: Yakov Zhdanov
Assignee: Yakov Zhdanov
Priority: Blocker
 Fix For: 2.1


Currently if late affinity assignment is on we send full partition map as a 
custom message to make sure all nodes get it. With greater number of nodes and 
caches this can cause significant slowdowns.

We suggest to move sending to communication. In this case scenario with 
coordinator failure requires special handling, since in this case some nodes 
may receive full map, complete exchange and proceed with cache operations, 
while others may not received full map yet. In this case full map should be 
resend from new coordinator - it should be recalculated if none has received 
one from former coordinator or should be requested from one of the lucky 
receivers to get forwarded to other nodes.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: contention on DataStructure creation/removing

[Ilya Lantukh]

This is how I see API to create data structures in user-defined caches:
https://github.com/apache/ignite/pull/2058

On Thu, Jun 1, 2017 at 2:56 PM, Ilya Lantukh  wrote:

> Hi all,
>
> There are other problems with current data structures implementation,
> which are related to new persistence mechanics. For example, take a look at
> this ticket: https://issues.apache.org/jira/browse/IGNITE-5324
> I think the best approach is to store data structures not in special
> system cache, but in user defined ones. All API methods to access data
> structures will have cacheName parameter, and unique identifier will be a
> pair (cacheName, dsName). In this case we won't need a single place to
> store all data structure metadata.
> What do you think?
>
>
> On Thu, Jun 1, 2017 at 1:32 PM, Mikhail Cherkasov  > wrote:
>
>> On Thu, Jun 1, 2017 at 1:29 AM, Dmitriy Setrakyan 
>> wrote:
>>
>> > Won't it be confusing from a user stand point to have multiple data
>> > structures with the same name?
>> >
>>
>> I won't change this, optionally I can allow to have the same name for
>> different data structures' types,
>> but it's better to keep single namespace for all data structures as it's
>> implemented now.
>>
>>
>> > Also, what is the performance impact of this
>> > change?
>> >
>>
>> I'm working on a benchmark.
>>
>>
>> --
>> Thanks,
>> Mikhail.
>>
>
>
>
> --
> Best regards,
> Ilya
>



-- 
Best regards,
Ilya

[GitHub] ignite pull request #2059: IGNITE-5075-pds

[gvvinblade]

GitHub user gvvinblade opened a pull request:

https://github.com/apache/ignite/pull/2059

IGNITE-5075-pds



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5075-pds

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2059.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2059


commit 28baebcd50ef1162623be81a8ebac0ed4b10283c
Author: Igor Seliverstov 
Date:   2017-06-01T09:08:20Z

IGNITE-5075-pds dataStructures test

commit 9c2fdfbfc5eb296236c18a59c48f60e60eec969f
Author: Igor Seliverstov 
Date:   2017-06-01T13:26:57Z

IGNITE-5075-pds dataStructures test




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #2058: Example of API changes.

[ilantukh]

GitHub user ilantukh opened a pull request:

https://github.com/apache/ignite/pull/2058

Example of API changes.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite 
ignite-5075-datastructures

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2058.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2058


commit 90c6345ef6c08eca35ecc35813f042f69f006ef3
Author: Ilya Lantukh 
Date:   2017-06-01T13:19:46Z

Example of API changes.

(cherry picked from commit c3b6c91)




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #2057: IGNITE-5309: Added schema to SqlFieldsQuery for C...

[isapego]

GitHub user isapego opened a pull request:

https://github.com/apache/ignite/pull/2057

IGNITE-5309: Added schema to SqlFieldsQuery for C++



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5309

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2057.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2057


commit 412183559fe16ed83eafd245c7abc0cf92f43090
Author: Igor Sapego 
Date:   2017-06-01T11:05:14Z

IGNITE-5309: Added schema to SqlFieldsQuery for C++

commit 711cdbce0a61a2e74e2161e652ce7f8d0e70c76d
Author: Igor Sapego 
Date:   2017-06-01T12:36:27Z

IGNITE-5309: Added test




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-5384) JDBC Driver: implement ResultSet#getDate(int, Calendar)

[Taras Ledkov (JIRA)]

Taras Ledkov created IGNITE-5384:


 Summary: JDBC Driver: implement ResultSet#getDate(int, Calendar)
 Key: IGNITE-5384
 URL: https://issues.apache.org/jira/browse/IGNITE-5384
 Project: Ignite
  Issue Type: Task
  Components: sql
Affects Versions: 2.1
Reporter: Taras Ledkov
 Fix For: 2.2


Current implementation doesn't use Calendar object to convert the date.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: contention on DataStructure creation/removing

[Ilya Lantukh]

Hi all,

There are other problems with current data structures implementation, which
are related to new persistence mechanics. For example, take a look at this
ticket: https://issues.apache.org/jira/browse/IGNITE-5324
I think the best approach is to store data structures not in special system
cache, but in user defined ones. All API methods to access data structures
will have cacheName parameter, and unique identifier will be a pair
(cacheName, dsName). In this case we won't need a single place to store all
data structure metadata.
What do you think?


On Thu, Jun 1, 2017 at 1:32 PM, Mikhail Cherkasov 
wrote:

> On Thu, Jun 1, 2017 at 1:29 AM, Dmitriy Setrakyan 
> wrote:
>
> > Won't it be confusing from a user stand point to have multiple data
> > structures with the same name?
> >
>
> I won't change this, optionally I can allow to have the same name for
> different data structures' types,
> but it's better to keep single namespace for all data structures as it's
> implemented now.
>
>
> > Also, what is the performance impact of this
> > change?
> >
>
> I'm working on a benchmark.
>
>
> --
> Thanks,
> Mikhail.
>



-- 
Best regards,
Ilya

[jira] [Created] (IGNITE-5383) Do not perform cache key validation when BinaryMarshaller is used

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-5383:
---

 Summary: Do not perform cache key validation when BinaryMarshaller 
is used
 Key: IGNITE-5383
 URL: https://issues.apache.org/jira/browse/IGNITE-5383
 Project: Ignite
  Issue Type: Task
  Components: binary, cache
Affects Versions: 2.1
Reporter: Vladimir Ozerov
 Fix For: 2.2


Currently whenever cache operation is performed, we invoke 
{{GridCacheAdapter.validateCacheKey}} to make sure that key overrides 
{{equals}} and {{hashCode}}. 

This check should not be performed when {{BinaryMarshaller}} is set, since we 
do not use these methods in binary mode. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5382) SQL: frequent switch between schemas cause severe slowdown

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-5382:
---

 Summary: SQL: frequent switch between schemas cause severe slowdown
 Key: IGNITE-5382
 URL: https://issues.apache.org/jira/browse/IGNITE-5382
 Project: Ignite
  Issue Type: Task
  Components: cache, sql
Reporter: Vladimir Ozerov


We have thread-bound cached connection to H2 database which is bound to 
specific schema. See {{IgniteH2Indexing.connectionForThread}}.

When query with different schema is executed, we call {{SET SCHEMA}} command, 
which is rather expensive and may cause slowdown when queries form different 
caches are executed.

To avoid this we should maintain thread-local map of such connections. Be 
careful with concurrency and resource leaks.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5381) JDBC thin: support JdbcPreparedStatement.getParameterMetaData method

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-5381:
---

 Summary: JDBC thin: support 
JdbcPreparedStatement.getParameterMetaData method
 Key: IGNITE-5381
 URL: https://issues.apache.org/jira/browse/IGNITE-5381
 Project: Ignite
  Issue Type: Task
  Components: sql
Reporter: Vladimir Ozerov
 Fix For: 2.2






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Persistent Store Stabilization for release

[Sergey Chugunov]

Hello Denis,

There are three suites [1] [2] [3] configured on TeamCity with bunch of
tests for PDS functionality.

As you can see most of tests are passing now (only 6 tests are
failing, for *partitionLoss
*tests problem was almost sorted out), it looks like that this
functionality is pretty stable.


[1] Ignite PDS 1

[2] Ignite PDS 2

[3] Ignite OutOfMemory


On Thu, Jun 1, 2017 at 1:24 AM, Dmitriy Setrakyan 
wrote:

> Thanks Denis. I also would like to encourage to post all the stabilization
> updates for the donated branch here. This will help the community to get
> familiar with the new code and functionality.
>
> In addition, everyone in the community should feel free to peruse through
> the code in this branch and ask any questions that may come up:
>
> https://github.com/apache/ignite/tree/ignite-5267/modules/pds
>
> D.
>
>
> On Wed, May 31, 2017 at 2:37 PM, Denis Magda  wrote:
>
> > Igniters,
> >
> > The Persistent Store donation was accepted by the community and now we
> > should think of its readiness for a public release.
> >
> > I know that many of you are involved it the donation stabilization. Could
> > you share the current status of the feature? How far is it from the
> > releasable state?
> >
> > —
> > Denis
>

[jira] [Created] (IGNITE-5380) Validate cache QueryEntities in discovery thread

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-5380:
---

 Summary: Validate cache QueryEntities in discovery thread
 Key: IGNITE-5380
 URL: https://issues.apache.org/jira/browse/IGNITE-5380
 Project: Ignite
  Issue Type: Task
  Components: cache, sql
Reporter: Vladimir Ozerov
Assignee: Alexander Paschenko
 Fix For: 2.1


Consider the following case:
1) Execute SQL: TABLE Person ...}}
2) Then again: TABLE Person ...}}
Second call will lead to exception in exchange thread and will hang the whole 
cluster. 

We need to add validation of {{CacheConfiguration.queryEntities}} wrt to other 
caches. This check should be performed in discovery thread. Note that we cannot 
rely on {{GridQueryProcessor}} or {{IgniteH2Indexing}} state, as some cache 
start requests may already be enqueued to exchange worker. Instead, we should 
perform cross-cache validation base only on two things:
1) {{DynamicCacheDescriptor.cacheCfg}}
2) {{DynamicCacheDescriptor.schema}}

That is, we should resolve cache schema name from configuration, tables and 
indexes from schema, and then cross-validate them with other caches.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5379) JDBC thin: get rid of cache name

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-5379:
---

 Summary: JDBC thin: get rid of cache name
 Key: IGNITE-5379
 URL: https://issues.apache.org/jira/browse/IGNITE-5379
 Project: Ignite
  Issue Type: Task
  Components: sql
Reporter: Vladimir Ozerov
 Fix For: 2.1


JDBC driver should not depend on cache name anyhow. We should remove it form 
configuration, and use {{GridQueryProcessor.querySqlFieldsNoCache}} method to 
start query.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5378) JDBC thin: support query cancellation

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-5378:
---

 Summary: JDBC thin: support query cancellation
 Key: IGNITE-5378
 URL: https://issues.apache.org/jira/browse/IGNITE-5378
 Project: Ignite
  Issue Type: Task
  Components: sql
Reporter: Vladimir Ozerov
 Fix For: 2.2


We need to support {{Statement.cancel()}} for thin JDBC driver. Ignite already 
able to cancel queries on a server side, so we should simply create new 
request-response pair for this.

Special attention should be dedicated to precise specification semantics. 
Namely, query could be started from the one thread, but cancelled from another. 
Hence, concurrency should be implemented accurately.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5376) JDBC thin: Expose SqlFieldsQuery hints as parameters

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-5376:
---

 Summary: JDBC thin: Expose SqlFieldsQuery hints as parameters
 Key: IGNITE-5376
 URL: https://issues.apache.org/jira/browse/IGNITE-5376
 Project: Ignite
  Issue Type: Task
  Components: sql
Reporter: Vladimir Ozerov
 Fix For: 2.1


We should expose:
1) {{enforceJoinOrder}}
2) {{distributedJoins}}
3) {{replicatedOnly}}
4) {{collocated}}
5) {{pageSize}}

As per the rest:
- {{partitions}} should be avoided for now, as they are very request-specific
- {{local}} flag cannot be supported for now, as "cacheless" query execution 
cannot be executed through "local" workflow for now (we cannot determine query 
parallelism)
- {{timeout}} - will be handled in another ticket.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5377) ODBC: Expose SqlFieldsQuery hints as parameters

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-5377:
---

 Summary: ODBC: Expose SqlFieldsQuery hints as parameters
 Key: IGNITE-5377
 URL: https://issues.apache.org/jira/browse/IGNITE-5377
 Project: Ignite
  Issue Type: Task
  Components: odbc
Reporter: Vladimir Ozerov
Assignee: Igor Sapego
 Fix For: 2.1


Same as for JDBC (IGNITE-5376):

We should expose:
1) {{enforceJoinOrder}}
2) {{distributedJoins}}
3) {{replicatedOnly}}
4) {{collocated}}
5) {{pageSize}}

As per the rest:
- {{partitions}} should be avoided for now, as they are very request-specific
- {{local}} flag cannot be supported for now, as "cacheless" query execution 
cannot be executed through "local" workflow for now (we cannot determine query 
parallelism)
- {{timeout}} - will be handled in another ticket.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Persistent Distributed Store Metrics

[Sergey Chugunov]

Guys,

I created a ticket [1] summing up results of our discussion. Please review
and leave comments if something is still unclear there.

Thanks,
Sergey

[1] https://issues.apache.org/jira/browse/IGNITE-5375

On Wed, May 31, 2017 at 11:56 PM, Dmitriy Setrakyan 
wrote:

> Denis, I do agree, the concept of virtual memory may work. I will start
> another discussion thread on that.
>
> We should extensively javadoc all these interfaces. The current one-liner
> javadoc documentation is not enough.
>
> Also, I would like to make a small correction to the MemoryMetrics
> interface, see below...
>
>
> On Wed, May 31, 2017 at 1:35 PM, Denis Magda  wrote:
>
> > Guys,
> > If to assume that the *page memory* is a sort of *virtual* memory that
> > maps a page to RAM or disk then MemoryMetrics interface makes sense if we
> > think of it as {Virtual}MemoryMetrics. It’s late to rename the interface
> > but this design flaw can be handled with a proper documentation.
> > At all, let’s gather all the page memory related metrics (both RAM and
> > disk) in MemoryMetrics interface. Here is it’s final version:
> > public interface MemoryMetrics {
> > public String getName();
> >
> > public long getTotalAllocatedPages();
>
>
> I think this method currently returns only the pages allocated in memory.
> To behave correctly, it now should return the total pages allocated within
> the virtual memory, which should be equal to the total pages on disk.
>
>
> > public long getPagesOnDisk();
> >
>
> This method should be removed as it represents the same value as
> "getTotalAllocatedPages()" method. Instead, we should add the following
> method:
>
> // Returns the number of pages allocated in physical off-heap memory.
> getPhysicalMemoryPages();
>
>
> >
> > public long getDirtyPages();
> >
> > public float getAllocationRate();
> >
> > public float getEvictionRate();
> >
> > public float getPagesFillFactor();
> >
> > public double getPagesMissRate();
> >
> > public float getLargeEntriesPagesPercentage();
> > }
> > I would rename getAllocationRate and getEvitionRate to
> > getPagesAllocationRate and getEvictionRates respectively but then we need
> > to deprecate the existing methods which is not good.
> >
> > As for the persistence metrics related to WAL and checkpointing let’s
> > store them here:
> >
> > public interface PersistentStoreMetrics {
> > public double getWalLoggingRate();
> >
> > public int getWalArchiveSegments();
> >
> > public double getWalFsyncTime();
> >
> > public double getCheckpointingTime();
> >
> > public double getCheckpointingFsyncTime();
> >
> > public long getCheckpointingTotalPagesNumber();
> >
> > public long[] getCheckpointingPagesByTypeNumber();
> >
> > public long getCheckpointingCopiedOnWritePagesNumber();
> > }
> >
> > Comments on the name of non released methods are welcomed.
> >
> > —
> > Denis
> >
> > > On May 30, 2017, at 2:59 PM, Dmitriy Setrakyan 
> > wrote:
> > >
> > > On Tue, May 30, 2017 at 2:46 PM, Denis Magda 
> wrote:
> > >
> > >> I don’t have any extra metrics in mind for now. All I want to say that
> > >> presently the Store metrics interface aggregates WAL and checkpointing
> > >> metrics and in the future we might add additional Store related
> metrics
> > >> there (that has nothing to do with WAL). So, this is why
> > IgniteWalMetrics
> > >> doesn’t sound like a generic interface.
> > >>
> > >
> > > I think it is getting more and more confusing. I, for instance, do not
> > > understand where memory metrics end and storage begins, neither I think
> > it
> > > is important to separate memory policy related metrics from the WAL or
> > > Checkpoint related metrics.
> > >
> > > How about we put them all together and have clear method names?
> > >
> > >
> > >>
> > >> —
> > >> Denis
> > >>
> > >>> On May 30, 2017, at 2:39 PM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> > >> wrote:
> > >>>
> > >>> On Tue, May 30, 2017 at 1:22 PM, Denis Magda 
> > wrote:
> > >>>
> >  I’m afraid that in the future we might need to add non WAL related
> > >> metrics
> >  under this interface. This is why it might make sense to keep the
> name
> >  generic.
> > 
> > >>>
> > >>> Hm... I am not sure we are going to have any other components in
> > addition
> > >>> to WAL and Store here. What do you have in mind?
> > >>
> > >>
> >
> >
>

[jira] [Created] (IGNITE-5375) New PersistentStoreMetrics, MemoryMetrics interface improvements

[Sergey Chugunov (JIRA)]

Sergey Chugunov created IGNITE-5375:
---

 Summary: New PersistentStoreMetrics, MemoryMetrics interface 
improvements
 Key: IGNITE-5375
 URL: https://issues.apache.org/jira/browse/IGNITE-5375
 Project: Ignite
  Issue Type: Task
  Components: persistence
Reporter: Sergey Chugunov
 Fix For: 2.1


h2. Notes
Original discussions on dev list: on [Persistent Store 
Metrics|http://apache-ignite-developers.2346864.n4.nabble.com/Persistent-Distributed-Store-Metrics-tt17730.html];
 on a new way of thinking of Memory Policies as [virtual 
memory|http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Ignite-memory-gt-virtual-memory-tt18291.html].

Existing {{MemoryMetrics}} interface will be enhanced with additional methods 
for monitoring persistence activities (only new methods are listed):
{code}
public interface MemoryMetrics {
...
public long getDirtyPages();

public float getPagesReplaceRate();

public long getPhysicalMemoryPages();
}
{code}

Proposed interface for Persistent Store Metrics:
{code}
public interface PersistentStoreMetrics {
public float getWalLoggingRate();

public int getWalArchiveSegments();

public float getWalFsyncTime();

public float getCheckpointingTime();

public float getCheckpointingFsyncTime();

public long getCheckpointingTotalPagesNumber();

public long[] getCheckpointingPagesByTypeNumber();

public long getCheckpointingCopiedOnWritePagesNumber();
}
{code}

h2. Acceptance Criteria
# MemoryMetrics interface is expanded with new methods
# Persistent Store Metrics are implemented according to the proposed interface
# Semantics of existing method {{MemoryMetrics::getTotalAllocatedPages}} is 
changed to return not only pages residing in memory, but total amount of paged 
including ones on disk. 
For amount of pages in RAM new method {{getPhysicalMemoryPages}} is added.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5374) JDBC/ODBC: make sure that server doesn't have idle timeout

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-5374:
---

 Summary: JDBC/ODBC: make sure that server doesn't have idle timeout
 Key: IGNITE-5374
 URL: https://issues.apache.org/jira/browse/IGNITE-5374
 Project: Ignite
  Issue Type: Task
  Components: odbc, sql
Reporter: Vladimir Ozerov
 Fix For: 2.1


{{GridNioServer}} has idle timeout with defaults to 7000ms. Let's make sure 
that it is disabled for now, as we do not support ping requests, and it is 
normal to have idle client connection.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5373) JDBC thin driver: improve network configuration

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-5373:
---

 Summary: JDBC thin driver: improve network configuration
 Key: IGNITE-5373
 URL: https://issues.apache.org/jira/browse/IGNITE-5373
 Project: Ignite
  Issue Type: Task
  Components: sql
Reporter: Vladimir Ozerov
 Fix For: 2.1


We need to add "send" and "receive" socket buffer size parameters.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5372) .NET: IgniteConfiguration.LongQueryWarningTimeout

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-5372:
--

 Summary: .NET: IgniteConfiguration.LongQueryWarningTimeout
 Key: IGNITE-5372
 URL: https://issues.apache.org/jira/browse/IGNITE-5372
 Project: Ignite
  Issue Type: Improvement
  Components: platforms
Affects Versions: 2.1
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 2.1


* Add {{IgniteConfiguration.LongQueryWarningTimeout}}
* Mark {{CacheConfiguration.LongQueryWarningTimeout}} as obsolete



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: contention on DataStructure creation/removing

[Mikhail Cherkasov]

On Thu, Jun 1, 2017 at 1:29 AM, Dmitriy Setrakyan 
wrote:

> Won't it be confusing from a user stand point to have multiple data
> structures with the same name?
>

I won't change this, optionally I can allow to have the same name for
different data structures' types,
but it's better to keep single namespace for all data structures as it's
implemented now.


> Also, what is the performance impact of this
> change?
>

I'm working on a benchmark.


-- 
Thanks,
Mikhail.

[GitHub] ignite pull request #2056: IGNITE-5327 PARTITIONED and REPLICATED cache temp...

[alexpaschenko]

GitHub user alexpaschenko opened a pull request:

https://github.com/apache/ignite/pull/2056

IGNITE-5327 PARTITIONED and REPLICATED cache templates



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5327

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2056.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2056


commit e03c7b552ea11e89f47ad4c3a34b076855e77487
Author: Alexander Paschenko 
Date:   2017-06-01T10:16:23Z

IGNITE-5327 PARTITIONED and REPLICATED cache templates




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #2045: MemoryPolicyConfiguration.rateTimeInterval proper...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2045


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #2008: IGNITE-5291 Correct handling of cache stop in Ign...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2008


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #2049: IGNITE-5243

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2049


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: nested SQL sub-queries with non-collocated joins

[Sergi Vladykin]

The approach you are suggesting will be very complex for current
implementation. Also most probably very inefficient.

Actually I was thinking about another but similar approach: in many cases
we can rewrite a subquery in WHERE clause into JOIN subquery.

Like the following:

SELECT x.* FROM x WHERE x.a = (SELECT MAX(y.a) FROM y WHERE y.b = x.b)

 ===>

SELECT x.* FROM x, (SELECT MAX(y.a), y.b FROM y GROUP BY y.b) z WHERE x.b =
z.b

There are still problems here:

1. We will not be able to rewrite all the queries.
2. We should not rewrite queries like this by default because this will
have a noticeable performance penalty for correctly collocated subqueries.
Probably we will need some flag for that.

Sergi

2017-05-31 21:26 GMT+03:00 Dmitriy Setrakyan :

> Igniters (specifically Sergi),
>
> It has come to my attention today that nested sub-select statements, when
> used in combination with non-collocated joins do not work properly in
> Ignite.
>
> So a query like this, where A, B, and C are all stored in Partitioned
> caches and are **not** collocated at all, will not work.
>
>
> > *select * from A, B where a.id  = b.a_id and b.somefield in
> > (select somefield from C where c.zipcode = ?)*
>
>
> The main reason it is not supported right now is because, in the absence of
> collocation, such query may create N^N complexity and it was decided that
> it is best not supporting it at all.
>
> However, I am not sure why N^N complexity is required. Why not support it
> as follows?
>
>1. execute the nested subquery and store the result in a temporary
>Replicated table.
>2. execute the original query and use the temporary Replicated table
>instead of the sub-query.
>
> Sergi, given that you are the author of the code, can you provide some
> insight here?
>
> Thanks,
> D.
>

Re: Summary of SQL changes in 2.1

[Vladimir Ozerov]

Dmitry, Pavel, regarding limitations:

1) Caches created through SQL should not interfere with caches created
through API and config. We do not understand all implications of this
interference for now, especially in the context of persistence, so it is
better to restrict this behavior for now. Especially provided that in most
situations this restricted case doesn't make sense.

2) CREATE INDEX on non-sql caches doesn't make sense. Yes, we implemented
this in 2.0, but in this versions users have to define table structure
through QueryTable before cache start, and cannot change it in future. As
such, in 2.0 any "CREATE INDEX" could be replaced with proper QueryEntity
configuration on cache start. That is, "CREATE INDEX" is effectively
useless in 2.0.

On Thu, Jun 1, 2017 at 12:17 PM, Sergi Vladykin 
wrote:

> I think it makes sense to reserve IGNITE schema for future use as well.
>
> 2017-06-01 0:26 GMT+03:00 Dmitriy Setrakyan :
>
> > Vladmir,
> >
> > Thanks for the detailed email. My comments are inline...
> >
> > On Wed, May 31, 2017 at 11:21 AM, Vladimir Ozerov 
> > wrote:
> >
> > > Folks,
> > >
> > > Let me summarize all recent changes to our SQL engine which are
> important
> > > from user perspective. Please think of them and let me know if you have
> > any
> > > objection and thoughts on how to improve them.
> > >
> > > 1) Default "PUBLIC" schema added. It always exists and cannot be
> dropped.
> > > Many caches can reside in this schema as opposed to earlier versions,
> > where
> > > every cache must be in separate schema.
> > >
> >
> > Nice!
> >
> >
> > > 2) Caches are still created in separate schemas by default. We should
> not
> > > change this behavior, because it could break SQL queries of virtually
> all
> > > users.
> > >
> >
> > We should document, however, that this behavior will change in 3.0. Also,
> > users should be able to specify that they wish to connect to the PUBLIC
> > schema explicitly.
> >
> >
> > > 3) "CREATE TABLE" creates a cache with special internal property
> > > "sql=true". Such cache cannot be destroyed through
> "Ignite.destroyCache".
> > > It can only be dropped through "DROP TABLE".The opposite is also holds:
> > > static and dynamic caches cannot be dropped through "DROP TABLE".
> > >
> >
> > Agree.
> >
> >
> > >
> > > 4) "CREATE INDEX" and "DROP INDEX" can only be executed on "sql"
> caches.
> > >
> >
> > Ouch! Many of current Ignite users wish to have this functionality
> enabled
> > for API-based caches. Any chance to lift this limitation?
> >
> >
> > >
> > > 5) There will be two predefined templates for "CREATE CACHE" command -
> > > "REPLICATED" and "PARTITIONED". They are always available on any node.
> > >
> > > 6) Additional parameters which could be passed to "CREATE TABLE":
> > > 6.1) "cacheTemplate" - name of cache template
> > > 6.2) "backups" - number of backups
> > > 6.3) "atomicityMode" - either "TRANSACTIONAL" or "ATOMIC"
> > > 6.4) "AFFINITY KEY" - if key field should be used for affinity.
> > >
> >
> > What are the defaults?
> >
> >
> > >
> > > Example:
> > > CREATE TABLE Employee (
> > > pk_id BIGINT PRIMARY KEY,
> > > name VARCHAR(255),
> > > org_id BIGINT AFFINITY KEY
> > > ) WITH "cacheTemplate=PARTITIONED, backups=1,
> > atomicityMode=TRANSACTIONAL"
> > >
> > > 7) Connetion string of new JDBC driver starts with
> "jdbc:ignite:thin://",
> > > and has only [host] as mandatory parameter.
> > >
> > > Example: "jdbc:ignite:thin://my_machine"
> > >
> >
> > Why not have "thin" driver by default? Will users even notice?
> >
> >
> > >
> > > 8) New bean "SqlConfiguration" will be added to "IgniteConfiguration":
> > >
> > > class SqlConfiguration {
> > > SqlListenerConfiguration listenerCfg; // Content of this class will
> > be
> > > copied from OdbcConfiguration;
> > > long longQryWarnTimeout; // Moved from CacheConfiguration
> > >
> > > // Will hold more common SQL stuff such as metrics frequency,
> > > predefined schemas, etc. in future.
> > > }
> > >
> > > class SqlListenerConfiguration {
> > > String host; // Optional, bind to all interfaces if ommitted;
> > > int port; // Port
> > > // Other stuff copied from OdbcConfiguration
> > > }
> > >
> > > Example of configuration with explicitly enabled listener:
> > > new IgniteConfiguration().setSqlConfiguration(new
> > > SqlConfiguration().setListenerConfiuration(new
> > > SqlListenerConfiguration()));
> > >
> >
> > Seems that there is one-to-one dependency between SqlConfiguration and
> > SqlListenerConfiguration. This looks a bit dirty. Why not just have
> > SqlConfiguration with all the properties?
> >
> >
> > >
> > > 9) SQL listener *will not be enabled by default* as it consumes
> resources
> > > and normally will be require only on small set of nodes.
> > >
> >
> > Again, seems to be very odd. I would like SqlConfiguration to be enabled
> by
> > default, given that many users will now connect 

[GitHub] ignite pull request #2006: IGNITE-5288: Inconsistency of committed and the m...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2006


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #2014: ignite-4947: binary basic test suite was removed ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2014


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #2055: IGNITE-5305 Correct schema in GridQueryProcessor....

[alexpaschenko]

GitHub user alexpaschenko opened a pull request:

https://github.com/apache/ignite/pull/2055

IGNITE-5305 Correct schema in GridQueryProcessor.dynamicTableCreate



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5305

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2055.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2055


commit a0504b74a37ac07228c828a5445cc556ed997d7d
Author: Alexander Paschenko 
Date:   2017-06-01T09:17:05Z

IGNITE-5305 Correct schema in GridQueryProcessor.dynamicTableCreate




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Summary of SQL changes in 2.1

[Sergi Vladykin]

I think it makes sense to reserve IGNITE schema for future use as well.

2017-06-01 0:26 GMT+03:00 Dmitriy Setrakyan :

> Vladmir,
>
> Thanks for the detailed email. My comments are inline...
>
> On Wed, May 31, 2017 at 11:21 AM, Vladimir Ozerov 
> wrote:
>
> > Folks,
> >
> > Let me summarize all recent changes to our SQL engine which are important
> > from user perspective. Please think of them and let me know if you have
> any
> > objection and thoughts on how to improve them.
> >
> > 1) Default "PUBLIC" schema added. It always exists and cannot be dropped.
> > Many caches can reside in this schema as opposed to earlier versions,
> where
> > every cache must be in separate schema.
> >
>
> Nice!
>
>
> > 2) Caches are still created in separate schemas by default. We should not
> > change this behavior, because it could break SQL queries of virtually all
> > users.
> >
>
> We should document, however, that this behavior will change in 3.0. Also,
> users should be able to specify that they wish to connect to the PUBLIC
> schema explicitly.
>
>
> > 3) "CREATE TABLE" creates a cache with special internal property
> > "sql=true". Such cache cannot be destroyed through "Ignite.destroyCache".
> > It can only be dropped through "DROP TABLE".The opposite is also holds:
> > static and dynamic caches cannot be dropped through "DROP TABLE".
> >
>
> Agree.
>
>
> >
> > 4) "CREATE INDEX" and "DROP INDEX" can only be executed on "sql" caches.
> >
>
> Ouch! Many of current Ignite users wish to have this functionality enabled
> for API-based caches. Any chance to lift this limitation?
>
>
> >
> > 5) There will be two predefined templates for "CREATE CACHE" command -
> > "REPLICATED" and "PARTITIONED". They are always available on any node.
> >
> > 6) Additional parameters which could be passed to "CREATE TABLE":
> > 6.1) "cacheTemplate" - name of cache template
> > 6.2) "backups" - number of backups
> > 6.3) "atomicityMode" - either "TRANSACTIONAL" or "ATOMIC"
> > 6.4) "AFFINITY KEY" - if key field should be used for affinity.
> >
>
> What are the defaults?
>
>
> >
> > Example:
> > CREATE TABLE Employee (
> > pk_id BIGINT PRIMARY KEY,
> > name VARCHAR(255),
> > org_id BIGINT AFFINITY KEY
> > ) WITH "cacheTemplate=PARTITIONED, backups=1,
> atomicityMode=TRANSACTIONAL"
> >
> > 7) Connetion string of new JDBC driver starts with "jdbc:ignite:thin://",
> > and has only [host] as mandatory parameter.
> >
> > Example: "jdbc:ignite:thin://my_machine"
> >
>
> Why not have "thin" driver by default? Will users even notice?
>
>
> >
> > 8) New bean "SqlConfiguration" will be added to "IgniteConfiguration":
> >
> > class SqlConfiguration {
> > SqlListenerConfiguration listenerCfg; // Content of this class will
> be
> > copied from OdbcConfiguration;
> > long longQryWarnTimeout; // Moved from CacheConfiguration
> >
> > // Will hold more common SQL stuff such as metrics frequency,
> > predefined schemas, etc. in future.
> > }
> >
> > class SqlListenerConfiguration {
> > String host; // Optional, bind to all interfaces if ommitted;
> > int port; // Port
> > // Other stuff copied from OdbcConfiguration
> > }
> >
> > Example of configuration with explicitly enabled listener:
> > new IgniteConfiguration().setSqlConfiguration(new
> > SqlConfiguration().setListenerConfiuration(new
> > SqlListenerConfiguration()));
> >
>
> Seems that there is one-to-one dependency between SqlConfiguration and
> SqlListenerConfiguration. This looks a bit dirty. Why not just have
> SqlConfiguration with all the properties?
>
>
> >
> > 9) SQL listener *will not be enabled by default* as it consumes resources
> > and normally will be require only on small set of nodes.
> >
>
> Again, seems to be very odd. I would like SqlConfiguration to be enabled by
> default, given that many users will now connect to Ignite using the JDBC or
> ODBC drivers.
>
>
> >
> > 10) OdbcConfiguration will be deprecated in favor of
> > SqlListenerConfiguration.
> >
>
> Again, let's just have one SqlConfiguration interface. I am OK with
> deprecating the OdbcConfiguration, assuming that it will still work.
>
>
>
> >
> > Please share your thoughts.
> >
>

Re: Summary of SQL changes in 2.1

[Pavel Tupitsyn]

> 3) "CREATE TABLE" creates a cache with special internal property
> "sql=true". Such cache cannot be destroyed through "Ignite.destroyCache".
> It can only be dropped through "DROP TABLE".The opposite is also holds:
> static and dynamic caches cannot be dropped through "DROP TABLE".

Hi Vladimir,
What is the reason for this limitation?

On Thu, Jun 1, 2017 at 12:26 AM, Dmitriy Setrakyan 
wrote:

> Vladmir,
>
> Thanks for the detailed email. My comments are inline...
>
> On Wed, May 31, 2017 at 11:21 AM, Vladimir Ozerov 
> wrote:
>
> > Folks,
> >
> > Let me summarize all recent changes to our SQL engine which are important
> > from user perspective. Please think of them and let me know if you have
> any
> > objection and thoughts on how to improve them.
> >
> > 1) Default "PUBLIC" schema added. It always exists and cannot be dropped.
> > Many caches can reside in this schema as opposed to earlier versions,
> where
> > every cache must be in separate schema.
> >
>
> Nice!
>
>
> > 2) Caches are still created in separate schemas by default. We should not
> > change this behavior, because it could break SQL queries of virtually all
> > users.
> >
>
> We should document, however, that this behavior will change in 3.0. Also,
> users should be able to specify that they wish to connect to the PUBLIC
> schema explicitly.
>
>
> > 3) "CREATE TABLE" creates a cache with special internal property
> > "sql=true". Such cache cannot be destroyed through "Ignite.destroyCache".
> > It can only be dropped through "DROP TABLE".The opposite is also holds:
> > static and dynamic caches cannot be dropped through "DROP TABLE".
> >
>
> Agree.
>
>
> >
> > 4) "CREATE INDEX" and "DROP INDEX" can only be executed on "sql" caches.
> >
>
> Ouch! Many of current Ignite users wish to have this functionality enabled
> for API-based caches. Any chance to lift this limitation?
>
>
> >
> > 5) There will be two predefined templates for "CREATE CACHE" command -
> > "REPLICATED" and "PARTITIONED". They are always available on any node.
> >
> > 6) Additional parameters which could be passed to "CREATE TABLE":
> > 6.1) "cacheTemplate" - name of cache template
> > 6.2) "backups" - number of backups
> > 6.3) "atomicityMode" - either "TRANSACTIONAL" or "ATOMIC"
> > 6.4) "AFFINITY KEY" - if key field should be used for affinity.
> >
>
> What are the defaults?
>
>
> >
> > Example:
> > CREATE TABLE Employee (
> > pk_id BIGINT PRIMARY KEY,
> > name VARCHAR(255),
> > org_id BIGINT AFFINITY KEY
> > ) WITH "cacheTemplate=PARTITIONED, backups=1,
> atomicityMode=TRANSACTIONAL"
> >
> > 7) Connetion string of new JDBC driver starts with "jdbc:ignite:thin://",
> > and has only [host] as mandatory parameter.
> >
> > Example: "jdbc:ignite:thin://my_machine"
> >
>
> Why not have "thin" driver by default? Will users even notice?
>
>
> >
> > 8) New bean "SqlConfiguration" will be added to "IgniteConfiguration":
> >
> > class SqlConfiguration {
> > SqlListenerConfiguration listenerCfg; // Content of this class will
> be
> > copied from OdbcConfiguration;
> > long longQryWarnTimeout; // Moved from CacheConfiguration
> >
> > // Will hold more common SQL stuff such as metrics frequency,
> > predefined schemas, etc. in future.
> > }
> >
> > class SqlListenerConfiguration {
> > String host; // Optional, bind to all interfaces if ommitted;
> > int port; // Port
> > // Other stuff copied from OdbcConfiguration
> > }
> >
> > Example of configuration with explicitly enabled listener:
> > new IgniteConfiguration().setSqlConfiguration(new
> > SqlConfiguration().setListenerConfiuration(new
> > SqlListenerConfiguration()));
> >
>
> Seems that there is one-to-one dependency between SqlConfiguration and
> SqlListenerConfiguration. This looks a bit dirty. Why not just have
> SqlConfiguration with all the properties?
>
>
> >
> > 9) SQL listener *will not be enabled by default* as it consumes resources
> > and normally will be require only on small set of nodes.
> >
>
> Again, seems to be very odd. I would like SqlConfiguration to be enabled by
> default, given that many users will now connect to Ignite using the JDBC or
> ODBC drivers.
>
>
> >
> > 10) OdbcConfiguration will be deprecated in favor of
> > SqlListenerConfiguration.
> >
>
> Again, let's just have one SqlConfiguration interface. I am OK with
> deprecating the OdbcConfiguration, assuming that it will still work.
>
>
>
> >
> > Please share your thoughts.
> >
>

Re: Geo spatial index

[Alexey Goncharuk]

Same think stays for the full-text indexes which are currently stored in
Lucene.

2017-05-24 21:56 GMT+03:00 Dmitriy Setrakyan :

> Sergi,
>
> While we are figuring this out, what happens to the GeoSpatial
> functionality in the mean time? Is it going to work at all? If not, should
> we throw some sort of exception?
>
> D.
>
> On Wed, May 24, 2017 at 1:44 AM, Sergi Vladykin 
> wrote:
>
> > Though this may require some changes in BPlusTree. Let me think.
> >
> > Sergi
> >
> > 2017-05-24 8:58 GMT+03:00 Sergi Vladykin :
> >
> > > It must not be too hard to implement kd-tree over b+tree [1]. Depending
> > on
> > > level we have to compare either X or Y coordinate.
> > >
> > > I think we will even have a performance boost for spatial indexes after
> > > this change.
> > >
> > > [1] https://en.wikipedia.org/wiki/K-d_tree
> > >
> > > Sergi
> > >
> > > 2017-05-23 18:59 GMT+03:00 Denis Magda :
> > >
> > >> +1
> > >>
> > >> This looks natural considering that we switched to the new memory
> > >> architecture. Sergi, how difficult is to support this?
> > >>
> > >> —
> > >> Denis
> > >>
> > >> > On May 23, 2017, at 4:25 AM, Sergi Vladykin <
> sergi.vlady...@gmail.com
> > >
> > >> wrote:
> > >> >
> > >> > Guys,
> > >> >
> > >> > Looks like we have to move our geospatial indexes to the new
> approach
> > >> with
> > >> > BPlusTree. Right now it stores data in Java heap. This is especially
> > >> > important because we are going to have a persistence layer donated
> by
> > >> > GridGain and obviously geo spatial indexes will not work with it at
> > all.
> > >> >
> > >> > Sergi
> > >>
> > >>
> > >
> >
>

Re: Cache.getAll can return partially commited results. IGNITE-4809

[ALEKSEY KUZNETSOV]

Ah, ok thanks for clarifying. I will leave my ticket unassigned

ср, 31 мая 2017 г. в 21:10, Alexey Goncharuk :

> Aleksey,
>
> Generally, this decision cannot be made on a single node because a
> transaction may affect multiple nodes, and one node may have already
> committed the transaction and the other - not. There is a dependant ticket
> in the ticket you are currently working on which will cover all the caes.
>
> Meanwhile, an easy way to verify reads is to start an optimistic
> serializable read-only transaction and commit it after reads. A commit will
> throw OptimisticException if you happen to read an inconsistent set of
> values.
>
> 2017-05-31 19:57 GMT+03:00 ALEKSEY KUZNETSOV :
>
> > one possible solution is to lock entry reading section while userCommit()
> > method is executing.
> >
> > ср, 31 мая 2017 г., 16:52 ALEKSEY KUZNETSOV :
> >
> > > Hi, Igntrs!
> > > When the first writer thread has commited values and cache entries are
> > > getting updated with this new values, another thread is getting
> partially
> > > assigned values through the
> > >
> > > cache.getAll(keys)
> > >
> > > should this method wait for all newly updated entries to finish , or
> > > should it throw the exception?
> > >
> > > If it should wait for entries, what mechanism shall i implement? would
> > > appreciate any examples
> > > --
> > >
> > > *Best Regards,*
> > >
> > > *Kuznetsov Aleksey*
> > >
> > --
> >
> > *Best Regards,*
> >
> > *Kuznetsov Aleksey*
> >
>
-- 

*Best Regards,*

*Kuznetsov Aleksey*

[jira] [Created] (IGNITE-5370) IgniteSet misses its size after cluster restart with enabled PDS feature

[Igor Seliverstov (JIRA)]

Igor Seliverstov created IGNITE-5370:


 Summary: IgniteSet misses its size after cluster restart with 
enabled PDS feature
 Key: IGNITE-5370
 URL: https://issues.apache.org/jira/browse/IGNITE-5370
 Project: Ignite
  Issue Type: Bug
  Components: data structures
Affects Versions: 2.0
Reporter: Igor Seliverstov


{{CacheDataStructuresManager}} tracks {{IgniteSet}} updates using an on-heap 
map ({{CacheDataStructuresManager#setDataMap}}), which is used for several 
operations (including {{IgniteSet.size()}}) and isn't restored after cluster 
restart.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [DISCUSS] Ignite memory -> virtual memory

[Alexey Goncharuk]

I am fine with this javadoc change as long as there is no confusion between
Ignite page memory buffers and the OS Virtual Memory concept.

2017-06-01 2:07 GMT+03:00 Dmitriy Setrakyan :

> Igniters,
>
> With the newly donated persistence functionality in Ignite, I have been
> struggling a bit on how to fit the notion of persistence into the current
> Ignite interfaces, that are almost completely memory oriented. For example,
> abstractions like MemoryConfiguration or MemoryMetrics will now have to
> include the persistence context, given that pages will be seamlessly mapped
> to disk, whenever the memory fills up (e.g. providing the number of pages
> on disk on MemoryMetrics interface).
>
> After looking around, I have noticed that our architecture is increasingly
> beginning to look like the Virtual Memory concept in operating systems [1],
> if you consider Ignite off-heap memory to be the physical memory, and disk
> to be the secondary memory space. Just like virtual memory, our
> architecture is based on memory pages and memory segments. The total set of
> all pages constitutes the total virtual memory space.
>
> If we document our memory interfaces as virtual memory, then we won't have
> to do any renaming and can comfortably add disk-based methods to these
> interfaces, as it becomes consistent with the virtual memory concept.
>
> Thoughts?
>
> [1] - https://en.wikipedia.org/wiki/Virtual_memory
>

[jira] [Created] (IGNITE-5369) Add possibility to generate alias for primary key in WebConsole

[Evgenii Zhuravlev (JIRA)]

Evgenii Zhuravlev created IGNITE-5369:
-

 Summary: Add possibility to generate alias for primary key in 
WebConsole
 Key: IGNITE-5369
 URL: https://issues.apache.org/jira/browse/IGNITE-5369
 Project: Ignite
  Issue Type: Bug
Reporter: Evgenii Zhuravlev
Assignee: Alexey Kuznetsov


Add field with alias name for primary key, that wiil be used in 
QueryEntity.setKeyFieldName



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5368) Cassandra store should support private/public fields serialization for POJO persistence strategy

[Igor Rudyak (JIRA)]

Igor Rudyak created IGNITE-5368:
---

 Summary: Cassandra store should support private/public fields 
serialization for POJO persistence strategy
 Key: IGNITE-5368
 URL: https://issues.apache.org/jira/browse/IGNITE-5368
 Project: Ignite
  Issue Type: Improvement
  Components: cassandra
Reporter: Igor Rudyak
Assignee: Igor Rudyak


As for now for POJO persistence strategy, Cassandra cache store supports only 
java classes following JavaBeans convention (having getters & setters).

It should also support serialization/deserialization for public/private class 
fields which are not following JavaBeans convention but:
- Annotated with @QuerySqlField
- Manually specified in persistence descriptor



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)