date:20170425

Re: Null cache names

2017-04-25 Thread Roman Shtykh

Denis,
PR is created. https://issues.apache.org/jira/browse/IGNITE-5083 for review.
We will have to find a good solution to "externalize" the default cache name in 
future, as Vladimir suggested. As Igor pointed out, probably for memcached 
too.I'll create another JIRA issue for that.
Roman
 

On Wednesday, April 26, 2017 1:16 PM, Denis Magda  wrote:
 

 > So it is approach 1 ("redis_cache") I proposed initially. Any concerns? If 
 > it is ok, I set Redis to that cache and update the documentation.

Let’s handle the way Roman suggests. Don’t see any reason why we keep 
discussing this if the whole story is about a “one line” fix.

Roman, could you implement your suggestion and do a pull-request?

—
Denis

> On Apr 25, 2017, at 6:53 PM, Roman Shtykh  wrote:
> 
> Vladimir,
> I agree. But, if I get it right, according to your concern for using 
> memcached and Redis simultaneously, we'll need a separate default cache for 
> each. So it is approach 1 ("redis_cache") I proposed initially. Any concerns? 
> If it is ok, I set Redis to that cache and update the documentation.
> Btw, I agree ConnectorConfiguration is not the proper place, but having 
> 'jettyPath' in it made me think it could be done ;) Probably we will need 
> refactoring to make it generic and more separate configuration classes as 
> more protocols get implemented.
> Roman 
> 
>    On Wednesday, April 26, 2017 4:49 AM, Sergi Vladykin 
> wrote:
> 
> 
> Agree, lets move forward with the simplest possible solution for now.
> 
> Sergi
> 
> 2017-04-25 13:07 GMT+03:00 Vladimir Ozerov :
> 
>> Folks,
>> 
>> I do not think it is legal to add such property to ConnectorConfiguration.
>> Connector is a generic gateway to cluster resources. It should not bother
>> about caches anyhow. What if there are multiple caches in the cluster? What
>> is I want to access "cache A" from Memcached and "cache B" from Redis
>> simultaneously? Etc.. This kind of property should be defined on client
>> level, not on the server.
>> 
>> For now, provided that 2.0 is about to be freezed, I propose to stick to
>> Dmitriy's approach and use "default" cache name instead of null. This
>> should work fine for AI 2.0. We will be able to improve it in further
>> releases.
>> 
>> Thoughts?
>> 
>> Vladimir.
>> 
>> On Tue, Apr 25, 2017 at 7:22 AM, Roman Shtykh 
>> wrote:
>> 
>>> Igor, +1 from me.We can add a field to ConnectorConfiguration (not sure
>> if
>>> it's a proper place, but it's shared by REST, memcached and Redis). A
>> user
>>> will have to create a cache, configure as needed and specify the name in
>>> ConnectorConfiguration.
>>> Roman
>>> 
>>> 
>>> 
>>>    On Monday, April 24, 2017 10:34 PM, Seliverstov Igor <
>>> gvvinbl...@gmail.com> wrote:
>>> 
>>> 
>>>  Dear Igniters,
>>> 
>>> Seems we have almost the same issue with Memcached protocol.
>>> 
>>> There is an ability to define a cache name via operation extras message
>>> part (
>>> https://github.com/memcached/memcached/wiki/
>> BinaryProtocolRevamped#packet-
>>> structure)
>>> but it looks a bit complicated from my point of view...
>>> 
>>> Different client implementations might provide such functionality or not
>> (I
>>> mean an additional info in an operation extras), so, potential user might
>>> have some difficultes using Ignite as a Memcached server because of this
>>> ignite-specific message part becomes mandatory.
>>> 
>>> An alternative (an the best way, I think) is introducing a configuration
>>> property to define which cache to use in case it hasn't be defined in a
>>> message.
>>> 
>>> I'll appreciate any thoughts on that.
>>> 
>>> Regards,
>>> Igor
>>> 
>>> 2017-04-24 12:43 GMT+03:00 Roman Shtykh :
>>> 
 Vladimir,
 Probably we may set the cache name via https://redis.io/commands/
 client-setname, save it and use until the user specifies another name.
 #That will be the name for the default cache (mainly for STRING data).
>>> For
 other data types, like hashes (https://redis.io/topics/data-types), I
>> am
 thinking about putting data into caches specified by key.
 Or use https://redis.io/commands/config-set CONFIG SET DFLT_CACHE
 cache_name,and save cache name somewhere in Ignite cluster (what is the
 proper place to store such info?).
 For that, we have to implement one of the above-mentioned commands.
 What do you think?
 
 
 
    On Monday, April 24, 2017 4:34 PM, Vladimir Ozerov <
 voze...@gridgain.com> wrote:
 
 
  Roman,
 Is it possible to define a kind of property in Redis connection string
>>> (or
 property map) for this purpose? IMO ideally we should "externalize"
>> cache
 name somehow, so that it can be changed with no changes to user's code.
 
 Alex,
 Not sure if this is a good idea as you will end up with PARTITIONED
>> cache
 without backups

[GitHub] ignite pull request #1874: IGNITE-5083: Add default cache for Redis.

2017-04-25 Thread shroman

GitHub user shroman opened a pull request:

https://github.com/apache/ignite/pull/1874

IGNITE-5083: Add default cache for Redis.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shroman/ignite IGNITE-5083

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1874.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1874


commit d9fcc6490272b5cdcf80a9f0c15c606a1096d852
Author: shtykh_roman 
Date:   2017-04-26T05:47:17Z

IGNITE-5083: Add default cache for Redis.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Cluster metrics - review for PageMemory

2017-04-25 Thread Dmitriy Setrakyan

Is there a way to provide such metrics for a specific memory policy?

On Tue, Apr 25, 2017 at 4:53 PM, Alexey Goncharuk <
alexey.goncha...@gmail.com> wrote:

> Igniters,
>
> Since we moved to the PageMemory architecture, several ClusterMetrics
> methods became questionable, so I would like to discuss this before the
> release. Currently, ClusterMetrics contains the following methods:
> getNonHeapMemoryCommitted(),
> getNonHeapMemoryUsed(),
> getNonHeapMemoryInitialized(),
> getNonHeapMemoryTotal(),
> getNonHeapMemoryMaximum()
>
> I suggest we remove Total and Committed metrics, and the rest of the
> methods will have the following semantics:
> Initialized() - start size of all memory policies
> Max - max size of all memory policies
> Used - size of all allocated pages
>
> Thoughts?
>

Re: Improve binary enum handling

2017-04-25 Thread Dmitriy Setrakyan

Vova,

I am not sure I am qualified to propose such a design, but I would like to
fully understand your proposal. Can you please suggest how a user would use
an enum-based query, after having specified an enum in the CREATE TABLE
command, as you suggested?

D.

On Tue, Apr 25, 2017 at 3:17 PM, Vladimir Ozerov 
wrote:

> Dima, Sergi,
>
> Please propose how are you going to work with enums from SQL perspective
> without registering them in advance. I already shared my thoughts and
> provided two examples how this is achieved in MySQL and PostgreSQL.
>
> On Tue, Apr 25, 2017 at 3:06 PM, Dmitriy Setrakyan 
> wrote:
>
> > Vladimir, can you please share your thoughts here?
> >
> > On Mon, Apr 24, 2017 at 5:21 PM, Sergi Vladykin <
> sergi.vlady...@gmail.com>
> > wrote:
> >
> > > I agree with Dmitriy, it is preferable to have this enum registration
> > > optional. It will be a better user experience.
> > >
> > > Why do we "inevitably" need it?
> > >
> > > Sergi
> > >
> > > 2017-04-24 17:02 GMT+03:00 Vladimir Ozerov :
> > >
> > > > Dima,
> > > >
> > > > No. It is normal (and inevitably) practice to register enums before
> > they
> > > > are used.
> > > >
> > > > This is how enum is created in MySQL:
> > > >
> > > > CREATE TABLE shirts (
> > > > name VARCHAR(40),
> > > > size ENUM('x-small', 'small', 'medium', 'large', 'x-large')
> > > > );
> > > >
> > > > And in PostgreSQL:
> > > >
> > > > CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
> > > > CREATE TABLE person (
> > > > name text,
> > > > current_mood mood
> > > > );
> > > >
> > > > We will do the same at some point. That is, in future users will
> > register
> > > > enums from SQL, not from native API or configuration.
> > > >
> > > > Vladimir.
> > > >
> > > > On Mon, Apr 24, 2017 at 4:37 PM, Dmitriy Setrakyan <
> > > dsetrak...@apache.org>
> > > > wrote:
> > > >
> > > > > Vladimir,
> > > > >
> > > > > I would really like to avoid special registration of Enums. Can you
> > > find
> > > > a
> > > > > way to handle it automatically?
> > > > >
> > > > > D.
> > > > >
> > > > > On Mon, Apr 24, 2017 at 6:33 AM, Vladimir Ozerov <
> > voze...@gridgain.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Sorry, looks like I mismanaged tickets in JIRA. In fact, we
> > > implemented
> > > > > H2
> > > > > > part, but Ignite's part is not ready yet and is managed in
> > > IGNITE-4575
> > > > > [1].
> > > > > > Ticket you mentioned was an umbrella.
> > > > > >
> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-4575
> > > > > >
> > > > > > On Mon, Apr 24, 2017 at 4:28 PM, Dmitriy Setrakyan <
> > > > > dsetrak...@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > Vladimir,
> > > > > > >
> > > > > > > I am very confused. I thought we already had resolved this
> issue
> > in
> > > > > this
> > > > > > > ticket:
> > > > > > > https://issues.apache.org/jira/browse/IGNITE-3595
> > > > > > >
> > > > > > > Can you clarify?
> > > > > > >
> > > > > > > D.
> > > > > > >
> > > > > > > On Mon, Apr 24, 2017 at 5:58 AM, Vladimir Ozerov <
> > > > voze...@gridgain.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Igniters,
> > > > > > > >
> > > > > > > > Currently we have limited support of binary enums. The main
> > > problem
> > > > > is
> > > > > > > that
> > > > > > > > we do not store any metadata about enum names. For this
> reason
> > it
> > > > is
> > > > > > > > impossible to use enums in SQL even though H2 already
> supports
> > it
> > > > > [1].
> > > > > > We
> > > > > > > > need to improve enum metadata support and provide some
> > additional
> > > > API
> > > > > > to
> > > > > > > > register new enums in runtime.
> > > > > > > >
> > > > > > > > Proposed API:
> > > > > > > >
> > > > > > > > 1) Enum mappings can be defined statically in
> > > > > BinaryTypeConfiguration:
> > > > > > > >
> > > > > > > > class BinaryTypeConfiguration {
> > > > > > > > boolean isEnum;  // Old method
> > > > > > > > *Map enumValues;* // New method
> > > > > > > > }
> > > > > > > >
> > > > > > > > 2) New enum could be registered through IgniteBinary (e.g. we
> > > will
> > > > > use
> > > > > > it
> > > > > > > > if enum is defined in CREATE TABLE statement). Elso it would
> be
> > > > > > possible
> > > > > > > to
> > > > > > > > build enum using only name.
> > > > > > > >
> > > > > > > > interface IgniteBinary {
> > > > > > > > BinaryObject buildEnum(String typeName, int ordinal);
> > > > > > //
> > > > > > > > Old
> > > > > > > > *BinaryObject buildEnum(String typeName, String name); *
> > > > > > >  //
> > > > > > > > New
> > > > > > > >
> > > > > > > > *BinaryType defineEnum(String typeName, Map > Integer>
> > > > > > vals);*
> > > > > > > //
> > > > > > > > New
> > > > > > > > }
> > > > > > > >
> > > > > > > > 3) BinaryObject will have new method "enumName":
> > > > > > > >
> > > > > > > > interface BinaryObject {
> >

Re: Ignite ML, DSL/Scripting (IGNITE-5065)

2017-04-25 Thread Denis Magda

Folks, 

Isn’t it to early to dig into DSL implementation? I would take this path only 
after Ignite supports basic algorithms used for ML and predictive analysis so 
that people can use it somehow. Then depending on a feedback we might need to 
start on this task.

—
Denis

> On Apr 25, 2017, at 12:38 PM, Sergi Vladykin  wrote:
> 
> I'm a bit out of the loop with ML and Web console, but Java scripting
> engines as in JSR-233 are supported in Java 7. You can use JSR-233 API in
> Web Console, implementation will be in IgniteML module, which requires Java
> 8 anyways. This way they will be decoupled.
> 
> Does this work for you?
> 
> Sergi
> 
> 2017-04-25 21:12 GMT+03:00 Yury Babak :
> 
>> Hi all!
>> 
>> Currently I'm working on adding scripting support for Ignite
>> ML(IGNITE-5065). The basic idea is to provide possibility to create and run
>> some scripts with ML algorithms over Ignite cluster using Ignite ML API and
>> JS/Python as script language.
>> 
>> I’ve exchanged several thoughts with Alexey K., Ignite web console
>> maintainer and according to him it's okey to add this feature to
>> web-console
>> module.
>> 
>> In this case we should add dependency to Ignite ML which require Java8.
>> 
>> On the other hand we could create new UI module for this purpose but it
>> will
>> require additional time for development separate ml web console.
>> 
>> If anyone know addition pros/cons for both ways - please advise.
>> 
>> Thanks,
>> Yury Babak.
>> 
>> 
>> 
>> --
>> View this message in context: http://apache-ignite-
>> developers.2346864.n4.nabble.com/Ignite-ML-DSL-Scripting-
>> IGNITE-5065-tp17216.html
>> Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
>>

Re: Apache Ignite 2.0 Release

2017-04-25 Thread Denis Magda

Alex G., 

Would you complete the review of IGNITE-4539 and merge the changes?

—
Denis

> On Apr 25, 2017, at 6:33 PM, Roman Shtykh  wrote:
> 
> I would also like to include IGNITE-4539. Its implementation is finished.
> Roman 
> 
>On Wednesday, April 26, 2017 1:59 AM, Vladimir Ozerov 
>  wrote:
> 
> 
> Igniters,
> 
> We are almost there. Outstanding issues which I hope we will be able to
> resolve quickly:
> 1) Hibernate 5 support [1]
> 2) Two SQL tickets (2], [3]
> 3) Two page memory tickets [4], [5]
> 4) TcpDiscoverySpi.heartbeatsFrequency [6]
> 5) Prohibit "null" as cache name [7]
> 
> Most of them are in "Patch Available". Let's do our best to merge them in
> the nearest day.
> 
> [1] https://issues.apache.org/jira/browse/IGNITE-1794
> [2] https://issues.apache.org/jira/browse/IGNITE-3481
> [3] https://issues.apache.org/jira/browse/IGNITE-3487
> ]4] https://issues.apache.org/jira/browse/IGNITE-5024
> [5] https://issues.apache.org/jira/browse/IGNITE-5072
> [6] https://issues.apache.org/jira/browse/IGNITE-4799
> [7] https://issues.apache.org/jira/browse/IGNITE-3488
> 
> 
> On Sat, Apr 22, 2017 at 2:25 AM, Denis Magda  wrote:
> 
>> Guys,
>> 
>> How close are we for the voting? What exactly are we waiting for to
>> initiate it (H2 release, TC tests, CREATE INDEX)?
>> 
>> —
>> Denis
>> 
>>> On Apr 19, 2017, at 4:03 PM, Denis Magda  wrote:
>>> 
>>> Roman,
>>> 
>>> If we squeeze your contribution into 2.0 please add a documentation. You
>> can handle the documentation separately as a subtask of this umbrella
>> ticket:
>>> https://issues.apache.org/jira/browse/IGNITE-4960
>>> 
>>> —
>>> Denis
>>> 
 On Apr 19, 2017, at 7:03 AM, Roman Shtykh 
>> wrote:
 
 Alexey,
 Sorry, I forgot to update dependencies. Now it should be fine.
 Roman
 
 
   On Wednesday, April 19, 2017 9:56 PM, Alexey Goncharuk <
>> alexey.goncha...@gmail.com> wrote:
 
 
 Roman,
 I see that you've changed the rocketmq version to 4.0.0-SNAPSHOT, but
>> this does not work because it requires a local copy of rocketmq. If I
>> revert it back to 4.0.0-incubating, then the test does not compile for me.
 2017-04-19 11:55 GMT+03:00 Roman Shtykh :
 
 Hi Alexey,
 Fixed now. Thanks for checking!
 Roman
 
 
 On Tuesday, April 18, 2017 10:17 PM, Alexey Goncharuk <
>> alexey.goncha...@gmail.com> wrote:
 
 
 Hi Roman,
 
 I tried to run tests for RocketMQ streamer, but they failed for me (I've
 added a comment in the ticket). Can you please take a look?
 
 2017-04-18 15:13 GMT+03:00 Vladimir Ozerov :
 
> Folks,
> 
> I created branch for AI 2.0: ignite-2.0. Relevant PR #1819. Let's
>> continue
> 2.0 finalization in this branch.
> 
> On Wed, Apr 12, 2017 at 7:58 AM, Dmitriy Setrakyan <
>> dsetrak...@apache.org>
> wrote:
> 
>> Awesome! Great addition to the project.
>> 
>> On Tue, Apr 11, 2017 at 8:27 PM, Denis Magda 
>> wrote:
>> 
>>> Spring Data integration caught up the train and will be released in
> 2.0:
>>> https://issues.apache.org/ jira/browse/IGNITE-1192 <
>>> https://issues.apache.org/ jira/browse/IGNITE-1192>
>>> 
>>> —
>>> Denis
>>> 
 On Apr 9, 2017, at 6:57 AM, Denis Magda 
>> wrote:
 
 Val, yes, as far as I know we still need to merge to the master. The
>>> branch should be ready later the next week.
 
 Denis
 
 On Sat, Apr 8, 2017 at 6:14 PM, Valentin Kulichenko <
>>> valentin.kuliche...@gmail.com >> wrote:
 Guys,
 
 There is no branch for 2.0 right now, correct? If I want to add
>> something
 there, do I just merge to master?
 
 -Val
 
 On Sat, Apr 8, 2017 at 10:18 AM, Alexander Paschenko <
 alexander.a.paschenko@gmail. com > gmail.com>>
>>> wrote:
 
> I've fixed https://issues.apache.org/ jira/browse/IGNITE-4354 <
>>> https://issues.apache.org/ jira/browse/IGNITE-4354>, PR is
> https://github.com/apache/ ignite/pull/1759 <
>> https://github.com/apache/
>>> ignite/pull/1759>.
> Pavel, thank you very much for bringing that to my attention.
> 
> - Alex
> 
> 2017-04-07 20:28 GMT+03:00 Sergi Vladykin <
> sergi.vlady...@gmail.com
>>> >:
>> A bunch of SQL related tickets is delayed until H2 release on the
>>> next
> week.
>> 
>> Sergi
>> 
>> 2017-04-07 19:36 GMT+03:00 Alexey Goncharuk <
>>> alexey.goncha...@gmail.com

[jira] [Created] (IGNITE-5083) Add default cache for Redis

2017-04-25 Thread Roman Shtykh (JIRA)

Roman Shtykh created IGNITE-5083:


 Summary: Add default cache for Redis
 Key: IGNITE-5083
 URL: https://issues.apache.org/jira/browse/IGNITE-5083
 Project: Ignite
  Issue Type: Task
Reporter: Roman Shtykh
Assignee: Roman Shtykh


Default "redis_cache" for Redis data.
NOTE: the name will have to be "externalized"  in future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Apache Ignite 2.0 Release

2017-04-25 Thread Roman Shtykh

I would also like to include IGNITE-4539. Its implementation is finished.
Roman 

On Wednesday, April 26, 2017 1:59 AM, Vladimir Ozerov 
 wrote:
 

 Igniters,

We are almost there. Outstanding issues which I hope we will be able to
resolve quickly:
1) Hibernate 5 support [1]
2) Two SQL tickets (2], [3]
3) Two page memory tickets [4], [5]
4) TcpDiscoverySpi.heartbeatsFrequency [6]
5) Prohibit "null" as cache name [7]

Most of them are in "Patch Available". Let's do our best to merge them in
the nearest day.

[1] https://issues.apache.org/jira/browse/IGNITE-1794
[2] https://issues.apache.org/jira/browse/IGNITE-3481
[3] https://issues.apache.org/jira/browse/IGNITE-3487
]4] https://issues.apache.org/jira/browse/IGNITE-5024
[5] https://issues.apache.org/jira/browse/IGNITE-5072
[6] https://issues.apache.org/jira/browse/IGNITE-4799
[7] https://issues.apache.org/jira/browse/IGNITE-3488


On Sat, Apr 22, 2017 at 2:25 AM, Denis Magda  wrote:

> Guys,
>
> How close are we for the voting? What exactly are we waiting for to
> initiate it (H2 release, TC tests, CREATE INDEX)?
>
> —
> Denis
>
> > On Apr 19, 2017, at 4:03 PM, Denis Magda  wrote:
> >
> > Roman,
> >
> > If we squeeze your contribution into 2.0 please add a documentation. You
> can handle the documentation separately as a subtask of this umbrella
> ticket:
> > https://issues.apache.org/jira/browse/IGNITE-4960
> >
> > —
> > Denis
> >
> >> On Apr 19, 2017, at 7:03 AM, Roman Shtykh 
> wrote:
> >>
> >> Alexey,
> >> Sorry, I forgot to update dependencies. Now it should be fine.
> >> Roman
> >>
> >>
> >>  On Wednesday, April 19, 2017 9:56 PM, Alexey Goncharuk <
> alexey.goncha...@gmail.com> wrote:
> >>
> >>
> >> Roman,
> >> I see that you've changed the rocketmq version to 4.0.0-SNAPSHOT, but
> this does not work because it requires a local copy of rocketmq. If I
> revert it back to 4.0.0-incubating, then the test does not compile for me.
> >> 2017-04-19 11:55 GMT+03:00 Roman Shtykh :
> >>
> >> Hi Alexey,
> >> Fixed now. Thanks for checking!
> >> Roman
> >>
> >>
> >>    On Tuesday, April 18, 2017 10:17 PM, Alexey Goncharuk <
> alexey.goncha...@gmail.com> wrote:
> >>
> >>
> >> Hi Roman,
> >>
> >> I tried to run tests for RocketMQ streamer, but they failed for me (I've
> >> added a comment in the ticket). Can you please take a look?
> >>
> >> 2017-04-18 15:13 GMT+03:00 Vladimir Ozerov :
> >>
> >>> Folks,
> >>>
> >>> I created branch for AI 2.0: ignite-2.0. Relevant PR #1819. Let's
> continue
> >>> 2.0 finalization in this branch.
> >>>
> >>> On Wed, Apr 12, 2017 at 7:58 AM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> >>> wrote:
> >>>
>  Awesome! Great addition to the project.
> 
>  On Tue, Apr 11, 2017 at 8:27 PM, Denis Magda 
> wrote:
> 
> > Spring Data integration caught up the train and will be released in
> >>> 2.0:
> > https://issues.apache.org/ jira/browse/IGNITE-1192 <
> > https://issues.apache.org/ jira/browse/IGNITE-1192>
> >
> > —
> > Denis
> >
> >> On Apr 9, 2017, at 6:57 AM, Denis Magda 
> wrote:
> >>
> >> Val, yes, as far as I know we still need to merge to the master. The
> > branch should be ready later the next week.
> >>
> >> Denis
> >>
> >> On Sat, Apr 8, 2017 at 6:14 PM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com  >>
> > wrote:
> >> Guys,
> >>
> >> There is no branch for 2.0 right now, correct? If I want to add
>  something
> >> there, do I just merge to master?
> >>
> >> -Val
> >>
> >> On Sat, Apr 8, 2017 at 10:18 AM, Alexander Paschenko <
> >> alexander.a.paschenko@gmail. com   gmail.com>>
> > wrote:
> >>
> >>> I've fixed https://issues.apache.org/ jira/browse/IGNITE-4354 <
> > https://issues.apache.org/ jira/browse/IGNITE-4354>, PR is
> >>> https://github.com/apache/ ignite/pull/1759 <
>  https://github.com/apache/
> > ignite/pull/1759>.
> >>> Pavel, thank you very much for bringing that to my attention.
> >>>
> >>> - Alex
> >>>
> >>> 2017-04-07 20:28 GMT+03:00 Sergi Vladykin <
> >>> sergi.vlady...@gmail.com
> > >:
>  A bunch of SQL related tickets is delayed until H2 release on the
> > next
> >>> week.
> 
>  Sergi
> 
>  2017-04-07 19:36 GMT+03:00 Alexey Goncharuk <
> > alexey.goncha...@gmail.com 
>  :
> 
> > Folks,
> >
> > The status of ignite-3477 is as follows: we've fixed almost all
> > tests
> >>> and
> > currently waiting for ignite-4535 and ignite-4534 to be merged,
> > which
> >>> will
> > happen once TC

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

2017-04-25 Thread Alexei Scherbakov

Nick,

Have you tried implementing delegating classloader and set it using
IgniteConfiguration.setClassLoader() ?

This should solve your problem without any API changes on Ignite's side.

2017-04-25 22:42 GMT+03:00 npordash :

> Hi Denis, Val,
>
> Please refer to my initial inquiry on the user forum for full context as I
> think that was lost while cross-posting to here:
> http://apache-ignite-users.70518.x6.nabble.com/BinaryObjectImpl-
> deserializeValue-with-specific-ClassLoader-td12055.html
>
> The setting in IgniteConfiguration does not help in this use-case because
> Ignite is started before user-code is dynamically loaded and ran. I also
> can't set it after it has been started because Ignite is a shared resource
> in the JVM and there would be multiple code paths trying to access it that
> resolved from different classloaders.
>
> I think the existing behavior is inconsistent and surprising to the user,
> but I do think this was just an oversight as not a lot of folks in general
> try to dynamically load and run code at runtime. Within the same JVM I can
> write objects to a cache for classes that Ignite can't resolve from it's
> own
> classpath (because it doesn't need to for writes), but if I try to read
> them
> back from the cache (even in the same JVM) I get a ClassNotFoundException.
>
> -Nick
>
>
>
> --
> View this message in context: http://apache-ignite-
> developers.2346864.n4.nabble.com/Re-BinaryObjectImpl-
> deserializeValue-with-specific-ClassLoader-tp17126p17221.html
> Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
>



-- 

Best regards,
Alexei Scherbakov

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

2017-04-25 Thread npordash

Hi Denis, Val,

Please refer to my initial inquiry on the user forum for full context as I
think that was lost while cross-posting to here:
http://apache-ignite-users.70518.x6.nabble.com/BinaryObjectImpl-deserializeValue-with-specific-ClassLoader-td12055.html

The setting in IgniteConfiguration does not help in this use-case because
Ignite is started before user-code is dynamically loaded and ran. I also
can't set it after it has been started because Ignite is a shared resource
in the JVM and there would be multiple code paths trying to access it that
resolved from different classloaders.

I think the existing behavior is inconsistent and surprising to the user,
but I do think this was just an oversight as not a lot of folks in general
try to dynamically load and run code at runtime. Within the same JVM I can
write objects to a cache for classes that Ignite can't resolve from it's own
classpath (because it doesn't need to for writes), but if I try to read them
back from the cache (even in the same JVM) I get a ClassNotFoundException.

-Nick



--
View this message in context: 
http://apache-ignite-developers.2346864.n4.nabble.com/Re-BinaryObjectImpl-deserializeValue-with-specific-ClassLoader-tp17126p17221.html
Sent from the Apache Ignite Developers mailing list archive at Nabble.com.

Re: Null cache names

2017-04-25 Thread Sergi Vladykin

Agree, lets move forward with the simplest possible solution for now.

Sergi

2017-04-25 13:07 GMT+03:00 Vladimir Ozerov :

> Folks,
>
> I do not think it is legal to add such property to ConnectorConfiguration.
> Connector is a generic gateway to cluster resources. It should not bother
> about caches anyhow. What if there are multiple caches in the cluster? What
> is I want to access "cache A" from Memcached and "cache B" from Redis
> simultaneously? Etc.. This kind of property should be defined on client
> level, not on the server.
>
> For now, provided that 2.0 is about to be freezed, I propose to stick to
> Dmitriy's approach and use "default" cache name instead of null. This
> should work fine for AI 2.0. We will be able to improve it in further
> releases.
>
> Thoughts?
>
> Vladimir.
>
> On Tue, Apr 25, 2017 at 7:22 AM, Roman Shtykh 
> wrote:
>
> > Igor, +1 from me.We can add a field to ConnectorConfiguration (not sure
> if
> > it's a proper place, but it's shared by REST, memcached and Redis). A
> user
> > will have to create a cache, configure as needed and specify the name in
> > ConnectorConfiguration.
> > Roman
> >
> >
> >
> > On Monday, April 24, 2017 10:34 PM, Seliverstov Igor <
> > gvvinbl...@gmail.com> wrote:
> >
> >
> >  Dear Igniters,
> >
> > Seems we have almost the same issue with Memcached protocol.
> >
> > There is an ability to define a cache name via operation extras message
> > part (
> > https://github.com/memcached/memcached/wiki/
> BinaryProtocolRevamped#packet-
> > structure)
> > but it looks a bit complicated from my point of view...
> >
> > Different client implementations might provide such functionality or not
> (I
> > mean an additional info in an operation extras), so, potential user might
> > have some difficultes using Ignite as a Memcached server because of this
> > ignite-specific message part becomes mandatory.
> >
> > An alternative (an the best way, I think) is introducing a configuration
> > property to define which cache to use in case it hasn't be defined in a
> > message.
> >
> > I'll appreciate any thoughts on that.
> >
> > Regards,
> > Igor
> >
> > 2017-04-24 12:43 GMT+03:00 Roman Shtykh :
> >
> > > Vladimir,
> > > Probably we may set the cache name via https://redis.io/commands/
> > > client-setname, save it and use until the user specifies another name.
> > > #That will be the name for the default cache (mainly for STRING data).
> > For
> > > other data types, like hashes (https://redis.io/topics/data-types), I
> am
> > > thinking about putting data into caches specified by key.
> > > Or use https://redis.io/commands/config-set CONFIG SET DFLT_CACHE
> > > cache_name,and save cache name somewhere in Ignite cluster (what is the
> > > proper place to store such info?).
> > > For that, we have to implement one of the above-mentioned commands.
> > > What do you think?
> > >
> > >
> > >
> > >On Monday, April 24, 2017 4:34 PM, Vladimir Ozerov <
> > > voze...@gridgain.com> wrote:
> > >
> > >
> > >  Roman,
> > > Is it possible to define a kind of property in Redis connection string
> > (or
> > > property map) for this purpose? IMO ideally we should "externalize"
> cache
> > > name somehow, so that it can be changed with no changes to user's code.
> > >
> > > Alex,
> > > Not sure if this is a good idea as you will end up with PARTITIONED
> cache
> > > without backups with no ability to change that.
> > >
> > > On Mon, Apr 24, 2017 at 9:35 AM, Alexey Kuznetsov <
> akuznet...@apache.org
> > >
> > > wrote:
> > >
> > > > Roman,
> > > >
> > > > Just as idea, how about in case of user does not configured
> > "REDIS_CACHE"
> > > >  then create it via ignite.getOrCreateCache(new
> > > > CacheConfiguration("REDIS_CACHE"))
> > > > and prin warning to log "REDIS_CACHE not configured, using default
> > > > partitioned cache".
> > > >
> > > > What do you think?
> > > >
> > > > On Mon, Apr 24, 2017 at 10:26 AM, Roman Shtykh
> >  > > >
> > > > wrote:
> > > >
> > > > > Denis, Igor,
> > > > > What can be done now
> > > > > 1. create a default cache name for Redis data, e.g. "REDIS_CACHE"
> > that
> > > > has
> > > > > to be configured explicitly in xml file (as it is done with other
> > > caches)
> > > > > by a user if he/she needs Redis protocol.
> > > > > 2. Force users to specify cache names as prefix to their keys, so
> > that
> > > we
> > > > > can parse and switch between caches.
> > > > > The 1st one is a very quick fix I can do today. This can be
> extended
> > in
> > > > > future to have a separate cache for each data type.
> > > > > Roman
> > > > >
> > > > >
> > > > >On Monday, April 24, 2017 12:15 AM, Denis Magda <
> > > dma...@gridgain.com
> > > > >
> > > > > wrote:
> > > > >
> > > > >
> > > > >  Roman, would you suggest a quick solution for the redis
> integration
> > or
> > > > > even
> > > > > implement it in the nearest couple of days? We need to change

Re: Cluster metrics - review for PageMemory

2017-04-25 Thread Sergi Vladykin

Looks good to me.

Sergi

2017-04-25 16:53 GMT+03:00 Alexey Goncharuk :

> Igniters,
>
> Since we moved to the PageMemory architecture, several ClusterMetrics
> methods became questionable, so I would like to discuss this before the
> release. Currently, ClusterMetrics contains the following methods:
> getNonHeapMemoryCommitted(),
> getNonHeapMemoryUsed(),
> getNonHeapMemoryInitialized(),
> getNonHeapMemoryTotal(),
> getNonHeapMemoryMaximum()
>
> I suggest we remove Total and Committed metrics, and the rest of the
> methods will have the following semantics:
> Initialized() - start size of all memory policies
> Max - max size of all memory policies
> Used - size of all allocated pages
>
> Thoughts?
>

Re: Ignite ML, next steps (IGNITE-5029)

2017-04-25 Thread Sergi Vladykin

It is preferable to avoid hard bindings to some exact scripting engines. If
user wants to plug in Groovy we should allow it.

As for DSL I believe it is a waste of time. Few years ago it was somewhat
popular idea to create DSLs for everything, but no one actually wants to
learn new quirky languages, so it it never worked out.

All in all the best choice is to have a good API that can be conveniently
used from any scripting language + ability to plug in any scripting engine
user likes.

Sergi

2017-04-25 20:31 GMT+03:00 Yury Babak :

> First of all thanks for this advice.
>
> And DSL/Scripting update:
>
> Actually it's a two separate features. The first is provide scripting from
> some web ui. I think we could use web-console as ui part and JSR 223 for
> scripting itself, Nashorn for JS and Jython for Python.
>
> And the second feature - DSL.
>
> For those features I've created IGNITE-5065.
>
> Thanks,
> Yury.
>
>
>
> --
> View this message in context: http://apache-ignite-
> developers.2346864.n4.nabble.com/Ignite-ML-next-steps-
> IGNITE-5029-tp17096p17211.html
> Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
>

Re: Ignite ML, DSL/Scripting (IGNITE-5065)

2017-04-25 Thread Sergi Vladykin

I'm a bit out of the loop with ML and Web console, but Java scripting
engines as in JSR-233 are supported in Java 7. You can use JSR-233 API in
Web Console, implementation will be in IgniteML module, which requires Java
8 anyways. This way they will be decoupled.

Does this work for you?

Sergi

2017-04-25 21:12 GMT+03:00 Yury Babak :

> Hi all!
>
> Currently I'm working on adding scripting support for Ignite
> ML(IGNITE-5065). The basic idea is to provide possibility to create and run
> some scripts with ML algorithms over Ignite cluster using Ignite ML API and
> JS/Python as script language.
>
> I’ve exchanged several thoughts with Alexey K., Ignite web console
> maintainer and according to him it's okey to add this feature to
> web-console
> module.
>
> In this case we should add dependency to Ignite ML which require Java8.
>
> On the other hand we could create new UI module for this purpose but it
> will
> require additional time for development separate ml web console.
>
> If anyone know addition pros/cons for both ways - please advise.
>
> Thanks,
> Yury Babak.
>
>
>
> --
> View this message in context: http://apache-ignite-
> developers.2346864.n4.nabble.com/Ignite-ML-DSL-Scripting-
> IGNITE-5065-tp17216.html
> Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
>

Ignite ML, DSL/Scripting (IGNITE-5065)

2017-04-25 Thread Yury Babak

Hi all!

Currently I'm working on adding scripting support for Ignite
ML(IGNITE-5065). The basic idea is to provide possibility to create and run
some scripts with ML algorithms over Ignite cluster using Ignite ML API and
JS/Python as script language.

I’ve exchanged several thoughts with Alexey K., Ignite web console
maintainer and according to him it's okey to add this feature to web-console
module. 

In this case we should add dependency to Ignite ML which require Java8. 

On the other hand we could create new UI module for this purpose but it will
require additional time for development separate ml web console.

If anyone know addition pros/cons for both ways - please advise.

Thanks,
Yury Babak.



--
View this message in context: 
http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-ML-DSL-Scripting-IGNITE-5065-tp17216.html
Sent from the Apache Ignite Developers mailing list archive at Nabble.com.

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

2017-04-25 Thread Denis Magda

It’s a good point. Thanks for reminding Val.

Nick, please check that the suggested method works for your use case.

—
Denis

> On Apr 25, 2017, at 11:16 AM, Valentin Kulichenko 
>  wrote:
> 
> We allow to provide custom class loader via
> IgniteConfiguration.setClassLoader. If this class loader is ignored during
> deserialization of cache objects, then I believe it's a bug.
> 
> -Val
> 
> On Tue, Apr 25, 2017 at 7:36 PM, Denis Magda  wrote:
> 
>> Nick,
>> 
>> This deserialization issue is related to cache objects. You will get the
>> same kind of exception if you try to deserialize a cache entry inside of a
>> compute task which class was preloaded using the peer-class-loading feature.
>> 
>> Frankly, this is not treated as an issue on Ignite side. We designed cache
>> objects serialization/deserialization in a way it works now - the class has
>> to be in the local classpath.
>> 
>> However, probably it makes sense to rethink this. *Val*, *Vovan*, what are
>> your thoughts on this?
>> 
>> —
>> Denis
>> 
>>> On Apr 24, 2017, at 6:06 PM, npordash  wrote:
>>> 
>>> Hi Denis,
>>> 
> if you want to deserialize an entry then most likely you’re doing this
>> on
> the app side that already has the class in the class path
>>> 
>>> In this particular case the app is a deployed service and the hosting
>> node
>>> doesn't have the class files on its classpath. I implemented a way to
>> deploy
>>> and run services on the grid even if the class files are not known by
>> ignite
>>> (which is current service grid limitation). It works fine except for this
>>> deserialization issue.
>>> 
>>> -Nick
>>> 
>>> 
>>> 
>>> --
>>> View this message in context: http://apache-ignite-
>> developers.2346864.n4.nabble.com/Re-BinaryObjectImpl-
>> deserializeValue-with-specific-ClassLoader-tp17126p17173.html
>>> Sent from the Apache Ignite Developers mailing list archive at
>> Nabble.com.
>> 
>>

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

2017-04-25 Thread Valentin Kulichenko

We allow to provide custom class loader via
IgniteConfiguration.setClassLoader. If this class loader is ignored during
deserialization of cache objects, then I believe it's a bug.

-Val

On Tue, Apr 25, 2017 at 7:36 PM, Denis Magda  wrote:

> Nick,
>
> This deserialization issue is related to cache objects. You will get the
> same kind of exception if you try to deserialize a cache entry inside of a
> compute task which class was preloaded using the peer-class-loading feature.
>
> Frankly, this is not treated as an issue on Ignite side. We designed cache
> objects serialization/deserialization in a way it works now - the class has
> to be in the local classpath.
>
> However, probably it makes sense to rethink this. *Val*, *Vovan*, what are
> your thoughts on this?
>
> —
> Denis
>
> > On Apr 24, 2017, at 6:06 PM, npordash  wrote:
> >
> > Hi Denis,
> >
> >>> if you want to deserialize an entry then most likely you’re doing this
> on
> >>> the app side that already has the class in the class path
> >
> > In this particular case the app is a deployed service and the hosting
> node
> > doesn't have the class files on its classpath. I implemented a way to
> deploy
> > and run services on the grid even if the class files are not known by
> ignite
> > (which is current service grid limitation). It works fine except for this
> > deserialization issue.
> >
> > -Nick
> >
> >
> >
> > --
> > View this message in context: http://apache-ignite-
> developers.2346864.n4.nabble.com/Re-BinaryObjectImpl-
> deserializeValue-with-specific-ClassLoader-tp17126p17173.html
> > Sent from the Apache Ignite Developers mailing list archive at
> Nabble.com.
>
>

Re: Cluster metrics - review for PageMemory

2017-04-25 Thread Denis Magda

Personally, Total and Committed metrics look confusing to me. Moreover, looks 
like they interfere with the rest of the metrics this or that way.

So, +1 for the changes proposed by Alex G.

—
Denis

> On Apr 25, 2017, at 6:53 AM, Alexey Goncharuk  
> wrote:
> 
> Igniters,
> 
> Since we moved to the PageMemory architecture, several ClusterMetrics
> methods became questionable, so I would like to discuss this before the
> release. Currently, ClusterMetrics contains the following methods:
> getNonHeapMemoryCommitted(),
> getNonHeapMemoryUsed(),
> getNonHeapMemoryInitialized(),
> getNonHeapMemoryTotal(),
> getNonHeapMemoryMaximum()
> 
> I suggest we remove Total and Committed metrics, and the rest of the
> methods will have the following semantics:
> Initialized() - start size of all memory policies
> Max - max size of all memory policies
> Used - size of all allocated pages
> 
> Thoughts?

Re: MemoryMetrics interface inconsistencies

2017-04-25 Thread Denis Magda

Totally agree with all Pavel’s and Vovan's suggestions and concerns, 
especially, with the following:

> 2) MemoryMetrics has methods that modify state, like enableMetrics
> and rateTimeInterval
> (IgniteCache.metrics() returns a read-only serializable snapshot)
> 
> 3) getSize method returns size in megabytes
> (IgniteCache.metrics() provides sizes in bytes)

In general, it’s unclear how to use it and the javadoc doesn’t explain whether 
these are the metrics for a pool/region/block defined by memory policy or the 
whole page memory available to a node will be monitored. Please mention all 
that in the javadoc and don’t refer to internal classes like PageMemory and 
FreeList the documentation. 

—
Denis

> On Apr 25, 2017, at 12:23 AM, Pavel Tupitsyn  wrote:
> 
> Can you give an example of such API usage?
> A piece of code to enable metrics and retrieve a snapshot?
> 
> On Mon, Apr 24, 2017 at 8:06 PM, Alexey Goncharuk <
> alexey.goncha...@gmail.com> wrote:
> 
>> Agree, I overlooked this during the review. However, the changes to fix
>> this is pretty simple - we just move all mutator methods to MBean, like
>> Vladimir suggested and make MBean return the live data, while the public
>> API will return a serializable snapshot.
>> 
>> Agreed?
>> 
>> 2017-04-24 19:33 GMT+03:00 Vladimir Ozerov :
>> 
>>> It seems that all mutator methods should be simply moved to MBean
>> interface
>>> and change MBytes to bytes. In this case metrics interface will be
>>> consistent.
>>> 
>>> On Mon, Apr 24, 2017 at 7:29 PM, Vladimir Ozerov 
>>> wrote:
>>> 
 Sergey,
 
 We need to keep API consistent. If we usually return bytes, then these
 metrics should return bytes as well. What is more important, when I
>> look
>>> at
 API I understand that this thing is not metrics at all. Metrics in
>> Ignte
 terms is a set of read-only numeric properties. But this interface
>>> contains
 strange things like "name", "swapPilePath". What even more strange, I
>> do
 not see how to get these metrics from public API.
 
 All in all, looks like this entity is not "metrics", but "MBean" in
>> usual
 Ignite terms.
 
 Vladimir.
 
 On Mon, Apr 24, 2017 at 7:20 PM, Pavel Tupitsyn 
 wrote:
 
> Sergey, I disagree.
> 
> 1) As a user, I would expect MemoryMetrics instance to be
> read-only and serializable, so I can send it somewhere, store,
> put into a collection and draw a graph in UI, etc.
> 
> Other metrics APIs allow this, MemoryMetrics does not.
> 
> 2) Methods like enableMetrics and rateTimeInterval placed in
>>> MemoryMetrics
> break single responsibility principle and are confusing.
> Why do I need to Get metrics to Enable them?
> 
> These methods should be placed somewhere else.
> 
> I would suggest some thing like this:
> - introduce Memory class with getMetrics, enableMetrics,
> setRateTimeInterval, etc
> - add Ignite.getMemory method
> 
> 
> On Mon, Apr 24, 2017 at 6:53 PM, Sergey Chugunov <
> sergey.chugu...@gmail.com>
> wrote:
> 
>> I guess the main reason to have IgniteCache returning snapshot
>> metrics
> was
>> to collect metrics about distributed cache across the cluster.
>> At the same time MemoryMetrics were initially designed to be local
>> on
> each
>> node, there were no requirements about collecting cluster-wide
>> MemoryMetrics.
>> 
>> Collecting MemoryMetrics is considered as an investigation action
>> when
>> something goes wrong, that's why it was decided to add
>> enable/disable
>> actions to the interface.
>> So for me it sounds reasonable.
>> 
>> The only debatable thing is having MemoryMetrics returning size in
>> megabytes, however I think about these metrics as designed only for
> user,
>> so I think it makes sense to return this metric in human-readable
>>> form.
>> 
>> On Mon, Apr 24, 2017 at 6:41 PM, Vladimir Ozerov <
>>> voze...@gridgain.com>
>> wrote:
>> 
>>> Agree, looks inconsistent to me.
>>> 
>>> Alex G., could you chime in?
>>> 
>>> On Mon, Apr 24, 2017 at 6:30 PM, Pavel Tupitsyn <
>>> ptupit...@apache.org
>> 
>>> wrote:
>>> 
 Igniters,
 
 I have noticed quite a lot of inconsistencies with the rest of
>> our
> APIs
>>> in
 our new MemoryMetrics API:
 
 1) MemoryMetrics is not a snapshot. It is a "live" instance
>> which
>> returns
 different values each time you access some property.
>>> (IgniteCache.metrics()
 returns a snapshot).
 
 2) MemoryMetrics has methods that modify state, like
>> enableMetrics
 and rateTimeInterval
 (IgniteCache.metrics() returns a read-only serializable
>> snapshot)
 
 3) getSize method returns

Re: Ignite ML, next steps (IGNITE-5029)

2017-04-25 Thread Yury Babak

First of all thanks for this advice.

And DSL/Scripting update:

Actually it's a two separate features. The first is provide scripting from
some web ui. I think we could use web-console as ui part and JSR 223 for
scripting itself, Nashorn for JS and Jython for Python.

And the second feature - DSL.

For those features I've created IGNITE-5065.

Thanks,
Yury.



--
View this message in context: 
http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-ML-next-steps-IGNITE-5029-tp17096p17211.html
Sent from the Apache Ignite Developers mailing list archive at Nabble.com.

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

2017-04-25 Thread Denis Magda

Nick,

This deserialization issue is related to cache objects. You will get the same 
kind of exception if you try to deserialize a cache entry inside of a compute 
task which class was preloaded using the peer-class-loading feature.

Frankly, this is not treated as an issue on Ignite side. We designed cache 
objects serialization/deserialization in a way it works now - the class has to 
be in the local classpath.

However, probably it makes sense to rethink this. *Val*, *Vovan*, what are your 
thoughts on this?

—
Denis

> On Apr 24, 2017, at 6:06 PM, npordash  wrote:
> 
> Hi Denis,
> 
>>> if you want to deserialize an entry then most likely you’re doing this on
>>> the app side that already has the class in the class path
> 
> In this particular case the app is a deployed service and the hosting node
> doesn't have the class files on its classpath. I implemented a way to deploy
> and run services on the grid even if the class files are not known by ignite
> (which is current service grid limitation). It works fine except for this
> deserialization issue.
> 
> -Nick
> 
> 
> 
> --
> View this message in context: 
> http://apache-ignite-developers.2346864.n4.nabble.com/Re-BinaryObjectImpl-deserializeValue-with-specific-ClassLoader-tp17126p17173.html
> Sent from the Apache Ignite Developers mailing list archive at Nabble.com.

Re: Apache Ignite 2.0 Release

2017-04-25 Thread Vladimir Ozerov

Igniters,

We are almost there. Outstanding issues which I hope we will be able to
resolve quickly:
1) Hibernate 5 support [1]
2) Two SQL tickets (2], [3]
3) Two page memory tickets [4], [5]
4) TcpDiscoverySpi.heartbeatsFrequency [6]
5) Prohibit "null" as cache name [7]

Most of them are in "Patch Available". Let's do our best to merge them in
the nearest day.

[1] https://issues.apache.org/jira/browse/IGNITE-1794
[2] https://issues.apache.org/jira/browse/IGNITE-3481
[3] https://issues.apache.org/jira/browse/IGNITE-3487
]4] https://issues.apache.org/jira/browse/IGNITE-5024
[5] https://issues.apache.org/jira/browse/IGNITE-5072
[6] https://issues.apache.org/jira/browse/IGNITE-4799
[7] https://issues.apache.org/jira/browse/IGNITE-3488


On Sat, Apr 22, 2017 at 2:25 AM, Denis Magda  wrote:

> Guys,
>
> How close are we for the voting? What exactly are we waiting for to
> initiate it (H2 release, TC tests, CREATE INDEX)?
>
> —
> Denis
>
> > On Apr 19, 2017, at 4:03 PM, Denis Magda  wrote:
> >
> > Roman,
> >
> > If we squeeze your contribution into 2.0 please add a documentation. You
> can handle the documentation separately as a subtask of this umbrella
> ticket:
> > https://issues.apache.org/jira/browse/IGNITE-4960
> >
> > —
> > Denis
> >
> >> On Apr 19, 2017, at 7:03 AM, Roman Shtykh 
> wrote:
> >>
> >> Alexey,
> >> Sorry, I forgot to update dependencies. Now it should be fine.
> >> Roman
> >>
> >>
> >>   On Wednesday, April 19, 2017 9:56 PM, Alexey Goncharuk <
> alexey.goncha...@gmail.com> wrote:
> >>
> >>
> >> Roman,
> >> I see that you've changed the rocketmq version to 4.0.0-SNAPSHOT, but
> this does not work because it requires a local copy of rocketmq. If I
> revert it back to 4.0.0-incubating, then the test does not compile for me.
> >> 2017-04-19 11:55 GMT+03:00 Roman Shtykh :
> >>
> >> Hi Alexey,
> >> Fixed now. Thanks for checking!
> >> Roman
> >>
> >>
> >>On Tuesday, April 18, 2017 10:17 PM, Alexey Goncharuk <
> alexey.goncha...@gmail.com> wrote:
> >>
> >>
> >> Hi Roman,
> >>
> >> I tried to run tests for RocketMQ streamer, but they failed for me (I've
> >> added a comment in the ticket). Can you please take a look?
> >>
> >> 2017-04-18 15:13 GMT+03:00 Vladimir Ozerov :
> >>
> >>> Folks,
> >>>
> >>> I created branch for AI 2.0: ignite-2.0. Relevant PR #1819. Let's
> continue
> >>> 2.0 finalization in this branch.
> >>>
> >>> On Wed, Apr 12, 2017 at 7:58 AM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> >>> wrote:
> >>>
>  Awesome! Great addition to the project.
> 
>  On Tue, Apr 11, 2017 at 8:27 PM, Denis Magda 
> wrote:
> 
> > Spring Data integration caught up the train and will be released in
> >>> 2.0:
> > https://issues.apache.org/ jira/browse/IGNITE-1192 <
> > https://issues.apache.org/ jira/browse/IGNITE-1192>
> >
> > —
> > Denis
> >
> >> On Apr 9, 2017, at 6:57 AM, Denis Magda 
> wrote:
> >>
> >> Val, yes, as far as I know we still need to merge to the master. The
> > branch should be ready later the next week.
> >>
> >> Denis
> >>
> >> On Sat, Apr 8, 2017 at 6:14 PM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com  >>
> > wrote:
> >> Guys,
> >>
> >> There is no branch for 2.0 right now, correct? If I want to add
>  something
> >> there, do I just merge to master?
> >>
> >> -Val
> >>
> >> On Sat, Apr 8, 2017 at 10:18 AM, Alexander Paschenko <
> >> alexander.a.paschenko@gmail. com   gmail.com>>
> > wrote:
> >>
> >>> I've fixed https://issues.apache.org/ jira/browse/IGNITE-4354 <
> > https://issues.apache.org/ jira/browse/IGNITE-4354>, PR is
> >>> https://github.com/apache/ ignite/pull/1759 <
>  https://github.com/apache/
> > ignite/pull/1759>.
> >>> Pavel, thank you very much for bringing that to my attention.
> >>>
> >>> - Alex
> >>>
> >>> 2017-04-07 20:28 GMT+03:00 Sergi Vladykin <
> >>> sergi.vlady...@gmail.com
> > >:
>  A bunch of SQL related tickets is delayed until H2 release on the
> > next
> >>> week.
> 
>  Sergi
> 
>  2017-04-07 19:36 GMT+03:00 Alexey Goncharuk <
> > alexey.goncha...@gmail.com 
>  :
> 
> > Folks,
> >
> > The status of ignite-3477 is as follows: we've fixed almost all
> > tests
> >>> and
> > currently waiting for ignite-4535 and ignite-4534 to be merged,
> > which
> >>> will
> > happen once TC for those branches is completed. This should
> >>> happen
> > over
> >>> the
> > weekend.
> >
> > 2017-04-06 19:12 GMT+03:00 Alexey Kuznetsov <

[GitHub] ignite pull request #1873: logging was improved

2017-04-25 Thread sergey-chugunov-1985

GitHub user sergey-chugunov-1985 opened a pull request:

https://github.com/apache/ignite/pull/1873

logging was improved



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5079

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1873.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1873


commit 0d2439d06cd64c56ca1c2e1aa21014a2345e386f
Author: Sergey Chugunov 
Date:   2017-04-25T15:26:14Z

IGNITE-5079 logging was improved




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-5081) SecurityPermissionSetBuilder duplicates permission if it's appended more than once

2017-04-25 Thread Valentin Kulichenko (JIRA)

Valentin Kulichenko created IGNITE-5081:
---

 Summary: SecurityPermissionSetBuilder duplicates permission if 
it's appended more than once
 Key: IGNITE-5081
 URL: https://issues.apache.org/jira/browse/IGNITE-5081
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 1.9
Reporter: Valentin Kulichenko
Assignee: Valentin Kulichenko
 Fix For: 2.1


If the same permission is appended to builder more than once, builder 
duplicates it in the resulting collection. This doesn't actually break 
anything, but seems to be redundant.

Example:
{code}
new SecurityPermissionSetBuilder()
.appendSystemPermissions(ADMIN_VIEW)
.appendSystemPermissions(ADMIN_VIEW, ADMIN_QUERY)
.build();
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5080) SecurityBasicPermissionSet implementation is incomplete

2017-04-25 Thread Valentin Kulichenko (JIRA)

Valentin Kulichenko created IGNITE-5080:
---

 Summary: SecurityBasicPermissionSet implementation is incomplete
 Key: IGNITE-5080
 URL: https://issues.apache.org/jira/browse/IGNITE-5080
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 1.9
Reporter: Valentin Kulichenko
Assignee: Valentin Kulichenko
 Fix For: 2.1


There are several issues with {{SecurityBasicPermissionSet}}:
* It doesn't implement {{hashCode}} and {{equals}} methods. This makes it 
impossible to use it as part of validation token.
* {{Collection}} fields are not marked with {{@GridToStringInclude}} 
annotation, so {{toString}} method doesn't actually print out all the 
information.
* {{systemPermissions}} method returns empty collection instead of {{null}} by 
default. This actually means 'no system permissions' even if 
{{defaultAllowAll}} is true.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5079) Improve debug logging for binary metadata exchange

2017-04-25 Thread Sergey Chugunov (JIRA)

Sergey Chugunov created IGNITE-5079:
---

 Summary: Improve debug logging for binary metadata exchange
 Key: IGNITE-5079
 URL: https://issues.apache.org/jira/browse/IGNITE-5079
 Project: Ignite
  Issue Type: Improvement
  Components: binary
Reporter: Sergey Chugunov
Assignee: Sergey Chugunov
Priority: Minor
 Fix For: 2.1






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Handling of @AffinityKeyMapped and @QuerySqlField annotations in Cassandra store

2017-04-25 Thread Valentin Kulichenko

I agree. Using @QuerySqlField in Cassandra store seems to be incorrect
design decision in the first place.

-Val

On Tue, Apr 25, 2017 at 2:22 PM, Vladimir Ozerov 
wrote:

> Hi Igor,
>
> During API stabilization and improvement for Apache Ignite 2.0 we
> restricted usage of @AffinityKeyMapped and @QuerySqlField annotations to
> fields only, because annotations on method level are not supported by
> BinaryMarshaller (which is default) and goes against our general approach
> of having no user classes on server.
>
> This affected Cassandra store as it relied on these annotations in several
> places. I propose the following plan:
> 1) Remove handling of these annotations from Cassandra module in 2.0 as it
> no longer work anyway.
> 2) Define new Cassandra-specific annotations and return this logic in AI
> 2.1.
>
> The main idea is that both mentioned annotations were created for different
> purpose, and their usage in Cassandra module appears to be wrong. We need
> to have different annotations for this.
>
> Thoughts?
>
> Vladimir.
>

[GitHub] ignite pull request #1872: MemoryMetrics interface cleanup, MemoryMetricsImp...

2017-04-25 Thread sergey-chugunov-1985

GitHub user sergey-chugunov-1985 opened a pull request:

https://github.com/apache/ignite/pull/1872

MemoryMetrics interface cleanup, MemoryMetricsImpl split



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4536-later-fixes

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1872.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1872


commit 00b7c51bc1f94d8a98709d61f48c19c7a4bc060f
Author: Sergey Chugunov 
Date:   2017-04-25T09:53:25Z

IGNITE-4536 one more obsolete cache metric was removed

commit 4ec1b65dda3e741abc09bfd342f958ce75b17a1a
Author: Sergey Chugunov 
Date:   2017-04-25T13:04:53Z

IGNITE-4536 MemoryMetricsImpl class was split, JMX part was extracted to a 
separate class




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1871: ignite-5041 NPE in deadlock detection fixed

2017-04-25 Thread agura

GitHub user agura opened a pull request:

https://github.com/apache/ignite/pull/1871

ignite-5041 NPE in deadlock detection fixed



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/agura/incubator-ignite ignite-5041

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1871.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1871


commit c2b4a42d2fc3aaa7f668ebeea8d4fe43bce71f15
Author: agura 
Date:   2017-04-20T17:45:58Z

ignite-5041 NPE in deadlock detection fixed




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1800: Obsolete cache metrics removed, test fixed

2017-04-25 Thread sergey-chugunov-1985

Github user sergey-chugunov-1985 closed the pull request at:

https://github.com/apache/ignite/pull/1800


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Handling of @AffinityKeyMapped and @QuerySqlField annotations in Cassandra store

2017-04-25 Thread Vladimir Ozerov

Hi Igor,

During API stabilization and improvement for Apache Ignite 2.0 we
restricted usage of @AffinityKeyMapped and @QuerySqlField annotations to
fields only, because annotations on method level are not supported by
BinaryMarshaller (which is default) and goes against our general approach
of having no user classes on server.

This affected Cassandra store as it relied on these annotations in several
places. I propose the following plan:
1) Remove handling of these annotations from Cassandra module in 2.0 as it
no longer work anyway.
2) Define new Cassandra-specific annotations and return this logic in AI
2.1.

The main idea is that both mentioned annotations were created for different
purpose, and their usage in Cassandra module appears to be wrong. We need
to have different annotations for this.

Thoughts?

Vladimir.

Re: Improve binary enum handling

2017-04-25 Thread Vladimir Ozerov

Dima, Sergi,

Please propose how are you going to work with enums from SQL perspective
without registering them in advance. I already shared my thoughts and
provided two examples how this is achieved in MySQL and PostgreSQL.

On Tue, Apr 25, 2017 at 3:06 PM, Dmitriy Setrakyan 
wrote:

> Vladimir, can you please share your thoughts here?
>
> On Mon, Apr 24, 2017 at 5:21 PM, Sergi Vladykin 
> wrote:
>
> > I agree with Dmitriy, it is preferable to have this enum registration
> > optional. It will be a better user experience.
> >
> > Why do we "inevitably" need it?
> >
> > Sergi
> >
> > 2017-04-24 17:02 GMT+03:00 Vladimir Ozerov :
> >
> > > Dima,
> > >
> > > No. It is normal (and inevitably) practice to register enums before
> they
> > > are used.
> > >
> > > This is how enum is created in MySQL:
> > >
> > > CREATE TABLE shirts (
> > > name VARCHAR(40),
> > > size ENUM('x-small', 'small', 'medium', 'large', 'x-large')
> > > );
> > >
> > > And in PostgreSQL:
> > >
> > > CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
> > > CREATE TABLE person (
> > > name text,
> > > current_mood mood
> > > );
> > >
> > > We will do the same at some point. That is, in future users will
> register
> > > enums from SQL, not from native API or configuration.
> > >
> > > Vladimir.
> > >
> > > On Mon, Apr 24, 2017 at 4:37 PM, Dmitriy Setrakyan <
> > dsetrak...@apache.org>
> > > wrote:
> > >
> > > > Vladimir,
> > > >
> > > > I would really like to avoid special registration of Enums. Can you
> > find
> > > a
> > > > way to handle it automatically?
> > > >
> > > > D.
> > > >
> > > > On Mon, Apr 24, 2017 at 6:33 AM, Vladimir Ozerov <
> voze...@gridgain.com
> > >
> > > > wrote:
> > > >
> > > > > Sorry, looks like I mismanaged tickets in JIRA. In fact, we
> > implemented
> > > > H2
> > > > > part, but Ignite's part is not ready yet and is managed in
> > IGNITE-4575
> > > > [1].
> > > > > Ticket you mentioned was an umbrella.
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-4575
> > > > >
> > > > > On Mon, Apr 24, 2017 at 4:28 PM, Dmitriy Setrakyan <
> > > > dsetrak...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Vladimir,
> > > > > >
> > > > > > I am very confused. I thought we already had resolved this issue
> in
> > > > this
> > > > > > ticket:
> > > > > > https://issues.apache.org/jira/browse/IGNITE-3595
> > > > > >
> > > > > > Can you clarify?
> > > > > >
> > > > > > D.
> > > > > >
> > > > > > On Mon, Apr 24, 2017 at 5:58 AM, Vladimir Ozerov <
> > > voze...@gridgain.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > Currently we have limited support of binary enums. The main
> > problem
> > > > is
> > > > > > that
> > > > > > > we do not store any metadata about enum names. For this reason
> it
> > > is
> > > > > > > impossible to use enums in SQL even though H2 already supports
> it
> > > > [1].
> > > > > We
> > > > > > > need to improve enum metadata support and provide some
> additional
> > > API
> > > > > to
> > > > > > > register new enums in runtime.
> > > > > > >
> > > > > > > Proposed API:
> > > > > > >
> > > > > > > 1) Enum mappings can be defined statically in
> > > > BinaryTypeConfiguration:
> > > > > > >
> > > > > > > class BinaryTypeConfiguration {
> > > > > > > boolean isEnum;  // Old method
> > > > > > > *Map enumValues;* // New method
> > > > > > > }
> > > > > > >
> > > > > > > 2) New enum could be registered through IgniteBinary (e.g. we
> > will
> > > > use
> > > > > it
> > > > > > > if enum is defined in CREATE TABLE statement). Elso it would be
> > > > > possible
> > > > > > to
> > > > > > > build enum using only name.
> > > > > > >
> > > > > > > interface IgniteBinary {
> > > > > > > BinaryObject buildEnum(String typeName, int ordinal);
> > > > > //
> > > > > > > Old
> > > > > > > *BinaryObject buildEnum(String typeName, String name); *
> > > > > >  //
> > > > > > > New
> > > > > > >
> > > > > > > *BinaryType defineEnum(String typeName, Map Integer>
> > > > > vals);*
> > > > > > //
> > > > > > > New
> > > > > > > }
> > > > > > >
> > > > > > > 3) BinaryObject will have new method "enumName":
> > > > > > >
> > > > > > > interface BinaryObject {
> > > > > > > enumOrdinal(); // Old
> > > > > > > *String enumName();* // New
> > > > > > > }
> > > > > > >
> > > > > > > 4) It would be possible to get the list of known values from
> > > > > BinaryType:
> > > > > > >
> > > > > > > interface BinaryType {
> > > > > > > boolean isEnum();  // Old
> > > > > > > *Collection enumValues();* // New
> > > > > > > }
> > > > > > >
> > > > > > > Thoughts?
> > > > > > >
> > > > > > > Vladimir.
> > > > > > >
> > > > > > > [1] https://github.com/h2database/h2database/pull/487/commits
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Improve binary enum handling

2017-04-25 Thread Dmitriy Setrakyan

Vladimir, can you please share your thoughts here?

On Mon, Apr 24, 2017 at 5:21 PM, Sergi Vladykin 
wrote:

> I agree with Dmitriy, it is preferable to have this enum registration
> optional. It will be a better user experience.
>
> Why do we "inevitably" need it?
>
> Sergi
>
> 2017-04-24 17:02 GMT+03:00 Vladimir Ozerov :
>
> > Dima,
> >
> > No. It is normal (and inevitably) practice to register enums before they
> > are used.
> >
> > This is how enum is created in MySQL:
> >
> > CREATE TABLE shirts (
> > name VARCHAR(40),
> > size ENUM('x-small', 'small', 'medium', 'large', 'x-large')
> > );
> >
> > And in PostgreSQL:
> >
> > CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
> > CREATE TABLE person (
> > name text,
> > current_mood mood
> > );
> >
> > We will do the same at some point. That is, in future users will register
> > enums from SQL, not from native API or configuration.
> >
> > Vladimir.
> >
> > On Mon, Apr 24, 2017 at 4:37 PM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> > wrote:
> >
> > > Vladimir,
> > >
> > > I would really like to avoid special registration of Enums. Can you
> find
> > a
> > > way to handle it automatically?
> > >
> > > D.
> > >
> > > On Mon, Apr 24, 2017 at 6:33 AM, Vladimir Ozerov  >
> > > wrote:
> > >
> > > > Sorry, looks like I mismanaged tickets in JIRA. In fact, we
> implemented
> > > H2
> > > > part, but Ignite's part is not ready yet and is managed in
> IGNITE-4575
> > > [1].
> > > > Ticket you mentioned was an umbrella.
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-4575
> > > >
> > > > On Mon, Apr 24, 2017 at 4:28 PM, Dmitriy Setrakyan <
> > > dsetrak...@apache.org>
> > > > wrote:
> > > >
> > > > > Vladimir,
> > > > >
> > > > > I am very confused. I thought we already had resolved this issue in
> > > this
> > > > > ticket:
> > > > > https://issues.apache.org/jira/browse/IGNITE-3595
> > > > >
> > > > > Can you clarify?
> > > > >
> > > > > D.
> > > > >
> > > > > On Mon, Apr 24, 2017 at 5:58 AM, Vladimir Ozerov <
> > voze...@gridgain.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > Currently we have limited support of binary enums. The main
> problem
> > > is
> > > > > that
> > > > > > we do not store any metadata about enum names. For this reason it
> > is
> > > > > > impossible to use enums in SQL even though H2 already supports it
> > > [1].
> > > > We
> > > > > > need to improve enum metadata support and provide some additional
> > API
> > > > to
> > > > > > register new enums in runtime.
> > > > > >
> > > > > > Proposed API:
> > > > > >
> > > > > > 1) Enum mappings can be defined statically in
> > > BinaryTypeConfiguration:
> > > > > >
> > > > > > class BinaryTypeConfiguration {
> > > > > > boolean isEnum;  // Old method
> > > > > > *Map enumValues;* // New method
> > > > > > }
> > > > > >
> > > > > > 2) New enum could be registered through IgniteBinary (e.g. we
> will
> > > use
> > > > it
> > > > > > if enum is defined in CREATE TABLE statement). Elso it would be
> > > > possible
> > > > > to
> > > > > > build enum using only name.
> > > > > >
> > > > > > interface IgniteBinary {
> > > > > > BinaryObject buildEnum(String typeName, int ordinal);
> > > > //
> > > > > > Old
> > > > > > *BinaryObject buildEnum(String typeName, String name); *
> > > > >  //
> > > > > > New
> > > > > >
> > > > > > *BinaryType defineEnum(String typeName, Map
> > > > vals);*
> > > > > //
> > > > > > New
> > > > > > }
> > > > > >
> > > > > > 3) BinaryObject will have new method "enumName":
> > > > > >
> > > > > > interface BinaryObject {
> > > > > > enumOrdinal(); // Old
> > > > > > *String enumName();* // New
> > > > > > }
> > > > > >
> > > > > > 4) It would be possible to get the list of known values from
> > > > BinaryType:
> > > > > >
> > > > > > interface BinaryType {
> > > > > > boolean isEnum();  // Old
> > > > > > *Collection enumValues();* // New
> > > > > > }
> > > > > >
> > > > > > Thoughts?
> > > > > >
> > > > > > Vladimir.
> > > > > >
> > > > > > [1] https://github.com/h2database/h2database/pull/487/commits
> > > > > >
> > > > >
> > > >
> > >
> >
>

[GitHub] ignite pull request #1870: IGNITE-5077 - Support service security permission...

2017-04-25 Thread dkarachentsev

GitHub user dkarachentsev opened a pull request:

https://github.com/apache/ignite/pull/1870

IGNITE-5077 - Support service security permissions



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5077

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1870.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1870


commit a19938d1fdce87ccaa5b1262921f7281348bffca
Author: dkarachentsev 
Date:   2017-04-25T11:53:48Z

IGNITE-5077 - Support service security permissions




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-5078) Partition lost event is fired only on coordinator when partition loss policy is IGNORE

2017-04-25 Thread Alexey Goncharuk (JIRA)

Alexey Goncharuk created IGNITE-5078:


 Summary: Partition lost event is fired only on coordinator when 
partition loss policy is IGNORE
 Key: IGNITE-5078
 URL: https://issues.apache.org/jira/browse/IGNITE-5078
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.0
Reporter: Alexey Goncharuk
 Fix For: 2.1






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] ignite pull request #1856: Ignite 5040

2017-04-25 Thread ybabak

Github user ybabak closed the pull request at:

https://github.com/apache/ignite/pull/1856


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-5077) Support service permissions

2017-04-25 Thread Dmitry Karachentsev (JIRA)

Dmitry Karachentsev created IGNITE-5077:
---

 Summary: Support service permissions
 Key: IGNITE-5077
 URL: https://issues.apache.org/jira/browse/IGNITE-5077
 Project: Ignite
  Issue Type: New Feature
  Components: managed services
Reporter: Dmitry Karachentsev
Assignee: Dmitry Karachentsev
 Fix For: 2.1


Need to add capability to specify permissions to allow/disallow executions of 
particular services (similar to compute tasks).

The following permissions should be added to the SecurityPermission enum:

SERVICE_DEPLOY - for IgniteServices.deployXXX methods.
SERVICE_CANCEL - for IgniteServices.cancel and IgniteServices.cancelAll 
methods.
SERVICE_INVOKE - for IgniteServices.service, IgniteServices.services and 
IgniteServices.serviceProxy methods.

SERVICE_INVOKE should allow fine-grained authorization based on service name, 
similar to TASK_EXECUTE. E.g., a particular user should be able to execute 
service A, but not service B.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] ignite pull request #1869: Ignite 3935

2017-04-25 Thread alt-freem

Github user alt-freem closed the pull request at:

https://github.com/apache/ignite/pull/1869


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1869: Ignite 3935

2017-04-25 Thread alt-freem

GitHub user alt-freem opened a pull request:

https://github.com/apache/ignite/pull/1869

Ignite 3935

ÐÑÐ¾ Ð½Ðµ PullRequest, Ð² ÑÐ¾Ð¼ ÑÐ¼ÑÑÐ»Ðµ ÑÑÐ¾ Ð·Ð°Ð´Ð°ÑÐ° Ð½Ðµ 
Ð²ÑÐ¿Ð¾Ð»Ð½ÐµÐ½Ð°, PR Ð·Ð°Ð²ÐµÐ» Ð¿ÑÐ¾ÑÑÐ¾ ÐºÐ°Ðº Ð¿Ð»Ð¾ÑÐ°Ð´ÐºÑ Ð´Ð»Ñ 
Ð¾Ð±ÑÑÐ¶Ð´ÐµÐ½Ð¸Ñ Ð¸ Ð²Ð¾Ð·Ð¼Ð¾Ð¶Ð½Ð¾ÑÑÐ¸ ÐºÐ¾Ð¼Ð¼ÐµÐ½ÑÐ¸ÑÐ¾Ð²Ð°ÑÑ 
Ð¿Ð¾ ÐºÐ¾Ð´Ñ.
Ð¢ÑÑ Ð½Ð°ÑÐ¾Ð´Ð¸ÑÑÑ Ð³ÑÑÐ·Ð½ÑÐ¹ ÐºÐ¾Ð´ Ð¼Ð¾Ð¸Ñ Ð¿Ð¾Ð¿ÑÑÐ¾Ðº 
Ð²Ð¾ÑÐ¿ÑÐ¾Ð¸Ð·Ð²ÐµÑÑÐ¸ Ð¿ÑÐ¾Ð±Ð»ÐµÐ¼Ñ, ÐºÐ¾Ð´ Ð½Ðµ Ð¿ÑÐ¸ÑÐµÑÑÐ²Ð°Ð», 
ÑÐ°Ðº Ð·Ð°Ð´Ð°ÑÐ° ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸ Ð½Ðµ ÑÐµÑÐµÐ½Ð° Ð¸ Ð½ÐµÑ 
Ð¾ÑÐ¾Ð±Ð¾Ð³Ð¾ ÑÐ¼ÑÑÐ»Ð° ÑÑÐ¾ Ð´ÐµÐ»Ð°ÑÑ.
Ð£Ð´Ð°Ð»Ð¾ÑÑ Ð²Ð¾ÑÐ¿ÑÐ¾Ð¸Ð·Ð²ÐµÑÑÐ¸ Ð¿ÑÐ¾Ð±Ð»ÐµÐ¼Ñ Ð½Ð° 
Ð¾Ð¿Ð¸ÑÐ°Ð½Ð½Ð¾Ð¼ Ð² ÑÐ¸ÐºÐµÑÐµ Ð¿ÑÐ¸Ð¼ÐµÑÐµ, Ð² ÑÐ»ÑÑÐ°Ðµ ÐºÐ¾Ð³Ð´Ð° 
ÐÐ»Ð¸Ð½ÐµÑ Ð¸ Ð¡ÐµÑÐ²ÐµÑ ÑÑÐ°ÑÑÑÑÑ Ð² ÑÐ°Ð·Ð½ÑÑ jvm Ð¸ Ð¸Ð¼ÐµÑÑ 
ÑÐ°Ð·Ð½ÑÐµ classpath, ÑÐ¾ ÐµÑÑÑ Ð½Ð° ÑÐµÑÐ²ÐµÑÐµ ÑÐ²Ð½Ð¾ 
Ð¾ÑÑÑÑÑÑÐ²ÑÐµÑ ÐºÐ»Ð¸ÐµÐ½ÑÑÐºÐ°Ñ ÑÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ñ 
CacheEntryProcessor.

ÐÑÐ¾Ð±Ð»ÐµÐ¼Ð°, Ñ Ð¿Ð¾Ð»Ð°Ð³Ð°Ñ Ð² Ð¼ÐµÑÐ¾Ð´Ðµ 
GridDeploymentManager#getGlobalDeployment - ÑÐ°Ð¼ ÐµÑÑÑ 
Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ñ ÐºÐ¾ÑÐ¾ÑÐ°Ñ ÑÐ½Ð°ÑÐ°Ð»Ð° Ð¸ÑÐµÑ 
Ð¿ÐµÑÐµÐ´Ð°Ð½Ð½ÑÐ¹ Ð¾Ñ ÐºÐ»Ð¸ÐµÐ½ÑÐ° ÐºÐ»Ð°ÑÑ Ð² localDeployment Ð¸ 
ÐµÑÐ»Ð¸ ÐµÐ³Ð¾ Ð½Ð°ÑÐ¾Ð´Ð¸Ñ, ÑÐ¾ Ð´Ð°Ð»ÑÐ½ÐµÐ¹ÑÑÑ 
Ð´ÐµÑÐµÑÐ¸Ð°Ð»Ð¸Ð·Ð°ÑÐ¸Ñ Ð¾ÑÐ²ÐµÑÐ° Ð¿ÑÐ¾Ð²Ð¾Ð´Ð¸Ñ Ñ classloader'Ð¾Ð¼ 
localDeployment'Ð°, ÑÑÐ¾ Ð¿ÑÐ¸Ð²Ð¾Ð´Ð¸Ñ Ðº Ð¿ÑÐ¾Ð±Ð»ÐµÐ¼Ð°Ð¼ Ð² 
Ð¾Ð¿Ð¸ÑÐ°Ð½Ð½Ð¾Ð¼ ÑÐ»ÑÑÐ°Ðµ: Ð² localDeployment Ð½Ð°ÑÐ¾Ð´Ð¸ÑÑÑÑ 
StreamTrasformer$1 Ð²Ð½ÑÑÑÐ¸ ÐºÐ¾ÑÐ¾ÑÐ¾Ð³Ð¾ ÑÑÑÐ»ÐºÐ° Ð½Ðµ 
ÐºÐ»Ð¸ÐµÐ½ÑÑÐºÐ¸Ð¹ ÐºÐ»Ð°ÑÑ ÑÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ð¸ CacheEntryProcessor, 
ÑÐ¾Ð¾ÑÐ²ÐµÑÑÑÐ²ÐµÐ½Ð½Ð¾ Ð´ÐµÑÑÐµÑÐ¸Ð°Ð»Ð¸Ð·Ð°ÑÐ¸Ñ 
ÑÐ°Ð·Ð²Ð°Ð»Ð¸Ð²Ð°ÐµÑÑÑ.
ÐÐµÐ½Ñ ÑÐ¼ÑÑÐ°ÐµÑ ÑÑÐ¾ ÐºÐ¾Ð´ 
GridDeploymentManager#getGlobalDeployment Ð½Ðµ Ð¼ÐµÐ½ÑÐ»ÑÑ Ñ 14Ð³Ð¾ 
Ð³Ð¾Ð´Ð°, ÑÐ¾ ÐµÑÑÑ Ð²ÑÐ¾Ð´Ðµ ÐºÐ°Ðº ÑÑÐ°Ð±Ð¸Ð»ÑÐ½ÑÐ¹ Ð¸ Ð²ÑÐ¾Ð´Ðµ 
ÐºÐ°Ðº Ð´Ð¾Ð»Ð¶ÐµÐ½ Ð±ÑÑÑ ÑÐ°Ð±Ð¾ÑÐ¸Ð¼, Ð² ÑÐ¾Ð¶Ðµ Ð²ÑÐµÐ¼Ñ Ñ Ð½Ðµ 
Ð½Ð°ÑÐµÐ» Ð½Ð° Ð½ÐµÐ³Ð¾ ÐºÐ°ÐºÐ¸Ñ Ð»Ð¸Ð±Ð¾ ÑÐµÑÑÐ¾Ð² ÐºÐ¾ÑÐ¾ÑÑÐµ 
ÐºÐ°Ðº-ÑÐ¾ Ð¿ÑÐ¾Ð²ÐµÑÑÐ»Ð¸ Ð±Ñ ÐµÐ³Ð¾ ÐºÐ¾ÑÑÐµÐºÑÐ½Ð¾ÑÑÑ.
Ð Ð»ÑÐ±Ð¾Ð¼ ÑÐ»ÑÑÐ°Ðµ Ð¿Ð¾Ð´Ð¾Ð±Ð½Ð°Ñ Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ñ Ð¼Ð½Ðµ 
ÐºÐ°Ð¶ÐµÑÑÑ ÑÐ¾Ð¼Ð½Ð¸ÑÐµÐ»ÑÐ½Ð¾Ð¹, ÐºÐ°Ðº ÑÐµÑÐµÐ½Ð¸Ðµ Ñ Ð±Ñ 
Ð¿ÑÐµÐ´Ð»Ð¾Ð¶Ð¸Ð» Ð²ÑÐ¿Ð¸Ð»Ð¸ÑÑ Ð²ÑÑ Ð²ÐµÑÐºÑ Ð²Ð¾ÐºÑÑÐ³ 
Ð¿ÐµÑÐµÐ¼ÐµÐ½Ð½Ð¾Ð¹ boolean reuse = true Ð¸ Ð±ÐµÐ·ÑÑÐ»Ð¾Ð²Ð½Ð¾ 
Ð²Ð¾Ð·Ð²ÑÐ°ÑÐ°ÑÑ verStore.getDeployment(meta).
Classloader ÐºÐ¾ÑÐ¾ÑÐ¾Ð³Ð¾ Ð¼Ð¾Ð³ Ð±Ñ Ð¸Ð¼ÐµÑÑ parent'Ð¾Ð¼ - 
localDeployment.classloader.
Ð¢Ð°ÐºÐ¸Ð¼ Ð¾Ð±ÑÐ°Ð·Ð¾Ð¼ GridDeploymentClassLoader

StreamTrasformer$1 Ð·Ð°Ð³ÑÑÐ·Ð¸Ñ Ð¸Ð· Ð»Ð¾ÐºÐ°Ð»ÑÐ½Ð¾Ð³Ð¾ 
classloader'Ð° 
 Ð° ÑÐ²ÑÐ·Ð°Ð½Ð½ÑÑ Ñ Ð½Ð¸Ð¼ ÑÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ñ 
CacheEntryProcessor (Ð½Ðµ Ð½Ð°Ð¹Ð´Ñ Ð² Ð»Ð¾ÐºÐ°Ð»ÑÐ½Ð¾Ð¼) 
Ð·Ð°Ð³ÑÑÐ·Ð¸Ñ Ð·Ð°Ð¿ÑÐ¾ÑÐ¾Ð¼ Ñ ÐºÐ»Ð¸ÐµÐ½ÑÐ°

Ð¡Ð¾Ð±ÑÑÐ²ÐµÐ½Ð½Ð¾ ÑÐºÑÐ¿ÐµÑÐ¸Ð¼ÐµÐ½Ñ Ð½Ð° Ð¾ÑÐ´ÐµÐ»ÑÐ½ÑÑ JVM 
ÑÑÐ¾ Ð¿Ð¾Ð´ÑÐ²ÐµÑÐ´Ð¸Ð» (ÐºÐ»Ð°ÑÑÑ Client, Server), 
ÑÑÑÐµÑÑÐ²ÑÑÑÐ¸Ð¹ Ð½Ð°Ð±Ð¾Ñ ÑÐµÑÑÐ¾Ð² Ð¿Ð°ÐºÐµÑÐ° 
org.apache.ignite.internal.processors.datastreamer Ð¿ÑÐ¾ÑÐµÐ» Ð±ÐµÐ· 
Ð¸Ð·Ð¼ÐµÐ½ÐµÐ½Ð¸Ð¹.

ÐÐ¾Ð³Ð´Ð° Ð½Ð°ÑÐ°Ð» Ð²ÑÐµ ÑÑÐ¾ Ð·Ð°Ð²Ð¾ÑÐ°ÑÐ¸Ð²Ð°ÑÑ Ð² Ð¾Ð´Ð¸Ð½ 
junit ÑÐµÑÑ Ð²Ð¾Ð·Ð½Ð¸ÐºÐ»Ð¸ Ð¿ÑÐ¾Ð±Ð»ÐµÐ¼Ñ Ñ ÑÐµÐ¼ ÑÑÐ¾ ÐºÐ»Ð¸ÐµÐ½Ñ 
ignite ÑÐ¾ ÑÑÑ ÑÐ¾ ÑÐ°Ð¼ ÑÐµÐ¿Ð»ÑÐ» Ð½Ðµ ÑÐ¾Ñ classloder' 
ÐºÐ¾ÑÐ¾ÑÑÐ¹ Ð¼Ð½Ðµ Ð±ÑÐ» Ð½ÑÐ¶ÐµÐ½,
Ð½Ð°Ð¿ÑÐ¸Ð¼ÐµÑ Ð² ÐºÐ°ÑÐµÑÑÐ²Ðµ Ð±Ð°Ð·Ð¾Ð²Ð¾Ð³Ð¾ classloader'a 
ÐºÐ»Ð¸ÐµÐ½Ñ Ð¿ÐµÑÐµÐ´ Ð¾ÑÐ¿ÑÐ°Ð²ÐºÐ¾Ð¹ ÑÑÑÐ¸Ð¼Ð° Ð½Ð° ÑÐµÑÐ²ÐµÑ 
Ð²ÑÐ±Ð¸ÑÐ°Ð» Ð½Ðµ ÑÐ¾Ñ classloader ÐºÐ¾ÑÐ¾ÑÑÐ¼ Ð±ÑÐ» Ð·Ð°Ð³ÑÑÐ¶ÐµÐ½ 
Ð¿ÐµÑÐµÐ´Ð°Ð²Ð°ÐµÐ¼ÑÐ¹ Ð¾Ð±ÑÐµÐºÑ, ÑÐ¾Ñ ÐºÐ¾ÑÐ¾ÑÑÐ¼ Ð±ÑÐ» 
Ð·Ð°Ð³ÑÑÐ¶ÐµÐ½ Ð¿ÐµÑÐ²ÑÐ¹ nonJdk ÐºÐ»Ð°ÑÑ ÑÐ²ÑÐ·Ð°Ð½Ð½ÑÐ¹ Ñ 
Ð¿ÐµÑÐµÐ´Ð°Ð²Ð°ÐµÐ¼ÑÐ¼ Ð¾Ð±ÑÐµÐºÑÐ¾Ð¼.
DataStreamerPda#deployClass
```java
for (Iterator it = objs.iterator(); (cls0 == null || U.isJdk(cls0)) 
&& it.hasNext(); ) {
Object o = it.next();
if (o != null)
cls0 = U.detectClass(o);
}
```
Ð¢Ð°ÐºÐ¾Ð³Ð¾ ÑÐ¾Ð´Ð° Ð¿ÑÐ¾Ð±Ð»ÐµÐ¼ Ð±ÑÐ»Ð¾ Ð½ÐµÑÐºÐ¾Ð»ÑÐºÐ¾ Ð¸ Ñ Ð½Ðµ 
ÑÑÐ¿ÐµÐ» Ð¸Ñ ÑÐµÑÐ¸ÑÑ.
ÐÐ°Ð´ Ð·Ð°Ð´Ð°ÑÐµÐ¹ ÑÐ°Ð±Ð¾ÑÐ°Ð» Ð¡Ð±. Ð¸ ÐÑ. Ð½Ñ Ð¸ ÑÐµÐ³Ð¾Ð´Ð½Ñ 
Ð²Ð¾Ñ ÑÐµÐ·ÑÐ¼Ðµ Ð¾ÑÐ¾ÑÐ¼Ð¸Ð» ÐµÑÑ Ð¼Ð¸Ð½ÑÑ 30.
ÐÐ°Ð½Ð¸Ð¼Ð°ÑÑÑÑ Ð·Ð°Ð´Ð°ÑÐµÐ¹ Ð½Ð° Ð±ÑÐ´Ð½ÑÑ - Ð½ÐµÑ 
Ð²ÑÐµÐ¼ÐµÐ½Ð¸, ÑÑÐ°ÑÐ¸ÑÑ ÐµÑÐµ Ð¾Ð´Ð½Ð¸ Ð²ÑÑÐ¾Ð´Ð½ÑÐµ - Ð½ÐµÑ 
Ð¶ÐµÐ»Ð°Ð½Ð¸Ñ.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/alt-freem/ignite IGNITE-3935

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1869.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1869

Re: Null cache names

2017-04-25 Thread Vladimir Ozerov

Folks,

I do not think it is legal to add such property to ConnectorConfiguration.
Connector is a generic gateway to cluster resources. It should not bother
about caches anyhow. What if there are multiple caches in the cluster? What
is I want to access "cache A" from Memcached and "cache B" from Redis
simultaneously? Etc.. This kind of property should be defined on client
level, not on the server.

For now, provided that 2.0 is about to be freezed, I propose to stick to
Dmitriy's approach and use "default" cache name instead of null. This
should work fine for AI 2.0. We will be able to improve it in further
releases.

Thoughts?

Vladimir.

On Tue, Apr 25, 2017 at 7:22 AM, Roman Shtykh 
wrote:

> Igor, +1 from me.We can add a field to ConnectorConfiguration (not sure if
> it's a proper place, but it's shared by REST, memcached and Redis). A user
> will have to create a cache, configure as needed and specify the name in
> ConnectorConfiguration.
> Roman
>
>
>
> On Monday, April 24, 2017 10:34 PM, Seliverstov Igor <
> gvvinbl...@gmail.com> wrote:
>
>
>  Dear Igniters,
>
> Seems we have almost the same issue with Memcached protocol.
>
> There is an ability to define a cache name via operation extras message
> part (
> https://github.com/memcached/memcached/wiki/BinaryProtocolRevamped#packet-
> structure)
> but it looks a bit complicated from my point of view...
>
> Different client implementations might provide such functionality or not (I
> mean an additional info in an operation extras), so, potential user might
> have some difficultes using Ignite as a Memcached server because of this
> ignite-specific message part becomes mandatory.
>
> An alternative (an the best way, I think) is introducing a configuration
> property to define which cache to use in case it hasn't be defined in a
> message.
>
> I'll appreciate any thoughts on that.
>
> Regards,
> Igor
>
> 2017-04-24 12:43 GMT+03:00 Roman Shtykh :
>
> > Vladimir,
> > Probably we may set the cache name via https://redis.io/commands/
> > client-setname, save it and use until the user specifies another name.
> > #That will be the name for the default cache (mainly for STRING data).
> For
> > other data types, like hashes (https://redis.io/topics/data-types), I am
> > thinking about putting data into caches specified by key.
> > Or use https://redis.io/commands/config-set CONFIG SET DFLT_CACHE
> > cache_name,and save cache name somewhere in Ignite cluster (what is the
> > proper place to store such info?).
> > For that, we have to implement one of the above-mentioned commands.
> > What do you think?
> >
> >
> >
> >On Monday, April 24, 2017 4:34 PM, Vladimir Ozerov <
> > voze...@gridgain.com> wrote:
> >
> >
> >  Roman,
> > Is it possible to define a kind of property in Redis connection string
> (or
> > property map) for this purpose? IMO ideally we should "externalize" cache
> > name somehow, so that it can be changed with no changes to user's code.
> >
> > Alex,
> > Not sure if this is a good idea as you will end up with PARTITIONED cache
> > without backups with no ability to change that.
> >
> > On Mon, Apr 24, 2017 at 9:35 AM, Alexey Kuznetsov  >
> > wrote:
> >
> > > Roman,
> > >
> > > Just as idea, how about in case of user does not configured
> "REDIS_CACHE"
> > >  then create it via ignite.getOrCreateCache(new
> > > CacheConfiguration("REDIS_CACHE"))
> > > and prin warning to log "REDIS_CACHE not configured, using default
> > > partitioned cache".
> > >
> > > What do you think?
> > >
> > > On Mon, Apr 24, 2017 at 10:26 AM, Roman Shtykh
>  > >
> > > wrote:
> > >
> > > > Denis, Igor,
> > > > What can be done now
> > > > 1. create a default cache name for Redis data, e.g. "REDIS_CACHE"
> that
> > > has
> > > > to be configured explicitly in xml file (as it is done with other
> > caches)
> > > > by a user if he/she needs Redis protocol.
> > > > 2. Force users to specify cache names as prefix to their keys, so
> that
> > we
> > > > can parse and switch between caches.
> > > > The 1st one is a very quick fix I can do today. This can be extended
> in
> > > > future to have a separate cache for each data type.
> > > > Roman
> > > >
> > > >
> > > >On Monday, April 24, 2017 12:15 AM, Denis Magda <
> > dma...@gridgain.com
> > > >
> > > > wrote:
> > > >
> > > >
> > > >  Roman, would you suggest a quick solution for the redis integration
> or
> > > > even
> > > > implement it in the nearest couple of days? We need to change the
> > defaul
> > > > cache name in 2.0. Otherwise, it can be done only in an year or so in
> > > 3.0.
> > > >
> > > > Denis
> > > >
> > > > On Sunday, April 23, 2017, Seliverstov Igor 
> > > wrote:
> > > >
> > > > > Hi Roman,
> > > > >
> > > > > The ticket number is IGNITE-3488.
> > > > >
> > > > > It's planned not to have null-named or default caches at all. All
> the
> > > > > caches must be defined

[jira] [Created] (IGNITE-5076) Optimization of multi-threaded start nodes in tests

2017-04-25 Thread Dmitriy Govorukhin (JIRA)

Dmitriy Govorukhin created IGNITE-5076:
--

 Summary: Optimization of multi-threaded start nodes in tests
 Key: IGNITE-5076
 URL: https://issues.apache.org/jira/browse/IGNITE-5076
 Project: Ignite
  Issue Type: Improvement
Reporter: Dmitriy Govorukhin
Assignee: Dmitriy Govorukhin
 Fix For: 2.0


Concurrent start,will more effective if we have coordinator before, 
multi-threaded start nodes. If start all nodes concurrent, they will be compete 
for coordinator role, it is not effective way.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] ignite pull request #1867: incorrect construction of swap files directory pa...

2017-04-25 Thread sergey-chugunov-1985

GitHub user sergey-chugunov-1985 opened a pull request:

https://github.com/apache/ignite/pull/1867

incorrect construction of swap files directory path fixed



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5067

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1867.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1867


commit 5e231cadb233ca1fff4c0a15f5672f93da8976cf
Author: Sergey Chugunov 
Date:   2017-04-25T08:21:58Z

IGNITE-5067 incorrect construction of swap files directory path fixed




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-5075) Implement logical 'cache groups' sharing the same physical caches

2017-04-25 Thread Semen Boikov (JIRA)

Semen Boikov created IGNITE-5075:


 Summary: Implement logical 'cache groups' sharing the same 
physical caches
 Key: IGNITE-5075
 URL: https://issues.apache.org/jira/browse/IGNITE-5075
 Project: Ignite
  Issue Type: Task
  Components: cache
Reporter: Semen Boikov
Assignee: Semen Boikov
Priority: Critical
 Fix For: 2.1


Currently started caches have pretty large memory overhead (e.g. to store 
affinity and partitions state information). If some application requires 
thousands caches with similar configuration then it would be useful to allow 
for 'logical' cache to reuse the same 'physical' cache. Let's introduce new 
CacheConfiguration property - 'groupName', caches with the same groupName will 
use the same 'physical' cache.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5074) Split up asynchronous and synchronous calls in PlatformCompute

2017-04-25 Thread Igor Sapego (JIRA)

Igor Sapego created IGNITE-5074:
---

 Summary: Split up asynchronous and synchronous calls in 
PlatformCompute
 Key: IGNITE-5074
 URL: https://issues.apache.org/jira/browse/IGNITE-5074
 Project: Ignite
  Issue Type: Task
  Components: platforms
Affects Versions: 1.9
Reporter: Igor Sapego
 Fix For: 2.1


Currently, there is only asynchronous version of calls in {{PlatformCompute}}, 
which means we create temporary objects (like futures and listeners) which we 
do not need during synchronous calls.

Add synchronous version of API calls which generate less garbage.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5073) Race between partition exchange process and client cache operations

2017-04-25 Thread Semen Boikov (JIRA)

Semen Boikov created IGNITE-5073:


 Summary: Race between partition exchange process and client cache 
operations
 Key: IGNITE-5073
 URL: https://issues.apache.org/jira/browse/IGNITE-5073
 Project: Ignite
  Issue Type: Bug
  Components: cache
Reporter: Semen Boikov
Assignee: Semen Boikov
Priority: Critical
 Fix For: 2.1


Added test reproducing issue IgniteCacheClientMultiNodeUpdateTopologyLockTest:
- 3 servers (node1, node2, node3), 1 client
- client starts pessimistic tx
- client locks key1 on node2
- new node joins, exchanges starts, on node3 there are no ongoging cache 
operations and node3 sends GridDhtPartitionsSingleMessage to coordinator
- client locks key2 on node3
- client commits tx, when tx started on node2 finishes then node2 will send 
GridDhtPartitionsSingleMessage to coordinator and exchange will be completed 
before tx on node3 finished

One potential fix for this issue is change exchange protocol to use two steps.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5072) Move memory metrics to snapshots

2017-04-25 Thread Alexey Goncharuk (JIRA)

Alexey Goncharuk created IGNITE-5072:


 Summary: Move memory metrics to snapshots
 Key: IGNITE-5072
 URL: https://issues.apache.org/jira/browse/IGNITE-5072
 Project: Ignite
  Issue Type: Improvement
  Components: cache
Affects Versions: 2.0
Reporter: Alexey Goncharuk
Assignee: Alexey Goncharuk
 Fix For: 2.0


This is a follow-up ticket for the following discussion:
http://apache-ignite-developers.2346864.n4.nabble.com/MemoryMetrics-interface-inconsistencies-td17156.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: MemoryMetrics interface inconsistencies

2017-04-25 Thread Pavel Tupitsyn

Can you give an example of such API usage?
A piece of code to enable metrics and retrieve a snapshot?

On Mon, Apr 24, 2017 at 8:06 PM, Alexey Goncharuk <
alexey.goncha...@gmail.com> wrote:

> Agree, I overlooked this during the review. However, the changes to fix
> this is pretty simple - we just move all mutator methods to MBean, like
> Vladimir suggested and make MBean return the live data, while the public
> API will return a serializable snapshot.
>
> Agreed?
>
> 2017-04-24 19:33 GMT+03:00 Vladimir Ozerov :
>
> > It seems that all mutator methods should be simply moved to MBean
> interface
> > and change MBytes to bytes. In this case metrics interface will be
> > consistent.
> >
> > On Mon, Apr 24, 2017 at 7:29 PM, Vladimir Ozerov 
> > wrote:
> >
> > > Sergey,
> > >
> > > We need to keep API consistent. If we usually return bytes, then these
> > > metrics should return bytes as well. What is more important, when I
> look
> > at
> > > API I understand that this thing is not metrics at all. Metrics in
> Ignte
> > > terms is a set of read-only numeric properties. But this interface
> > contains
> > > strange things like "name", "swapPilePath". What even more strange, I
> do
> > > not see how to get these metrics from public API.
> > >
> > > All in all, looks like this entity is not "metrics", but "MBean" in
> usual
> > > Ignite terms.
> > >
> > > Vladimir.
> > >
> > > On Mon, Apr 24, 2017 at 7:20 PM, Pavel Tupitsyn 
> > > wrote:
> > >
> > >> Sergey, I disagree.
> > >>
> > >> 1) As a user, I would expect MemoryMetrics instance to be
> > >> read-only and serializable, so I can send it somewhere, store,
> > >> put into a collection and draw a graph in UI, etc.
> > >>
> > >> Other metrics APIs allow this, MemoryMetrics does not.
> > >>
> > >> 2) Methods like enableMetrics and rateTimeInterval placed in
> > MemoryMetrics
> > >> break single responsibility principle and are confusing.
> > >> Why do I need to Get metrics to Enable them?
> > >>
> > >> These methods should be placed somewhere else.
> > >>
> > >> I would suggest some thing like this:
> > >> - introduce Memory class with getMetrics, enableMetrics,
> > >> setRateTimeInterval, etc
> > >> - add Ignite.getMemory method
> > >>
> > >>
> > >> On Mon, Apr 24, 2017 at 6:53 PM, Sergey Chugunov <
> > >> sergey.chugu...@gmail.com>
> > >> wrote:
> > >>
> > >> > I guess the main reason to have IgniteCache returning snapshot
> metrics
> > >> was
> > >> > to collect metrics about distributed cache across the cluster.
> > >> > At the same time MemoryMetrics were initially designed to be local
> on
> > >> each
> > >> > node, there were no requirements about collecting cluster-wide
> > >> > MemoryMetrics.
> > >> >
> > >> > Collecting MemoryMetrics is considered as an investigation action
> when
> > >> > something goes wrong, that's why it was decided to add
> enable/disable
> > >> > actions to the interface.
> > >> > So for me it sounds reasonable.
> > >> >
> > >> > The only debatable thing is having MemoryMetrics returning size in
> > >> > megabytes, however I think about these metrics as designed only for
> > >> user,
> > >> > so I think it makes sense to return this metric in human-readable
> > form.
> > >> >
> > >> > On Mon, Apr 24, 2017 at 6:41 PM, Vladimir Ozerov <
> > voze...@gridgain.com>
> > >> > wrote:
> > >> >
> > >> > > Agree, looks inconsistent to me.
> > >> > >
> > >> > > Alex G., could you chime in?
> > >> > >
> > >> > > On Mon, Apr 24, 2017 at 6:30 PM, Pavel Tupitsyn <
> > ptupit...@apache.org
> > >> >
> > >> > > wrote:
> > >> > >
> > >> > > > Igniters,
> > >> > > >
> > >> > > > I have noticed quite a lot of inconsistencies with the rest of
> our
> > >> APIs
> > >> > > in
> > >> > > > our new MemoryMetrics API:
> > >> > > >
> > >> > > > 1) MemoryMetrics is not a snapshot. It is a "live" instance
> which
> > >> > returns
> > >> > > > different values each time you access some property.
> > >> > > (IgniteCache.metrics()
> > >> > > > returns a snapshot).
> > >> > > >
> > >> > > > 2) MemoryMetrics has methods that modify state, like
> enableMetrics
> > >> > > > and rateTimeInterval
> > >> > > > (IgniteCache.metrics() returns a read-only serializable
> snapshot)
> > >> > > >
> > >> > > > 3) getSize method returns size in megabytes
> > >> > > > (IgniteCache.metrics() provides sizes in bytes)
> > >> > > >
> > >> > > >
> > >> > > > I propose to rework this API until it is not too late.
> > >> > > >
> > >> > > > Thoughts?
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

Re: SQL usability: catalogs, schemas and tables

2017-04-25 Thread Dmitriy Setrakyan

Vladimir, great suggestions. Can you please make an umbrella ticket with
sub-tasks, so we can tackle some of them for 2.1?

D.

On Mon, Apr 24, 2017 at 5:28 PM, Sergi Vladykin 
wrote:

> Yes, we need to move on making Ignite work as any usual SQL db.
>
> But please avoid mixing all this stuff together, lets have a separate task
> (and discussion if needed) for each item in your list.
>
> Sergi
>
> 2017-04-24 16:58 GMT+03:00 Vladimir Ozerov :
>
> > Igniters,
> >
> > [long read, take a cup of coffee]
> >
> > Historically every SQL in Ignite must be executed against particular
> cache:
> > SqlQuery requires cache, JDBC and ODBC drivers require cache name. This
> > works fine until we add CREATE TABLE. Consider an empty cluster - how do
> > you connect to it if you have no caches yet? Nohow.
> >
> > It looks like we cannot have convenient access to cluster unless we
> > properly define and implement *schema* abstraction. ANSI SQL'92 defines
> > several abstractions: cluster -> catalog -> schema -> table/view/etc..
> > Every "catalog" has *INFORMATION_SCHEMA* schema, containing database
> > metadata. Almost all vendors support it (notable exclusion - Oracle).
> Looks
> > like we need to introduce similar concept and finally decouple caches
> from
> > schemas.
> >
> > High-level proposal from my side
> >
> > 1) Let's consider Ignite cluster as a single database ("catalog" in ANSI
> > SQL'92 terms).
> >
> > 2) It should be possible to connect to the cluster without a single user
> > cache. In this case schema is not defined.
> >
> > 3) We must have a kind of storage for metadata. It could be either
> another
> > system cache, or something analogous to binary metadata cache, which is
> > essentially node-local data structure exchanged on node join. It should
> be
> > aligned well with persistence feature, which is expected in AI 2.1.
> >
> > 4) Content of this storage will be accessible through INFORMATION_SCHEMA
> > abstraction.
> >
> > 5) We must support "CREATE SCHEMA", "DROP SCHEMA" commands which will
> > effectively create records in system cache and invoke relevant commands
> on
> > local H2 engines of every node (now it happens implicitly on cache
> > start/stop).
> >
> > 6) From JDBC/ODBC driver perspective schema will be defined either in
> > connection string, or in runtime through "SET SCHEMA" command which is
> > already supported by H2.
> >
> > 7) We must finally introduce new native SQL API, which will not use
> caches
> > directly. Something like "IgniteSql sql()". See *IGNITE-4701*.
> >
> > Once schema is there, usage of "CREATE TABLE" and "DROP TABLE" commands
> > will be simple and convenient, and it will fit naturally in user's past
> > experience with conventional RDBMS.
> >
> > Thoughts?
> >
> > P.S.: CREATE/DROP TABLE feature is not blocked by this problem. It will
> > work, but will be inconvenient for users.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-4701
> >
>

48 matches

Mail list logo