Fwd: Re: When cache node switch between primary and backup any notification be received?

[Dmitriy Setrakyan]

Cross-sending to dev@.

Igniters, is it really this difficult to iterate over local partitions, or
is there a simpler way? What is worse, is that I could not find a single
example anywhere of how to do it. Seems like a pretty common use case.

D.

-- Forwarded message --
From: Dmitriy Setrakyan 
Date: Mon, Sep 11, 2017 at 10:41 PM
Subject: Re: Re: When cache node switch between primary and backup any
notification be received?
To: user 
Cc: aa...@tophold.com




On Mon, Sep 11, 2017 at 6:54 PM, aa...@tophold.com 
wrote:

> Thanks Alexey!   what we real want, we deploy service on each Cache Node.
>  those service will use data from its' local cache.
>
> Client will call those remote service, Client should only call  the
> service on primary node,  this make those nodes work like master-slave mode
> automatically.
>

In Ignite, a node is a primary node for a certain partition. A key belongs
to a partition and a partition belongs to a node. A node may be primary for
key1 (partition N)  and the same node may be a back up for key 2 (partition
M).

I think you simply should invoke your service on each node and only check
or iterate through primary keys stored on that node. You can get a list of
primary keys by using org.apache.ignite.cache.affinity.Affinity API, for
example Affinty.primaryPartitions(ClusterNode) method.

*int[] primaryPartitions =
> Ignite.affinity("cacheName").primaryPartitions(Ignite.cluster().localNode());*




>
> *for (int primaryPartition : primaryPartitions) {// Cursor over local
> entries for the given partition.QueryCursor> cur =
> cache.query(new ScanQuery(primaryPartition));*




>
>
> * for (Entry entry : cur) { // Do something on local entries. }}*


Does this make sense?

[jira] [Created] (IGNITE-6347) Exception in GridDhtPartitionMap.readExternal

[Alexandr Kuramshin (JIRA)]

Alexandr Kuramshin created IGNITE-6347:
--

 Summary: Exception in GridDhtPartitionMap.readExternal
 Key: IGNITE-6347
 URL: https://issues.apache.org/jira/browse/IGNITE-6347
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 2.1
Reporter: Alexandr Kuramshin
 Fix For: 2.1


Reading partition state with {{id > Short.MAX_VALUE}} causes to read negative 
value in {{int part = in.readShort()}}

{{in.readUnsignedShort()}} should be used instead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2640: IGNITE-6345: Wrong message about cluster activati...

[shroman]

GitHub user shroman opened a pull request:

https://github.com/apache/ignite/pull/2640

IGNITE-6345: Wrong message about cluster activation.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shroman/ignite IGNITE-6345

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2640.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2640


commit 824d95a8e00f3918d09fe7ff7da5718536378461
Author: shroman 
Date:   2017-09-12T02:34:15Z

IGNITE-6345: Wrong message about cluster activation.




---

Re: AffinityKeyMapper alternatives

[Dmitriy Setrakyan]

On Mon, Sep 11, 2017 at 4:47 PM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Even if CacheKeyConfiguration is part of CacheConfiguration, the affinity
> key field name can be provided only on cache startup. In many cases this
> name can be resolved only based on the actual key instance, e.g. during
> first put. Per my understanding, this already works with annotation, I just
> propose more flexible solution for rare cases when annotation can't be
> used. Basically, the logic we currently have would become the default
> implementation of the resolver.
>

Val, the use case seems very strange to me. How can you not know the
affinity key field in advance? Can you provide an example from the field?

Re: Ignite Benchmarking page on readme.io

[Dmitriy Setrakyan]

On Mon, Sep 11, 2017 at 6:10 PM, Denis Magda  wrote:

> 1. It’s fine to have it under the current section that will include
> testing related documentation proposed by Yakov in other discussion thread.
>

I think we should avoid having 1-page sections. They look bad. Also,
benchmarking is a very common topic during evaluation process and deserves
a spot up top. I would still place it under Basic Concepts for now. You can
always move it to a separate section once we have more content.

Re: [VOTE] Apache Ignite 2.2.0 RC1

[Denis Magda]

+1 (binding)

Built from sources and confirmed that ML profile is recovered and its examples 
are running.

—
Denis

> On Sep 11, 2017, at 6:31 AM, Anton Vinogradov  wrote:
> 
> Igniters,
> 
> We have uploaded a 2.2.0 release candidate to
> https://dist.apache.org/repos/dist/dev/ignite/2.2.0-rc1/
> 
> Git tag name is
> 2.2.0-rc1
> 
> This release includes the following changes:
> 
> Ignite:
> * Checkpointing algorithm optimized
> * Default max memory size changed from 80% to 20%
> 
> Ignite CPP:
> * Now possible to start node with persistent store
> * Ignite.setActive method added to C++ API
> 
> Complete list of closed issues:
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%202.2%20AND%20(status%20%3D%20closed%20or%20status%20%3D%20resolved)
> 
> DEVNOTES
> https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_plain;f=DEVNOTES.txt;hb=refs/tags/2.2.0-rc1
> 
> RELEASE NOTES
> https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_plain;f=RELEASE_NOTES.txt;hb=refs/tags/2.2.0-rc1
> 
> Please start voting.
> 
> +1 - to accept Apache Ignite 2.2.0-RC1
> 0 - don't care either way
> -1 - DO NOT accept Apache Ignite 2.2.0-RC1 (explain why)
> 
> This vote will go for 72 hours.

Re: Ignite Benchmarking page on readme.io

[Denis Magda]

1. It’s fine to have it under the current section that will include testing 
related documentation proposed by Yakov in other discussion thread.

2. That’s easy and makes total sense. Renamed.

3. Mikhail could you add this?

—
Denis

> On Sep 11, 2017, at 6:01 PM, Dmitriy Setrakyan  wrote:
> 
> Several comments:
> I would place this page directly under the top Basic Concepts section 
> I would rename the link to Benchmarking (performance is assumed)
> I would add a few benchmark runs with examples graphs there
> D.
> 
> On Mon, Sep 11, 2017 at 5:03 PM, Denis Magda  > wrote:
> I’ve done the primarily review and rename the page address. That’s it:
> https://apacheignite.readme.io/docs/perfomance-benchmarking 
> 
> 
> Prachi, please do the final review. The page is already online.
> 
> —
> Denis
> 
>> On Sep 8, 2017, at 10:40 AM, Aleksei Zaitsev > > wrote:
>> 
>> Thanks. Here is a direct link on the new page about Ignite 
>> Benchmarking(hidden for now): 
>> https://apacheignite.readme.io/v2.1/docs/perfomance-tests 
>> . Please review.
>> Mostly it was aggregated from the information in the repo
>> 
>> 07.09.2017, 20:57, "Dmitriy Setrakyan" > >:
>>> Actually, it can be contributed directly to readme.io . 
>>> Alexey, I just sent you an invite.
>>> 
>>> You can create a new page for benchmarking and make it invisible for now, 
>>> while you are working on it. If you need us to review it, please send a 
>>> direct link to it here.
>>> 
>>> D.
>>> 
>>> On Thu, Sep 7, 2017 at 8:26 AM, Anton Vinogradov >> > wrote:
 Aleksei,
 
 Good idea.
 Could you please prepare page in MD format and attach link to branch
 containing it to the issue to review before publication?
 
 On Thu, Sep 7, 2017 at 1:45 PM, Aleksei Zaitsev >
 wrote:
 
> Hi, guys.
> 
> Issue[1] stated that we have not any information about Apache Ignite
> benchmarking on our site https://apacheignite.readme.io 
> . I want to add
> such a page, but I can’t find appropriate section for it. I suggest to 
> make
> new section named ‘Tests and Benchmarking’ and put all information about
> benchmaring in section page ‘Perfomance Tests’
> 
> Thoughts?
> 
> [1] https://issues.apache.org/jira/browse/IGNITE-6278 
> 
> 
> 
> 

Re: Ignite Benchmarking page on readme.io

[Dmitriy Setrakyan]

Several comments:

   1. I would place this page directly under the top Basic Concepts section
   2. I would rename the link to Benchmarking (performance is assumed)
   3. I would add a few benchmark runs with examples graphs there

D.

On Mon, Sep 11, 2017 at 5:03 PM, Denis Magda  wrote:

> I’ve done the primarily review and rename the page address. That’s it:
> https://apacheignite.readme.io/docs/perfomance-benchmarking
>
> Prachi, please do the final review. The page is already online.
>
> —
> Denis
>
> On Sep 8, 2017, at 10:40 AM, Aleksei Zaitsev 
> wrote:
>
> Thanks. Here is a direct link on the new page about Ignite
> Benchmarking(hidden for now): https://apacheignite.readme.
> io/v2.1/docs/perfomance-tests. Please review.
> Mostly it was aggregated from the information in the repo
>
> 07.09.2017, 20:57, "Dmitriy Setrakyan" :
>
> Actually, it can be contributed directly to readme.io. Alexey, I just
> sent you an invite.
>
> You can create a new page for benchmarking and make it invisible for now,
> while you are working on it. If you need us to review it, please send a
> direct link to it here.
>
> D.
>
> On Thu, Sep 7, 2017 at 8:26 AM, Anton Vinogradov 
> wrote:
>
> Aleksei,
>
> Good idea.
> Could you please prepare page in MD format and attach link to branch
> containing it to the issue to review before publication?
>
> On Thu, Sep 7, 2017 at 1:45 PM, Aleksei Zaitsev 
> wrote:
>
> Hi, guys.
>
> Issue[1] stated that we have not any information about Apache Ignite
> benchmarking on our site https://apacheignite.readme.io. I want to add
> such a page, but I can’t find appropriate section for it. I suggest to make
> new section named ‘Tests and Benchmarking’ and put all information about
> benchmaring in section page ‘Perfomance Tests’
>
> Thoughts?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-6278
>
>
>

Re: Start contributing to Apache Ignite

[Denis Magda]

Mikhail, welcome!

Granted you required permissions! Look forward to your contributions!

—
Denis

> On Sep 11, 2017, at 4:33 AM, Mikhail Lipkovich  
> wrote:
> 
> Hello,
> 
> I would like to start contributing to Ignite.
> This issue seems suitable for me
> https://issues.apache.org/jira/browse/IGNITE-6284
> 
> Can I get permissions to assign issues to myself?
> 
> My username at issues.apache.org is mlipkovich
> Full name is Mikhail Lipkovich
> 
> 
> 
> Thanks,
> 
> Mikhail

Re: Ignite Benchmarking page on readme.io

[Denis Magda]

I’ve done the primarily review and rename the page address. That’s it:
https://apacheignite.readme.io/docs/perfomance-benchmarking 


Prachi, please do the final review. The page is already online.

—
Denis

> On Sep 8, 2017, at 10:40 AM, Aleksei Zaitsev  wrote:
> 
> Thanks. Here is a direct link on the new page about Ignite 
> Benchmarking(hidden for now): 
> https://apacheignite.readme.io/v2.1/docs/perfomance-tests. Please review.
> Mostly it was aggregated from the information in the repo
> 
> 07.09.2017, 20:57, "Dmitriy Setrakyan" :
>> Actually, it can be contributed directly to readme.io. Alexey, I just sent 
>> you an invite.
>> 
>> You can create a new page for benchmarking and make it invisible for now, 
>> while you are working on it. If you need us to review it, please send a 
>> direct link to it here.
>> 
>> D.
>> 
>> On Thu, Sep 7, 2017 at 8:26 AM, Anton Vinogradov  
>> wrote:
>>> Aleksei,
>>> 
>>> Good idea.
>>> Could you please prepare page in MD format and attach link to branch
>>> containing it to the issue to review before publication?
>>> 
>>> On Thu, Sep 7, 2017 at 1:45 PM, Aleksei Zaitsev 
>>> wrote:
>>> 
 Hi, guys.
 
 Issue[1] stated that we have not any information about Apache Ignite
 benchmarking on our site https://apacheignite.readme.io. I want to add
 such a page, but I can’t find appropriate section for it. I suggest to make
 new section named ‘Tests and Benchmarking’ and put all information about
 benchmaring in section page ‘Perfomance Tests’
 
 Thoughts?
 
 [1] https://issues.apache.org/jira/browse/IGNITE-6278
 

Re: AffinityKeyMapper alternatives

[Valentin Kulichenko]

Even if CacheKeyConfiguration is part of CacheConfiguration, the affinity
key field name can be provided only on cache startup. In many cases this
name can be resolved only based on the actual key instance, e.g. during
first put. Per my understanding, this already works with annotation, I just
propose more flexible solution for rare cases when annotation can't be
used. Basically, the logic we currently have would become the default
implementation of the resolver.

-Val

On Mon, Sep 11, 2017 at 4:41 PM, Dmitriy Setrakyan 
wrote:

> On Mon, Sep 11, 2017 at 4:01 PM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > Guys,
> >
> > Some time ago we deprecated AffinityKeyMapper in favor
> > of CacheKeyConfiguration#affinityKeyFieldName and AffinityKeyMapped
> > annotation. While I understand the reasons why we did this, I think it's
> > not very flexible as requires to specify the field name on node startup.
> >
> > First of all, CacheKeyConfiguration is set on IgniteConfiguration, but
> not
> > CacheConfiguration. Does anyone knows why? How can I specify the affinity
> > field name if I create a new cache dynamically?
> >
>
> Ouch... really? Just looking at the name of the configuration class, I
> would assume that it belongs to CacheConfiguration. Can this be fixed?
>
>
> >
> > Second of all, AffinityKeyMapped doesn't always work. There are cases
> when
> > model classes can't be modified with Ignite annotations, for example. For
> > this case I suggest to introduce something like
> > AffinityKeyFieldNameResolver that will allow to implement custom logic
> > instead. Of course, it will work in the same way as annotation, i.e.
> > invoked on client side only. Is this possible?
> >
>
> But wouldn't CacheKeyConfiguration provide this information if it was the
> property of the CacheConfiguration and not IgniteConfiguration? I don't
> think we need a resolver.
>

Re: AffinityKeyMapper alternatives

[Dmitriy Setrakyan]

On Mon, Sep 11, 2017 at 4:01 PM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Guys,
>
> Some time ago we deprecated AffinityKeyMapper in favor
> of CacheKeyConfiguration#affinityKeyFieldName and AffinityKeyMapped
> annotation. While I understand the reasons why we did this, I think it's
> not very flexible as requires to specify the field name on node startup.
>
> First of all, CacheKeyConfiguration is set on IgniteConfiguration, but not
> CacheConfiguration. Does anyone knows why? How can I specify the affinity
> field name if I create a new cache dynamically?
>

Ouch... really? Just looking at the name of the configuration class, I
would assume that it belongs to CacheConfiguration. Can this be fixed?


>
> Second of all, AffinityKeyMapped doesn't always work. There are cases when
> model classes can't be modified with Ignite annotations, for example. For
> this case I suggest to introduce something like
> AffinityKeyFieldNameResolver that will allow to implement custom logic
> instead. Of course, it will work in the same way as annotation, i.e.
> invoked on client side only. Is this possible?
>

But wouldn't CacheKeyConfiguration provide this information if it was the
property of the CacheConfiguration and not IgniteConfiguration? I don't
think we need a resolver.

AffinityKeyMapper alternatives

[Valentin Kulichenko]

Guys,

Some time ago we deprecated AffinityKeyMapper in favor
of CacheKeyConfiguration#affinityKeyFieldName and AffinityKeyMapped
annotation. While I understand the reasons why we did this, I think it's
not very flexible as requires to specify the field name on node startup.

First of all, CacheKeyConfiguration is set on IgniteConfiguration, but not
CacheConfiguration. Does anyone knows why? How can I specify the affinity
field name if I create a new cache dynamically?

Second of all, AffinityKeyMapped doesn't always work. There are cases when
model classes can't be modified with Ignite annotations, for example. For
this case I suggest to introduce something like
AffinityKeyFieldNameResolver that will allow to implement custom logic
instead. Of course, it will work in the same way as annotation, i.e.
invoked on client side only. Is this possible?

-Val

Re: How to configure new thin client connector on the server?

[Dmitriy Setrakyan]

In my opinion all clients, including JDBC, ODBC, and REST should be
implemented over the same protocol and we should not have the protocol mess
we have today.

Vladimir, does your suggestion about ODBC and JDBC include migration to the
new client?

D.

On Mon, Sep 11, 2017 at 12:39 AM, Vladimir Ozerov 
wrote:

> Pavel,
>
> Note that this would not be renames, but rather copy+deprecated. Otherwise
> we will break compilation of existing applications.
>
> On Mon, Sep 11, 2017 at 10:32 AM, Pavel Tupitsyn 
> wrote:
>
> > Renaming SqlConnectorConfiguration to ClientConnectorConfiguration seems
> > reasonable to me.
> >
> > On Mon, Sep 11, 2017 at 10:28 AM, Vladimir Ozerov 
> > wrote:
> >
> > > Igniters,
> > >
> > > As you know we are working on a new thin client protocol [1]. Core
> logic
> > > for base cache operations is ready, so now it is time to discuss how to
> > > configure it on the server. Things are a bit complicated here.
> > >
> > > First, we have connector configuration for our old so-called REST
> client,
> > > see ConnectorConfiguration and IgniteConfiguraion.
> > connectorConfiguration.
> > > We cannot re-use it because it is very heavy-weight and contains a lot
> of
> > > properties which are simply inapplicable to the new client. E.g.
> > > SSL-related stuff, Jetty configs, etc..
> > >
> > > Second, we have connector configuration for our thin SQL drivers (JDBC,
> > > ODBC). See SqlConnectorConfiguration and
> > > IgniteConfiguration.sqlConnectorConfiguration. This configuration is
> > fine
> > > for the new thin client, but it's name is inappropriate.
> > >
> > > Essentially, we should rename our SqlConnectorConfiguration to
> something
> > > more generic, but this "something" cannot be ConnectorConfiguration.
> > Let's
> > > brainstorm how to name it.
> > >
> > > My ideas:
> > > - ClientConnectorConfiguration - preferrable
> > > - ListenerConfiguration
> > > - EndpointConfiguration
> > >
> > > Any more ideas?
> > >
> > > Vladimir.
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-5896
> > >
> >
>

[jira] [Created] (IGNITE-6346) Distributed set does not work in REPLICATED mode

[Valentin Kulichenko (JIRA)]

Valentin Kulichenko created IGNITE-6346:
---

 Summary: Distributed set does not work in REPLICATED mode
 Key: IGNITE-6346
 URL: https://issues.apache.org/jira/browse/IGNITE-6346
 Project: Ignite
  Issue Type: Bug
  Components: data structures
Affects Versions: 2.1
Reporter: Valentin Kulichenko
 Fix For: 2.2


When {{IgniteSet}} is created with {{REPLICATED}} cache mode, any attempt to 
read it fails with exception:
{noformat}
Exception in thread "main" class org.apache.ignite.IgniteException: Cluster 
group is empty.
at 
org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:48)
at ReplicatedSet.main(ReplicatedSet.java:36)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: class 
org.apache.ignite.internal.cluster.ClusterGroupEmptyCheckedException: Cluster 
group is empty.
at 
org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter.execute0(GridCacheQueryAdapter.java:481)
at 
org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter.execute(GridCacheQueryAdapter.java:442)
at 
org.apache.ignite.internal.processors.datastructures.GridCacheSetImpl.iterator0(GridCacheSetImpl.java:420)
at 
org.apache.ignite.internal.processors.datastructures.GridCacheSetImpl.iterator(GridCacheSetImpl.java:375)
at 
org.apache.ignite.internal.processors.datastructures.GridCacheSetProxy.iterator(GridCacheSetProxy.java:342)
... 6 more
{noformat}
To reproduce run the following code:
{code}
Ignition.start(new IgniteConfiguration().setIgniteInstanceName("server-1"));
Ignition.start(new IgniteConfiguration().setIgniteInstanceName("server-2"));

Ignition.setClientMode(true);

Ignite ignite = Ignition.start();

IgniteSet set = ignite.set("my-set", new 
CollectionConfiguration().setCacheMode(CacheMode.REPLICATED));

for (String s : set) {
System.out.println(s);
}
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: IGNITE-5655: Mixing binary string encodings in Ignite cluster

[Dmitriy Setrakyan]

Vova, generally agree, but why not also support per-cache (per-table)
settings?


On Mon, Sep 11, 2017 at 1:16 AM, Vladimir Ozerov 
wrote:

> OK, here is implementation plan I propose:
> 1) Add global character set configuration -
> IgniteConfiguration.characterSet. Note, it is located in
> IgniteConfiguration, not BinaryConfiguration.
> 2) All cluster nodes must have the same character set.
> 3) Once defined, character set cannot be changed ever. In future we will
> probably have import/export utilities, which will help users migrate
> between character sets. Such strict behavior is normal for other major DBMS
> vendors (e.g. Oracle), so it should work for us as well.
> 4) We will add "characterSet" property to all clients (ODBC, JDBC, thin
> client). It will be validated during handshake phase. Exception is thrown
> in case of mismatch.
> 5) In future we will work on relaxing these restrictions in favor of
> runtime conversions on fly.
>
> Thoughts?
>
>
>
> On Mon, Sep 11, 2017 at 11:01 AM, Vladimir Ozerov 
> wrote:
>
> > Dima,
> >
> > You contradict yourself - vote for per-column encoding on the one hand,
> > but telling that it is "over-architected" on the other. This is exactly
> > what I am talking about - anything more that hard-coded cluster-wide
> > encoding is complex. You cannot simply define per-column encoding. In
> > addition you should either pass information about this encoding too all
> > cluster members and to all clients, so that they construct correct binary
> > object in the first place, or you should re-convert binary object on fly,
> > this is what I suggested. No simple solution here.
> >
> > I vote for cluster-wide encoding for now, but with transparent conversion
> > when needed.
> >
> >
> > On Thu, Sep 7, 2017 at 4:50 AM, Dmitriy Setrakyan  >
> > wrote:
> >
> >> I would agree with Andrey, it does look a bit over-architected to me.
> Why
> >> would anyone try to move data from one encoding to another? Is it a real
> >> use case that needs to be handled automatically?
> >>
> >> Here is what I think we should handle:
> >>
> >>1. Ability to set cluster-wide encoding. This should be easy.
> >>2. Ability to set per-column encoding. Such encoding should be set on
> >>per-column level, perhaps at cache creation or table creation. For
> >> example,
> >>at the cache creation time, we could let user define all column names
> >> that
> >>will have non-default encodings.
> >>
> >> Thoughts?
> >>
> >> D.
> >>
> >> On Wed, Sep 6, 2017 at 6:27 AM, Andrey Kuznetsov 
> >> wrote:
> >>
> >> > As of option #1, it's not so bad. Currently we've implemented global
> >> level
> >> > encoding switch, and this looks similar to DBMS: if server works with
> >> > certain encoding, then all clients should be configured to use the
> same
> >> > encoding for correct string processing.
> >> >
> >> > Option #2 provokes a number of questions.
> >> >
> >> > What are performance implications of such hidden binary reencoding?
> >> >
> >> > Who will check for possible data loss on transparent reencoding (when
> >> > object walks between caches/fields with distinct encodings)?
> >> >
> >> > How should we handle nested binary objects? On the one hand, they
> >> should be
> >> > reencoded in a way described by Vladimir. On the other hand,
> >> BinaryObject
> >> > is an independent entity, that can be serialized/deserialized freely,
> >> moved
> >> > between various data structures, etc. It will be frustrating for user
> to
> >> > find its binary state changed after storing in a grid, with possible
> >> data
> >> > corruption.
> >> >
> >> >
> >> > As far as I can see, we are trying to couple orthogonal APIs:
> >> > BinaryMarshaller, IgniteCache and SQL. BinaryMarshaller is
> >> > Java-datatype-driven, it creates 1-to-1 mapping between Java types and
> >> > their binary representations, and now we are trying to map two binary
> >> types
> >> > (STRING and ENCODED_STRING) to single String class. IgniteCache is
> much
> >> > more flexible API, than SQL, but it lacks encoded string datatype,
> that
> >> > exists in SQLs of some RDBMSs: `varchar(n) character set
> some_charset`.
> >> > It's not a popular idea, but many problems could be solved by adding
> >> such
> >> > type. Those IgniteCache API users who don't need it won't use it, but
> it
> >> > could become a bridge between SQL and BinaryMarshaller encoded-string
> >> > types.
> >> >
> >> > 2017-09-06 10:32 GMT+03:00 Vladimir Ozerov :
> >> >
> >> > > What we tried to achieve is that several encoding could co-exist in
> a
> >> > > single cluster or even single cache. This would be great from UX
> >> > > perspective. However, from what Andrey wrote, I understand that this
> >> > would
> >> > > be pretty hard to achieve as we rely heavily on similar binary
> >> > > representation of objects being compared. That said, while this
> could
> >> > work

Re: NPE from service processor

[Dmitriy Setrakyan]

Well... would be nice to see a response on Stack Overflow. So far, there is
silence there.

D.

On Mon, Sep 11, 2017 at 3:11 AM, Denis Mekhanikov 
wrote:

> I can only say that recent changes introduced by IGNITE-5145
>  in
> GridServiceProcessor
> shouldn't have changed this behavior in any way.
> It must be somehow connected with a lifecycle of IgniteKernal and other
> components. A reproducing example would help to find the problem.
>
> - Denis
>
> пн, 11 сент. 2017 г. в 1:32, Dmitriy Setrakyan :
>
> > Igniters,
> >
> > I am seeing an NPE reported from the GridServiceProcessor class here:
> >
> > https://stackoverflow.com/questions/46144413/apache-
> ignite-cluster-does-not-start-with-persistent-storage
> >
> > Has this been fixed?
> >
> > D.
> >
>

Specifying schema name with thin JDBC driver

[Valentin Kulichenko]

Guys,

Is there a way to specify default schema name when connecting to a cluster
using thin JDBC driver? This seems to be possible with other versions of
JDBC driver, but not with thin one. This forces users to prefix each table
with cache name in all queries, which is not very usable. Do we have any
plans to improve this?

Looks like this user faced this exact problem:
https://stackoverflow.com/questions/46158641/apache-ignite-jdbc-thin-client-does-not-work-with-existing-cache

-Val

[jira] [Created] (IGNITE-6345) Wrong message about cluster activation

[Evgenii Zhuravlev (JIRA)]

Evgenii Zhuravlev created IGNITE-6345:
-

 Summary: Wrong message about cluster activation
 Key: IGNITE-6345
 URL: https://issues.apache.org/jira/browse/IGNITE-6345
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.1
Reporter: Evgenii Zhuravlev
Priority: Minor
 Fix For: 2.3


Message of an exception that threw while performing an operation on inactive 
cluster relates to method Ignite.activate(true), which doesn't exist. 

Method in message should be changed to Ignite.active(true)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2639: IGNITE-6314 Tests fix.

[alexpaschenko]

GitHub user alexpaschenko opened a pull request:

https://github.com/apache/ignite/pull/2639

IGNITE-6314 Tests fix.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6314

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2639.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2639


commit 18db626e5c8032084205a70b4542a0fd6c55df30
Author: Alexander Paschenko 
Date:   2017-09-11T16:53:29Z

IGNITE-6314 Tests fix.




---

Re: Deprecate IgniteCache.query(Query, IgniteClosure<T,R>)

[Valentin Kulichenko]

Vova,

I doubt you'll be able to easily add it directly to ScanQuery because of
generics. Do you have an alternative API in mind?

-Val

On Mon, Sep 11, 2017 at 7:32 AM, Dmitriy Setrakyan 
wrote:

> On Mon, Sep 11, 2017 at 7:13 AM, Vladimir Ozerov 
> wrote:
>
> > Igniters,
> >
> > We have that very strange method, which accepts transformer. It is only
> > supported for ScanQuery and we do not have any plans to extend it in
> > future.
> >
> > Let's just deprecate it and add transformer to ScanQuery directly.
> >
> > Thoughts?
> >
>
> Vladimir, what is wrong with this method?
>

[jira] [Created] (IGNITE-6344) AWS AMI startup.sh gets spurious error on export statment

[David Harvey (JIRA)]

David Harvey created IGNITE-6344:


 Summary: AWS AMI startup.sh gets spurious error on export statment
 Key: IGNITE-6344
 URL: https://issues.apache.org/jira/browse/IGNITE-6344
 Project: Ignite
  Issue Type: Bug
  Components: aws
Reporter: David Harvey
Priority: Minor


The export statement in this script should look like
export "$p"
rather than
export $p
or removed altogether

If something with blanks is specified, like
JVM_OPTS=-Xg1 -Xg2
the script reports errors because it splits the line on blanks.  However, the 
result of the export statement is not used, so the basic AMI works.   But it 
you are debugging another issue, you get very confused by this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2638: IGNITE-6151: Impemented getting and setting of th...

[isapego]

GitHub user isapego opened a pull request:

https://github.com/apache/ignite/pull/2638

IGNITE-6151: Impemented getting and setting of the SQL_ATTR_CONNECTION_DEAD



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6151

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2638.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2638


commit b710366f8cf58f0fb5b598150601b016217489e9
Author: Igor Sapego 
Date:   2017-09-11T15:05:17Z

IGNITE-6151: Impemented getting and setting of the SQL_ATTR_CONNECTION_DEAD

commit b7cd0bac4ef140c51b08fcb9edce6fc820c1dd64
Author: Igor Sapego 
Date:   2017-09-11T16:13:15Z

IGNITE-6151: Added tests




---

[GitHub] ignite pull request #2362: IGNITE-5621: Support BINARY and VARBINARY SQL typ...

[isapego]

Github user isapego closed the pull request at:

https://github.com/apache/ignite/pull/2362


---

[GitHub] ignite pull request #2543: IGNITE-6119: Added 'lazy' flag to ODBC

[isapego]

Github user isapego closed the pull request at:

https://github.com/apache/ignite/pull/2543


---

[GitHub] ignite pull request #2468: IGNITE-3987: ODBC: Improved error output when que...

[isapego]

Github user isapego closed the pull request at:

https://github.com/apache/ignite/pull/2468


---

[GitHub] ignite pull request #2573: IGNITE-6211: ODBC: SQLBindParameter now does not ...

[isapego]

Github user isapego closed the pull request at:

https://github.com/apache/ignite/pull/2573


---

[GitHub] ignite pull request #2581: IGNITE-6188: ODBC: Fix for SQLFreeStmt(SQL_CLOSE)...

[isapego]

Github user isapego closed the pull request at:

https://github.com/apache/ignite/pull/2581


---

[GitHub] ignite pull request #2601: IGNITE-6245: Implemented returning of the number ...

[isapego]

Github user isapego closed the pull request at:

https://github.com/apache/ignite/pull/2601


---

[GitHub] ignite pull request #2612: IGNITE-6282: Implemented lazy initialization of I...

[isapego]

Github user isapego closed the pull request at:

https://github.com/apache/ignite/pull/2612


---

[jira] [Created] (IGNITE-6343) Index is not used properly if changing sort order.

[Alexei Scherbakov (JIRA)]

Alexei Scherbakov created IGNITE-6343:
-

 Summary: Index is not used properly if changing sort order.
 Key: IGNITE-6343
 URL: https://issues.apache.org/jira/browse/IGNITE-6343
 Project: Ignite
  Issue Type: Improvement
Affects Versions: 2.0
Reporter: Alexei Scherbakov
 Fix For: 2.3


Unit test reproducer:

{noformat}
/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *  http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.apache.ignite.internal.processors.cache;

import java.util.Calendar;
import java.util.Collections;
import java.util.Date;
import java.util.LinkedHashMap;
import java.util.List;
import org.apache.ignite.IgniteCache;
import org.apache.ignite.cache.CacheMode;
import org.apache.ignite.cache.QueryEntity;
import org.apache.ignite.cache.QueryIndex;
import org.apache.ignite.cache.QueryIndexType;
import org.apache.ignite.cache.query.SqlFieldsQuery;
import org.apache.ignite.configuration.CacheConfiguration;
import org.apache.ignite.configuration.IgniteConfiguration;
import org.apache.ignite.configuration.MemoryConfiguration;
import org.apache.ignite.configuration.MemoryPolicyConfiguration;
import org.apache.ignite.internal.util.typedef.internal.U;
import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
import org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinder;
import org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;

import static org.apache.ignite.cache.CacheMode.PARTITIONED;
import static org.apache.ignite.cache.CacheWriteSynchronizationMode.FULL_SYNC;
import static java.util.Calendar.*;

/**
 * Tests for cache query results serialization.
 */
public class GridCacheQueryIndexUsageSelfTest extends GridCommonAbstractTest {
/** */
private static final int GRID_CNT = 1;

/** */
private static final String CACHE_NAME = "A";

/** */
private static final CacheMode CACHE_MODE = PARTITIONED;

/** */
private static TcpDiscoveryIpFinder ipFinder = new 
TcpDiscoveryVmIpFinder(true);

/** {@inheritDoc} */
@Override protected void beforeTest() throws Exception {
startGridsMultiThreaded(GRID_CNT);
}

/** {@inheritDoc} */
@Override protected void afterTest() throws Exception {
stopAllGrids();
}

/** {@inheritDoc} */
@Override protected IgniteConfiguration getConfiguration(String 
igniteInstanceName) throws Exception {
IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);

MemoryPolicyConfiguration mpcfg = new MemoryPolicyConfiguration();
//mpcfg.setMaxSize(2 * 1024 * 1024 * 1024L);
mpcfg.setName("def");

MemoryConfiguration mcfg = new MemoryConfiguration();
mcfg.setDefaultMemoryPolicyName("def");
mcfg.setMemoryPolicies(mpcfg);

cfg.setMemoryConfiguration(mcfg);

TcpDiscoverySpi disco = new TcpDiscoverySpi();

disco.setIpFinder(ipFinder);

cfg.setDiscoverySpi(disco);

CacheConfiguration cacheCfg = defaultCacheConfiguration();

cacheCfg.setName(CACHE_NAME);
cacheCfg.setCacheMode(CACHE_MODE);
cacheCfg.setWriteSynchronizationMode(FULL_SYNC);

QueryEntity qe = new QueryEntity();
qe.setKeyType(Long.class.getName());
qe.setValueType(IndexedValue.class.getName());

LinkedHashMap fields = U.newLinkedHashMap(3);

fields.put("id", Long.class.getName());
fields.put("startDate", Date.class.getName());
qe.setFields(fields);

QueryIndex idx = new QueryIndex();
idx.setIndexType(QueryIndexType.SORTED);
LinkedHashMap idxFields = U.newLinkedHashMap(3);

idxFields.put("startDate", Boolean.TRUE);

idx.setFields(idxFields);

qe.setIndexes(Collections.singleton(idx));

cacheCfg.setQueryEntities(Collections.singleton(qe));

cfg.setCacheConfiguration(cacheCfg);

return cfg;
}

/** */
public void testIndexUsageAscSort() {
testIndexUsage0(true);
}

/** */
public void testIndexUsageDescSort() {
  

[jira] [Created] (IGNITE-6342) Exchange is hanging on eviction

[Eduard Shangareev (JIRA)]

Eduard Shangareev created IGNITE-6342:
-

 Summary: Exchange is hanging on eviction
 Key: IGNITE-6342
 URL: https://issues.apache.org/jira/browse/IGNITE-6342
 Project: Ignite
  Issue Type: Bug
Reporter: Eduard Shangareev


I wrote test which stop/start random node, the test hangs time to time on 
exchange.
As I see, exchange-thread waiting for partition eviction.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2637: jdkMarshaller(String nodeName) didn't set nodeNam...

[ahumellihuk]

GitHub user ahumellihuk opened a pull request:

https://github.com/apache/ignite/pull/2637

jdkMarshaller(String nodeName) didn't set nodeName

Node name was being set on a new instance of JdkMarshaller, not the one 
being returned

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ahumellihuk/ignite master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2637.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2637






---

Re: Deprecate IgniteCache.query(Query, IgniteClosure<T,R>)

[Dmitriy Setrakyan]

On Mon, Sep 11, 2017 at 7:13 AM, Vladimir Ozerov 
wrote:

> Igniters,
>
> We have that very strange method, which accepts transformer. It is only
> supported for ScanQuery and we do not have any plans to extend it in
> future.
>
> Let's just deprecate it and add transformer to ScanQuery directly.
>
> Thoughts?
>

Vladimir, what is wrong with this method?

[jira] [Created] (IGNITE-6341) WAL: Use direct IO or libaio for writer thread

[Andrey Gura (JIRA)]

Andrey Gura created IGNITE-6341:
---

 Summary: WAL: Use direct IO or libaio for writer thread
 Key: IGNITE-6341
 URL: https://issues.apache.org/jira/browse/IGNITE-6341
 Project: Ignite
  Issue Type: Improvement
  Components: persistence
Reporter: Andrey Gura
Assignee: Andrey Gura


Need try to use direct IO or libaio for WAL writer thread because once data 
buffer is serialized, we can write it as disk pages (this will also make it 
easier if we decide to open file as a block device).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Deprecate IgniteCache.query(Query, IgniteClosure<T,R>)

[Николай Ижиков]

+1 from me as I proposed similar design in my implementation of
ContinuousQueryWithTransformer [1]

https://issues.apache.org/jira/browse/IGNITE-425


2017-09-11 17:13 GMT+03:00 Vladimir Ozerov :

> Igniters,
>
> We have that very strange method, which accepts transformer. It is only
> supported for ScanQuery and we do not have any plans to extend it in
> future.
>
> Let's just deprecate it and add transformer to ScanQuery directly.
>
> Thoughts?
>



-- 
Nikolay Izhikov
nizhikov@gmail.com

[GitHub] ignite pull request #2636: For testing.

[ilantukh]

GitHub user ilantukh opened a pull request:

https://github.com/apache/ignite/pull/2636

For testing.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.1.5-6340

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2636.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2636


commit 1e08c3fb5c02ec8acafd71b50b6ad3b749259f1a
Author: Andrey V. Mashenkov 
Date:   2017-07-31T11:14:56Z

IGNITE-4800: Lucene query may fails with NPE. This closes #2315.

commit 3fdf453e89a7bd76dff6b6d0646e3821ea3921d5
Author: Andrey V. Mashenkov 
Date:   2017-07-31T14:32:12Z

IGNITE-4800: Lucene query may fails with NPE.
Test fixed.

commit e255a564985a12113984ec02f15a4443495b8ffc
Author: Nikolay Izhikov 
Date:   2017-08-02T08:52:44Z

ignite-5712 Context switching for optimistic transactions

commit 772d462b68c7de8517d1f61e2e05ec8eefb18eac
Author: Alexey Kuznetsov 
Date:   2017-08-03T04:55:15Z

Merge branch ignite-2.1.3 into ignite-2.1.4

commit 0f3b7ca25313083e4dc35e7842931a655abd
Author: tledkov-gridgain 
Date:   2017-08-04T08:46:14Z

IGNITE-5126: Batch support for this JDBC driver. This closes #2162.

commit d1a74a4be8744528e6ed23706174041ddb0f2618
Author: devozerov 
Date:   2017-08-04T09:04:38Z

Merge remote-tracking branch 'upstream/ignite-2.1.4' into ignite-2.1.4

commit 0b3a9a7176f5ae44a96ecf700c8147193dfbf064
Author: Igor Sapego 
Date:   2017-08-04T10:18:00Z

IGNITE-5923: ODBC: SQLGetTypeInfo now works with SQL_ALL_TYPES

(cherry picked from commit 48c914d)

commit 4e0385fbc0f50548f2da3407fdfdfe939b463c67
Author: Igor Sapego 
Date:   2017-08-04T15:34:27Z

IGNITE-5939: ODBC: SQLColAttributes now works with legacy attribute codes.

(cherry picked from commit 70ffa2c)

commit 4f02504475fd1e5cc3b9f4754856e44d20fdc1cb
Author: Alexey Kuznetsov 
Date:   2017-08-07T02:41:22Z

Merge branch ignite-2.1.3 into ignite-2.1.4.

commit b093afb8231135a4904f5fffd62f5b4c332f1d47
Author: tledkov-gridgain 
Date:   2017-08-08T09:05:36Z

IGNITE-5211: Added new constructor: QueryEntity(Class keyCls, Class 
valCls). This closes #2371. This closes #2388. This closes #2407.

commit 879f19106b22e66d5f6ea94424d961d049397410
Author: devozerov 
Date:   2017-08-08T12:16:58Z

IGNITE-5982: GridMapQueryExecutor was split into several pieces.

commit 7c77b869bc3efdf19e53cc2b064f4290fd73e2b2
Author: devozerov 
Date:   2017-08-09T08:47:58Z

IGNITE-5993: Removed unused SQL-related classes and methods (old tree 
index, snapshots, etc). This closes #2414.

commit ab18fdfcc4f6db1e54fb1f3b68ba7fbc31a7f6e7
Author: Andrey V. Mashenkov 
Date:   2017-07-14T17:14:47Z

IGNITE-5452: GridTimeoutProcessor can hang on stop. This closes #2279.

commit 580b6aa8e5a8a887397eab5c4c830ec28f45cd30
Author: Alexey Kuznetsov 
Date:   2017-08-09T10:22:54Z

IGNITE-5734 Web Console: Fixed npm dependencies.
(cherry picked from commit aeafbf1)

commit 841db65e56063605475710bc170de4aea672c31d
Author: Alexey Kuznetsov 
Date:   2017-08-09T11:55:04Z

IGNITE-5987 Added -nq (visor will not quit in batch mode) option for Visor 
Cmd.
(cherry picked from commit 8d6e842)

commit 5c2097856714a7803956d754735c68b21156019c
Author: Alexey Kuznetsov 
Date:   2017-08-11T03:25:36Z

IGNITE-5902 Implemented stop caches at once.
(cherry picked from commit ebb8765)

commit 8b2461942c18f228c0107020aa28c03711bdceda
Author: Alexey Kuznetsov 
Date:   2017-08-11T04:07:26Z

IGNITE-6012 Refactored GridJettyRestHandler.processRequest(): replace 
mapper.writeValueAsString with writeValue(stream, v).
(cherry picked from commit 3a390c8)

commit 3a7d4f4a79e7c0a23244387e3a68535375a66a87
Author: Alexey Kuznetsov 
Date:   2017-08-11T04:18:42Z

IGNITE-6013 Optimized processing response from cluster.
(cherry picked from commit b02c481)

commit 74d6ab9916b3a01c78cdf1ad86211c9fcbb2214d
Author: Alexey Goncharuk 
Date:   2017-08-14T08:08:28Z

Merge branch 'ignite-2.1.3' into ignite-2.1.4

commit fde550bac56fd0cc7c51c62a9c291dd4c3f3030c
Author: Ilya Lantukh 
Date:   2017-08-14T08:32:11Z

IGNITE-5941 - Fixed index name length restrictions. This closes #2408

commit 13f38d79b57b395e43d42a8f3c278cf48336d7c5
Author: Dmitriy Govorukhin 
Date:   

[GitHub] ignite pull request #2635: IGNITE-6340 : Fixed creation of client cache on s...

[ilantukh]

GitHub user ilantukh opened a pull request:

https://github.com/apache/ignite/pull/2635

IGNITE-6340 : Fixed creation of client cache on server node.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6340

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2635.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2635


commit 8b6f7dfe5b87154c77cd782be2597df2248700df
Author: Ilya Lantukh 
Date:   2017-09-11T14:07:43Z

ignite-6340 : Fixed creation of client cache on server node.

commit 4852c80d095767d94326ce8f894541ea8ecb85ee
Author: Ilya Lantukh 
Date:   2017-09-11T14:14:50Z

ignite-6340 : fixed code style.




---

Deprecate IgniteCache.query(Query, IgniteClosure<T,R>)

[Vladimir Ozerov]

Igniters,

We have that very strange method, which accepts transformer. It is only
supported for ScanQuery and we do not have any plans to extend it in
future.

Let's just deprecate it and add transformer to ScanQuery directly.

Thoughts?

[jira] [Created] (IGNITE-6340) Attempting to create the same cache from multiple threads leads to creation of client cache on server node

[Ilya Lantukh (JIRA)]

Ilya Lantukh created IGNITE-6340:


 Summary: Attempting to create the same cache from multiple threads 
leads to creation of client cache on server node
 Key: IGNITE-6340
 URL: https://issues.apache.org/jira/browse/IGNITE-6340
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 2.1
Reporter: Ilya Lantukh
Assignee: Ilya Lantukh






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6339) WAL: Avoid closed by interruption exception when user thread is interrupted

[Andrey Gura (JIRA)]

Andrey Gura created IGNITE-6339:
---

 Summary: WAL: Avoid closed by interruption exception when user 
thread is interrupted
 Key: IGNITE-6339
 URL: https://issues.apache.org/jira/browse/IGNITE-6339
 Project: Ignite
  Issue Type: Bug
  Components: persistence
Reporter: Andrey Gura
Assignee: Andrey Gura
 Fix For: 2.3


We should have a separate writer thread for WAL that will write completed 
serialized chunks of data. This will allow us avoid closed by interruption 
exception when user thread is interrupted.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [VOTE] Apache Ignite 2.2.0 RC1

[Николай Ижиков]

+1

2017-09-11 16:31 GMT+03:00 Anton Vinogradov :

> Igniters,
>
> We have uploaded a 2.2.0 release candidate to
> https://dist.apache.org/repos/dist/dev/ignite/2.2.0-rc1/
>
> Git tag name is
> 2.2.0-rc1
>
> This release includes the following changes:
>
> Ignite:
> * Checkpointing algorithm optimized
> * Default max memory size changed from 80% to 20%
>
> Ignite CPP:
> * Now possible to start node with persistent store
> * Ignite.setActive method added to C++ API
>
> Complete list of closed issues:
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%
> 20fixVersion%20%3D%202.2%20AND%20(status%20%3D%
> 20closed%20or%20status%20%3D%20resolved)
>
> DEVNOTES
> https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_
> plain;f=DEVNOTES.txt;hb=refs/tags/2.2.0-rc1
>
> RELEASE NOTES
> https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_
> plain;f=RELEASE_NOTES.txt;hb=refs/tags/2.2.0-rc1
>
> Please start voting.
>
> +1 - to accept Apache Ignite 2.2.0-RC1
> 0 - don't care either way
> -1 - DO NOT accept Apache Ignite 2.2.0-RC1 (explain why)
>
> This vote will go for 72 hours.
>



-- 
Nikolay Izhikov
nizhikov@gmail.com

[VOTE] Apache Ignite 2.2.0 RC1

[Anton Vinogradov]

Igniters,

We have uploaded a 2.2.0 release candidate to
https://dist.apache.org/repos/dist/dev/ignite/2.2.0-rc1/

Git tag name is
2.2.0-rc1

This release includes the following changes:

Ignite:
* Checkpointing algorithm optimized
* Default max memory size changed from 80% to 20%

Ignite CPP:
* Now possible to start node with persistent store
* Ignite.setActive method added to C++ API

Complete list of closed issues:
https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%202.2%20AND%20(status%20%3D%20closed%20or%20status%20%3D%20resolved)

DEVNOTES
https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_plain;f=DEVNOTES.txt;hb=refs/tags/2.2.0-rc1

RELEASE NOTES
https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_plain;f=RELEASE_NOTES.txt;hb=refs/tags/2.2.0-rc1

Please start voting.

+1 - to accept Apache Ignite 2.2.0-RC1
0 - don't care either way
-1 - DO NOT accept Apache Ignite 2.2.0-RC1 (explain why)

This vote will go for 72 hours.

[jira] [Created] (IGNITE-6338) .NET: Thin client: LINQ

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-6338:
--

 Summary: .NET: Thin client: LINQ
 Key: IGNITE-6338
 URL: https://issues.apache.org/jira/browse/IGNITE-6338
 Project: Ignite
  Issue Type: Improvement
  Components: platforms
Reporter: Pavel Tupitsyn
 Fix For: 2.3


Make sure LINQ works via thin client. This requires implementing 
{{ICacheInternal}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6337) .NET: Thin client: SQL queries

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-6337:
--

 Summary: .NET: Thin client: SQL queries
 Key: IGNITE-6337
 URL: https://issues.apache.org/jira/browse/IGNITE-6337
 Project: Ignite
  Issue Type: Improvement
  Components: platforms
Reporter: Pavel Tupitsyn
 Fix For: 2.3


SQL and Fields queries in thin client.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6336) .NET: Thin client: Create cache

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-6336:
--

 Summary: .NET: Thin client: Create cache
 Key: IGNITE-6336
 URL: https://issues.apache.org/jira/browse/IGNITE-6336
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Reporter: Pavel Tupitsyn
 Fix For: 2.3


Create caches from thin client (by name and from {{CacheConfiguration}}).

Implement {{ICache.GetConfiguration}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6335) .NET: Thin client: cache binary mode

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-6335:
--

 Summary: .NET: Thin client: cache binary mode
 Key: IGNITE-6335
 URL: https://issues.apache.org/jira/browse/IGNITE-6335
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Reporter: Pavel Tupitsyn
 Fix For: 2.3


{{WithKeepBinary}}, all operations in binary mode, {{IIgnite.GetBinary()}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2634: For testing only.

[ilantukh]

GitHub user ilantukh opened a pull request:

https://github.com/apache/ignite/pull/2634

For testing only.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.1.5-5549

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2634.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2634


commit 1e08c3fb5c02ec8acafd71b50b6ad3b749259f1a
Author: Andrey V. Mashenkov 
Date:   2017-07-31T11:14:56Z

IGNITE-4800: Lucene query may fails with NPE. This closes #2315.

commit 3fdf453e89a7bd76dff6b6d0646e3821ea3921d5
Author: Andrey V. Mashenkov 
Date:   2017-07-31T14:32:12Z

IGNITE-4800: Lucene query may fails with NPE.
Test fixed.

commit e255a564985a12113984ec02f15a4443495b8ffc
Author: Nikolay Izhikov 
Date:   2017-08-02T08:52:44Z

ignite-5712 Context switching for optimistic transactions

commit 772d462b68c7de8517d1f61e2e05ec8eefb18eac
Author: Alexey Kuznetsov 
Date:   2017-08-03T04:55:15Z

Merge branch ignite-2.1.3 into ignite-2.1.4

commit 0f3b7ca25313083e4dc35e7842931a655abd
Author: tledkov-gridgain 
Date:   2017-08-04T08:46:14Z

IGNITE-5126: Batch support for this JDBC driver. This closes #2162.

commit d1a74a4be8744528e6ed23706174041ddb0f2618
Author: devozerov 
Date:   2017-08-04T09:04:38Z

Merge remote-tracking branch 'upstream/ignite-2.1.4' into ignite-2.1.4

commit 0b3a9a7176f5ae44a96ecf700c8147193dfbf064
Author: Igor Sapego 
Date:   2017-08-04T10:18:00Z

IGNITE-5923: ODBC: SQLGetTypeInfo now works with SQL_ALL_TYPES

(cherry picked from commit 48c914d)

commit 4e0385fbc0f50548f2da3407fdfdfe939b463c67
Author: Igor Sapego 
Date:   2017-08-04T15:34:27Z

IGNITE-5939: ODBC: SQLColAttributes now works with legacy attribute codes.

(cherry picked from commit 70ffa2c)

commit 4f02504475fd1e5cc3b9f4754856e44d20fdc1cb
Author: Alexey Kuznetsov 
Date:   2017-08-07T02:41:22Z

Merge branch ignite-2.1.3 into ignite-2.1.4.

commit b093afb8231135a4904f5fffd62f5b4c332f1d47
Author: tledkov-gridgain 
Date:   2017-08-08T09:05:36Z

IGNITE-5211: Added new constructor: QueryEntity(Class keyCls, Class 
valCls). This closes #2371. This closes #2388. This closes #2407.

commit 879f19106b22e66d5f6ea94424d961d049397410
Author: devozerov 
Date:   2017-08-08T12:16:58Z

IGNITE-5982: GridMapQueryExecutor was split into several pieces.

commit 7c77b869bc3efdf19e53cc2b064f4290fd73e2b2
Author: devozerov 
Date:   2017-08-09T08:47:58Z

IGNITE-5993: Removed unused SQL-related classes and methods (old tree 
index, snapshots, etc). This closes #2414.

commit ab18fdfcc4f6db1e54fb1f3b68ba7fbc31a7f6e7
Author: Andrey V. Mashenkov 
Date:   2017-07-14T17:14:47Z

IGNITE-5452: GridTimeoutProcessor can hang on stop. This closes #2279.

commit 580b6aa8e5a8a887397eab5c4c830ec28f45cd30
Author: Alexey Kuznetsov 
Date:   2017-08-09T10:22:54Z

IGNITE-5734 Web Console: Fixed npm dependencies.
(cherry picked from commit aeafbf1)

commit 841db65e56063605475710bc170de4aea672c31d
Author: Alexey Kuznetsov 
Date:   2017-08-09T11:55:04Z

IGNITE-5987 Added -nq (visor will not quit in batch mode) option for Visor 
Cmd.
(cherry picked from commit 8d6e842)

commit 5c2097856714a7803956d754735c68b21156019c
Author: Alexey Kuznetsov 
Date:   2017-08-11T03:25:36Z

IGNITE-5902 Implemented stop caches at once.
(cherry picked from commit ebb8765)

commit 8b2461942c18f228c0107020aa28c03711bdceda
Author: Alexey Kuznetsov 
Date:   2017-08-11T04:07:26Z

IGNITE-6012 Refactored GridJettyRestHandler.processRequest(): replace 
mapper.writeValueAsString with writeValue(stream, v).
(cherry picked from commit 3a390c8)

commit 3a7d4f4a79e7c0a23244387e3a68535375a66a87
Author: Alexey Kuznetsov 
Date:   2017-08-11T04:18:42Z

IGNITE-6013 Optimized processing response from cluster.
(cherry picked from commit b02c481)

commit 74d6ab9916b3a01c78cdf1ad86211c9fcbb2214d
Author: Alexey Goncharuk 
Date:   2017-08-14T08:08:28Z

Merge branch 'ignite-2.1.3' into ignite-2.1.4

commit fde550bac56fd0cc7c51c62a9c291dd4c3f3030c
Author: Ilya Lantukh 
Date:   2017-08-14T08:32:11Z

IGNITE-5941 - Fixed index name length restrictions. This closes #2408

commit 13f38d79b57b395e43d42a8f3c278cf48336d7c5
Author: Dmitriy Govorukhin 
Date:   

Start contributing to Apache Ignite

[Mikhail Lipkovich]

Hello,

I would like to start contributing to Ignite.
This issue seems suitable for me
https://issues.apache.org/jira/browse/IGNITE-6284

Can I get permissions to assign issues to myself?

My username at issues.apache.org is mlipkovich
Full name is Mikhail Lipkovich



Thanks,

Mikhail

Re: BinaryObject model replace() in client/server topology fails with ClassNotFoundFunction

[Andrey Mashenkov]

Crossposting to dev list.

Guys,
Do we have any ETA for next releases?
I see no dates on release page [1].

[1]
https://issues.apache.org/jira/projects/IGNITE?selectedItem=com.atlassian.jira.jira-projects-plugin:release-page

On Mon, Sep 11, 2017 at 2:00 PM, Andrey Mashenkov <
andrey.mashen...@gmail.com> wrote:

> I've created a ticket for this [1].
> As a workaround you can try to use cache.invoke() with own comparison
> implementation inside EntryProcessor.
>
> Unfortunately, there is no release dated filled on apache ignite releases
> page [2].
> Usually, new Ignite release become available twice a year.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-6332
> [2] https://issues.apache.org/jira/projects/IGNITE?
> selectedItem=com.atlassian.jira.jira-projects-plugin:release-page
>
> On Thu, Sep 7, 2017 at 9:50 PM, sai kiran nukala 
> wrote:
>
>> Thanks for the reply.
>>
>> How do I file a ticket ?
>>
>> I know it is still early stage, usually how long it takes to fix this bug
>> and release a version because we need this functionality working for our
>> use case.
>>
>> On Sep 7, 2017 10:22 PM, "Andrey Mashenkov" 
>> wrote:
>>
>> Hi,
>> Looks like a bug and CacheEntryPredicateContainsValue shouldn't
>> deserialize value to compare BinaryObjects in case of replace() operation
>> .
>> Feel free to fill a ticket for this.
>>
>>
>>
>> On Thu, Sep 7, 2017 at 9:50 AM, saikiran939 
>> wrote:
>>
>>> Hi Team,
>>>
>>> Our team on working on a usecase in which we don't want to have any
>>> classes
>>> on Ignite Server node's classpath. To achieve this we are making use of
>>> BinaryObject based querying and putting/replacing values into cache.
>>>
>>> We are also using Optimistic Locking to replace the values into cache
>>> using
>>> "binaryObjectcache.replace(key, oldValue, newValue)" API - this method
>>> fails
>>> with ClassNotFoundException when used in client/server topology if the
>>> cache
>>> value class is not present at server's classpath. Sample piece of code is
>>> given below:
>>>
>>> String key = "key1";
>>> CacheValue entry1 = new CacheValue("putsomevalue");
>>> IgniteCache cache = ignite.getOrCreateCache(cacheC
>>> fg);
>>> cache.put(key, entry1); //put works fine even if there is no class
>>> present
>>> in server's classpath
>>>
>>> CacheValue replaceEntry1 = cache.get(key);
>>> replaceEntry1.location= "test";
>>>
>>> IgniteCache binaryCacheProjection =
>>> cache.withKeepBinary();
>>> BinaryObject oldValueInBinary = binaryCacheProjection.get(key);
>>> BinaryObject newValueInBinary = ignite.binary().toBinary(replaceEntry1);
>>> binaryCacheProjection.replace(key, oldValueInBinary, newValueInBinary);
>>>
>>> The last replace() method call fails with below exception, is this bug in
>>> ignite because put() API works or is there anyway to workaround this
>>> exception ? I think from the stacktrace ignite server is trying to
>>> deserialize value object even if it is binary. I get the same exception
>>> with
>>> normal cache.replace() API .
>>>
>>> Exception in thread "main"
>>> org.apache.ignite.cache.CachePartialUpdateException: Failed to update
>>> keys
>>> (retry update if possible).: [OP21|SHARED]
>>> at
>>> org.apache.ignite.internal.processors.cache.GridCacheUtils.c
>>> onvertToCacheException(GridCacheUtils.java:1488)
>>> at
>>> org.apache.ignite.internal.processors.cache.IgniteCacheProxy
>>> .cacheException(IgniteCacheProxy.java:2021)
>>> at
>>> org.apache.ignite.internal.processors.cache.IgniteCacheProxy
>>> .replace(IgniteCacheProxy.java:1393)
>>> at
>>> com.ignite.binary.TestReplaceBinaryObject.populateCache(Test
>>> ReplaceBinaryObject.java:166)
>>> at
>>> com.ignite.binary.TestReplaceBinaryObject.main(TestReplaceBi
>>> naryObject.java:60)
>>> at com.ignite.binary.IgniteDriver.main(IgniteDriver.java:11)
>>> Caused by: class
>>> org.apache.ignite.internal.processors.cache.CachePartialUpda
>>> teCheckedException:
>>> Failed to update keys (retry update if possible).: [OP21|SHARED]
>>> at
>>> org.apache.ignite.internal.processors.cache.distributed.dht.
>>> atomic.GridNearAtomicSingleUpdateFuture.onResult(GridNearAto
>>> micSingleUpdateFuture.java:232)
>>> at
>>> org.apache.ignite.internal.processors.cache.distributed.dht.
>>> atomic.GridDhtAtomicCache.processNearAtomicUpdateResponse(Gr
>>> idDhtAtomicCache.java:2969)
>>> at
>>> org.apache.ignite.internal.processors.cache.distributed.dht.
>>> atomic.GridDhtAtomicCache.access$700(GridDhtAtomicCache.java:130)
>>> at
>>> org.apache.ignite.internal.processors.cache.distributed.dht.
>>> atomic.GridDhtAtomicCache$6.apply(GridDhtAtomicCache.java:274)
>>> at
>>> org.apache.ignite.internal.processors.cache.distributed.dht.
>>> atomic.GridDhtAtomicCache$6.apply(GridDhtAtomicCache.java:272)
>>> at
>>> 

[GitHub] ignite pull request #2633: IGNITE-6331

[devozerov]

GitHub user devozerov opened a pull request:

https://github.com/apache/ignite/pull/2633

IGNITE-6331



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6331

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2633.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2633


commit ef8dec45d777d3ca6443bf9ca021fa9fa7bfeab2
Author: devozerov 
Date:   2017-09-11T11:02:56Z

IGNITE-6331




---

[jira] [Created] (IGNITE-6333) Improve cache transaction metrics to understand transactions efficiency.

[Alexei Scherbakov (JIRA)]

Alexei Scherbakov created IGNITE-6333:
-

 Summary: Improve cache transaction metrics to understand 
transactions efficiency.
 Key: IGNITE-6333
 URL: https://issues.apache.org/jira/browse/IGNITE-6333
 Project: Ignite
  Issue Type: Improvement
Affects Versions: 2.0
Reporter: Alexei Scherbakov
 Fix For: 2.3


I suggest to add:

1. Count of rollbacks due to transaction timeout.

2. Count of rollbacks due to deadlocks.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6332) BinaryObject model replace() in client/server topology fails with ClassNotFoundFunction

[Andrew Mashenkov (JIRA)]

Andrew Mashenkov created IGNITE-6332:


 Summary: BinaryObject model replace() in client/server topology 
fails with ClassNotFoundFunction
 Key: IGNITE-6332
 URL: https://issues.apache.org/jira/browse/IGNITE-6332
 Project: Ignite
  Issue Type: Bug
  Components: binary, cache
Affects Versions: 2.1
Reporter: Andrew Mashenkov


cache.withBinary().replace() method can fails due to ClassNotFoundException on 
unstable topology that is not expected from user side as withBinary projection 
is used.

Looks like CacheEntryPredicateContainsValue shouldn't deserialize value to 
compare BinaryObjects in case of replace() operation. 

See also discussion on userlist [1].

[1] 
http://apache-ignite-users.70518.x6.nabble.com/BinaryObject-model-replace-in-client-server-topology-fails-with-ClassNotFoundFunction-td16788.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6330) Thin clients: close query cursors on disconnect

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-6330:
--

 Summary: Thin clients: close query cursors on disconnect
 Key: IGNITE-6330
 URL: https://issues.apache.org/jira/browse/IGNITE-6330
 Project: Ignite
  Issue Type: Bug
  Components: jdbc, odbc
Reporter: Pavel Tupitsyn
 Fix For: 2.3


{{JdbcRequestHandler}} and {{OdbcRequestHandler}} store active cursors in 
{{qryCursors}} map. If client connection abruptly ends, these cursors are not 
closed properly.

{{SqlListenerNioListener.onDisconnected}} should call 
{{SqlListenerConnectionContext.onDisconnected}}, and connection context should 
clean up all resources there.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2632: IGNTIE-6328: in ML examples, class ExampleMatrixS...

[ybabak]

GitHub user ybabak opened a pull request:

https://github.com/apache/ignite/pull/2632

IGNTIE-6328: in ML examples, class ExampleMatrixStorage doesn't compile

fixed DistributedRegressionExample and ExampleMatrixStorage

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6328

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2632.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2632


commit e9890d22f883d0282db16d39ea7bf5574cdf89d0
Author: Yury Babak 
Date:   2017-09-11T10:04:33Z

IGNITE-6328: in ML examples, class ExampleMatrixStorage doesn't compile
Fixed.

commit 90a7d3227c4d3c694104d57ca5deb1fa2db9c04a
Author: Yury Babak 
Date:   2017-09-11T10:08:30Z

IGNITE-6328: in ML examples, class ExampleMatrixStorage doesn't compile
Fixed.




---

Re: NPE from service processor

[Denis Mekhanikov]

I can only say that recent changes introduced by IGNITE-5145
 in GridServiceProcessor
shouldn't have changed this behavior in any way.
It must be somehow connected with a lifecycle of IgniteKernal and other
components. A reproducing example would help to find the problem.

- Denis

пн, 11 сент. 2017 г. в 1:32, Dmitriy Setrakyan :

> Igniters,
>
> I am seeing an NPE reported from the GridServiceProcessor class here:
>
> https://stackoverflow.com/questions/46144413/apache-ignite-cluster-does-not-start-with-persistent-storage
>
> Has this been fixed?
>
> D.
>

[GitHub] ignite pull request #2630: IGNITE-6291 JDBC thin protocol compatibility is b...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2630


---

[jira] [Created] (IGNITE-6329) add Teamcity task to verify the build of ml examples

[Oleg Ignatenko (JIRA)]

Oleg Ignatenko created IGNITE-6329:
--

 Summary: add Teamcity task to verify the build of ml examples
 Key: IGNITE-6329
 URL: https://issues.apache.org/jira/browse/IGNITE-6329
 Project: Ignite
  Issue Type: Task
  Components: build, examples
Reporter: Oleg Ignatenko
Assignee: Oleg Ostanin


Add a teamcity task to verify that ml examples build successfully.

For an example of how to build refer [this doc at 
readme.io|https://apacheignite.readme.io/docs/machine-learning]:
{quote}
* ...Make sure you're using Java 8 or later.
* Build and install Apache Ignite Data Fabric from the project's root directory:
{noformat}mvn clean install -DskipTests -Dmaven.javadoc.skip=true -P 
java8{noformat}
* Build and install ML Grid from the project's root directory:
{noformat}mvn install -Pml -DskipTests -U -pl modules/ml -am{noformat}
* ...If you need to build the ML Grid examples from sources execute the 
commands below:
{noformat}cd examples
mvn clean package -DskipTests -Pml{noformat}
{quote}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2631: IGNITE-6326 JDBC thin: statement type mismatch er...

[tledkov-gridgain]

GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/2631

IGNITE-6326  JDBC thin: statement type mismatch error is handled incorrect



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6326

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2631.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2631


commit 643e6a90f2a778a0071dba6d2c2602de32ee9fc1
Author: tledkov-gridgain 
Date:   2017-09-07T12:38:48Z

IGNITE-6291: add compatibility handshake

commit 2711400412f369ff65ef6169d206bf28e1908ef1
Author: tledkov-gridgain 
Date:   2017-09-11T06:57:41Z

Merge branch '_master' into ignite-6291

commit bbb35edd2aa2682bc0a5cf39035e4f6c3617da9f
Author: tledkov-gridgain 
Date:   2017-09-11T07:34:12Z

IGNITE-6291: cosmetic

commit 3eaeafb8af44a6a1b00a19636427247c01acfd06
Author: tledkov-gridgain 
Date:   2017-09-11T08:44:54Z

IGNITE-6326 JDBC thin: statement type mismatch error is handled incorrect

commit e9fda5c73a66ebc4a24addf968707b9f0322913d
Author: tledkov-gridgain 
Date:   2017-09-11T09:11:19Z

IGNITE-6326: remove unused




---

[jira] [Created] (IGNITE-6328) in ML examples, class ExampleMatrixStorage doesn't compile

[Oleg Ignatenko (JIRA)]

Oleg Ignatenko created IGNITE-6328:
--

 Summary: in ML examples, class ExampleMatrixStorage doesn't compile
 Key: IGNITE-6328
 URL: https://issues.apache.org/jira/browse/IGNITE-6328
 Project: Ignite
  Issue Type: Bug
  Components: examples
Reporter: Oleg Ignatenko
Assignee: Yury Babak


As of now compilation fails with message:
{noformat}Error:(33, 1) java: 
org.apache.ignite.examples.ml.math.matrix.ExampleMatrixStorage
 is not abstract and does not override abstract method
 accessMode() in org.apache.ignite.ml.math.MatrixStorage{noformat}

Per preliminary discussion it was likely broken by [commit 
518238473c0a12380cb9ed96c67597d1fe42652c|https://github.com/apache/ignite/commit/518238473c0a12380cb9ed96c67597d1fe42652c].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Check Ignite build for different Java before VOTE

[Sergey Kozlov]

Alexey

Thanks for contributing! The idea is good for the improving of product
quality.

Could you explain which values the release engineer should put in the
parameters section?

On Wed, Sep 6, 2017 at 10:29 PM, Aleksey Chetaev 
wrote:

> Dmitriy,
> Sorry, we had problem on Ignite TeamCity, I fixed it, now project available
> for all developers.
>
>
> Dmitry Pavlov wrote
> > Hi Aleksey,
> >
> > I think all types of release automation is very good for improving our
> > processes and provide stable product.
> >
> > Currently I am not able to view new Run configs:
> > You do not have enough permissions to access build type with id:
> > IgniteRelease_427CheckBuildIgniteFabricOpenJDK8
> >
> > What should I do to see new builds?
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > ср, 6 сент. 2017 г. в 15:57, Aleksey Chetaev 
>
> > alex.chetaev@
>
> > :
> >
> >> Igniters,
> >>
> >> In Ignite DEVNOTES we have full instruction how we can build Ignite, but
> >> we
> >> have a lot of different options with which Ignite can be builded:
> >>  1. Ignite Fabric without Scala modules
> >>  2. Ignite Fabric with Scala modules
> >>  3. Ignite Fabric LGPL without Scala
> >>  4. Ignite Fabric LGPL with Scala
> >>  5. Ignite Hadoop for default Hadoop release.
> >>  6. Ignite Hadoop for one of last Hadoop release
> >>  7. Ignite Hadoop for one of last Hadoop version and one of Spark
> >> version.
> >> Because we provide source code to users, success build for all this
> >> options
> >> should be tested with all supported Java version before release. Make it
> >> manually it's really difficult.
> >>
> >> So that release developer can check all this build option automatically
> I
> >> created 9 task on TeamCity. All task download source archive from svn
> for
> >> release candidate using params and start build on downloaded source.
> (now
> >> source downloading from release svn for testing, because release
> >> candidate
> >> svn is empty)
> >>
> >> 1. Task check that we have license headers in all source files:
> >>
> >> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteRelease_
> 41CheckLicenseHeadersInFiles
> >> 2. Tasks check build option 1-4 for Oracle Java 7,8 and for OpenJDK 7,8:
> >>*
> >>
> >> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteRelease_
> 421CheckBuildIgniteFabricOracleJava7
> >>*
> >>
> >> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteRelease_
> 423CheckBuildIgniteFabricOracleJava8
> >>*
> >>
> >> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteRelease_
> 425CheckBuildIgniteFabricOpenJDK7
> >>*
> >>
> >> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteRelease_
> 427CheckBuildIgniteFabricOpenJDK8
> >> 3. Tasks check build option 5-7 for  Oracle Java 7,8 and for OpenJDK
> 7,8,
> >> for Hadoop(default, 2.6.5, 2.7.3) and for Spark(2.1.1, 2.2.0)
> >>*
> >>
> >> https://ci.ignite.apache.org/viewLog.html?buildId=811586;
> tab=buildResultsDiv=IgniteRelease_
> 422CheckBuildIgniteHadoopOracleJava7
> >>*
> >>
> >> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteRelease_
> 424CheckBuildIgniteHadoopOracleJava8
> >>*
> >>
> >> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteRelease_
> 426CheckBuildIgniteHadoopOpenJDK7
> >>*
> >>
> >> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteRelease_
> 428CheckBuildIgniteHadoopOpenJDK8
> >>
> >> I think we should change release process and release developer should
> run
> >> all this task before send release to VOTE.
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> --
> >> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>



-- 
Sergey Kozlov
GridGain Systems
www.gridgain.com

Re: IGNITE-5655: Mixing binary string encodings in Ignite cluster

[Vladimir Ozerov]

OK, here is implementation plan I propose:
1) Add global character set configuration -
IgniteConfiguration.characterSet. Note, it is located in
IgniteConfiguration, not BinaryConfiguration.
2) All cluster nodes must have the same character set.
3) Once defined, character set cannot be changed ever. In future we will
probably have import/export utilities, which will help users migrate
between character sets. Such strict behavior is normal for other major DBMS
vendors (e.g. Oracle), so it should work for us as well.
4) We will add "characterSet" property to all clients (ODBC, JDBC, thin
client). It will be validated during handshake phase. Exception is thrown
in case of mismatch.
5) In future we will work on relaxing these restrictions in favor of
runtime conversions on fly.

Thoughts?



On Mon, Sep 11, 2017 at 11:01 AM, Vladimir Ozerov 
wrote:

> Dima,
>
> You contradict yourself - vote for per-column encoding on the one hand,
> but telling that it is "over-architected" on the other. This is exactly
> what I am talking about - anything more that hard-coded cluster-wide
> encoding is complex. You cannot simply define per-column encoding. In
> addition you should either pass information about this encoding too all
> cluster members and to all clients, so that they construct correct binary
> object in the first place, or you should re-convert binary object on fly,
> this is what I suggested. No simple solution here.
>
> I vote for cluster-wide encoding for now, but with transparent conversion
> when needed.
>
>
> On Thu, Sep 7, 2017 at 4:50 AM, Dmitriy Setrakyan 
> wrote:
>
>> I would agree with Andrey, it does look a bit over-architected to me. Why
>> would anyone try to move data from one encoding to another? Is it a real
>> use case that needs to be handled automatically?
>>
>> Here is what I think we should handle:
>>
>>1. Ability to set cluster-wide encoding. This should be easy.
>>2. Ability to set per-column encoding. Such encoding should be set on
>>per-column level, perhaps at cache creation or table creation. For
>> example,
>>at the cache creation time, we could let user define all column names
>> that
>>will have non-default encodings.
>>
>> Thoughts?
>>
>> D.
>>
>> On Wed, Sep 6, 2017 at 6:27 AM, Andrey Kuznetsov 
>> wrote:
>>
>> > As of option #1, it's not so bad. Currently we've implemented global
>> level
>> > encoding switch, and this looks similar to DBMS: if server works with
>> > certain encoding, then all clients should be configured to use the same
>> > encoding for correct string processing.
>> >
>> > Option #2 provokes a number of questions.
>> >
>> > What are performance implications of such hidden binary reencoding?
>> >
>> > Who will check for possible data loss on transparent reencoding (when
>> > object walks between caches/fields with distinct encodings)?
>> >
>> > How should we handle nested binary objects? On the one hand, they
>> should be
>> > reencoded in a way described by Vladimir. On the other hand,
>> BinaryObject
>> > is an independent entity, that can be serialized/deserialized freely,
>> moved
>> > between various data structures, etc. It will be frustrating for user to
>> > find its binary state changed after storing in a grid, with possible
>> data
>> > corruption.
>> >
>> >
>> > As far as I can see, we are trying to couple orthogonal APIs:
>> > BinaryMarshaller, IgniteCache and SQL. BinaryMarshaller is
>> > Java-datatype-driven, it creates 1-to-1 mapping between Java types and
>> > their binary representations, and now we are trying to map two binary
>> types
>> > (STRING and ENCODED_STRING) to single String class. IgniteCache is much
>> > more flexible API, than SQL, but it lacks encoded string datatype, that
>> > exists in SQLs of some RDBMSs: `varchar(n) character set some_charset`.
>> > It's not a popular idea, but many problems could be solved by adding
>> such
>> > type. Those IgniteCache API users who don't need it won't use it, but it
>> > could become a bridge between SQL and BinaryMarshaller encoded-string
>> > types.
>> >
>> > 2017-09-06 10:32 GMT+03:00 Vladimir Ozerov :
>> >
>> > > What we tried to achieve is that several encoding could co-exist in a
>> > > single cluster or even single cache. This would be great from UX
>> > > perspective. However, from what Andrey wrote, I understand that this
>> > would
>> > > be pretty hard to achieve as we rely heavily on similar binary
>> > > representation of objects being compared. That said, while this could
>> > work
>> > > for SQL with some adjustments, we will have severe problems with
>> > > BinaryObject.equals().
>> > >
>> > > Let's think on how we can resolve this. I see two options:
>> > > 1) Allow only single encoding in the whole cluster. Easy to implement,
>> > but
>> > > very bad from usability perspective. Especially this would affect
>> > clients -
>> > > client nodes, and what is worse, 

Re: IGNITE-5655: Mixing binary string encodings in Ignite cluster

[Vladimir Ozerov]

Dima,

You contradict yourself - vote for per-column encoding on the one hand, but
telling that it is "over-architected" on the other. This is exactly what I
am talking about - anything more that hard-coded cluster-wide encoding is
complex. You cannot simply define per-column encoding. In addition you
should either pass information about this encoding too all cluster members
and to all clients, so that they construct correct binary object in the
first place, or you should re-convert binary object on fly, this is what I
suggested. No simple solution here.

I vote for cluster-wide encoding for now, but with transparent conversion
when needed.


On Thu, Sep 7, 2017 at 4:50 AM, Dmitriy Setrakyan 
wrote:

> I would agree with Andrey, it does look a bit over-architected to me. Why
> would anyone try to move data from one encoding to another? Is it a real
> use case that needs to be handled automatically?
>
> Here is what I think we should handle:
>
>1. Ability to set cluster-wide encoding. This should be easy.
>2. Ability to set per-column encoding. Such encoding should be set on
>per-column level, perhaps at cache creation or table creation. For
> example,
>at the cache creation time, we could let user define all column names
> that
>will have non-default encodings.
>
> Thoughts?
>
> D.
>
> On Wed, Sep 6, 2017 at 6:27 AM, Andrey Kuznetsov 
> wrote:
>
> > As of option #1, it's not so bad. Currently we've implemented global
> level
> > encoding switch, and this looks similar to DBMS: if server works with
> > certain encoding, then all clients should be configured to use the same
> > encoding for correct string processing.
> >
> > Option #2 provokes a number of questions.
> >
> > What are performance implications of such hidden binary reencoding?
> >
> > Who will check for possible data loss on transparent reencoding (when
> > object walks between caches/fields with distinct encodings)?
> >
> > How should we handle nested binary objects? On the one hand, they should
> be
> > reencoded in a way described by Vladimir. On the other hand, BinaryObject
> > is an independent entity, that can be serialized/deserialized freely,
> moved
> > between various data structures, etc. It will be frustrating for user to
> > find its binary state changed after storing in a grid, with possible data
> > corruption.
> >
> >
> > As far as I can see, we are trying to couple orthogonal APIs:
> > BinaryMarshaller, IgniteCache and SQL. BinaryMarshaller is
> > Java-datatype-driven, it creates 1-to-1 mapping between Java types and
> > their binary representations, and now we are trying to map two binary
> types
> > (STRING and ENCODED_STRING) to single String class. IgniteCache is much
> > more flexible API, than SQL, but it lacks encoded string datatype, that
> > exists in SQLs of some RDBMSs: `varchar(n) character set some_charset`.
> > It's not a popular idea, but many problems could be solved by adding such
> > type. Those IgniteCache API users who don't need it won't use it, but it
> > could become a bridge between SQL and BinaryMarshaller encoded-string
> > types.
> >
> > 2017-09-06 10:32 GMT+03:00 Vladimir Ozerov :
> >
> > > What we tried to achieve is that several encoding could co-exist in a
> > > single cluster or even single cache. This would be great from UX
> > > perspective. However, from what Andrey wrote, I understand that this
> > would
> > > be pretty hard to achieve as we rely heavily on similar binary
> > > representation of objects being compared. That said, while this could
> > work
> > > for SQL with some adjustments, we will have severe problems with
> > > BinaryObject.equals().
> > >
> > > Let's think on how we can resolve this. I see two options:
> > > 1) Allow only single encoding in the whole cluster. Easy to implement,
> > but
> > > very bad from usability perspective. Especially this would affect
> > clients -
> > > client nodes, and what is worse, drivers and thin clients! They all
> would
> > > have to bother about which encoding to use. But may be we can share
> this
> > > information during handshake (as every client has a handshake).
> > >
> > > 2) Add custom eocnding flag/ID to object header if non-standard
> enconding
> > > appears somewhere inside the object (even in nested objects). This way,
> > we
> > > will be able to re-create the object if needed if expected and actual
> > > encoding doesn't match. For example, consider we have two caches/tables
> > > with different encoding (not implemented in current iteration, but we
> may
> > > decide to implement per-cache encodings in future, as this any RDBMS
> > > support it). And then I decide to move object A from cache 1 with UTF-8
> > > encoding to cache 2 with Cp1251 encoding. In this case I will detect
> > > encoding mismatch through object header (or footer) and re-build it
> > > transparently for user.
> > >
> > > Second option is more preferable to me as a 

[GitHub] ignite pull request #2630: IGNITE-6291 JDBC thin protocol compatibility is b...

[tledkov-gridgain]

GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/2630

IGNITE-6291 JDBC thin protocol compatibility is broken



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6291

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2630.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2630


commit 643e6a90f2a778a0071dba6d2c2602de32ee9fc1
Author: tledkov-gridgain 
Date:   2017-09-07T12:38:48Z

IGNITE-6291: add compatibility handshake

commit 2711400412f369ff65ef6169d206bf28e1908ef1
Author: tledkov-gridgain 
Date:   2017-09-11T06:57:41Z

Merge branch '_master' into ignite-6291

commit bbb35edd2aa2682bc0a5cf39035e4f6c3617da9f
Author: tledkov-gridgain 
Date:   2017-09-11T07:34:12Z

IGNITE-6291: cosmetic




---

Re: How to configure new thin client connector on the server?

[Vladimir Ozerov]

Pavel,

Note that this would not be renames, but rather copy+deprecated. Otherwise
we will break compilation of existing applications.

On Mon, Sep 11, 2017 at 10:32 AM, Pavel Tupitsyn 
wrote:

> Renaming SqlConnectorConfiguration to ClientConnectorConfiguration seems
> reasonable to me.
>
> On Mon, Sep 11, 2017 at 10:28 AM, Vladimir Ozerov 
> wrote:
>
> > Igniters,
> >
> > As you know we are working on a new thin client protocol [1]. Core logic
> > for base cache operations is ready, so now it is time to discuss how to
> > configure it on the server. Things are a bit complicated here.
> >
> > First, we have connector configuration for our old so-called REST client,
> > see ConnectorConfiguration and IgniteConfiguraion.
> connectorConfiguration.
> > We cannot re-use it because it is very heavy-weight and contains a lot of
> > properties which are simply inapplicable to the new client. E.g.
> > SSL-related stuff, Jetty configs, etc..
> >
> > Second, we have connector configuration for our thin SQL drivers (JDBC,
> > ODBC). See SqlConnectorConfiguration and
> > IgniteConfiguration.sqlConnectorConfiguration. This configuration is
> fine
> > for the new thin client, but it's name is inappropriate.
> >
> > Essentially, we should rename our SqlConnectorConfiguration to something
> > more generic, but this "something" cannot be ConnectorConfiguration.
> Let's
> > brainstorm how to name it.
> >
> > My ideas:
> > - ClientConnectorConfiguration - preferrable
> > - ListenerConfiguration
> > - EndpointConfiguration
> >
> > Any more ideas?
> >
> > Vladimir.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-5896
> >
>

Re: How to configure new thin client connector on the server?

[Pavel Tupitsyn]

Renaming SqlConnectorConfiguration to ClientConnectorConfiguration seems
reasonable to me.

On Mon, Sep 11, 2017 at 10:28 AM, Vladimir Ozerov 
wrote:

> Igniters,
>
> As you know we are working on a new thin client protocol [1]. Core logic
> for base cache operations is ready, so now it is time to discuss how to
> configure it on the server. Things are a bit complicated here.
>
> First, we have connector configuration for our old so-called REST client,
> see ConnectorConfiguration and IgniteConfiguraion.connectorConfiguration.
> We cannot re-use it because it is very heavy-weight and contains a lot of
> properties which are simply inapplicable to the new client. E.g.
> SSL-related stuff, Jetty configs, etc..
>
> Second, we have connector configuration for our thin SQL drivers (JDBC,
> ODBC). See SqlConnectorConfiguration and
> IgniteConfiguration.sqlConnectorConfiguration. This configuration is fine
> for the new thin client, but it's name is inappropriate.
>
> Essentially, we should rename our SqlConnectorConfiguration to something
> more generic, but this "something" cannot be ConnectorConfiguration. Let's
> brainstorm how to name it.
>
> My ideas:
> - ClientConnectorConfiguration - preferrable
> - ListenerConfiguration
> - EndpointConfiguration
>
> Any more ideas?
>
> Vladimir.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-5896
>

[jira] [Created] (IGNITE-6327) Configuration for thin client

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-6327:
---

 Summary: Configuration for thin client
 Key: IGNITE-6327
 URL: https://issues.apache.org/jira/browse/IGNITE-6327
 Project: Ignite
  Issue Type: Task
  Components: clients, platforms
Reporter: Vladimir Ozerov
 Fix For: 2.3


We need to create configuration for thin client connector on the server side. 
See linked devlist discussion.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

How to configure new thin client connector on the server?

[Vladimir Ozerov]

Igniters,

As you know we are working on a new thin client protocol [1]. Core logic
for base cache operations is ready, so now it is time to discuss how to
configure it on the server. Things are a bit complicated here.

First, we have connector configuration for our old so-called REST client,
see ConnectorConfiguration and IgniteConfiguraion.connectorConfiguration.
We cannot re-use it because it is very heavy-weight and contains a lot of
properties which are simply inapplicable to the new client. E.g.
SSL-related stuff, Jetty configs, etc..

Second, we have connector configuration for our thin SQL drivers (JDBC,
ODBC). See SqlConnectorConfiguration and
IgniteConfiguration.sqlConnectorConfiguration. This configuration is fine
for the new thin client, but it's name is inappropriate.

Essentially, we should rename our SqlConnectorConfiguration to something
more generic, but this "something" cannot be ConnectorConfiguration. Let's
brainstorm how to name it.

My ideas:
- ClientConnectorConfiguration - preferrable
- ListenerConfiguration
- EndpointConfiguration

Any more ideas?

Vladimir.

[1] https://issues.apache.org/jira/browse/IGNITE-5896

[GitHub] ignite pull request #2617: IGNITE-6288 NPE on SQL query with parameters on c...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2617


---

[jira] [Created] (IGNITE-6326) JDBC thin: statement type mismatch error is handled incorrect

[Taras Ledkov (JIRA)]

Taras Ledkov created IGNITE-6326:


 Summary: JDBC thin: statement type mismatch error is handled 
incorrect
 Key: IGNITE-6326
 URL: https://issues.apache.org/jira/browse/IGNITE-6326
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.2
Reporter: Taras Ledkov
Assignee: Taras Ledkov
 Fix For: 2.3


e.g.: if the update query is executed with {{executeQuery}} method the 
{{SQLException}} will be  thrown AFTER the data has been updates.

{{JdbcSqlFieldsQuery}} must be used for {{executeQuery}} and {{executeUpdate}} 
methods.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)