RE: OSGi migration may require a special marshaller

[Andrey Kornev]

Raul,

Could you please be a bit more specific about the nature of your disagreement? 
Is the proposed SPI not generic enough? Or, is it just the naming?

As per Romain's suggestion, could you please just make sure the SPI is hooked 
into the new marshalling implementation and  a no-op codec implementation is 
available in time for the 1.5 release? This would be the first step toward full 
OSGi support in 1.6.

Thanks
Andrey

> From: ra...@apache.org
> Date: Tue, 10 Nov 2015 18:22:59 +
> Subject: Re: OSGi migration may require a special marshaller
> To: dev@ignite.apache.org
> 
> Hi Romain,
> 
> The implementation I have in mind won't be costly. I'm working on it this
> week.
> 
> I still don't agree with a ClassLoaderCodec SPI as-is. If we create an SPI
> it should be for more generic hinting applicable in other circumstances
> during serialisation with other frameworks and what not.
> 
> Otherwise, we should not create a pluggable SPI at all and simply implement
> an option. As a matter of fact, with the last solution I proposed I don't
> think there will be any edge cases at all where users would need to create
> their own "ClassLoaderCodec" at all.
> 
> Regards,
> 
> *Raúl Kripalani*
> PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
> Messaging Engineer
> http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
> http://blog.raulkr.net | twitter: @raulvk
> 
> On Tue, Nov 10, 2015 at 3:06 PM, Romain Gilles <romain.gil...@gmail.com>
> wrote:
> 
> > To be honest Raoul, I'm more interesting in the interface (SPI) declaration
> > than the implementation. If you can do it for the 1.5 I'm ok. Otherwise I
> > ready to start it by introducing the interface and implementing the dummy
> > version for non OSGi environment.
> >
> > It will free me to implement a temporary OSGi version by the time you are
> > done with the overall IGNITE-1270 tasks.
> >
> > Thanks,
> >
> > Romain
> >
> > Le mar. 10 nov. 2015 à 15:42, Raul Kripalani <r...@evosent.com> a écrit :
> >
> > > Hi Romain,
> > >
> > > If you don't mind, I'm working on the entire OSGi integration including
> > the
> > > serialisation technique.
> > >
> > > I'll ping you if I need help.
> > >
> > > Thanks for your collaboration,
> > >
> > > *Raúl Kripalani*
> > > PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
> > > Messaging Engineer
> > > http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
> > > http://blog.raulkr.net | twitter: @raulvk
> > >
> > > On Tue, Nov 10, 2015 at 1:50 PM, Romain Gilles <romain.gil...@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > > I have put my comments. Hope they will make the things progress :)
> > > > Should I start to implement this ticket or should I wait and see if
> > Raoul
> > > > want to take it?
> > > >
> > > > Romain
> > > >
> > > > Le mar. 10 nov. 2015 à 02:58, Dmitriy Setrakyan <dsetrak...@apache.org
> > >
> > > a
> > > > écrit :
> > > >
> > > > > Thanks Raul, great points! I have created a ticket for the
> > > > > class-loader-detection design, described on wiki:
> > > > >
> > > > > https://issues.apache.org/jira/browse/IGNITE-1877
> > > > >
> > > > > D.
> > > > >
> > > > > On Mon, Nov 9, 2015 at 3:42 PM, Raul Kripalani <ra...@apache.org>
> > > wrote:
> > > > >
> > > > > > Hey Romain,
> > > > > >
> > > > > > I've updated the design proposal in the Wiki with my input. Could
> > you
> > > > > > please have a look and let me know what you think?
> > > > > >
> > > > > > I'll implement a proof of concept of my proposed
> > > > > marshalling/unmarshalling
> > > > > > strategy and push it to Git so you can take it for a spin.
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > *Raúl Kripalani*
> > > > > > PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big
> > Data
> > > > and
> > > > > > Messaging Engineer
> > > > > > http://about.me/raulkripalani |
> > > > http://www.linkedin.com/in/raulkripalani
> > > > > > http://blog.raulkr.

RE: Custom ThreadFactory

[Andrey Kornev]

There you go: 
https://issues.apache.org/jira/browse/IGNITE-1898
and
https://issues.apache.org/jira/browse/IGNITE-1899

Cheers
Andrey

> From: dsetrak...@apache.org
> Date: Thu, 12 Nov 2015 12:53:52 -0800
> Subject: Re: Custom ThreadFactory
> To: dev@ignite.apache.org
> 
> On Thu, Nov 12, 2015 at 10:26 AM, Andrey Kornev <andrewkor...@hotmail.com>
> wrote:
> 
> > Even better, Ignite might provide out-of-the-box access to the local
> > instance via a
> > thread local. It could be in a form of a static public
> > method on the Ignition class.
> >
> > Ignite itself could benefit from
> > this feature as it does get it wrong occasionally. A good example of
> > this is the ClusterGroupAdapter class or any other class that serializes
> >  the instance of IgniteKernal. Imagine the situation where you have a
> > single JVM with multiple Ignite nodes started -- Ignite requires the
> > grid names to be different. But since only the name of the grid is
> > serialized, during deserialization an invalid (unexpected, to put it
> > mildly) instance of IgniteKernal is looked up. I don't know how serious
> > it is, but it is probably a bug.
> >
> 
> Andrey, I think its best to summarize the design thoughts in a ticket. Then
> someone from the community will pick it up.
> 
> 
> >
> > Regards
> > Andrey
> >
> > > Date: Thu, 12 Nov 2015 21:11:38 +0300
> > > Subject: Re: Custom ThreadFactory
> > > From: voze...@gridgain.com
> > > To: dev@ignite.apache.org
> > >
> > > I would avoid running any external payloads in public pool because it
> > could
> > > unpredictably affect Ignite internals. "Public" doesn't mean "opened for
> > > everyone" here.
> > >
> > > On the other hand, I abosuletly agree that removing possibility to
> > > configure custom pools was not very good idea. I do not see any problems
> > > with returning it back while still keeping "thread count" property for
> > the
> > > most common use case when custom pool is not needed/
> > >
> > > On Thu, Nov 12, 2015 at 9:02 PM, Andrey Kornev <andrewkor...@hotmail.com
> > >
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > If my memory doesn't fail me, in the pre-Ignite versions of GridGain,
> > it
> > > > was possible to configure custom executor services which would then be
> > used
> > > > to create the public, system, utility, etc. thread pools. In Ignite
> > however
> > > > it's only possible to configure the size of the thread pools and not
> > their
> > > > implementations.
> > > >
> > > > This is unfortunate as I'd like to be able to configure my own
> > > > ThreadFactory. My implementation would for example ensure that newly
> > > > created threads have their thread locals properly initialized (for
> > example,
> > > > by storing the local instance of Ignite in it). Specific use case is
> > being
> > > > able to get hold of the local Ignite instance during deserialization
> > when
> > > > the JVM instance has multiple Ignite nodes started. Some of my classes
> > must
> > > > be able to access resources that are local to the node on which they
> > are
> > > > being deserialized. At the moment there is absolutely no way of
> > achieving
> > > > something like that.
> > > >
> > > > I'm wondering if it would be possible to add this feature back to
> > Ignite?
> > > > It seems to be indispensable for unit testing.
> > > >
> > > > Alternatively, to reduce the impact on the public API, an environment
> > > > variable that takes an FQN of the ThreadFactory to use would also
> > work. It
> > > > would be injectable with the Ignite resources in the manner similar to
> > how
> > > > it's done for the closures and factories...
> > > >
> > > > Regards
> > > > Andrey
> > > >
> > > > PS. While we're at it, I also remember that in the pre-Ignite versions
> > it
> > > > was possible to inject an instance of the public executor service into
> > the
> > > > closures. Not anymore. It causes the inconvenience of starting another
> > > > thread pool while there is already a public pool managed by Ignite with
> > > > plenty of threads idling most of the time... It feels wasteful.
> > > >
> >
> >
  

RE: OSGi migration may require a special marshaller

[Andrey Kornev]

Raul, if you don't mind, for posterity, could you please collect all your ideas 
spread all over this two-week long email thread and update the wiki page here 
https://cwiki.apache.org/confluence/display/IGNITE/OSGI+Compatibility? The 
details of your de-/serialization implementation would be especially 
interesting as personally I still don't understand what's being proposed. Also, 
please update the "Bundlezation" section with your approach to determining when 
an optional Ignite dependency should be a bundle and when a fragment.

Thanks a lot!
Andrey

> From: ra...@apache.org
> Date: Wed, 11 Nov 2015 13:31:04 +
> Subject: Re: OSGi migration may require a special marshaller
> To: dev@ignite.apache.org
> 
> On Wed, Nov 11, 2015 at 1:48 AM, Dmitriy Setrakyan 
> wrote:
> 
> > Raul, we cannot be adding package name and version in all cases, as it will
> > have a negative impact on performance, especially when there is an
> > *optimistic* approach that requires no over-the-wire overhead whatsoever.
> >
> 
> Why do we need to add the package name? The OptimizedObjectOutputStream
> already encodes the classname, which is then read on line 310 of
> OptimizedObjectInputStream (ignite-1.5 branch). Therefore, we already have
> the package name. All we need to encode is the version number which will be
> an additional String which can be further compressed:
> 
> * For "1.0.0" we transmit "1".
> * For "1.1.0", we transmit "1.1".
> * etc.
> 
> And this will only occur on a conditional basis when we detect we're
> running in an OSGi environment, although we need to cater for hybrid grid
> scenarios (comprising OSGi containers + non-OSGi containers). For all other
> cases, it would be skipped.
> 
> If you'd still like to avoid these additional bytes, then a mode switch in
> OptimizedMarshaller: setTransmitOsgiPkgVer(boolean) should suffice.
> 
> As I said, I don't see enough justification for pluggability in this
> context. I could see it if we were creating a mechanism for transmitting
> generic serialisation context. But everybody seems fixated on tackling
> classloader complexity only.
> 
> I could also see some level of pluggability if this was applicable to any
> other marshallers. But how exactly do you see this being applicable to
> Kryo, Protobuf, or other potential ones (which we don't offer now)? How
> would you "encode the classloader" and where in the OutputStream? At the
> header? Before delegating the serialisation to the selected framework?
> 
> Raul, personally I understand your sentiments, but to be honest I dislike
> > the names you are proposing even more. I still consider ClassLoaderCodec to
> > be the most elegant name here, considering all the other options I have
> > seen. It is concise and symmetric. I am open to changing it, but it must be
> > for the better, not for the worse.
> 
> 
> It's not about sentiment, it's about objectivity. The first step is to move
> away from the ClassLoaderCodec name, it's inaccurate. If anything I would
> propose SerializationContextCodec. But it implies extending its role to
> what I proposed: to provide and interpret information that assists the
> serialisation/deserialisation context.
> 
> Let’s make sure that we follow the 80/20 rule and keep the default
> > implementation suitable for 80% of use cases with zero performance
> > overhead.
> 
> 
> No one questions that. The additional package version transmission would
> only kick in if we're in an OSGi environment.
> 
> Having said that, I would be against artificially forcing this change into
> > 1.5, especially given the amount of controversy it generated.
> 
> 
> I don't think it's impossible to get it into 1.5 if that's the timeline.
> 
> Regards,
> 
> *Raúl Kripalani*
> PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
> Messaging Engineer
> http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
> http://blog.raulkr.net | twitter: @raulvk
  

Custom ThreadFactory

[Andrey Kornev]

Hello,

If my memory doesn't fail me, in the pre-Ignite versions of GridGain, it was 
possible to configure custom executor services which would then be used to 
create the public, system, utility, etc. thread pools. In Ignite however it's 
only possible to configure the size of the thread pools and not their 
implementations. 

This is unfortunate as I'd like to be able to configure my own ThreadFactory. 
My implementation would for example ensure that newly created threads have 
their thread locals properly initialized (for example, by storing the local 
instance of Ignite in it). Specific use case is being able to get hold of the 
local Ignite instance during deserialization when the JVM instance has multiple 
Ignite nodes started. Some of my classes must be able to access resources that 
are local to the node on which they are being deserialized. At the moment there 
is absolutely no way of achieving something like that.

I'm wondering if it would be possible to add this feature back to Ignite? It 
seems to be indispensable for unit testing. 

Alternatively, to reduce the impact on the public API, an environment variable 
that takes an FQN of the ThreadFactory to use would also work. It would be 
injectable with the Ignite resources in the manner similar to how it's done for 
the closures and factories...

Regards
Andrey

PS. While we're at it, I also remember that in the pre-Ignite versions it was 
possible to inject an instance of the public executor service into the 
closures. Not anymore. It causes the inconvenience of starting another thread 
pool while there is already a public pool managed by Ignite with plenty of 
threads idling most of the time... It feels wasteful.
  

RE: Custom ThreadFactory

[Andrey Kornev]

Even better, Ignite might provide out-of-the-box access to the local instance 
via a 
thread local. It could be in a form of a static public 
method on the Ignition class. 

Ignite itself could benefit from 
this feature as it does get it wrong occasionally. A good example of 
this is the ClusterGroupAdapter class or any other class that serializes
 the instance of IgniteKernal. Imagine the situation where you have a 
single JVM with multiple Ignite nodes started -- Ignite requires the 
grid names to be different. But since only the name of the grid is 
serialized, during deserialization an invalid (unexpected, to put it 
mildly) instance of IgniteKernal is looked up. I don't know how serious 
it is, but it is probably a bug.

Regards
Andrey

> Date: Thu, 12 Nov 2015 21:11:38 +0300
> Subject: Re: Custom ThreadFactory
> From: voze...@gridgain.com
> To: dev@ignite.apache.org
> 
> I would avoid running any external payloads in public pool because it could
> unpredictably affect Ignite internals. "Public" doesn't mean "opened for
> everyone" here.
> 
> On the other hand, I abosuletly agree that removing possibility to
> configure custom pools was not very good idea. I do not see any problems
> with returning it back while still keeping "thread count" property for the
> most common use case when custom pool is not needed/
> 
> On Thu, Nov 12, 2015 at 9:02 PM, Andrey Kornev <andrewkor...@hotmail.com>
> wrote:
> 
> > Hello,
> >
> > If my memory doesn't fail me, in the pre-Ignite versions of GridGain, it
> > was possible to configure custom executor services which would then be used
> > to create the public, system, utility, etc. thread pools. In Ignite however
> > it's only possible to configure the size of the thread pools and not their
> > implementations.
> >
> > This is unfortunate as I'd like to be able to configure my own
> > ThreadFactory. My implementation would for example ensure that newly
> > created threads have their thread locals properly initialized (for example,
> > by storing the local instance of Ignite in it). Specific use case is being
> > able to get hold of the local Ignite instance during deserialization when
> > the JVM instance has multiple Ignite nodes started. Some of my classes must
> > be able to access resources that are local to the node on which they are
> > being deserialized. At the moment there is absolutely no way of achieving
> > something like that.
> >
> > I'm wondering if it would be possible to add this feature back to Ignite?
> > It seems to be indispensable for unit testing.
> >
> > Alternatively, to reduce the impact on the public API, an environment
> > variable that takes an FQN of the ThreadFactory to use would also work. It
> > would be injectable with the Ignite resources in the manner similar to how
> > it's done for the closures and factories...
> >
> > Regards
> > Andrey
> >
> > PS. While we're at it, I also remember that in the pre-Ignite versions it
> > was possible to inject an instance of the public executor service into the
> > closures. Not anymore. It causes the inconvenience of starting another
> > thread pool while there is already a public pool managed by Ignite with
> > plenty of threads idling most of the time... It feels wasteful.
> >
  

RE: OSGi migration may require a special marshaller

[Andrey Kornev]

Raul,

While we're waiting for the specific proposal(s) from you, I'd like to point 
out that OSGi bundle is much more than just a jar or a packaging unit. First 
and foremost an OSGi bundle is also a class loader unlike the jars in non-OSGi 
environment. We're after the class loader hence the proposal to use the bundle 
symbolic name and bundle version for serialization of the *class loader*.

Have a safe trip and please, do get back to us at your earliest.

Regards
Andrey

> Date: Wed, 4 Nov 2015 10:56:32 +
> Subject: Re: OSGi migration may require a special marshaller
> From: r...@evosent.com
> To: dev@ignite.apache.org
> 
> Could we rewind a little bit please? What is the point of using the bundle
> symbolic name in the first place?
> 
> In OSGi, it is bad practice to refer to bundles directly. A bundle is a
> packaging unit, that's all, but it should not have any further
> consideration.
> 
> It is perfectly feasible that a given package is exported by bundle A in a
> container, and by bundle B in another. One should not make any assumptions
> in that regard. A simple example of this is an enterprise-wide deployment
> of Ignite, where multiple apps in different OSGi containers share cache
> entities. Why would one expect these entities to be hosted in the same
> bundle across all apps?
> 
> Referring to a bundle in OSGi world is like referring to a JAR in a normal
> Java environment. One would not always expect to resolve class A.B.C from
> bundle xyz-1.2.3.jar, right? You would resolve from the classpath.
> 
> There are other mechanisms to reach into the OSGi environment and resolve a
> class arbitrarily from another bundle. Some of which I've pointed out. But
> relying on a bundle symbolic name receives an -1 from me.
> 
> Can't elaborate now, I'm travelling.
> 
> Will get back to my office tomorrow and I'll dig deeper into possibilities.
> 
> Regards,
> Raúl.
> On 4 Nov 2015 03:07, "Dmitriy Setrakyan" <dsetrak...@apache.org> wrote:
> 
> > Andrey, et al,
> >
> > Can you provide a way on how to get a required Bundle object based on, say,
> > bundle symbolic name and version?
> >
> > I have reached the end of the internet trying to find a way in OSGI to look
> > up a Bundle based on something other than an actual Class belonging to that
> > bundle, but no luck.
> >
> > D.
> >
> > On Tue, Nov 3, 2015 at 5:15 PM, Andrey Kornev <andrewkor...@hotmail.com>
> > wrote:
> >
> > > Dmitriy,
> > >
> > > I think your approach will work, but I let Romain respond.
> > >
> > > Also, in terms of the implementation, please keep in mind that the
> > > resolver must be called for each non-JDK and non-Ignite core class (it
> > > would probably make sense to eschew storing class loaders for such
> > classes
> > > in favor of compactness of the serialized representation -- see below).
> > > Also, it's worth keeping a cache of already resolved class loaders per
> > > marshaller invocation (this is probably the context that Romain has
> > > mentioned in his previous posting) to minimize the number of the resolver
> > > calls.
> > >
> > > In terms of the resolver's implementation, the simplest way to serialize
> > > the class loader would be by capturing two pieces of information (both
> > > strings): the bundle symbolic name and the bundle version. This approach
> > > however may result in bloating of the serialized representation: I'd
> > > roughly estimate the overhead per element to be at least 20-30 bytes (the
> > > length of the symbolic name string, plus the length of the version
> > string).
> > > There are way to reduce the overhead (like serializing the hash code of
> > the
> > > bundle id string rather than the string itself, and then come up with a
> > > clever way of resolving the hash collisions), but they all come at cost
> > of
> > > increased complexity...
> > >
> > > An alternative approach would be rely on the special bundle id which is
> > an
> > > integer and is generated by the OSGi container. But in this case, all
> > nodes
> > > must ensure that all the bundles have consistent ids (bundle A with id 42
> > > on node N1, has the same id on every other node) which is not easy --
> > while
> > > not entirely impossible -- to guarantee. As long as the nodes are
> > > homogeneous (have the same set of bundles deployed) the OSGi container is
> > > guaranteed to assign to the bundles the same ids.
> > >
> > > Thanks
> > > Andrey
> >

RE: OSGi migration may require a special marshaller

[Andrey Kornev]

Dmitriy,

I think your approach will work, but I let Romain respond. 

Also, in terms of the implementation, please keep in mind that the resolver 
must be called for each non-JDK and non-Ignite core class (it would probably 
make sense to eschew storing class loaders for such classes in favor of 
compactness of the serialized representation -- see below). Also, it's worth 
keeping a cache of already resolved class loaders per marshaller invocation 
(this is probably the context that Romain has mentioned in his previous 
posting) to minimize the number of the resolver calls.

In terms of the resolver's implementation, the simplest way to serialize the 
class loader would be by capturing two pieces of information (both strings): 
the bundle symbolic name and the bundle version. This approach however may 
result in bloating of the serialized representation: I'd roughly estimate the 
overhead per element to be at least 20-30 bytes (the length of the symbolic 
name string, plus the length of the version string). There are way to reduce 
the overhead (like serializing the hash code of the bundle id string rather 
than the string itself, and then come up with a clever way of resolving the 
hash collisions), but they all come at cost of increased complexity...

An alternative approach would be rely on the special bundle id which is an 
integer and is generated by the OSGi container. But in this case, all nodes 
must ensure that all the bundles have consistent ids (bundle A with id 42 on 
node N1, has the same id on every other node) which is not easy -- while not 
entirely impossible -- to guarantee. As long as the nodes are homogeneous (have 
the same set of bundles deployed) the OSGi container is guaranteed to assign to 
the bundles the same ids.

Thanks
Andrey

> From: dsetrak...@apache.org
> Date: Tue, 3 Nov 2015 16:29:41 -0800
> Subject: Re: OSGi migration may require a special marshaller
> To: dev@ignite.apache.org
> 
> Romain,
> 
> In the upcoming release we will be deprecating the OptimizedMarshaller and
> will be switching to a default internal marshaller (which is based on the
> new PortableMarshaller donated by GridGain).
> 
> Having said that, we may be able to pass BinaryWriter and BinaryReader
> instead of byte arrays. This will be pretty close to passing the stream, as
> suggested by Andrey.
> 
> Also, I still think that we should only resolve class loaders and not the
> class itself. The main reason is that Ignite will encode class names into
> an integer hash code and will store the integer->class-fqn mapping in
> internal replicated cache. I doubt users will get more efficient than an
> integer (4 bytes) for a class name.
> 
> On the receiving side, once we are able to get the right class loader, we
> can easily get the proper class by calling ClassLoader.findClass(class-fqn).
> 
> Thoughts?
> 
> D.
> 
> On Tue, Nov 3, 2015 at 2:01 PM, Romain Gilles <romain.gil...@gmail.com>
> wrote:
> 
> > Hi,
> > Maybe a missing point. I think but I'm not sure that in the
> > OptimizedMarshaller there is a caching of already serialized classes. Due
> > to the dynamic nature of OSGi this may lead to memory leak. In fact if a
> > bundle is refreshed, it will produce a new BundleRevision and therefore a
> > new classloader. And if you don't release the class from the previous
> > BundleRevision then you endup with memory leak. So maybe the Marshaller
> > interface or somewhere should provide a way to free those classes /
> > classloaders.
> >
> > Regards,
> >
> > Romain
> >
> > Le mar. 3 nov. 2015 à 22:42, Andrey Kornev <andrewkor...@hotmail.com> a
> > écrit :
> >
> > > Romain/Dmitriy,
> > >
> > > I prefer Romain's approach, but just curious, in the API you guys are
> > > proposing why use a byte[] rather than OutputStream/InputStream? With a
> > > byte[], one would inevitably end up wrapping it into a byte stream class.
> > > Also, the stream-based interface would be more in line with the
> > Marshaller
> > > API.
> > >
> > > Also for symmetry with the readClass() method, I suggest the writeClass()
> > > take a Class rather than an object.
> > >
> > > Regards
> > > Andrey
> > >
> > > > From: romain.gil...@gmail.com
> > > > Date: Tue, 3 Nov 2015 21:24:01 +
> > > > Subject: Re: OSGi migration may require a special marshaller
> > > > To: dev@ignite.apache.org
> > > >
> > > > Hi Dmitriy,
> > > > I think your solution is good. Maybe I will change it a little bit...
> > :P
> > > > I think you should delegate the Class resolution to the resolver.
> > Because
> > &

RE: OSGi migration may require a special marshaller

[Andrey Kornev]

nt that OSGi allows for multiple versions of the same
> > > > bundle/packages to co-exist in the same  container. So it becomes more
> > > > complex.
> > > >
> > > > Using the TCCL should work when serialising, but it'll probably be of
> > no
> > > > use when deserialising on the other end.
> > > >
> > > > I need to enhance Ignite to:
> > > >
> > > > 1. Use the TCCL when marshalling on one end.
> > > > 2. Incorporate the package version of the class in the serialised form
> > if
> > > > Ignite is running in an OSGi environment.
> > > > 3. On the receiver end, discover cache entities / DTOs in all bundles
> > > > through a custom OSGi manifest header or the like, as I explained
> > before.
> > > > Package version must be taken into account.
> > > >
> > > > What do you think?
> > > >
> > > > Raúl.
> > > > On 2 Nov 2015 17:25, "Andrey Kornev" <andrewkor...@hotmail.com> wrote:
> > > >
> > > > > Raul,
> > > > >
> > > > > The fundamental hurdle we need to jump over to make Ignite
> > OSGi-enabled
> > > > is
> > > > > the marshalling. More specifically the issue is with deserialization
> > of
> > > > the
> > > > > classes that are provided by the bundles other than the Ignite bundle
> > > > > itself.
> > > > >
> > > > > When the Ignite transport layer receives a message it needs to figure
> > > out
> > > > > how to deserialize the bytes and for that it needs to know the bundle
> > > > that
> > > > > provides the class to be deserailized. At this point TCCL is of no
> > use.
> > > > To
> > > > > make things more complex, the class may contain other classes that
> > come
> > > > > from other bundles, and so on recursively. This means that each
> > object
> > > in
> > > > > the hierarchy must be serialized with its bundle name (or bundle id),
> > > so
> > > > > that the deserializer will then be able to correctly resolve the
> > class
> > > > > while traversing the object hierarchy during deserialization.
> > > > >
> > > > > Unfortunately, Ignite's OptimizedMarshaller is lacking the ability to
> > > > plug
> > > > > in a custom class resolver. Romain's solution was to use Kryo that
> > does
> > > > > provide a way to customize class resolution. It has solved the
> > problem
> > > of
> > > > > capturing the bundle info and he was able to successfully run Ignite
> > > as a
> > > > > bundle in an OSGi container (after some repackaging and inclusion of
> > > the
> > > > > manifest). But Kryo-based marshalling introduced a lot of complexity
> > to
> > > > the
> > > > > code and incorrect use of Kryo's numerous serializers caused some
> > weird
> > > > > hard-to-debug issues in the Ignite core (like duplicate cache entries
> > > due
> > > > > to incorrect marshalling of the GridDhtPArtitonFullMap class -- go
> > > > > figure!). Overall the Kryo-based solution is brittle and hard to
> > > > maintain.
> > > > >
> > > > > I feel the correct solution to OSGi problem would be to
> > > > > 1) enhance the OptimizedMarshaller to allow custom class resolution.
> > > > > 2) provide an OSGi-enabled OptimizedMarshaller (in addition to the
> > > > > original one) to be used in OSGi environment.
> > > > >
> > > > > Regards
> > > > > Andrey
> > > > >
> > > > > > From: ra...@apache.org
> > > > > > Date: Mon, 2 Nov 2015 12:41:47 +
> > > > > > Subject: Re: OSGi migration may require a special marshaller
> > > > > > To: dev@ignite.apache.org
> > > > > >
> > > > > > Hi Romain,
> > > > > >
> > > > > > I'm working on the OSGi compatibility of Ignite. I appreciate your
> > > > input.
> > > > > >
> > > > > > I'm thinking about the situation you describe and I wonder if
> > you're
> > > > > > exporting Ignite as an OSGi service which is then consumed from
> > other
> > > > > > bundles. Under this situation, it would be quite 

RE: OSGi migration may require a special marshaller

[Andrey Kornev]

Raul,

The fundamental hurdle we need to jump over to make Ignite OSGi-enabled is the 
marshalling. More specifically the issue is with deserialization of the classes 
that are provided by the bundles other than the Ignite bundle itself. 

When the Ignite transport layer receives a message it needs to figure out how 
to deserialize the bytes and for that it needs to know the bundle that provides 
the class to be deserailized. At this point TCCL is of no use. To make things 
more complex, the class may contain other classes that come from other bundles, 
and so on recursively. This means that each object in the hierarchy must be 
serialized with its bundle name (or bundle id), so that the deserializer will 
then be able to correctly resolve the class while traversing the object 
hierarchy during deserialization.

Unfortunately, Ignite's OptimizedMarshaller is lacking the ability to plug in a 
custom class resolver. Romain's solution was to use Kryo that does provide a 
way to customize class resolution. It has solved the problem of capturing the 
bundle info and he was able to successfully run Ignite as a bundle in an OSGi 
container (after some repackaging and inclusion of the manifest). But 
Kryo-based marshalling introduced a lot of complexity to the code and incorrect 
use of Kryo's numerous serializers caused some weird hard-to-debug issues in 
the Ignite core (like duplicate cache entries due to incorrect marshalling of 
the GridDhtPArtitonFullMap class -- go figure!). Overall the Kryo-based 
solution is brittle and hard to maintain.

I feel the correct solution to OSGi problem would be to 
1) enhance the OptimizedMarshaller to allow custom class resolution.
2) provide an OSGi-enabled OptimizedMarshaller (in addition to the original 
one) to be used in OSGi environment.

Regards
Andrey

> From: ra...@apache.org
> Date: Mon, 2 Nov 2015 12:41:47 +
> Subject: Re: OSGi migration may require a special marshaller
> To: dev@ignite.apache.org
> 
> Hi Romain,
> 
> I'm working on the OSGi compatibility of Ignite. I appreciate your input.
> 
> I'm thinking about the situation you describe and I wonder if you're
> exporting Ignite as an OSGi service which is then consumed from other
> bundles. Under this situation, it would be quite easy to reproduce the
> behaviour you describe if Ignite is not resolving classes via the TCCL.
> Need to dig deeper into that.
> 
> Off the top of my head, there are two alternatives to solve it:
> 
> 1. Use the TCCL for marshalling/unmarshalling (if not already used) – we
> gotta be wary of possible regressions.
> 2. Create a special OSGi header Ignite-Export-Package so that bundles
> containing DTOs can expose packages to Ignite's marshallers.
> 
> Regards,
> 
> *Raúl Kripalani*
> PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
> Messaging Engineer
> http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
> http://blog.raulkr.net | twitter: @raulvk
> 
> On Mon, Nov 2, 2015 at 9:56 AM, Gilles, Romain 
> wrote:
> 
> > Hi all,
> >
> > I'm really interested in this issue:
> > https://issues.apache.org/jira/browse/IGNITE-1270 . We some stuff to make
> > it work in our osgi environment. The main issue for us now is the
> > serialization. I think it you will have to rework the OptimizedMarshaller
> > or any other marshaller that works with object that come from outside your
> > class space.
> >
> > We have try kryo that works. Kryo provide an extension point in order to
> > resolve the classes:
> > https://github.com/EsotericSoftware/kryo/blob/master/src/com/esotericsoftware/kryo/ClassResolver.java
> > . With this extension we are able to solve the problem of external classes.
> > The only issue with kryo is that some classes need a certain care in the
> > serialization process and therefore a specialized serializer.
> >
> > So I would like to know from the community what do think of changing the
> > way the optimized marshaller works or introducing the support of yet
> > another marshaller based on a kryo like technology?
> >
> >
> > Thanks in advance,
> >
> >
> > Best regards,
> >
> >
> > Romain.
> >
> >
> > PS: I'm ready to help in the both cases.
> > "Misys" is the trade name of the Misys group of companies. This email and
> > any attachments have been scanned for known viruses using multiple
> > scanners. This email message is intended for the named recipient only. It
> > may be privileged and/or confidential. If you are not the named recipient
> > of this email please notify us immediately and do not copy it or use it for
> > any purpose, nor disclose its contents to any other person. This email does
> > not constitute the commencement of legal relations between you and Misys.
> > Please refer to the executed contract between you and the relevant member
> > of the Misys group for the identity of the contracting party with which you
> > are dealing.
> >
  

RE: OSGi migration may require a special marshaller

[Andrey Kornev]

Hey Raul,

What kind of use case do you have in mind when you suggest that a classdef may 
be located in different bundles on different machines? What would one have to 
do to end up in such a predicament? It feels more like a 
configuration/deployment issue rather than a desirable feature.

A more realistic assumption I believe is that the nodes will have (mostly) the 
same set of bundles deployed. (We even enforce it at runtime  by leveraging 
Ignite's PluginProvider.validateNewNode() method: we do not allow new nodes 
join the cluster if their OSGi environment is inconsistent.)

The "bundle name" in my original email really meant "bundle symbolic name + 
bundle version". Sorry for not being clear.

I'm not sure I understand why TCCL is even required for serialization?

Finally, I didn't quite grok your thinking about the custom manifest header. 
Could you please elaborate? What type of information do you propose to store in 
the header? How will the serializer be able to provide that info, and how will 
the deserializer use the info to load the classes? Also, who is responsible for 
making sure that the custom header includes all classes to be serialized? And 
how/when/by whom would such a header be generated?

Regards
Andrey

> Date: Mon, 2 Nov 2015 23:20:41 +
> Subject: RE: OSGi migration may require a special marshaller
> From: r...@evosent.com
> To: dev@ignite.apache.org
> 
> Hi Andrey,
> 
> Thanks for the participation in this topic.
> 
> I don't like the solution to incorporate the bundle symbolic name in the
> serialised form. Nothing guarantees that the classdef will be located under
> the same bundle in both source and target machines. We also have to take
> into account that OSGi allows for multiple versions of the same
> bundle/packages to co-exist in the same  container. So it becomes more
> complex.
> 
> Using the TCCL should work when serialising, but it'll probably be of no
> use when deserialising on the other end.
> 
> I need to enhance Ignite to:
> 
> 1. Use the TCCL when marshalling on one end.
> 2. Incorporate the package version of the class in the serialised form if
> Ignite is running in an OSGi environment.
> 3. On the receiver end, discover cache entities / DTOs in all bundles
> through a custom OSGi manifest header or the like, as I explained before.
> Package version must be taken into account.
> 
> What do you think?
> 
> Raúl.
> On 2 Nov 2015 17:25, "Andrey Kornev" <andrewkor...@hotmail.com> wrote:
> 
> > Raul,
> >
> > The fundamental hurdle we need to jump over to make Ignite OSGi-enabled is
> > the marshalling. More specifically the issue is with deserialization of the
> > classes that are provided by the bundles other than the Ignite bundle
> > itself.
> >
> > When the Ignite transport layer receives a message it needs to figure out
> > how to deserialize the bytes and for that it needs to know the bundle that
> > provides the class to be deserailized. At this point TCCL is of no use. To
> > make things more complex, the class may contain other classes that come
> > from other bundles, and so on recursively. This means that each object in
> > the hierarchy must be serialized with its bundle name (or bundle id), so
> > that the deserializer will then be able to correctly resolve the class
> > while traversing the object hierarchy during deserialization.
> >
> > Unfortunately, Ignite's OptimizedMarshaller is lacking the ability to plug
> > in a custom class resolver. Romain's solution was to use Kryo that does
> > provide a way to customize class resolution. It has solved the problem of
> > capturing the bundle info and he was able to successfully run Ignite as a
> > bundle in an OSGi container (after some repackaging and inclusion of the
> > manifest). But Kryo-based marshalling introduced a lot of complexity to the
> > code and incorrect use of Kryo's numerous serializers caused some weird
> > hard-to-debug issues in the Ignite core (like duplicate cache entries due
> > to incorrect marshalling of the GridDhtPArtitonFullMap class -- go
> > figure!). Overall the Kryo-based solution is brittle and hard to maintain.
> >
> > I feel the correct solution to OSGi problem would be to
> > 1) enhance the OptimizedMarshaller to allow custom class resolution.
> > 2) provide an OSGi-enabled OptimizedMarshaller (in addition to the
> > original one) to be used in OSGi environment.
> >
> > Regards
> > Andrey
> >
> > > From: ra...@apache.org
> > > Date: Mon, 2 Nov 2015 12:41:47 +
> > > Subject: Re: OSGi migration may require a special marshaller
> > > To: dev@ignite.apache.org
> > >
> > >

RE: OSGi migration may require a special marshaller

[Andrey Kornev]

Hello,

We've made an attempt to formalize the approach to Ignite's OSGi enablement: 
https://cwiki.apache.org/confluence/display/IGNITE/OSGI+Compatibility. Please 
have a look and feel free to provide your positive feedback :)))

Thanks
Andrey

> From: dsetrak...@apache.org
> Date: Wed, 4 Nov 2015 09:26:17 -0800
> Subject: Re: OSGi migration may require a special marshaller
> To: dev@ignite.apache.org
> 
> On Wed, Nov 4, 2015 at 9:07 AM, Romain Gilles 
> wrote:
> 
> > Hi Dmitriy,
> > I found this post that explain how to find a bundle based on its bundle
> > name and version.
> > This post explain for the past to now how to do it in the standard way with
> > "pull" approach: https://www.eclipse.org/forums/index.php/t/205719/
> > Regarding the version of OSGi you want to support then some solutions will
> > works some others will not.
> > There is an other way to do this stuff without using those "pull" style
> > approach based on BundleTracker. If you want I can give you the code to do
> > it with BundlTracker. I think with this solution you will support a wider
> > range of OSGi version.
> >
> 
> Romain, if you can provide a generic code sample to look up a ClassLoader
> in OSGI based on manifest properties, would be great.
> 
> 
  

LOCAL cache serializes the entries?

[Andrey Kornev]

Hello,

It seems that even if the cache is configured as LOCAL, the entries are 
nevertheless stored in serialized form. If that's the case, could someone 
explain the reasons?

Regards
Andrey
  

RE: LOCAL cache serializes the entries?

[Andrey Kornev]

Dmitriy,

This design is unfortunate, but I understand the reason. My issue is not with 
performance impact of serialization, but with having to make my key/value 
serializable which is not always possible nor desirable. Well, back to Guava's 
Cache!

Thanks
Andrey

> From: dsetrak...@apache.org
> Date: Thu, 5 Nov 2015 17:34:28 -0800
> Subject: Re: LOCAL cache serializes the entries?
> To: dev@ignite.apache.org
> 
> On Thu, Nov 5, 2015 at 6:44 AM, Andrey Kornev <andrewkor...@hotmail.com>
> wrote:
> 
> > Hello,
> >
> > It seems that even if the cache is configured as LOCAL, the entries are
> > nevertheless stored in serialized form. If that's the case, could someone
> > explain the reasons?
> >
> 
> Andrey, JCache has pass-by-value semantics by default, which means that
> whenever you do a get from cache, you have to get a copy of the value, not
> the same reference. To achieve that, we must serialize value on Put, so
> that we can deserialize it on Get and get a different copy.
> 
> You have an option to set copyOnGet parameter to false, in which case
> Ignite will cache the deserialized reference after the 1st Get.
> 
> 
> >
> > Regards
> > Andrey
> >
  

RE: Cluster group affinity

[Andrey Kornev]

Dmitriy,

This approach would definitely work, if it wasn't for the fact that the cluster 
groups in my case are created dynamically and may include any combination of 
nodes in the cluster (where the number of combinations grows exponentially with 
the number of nodes in the cluster). I don't think it's practical to create 
that many caches.

I still can't get why the affinity function can't be applied to an arbitrary 
cluster group, and why it must necessarily be a cache. Isn't the cache affinity 
just a special case of the cluster group affinity defined as 
ClusterGroup.forCache()?

Thanks
Andrey

> From: dsetrak...@apache.org
> Date: Tue, 6 Oct 2015 12:07:39 -0700
> Subject: Re: Cluster group affinity
> To: dev@ignite.apache.org
> 
> On Tue, Oct 6, 2015 at 8:46 AM, Andrey Kornev <andrewkor...@hotmail.com>
> wrote:
> 
> > Thanks, Andrey! This definitely helps.
> >
> > It's just that implementing such a simple feature in the "user space"
> > feels awkward and requires intimate knowledge of  fairly low-level details
> > of how things work in the current version.
> >
> > Just curios, how about providing an override for Ignite.affinity() method
> > that ClusterGroup? Is there something fundamentally wrong about calculating
> > the affinity for an arbitrary collection of nodes (such as a ClusterGroup
> > is)?
> >
> 
> Affinity is usually associated with data. In your case you have no data,
> but you still need keys to be always mapped to the same node.  How about
> creating an empty cache and using standard cache API for determining the
> affinity for a key?
> 
> 
> > Regards
> > Andrey
> >
> > > Date: Tue, 6 Oct 2015 18:12:48 +0300
> > > Subject: Re: Cluster group affinity
> > > From: ag...@gridgain.com
> > > To: dev@ignite.apache.org
> > >
> > > Andrey,
> > >
> > >
> > > > 1) I'm expected to return an instance of the internal class
> > > > AffinityTopologyVersion.
> > >
> > >
> > > If you are talking about AffinityContextFunction.currentTopologyVersion
> > > method then for now this method is nowhere uses. But it make sense to
> > > return non null value in order to avoid problems in the future.
> > >
> > > 2) the consequences of returning null from
> > > > AffinityFunctionContext.previousAssignment and
> > > > AffinityFunctionContext.discoveryEvent methods (because I can't
> > provide any
> > > > meaningful implementation for them) are not clear.
> > > >
> > >
> > > Both methods declared as @Nullable, so affinity function developer should
> > > correctly handle this cases. In Ignite only FairAffinityFunction uses
> > these
> > > methods. FairAffinityFunction tries to obtain left node Id from event of
> > > EventType.EVT_NODE_LEFT or EventType.EVT_NODE_FAILED type. It needs to
> > > exclude this node assignment from previous assignments. So if your
> > cluster
> > > group lost node you can return EVT_NODE_LEFT discovery event with Id of
> > > lost node from discoveryEvent method and assignments for previous cluster
> > > group state from previousAssignment method.
> > >
> > > RendezvousAffinityFunction uses only currentTopologySnapshot() and
> > > backups() methods of AffinityFunctionContext interface.
> > >
> > >
> > > On Tue, Oct 6, 2015 at 5:07 PM, Andrey Kornev <andrewkor...@hotmail.com>
> > > wrote:
> > >
> > > > Andrey, thanks!
> > > >
> > > > But a "properly formed AffinityFunctionContext" is the problem:
> > > > 1) I'm expected to return an instance of the internal class
> > > > AffinityTopologyVersion.
> > > > 2) the consequences of returning null from
> > > > AffinityFunctionContext.previousAssignment and
> > > > AffinityFunctionContext.discoveryEvent methods (because I can't
> > provide any
> > > > meaningful implementation for them) are not clear.
> > > >
> > > > Please advise.
> > > >
> > > > Thanks
> > > > Andrey
> > > >
> > > > > Date: Tue, 6 Oct 2015 16:43:10 +0300
> > > > > Subject: Re: Cluster group affinity
> > > > > From: ag...@gridgain.com
> > > > > To: dev@ignite.apache.org
> > > > >
> > > > > Andrey,
> > > > >
> > > > > See AffinityFunction.assignPartitions method. It returns assignment
> > list
> > > > as
> &

RE: Cluster group affinity

[Andrey Kornev]

Thanks, Yakov! I'm going to use your code (I hope it's Apache2 licensed:)))

As for the use case, it's quite simple. I have a group of compute nodes that 
are being sent a stream of work items and I'd like to parallelize processing by 
partitioning the stream across all the nodes in the group by some attribute. 
This is pure compute, no caches.

Thanks
Andrey

> Date: Wed, 7 Oct 2015 13:46:23 +0300
> Subject: Re: Cluster group affinity
> From: yzhda...@apache.org
> To: dev@ignite.apache.org
> 
> Andrey, probably it is, but I am not sure if I have ever had a thought for
> mentioned scenario.
> 
> I think you should get existing implementation and use it like this. Make
> sure to cache assignments as this may be quiet expensive operation.
> 
> Ignite ignite = Ignition.start(cfg);
> 
> ClusterGroup group = ignite.cluster().forPredicate(new Predicate());
> 
> final List snapshot = new ArrayList<>(group.nodes());
> 
> RendezvousAffinityFunction aff = new RendezvousAffinityFunction();
> 
> List<List> parts = aff.assignPartitions(new
> AffinityFunctionContext() {
> @Nullable @Override public List
> previousAssignment(int part) {
> return null;
> }
> 
> @Override public int backups() {
> return 0;
> }
> 
> @Override public List currentTopologySnapshot() {
> return snapshot;
> }
> 
> @Override public AffinityTopologyVersion
> currentTopologyVersion() {
> return null;
> }
> 
> @Nullable @Override public DiscoveryEvent discoveryEvent() {
> return null;
> }
> });
> 
>     // Picking node.
> ClusterNode node = parts.get(aff.partition(key)).get(0);
> 
> 
> --Yakov
> 
> 2015-10-07 11:39 GMT+03:00 Andrey Kornev <andrewkor...@hotmail.com>:
> 
> > Dmitriy,
> >
> > This approach would definitely work, if it wasn't for the fact that the
> > cluster groups in my case are created dynamically and may include any
> > combination of nodes in the cluster (where the number of combinations grows
> > exponentially with the number of nodes in the cluster). I don't think it's
> > practical to create that many caches.
> >
> > I still can't get why the affinity function can't be applied to an
> > arbitrary cluster group, and why it must necessarily be a cache. Isn't the
> > cache affinity just a special case of the cluster group affinity defined as
> > ClusterGroup.forCache()?
> >
> > Thanks
> > Andrey
> >
> > > From: dsetrak...@apache.org
> > > Date: Tue, 6 Oct 2015 12:07:39 -0700
> > > Subject: Re: Cluster group affinity
> > > To: dev@ignite.apache.org
> > >
> > > On Tue, Oct 6, 2015 at 8:46 AM, Andrey Kornev <andrewkor...@hotmail.com>
> > > wrote:
> > >
> > > > Thanks, Andrey! This definitely helps.
> > > >
> > > > It's just that implementing such a simple feature in the "user space"
> > > > feels awkward and requires intimate knowledge of  fairly low-level
> > details
> > > > of how things work in the current version.
> > > >
> > > > Just curios, how about providing an override for Ignite.affinity()
> > method
> > > > that ClusterGroup? Is there something fundamentally wrong about
> > calculating
> > > > the affinity for an arbitrary collection of nodes (such as a
> > ClusterGroup
> > > > is)?
> > > >
> > >
> > > Affinity is usually associated with data. In your case you have no data,
> > > but you still need keys to be always mapped to the same node.  How about
> > > creating an empty cache and using standard cache API for determining the
> > > affinity for a key?
> > >
> > >
> > > > Regards
> > > > Andrey
> > > >
> > > > > Date: Tue, 6 Oct 2015 18:12:48 +0300
> > > > > Subject: Re: Cluster group affinity
> > > > > From: ag...@gridgain.com
> > > > > To: dev@ignite.apache.org
> > > > >
> > > > > Andrey,
> > > > >
> > > > >
> > > > > > 1) I'm expected to return an instance of the internal class
> > > > > > AffinityTopologyVersion.
> > > > >
> > > > >
> > > > > If you are talking about
> > AffinityContextFunction.currentTopologyVersion
> > > > > method th

Re: Cluster group affinity

[Andrey Kornev]

Maybe I don't quite get your point, Andrey, but to me, the cache affinity nodes 
could just be equally expressed as a ClusterGroup.forCache(...) cluster group. 
Then, why can't Ignite expose just a single method that takes a cluster group 
(rather than a cache name) as a parameter and returns an Affinity instance 
corresponding to the group? Cheers!
_
From: Andrey Gura <ag...@gridgain.com>
Sent: Tuesday, October 6, 2015 9:31 PM
Subject: Re: Cluster group affinity
To:  <dev@ignite.apache.org>


   >   
 > Just curios, how about providing an override for Ignite.affinity() method   
 > that ClusterGroup? Is there something fundamentally wrong about calculating  
 >  
 > the affinity for an arbitrary collection of nodes (such as a ClusterGroup   
 > is)?   
 >   

 Andrey,   

 I think Ignite.affinity() method for ClusterGroup does not make sense in   
 API because for each cache topology snapshot always consist of cache   
 affinity nodes. However, I don't see any fundamentally problems in using   
 affinity fucntion for some collection of nodes. But you should make sure   
 that your code is aware of changes in this collection.   

 On Tue, Oct 6, 2015 at 6:46 PM, Andrey Kornev <andrewkor...@hotmail.com>   
 wrote:   

 > Thanks, Andrey! This definitely helps.   
 >   
 > It's just that implementing such a simple feature in the "user space"   
 > feels awkward and requires intimate knowledge of  fairly low-level details   
 > of how things work in the current version.   
 >   
 > Just curios, how about providing an override for Ignite.affinity() method   
 > that ClusterGroup? Is there something fundamentally wrong about calculating  
 >  
 > the affinity for an arbitrary collection of nodes (such as a ClusterGroup   
 > is)?   
 >   
 > Regards   
 > Andrey   
 >   
 > > Date: Tue, 6 Oct 2015 18:12:48 +0300   
 > > Subject: Re: Cluster group affinity   
 > > From: ag...@gridgain.com   
 > > To: dev@ignite.apache.org   
 > >   
 > > Andrey,   
 > >   
 > >   
 > > > 1) I'm expected to return an instance of the internal class   
 > > > AffinityTopologyVersion.   
 > >   
 > >   
 > > If you are talking about AffinityContextFunction.currentTopologyVersion   
 > > method then for now this method is nowhere uses. But it make sense to   
 > > return non null value in order to avoid problems in the future.   
 > >   
 > > 2) the consequences of returning null from   
 > > > AffinityFunctionContext.previousAssignment and   
 > > > AffinityFunctionContext.discoveryEvent methods (because I can't   
 > provide any   
 > > > meaningful implementation for them) are not clear.   
 > > >   
 > >   
 > > Both methods declared as @Nullable, so affinity function developer should  
 > >  
 > > correctly handle this cases. In Ignite only FairAffinityFunction uses   
 > these   
 > > methods. FairAffinityFunction tries to obtain left node Id from event of   
 > > EventType.EVT_NODE_LEFT or EventType.EVT_NODE_FAILED type. It needs to   
 > > exclude this node assignment from previous assignments. So if your   
 > cluster   
 > > group lost node you can return EVT_NODE_LEFT discovery event with Id of   
 > > lost node from discoveryEvent method and assignments for previous cluster  
 > >  
 > > group state from previousAssignment method.   
 > >   
 > > RendezvousAffinityFunction uses only currentTopologySnapshot() and   
 > > backups() methods of AffinityFunctionContext interface.   
 > >   
 > >   
 > > On Tue, Oct 6, 2015 at 5:07 PM, Andrey Kornev <andrewkor...@hotmail.com>   
 > > wrote:   
 > >   
 > > > Andrey, thanks!   
 > > >   
 > > > But a "properly formed AffinityFunctionContext" is the problem:   
 > > > 1) I'm expected to return an instance of the internal class   
 > > > AffinityTopologyVersion.   
 > > > 2) the consequences of returning null from   
 > > > AffinityFunctionContext.previousAssignment and   
 > > > AffinityFunctionContext.discoveryEvent methods (because I can't   
 > provide any   
 > > > meaningful implementation for them) are not clear.   
 > > >   
 > > > Please advise.   
 > > >   
 > > > Thanks   
 > > > Andrey   
 > > >   
 > > > > Date: Tue, 6 Oct 2015 16:43:10 +0300   
 > > > > Subject: Re: Cluster group affinity   
 > > > > From: ag...@gridgain.com   
 > > > > To: dev@ignite.apache.org   
 > > > >   
 > > > > Andrey,   
 > > > >

Re: Which thread is used to run IgniteFuture continuations?

[Andrey Kornev]

Excellent idea! +10
PS. Got burnt by this a few times already
_
From: Vladimir Ozerov 
Sent: Friday, October 9, 2015 4:22 PM
Subject: Which thread is used to run IgniteFuture continuations?
To:  


   Igniters,   

 We are missing an ability to specify which thread should run continuation   
 routine when future is completed.   

 Currently we either run routine in the callee thread or in completion   
 thread (usually system pool thread). It means user cannot have any blocking   
 or cache operations inside the continuation. The code below could surprise   
 user with either a deadlock or system pool starvation:   

 final CountDownLatch latch = new CountDownLatch();   

 Cache cache = ignite.cache().withAsync();   
 cache.invoke(...);   

 cache.future().listen(() => {   
     latch.await();   
     /** Do something useful. */   
 });   

 /** Do something else. */   
 latch.countDown();   

 Java 8 and Hazelcast already support custom thread pools for continuations.   
 E.g.:   
 Hazelcast.CompletableFutureTask.andThen(ExecutionCallback callback,   
 Executor executor);   

 Looks like we should allow users to specify optional thread pool in futures   
 likewise.   

 Thoughts?   

 Vladimir.

Services collocation

[Andrey Kornev]

Hello,

I'd like to be able to deploy my cluster-wide singleton services all collocated 
on the same cluster node. What is the best way to achieve that?

Ideally, I'd like to be able to define the "service groups" where all services 
in a group would be deployed on the same node, whereas the different groups 
would be load balanced all over the cluster. The services included in the same 
group would usually the ones that share their data or/and closely collaborate 
on the same task. By collocating services in such a way, I'll be able to 
optimize some of the execution paths (reducing the number of remote 
invocations, for example) and have in-process coordination between the services 
(if necessary) as a bonus.

I've considered the affinity-based collocation (where the "group" would consist 
of the all services configured with the same affinity key), but the drawback, 
IMHO, is that topology changes may cause unnecessary mass migration of the 
services to the new primary node where the services get then restarted. I'd 
like to avoid paying the service shutdown/start-up overhead as much as 
possible, since in my case it's pretty expensive.

Any ideas/suggestion are very much appreciated!

Thanks
Andrey
  

Cluster group affinity

[Andrey Kornev]

Hello,

I have a user-defined cluster group and I'd like to be able to consistently 
pick the same node in the group for a given key. Essentially, what I want is a 
cluster group affinity that is not associated with any cache. How can I do it?

Thanks
Andrey
  

RE: Azul Zing

[Andrey Kornev]

According to my sources ;) they are quite popular in the finance.

I'll create a ticket anyway and let the community decide. :)

Andrey

> Date: Tue, 22 Sep 2015 10:09:48 -0700
> From: c...@apache.org
> To: dev@ignite.apache.org
> Subject: Re: Azul Zing
> 
> I have heard some conversations that most of their sales are coming from the
> commercial support of OpenJDK. What I am saying that if the support effort of
> Azul is huge, may be there's no point of doing this... ;)?
> 
> Cos
> 
> On Tue, Sep 22, 2015 at 11:47AM, Dmitriy Setrakyan wrote:
> > On Tue, Sep 22, 2015 at 10:45 AM, Andrey Kornev <andrewkor...@hotmail.com>
> > wrote:
> > 
> > > Hello,
> > >
> > > Since it looks like there is an effort underway to certify Ignite on
> > > different JVMs, I'm wondering if Azul Zing could also be considered as a
> > > potential candidate?
> > >
> > 
> > Andrey, I don't see why not. Can you please file a ticket?
> > 
> > 
> > >
> > > Regards
> > > Andrey
> > >
  

Azul Zing

[Andrey Kornev]

Hello,

Since it looks like there is an effort underway to certify Ignite on different 
JVMs, I'm wondering if Azul Zing could also be considered as a potential 
candidate?

Regards
Andrey
  

RE: EntryProcessor invoked twice

[Andrey Kornev]

Dmitriy,

Alexey's point is just an explanation of the existing implementation and not 
how it is supposed to be.  And while nobody really knows what the correct 
behavior should be, the one implemented by Ignite is incredibly 
over-engineered. As I argued in a different thread, other vendors provide more 
reasonable implementations and their users seem to be quite content with the 
performance.

Cheers
Andrey

> From: dsetrak...@gridgain.com
> Date: Thu, 3 Dec 2015 10:35:34 -0800
> Subject: Re: EntryProcessor invoked twice
> To: dev@ignite.apache.org
> 
> Point taken.
> 
> However I still don’t get why it should be called twice in PESSIMISTIC
> mode. Can someone explain?
> 
> D.
> 
> On Thu, Dec 3, 2015 at 10:04 AM, Andrey Gura  wrote:
> 
> > Hi,
> >
> > Recently Alexey Goncharuk wrote in thread "EntryProcessor execution
> > semantics":
> >
> > For example, in a case of explicit optimistic
> > > READ_COMMITTED transaction it may be called more than once because Ignite
> > > needs to calculate a return value for the first invoke() and then it
> > should
> > > be called second time during commit when transactional locks are held.
> >
> >
> > Current requirement is that an EntryProcessor should be a stateless
> > > function, and it may be called more than once (but of course it will
> > > receive the same cache value every time).
> > >
> >
> >
> >
> > On Thu, Dec 3, 2015 at 8:46 PM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> > > Dmitry,
> > >
> > > Counter will not be incremented twice, because the entry is locked and EP
> > > is guaranteed to be called both times for the same old value. So there is
> > > no critical bug here in my view, but I just don't see the reason why we
> > > call it two times instead of one for one operation.
> > >
> > > -Val
> > >
> > > On Thu, Dec 3, 2015 at 1:22 AM, Dmitriy Setrakyan  > >
> > > wrote:
> > >
> > > > I think this is definitely a bug. Imagine that EP is used to maintain a
> > > > counter, and the counter is incremented with every update. In this case
> > > the
> > > > counter will be updated 2 times instead of 1.
> > > >
> > > > Alexey G., would be nice to hear your thoughts here.
> > > >
> > > > D.
> > > >
> > > > On Wed, Dec 2, 2015 at 9:08 PM, Valentin Kulichenko <
> > > > valentin.kuliche...@gmail.com> wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > I noticed that when I execute cache.invoke() in transactional cache,
> > > it's
> > > > > invoked twice for the same old value on the primary node. First
> > > > invocation
> > > > > is done on prepare step [1], the second one happens on commit [2]. Is
> > > > this
> > > > > expected behavior? Why can't we reuse already calculated new value
> > > during
> > > > > commit?
> > > > >
> > > > > [1]
> > > > > java.lang.RuntimeException: XXX
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> > com.workday.fabric.ScratchClient$ScratchEntryProcessor.process(ScratchClient.java:106)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> > com.workday.fabric.ScratchClient$ScratchEntryProcessor.process(ScratchClient.java:96)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.onEntriesLocked(GridDhtTxPrepareFuture.java:381)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare0(GridDhtTxPrepareFuture.java:909)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.mapIfLocked(GridDhtTxPrepareFuture.java:527)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare(GridDhtTxPrepareFuture.java:822)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.prepareAsync(GridDhtTxLocal.java:462)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> > org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.prepareNearTx(IgniteTxHandler.java:406)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> > org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.prepareTx(IgniteTxHandler.java:200)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> > org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxPrepareRequest(IgniteTxHandler.java:101)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> > org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$1.apply(IgniteTxHandler.java:114)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> > org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$1.apply(IgniteTxHandler.java:112)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> > 

EntryProcessor execution semantics

[Andrey Kornev]

Hello,

I'd like to ask the community members to share their thoughts/opinions on the 
following subject.

JCache provides a way to atomically execute one or more actions against a cache 
entry using the Entry Processor mechanism. The Cache interface exposes an 
invoke() method that takes a key of the cache entry to be acted upon and an 
instance of the EntryProcessor interface as its parameters. When invoked, the 
entry processor will be passed an instance of JCache's MutableEntry which gives 
the processor exclusive access to the cache entry for the duration of the 
EntryProcessor.process() call. Great feature for delta updates, in-place 
compute, coordination/agreement (a la zookeeper), and so on!

Now, if one were to put his Ignite user hat on, what execution semantics would 
you as a user expect? Specifically,
1) the EntryProcessor is executed on the key's primary node as well as all 
backup nodes.
2) the EntryProcessor is executed only on the key's primary node.
3) something else.

Unfortunately JCache spec doesn't provide much details on this feature. Ignite 
documentation is silent, too.

Thanks
Andrey
  

RE: EntryProcessor execution semantics

[Andrey Kornev]

Thank you, Alexey!

By stating that "sending a serialized EntryProcessor should be cheaper" you 
implicitly assume that the cache entry is big and the computation done by the 
processor is cheap. But what if it's not the case? What if the computation 
itself is quite expensive and depends on external data (which may happen to be 
constantly changing -- like the stock tickers?), or is done for a side effect? 
What is the EP feature good for after all, given the constraints you posed 
below? Incrementing an integer counter, as the example in Ignite documentation 
does? :)

Of course, JCache specification is open to interpretation, and one might argue 
that the EntryProcessor is a performance feature, but my reading of the spec 
makes me think  (and it looks like both Coherence and Hazelcast agree with me) 
that it's first and foremost a way to atomically mutate a cache entry without 
incurring an overhead of locking.

Let's see now. A single call to Cache.invoke() produces
- a single EP invocation on the key's primary node in Coherence. Period.
- a single EP invocation on the key's primary node in Hazelcast, but they offer 
the non-JCache BackupAwareEntryProcessor class that allows the user "to create 
or pass another EntryProcessor to run on backup
partitions and apply delta changes to the backup entries".
- In Ignite: 
-- a single invocation on the key's primary node if the cache is ATOMIC (both 
REPLICATED and PARTITIONED).
-- N+1 invocations (where N is the number of nodes the cache is started on) if 
the cache is REPLICATED and TRANSACTIONAL.
-- B+2 invocations (where B is the number of replicas) if the cache is 
PARTITIONED and TRANSACTIONAL.

Go figure! Alexey, you're suggesting that a user without deep knowledge of 
Ignite internals would find such behavior expected and natural? Even with deep 
knowledge of Ignite internals it's hard to understand the logic.

Neither Coherence nor Hazelcast require the EP to be stateless and side-effect 
free. Even better Hazelcast makes the choice explicit by providing the backup 
aware processor API and it's then up to the user to ensure statelessness etc. 
But Ignite is just too clever.

I'd really like to ask the brains behind the current design to reconsider.

Regards
Andrey

> Date: Mon, 30 Nov 2015 13:11:13 +0300
> Subject: Re: EntryProcessor execution semantics
> From: alexey.goncha...@gmail.com
> To: dev@ignite.apache.org
> 
> Andrey,
> 
> If I leave behind my knowledge about Ignite internals, my expectation would
> be that an EntryProcessor is invoked on all affinity - both primary and
> backup - nodes in the grid. The main reason behind this expectation is that
> usually a serialized EntryProcessor instance is smaller than resulting
> object being stored in the cache, so sending a serialized EntryProcessor
> should be cheaper. Is there a specific reason you expect an EntryProcessor
> to be called only once across all the nodes?
> 
> I would not imply any restrictions on how many times an EntryProcessor is
> called during a cache update. For example, in a case of explicit optimistic
> READ_COMMITTED transaction it may be called more than once because Ignite
> needs to calculate a return value for the first invoke() and then it should
> be called second time during commit when transactional locks are held.
> 
> Current requirement is that an EntryProcessor should be a stateless
> function, and it may be called more than once (but of course it will
> receive the same cache value every time). I agree that this should be
> properly articulated in the documentation, I will make sure that it will be
> reflected in the forthcoming 1.5 release javadocs.
  

RE: [VOTE] Apache Ignite 1.5.0-EA

[Andrey Kornev]

Raul,

I'm not associated with GG and my opinion is that Yakov's naming is good enough.

And since you've asked for opinions, it's also my opinion that the discussion 
about the naming was a non-consequential bikeshedding 
(https://en.wikipedia.org/wiki/Parkinson's_law_of_triviality) and potentially 
causing unnecessary delays of the release which is already long overdue.

Regards
Andrey

> Date: Mon, 30 Nov 2015 22:59:44 +
> Subject: Re: [VOTE] Apache Ignite 1.5.0-EA
> From: ra...@apache.org
> To: dev@ignite.apache.org
> 
> Yakov,
> 
> We were having an active discussion in the dev mailing list about the name
> of this version.
> 
> The discussion had not concluded. It is not respectful for you to override
> the disagreement and take a decision.
> 
> I would like to hear the opinion of other members outside of GG about the
> naming.
> 
> What's the urgency with this release? Has this community communicated a
> timeline to users@?
> 
> I'm tempted to -1 this release just due to the manners.
> 
> Regards,
> Raúl.
> Dear Sirs!
> 
> We have uploaded release candidate to
> https://dist.apache.org/repos/dist/dev/ignite/1.5.0-EA-rc2/
> 
> This is very important release containing initial versions of Ignite.NET
> and Ignite C++ and many other important features and improvements.
> We want community to test this EA in order to gather feedback and move on
> to releasing final version.
> 
> Tag name is
> ignite-1.5.0-EA-rc2
> 
> 1.5.0 changes:
> * Ignite.NET: Initial Release.
> * Ignite C++: Initial Release.
> * Massive performance improvements for cache operations and SQL.
> * Added new binary cache object marshalling implementation.
> * Added IgniteSemaphore data structure.
> * Added MQTT Streamer.
> * Fixed failover for continuous queries.
> * Fixed compilation and runtime errors under OpenJDK and IBM JDK.
> * Fixed Integer.size limitation for cache.
> * Fixed and improved cache types configuration.
> * Fixed cache rebalancing.
> * Many stability and fault-tolerance fixes.
> 
> Complete list of closed issues:
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%201.5%20AND%20status%20%3D%20closed
> 
> DEVNOTES
> https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_plain;f=DEVNOTES.txt;hb=refs/tags/ignite-1.5.0-EA-rc2
> 
> RELEASENOTES
> https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_plain;f=RELEASE_NOTES.txt;hb=refs/tags/ignite-1.5.0-EA-rc2
> 
> Please start voting.
> 
> +1 - to accept Apache Ignite 1.5.0-EA
> 0 - don't care either way
> -1 - DO NOT accept Apache Ignite 1.5.0-EA (explain why)
> 
> This vote will go for 72 hours.
> 
> --Yakov
  

RE: EntryProcessor execution semantics

[Andrey Kornev]

Dmitriy,

Here, by "stateless" I meant whatever Alexey meant in his previous post in this 
thread. But I'm really talking about being able to have EPs with side effects 
and therefore the execution semantics should be "exactly-once" by default. 
Besides, maybe it's just me, but intuitively the expectation of Cache.invoke() 
is that the EP will be executed only once because *logically* there can only be 
one entry with the given key in the cache to which the EP is applied. Having 
the EP executed many times for the same entry comes as a big surprise, at least 
to me.

Maybe it's worth considering an API similar to what Hazelcast has to make it 
possible to explicitly control EP's execution semantics.

Regards
Andrey

> From: dsetrak...@apache.org
> Date: Mon, 30 Nov 2015 23:16:58 -0800
> Subject: Re: EntryProcessor execution semantics
> To: dev@ignite.apache.org
> 
> On Mon, Nov 30, 2015 at 9:02 AM, Andrey Kornev <andrewkor...@hotmail.com>
> wrote:
> 
> >
> > Neither Coherence nor Hazelcast require the EP to be stateless and
> > side-effect free. Even better Hazelcast makes the choice explicit by
> > providing the backup aware processor API and it's then up to the user to
> > ensure statelessness etc. But Ignite is just too clever.
> >
> 
> Andrey, stateful EP seems a bit utopian to me, since the state would not
> survive between executions anyway. Can you elaborate?
  

RE: API for asynchronous execution.

[Andrey Kornev]

Hello,

I'd like to suggest making the point about the callback execution ordering more 
prominent in the Ignite documentation and/or the javadocs of 
@IgniteAsyncCallback. I think the documentation should emphasize the fact  that 
the events for the same keys are guaranteed to be delivered in-order, despite 
the asynchronous execution of the callback. Without such guarantee, the 
applicability of the async callback feature would be very limited.

Thanks
Andrey 

> Date: Tue, 29 Mar 2016 19:22:50 +0300
> Subject: Re: API for asynchronous execution.
> From: ntikho...@gridgain.com
> To: dev@ignite.apache.org
> 
> Vova,
> 
> 1) As far as I remember, we enforce ordering of updates for the same key,
> > right? If yes, then stalled filter invoke for update #1 will stall all
> > further updates.
> >
> 
> Invocation of filter will be implemented in thread-partition style for
> supporting ordering of updates.
> 
> 
> > 2) Listener notification is final stage of update. To the contrast remote
> > filter is intermediate stage. It means that you will have to maintain some
> > kind of beckpressure control for it's async invocations. If there are too
> > much concurrent invokes, you will eventually block system pool waiting for
> > some invokes to finish. Ultimately it means that user cannot execute any
> > other cache operations inside the filter irrespecitve of whether it is
> > synchronous or async.
> > Having these limitations in place - what is the reason to have async remote
> > filter?
> 
> From my point of view, neither remote filter, nor invoke are valid use
> > cases for async execution.
> >
> 
> Main reason async execution to allow users use cache operations in remote
> filter.
  

IgniteBinary, POJOs and indexing

[Andrey Kornev]

Hello, Ignite Community,

IgniteBinary javadocs mention the ability to dynamically change the structure 
of the "classes" without restarting the cluster. The javadocs even say that the 
new fields become automatically available to the SQL queries with no extra 
effort. It's all fine and dandy, but how can I tell Ignite that some of the new 
fields should also be indexed?

Using the example from the same javadocs, my binary object initially has two 
field A and B. I configure the cache entry metadata to index the field A and 
then create the cache. When later I change the structure by adding a new field 
C, how can I tell Ignite that I now want only fields B and C to be included in 
the index?

Related to above,  is there any way to modify the indexes after the cache has 
been started? In this case I do not modify the structure of the class, but 
rather change which fields get indexed, the sorting properties, etc.

Finally, how about introducing new POJO classes at runtime (yes, I can do it -- 
I run in OSGi environment)? For example, at the cache creation time the cache 
metadata only had the POJOA class annotated to index its field "foo", and then 
later the user introduces a new POJOB class annotated to index its field "bar". 
Would POJOB start getting indexed automatically, or the user will be given the 
finger?

Any input will be very much appreciated!
Andrey
  

RE: IgniteBinary, POJOs and indexing

[Andrey Kornev]

Sergi,

Thanks for your reply.

I didn't quite get the point about the "persistent configuration storage". 
Could you please elaborate? 

>From my point of view, dynamic index management requires something similar to 
>the SQL DDL, like create/drop index, alter index, etc. So in addition to the 
>indexing metadata in the cache config, Ignite could provide an index 
>management API: create, drop, alter, describe. It would be up to the user to 
>ensure that the indexes get recreated after the cluster is restarted.

Such approach is no different from the way Ignite currently handles dynamically 
created caches -- there is no "persistent configuration storage" to store the 
dynamic cache configs, and Ignite doesn't even try to recreate them after a 
full cluster restart -- it's the use who does that, either in the code or thru 
the configuration files.

Thanks
Andrey

> From: sergi.vlady...@gmail.com
> Date: Sat, 9 Jan 2016 13:10:13 +0300
> Subject: Re: IgniteBinary, POJOs and indexing
> To: dev@ignite.apache.org
> 
> I don't think we can easily implement this feature. Because it means that
> we need some kind of "persistent configuration storage", which will be
> consistent in all the scenarios of failures/restarts. So we have to design
> this thing first keeping in mind that in the future we will store there not
> only "dynamic indexes" but some other dynamic configuration as well.
> 
> I don't think I will be able to work on it myself but since it is going to
> be a general mechanism mostly unrelated to SQL we can ask someone else to
> pick it up.
> 
> Sergi
> 
> 
> 2016-01-08 21:20 GMT+03:00 Dmitriy Setrakyan <dsetrak...@apache.org>:
> 
> > Hi Andrey,
> >
> > The answer right now is simple. Yes, you can create classes and add fields
> > at runtime. No, you cannot change indexes that you have created in cache
> > configuration.
> >
> > However, you are raising a very valid point. We should be able to create
> > and update indexes dynamically.  There is a ticket, IGNITE-735 [1], that
> > has been sitting around for a while. I think we should reprioritize it.
> >
> > Sergi, can you please comment on how hard this would be to implement?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-735
> >
> > On Fri, Jan 8, 2016 at 9:02 AM, Andrey Kornev <andrewkor...@hotmail.com>
> > wrote:
> >
> > > Hello, Ignite Community,
> > >
> > > IgniteBinary javadocs mention the ability to dynamically change the
> > > structure of the "classes" without restarting the cluster. The javadocs
> > > even say that the new fields become automatically available to the SQL
> > > queries with no extra effort. It's all fine and dandy, but how can I tell
> > > Ignite that some of the new fields should also be indexed?
> > >
> > > Using the example from the same javadocs, my binary object initially has
> > > two field A and B. I configure the cache entry metadata to index the
> > field
> > > A and then create the cache. When later I change the structure by adding
> > a
> > > new field C, how can I tell Ignite that I now want only fields B and C to
> > > be included in the index?
> > >
> > > Related to above,  is there any way to modify the indexes after the cache
> > > has been started? In this case I do not modify the structure of the
> > class,
> > > but rather change which fields get indexed, the sorting properties, etc.
> > >
> > > Finally, how about introducing new POJO classes at runtime (yes, I can do
> > > it -- I run in OSGi environment)? For example, at the cache creation time
> > > the cache metadata only had the POJOA class annotated to index its field
> > > "foo", and then later the user introduces a new POJOB class annotated to
> > > index its field "bar". Would POJOB start getting indexed automatically,
> > or
> > > the user will be given the finger?
> > >
> > > Any input will be very much appreciated!
> > > Andrey
> > >
> >
  

Semaphore action

[Andrey Kornev]

Hi there,

The Semaphore feature was a great addition to Ignite's synchronization 
primitive toolkit. I'd like to propose an enhancement to make Semaphore API 
even more useful.

My biggest complaint about the current API is its blocking nature. For example, 
the only way to acquire a permit is by using the blocking acquire() method 
(tryAcquire() is not very useful in the cases when you MUST acquire before you 
can proceed). I believe that in the 21st century blocking APIs is anachronism. 
:))) It's a bit embarrassing even. I'm sure Ignite can do better than that! :)

Usually a permit acquisition is followed by an action, followed by a release of 
the permit. I propose a simple enhancement to the Semaphore API that will 
reduce the boilerplate and make it possible to do all of that asynchronously! 
The new method might look like this:

 IgniteFuture acquireAndExecute(Callable action);
 IgniteFuture acquireAndExecute(Callable action, Executor executor);

(The name could be improved, of course...) The first method would immediately 
return a future to be later completed by the action's result. The action will 
be executed on a thread from the Ignite's public thread pool once a permit has 
been acquired. Optionally, the user could specify a different executor using 
the second signature.

Would the community be interested in such enhancement?

Thanks
Andrey

PS. A subject for a separate discussion, but I believe that in the next major 
release Ignite should just drop the clumsy withAsync() API and make all 
blocking calls explicitly async by returning a future. In my opinion it would 
greatly simplify the API and make it more modern. Besides, I'm pretty sure that 
internally Ignite already creates futures for pretty much most of the API 
calls, so it should not cause any performance issue.
  

RE: Semaphore action

[Andrey Kornev]

Just to clarify, 7.4 below refers to GridGain 7.4, which is Ignite 1.4.
_
From: Andrey Kornev <andrewkor...@hotmail.com>
Sent: Sunday, January 17, 2016 10:27 AM
Subject: RE: Semaphore action
To:  <dev@ignite.apache.org>


   Vladisav,   

 It would be great if you could implement the enhancements!

 And while we're at it, here's something else I'd like us to consider. The 
proposed API only gets us half way there: there is no longer a blocking wait 
for the permit, but the action must be blocking. I really think it should be 
possible to do the whole thing asynchronously including the action and release 
of the permit. (In fact, this is what I need for my project. In 7.4, I was able 
to implement this feature using just the public APIs, but I'd really like to 
drop that code and start using the Semaphore API.)   

 For example, upon the permit acquisition, the action could fire off a 
distributed compute and rather than block waiting for the results to come back, 
it would simply return the future provided by Ignite's withAsync(). The 
semaphore implementation would not release the permit immediately upon the 
action return. Instead, it'd do so only when the action future completes.   

 Here's how the API might look like:   

  IgniteFuture acquireAndExecuteAsyncAction(Callable<IgniteFuture> 
action);   

 This API makes it possible to execute a protected action with no blocking 
whatsoever! It's a lot more efficient, reduces the pressure on Ignite's thread 
pools and prevents thread starvation, and so on.   

 Regards   
 Andrey   

 > Date: Sun, 17 Jan 2016 10:40:53 +0100   
 > Subject: Re: Semaphore action   
 > From: vladis...@gmail.com   
 > To: dev@ignite.apache.org   
 >
 > It does sounds like a useful addition,   
 > and I believe it could be implemented easily.   
 > I can do it if the community agrees on changing the API of the Semaphore.   
 >
 > Best regards,   
 > VJ   
 >
 > On Sun, Jan 17, 2016 at 5:53 AM, Andrey Kornev <andrewkor...@hotmail.com>   
 > wrote:   
 >
 > > Dmitriy,I don't believe it makes sense to have the action execute   
 > > remotely. At least I didn't think of it.In my mind, the action is always   
 > > local (that's the reason I suggested the Java's Callable in the API and 
 > > not   
 > > serializable IgniteCallable, in the first place). What the action does is  
 > >  
 > > entirely up to the user. One could fire off a remote compute if that's 
 > > what   
 > > one wants.I hope I make myself clear.ThanksAndrey   
 > > _   
 > > From: Dmitriy Setrakyan <dsetrak...@apache.org>   
 > > Sent: Saturday, January 16, 2016 7:54 AM   
 > > Subject: Re: Semaphore action   
 > > To:  <dev@ignite.apache.org>   
 > >   
 > >   
 > >    Andrey,   
 > >   
 > >  In general this seems like a good addition. However, I do not understand  
 > > 
 > >  how you can specify an executor, given that execution could happen   
 > >  remotely. Can you please clarify?   
 > >   
 > >  D.   
 > >   
 > >  On Fri, Jan 15, 2016 at 1:00 PM, Andrey Kornev <andrewkor...@hotmail.com> 
 > >  
 > >  wrote:   
 > >   
 > >  > Hi there,   
 > >  >   
 > >  > The Semaphore feature was a great addition to Ignite's synchronization  
 > > 
 > >  > primitive toolkit. I'd like to propose an enhancement to make Semaphore 
 > >  
 > > API   
 > >  > even more useful.   
 > >  >   
 > >  > My biggest complaint about the current API is its blocking nature. For  
 > > 
 > >  > example, the only way to acquire a permit is by using the blocking   
 > >  > acquire() method (tryAcquire() is not very useful in the cases when you 
 > >  
 > >  > MUST acquire before you can proceed). I believe that in the 21st 
 > >century   
 > >  > blocking APIs is anachronism. :))) It's a bit embarrassing even. I'm   
 > > sure   
 > >  > Ignite can do better than that! :)   
 > >  >   
 > >  > Usually a permit acquisition is followed by an action, followed by a   
 > >  > release of the permit. I propose a simple enhancement to the Semaphore  
 > > 
 > > API   
 > >  > that will reduce the boilerplate and make it possible to do all of that 
 > >  
 > >  > asynchronously! The new method might look like this:   
 > >  >   
 > >  >  IgniteFuture acquireAndExecute(Callable action);   
 > >  >  IgniteFuture acquireAndExecute(Callable action, Executor   
 > >  > executor);   
 > >  &

RE: Semaphore action

[Andrey Kornev]

Vladisav,

It would be great if you could implement the enhancements! 

And while we're at it, here's something else I'd like us to consider. The 
proposed API only gets us half way there: there is no longer a blocking wait 
for the permit, but the action must be blocking. I really think it should be 
possible to do the whole thing asynchronously including the action and release 
of the permit. (In fact, this is what I need for my project. In 7.4, I was able 
to implement this feature using just the public APIs, but I'd really like to 
drop that code and start using the Semaphore API.)

For example, upon the permit acquisition, the action could fire off a 
distributed compute and rather than block waiting for the results to come back, 
it would simply return the future provided by Ignite's withAsync(). The 
semaphore implementation would not release the permit immediately upon the 
action return. Instead, it'd do so only when the action future completes.

Here's how the API might look like:

 IgniteFuture acquireAndExecuteAsyncAction(Callable<IgniteFuture> 
action);

This API makes it possible to execute a protected action with no blocking 
whatsoever! It's a lot more efficient, reduces the pressure on Ignite's thread 
pools and prevents thread starvation, and so on.

Regards
Andrey

> Date: Sun, 17 Jan 2016 10:40:53 +0100
> Subject: Re: Semaphore action
> From: vladis...@gmail.com
> To: dev@ignite.apache.org
> 
> It does sounds like a useful addition,
> and I believe it could be implemented easily.
> I can do it if the community agrees on changing the API of the Semaphore.
> 
> Best regards,
> VJ
> 
> On Sun, Jan 17, 2016 at 5:53 AM, Andrey Kornev <andrewkor...@hotmail.com>
> wrote:
> 
> > Dmitriy,I don't believe it makes sense to have the action execute
> > remotely. At least I didn't think of it.In my mind, the action is always
> > local (that's the reason I suggested the Java's Callable in the API and not
> > serializable IgniteCallable, in the first place). What the action does is
> > entirely up to the user. One could fire off a remote compute if that's what
> > one wants.I hope I make myself clear.ThanksAndrey
> > _
> > From: Dmitriy Setrakyan <dsetrak...@apache.org>
> > Sent: Saturday, January 16, 2016 7:54 AM
> > Subject: Re: Semaphore action
> > To:  <dev@ignite.apache.org>
> >
> >
> >Andrey,
> >
> >  In general this seems like a good addition. However, I do not understand
> >  how you can specify an executor, given that execution could happen
> >  remotely. Can you please clarify?
> >
> >  D.
> >
> >  On Fri, Jan 15, 2016 at 1:00 PM, Andrey Kornev <andrewkor...@hotmail.com>
> >  wrote:
> >
> >  > Hi there,
> >  >
> >  > The Semaphore feature was a great addition to Ignite's synchronization
> >  > primitive toolkit. I'd like to propose an enhancement to make Semaphore
> > API
> >  > even more useful.
> >  >
> >  > My biggest complaint about the current API is its blocking nature. For
> >  > example, the only way to acquire a permit is by using the blocking
> >  > acquire() method (tryAcquire() is not very useful in the cases when you
> >  > MUST acquire before you can proceed). I believe that in the 21st century
> >  > blocking APIs is anachronism. :))) It's a bit embarrassing even. I'm
> > sure
> >  > Ignite can do better than that! :)
> >  >
> >  > Usually a permit acquisition is followed by an action, followed by a
> >  > release of the permit. I propose a simple enhancement to the Semaphore
> > API
> >  > that will reduce the boilerplate and make it possible to do all of that
> >  > asynchronously! The new method might look like this:
> >  >
> >  >  IgniteFuture acquireAndExecute(Callable action);
> >  >  IgniteFuture acquireAndExecute(Callable action, Executor
> >  > executor);
> >  >
> >  > (The name could be improved, of course...) The first method would
> >  > immediately return a future to be later completed by the action's
> > result.
> >  > The action will be executed on a thread from the Ignite's public thread
> >  > pool once a permit has been acquired. Optionally, the user could
> > specify a
> >  > different executor using the second signature.
> >  >
> >  > Would the community be interested in such enhancement?
> >  >
> >  > Thanks
> >  > Andrey
> >  >
> >  > PS. A subject for a separate discussion, but I believe that in the next
> >  > major release Ignite should just drop the clumsy withAsync() API and
> > make
> >  > all blocking calls explicitly async by returning a future. In my
> > opinion it
> >  > would greatly simplify the API and make it more modern. Besides, I'm
> > pretty
> >  > sure that internally Ignite already creates futures for pretty much
> > most of
> >  > the API calls, so it should not cause any performance issue.
> >  >
> >
> >
> >
> >
> >
  

RE: Semaphore action

[Andrey Kornev]

Yakov,

Thank you for your feedback.

I agree with the first point. My thinking was based on previous unfortunate 
experiences with user callbacks being executed on Ignite system threads while 
some internal locks are held, which made the Ignite API non-reentrant (a 
salient example in 1.4 is the Event API's listeners handling the discovery 
events). The Semaphore API must guarantee the Ignite API's reentrancy and 
clearly document the semantics. In such case, the overload with the Executor is 
redundant.

Wrt the second point, the original signature correctly unwraps the action's 
result type as T. Your suggestion would result in the method's return type 
becoming IgniteFuture<IgniteFuture<...>>  which is undesirable. Besides, the 
original signature makes the asynchronous contract explicit -- the action must 
return a future in order to be asynchronous. 

I only wish IgniteFuture extended JUC's Future! Is there any particular reason 
it doesn't?

Regards
Andrey

> Date: Mon, 18 Jan 2016 14:28:24 +0300
> Subject: Re: Semaphore action
> From: yzhda...@apache.org
> To: dev@ignite.apache.org
> 
> I like the idea. Couple of comments though.
> 
> I would leave only  IgniteFuture acquireAndExecute(Callable
> action); Overload with executor can be omitted because callable
> implementation can use any executor. As far as moving acquire logic to
> another thread pool - I think we'd better go without it same as for any
> other Ignite operations. Andrey, what was the pool intended for?
> 
> This T> IgniteFuture acquireAndExecuteAsyncAction(Callable<IgniteFuture>
> action); can be implemented with   IgniteFuture
> acquireAndExecute(Callable action); where T is IgniteFuture for
> operations initiated and returned by callable. Makes sense?
> 
> 
> --Yakov
> 
> 2016-01-18 0:39 GMT+03:00 Andrey Kornev <andrewkor...@hotmail.com>:
> 
> > Just to clarify, 7.4 below refers to GridGain 7.4, which is Ignite 1.4.
> > _
> > From: Andrey Kornev <andrewkor...@hotmail.com>
> > Sent: Sunday, January 17, 2016 10:27 AM
> > Subject: RE: Semaphore action
> > To:  <dev@ignite.apache.org>
> >
> >
> >Vladisav,
> >
> >  It would be great if you could implement the enhancements!
> >
> >  And while we're at it, here's something else I'd like us to consider. The
> > proposed API only gets us half way there: there is no longer a blocking
> > wait for the permit, but the action must be blocking. I really think it
> > should be possible to do the whole thing asynchronously including the
> > action and release of the permit. (In fact, this is what I need for my
> > project. In 7.4, I was able to implement this feature using just the public
> > APIs, but I'd really like to drop that code and start using the Semaphore
> > API.)
> >
> >  For example, upon the permit acquisition, the action could fire off a
> > distributed compute and rather than block waiting for the results to come
> > back, it would simply return the future provided by Ignite's withAsync().
> > The semaphore implementation would not release the permit immediately upon
> > the action return. Instead, it'd do so only when the action future
> > completes.
> >
> >  Here's how the API might look like:
> >
> >   IgniteFuture
> > acquireAndExecuteAsyncAction(Callable<IgniteFuture> action);
> >
> >  This API makes it possible to execute a protected action with no blocking
> > whatsoever! It's a lot more efficient, reduces the pressure on Ignite's
> > thread pools and prevents thread starvation, and so on.
> >
> >  Regards
> >  Andrey
> >
> >  > Date: Sun, 17 Jan 2016 10:40:53 +0100
> >  > Subject: Re: Semaphore action
> >  > From: vladis...@gmail.com
> >  > To: dev@ignite.apache.org
> >  >
> >  > It does sounds like a useful addition,
> >  > and I believe it could be implemented easily.
> >  > I can do it if the community agrees on changing the API of the
> > Semaphore.
> >  >
> >  > Best regards,
> >  > VJ
> >  >
> >  > On Sun, Jan 17, 2016 at 5:53 AM, Andrey Kornev <
> > andrewkor...@hotmail.com>
> >  > wrote:
> >  >
> >  > > Dmitriy,I don't believe it makes sense to have the action execute
> >  > > remotely. At least I didn't think of it.In my mind, the action is
> > always
> >  > > local (that's the reason I suggested the Java's Callable in the API
> > and not
> >  > > serializable IgniteCallable, in the first place). What the action
> > does is
> >  > > 

LOCAL cache

[Andrey Kornev]

Hello,


It appears that a LOCAL cache gets created on all cluster nodes despite it 
being LOCAL. In other words, a call to Ignite.getOrCreateCache(...) results in 
the cache started on all nodes:


3871 [exchange-worker-#74%Node1%] INFO GridCacheProcessor - Started cache 
[name=myCache, mode=LOCAL]
3871 [exchange-worker-#33%Node0%] INFO GridCacheProcessor - Started cache 
[name=myCache, mode=LOCAL]


Why? I find it counter intuitive, and rather redundant - I don't want the 
creation of a LOCAL cache on a one node to result in the same cache getting 
created on all other cluster nodes. I really think that creation of a LOCAL 
cache should be a node local operation. Each node will create its own LOCAL 
cache instance when it needs it.


Can anyone shed some light on the rationale behind such design?


Thanks

Andrey

Re: Transformers in SCAN queries

[Andrey Kornev]

Another perhaps bigger problem with running queries (including scan queries) 
using closures was discussed at length on the @dev not so long ago. It has to 
do with partitions migration due to cluster topology changes which may result 
in the query returning incomplete result. And while it is possible to solve 
this problem for the scan queries by using some clever tricks, all bets are off 
with the SQL queries.Andrey
_
From: Valentin Kulichenko 
Sent: Thursday, February 4, 2016 6:29 AM
Subject: Re: Transformers in SCAN queries
To:  


   Dmitry,   

 The main difference in my view is that you lose pagination when sending   
 results from servers to client. What if one wants to iterate through all   
 entries in cache?   

 On Wed, Feb 3, 2016 at 9:47 PM, Dmitriy Setrakyan    
 wrote:   

 > Valentin,   
 >   
 > Wouldn’t the same effect be achieved by broadcasting a closure to the   
 > cluster and executing scan-query on every node locally?   
 >   
 > D.   
 >   
 > On Wed, Feb 3, 2016 at 9:17 PM, Valentin Kulichenko <   
 > valentin.kuliche...@gmail.com> wrote:   
 >   
 > > Igniters,   
 > >   
 > > I keep getting requests from our users to add optional transformers to   
 > SCAN   
 > > queries. This will allow to iterate through cache, but do not transfer   
 > > whole key-value pairs across networks (e.g., get only keys). The feature   
 > > looks useful and I created a ticket [1].   
 > >   
 > > I am struggling with the design now. The problem is that I wanted to   
 > extend   
 > > existing ScanQuery object for this, but this seems to be impossible   
 > because   
 > > it already extends Query> and thus can iterate only   
 > > through entries.   
 > >   
 > > The only option I see now is to create a separate query type, copy-paste   
 > > everything from ScanQuery and add *mandatory* transformer. Something like  
 > >  
 > > this:   
 > >   
 > > ScanTransformQuery extends Query {   
 > > IgniteBiPredicate filter;   
 > > IgniteClosure, R> transformer;   
 > > int part;   
 > > ...   
 > > }   
 > >   
 > > Thoughts? Does anyone has other ideas?   
 > >   
 > > [1]https://issues.apache.org/jira/browse/IGNITE-2546   
 > >   
 > > -Val   
 > >   
 >   
   


  

RE: Binary object inside Externalizable

[Andrey Kornev]

Val,

In our case, it's other way around -- use of Externalizable is pretty rare and 
it's mostly Serializables throughout the code base (not our code, but of our 
clients that use our software). Besides there are always those pesky 3d party 
libraries and the JDK itself (for example, the Throwable class) that use 
Serializable.

My point is that no matter how much we all love quick fixes, when it comes to 
serialization, all cases must be properly supported.

Regards
Andrey

> From: valentin.kuliche...@gmail.com
> Date: Tue, 23 Feb 2016 20:50:14 -0800
> Subject: Re: Binary object inside Externalizable
> To: dev@ignite.apache.org
> 
> Actually, adding Externalizable support to binary marshaller should not be
> difficult, no? BinaryWriterExImpl already implements ObjectOutput, so we
> just need to pass to writeExternal (and the same for deserialization). To
> be honest, I don't understand why we use optimized marshaller here and
> making this change should fix majority of use cases.
> 
> The issue will remain for writeObject/readObject which is much more
> complicated and where using optimized marshaller makes more sense. But this
> is a relatively rare case and I think we can fix this as a next step.
> 
> Thoughts?
> 
> On Sat, Feb 20, 2016 at 10:01 AM, Dmitriy Setrakyan <dsetrak...@apache.org>
> wrote:
> 
> > Andrey, you are absolutely right. I was suggesting a quick fix, which in my
> > view, would fix most of the issues. In the long term, we should fix the
> > binary serialization to work the way you are describing.
> >
> > D.
> >
> > On Sat, Feb 20, 2016 at 9:26 AM, Andrey Kornev <andrewkor...@hotmail.com>
> > wrote:
> >
> > > Val,
> > >
> > > I think Ignite Serialization should follow the approach similar to what
> > > Java serialization does: it can transparently and consistently handle
> > both
> > > Serializable and Externalizable classes. It can do it because it relies
> > on
> > > a single class - ObjectOutputStream - to do serialization (and a single
> > > class - ObjectInputStream - to do the opposite, but let's ignore it for
> > the
> > > moment). ObjectOutputStream delegates to proper user callbacks
> > (writeObject
> > > or writeExternal) at the right moments, but otherwise it fully controls
> > the
> > > on-the-wire representation and takes care of all the necessary stream
> > > framing/marking so that ObjectInputStream can then correctly deserialize
> > > the bytes. It's never a problem for Java to serialize an Externalizable
> > > field from a Serializable object and vice versa. Users can mix and match
> > > serialization APIs as they please.
> > >
> > > I think Ignite should just have a single marshaller, let's say the
> > Binary,
> > > which drives the whole serialization process. It doesn't delegate to
> > > OptimizedMarshaller as it does today. Instead it detects the
> > > Serializable/Externalizable classes and calls their serialization
> > callbacks
> > > - writeObject/writeExternal - passing in *itself* as
> > > ObjectOutputStream/ObjectOutput correspondingly. This way the entire
> > > serialization process never leaves the context of a single Binary
> > > marshaller and allows Ignite to handle such complex cases as the one
> > we're
> > > discussing now.
> > >
> > > I hope it makes sense.
> > >
> > > Andrey
> > >
> > > > From: valentin.kuliche...@gmail.com
> > > > Date: Fri, 19 Feb 2016 19:15:07 -0800
> > > > Subject: Binary object inside Externalizable
> > > > To: dev@ignite.apache.org
> > > >
> > > > Folks,
> > > >
> > > > I recently faced an issue which seems to be pretty critical. As you
> > know,
> > > > when binary marshaller meets an Externalizable object, it switches to
> > > > optimized marshaller. And if there is a regular object inside, it's
> > still
> > > > serialized with optimized, even if its type is declared in binary
> > > > configuration with custom mapper, etc. This looks wrong.
> > > >
> > > > It's getting even worse in compute grid, because closure processor
> > wraps
> > > > user's closures in some internal classes, which are Externalizable, so
> > > > these closures are always serialized with optimized marshaller.
> > > >
> > > > Does anyone have ideas on what is the proper fix for this?
> > > >
> > > > -Val
> > >
> > >
> >
  

CacheEntryEventFilter with Replicated caches

[Andrey Kornev]

Hello,

It's come to my attention, when registering the cache event listener, the 
filters get deployed on all the nodes of the cache, despite the fact that the 
cache is configured as REPLICATED.  This seems redundant since it's sufficient 
to have the filter deployed only on the node that has the local cache listener 
(in other words, the same node that registers the listener). Since the filter 
may execute some non-trivial computationally intensive logic and it doesn't 
make sense to waste CPU on the nodes that are not going to call back the 
listener. Not deploying filters to remote nodes would also reduce the 
registration/unregistration overhead since only the local node needs to be 
involved.

The only case that would require special attention would be the case when a 
listener is registered from a node which doesn't have the cache started.

Please advise.
Andrey
  

RE: CacheEntryEventFilter with Replicated caches

[Andrey Kornev]

Alexey,

I'm talking about JCache's CacheEntry listener (which I think is implemented on 
top of the continuous query feature).

Andrey

> Date: Wed, 9 Mar 2016 15:52:48 -0800
> Subject: Re: CacheEntryEventFilter with Replicated caches
> From: alexey.goncha...@gmail.com
> To: dev@ignite.apache.org
> 
> Andrey,
> 
> Are you talking about a continuous query or a distributed event listener?
  

RE: CacheEntryEventFilter with Replicated caches

[Andrey Kornev]

Dmitriy,

The reason for my posting was exactly that: the filter is both deployed and 
invoked on all nodes where the REPLICATED cache is started. My point is that 
the filter should only be deployed and invoked on the node where its 
corresponding listener is, namely the local node (the node that registered the 
listener by calling IgniteCache.registerCacheEntryListener). Executing filter 
on remote nodes in case of a REPLICATED cache is a waste of resources that can 
and should be avoided.

Thanks
Andrey

> From: dsetrak...@apache.org
> Date: Wed, 9 Mar 2016 15:09:20 -0800
> Subject: Re: CacheEntryEventFilter with Replicated caches
> To: dev@ignite.apache.org
> 
> Hi Andrey.
> 
> The replicated cache is just a partitioned cache with more backups. I think
> the filter is deployed on all nodes, but is only invoked on the primary
> node (correct me if I am wrong). In that case, it will be impossible to
> deploy it only on the node that registered it.
> 
> D.
> 
> On Wed, Mar 9, 2016 at 1:44 PM, Andrey Kornev <andrewkor...@hotmail.com>
> wrote:
> 
> > Hello,
> >
> > It's come to my attention, when registering the cache event listener, the
> > filters get deployed on all the nodes of the cache, despite the fact that
> > the cache is configured as REPLICATED.  This seems redundant since it's
> > sufficient to have the filter deployed only on the node that has the local
> > cache listener (in other words, the same node that registers the listener).
> > Since the filter may execute some non-trivial computationally intensive
> > logic and it doesn't make sense to waste CPU on the nodes that are not
> > going to call back the listener. Not deploying filters to remote nodes
> > would also reduce the registration/unregistration overhead since only the
> > local node needs to be involved.
> >
> > The only case that would require special attention would be the case when
> > a listener is registered from a node which doesn't have the cache started.
> >
> > Please advise.
> > Andrey
> >
  

RE: CacheEntryEventFilter with Replicated caches

[Andrey Kornev]

Alexey,

Thank you for the explanation!

But why any of that matters in this case? I'm talking about the REPLICATED 
cache -- all data changes get applied to all nodes in the same order on all 
nodes. And I'm talking about a cache event listener that is instantiated on one 
of thise nodes. It can be a primary or back up node, why does it matter? If the 
node is gone, that the listener is gone. Nobody's listening anymore. Time to 
forget the listener, clean up and move on. There is no way on a different node 
to pick from where the crushed node has left off (at least not thru JCache 
API). And  it's doubtful one would want to do that even if it was possible.

In the use case I'm describing, the whole filtering/listening thing could be 
entirely local making things a lot more efficient. I don't care about 
exactly-once delivery semantics that Ignite seems to be trying to guarantee, 
and I really hate to pay for things I don't need (the overhead of deploying 
filters to remote nodes, gratuitous evaluation of remote filters, the internal 
data structures and logic employed by Ignite to maintain the backup queues, 
etc, etc). I just want a lightweight all-local cache event listener.

How can I do it? Please advise.

Thanks
Andrey

> Date: Wed, 9 Mar 2016 16:21:19 -0800
> Subject: Re: CacheEntryEventFilter with Replicated caches
> From: alexey.goncha...@gmail.com
> To: dev@ignite.apache.org
> 
> Dmitriy is right, currently REPLICATED cache works the same way as
> PARTITIONED does, and in PARTITIONED cache filters should be evaluated on
> backups in order to maintain a backup queue in case a primary node fails.
> 
> For the case when query is executed on an affinity node of a REPLICATED
> cache _and_ auto-unsubscribe is true, I believe we can change the behavior,
> however it will be inconsistent with all other modes.
> 
> It can be easily overridden in Ignite API by setting local flag on a
> continuous query. I think we can provide a way to set the local flag for a
> JCache event listener, but I am not sure how it will look API-wise.
  

RE: 1.5.0.final is breaking packaging: osgi dependency is non-existent

[Andrey Kornev]

I'd love to elaborate, Konstantin, but I can't. I have no experience with the 
package management tools, neither yum nor rpm.

BTW, those Spring dependencies are optional, as far as I know. They are only 
required if Spring framework is used by the application (Spring XML 
configuration, for example).

Andrey

> Date: Sun, 15 May 2016 10:20:57 -0400
> From: c...@apache.org
> To: dev@ignite.apache.org
> Subject: Re: 1.5.0.final is breaking packaging: osgi dependency is 
> non-existent
> 
> Thanks for checking out Andrey. Do you mind to elaborate on 'something'? The
> way I see it, is that rpmbuilder performs certain checks to declare the
> package dependencies, and the osgi once are popping up from somewhere. Which
> wasn't the case before OSGI was added. Which lead me to believe that the issue
> might be rooting from missing jars or something similar. Hence, my this email
> thread.
> 
> Were my assumptions incorrect.
> Thanks
>   Cos
> 
> On Thu, May 05, 2016 at 04:39AM, Andrey Kornev wrote:
> > Anton,
> > 
> > My first impression after having taken a look at bigtop's JIRA ticket is
> > that the issue is not so much with OSGi per se, but rather it has something
> > to do with Ignite's HDFS accelerator yum packaging.
> > 
> > Regards
> > Andrey
> > 
> > > Date: Thu, 5 May 2016 13:55:26 +0300
> > > Subject: Re: 1.5.0.final is breaking packaging: osgi dependency is 
> > > non-existent
> > > From: avinogra...@gridgain.com
> > > To: dev@ignite.apache.org
> > > 
> > > Igniters,
> > > 
> > > Issue is critical for Bigtop project.
> > > In case someone familiar with OSGI please have a look.
> > > 
> > > On Wed, May 4, 2016 at 11:15 AM, Anton Vinogradov 
> > > <avinogra...@gridgain.com>
> > > wrote:
> > > 
> > > > Roman, Raul,
> > > >
> > > > As far as I remember you're familiar with OSGI. Could you please have a
> > > > look at this?
> > > >
> > > > On Wed, May 4, 2016 at 6:09 AM, Konstantin Boudnik <c...@boudnik.org>
> > > > wrote:
> > > >
> > > >> Hey guys
> > > >>
> > > >> For your information: adding osgi into the product is breaking the
> > > >> dependency matrix of the bigtop packages. For more info see
> > > >> https://issues.apache.org/jira/browse/BIGTOP-2421
> > > >>
> > > >> --
> > > >> Take care,
> > > >> Cos
> > > >> 2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622
> > > >> Cos' pubkey: http://people.apache.org/~cos/cos.asc
> > > >>
> > > >>   Wisdom of the hour 
> > > >>
> > > >>
> > > >
> >   
  

RE: 1.5.0.final is breaking packaging: osgi dependency is non-existent

[Andrey Kornev]

Anton,

My first impression after having taken a look at bigtop's JIRA ticket is that 
the issue is not so much with OSGi per se, but rather it has something to do 
with Ignite's HDFS accelerator yum packaging.

Regards
Andrey

> Date: Thu, 5 May 2016 13:55:26 +0300
> Subject: Re: 1.5.0.final is breaking packaging: osgi dependency is 
> non-existent
> From: avinogra...@gridgain.com
> To: dev@ignite.apache.org
> 
> Igniters,
> 
> Issue is critical for Bigtop project.
> In case someone familiar with OSGI please have a look.
> 
> On Wed, May 4, 2016 at 11:15 AM, Anton Vinogradov 
> wrote:
> 
> > Roman, Raul,
> >
> > As far as I remember you're familiar with OSGI. Could you please have a
> > look at this?
> >
> > On Wed, May 4, 2016 at 6:09 AM, Konstantin Boudnik 
> > wrote:
> >
> >> Hey guys
> >>
> >> For your information: adding osgi into the product is breaking the
> >> dependency matrix of the bigtop packages. For more info see
> >> https://issues.apache.org/jira/browse/BIGTOP-2421
> >>
> >> --
> >> Take care,
> >> Cos
> >> 2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622
> >> Cos' pubkey: http://people.apache.org/~cos/cos.asc
> >>
> >>   Wisdom of the hour 
> >>
> >>
> >
  

Re: Data compression in Ignite 2.0

[Andrey Kornev]

I'm guessing the suggestion here is to use the compressed form directly for 
WHERE clause evaluation. If that's the case I think there are a couple of 
issues:

1) the LIKE predicate.

2) predicates other than equality (for example, <, >, etc.)


But since Ignite isn't just about SQL queries (surprisingly some people still 
use it just as distributed cache!), in general I think compression is a great 
data. The cleanest way to achieve that would be to just make it possible to 
chain the marshallers. It is possible to do it already without any Ignite code 
changes, but unfortunately it would force people to use the non-public 
BinaryMarshaller class directly (as the first element of the chain).


Cheers

Andrey


From: Dmitriy Setrakyan 
Sent: Monday, July 25, 2016 1:53 PM
To: dev@ignite.apache.org
Subject: Re: Data compression in Ignite 2.0

Nikita, this sounds like a pretty elegant approach.

Does anyone in the community see a problem with this design?

On Mon, Jul 25, 2016 at 4:49 PM, Nikita Ivanov  wrote:

> SAP Hana does the compression by 1) compressing SQL parameters before
> execution, and 2) storing only compressed data in memory. This way all SQL
> queries work as normal with zero modifications or performance overhead.
> Only results of the query can be (optionally) decompressed back before
> returning to the user.
>
> --
> Nikita Ivanov
>
>
> On Mon, Jul 25, 2016 at 1:40 PM, Sergey Kozlov 
> wrote:
>
> > Hi
> >
> > For approach 1: Put a large object into a partiton cache will force to
> > update the dictionary placed on replication cache. It seeis it may be
> > time-expense operation.
> > Appoach 2-3 are make sense for rare cases as Sergi commented.
> > Aslo I see a danger of OOM if we've got high compression level and try to
> > restore orginal value in memory.
> >
> > On Mon, Jul 25, 2016 at 10:39 AM, Alexey Kuznetsov <
> > akuznet...@gridgain.com>
> > wrote:
> >
> > > Sergi,
> > >
> > > Of course it will introduce some slowdown, but with compression more
> data
> > > could be stored in memory
> > > and not will be evicted to disk. In case of compress by dictionary
> > > substitution it will be only one more lookup
> > > and should be fast.
> > >
> > > In general we could provide only API for compression out of the box,
> and
> > > users that really need some sort of compression
> > > will implement it by them self. This will not require much effort I
> > think.
> > >
> > >
> > >
> > > On Mon, Jul 25, 2016 at 2:18 PM, Sergi Vladykin <
> > sergi.vlady...@gmail.com>
> > > wrote:
> > >
> > > > This will make sense only for rare cases when you have very large
> > objects
> > > > stored, which can be effectively compressed. And even then it will
> > > > introduce slowdown on all the operations, which often will not be
> > > > acceptable. I guess only few users will find this feature useful,
> thus
> > I
> > > > think it does not worth the effort.
> > > >
> > > > Sergi
> > > >
> > > >
> > > >
> > > > 2016-07-25 9:28 GMT+03:00 Alexey Kuznetsov  >:
> > > >
> > > > > Hi, All!
> > > > >
> > > > > I would like to propose one more feature for Ignite 2.0.
> > > > >
> > > > > Data compression for data in binary format.
> > > > >
> > > > > Binary format is stored as field name + field data.
> > > > > So we have a description.
> > > > > How about to add one more byte to binary data descriptor:
> > > > >
> > > > > *Compressed*:
> > > > >  0 - Data stored as is (no compression).
> > > > >  1 - Data compressed by dictionary (something like DB2 row
> > compression
> > > > [1],
> > > > >  but for all binary types). We could have system or user defined
> > > > replicated
> > > > > cache for such dictionary and *cache.compact()* method that will
> scan
> > > > > cache, build dictionary and compact data.
> > > > >  2 - Data compressed by Java built in ZIP.
> > > > >  3 - Data compressed by some user custom algorithm.
> > > > >
> > > > > Of course it is possible to compress data in current Ignite 1.x but
> > in
> > > > this
> > > > > case compressed data cannot be accessed from SQL engine, if we
> > > implement
> > > > > support for compression on Ignite core level SQL engine will be
> able
> > to
> > > > > detect that data is compressed and properly handle such data.
> > > > >
> > > > > What do you think?
> > > > > If community consider this feature useful I will create issue in
> > JIRA.
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> http://www.ibm.com/developerworks/data/library/techarticle/dm-1205db210compression/
Optimize storage with deep compression in DB2 
10
www.ibm.com
Thomas Fanghaenel has been with IBM for nine years and is a Senior Software 
Engineer with the DB2 Data Warehouse Storage and Indexing ...


> > > > >
> > > > > --
> > > > > Alexey Kuznetsov
> > > > >
> > > >
> > >
> > >

Re: Data compression in Ignite 2.0

[Andrey Kornev]

Dictionary compression requires some knowledge about data being compressed. For 
example, for numeric types a range of values must be known so that the 
dictionary can be generated. For strings, the number of unique values of the 
column is the key piece of input into the dictionary generation.
SAP HANA is a column-based database system: it stores the fields of the data 
tuple individually using the best compression for the given data type and the 
particular set of values. HANA has been specifically built as a general purpose 
database, rather than as an afterthought layer on top of an already existing 
distributed cache.
On the other hand, Ignite is a distributed cache implementation (a pretty good 
one!) that in general requires no schema and stores its data in the row-based 
fashion. Its current design doesn't land itself readily to the kind of 
optimizations HANA provides out of the box.
For the curios types among us, the implementation details of HANA are well 
documented in "In-memory Data Management", by Hasso Plattner & Alexander Zeier.
Cheers
Andrey
_
From: Alexey Kuznetsov <akuznet...@gridgain.com<mailto:akuznet...@gridgain.com>>
Sent: Tuesday, July 26, 2016 5:36 AM
Subject: Re: Data compression in Ignite 2.0
To: <dev@ignite.apache.org<mailto:dev@ignite.apache.org>>


Sergey Kozlov wrote:
>> For approach 1: Put a large object into a partition cache will
force to update
the dictionary placed on replication cache. It may be time-expense
operation.
The dictionary will be built only once. And we could control what should be
put into dictionary, for example, we could check min and max size and
decide - put value to dictionary or not.

>> Approach 2-3 are make sense for rare cases as Sergi commented.
But it is better at least have a possibility to plug user code for
compression than not to have it at all.

>> Also I see a danger of OOM if we've got high compression level and try
to restore original value in memory.
We could easily get OOM with many other operations right now without
compression, I think it is not an issue, we could add a NOTE to
documentation about such possibility.

Andrey Kornev wrote:
>> ... in general I think compression is a great data. The cleanest way to
achieve that would be to just make it possible to chain the marshallers...
I think it is also good idea. And looks like it could be used for
compression with some sort of ZIP algorithm, but how to deal with
compression by dictionary substitution?
We need to build dictionary first. Any ideas?

Nikita Ivanov wrote:
>> SAP Hana does the compression by 1) compressing SQL parameters before
execution...
Looks interesting, but my initial point was about compression of cache
data, not SQL queries.
My idea was to make compression transparent for SQL engine when it will
lookup for data.

But idea of compressing SQL queries result looks very interesting, because
it is known fact, that SQL engine could consume quite a lot of heap for
storing result sets.
I think this should be discussed in separate thread.

Just for you information, in first message I mentioned that DB2 has
compression by dictionary and according to them it is possible to
compress usual data to 50-80%.
I have some experience with DB2 and can confirm this.

--
Alexey Kuznetsov

Re: Rework "withAsync" in Apache 2.0

[Andrey Kornev]

+1 "Async" suffix.



From: Vladimir Ozerov 
Sent: Thursday, July 21, 2016 2:41 AM
To: dev@ignite.apache.org
Subject: Re: Rework "withAsync" in Apache 2.0

Moreover, have a look at *CompletableFuture *interface:

handle
handle*Async*

thenApply
thenApply*Async*

And so on. One more argument to return methods with "Async" suffix from
good old GridGain days.

On Thu, Jul 21, 2016 at 12:38 PM, Vladimir Ozerov 
wrote:

> It is a matter of taste. In .NET we have "Async" methods near synchronous
> methods because it is common practice in .NET world:
> V Get(K key);
> Task GetAsync(K key);
>
> For Java we can go the same way, or introduce separate interface. It will
> not be identical to synchronous, because it will have different return
> types and probably will contain only those methods which support async
> execution.
>
> Personally I like .NET approach. It is simple and straightforward. User do
> not have to bother about whether current instance is sync or async (like
> now), and will not have to perform additional steps like
> *IgniteCache.withAsync().get()*. Do you want to call GET asynchronously?
> No problem, *IgniteCache.getAsync()*. Simple, expressive, straightforward.
>
> The drawback is that our interfaces will become "heavier". But it is 2016
> year, we all have IDEs. I do not see any problems if we will have 62 + 36 =
> 98 methods instead of current 62 on cache API.
>
> Vladimir.
>
>
> On Thu, Jul 21, 2016 at 12:21 PM, Dmitriy Setrakyan  > wrote:
>
>> Do I understand correctly that the community is proposing to have 2
>> identical interfaces, one for sync operations and another for async
>> operations?
>>
>> On Thu, Jul 21, 2016 at 12:15 PM, Sergi Vladykin <
>> sergi.vlady...@gmail.com>
>> wrote:
>>
>> > +1
>> >
>> > Finally it is time to drop this "cool feature" from Ignite!
>> >
>> > Sergi
>> >
>> > On Thu, Jul 21, 2016 at 11:13 AM, Vladimir Ozerov > >
>> > wrote:
>> >
>> > > Alex.
>> > >
>> > > Of course, some distributed operations will involve some kind of
>> > asynchrony
>> > > even in synchronous mode. My point is that we should not blindly do
>> > things
>> > > like that:
>> > >
>> > > V get(K key) {
>> > > return getAsync(key),get();
>> > > }
>> > >
>> > > Instead, get() has it's own path, getAsync() another path. But of
>> course
>> > > they could share some common places. E.g. I remember we already fixed
>> > some
>> > > cache operations in this regard when users hit async semaphore limit
>> when
>> > > calling synchronous gets.
>> > >
>> > > Another point is that async instances may possibly accept
>> user-provided
>> > > Executor.
>> > >
>> > > Vladimir,
>> > >
>> > > On Thu, Jul 21, 2016 at 11:04 AM, Semyon Boikov > >
>> > > wrote:
>> > >
>> > > > Another issue which usually confuses users is Ignite 'implementation
>> > > > details' of asynchronous execution: it operation is local it can be
>> > > > executed from calling thread (for example, if 'async put' is
>> executed
>> > in
>> > > > atomic cache from primary node then cache store will be updated from
>> > > > calling thread). Does it make sense to fix this as well?
>> > > >
>> > > >
>> > > > On Thu, Jul 21, 2016 at 10:55 AM, Yakov Zhdanov <
>> yzhda...@apache.org>
>> > > > wrote:
>> > > >
>> > > > > Agree with Alex. Vova, please go on with issues taking Alex's
>> > comments
>> > > > into
>> > > > > consideration.
>> > > > >
>> > > > > Thanks!
>> > > > >
>> > > > > --Yakov
>> > > > >
>> > > > > 2016-07-21 10:43 GMT+03:00 Alexey Goncharuk <
>> > > alexey.goncha...@gmail.com
>> > > > >:
>> > > > >
>> > > > > > Big +1 on this in general.
>> > > > > >
>> > > > > > I would also relax our guarantees on operations submitted from
>> the
>> > > same
>> > > > > > thread. Currently we guarantee that sequential invocations of
>> async
>> > > > > > operations happen in the same order. I think that if a user
>> wants
>> > > such
>> > > > > > guarantees, he must define these dependencies explicitly by
>> calling
>> > > > > chain()
>> > > > > > on returning futures.
>> > > > > >
>> > > > > > This change will significantly improve cache operations
>> performance
>> > > in
>> > > > > > async mode.
>> > > > > >
>> > > > > > 3) Sync operations normally* should not* be implemented through
>> > > async.
>> > > > > This
>> > > > > > > is a long story - if we delegate to async, then we have to
>> bother
>> > > > with
>> > > > > > > additional threads, associated back-pressure control and all
>> that
>> > > > crap.
>> > > > > > > Sync call must be sync unless there is a very strong reason
>> to go
>> > > > > through
>> > > > > > > async path.
>> > > > > > >
>> > > > > > Not sure about this, though. In most cases a cache operation
>> > implies
>> > > > > > request/response over the network, so I think we should have
>> > explicit
>> > > > > > synchronous counterparts only for methods that are guaranteed
>> 

Re: Rework "withAsync" in Apache 2.0

[Andrey Kornev]

Totally agree with this particular suggestion. On more than just a few 
occasions I've been greatly confused by  Ignite's "asynchronous" operations 
that in reality are blocking under some circumstances and non-blocking under 
others... Go figure! The reason I was given for such design was performance, if 
I remember correctly. I hope the Ignite experts on this mailing list will be 
able to provide more specifics if the community is interested.


In general an asynchronous operation is the one that returns a future 
immediately without any blocking of the calling thread. The outcome of the 
invocation is then communicated back exclusively via the future.


With the growing popularity of reactive programming style (a la Rx Java, etc.), 
I think it is crucial to get this fixed.


Regards

Andrey


From: Semyon Boikov 
Sent: Thursday, July 21, 2016 1:04 AM
To: dev@ignite.apache.org
Subject: Re: Rework "withAsync" in Apache 2.0

Another issue which usually confuses users is Ignite 'implementation
details' of asynchronous execution: it operation is local it can be
executed from calling thread (for example, if 'async put' is executed in
atomic cache from primary node then cache store will be updated from
calling thread). Does it make sense to fix this as well?


On Thu, Jul 21, 2016 at 10:55 AM, Yakov Zhdanov  wrote:

> Agree with Alex. Vova, please go on with issues taking Alex's comments into
> consideration.
>
> Thanks!
>
> --Yakov
>
> 2016-07-21 10:43 GMT+03:00 Alexey Goncharuk :
>
> > Big +1 on this in general.
> >
> > I would also relax our guarantees on operations submitted from the same
> > thread. Currently we guarantee that sequential invocations of async
> > operations happen in the same order. I think that if a user wants such
> > guarantees, he must define these dependencies explicitly by calling
> chain()
> > on returning futures.
> >
> > This change will significantly improve cache operations performance in
> > async mode.
> >
> > 3) Sync operations normally* should not* be implemented through async.
> This
> > > is a long story - if we delegate to async, then we have to bother with
> > > additional threads, associated back-pressure control and all that crap.
> > > Sync call must be sync unless there is a very strong reason to go
> through
> > > async path.
> > >
> > Not sure about this, though. In most cases a cache operation implies
> > request/response over the network, so I think we should have explicit
> > synchronous counterparts only for methods that are guaranteed to be
> local.
> >
>

Re: Global off-heap memory limit

[Andrey Kornev]

+1
I'm particularly interested in the use case where I have caches that are 
created dynamically and in general I don't know in advance how many of them 
will exist at any point in time. But I do know the total amount of memory I'm 
willing to make available to the caches.
So it would be nice if the newly introduced Memory Policies would allow this 
type of configuration  possible.
Regards
Andrey
_
From: Dmitriy Setrakyan >
Sent: Thursday, September 15, 2016 12:44 PM
Subject: Re: Global off-heap memory limit
To: >


To my knowledge, in Ignite 2.0 we will have Memory Policies, and multiple
caches will be able to reuse the same policy. Since the memory size will be
defined at the policy level, it will essentially solve this problem.

Am I wrong?

D.

On Thu, Sep 15, 2016 at 12:15 PM, Igor Rudyak 
> wrote:

> +1
>
> On Thu, Sep 15, 2016 at 12:11 PM, Denis Magda 
> > wrote:
>
> > +1
> >
> > I've faced with such injuries as well and this feature sounds reasonable
> > for me.
> >
> > However, is this going to be supported in the new memory architecture the
> > community has been working on? Alex G., can you comment?
> >
> > -
> > Denis
> >
> > > On Sep 15, 2016, at 12:07 PM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> > >
> > > Folks,
> > >
> > > I'm seeing more and more requests from users to add an option to
> specify
> > > the off-heap memory limit globally, for all the caches in total.
> > Currently
> > > we allow this only on per-cache level, which can be not very usable
> > > sometimes.
> > >
> > > E.g., imagine that you have 100GB of RAM and two caches with 50GB limit
> > for
> > > each of them. At some point one of the caches can consume all 50GB,
> while
> > > the second is almost empty. You can't add anything to the first cache
> > > regardless of the fact that you still have lots of memory available.
> > >
> > > Can we add such parameter?
> > >
> > > -Val
> >
> >
>

Re: Create separate thread pool for query execution

[Andrey Kornev]

Vladimir,


Thanks for your reply!


Unfortunately, most of the time users end up having to think about configuring 
the thread pools, because the "sensible" default value for most of the thread 
pools is the number of processors multiplied by 2. With modern production-grade 
multicore machines, the default thread pools size is simply too high.


Another problem with the current approach to thread management is that some 
resources inevitably go wasted. For example, if one thread pool runs out of 
threads it won't be able to borrow from another thread pool even though that 
one may have plenty of threads sitting idle.


And even if there were really good reasons for maintaining all those thread 
pools, complete lack of any documentation makes it pretty difficult (if not 
impossible) to configure the thread pools correctly.


Finally, your explanation as to why another thread pool - dedicated exclusively 
to queries - is necessary, doesn't explain anything, but rather raises more 
questions:

- why would a query block cache operations? Even if it absolutely must block, 
so what? In this scenario, how is a query different from another "regular" 
cache operation that may also block some other cache operations? Why special 
treatment?

- why wouldn't it possible to run queries from the Ignite closures and tasks? I 
don't see any connection.


Thanks

Andrey


From: Vladimir Ozerov <voze...@gridgain.com>
Sent: Tuesday, October 25, 2016 11:40 PM
To: dev@ignite.apache.org
Subject: Re: Create separate thread pool for query execution

Andrey,

Ignite already works the way you described except of listeners/callbacks.
Only size of each thread pool is exposed to configuration. And each one
already have sensible default value, so normally users do not think of it
at all. For some pools we also have timeouts, but they will be deprecated
soon.

Queries cannot be executed in system pool because they can block cache
operations. And they cannot be executed in public pool because in this case
it will be impossible to run queries from closures and tasks.

Vladimir.

On Wed, Oct 26, 2016 at 8:37 AM, Andrey Kornev <andrewkor...@hotmail.com>
wrote:

> Guys,
> I feel very uneasy about proliferation of thread pools in Ignite. I
> suggest we take step back.
> Most of the users (including yours truly) have no idea how to correctly
> configure the existing thread pools, what those thread pools are for (like,
> management thread pool, or marshaller cache thread pool, or async callback
> thread pool, or peer class loading thread pool, or - my personal favorite -
> utility cache thread pool, just to name a few), and why they should worry
> about them altogether.
> In reality, there should probably be only 2 parameters exposed at the
> configuration level: the size of Ignite's internal (or "system") thread
> pool, and the size of the user thread pool. Or alternatively, one number
> could indicate the total number of thread available to Ignite (hard upper
> bound) and the other would be a ratio of system threads to user threads.
> Either way, it's Ignite's internal business how to manage the thread pools
> given the constraints. For example, Ignite may choose to allocate the
> system threads across twenty thread pools, or maybe just one -- the users
> do not care. What users do care about is the size of the user thread pool
> because it's the one that executes their code. I know it's not the case in
> the current version, but going forward, Ignite must ensure that all user
> code (including all sorts of listeners and callbacks) is executed on the
> user threads.
> In any case, it's the user who is ultimately in charge of figuring out the
> correct thread pool sizes, and I believe having just 2 knobs to turn vs. 7
> or 8 as is the case today, would make things a lot simpler and reduce the
> guess work.
> Speaking of query execution, it feels natural to have it executed on the
> user thread pool, as the query is a user-triggered action executing a
> user's code (even though the code is not Java in this case, but something
> that looks rather like SQL).
> Regards,
> Andrey
> _
> From: Dmitriy Setrakyan <dsetrak...@apache.org dsetrak...@apache.org>>
> Sent: Monday, October 24, 2016 9:23 AM
> Subject: Re: Create separate thread pool for query execution
> To: <dev@ignite.apache.org<mailto:dev@ignite.apache.org>>
>
>
> I think this makes sense. Do we have a ticket filed?
>
> On Mon, Oct 24, 2016 at 2:17 AM, Yakov Zhdanov <yzhda...@apache.org
> <mailto:yzhda...@apache.org>> wrote:
>
> > > As far as the query thread pool, how many threads should be in it by
> > > default? What happens is the query is run from compute task - will the
> > > execution be shifte

Re: Create separate thread pool for query execution

[Andrey Kornev]

Guys,
I feel very uneasy about proliferation of thread pools in Ignite. I suggest we 
take step back.
Most of the users (including yours truly) have no idea how to correctly 
configure the existing thread pools, what those thread pools are for (like, 
management thread pool, or marshaller cache thread pool, or async callback 
thread pool, or peer class loading thread pool, or - my personal favorite - 
utility cache thread pool, just to name a few), and why they should worry about 
them altogether.
In reality, there should probably be only 2 parameters exposed at the 
configuration level: the size of Ignite's internal (or "system") thread pool, 
and the size of the user thread pool. Or alternatively, one number could 
indicate the total number of thread available to Ignite (hard upper bound) and 
the other would be a ratio of system threads to user threads. Either way, it's 
Ignite's internal business how to manage the thread pools given the 
constraints. For example, Ignite may choose to allocate the system threads 
across twenty thread pools, or maybe just one -- the users do not care. What 
users do care about is the size of the user thread pool because it's the one 
that executes their code. I know it's not the case in the current version, but 
going forward, Ignite must ensure that all user code (including all sorts of 
listeners and callbacks) is executed on the user threads.
In any case, it's the user who is ultimately in charge of figuring out the 
correct thread pool sizes, and I believe having just 2 knobs to turn vs. 7 or 8 
as is the case today, would make things a lot simpler and reduce the guess work.
Speaking of query execution, it feels natural to have it executed on the user 
thread pool, as the query is a user-triggered action executing a user's code 
(even though the code is not Java in this case, but something that looks rather 
like SQL).
Regards,
Andrey
_
From: Dmitriy Setrakyan >
Sent: Monday, October 24, 2016 9:23 AM
Subject: Re: Create separate thread pool for query execution
To: >


I think this makes sense. Do we have a ticket filed?

On Mon, Oct 24, 2016 at 2:17 AM, Yakov Zhdanov 
> wrote:

> > As far as the query thread pool, how many threads should be in it by
> > default? What happens is the query is run from compute task - will the
> > execution be shifted from the public to the query thread pool?
>
> Pool management should be the same as for other pools.
>
> Remote executions should be executed in query pool. Local should run
> synchronously.
>
> --Yakov
>
> 2016-10-24 11:53 GMT+03:00 Dmitriy Setrakyan 
> >:
>
> > Sergey, I think we already have a general approach with 3 or 4 thread
> pools
> > we have today.
> >
> > As far as the query thread pool, how many threads should be in it by
> > default? What happens is the query is run from compute task - will the
> > execution be shifted from the public to the query thread pool?
> >
> > D.
> >
> > On Mon, Oct 24, 2016 at 1:47 AM, Sergey Kozlov 
> > >
> > wrote:
> >
> > > Hi
> > >
> > > It seems we need a set of dedicated pools for various purposes. Could
> we
> > > design a general apporach for this like following:
> > > - define dedicated pools in the grid configuration
> > > - run a query/compute/etc in the particular pool
> > >
> > > On Mon, Oct 24, 2016 at 9:49 AM, Vladimir Ozerov 
> > > 
> >
> > > wrote:
> > >
> > > > Romain,
> > > > We do not pre-start threads in our pools on Ignite start. Moreover,
> > idle
> > > > threads are stopped after some timeout.
> > > >
> > > > 24 окт. 2016 г. 8:57 пользователь "Gilles, Romain" <
> > > > romain.gil...@misys.com>
> > > > написал:
> > > >
> > > > Hi igniters,
> > > > I'm not against to create a thread pool for each features but I have
> > some
> > > > difficulty to minimize the number of threads required to start
> > ignite...
> > > If
> > > > you add too many thread pools does this will not increase the number
> of
> > > > threads at startup?
> > > >
> > > > Thanks.
> > > >
> > > > Romain
> > > >
> > > >
> > > > 
> > > > De: Dmitriy Setrakyan 
> > > > >
> > > > Envoye: 23 oct. 2016 23:00
> > > > A: dev@ignite.apache.org
> > > > Objet: Re: Create separate thread pool for query execution
> > > >
> > > > How about long running compute? Do we need a separate thread pool for
> > it
> > > as
> > > > well?
> > > >
> > > > On Sun, Oct 23, 2016 at 3:57 AM, Sergi Vladykin <
> > > sergi.vlady...@gmail.com>
> > > > wrote:
> > > >
> > > > > +1
> > > > >
> > > > > Sergi
> > > > >
> > > > > 

Re: Indexing fields of non-POJO cache values

[Andrey Kornev]

Of course it does, Dmitriy! However as I suggested below, the feature should be 
optional. The current behavior (not requiring user classes on the server, etc.) 
would remain the default one.

Also, please realize that not everyone stores their data as POJOs or uses 
Ignite as a JDBC source -- the use cases that appear to have been the main 
focus of Ignite community lately.

Payloads with dynamic structures require more advanced mechanisms for indexing, 
for example, to avoid the overhead of duplicating the indexable fields as top 
level fields of the BinaryObjects. In cases where the cache sizes are in tens 
of millions of entries, the ability to generate index values on the fly rather 
than store them, would go a long way in terms of reducing memory utilization.

In Ignite community finds this feature generally useful, I'd be more than happy 
to contribute its implementation.

Regards
Andrey


From: Dmitriy Setrakyan <dsetrak...@apache.org>
Sent: Monday, October 16, 2017 6:14 PM
To: dev@ignite.apache.org
Subject: Re: Indexing fields of non-POJO cache values

On Mon, Oct 16, 2017 at 12:35 PM, Andrey Kornev <andrewkor...@hotmail.com>
wrote:

> [Crossposting to the dev list]
>
> Alexey,
>
> Yes, something like that, where the "reference"/"alias" is expressed as a
> piece of Java code (as part of QueryEntity definition, perhaps) that is
> invoked by Ignite at the cache entry indexing time.
>
> My point is that rather than limiting indexable fields only to predefined
> POJO attributes (or BinaryObject fields) Ignite could adopt a more general
> approach by allowing users designate an arbitrary piece of code (a
> lambda/closure) to be used as an index value extractor. In such case, the
> current functionality (extracting index values from POJO attributes)
> becomes just a special case that's supported by Ignite out of the box.
>

Andrey, this would require deserialization on the server side. It would
also require that user classes are present on the server side. Both of this
scenarios Ignite tries to avoid.

Makes sense?

Re: Indexing fields of non-POJO cache values

[Andrey Kornev]

[Crossposting to the dev list]

Alexey,

Yes, something like that, where the "reference"/"alias" is expressed as a piece 
of Java code (as part of QueryEntity definition, perhaps) that is invoked by 
Ignite at the cache entry indexing time.

My point is that rather than limiting indexable fields only to predefined POJO 
attributes (or BinaryObject fields) Ignite could adopt a more general approach 
by allowing users designate an arbitrary piece of code (a lambda/closure) to be 
used as an index value extractor. In such case, the current functionality 
(extracting index values from POJO attributes) becomes just a special case 
that's supported by Ignite out of the box.

This would really help in cases (like mine) where the cache values are non-POJO 
entities.

Thanks
Andrey

From: Alexey Kuznetsov <akuznet...@apache.org>
Sent: Thursday, October 12, 2017 5:53 PM
To: u...@ignite.apache.org
Subject: Re: Indexing fields of non-POJO cache values

Just as idea.

What if we can to declare a kind of "references" or "aliases" for fields in 
such cases?
And this will help us to avoid duplication of data.

For example in JavaScript I could (almost on the fly) declare getters and 
setters that could be as aliases for my data.


On Fri, Oct 13, 2017 at 12:39 AM, Andrey Kornev 
<andrewkor...@hotmail.com<mailto:andrewkor...@hotmail.com>> wrote:
Hey Andrey,

Thanks for your reply!

We've been using a slightly different approach, where we extract the values of 
the indexable leaf nodes and store them as individual fields of the binary 
object along with the serialized tree itself. Then we configure the cache to 
use those fields as QueryEntities. It works fine and this way we avoid using 
joins in our queries.

However an obvious drawback of such approach is data duplication. We end up 
with three copies of a field value:

1) the leaf node of the tree,
2) the field of the binary object, and
3) Ignite index

I was hoping that there may be a better way to achieve this. In particular I'd 
like to avoid storing the value as a field of a binary object (copy #2).

One possible (and elegant) approach to solving this problem would be to 
introduce a way to specify a method (or a closure) for a QueryEntity in 
addition to currently supported BinaryObject field/POJO attribute.

Regards
Andrey


From: Andrey Mashenkov 
<andrey.mashen...@gmail.com<mailto:andrey.mashen...@gmail.com>>
Sent: Thursday, October 12, 2017 6:25 AM
To: u...@ignite.apache.org<mailto:u...@ignite.apache.org>
Subject: Re: Indexing fields of non-POJO cache values

Hi,

Another way here is to implement your own query engine by extending IndexingSPI 
interface, which looks much more complicated.

On Thu, Oct 12, 2017 at 4:23 PM, Andrey Mashenkov 
<andrey.mashen...@gmail.com<mailto:andrey.mashen...@gmail.com>> wrote:
Hi,

There is no way to index such data as is. To index data you need to have 
entry_field<->column mapping configured.
As a workaround here, leaves can be stored in cache as values.

E.g. you can have a separate cache to index leaf nodes, where entries will have 
2 fields: "original tree key" field and "leaf node value" indexed field.
So, you will be able to query serialized tree-like structures via SQL query 
with JOIN condition on  "original tree key" and WHERE condition on "leaf node 
value" field.
Obviously, you will need to implement intermediate logic to keep data of both 
caches consistent.


On Wed, Oct 11, 2017 at 9:40 PM, Andrey Kornev 
<andrewkor...@hotmail.com<mailto:andrewkor...@hotmail.com>> wrote:
Hello,

Consider the following use case: my cache values are a serialized tree-like 
structure (as opposed to a POJO). The leaf nodes of the tree are Java 
primitives. Some of the leaf nodes are used by the queries and should be 
indexed.

What are my options for indexing such data?

Thanks
Andrey



--
Best regards,
Andrey V. Mashenkov



--
Best regards,
Andrey V. Mashenkov



--
Alexey Kuznetsov

Re: IgniteJdbcDriver's usage of JavaLogger

[Andrey Kornev]

Hey Vladimir,

Maybe it'd be better to just use JUL logger directly?

Or, better yet, just get rid of that nagging patronizing warning on line 434 
(the only reason the logger is created in the first place) altogether and 
instead optionally throw an IAE?

Or, include a dummy config/java.util.logging.properties with ignite-indexing 
distribution (under META-INF, perhaps) just to keep JavaLogger happy?

Cheers
Andrey

From: Vladimir Ozerov <voze...@gridgain.com>
Sent: Monday, October 23, 2017 9:03 AM
To: dev@ignite.apache.org
Subject: Re: IgniteJdbcDriver's usage of JavaLogger

Hi Andrey,

What kind of fix do you suggest?

On Mon, Oct 23, 2017 at 6:58 PM, Andrey Kornev <andrewkor...@hotmail.com>
wrote:

> Hello,
>
> Just curious if anyone knows why IgniteJdbcDriver class instantiates a
> JavaLogger() on line 410 rather than using the globally configured logger
> instance?
>
> I have an slf4j logger configured and with ignite-indexing module in the
> classpath, I get scary looking (albeit benign) message in my logs during
> startup:
>
> Oct 23, 2017 9:02:23 AM java.util.logging.LogManager$RootLogger log
> SEVERE: Failed to resolve default logging config file:
> config/java.util.logging.properties
>
> Shouldn't IgniteJdbcDriver be fixed?
>
> Thanks
> Andrey
>

Re: Thin Client Protocol documentation

[Andrey Kornev]

Pavel,

Thanks! While we're at it, are there any plans to add cluster-related 
operations? For example, I think it'd be nice to allow the thin clients to 
obtain a current topology snapshot. This would make it possible the clients to 
send requests directly to the affinity host for colocated computation. To make 
it even more useful, all server responses could optionally include the topology 
version the operation has been executed against. This would effectively give us 
a kind out-of-band topology change notification mechanism. This way the clients 
can detect a topology change and refresh the topology snapshot next time they 
need to compute affinity.

Regards
Andrey

From: Pavel Tupitsyn 
Sent: Sunday, December 3, 2017 9:23 AM
To: dev@ignite.apache.org
Subject: Re: Thin Client Protocol documentation

Hi Andrey,

Compute and other APIs are certainly planned, cache is just a start.
We intentionally limit the scope to actually release something in 2.4 and
not delay it further.

Adding operations to existing protocol is relatively easy.
Current focus is to make sure that the protocol itself is solid and
future-proof.

Thanks,
Pavel

Re: Thin Client Protocol documentation

[Andrey Kornev]

Pavel, could you please at least share your time frame wrt adding compute, 
clustering and services features to the spec?

Thanks
Andrey

From: Pavel Tupitsyn <ptupit...@gridgain.com>
Sent: Tuesday, December 5, 2017 11:15 PM
To: dev@ignite.apache.org
Subject: Re: Thin Client Protocol documentation

Andrey,

As I understand, you suggest to document every prospective feature right
now.
That would be (at least) compute, clustering, transactions, services,
messages, events, failover, data structures, metrics.

I don't think it makes sense to do that. Another problem is that it will
take weeks :)
But, of course, you are welcome to take the initiative.

Thanks,
Pavel

On Tue, Dec 5, 2017 at 10:20 PM, Andrey Kornev <andrewkor...@hotmail.com>
wrote:

> Pavel,
>
> I have absolutely no doubts that support for all those features will be
> added eventually. And if so, wouldn't it be the right thing to do to
> document the operations and their semantics right now without
> necessarily implementing them? It should really help to ensure that the
> protocol can accommodate all those use cases before it gets released to the
> public.
>
> Thanks
> Andrey
> --
> *From:* Pavel Tupitsyn <ptupit...@gridgain.com>
> *Sent:* Tuesday, December 5, 2017 12:07 AM
> *Cc:* dev@ignite.apache.org
> *Subject:* Re: Thin Client Protocol documentation
>
> Andrey,
>
> All of this is to come :)
>
>
> Prachi,
>
> 1) There are no flags, see the doc
> 2) String is simply 4 byte length (n) + n bytes
> 3) Op codes have changed
>
> writeByteLittleEndian(1051, out); // Type code
> writeIntLittleEndian(20, out); // String length
> out.writeUTF("myNewCache"); // Cache name
>
> On Tue, Dec 5, 2017 at 4:39 AM, Prachi Garg <pg...@gridgain.com> wrote:
>
> > Hi Pavel,
> >
> > I am trying to use the OP_CACHE_CREATE_WITH_NAME operation, but can't get
> > it to work.  Digging deeper into the source code, it seems like I have to
> > provide a flag, string length, and position, in addition to the type code
> > and the actual string. Is that correct?
> >
> > Here is the request I am sending to the server-
> >
> > DataOutputStream out = new DataOutputStream(socket.getOutputStream());
> >
> > // Message length
> > writeIntLittleEndian(24, out);
> >
> > // Op code = OP_CACHE_CREATE_WITH_NAME
> > writeShortLittleEndian(1051, out);
> >
> > // Request id
> > long reqId = 1;
> > writeLongLittleEndian(reqId, out);
> >
> > // String (cache name)
> > writeByteLittleEndian(9, out); // Type code
> > writeByteLittleEndian(0, out); // Flag
> > writeIntLittleEndian(20, out); // String length
> > writeIntLittleEndian(0, out); // Position
> > out.writeUTF("myNewCache"); // Cache name
> >
> > // Send request
> > out.flush();
> >
> > But I get the following error on the server side.
> >
> > [2017-12-04 17:27:39,421][ERROR][client-connector-#53][
> ClientListenerNioListener]
> > Failed to parse client request.
> > java.lang.StringIndexOutOfBoundsException: String index out of range:
> 2575
> > at java.lang.String.checkBounds(String.java:385)
> > at java.lang.String.(String.java:462)
> > at org.apache.ignite.internal.binary.BinaryUtils.
> > doReadString(BinaryUtils.java:1314)
> > at org.apache.ignite.internal.binary.BinaryReaderExImpl.
> > readString(BinaryReaderExImpl.java:1055)
> > at org.apache.ignite.internal.processors.platform.client.cache.
> > ClientCacheCreateWithNameRequest.(ClientCacheCreateWithNameReque
> > st.java:43)
> > at org.apache.ignite.internal.processors.platform.client.
> > ClientMessageParser.decode(ClientMessageParser.java:318)
> > at org.apache.ignite.internal.processors.platform.client.
> > ClientMessageParser.decode(ClientMessageParser.java:220)
> > at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> > onMessage(ClientListenerNioListener.java:119)
> > at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> > onMessage(ClientListenerNioListener.java:40)
> > at org.apache.ignite.internal.util.nio.GridNioFilterChain$
> > TailFilter.onMessageReceived(GridNioFilterChain.java:279)
> > at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.
> > proceedMessageReceived(GridNioFilterAdapter.java:109)
> > at org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.
> > body(GridNioAsyncNotifyFilter.java:97)
> > at org.apache.ignite.internal.util.worker.GridWorker.run(
> > GridWorker.java:110)
> > at org.apache.ignite.internal.util.worker.GridW

Re: Facility to detect long STW pauses and other system response degradations

[Andrey Kornev]

My 2 cents. Don’t do it. JVM provides sufficient means of detecting a 
struggling process out of the box. SRE/Operations teams usually know how to 
monitor JVMs and can handle killing of such processes themselves.

The feature adds no value, just complexity (and more configuration parameters 
(!) — as if Ignite didn’t have enough of them already).

Regards,
Andrey
_
From: Denis Magda 
Sent: Monday, November 20, 2017 3:10 PM
Subject: Re: Facility to detect long STW pauses and other system response 
degradations
To: 


My 2 cents.

1. Totally for a separate native process that will handle the monitoring of an 
Ignite process. The watchdog process can simply start a JVM tool like jstat and 
parse its GC logs: https://dzone.com/articles/how-monitor-java-garbage 


2. As for the STW handling, I would make a possible reaction more generic. 
Let’s define a policy (enumeration) that will define how to deal with an 
unstable node. The events might be as follows - kill a node, restart a node, 
trigger a custom script using Runtime.exec or other methods.

What’d you think? Specifically on point 2.

—
Denis

> On Nov 20, 2017, at 6:47 AM, Anton Vinogradov  
> wrote:
>
> Yakov,
>
> Issue is https://issues.apache.org/jira/browse/IGNITE-6171
>
> We split issue to
> #1 STW duration metrics
> #2 External monitoring allows to stop node during STW
>
>> Testing GC pause with java thread is
>> a bit strange and can give info only after GC pause finishes.
>
> That's ok since it's #1
>
> On Mon, Nov 20, 2017 at 5:45 PM, Dmitriy_Sorokin 
> wrote:
>
>> I have tested solution with java-thread and GC logs had contain same pause
>> values of thread stopping which was detected by java-thread.
>>
>>
>> My log (contains pauses > 100ms):
>> [2017-11-20 17:33:28,822][WARN ][Thread-1][root] Possible too long STW
>> pause: 507 milliseconds.
>> [2017-11-20 17:33:34,522][WARN ][Thread-1][root] Possible too long STW
>> pause: 5595 milliseconds.
>> [2017-11-20 17:33:37,896][WARN ][Thread-1][root] Possible too long STW
>> pause: 3262 milliseconds.
>> [2017-11-20 17:33:39,714][WARN ][Thread-1][root] Possible too long STW
>> pause: 1737 milliseconds.
>>
>> GC log:
>> gridgain@dell-5580-92zc8h2:~$ cat
>> ./dev/ignite-logs/gc-2017-11-20_17-33-27.log | grep Total
>> 2017-11-20T17:33:27.608+0300: 0,116: Total time for which application
>> threads were stopped: 0,845 seconds, Stopping threads took: 0,246
>> seconds
>> 2017-11-20T17:33:27.667+0300: 0,175: Total time for which application
>> threads were stopped: 0,0001072 seconds, Stopping threads took: 0,252
>> seconds
>> 2017-11-20T17:33:28.822+0300: 1,330: Total time for which application
>> threads were stopped: 0,5001082 seconds, Stopping threads took: 0,178
>> seconds // GOT!
>> 2017-11-20T17:33:34.521+0300: 7,030: Total time for which application
>> threads were stopped: 5,5856603 seconds, Stopping threads took: 0,229
>> seconds // GOT!
>> 2017-11-20T17:33:37.896+0300: 10,405: Total time for which application
>> threads were stopped: 3,2595700 seconds, Stopping threads took: 0,223
>> seconds // GOT!
>> 2017-11-20T17:33:39.714+0300: 12,222: Total time for which application
>> threads were stopped: 1,7337123 seconds, Stopping threads took: 0,121
>> seconds // GOT!
>>
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>>

Re: Deserialization of BinaryObject's byte arrays

[Andrey Kornev]

Will do! Thanks!

From: Valentin Kulichenko <valentin.kuliche...@gmail.com>
Sent: Monday, November 6, 2017 9:17 AM
To: dev@ignite.apache.org
Subject: Re: Deserialization of BinaryObject's byte arrays

This use case was discussed couple of times already, so it seems to be
pretty important for users. And I like the approach suggested by Andrey.

Andrey, can you create a Jira ticket for this change? Would you like to
contribute it?

-Val

On Fri, Nov 3, 2017 at 4:18 PM, Dmitriy Setrakyan <dsetrak...@apache.org>
wrote:

> This makes sense to me. Sounds like a useful feature.
>
> Would be nice to hear what others in the community think?
>
> D.
>
> On Fri, Nov 3, 2017 at 1:07 PM, Andrey Kornev <andrewkor...@hotmail.com>
> wrote:
>
> > Hello,
> >
> > We store lots of byte arrays (serialized graph-like data structures) as
> > fields of BinaryObject. Later on, while reading such field,
> > BinaryInputStream implementation creates an on-heap array and copies the
> > bytes from the BinaryObject's internal backing array to the new array.
> >
> > While in general case it works just fine, in our case, a lot of CPU is
> > spent on copying of the bytes and significant amount garbage is generated
> > as well.
> >
> > I'd like Ignite Community to consider the following proposal: introduce
> > support for serialization/deserialization of the ByteBuffer class.  A
> > BinaryInputStream implementation would special case ByteBuffer fields
> same
> > way it currently special cases various Java types (collections, dates,
> > timestamps, enums, etc.)
> >
> > The application developer would simply call
> BinaryObjectBuilder.setField(...)
> > passing in an instance of ByteBuffer. During serialization, the
> > ByteBuffer's bytes would simply be copied to the backing array (similar
> to
> > how regular byte arrays are serialized today) and in the binary object's
> > metadata the field marked as, say, GridBinaryMarshaller.BYTE_BUFFER.
> >
> > During deserialization of the field, instead of allocating a new byte
> > array and transferring the bytes from the BinaryObject's backing array,
> > BinaryInputStream would simply wrap the required sub-array into a
> read-only
> > instance of ByteBuffer using "public static ByteBuffer wrap(byte[] array,
> > int offset, int length)" and return the instance to the application.
> >
> > This approach doesn't require any array allocation/data copying and is
> > faster and significantly reduces pressure on the garbage collector. The
> > benefits are especially great when application stores significant amount
> of
> > its data as byte arrays.
> >
> > Clearly, this approach equally applies to arrays of other primitive
> types.
> >
> > A minor complication does arise when dealing with off-heap BinaryObjects,
> > but nothing that can't solved with a little bit of Java reflection
> wizardry
> > (or by other means).
> >
> > Thanks,
> > Andrey
> >
>

Query duration metrics

[Andrey Kornev]

Hi,

It appears as if the query duration metric is a bit misleading.

GridQueryProcessor.executeQuery() doesn't actually "run" the query. It simply 
delegates query execution to a closure passed in as a parameter. The closure 
may or may not actually execute the query. In some (all?) cases, the closure 
simply creates and returns an Iterable (how does 
IgniteH2Indexing.runQueryTwoStep, for example). Actual query execution happens 
at some point later when user instantiates an Iterator from the Iterable. 
What's in fact recorded (by GridQueryProcessor on line 2477) as the "query 
duration" is just the query parsing stage as well as some query's AST 
manipulation logic that tries to convert the regular regular query to a 
map/reduce style query. So, it's basically a "prepare statement duration" that 
is currently reported as "query duration".

Am I missing something?

Thanks
Andrey

Re: Deserialization of BinaryObject's byte arrays

[Andrey Kornev]

Vladimir,

Could you please elaborate as to why adding ByteBuffer to the Binary protocol 
is “not a good idea”? What is inherently wrong about it?

As for your suggestion, while I certainly see your point, it comes with the 
inconvenience of implementing Binarylizable for every entity type in an 
application, which makes me wonder if the self imposed rules such as “type 
system should be kept as small as possible” could be relaxed in the name of 
greater good and user friendliness. Ignite API is not easy to grok as it is 
already.

Thanks
Andrey
_
From: Vladimir Ozerov <voze...@gridgain.com<mailto:voze...@gridgain.com>>
Sent: Monday, November 6, 2017 10:22 PM
Subject: Re: Deserialization of BinaryObject's byte arrays
To: <dev@ignite.apache.org<mailto:dev@ignite.apache.org>>
Cc: Valentin Kulichenko 
<valentin.kuliche...@gmail.com<mailto:valentin.kuliche...@gmail.com>>


Hi Andrey,

While we deifnitely need to support byte buffers, adding new type to
protocol is not very good idea. Binary protocol is the heart of our ssytem,
which is used not oly for plain Java, but for other platforms and SQL. And
ByteBuffer is essentially the same byte array, but with different API. What
you describe is a special case of byte array read/write. For this reason
our type system should be kept as small as possible and it doesn't require
new type to be introduced. Instead, we'd better to handle everything on API
level only. For example, we can add new methods to reader/writer/builder,
which will handle this. E.g.:

BinaryWriter.writeByteArray();
BinaryReader.readByteArray();

Vladimir.

On Mon, Nov 6, 2017 at 9:25 PM, Andrey Kornev 
<andrewkor...@hotmail.com<mailto:andrewkor...@hotmail.com>>
wrote:

> Will do! Thanks!
> 
> From: Valentin Kulichenko 
> <valentin.kuliche...@gmail.com<mailto:valentin.kuliche...@gmail.com>>
> Sent: Monday, November 6, 2017 9:17 AM
> To: dev@ignite.apache.org<mailto:dev@ignite.apache.org>
> Subject: Re: Deserialization of BinaryObject's byte arrays
>
> This use case was discussed couple of times already, so it seems to be
> pretty important for users. And I like the approach suggested by Andrey.
>
> Andrey, can you create a Jira ticket for this change? Would you like to
> contribute it?
>
> -Val
>
> On Fri, Nov 3, 2017 at 4:18 PM, Dmitriy Setrakyan 
> <dsetrak...@apache.org<mailto:dsetrak...@apache.org>>
> wrote:
>
> > This makes sense to me. Sounds like a useful feature.
> >
> > Would be nice to hear what others in the community think?
> >
> > D.
> >
> > On Fri, Nov 3, 2017 at 1:07 PM, Andrey Kornev 
> > <andrewkor...@hotmail.com<mailto:andrewkor...@hotmail.com>>
> > wrote:
> >
> > > Hello,
> > >
> > > We store lots of byte arrays (serialized graph-like data structures) as
> > > fields of BinaryObject. Later on, while reading such field,
> > > BinaryInputStream implementation creates an on-heap array and copies
> the
> > > bytes from the BinaryObject's internal backing array to the new array.
> > >
> > > While in general case it works just fine, in our case, a lot of CPU is
> > > spent on copying of the bytes and significant amount garbage is
> generated
> > > as well.
> > >
> > > I'd like Ignite Community to consider the following proposal: introduce
> > > support for serialization/deserialization of the ByteBuffer class. A
> > > BinaryInputStream implementation would special case ByteBuffer fields
> > same
> > > way it currently special cases various Java types (collections, dates,
> > > timestamps, enums, etc.)
> > >
> > > The application developer would simply call
> > BinaryObjectBuilder.setField(...)
> > > passing in an instance of ByteBuffer. During serialization, the
> > > ByteBuffer's bytes would simply be copied to the backing array (similar
> > to
> > > how regular byte arrays are serialized today) and in the binary
> object's
> > > metadata the field marked as, say, GridBinaryMarshaller.BYTE_BUFFER.
> > >
> > > During deserialization of the field, instead of allocating a new byte
> > > array and transferring the bytes from the BinaryObject's backing array,
> > > BinaryInputStream would simply wrap the required sub-array into a
> > read-only
> > > instance of ByteBuffer using "public static ByteBuffer wrap(byte[]
> array,
> > > int offset, int length)" and return the instance to the application.
> > >
> > > This approach doesn't require any array allocation/data copying and is
> > > faster and significantly reduces pressure on the garbage collector. The
> > > benefits are especially great when application stores significant
> amount
> > of
> > > its data as byte arrays.
> > >
> > > Clearly, this approach equally applies to arrays of other primitive
> > types.
> > >
> > > A minor complication does arise when dealing with off-heap
> BinaryObjects,
> > > but nothing that can't solved with a little bit of Java reflection
> > wizardry
> > > (or by other means).
> > >
> > > Thanks,
> > > Andrey
> > >
> >
>

Misleading query duration metrics

[Andrey Kornev]

(Changed the subject to maybe make people actually read the posting.)

Anyway, if somebody (who knows that code) could take a quick look to see if the 
duration is indeed computed correctly (or not), that would be greatly 
appreciated!

Thanks
Andrey


From: Andrey Kornev <andrewkor...@hotmail.com>
Sent: Sunday, November 5, 2017 12:17 PM
To: dev@ignite.apache.org
Subject: Query duration metrics

Hi,

It appears as if the query duration metric is a bit misleading.

GridQueryProcessor.executeQuery() doesn't actually "run" the query. It simply 
delegates query execution to a closure passed in as a parameter. The closure 
may or may not actually execute the query. In some (all?) cases, the closure 
simply creates and returns an Iterable (how does 
IgniteH2Indexing.runQueryTwoStep, for example). Actual query execution happens 
at some point later when user instantiates an Iterator from the Iterable. 
What's in fact recorded (by GridQueryProcessor on line 2477) as the "query 
duration" is just the query parsing stage as well as some query's AST 
manipulation logic that tries to convert the regular regular query to a 
map/reduce style query. So, it's basically a "prepare statement duration" that 
is currently reported as "query duration".

Am I missing something?

Thanks
Andrey

Deserialization of BinaryObject's byte arrays

[Andrey Kornev]

Hello,

We store lots of byte arrays (serialized graph-like data structures) as fields 
of BinaryObject. Later on, while reading such field, BinaryInputStream 
implementation creates an on-heap array and copies the bytes from the 
BinaryObject's internal backing array to the new array.

While in general case it works just fine, in our case, a lot of CPU is spent on 
copying of the bytes and significant amount garbage is generated as well.

I'd like Ignite Community to consider the following proposal: introduce support 
for serialization/deserialization of the ByteBuffer class.  A BinaryInputStream 
implementation would special case ByteBuffer fields same way it currently 
special cases various Java types (collections, dates, timestamps, enums, etc.)

The application developer would simply call BinaryObjectBuilder.setField(...) 
passing in an instance of ByteBuffer. During serialization, the ByteBuffer's 
bytes would simply be copied to the backing array (similar to how regular byte 
arrays are serialized today) and in the binary object's metadata the field 
marked as, say, GridBinaryMarshaller.BYTE_BUFFER.

During deserialization of the field, instead of allocating a new byte array and 
transferring the bytes from the BinaryObject's backing array, BinaryInputStream 
would simply wrap the required sub-array into a read-only instance of 
ByteBuffer using "public static ByteBuffer wrap(byte[] array, int offset, int 
length)" and return the instance to the application.

This approach doesn't require any array allocation/data copying and is faster 
and significantly reduces pressure on the garbage collector. The benefits are 
especially great when application stores significant amount of its data as byte 
arrays.

Clearly, this approach equally applies to arrays of other primitive types.

A minor complication does arise when dealing with off-heap BinaryObjects, but 
nothing that can't solved with a little bit of Java reflection wizardry (or by 
other means).

Thanks,
Andrey

Re: Deserialization of BinaryObject's byte arrays

[Andrey Kornev]

e consumer and read function.

Vladimir.


On Tue, Nov 7, 2017 at 10:16 AM, Vladimir Ozerov <voze...@gridgain.com>
wrote:

> ByteBuffer is not fundamental data type. It is a thin wrapper over array.
> Protocol should contain only fundamental types.
>
> вт, 7 нояб. 2017 г. в 10:05, Andrey Kornev <andrewkor...@hotmail.com>:
>
>> Vladimir,
>>
>> Could you please elaborate as to why adding ByteBuffer to the Binary
>> protocol is “not a good idea”? What is inherently wrong about it?
>>
>> As for your suggestion, while I certainly see your point, it comes with
>> the inconvenience of implementing Binarylizable for every entity type in an
>> application, which makes me wonder if the self imposed rules such as “type
>> system should be kept as small as possible” could be relaxed in the name of
>> greater good and user friendliness. Ignite API is not easy to grok as it is
>> already.
>>
>> Thanks
>> Andrey
>> _
>> From: Vladimir Ozerov <voze...@gridgain.com<mailto:voze...@gridgain.com>>
>> Sent: Monday, November 6, 2017 10:22 PM
>> Subject: Re: Deserialization of BinaryObject's byte arrays
>> To: <dev@ignite.apache.org<mailto:dev@ignite.apache.org>>
>> Cc: Valentin Kulichenko <valentin.kuliche...@gmail.com> valentin.kuliche...@gmail.com>>
>>
>>
>> Hi Andrey,
>>
>> While we deifnitely need to support byte buffers, adding new type to
>> protocol is not very good idea. Binary protocol is the heart of our
>> ssytem,
>> which is used not oly for plain Java, but for other platforms and SQL. And
>> ByteBuffer is essentially the same byte array, but with different API.
>> What
>> you describe is a special case of byte array read/write. For this reason
>> our type system should be kept as small as possible and it doesn't require
>> new type to be introduced. Instead, we'd better to handle everything on
>> API
>> level only. For example, we can add new methods to reader/writer/builder,
>> which will handle this. E.g.:
>>
>> BinaryWriter.writeByteArray();
>> BinaryReader.readByteArray();
>>
>> Vladimir.
>>
>> On Mon, Nov 6, 2017 at 9:25 PM, Andrey Kornev <andrewkor...@hotmail.com<
>> mailto:andrewkor...@hotmail.com>>
>> wrote:
>>
>> > Will do! Thanks!
>> > 
>> > From: Valentin Kulichenko <valentin.kuliche...@gmail.com> valentin.kuliche...@gmail.com>>
>> > Sent: Monday, November 6, 2017 9:17 AM
>> > To: dev@ignite.apache.org<mailto:dev@ignite.apache.org>
>> > Subject: Re: Deserialization of BinaryObject's byte arrays
>> >
>> > This use case was discussed couple of times already, so it seems to be
>> > pretty important for users. And I like the approach suggested by Andrey.
>> >
>> > Andrey, can you create a Jira ticket for this change? Would you like to
>> > contribute it?
>> >
>> > -Val
>> >
>> > On Fri, Nov 3, 2017 at 4:18 PM, Dmitriy Setrakyan <
>> dsetrak...@apache.org<mailto:dsetrak...@apache.org>>
>> > wrote:
>> >
>> > > This makes sense to me. Sounds like a useful feature.
>> > >
>> > > Would be nice to hear what others in the community think?
>> > >
>> > > D.
>> > >
>> > > On Fri, Nov 3, 2017 at 1:07 PM, Andrey Kornev <
>> andrewkor...@hotmail.com<mailto:andrewkor...@hotmail.com>>
>> > > wrote:
>> > >
>> > > > Hello,
>> > > >
>> > > > We store lots of byte arrays (serialized graph-like data
>> structures) as
>> > > > fields of BinaryObject. Later on, while reading such field,
>> > > > BinaryInputStream implementation creates an on-heap array and copies
>> > the
>> > > > bytes from the BinaryObject's internal backing array to the new
>> array.
>> > > >
>> > > > While in general case it works just fine, in our case, a lot of CPU
>> is
>> > > > spent on copying of the bytes and significant amount garbage is
>> > generated
>> > > > as well.
>> > > >
>> > > > I'd like Ignite Community to consider the following proposal:
>> introduce
>> > > > support for serialization/deserialization of the ByteBuffer class. A
>> > > > BinaryInputStream implementation would special case ByteBuffer
>> fields
>> > > same
>> > > > way it currentl

Re: Thin Client Protocol documentation

[Andrey Kornev]

Pavel,

I have absolutely no doubts that support for all those features will be added 
eventually. And if so, wouldn't it be the right thing to do to document the 
operations and their semantics right now without necessarily implementing them? 
It should really help to ensure that the protocol can accommodate all those use 
cases before it gets released to the public.

Thanks
Andrey

From: Pavel Tupitsyn <ptupit...@gridgain.com>
Sent: Tuesday, December 5, 2017 12:07 AM
Cc: dev@ignite.apache.org
Subject: Re: Thin Client Protocol documentation

Andrey,

All of this is to come :)


Prachi,

1) There are no flags, see the doc
2) String is simply 4 byte length (n) + n bytes
3) Op codes have changed

writeByteLittleEndian(1051, out); // Type code
writeIntLittleEndian(20, out); // String length
out.writeUTF("myNewCache"); // Cache name

On Tue, Dec 5, 2017 at 4:39 AM, Prachi Garg <pg...@gridgain.com> wrote:

> Hi Pavel,
>
> I am trying to use the OP_CACHE_CREATE_WITH_NAME operation, but can't get
> it to work.  Digging deeper into the source code, it seems like I have to
> provide a flag, string length, and position, in addition to the type code
> and the actual string. Is that correct?
>
> Here is the request I am sending to the server-
>
> DataOutputStream out = new DataOutputStream(socket.getOutputStream());
>
> // Message length
> writeIntLittleEndian(24, out);
>
> // Op code = OP_CACHE_CREATE_WITH_NAME
> writeShortLittleEndian(1051, out);
>
> // Request id
> long reqId = 1;
> writeLongLittleEndian(reqId, out);
>
> // String (cache name)
> writeByteLittleEndian(9, out); // Type code
> writeByteLittleEndian(0, out); // Flag
> writeIntLittleEndian(20, out); // String length
> writeIntLittleEndian(0, out); // Position
> out.writeUTF("myNewCache"); // Cache name
>
> // Send request
> out.flush();
>
> But I get the following error on the server side.
>
> [2017-12-04 
> 17:27:39,421][ERROR][client-connector-#53][ClientListenerNioListener]
> Failed to parse client request.
> java.lang.StringIndexOutOfBoundsException: String index out of range: 2575
> at java.lang.String.checkBounds(String.java:385)
> at java.lang.String.(String.java:462)
> at org.apache.ignite.internal.binary.BinaryUtils.
> doReadString(BinaryUtils.java:1314)
> at org.apache.ignite.internal.binary.BinaryReaderExImpl.
> readString(BinaryReaderExImpl.java:1055)
> at org.apache.ignite.internal.processors.platform.client.cache.
> ClientCacheCreateWithNameRequest.(ClientCacheCreateWithNameReque
> st.java:43)
> at org.apache.ignite.internal.processors.platform.client.
> ClientMessageParser.decode(ClientMessageParser.java:318)
> at org.apache.ignite.internal.processors.platform.client.
> ClientMessageParser.decode(ClientMessageParser.java:220)
> at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> onMessage(ClientListenerNioListener.java:119)
> at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> onMessage(ClientListenerNioListener.java:40)
> at org.apache.ignite.internal.util.nio.GridNioFilterChain$
> TailFilter.onMessageReceived(GridNioFilterChain.java:279)
> at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.
> proceedMessageReceived(GridNioFilterAdapter.java:109)
> at org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.
> body(GridNioAsyncNotifyFilter.java:97)
> at org.apache.ignite.internal.util.worker.GridWorker.run(
> GridWorker.java:110)
> at org.apache.ignite.internal.util.worker.GridWorkerPool$1.
> run(GridWorkerPool.java:70)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
> What am I missing here? Can you provide an example of how to send a
> 'String' type request?
>
> -Prachi
>
> On Mon, Dec 4, 2017 at 1:06 PM, Andrey Kornev <andrewkor...@hotmail.com>
> wrote:
>
>> Pavel,
>>
>> Thanks! While we're at it, are there any plans to add cluster-related
>> operations? For example, I think it'd be nice to allow the thin clients to
>> obtain a current topology snapshot. This would make it possible the clients
>> to send requests directly to the affinity host for colocated computation.
>> To make it even more useful, all server responses could optionally include
>> the topology version the operation has been executed against. This would
>> effectively give us a kind out-of-band topology change notification
>> mechanism. This way the clients can detect a topology change and refresh
>> the topology snapshot next time they need to compute affinity.
>>

Re: Thin Client Protocol documentation

[Andrey Kornev]

Pavel,

Great job on the Thin Client protocol design!

I'm a bit late to the party, but I'm wondering why none of the compute features 
are supported? Such omission is unfortunate. Is it intentional? If so, what are 
the reasons?

I think it would be useful for the clients to be able to invoke already 
deployed tasks/services, or send a closure (if the client happens to be written 
in Java) for execution.

Thanks
Andrey


From: Pavel Tupitsyn 
Sent: Friday, December 1, 2017 5:48 AM
To: dev@ignite.apache.org
Subject: Re: Thin Client Protocol documentation

There are no legacy formats. JDBC and ODBC clients are not "legacy", quite
the opposite.

In future we may even want to combine JDBC Thin and general-purpose Thin
clients since they have a lot in common.
So let's keep the handshake format consistent across clients.

> exceptions for message formats
Handshake is an exception anyway, it does not have (or need) requestId, etc.

On Fri, Dec 1, 2017 at 1:38 PM, Alexey Popov  wrote:

> Pavel,
>
> I believe ClientListenerNioListener.onHandshake() could support more than
> one Handshake request format (be backward compatible), for instance, if we
> will have a new handshake code = 0xABCD that differs from 0x01 byte.
>
> It is a design vs architecture question.
>
> I can’t understand why the legacy Handshake format should be used for a
> new protocol. If this protocol is supposed to be public it should have no
> exceptions for message formats.
>
> Thank you,
> Alexey
>
> From: Pavel Tupitsyn
> Sent: Thursday, November 30, 2017 12:11 PM
> To: dev@ignite.apache.org
> Subject: Re: Thin Client Protocol documentation
>
> Hi Alexey,
>
> 1,2,3 are related only to handshake. All other operations are consistent.
>
> Handshake request format is dictated by existing client connector that is
> shared with ODBC and JDBC clients (see
> ClientListenerNioListener.onHandshake).
> so we can't add magic numbers or change operation code.
>
> But yes, we can add server version to the handshake response, and I think
> this makes sense.
>
> > 4. The same comments for success flag (1 byte) and status code (4 bytes)
> in responses. Let's leave only status code.
> We don't have a success flag in responses, there is just a 4-byte status
> code, 0 indicates success, everything else is an error.
>
> Thanks,
> Pavel
>
> On Thu, Nov 30, 2017 at 12:01 PM, Alexey Popov 
> wrote:
>
> > Hi Pavel,
> >
> > Let me add my 5 cents.
> >
> > 1. It would be great if both Handshake request & response have some
> > "magic" number (2 or 4 bytes) inside their msg body. That will simplify
> > handling situations when non-Ignite client connects to Ignite server and
> > vice versa.
> >
> > 2. It makes sense to add server version to successful Handshake response
> > as well. It will help to understand & debug possible backward
> compatibility
> > issues in the field by *.pcap logs analysis & etc.
> >
> > 3. Can we have a more strict header for all message types?
> > As far as I understand,
> > Handshake request has:
> > 1) length - 4 byte
> > 2) Handshake code - 1 byte
> > 3) body - (length - 1) bytes
> >
> > while OP_CACHE_GET request has:
> > 1) length - 4 byte
> > 2) OP_CACHE_GET code - 2 bytes
> > 3) request id - 4 bytes
> > 4) body - (length - 2 - 4) bytes
> >
> > Why some messages have Operation code with 1 byte while others - 2 bytes?
> > Why some requests/responses have request-id while others don't? Let's
> > simplify parser work )
> >
> > 4. The same comments for success flag (1 byte) and status code (4 bytes)
> > in responses. Let's leave only status code.
> >
> > Thank you,
> > Alexey
> >
> > From: Pavel Tupitsyn
> > Sent: Wednesday, November 22, 2017 4:04 PM
> > To: dev@ignite.apache.org
> > Subject: Thin Client Protocol documentation
> >
> > Igniters,
> >
> > I've put together a detailed description of our Thin Client protocol
> > in form of IEP on wiki:
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> > 9+Thin+Client+Protocol
> >
> >
> > To clarify:
> > - Protocol implementation is in master (see ClientMessageParser class)
> > - Protocol has not been released yet, so we are free to change anything
> > - Protocol is only used by .NET Thin Client for now, but is supposed to
> be
> > used from other languages by third party contributors
> > - More operations will be added in future, this is a first set of them,
> > cache-related
> >
> >
> > Please review the document and let me know your thoughts.
> > Is there anything missing or wrong?
> >
> > We should make sure that the foundation is solid and extensible.
> >
> >
> > Thanks,
> > Pavel
> >
> >
>
>

Re: Thin Client Protocol documentation

[Andrey Kornev]

>>> I don't think it makes sense to do that. Another problem is that it will 
>>> take weeks :)

I'm sorry, Pavel. None of what you've said so far makes it obvious why it 
doesn't make sense to produce the complete spec. Would you care to elaborate, 
please?

Also, what's the problem with the "another problem" of the effort taking weeks? 
If based on your best estimate the effort requires a few weeks of your time, 
why is this a problem then?

>>> But, of course, you are welcome to take the initiative.

Saying something like that is not very helpful, Pavel. But let me just say, I 
believe that as the current spec lead and an employee of the company behind 
Apache Ignite project you're in a much better position to continue to lead the 
effort (you've been doing great so far!). From my side, I'd be more than happy 
to review the spec and provide my feedback. Also, I'm very interested in 
becoming a contributor/early adopter of the Java-based implementation of the 
protocol.

Regards
Andrey


From: Pavel Tupitsyn <ptupit...@gridgain.com>
Sent: Tuesday, December 5, 2017 11:15 PM
To: dev@ignite.apache.org
Subject: Re: Thin Client Protocol documentation

Andrey,

As I understand, you suggest to document every prospective feature right
now.
That would be (at least) compute, clustering, transactions, services,
messages, events, failover, data structures, metrics.

I don't think it makes sense to do that. Another problem is that it will
take weeks :)
But, of course, you are welcome to take the initiative.

Thanks,
Pavel

On Tue, Dec 5, 2017 at 10:20 PM, Andrey Kornev <andrewkor...@hotmail.com>
wrote:

> Pavel,
>
> I have absolutely no doubts that support for all those features will be
> added eventually. And if so, wouldn't it be the right thing to do to
> document the operations and their semantics right now without
> necessarily implementing them? It should really help to ensure that the
> protocol can accommodate all those use cases before it gets released to the
> public.
>
> Thanks
> Andrey
> --
> *From:* Pavel Tupitsyn <ptupit...@gridgain.com>
> *Sent:* Tuesday, December 5, 2017 12:07 AM
> *Cc:* dev@ignite.apache.org
> *Subject:* Re: Thin Client Protocol documentation
>
> Andrey,
>
> All of this is to come :)
>
>
> Prachi,
>
> 1) There are no flags, see the doc
> 2) String is simply 4 byte length (n) + n bytes
> 3) Op codes have changed
>
> writeByteLittleEndian(1051, out); // Type code
> writeIntLittleEndian(20, out); // String length
> out.writeUTF("myNewCache"); // Cache name
>
> On Tue, Dec 5, 2017 at 4:39 AM, Prachi Garg <pg...@gridgain.com> wrote:
>
> > Hi Pavel,
> >
> > I am trying to use the OP_CACHE_CREATE_WITH_NAME operation, but can't get
> > it to work.  Digging deeper into the source code, it seems like I have to
> > provide a flag, string length, and position, in addition to the type code
> > and the actual string. Is that correct?
> >
> > Here is the request I am sending to the server-
> >
> > DataOutputStream out = new DataOutputStream(socket.getOutputStream());
> >
> > // Message length
> > writeIntLittleEndian(24, out);
> >
> > // Op code = OP_CACHE_CREATE_WITH_NAME
> > writeShortLittleEndian(1051, out);
> >
> > // Request id
> > long reqId = 1;
> > writeLongLittleEndian(reqId, out);
> >
> > // String (cache name)
> > writeByteLittleEndian(9, out); // Type code
> > writeByteLittleEndian(0, out); // Flag
> > writeIntLittleEndian(20, out); // String length
> > writeIntLittleEndian(0, out); // Position
> > out.writeUTF("myNewCache"); // Cache name
> >
> > // Send request
> > out.flush();
> >
> > But I get the following error on the server side.
> >
> > [2017-12-04 17:27:39,421][ERROR][client-connector-#53][
> ClientListenerNioListener]
> > Failed to parse client request.
> > java.lang.StringIndexOutOfBoundsException: String index out of range:
> 2575
> > at java.lang.String.checkBounds(String.java:385)
> > at java.lang.String.(String.java:462)
> > at org.apache.ignite.internal.binary.BinaryUtils.
> > doReadString(BinaryUtils.java:1314)
> > at org.apache.ignite.internal.binary.BinaryReaderExImpl.
> > readString(BinaryReaderExImpl.java:1055)
> > at org.apache.ignite.internal.processors.platform.client.cache.
> > ClientCacheCreateWithNameRequest.(ClientCacheCreateWithNameReque
> > st.java:43)
> > at org.apache.ignite.internal.processors.platform.client.
> > ClientMessageParser.decode(ClientMessageParser.java:318)
> > at org.apache.igni

Sorted index includes the cache key and the affinity key columns. Why?

[Andrey Kornev]

Hello,

Ignite adds the cache key and optionally the affinity key (if defined) as two 
last columns of all user-defined indexes (line 301 in H2TableDescriptor). For 
example, given a user index salary_idx including a single field "salary", the 
actual index is going to look like this:

CREATE INDEX "SALARY_IDX" ON "mycache"."PERSON" ("SALARY" ASC, "_KEY" ASC)

I'm wondering what is the purpose of doing so. Could someone knowledgeable 
(Vladimir O., for example) share their insights?

Thanks
Andrey

IgniteJdbcDriver's usage of JavaLogger

[Andrey Kornev]

Hello,

Just curious if anyone knows why IgniteJdbcDriver class instantiates a 
JavaLogger() on line 410 rather than using the globally configured logger 
instance?

I have an slf4j logger configured and with ignite-indexing module in the 
classpath, I get scary looking (albeit benign) message in my logs during 
startup:

Oct 23, 2017 9:02:23 AM java.util.logging.LogManager$RootLogger log
SEVERE: Failed to resolve default logging config file: 
config/java.util.logging.properties

Shouldn't IgniteJdbcDriver be fixed?

Thanks
Andrey

Re: DiscoverySpi based on Apache ZooKeeper

[Andrey Kornev]

Semyon,

Not to discourage you or anything, just a interesting fact from recent history.

I vaguely remember already trying to implement DiscoverySpi on top of Zookeeper 
back in 2012. After a few failed attempts and a lot of help from Zookeeper's 
original developers (Flavio Junqueira and Ben Reed) we (Dmitriy S. and I) 
concluded that its not possible to  implement DiscoverySpi on top of Zookeeper 
due to strict(er) semantics of DiscoverySpi. Unfortunately I do not remember 
details, but essentially, in some cases it was not possible to establish total 
ordering of watcher events and under certain circumstances loss of such events 
was possible.

It's not to say that Zookeeper can't be used to implement the cluster 
membership tracking in general. The problem is rather with DiscoverySpi 
semantics that require a different set of APIs than what Zookeeper provides.

Regards
Andrey


From: Semyon Boikov 
Sent: Tuesday, January 9, 2018 3:39 AM
To: dev@ignite.apache.org
Subject: DiscoverySpi based on Apache ZooKeeper

Hi all,

Currently I'm working on implementation of DiscoverySpi based on Apache
ZooKeeper (ZookeeperDiscoverySpi) and want to share results of this work.

In very large clusters (>1000 nodes) current default implementation of
DiscoverySpi - TcpDiscoverySpi - has some significant drawbacks:
- TcpDiscoverySpi organizes nodes in ring, and all messages are passed
sequentially via ring. More nodes there are in ring, more time it takes to
pass message. In very large clusters such architecture can cause slowdown
of important operations (node join, node stop, cache start, etc).
- in TcpDiscoverySpi's protocol each node in ring is able to fail next one
in case of network issues, and it is possible that two nodes can 'kill'
each other (it is possible in complex scenarios when network is broken and
then restored back after some time, such problems were observed in real
environments), and with current TcpDiscoverySpi protocol there is no easy
way to completely fix such problems.
- when some node in ring fails, then previous node tries to restore ring
and sequentially tries to connect to next nodes. If large part of ring
fails then it takes long time to sequentially detect failure of all nodes.
- with TcpDiscoverySpi split brain is possible (one ring can split into two
independent parts), separate mechanism is needed to protect from split
brain when TcpDiscoverySpi is used

Even though most probably some of these problems can be somehow fixed in
TcpDiscoverySpi, it seems more robust and fast DiscoverySpi can be
implemented on top of some existing coordination service.

Apache ZooKeeper is known reliable service and it provides all mechanisms
and consistency guarantees required for Ignite's DiscoverySpi. Some
technical details of ZookeeperDiscoverySpi implementation can be found in
description prepared by Sergey Puchnin:
https://cwiki.apache.org/confluence/display/IGNITE/Discovery+SPI+by+ZooKeeper
.

In our preliminary tests we were able to successfully start 4000+ nodes
with ZookeeperDiscoverySpi. New implementation works faster than
TcpDiscoverySpi and does not have mentioned TcpDiscoverySpi's drawbacks:
- nodes alive status is controlled by ZooKeeper, so nodes never can kill
each other
- ZooKeeper has protection from split brain
- with ZooKeeper it is possible to detect nodes join/failures in batches so
time to detect join/failure of 1 vs 100+ nodes is almost the same

I'm going to finalize implementation of ZookeeperDiscoverySpi in next few
days. But in Ignite there is one more issue caused by the fact that
DiscoverySpi and CommunicationSpi are two independent component: it is
possible that DiscoverySpi considers some node as alive, but at the same
time CommunicationSpi is not able to send message to this node (this issue
exists not only for ZookeeperDiscoverySpi for but for TcpDiscoverySpi too).
Such case is very critical since all internal Ignite's code assumes that if
node is alive then CommunicationSpi is able to send/receive messages
to/from this node. If it is not the case, then any Ignite operation can
hang (with ZooKeeper such situation is possible when due to network
failures nodes have connection with ZooKeeper, but can not establish TCP
connection to each other).

If such case arises, then Ignite should have some mechanism to forcibly
kill faulty nodes to keep cluster operational. But note that in case of
'split brain' scenarios each independent part of cluster will consider
others as 'failed' and there should be some way to choose which part should
be killed. It would be good to provide generic solution for this problem as
part of work on new DiscoverySpi.

We discussed this with Yakov Zhdanov and suggest following: in case when
communication fails to send message to some node and this node is
considered as alive, then Ignite should trigger global 'communication error
resolve' process (obviously, this process should use for messaging 

Re: DiscoverySpi based on Apache ZooKeeper

[Andrey Kornev]

Vladimir,

I wish I could be more helpful, but as I mentioned, I can't remember the exact 
details. It had something to do with rather low level technical details of the 
watcher implementation in Zookeeper.

Also, I never said Zookeeper doesn't provide total ordering guarantees. In 
fact, its sole purpose of existence is to do just that: totally ordered atomic 
broadcast. My point was more about "impedance mismatch" between Zookeeper API 
and DiscoverySpi.

Regards
Andrey


From: Vladimir Ozerov <voze...@gridgain.com>
Sent: Tuesday, January 9, 2018 11:38 PM
To: dev@ignite.apache.org
Cc: Semyon Boikov
Subject: Re: DiscoverySpi based on Apache ZooKeeper

Hi Andrey,

Could you please share detail of this API mismatch? AFAIK, the main
guarantee we need for disco SPI is total message ordering. Zookeeper
provides this guarantee. Moreover, Zookeeper is proven to be correct and
reliable coordinator service by many users and Jepsen tests, as opposed to
various in-house implementations (e.g. Zen of Elasticsearch).

вт, 9 янв. 2018 г. в 21:53, Andrey Kornev <andrewkor...@hotmail.com>:

> Semyon,
>
> Not to discourage you or anything, just a interesting fact from recent
> history.
>
> I vaguely remember already trying to implement DiscoverySpi on top of
> Zookeeper back in 2012. After a few failed attempts and a lot of help from
> Zookeeper's original developers (Flavio Junqueira and Ben Reed) we (Dmitriy
> S. and I) concluded that its not possible to  implement DiscoverySpi on top
> of Zookeeper due to strict(er) semantics of DiscoverySpi. Unfortunately I
> do not remember details, but essentially, in some cases it was not possible
> to establish total ordering of watcher events and under certain
> circumstances loss of such events was possible.
>
> It's not to say that Zookeeper can't be used to implement the cluster
> membership tracking in general. The problem is rather with DiscoverySpi
> semantics that require a different set of APIs than what Zookeeper provides.
>
> Regards
> Andrey
>
> 
> From: Semyon Boikov <sboi...@apache.org>
> Sent: Tuesday, January 9, 2018 3:39 AM
> To: dev@ignite.apache.org
> Subject: DiscoverySpi based on Apache ZooKeeper
>
> Hi all,
>
> Currently I'm working on implementation of DiscoverySpi based on Apache
> ZooKeeper (ZookeeperDiscoverySpi) and want to share results of this work.
>
> In very large clusters (>1000 nodes) current default implementation of
> DiscoverySpi - TcpDiscoverySpi - has some significant drawbacks:
> - TcpDiscoverySpi organizes nodes in ring, and all messages are passed
> sequentially via ring. More nodes there are in ring, more time it takes to
> pass message. In very large clusters such architecture can cause slowdown
> of important operations (node join, node stop, cache start, etc).
> - in TcpDiscoverySpi's protocol each node in ring is able to fail next one
> in case of network issues, and it is possible that two nodes can 'kill'
> each other (it is possible in complex scenarios when network is broken and
> then restored back after some time, such problems were observed in real
> environments), and with current TcpDiscoverySpi protocol there is no easy
> way to completely fix such problems.
> - when some node in ring fails, then previous node tries to restore ring
> and sequentially tries to connect to next nodes. If large part of ring
> fails then it takes long time to sequentially detect failure of all nodes.
> - with TcpDiscoverySpi split brain is possible (one ring can split into two
> independent parts), separate mechanism is needed to protect from split
> brain when TcpDiscoverySpi is used
>
> Even though most probably some of these problems can be somehow fixed in
> TcpDiscoverySpi, it seems more robust and fast DiscoverySpi can be
> implemented on top of some existing coordination service.
>
> Apache ZooKeeper is known reliable service and it provides all mechanisms
> and consistency guarantees required for Ignite's DiscoverySpi. Some
> technical details of ZookeeperDiscoverySpi implementation can be found in
> description prepared by Sergey Puchnin:
>
> https://cwiki.apache.org/confluence/display/IGNITE/Discovery+SPI+by+ZooKeeper
> .
>
> In our preliminary tests we were able to successfully start 4000+ nodes
> with ZookeeperDiscoverySpi. New implementation works faster than
> TcpDiscoverySpi and does not have mentioned TcpDiscoverySpi's drawbacks:
> - nodes alive status is controlled by ZooKeeper, so nodes never can kill
> each other
> - ZooKeeper has protection from split brain
> - with ZooKeeper it is possible to detect nodes join/failures in batches so
> time to detect join/failure of 1 vs 100+ nodes is almost the same
>
> I'm going 

Re: TcpCommunicationSpi in dockerized environment

[Andrey Kornev]

Sergey,

The way I "solved" this problem was to modify both

org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi#getNodeAddresses(TcpDiscoveryNode,
 boolean)

and

org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi#nodeAddresses(ClusterNode)

to make sure the external IP addresses (the ones in ATTR_EXT_ADDRS attribute of 
the cluster node) are listed first in the returned collection.

It did fix the problem and significantly reduced the connection time as Ignite 
no longer had to waste time attempting to connect to the remote node's Docker's 
internal IP. It always results in a socket timeout (2 seconds, by default), and 
in case of multiple nodes, making the cluster startup very slow and unreliable.

Of course, having a Docker Swarm with an overlay network would probably solve 
this problem more elegantly without any code changes, but I'm not a Docker 
expert and Docker Swarm is not my target execution environment anyway. I'd like 
to be able to deploy Ignite nodes in standalone containers and have them join 
the cluster as if they were running on physical hardware.

Hope it helps.
Andrey



From: Sergey Chugunov 
Sent: Friday, February 9, 2018 3:54 AM
To: dev@ignite.apache.org
Subject: TcpCommunicationSpi in dockerized environment

Hello Ignite community,

When testing Ignite in dockerized environment I faced the following issue
with current TcpComminicationSpi implementation.

I had several physical machines and each Ignite node running inside Docker
container had at least two InetAddresses associated with it: one IP address
associated with physical host and one additional IP address of Docker
bridge interface *which was default and the same accross all physical
machines*.

Each node publishes address of its Docker bridge in the list of its
addresses although it is not reachable from remote nodes.
So when node tries to establish communication connection using remote
node's Docker address its request goes to itself like it was a loopback
address.

I would suggest to implement a simple heuristic to avoid this: before
connecting to some remote node's address CommunicationSpi should check
whether local node has exactly the same address. If "remote" and local
addresses are the same CommunicationSpi should skip such address from
remote node's list and proceed with the next one.

Is it safe to implement such heuristic in TcpCommunicationSpi or there are
some risks I'm missing? I would really appreciate any help from expert with
deep knowledge of Communication mechanics.

If such improvement makes sense I'll file a ticket and start working on it.

Thanks,
Sergey.

Re: IEP-14: Ignite failures handling (Discussion)

[Andrey Kornev]

I'm not disagreeing with you, Dmitriy.

What I'm trying to say is that if we assume that a serious enough bug or some 
environmental issue prevents Ignite node from functioning correctly, then it's 
only logical to assume that Ignite process is completely hosed (for example, 
due to a very very long STW pause) and can't make any progress at all. In a 
situation like this the application can't reason about the process state, and 
the process itself may not be able to even kill itself. The only reliable way 
to handle cases like that is to have an external observer (a health monitoring 
tool) that is not itself affected by the bug or the env issue and can either 
make a decision by itself or send a notification to the SRE team.

In my previous post I only suggest to go easy on the "cleverness" of the 
self-monitoring implementation as IMHO it won't be used much in production 
environment. I think Ignite as it is already provides sufficient means of 
monitoring its health (they may or may not be robust enough, which is a 
different issue).

Regards
Andrey


From: Dmitriy Setrakyan <dsetrak...@apache.org>
Sent: Wednesday, March 14, 2018 6:22 PM
To: dev@ignite.apache.org
Subject: Re: IEP-14: Ignite failures handling (Discussion)

On Wed, Mar 14, 2018 at 3:36 PM, Andrey Kornev <andrewkor...@hotmail.com>
wrote:

> If I were the one responsible for running Ignite-based applications (be it
> embedded or standalone Ignite) in my company's datacenter, I'd prefer the
> application nodes simply make their current state readily available to
> external tools (via JMX, health checks, etc.) and leave the decision of
> when to die and when to continue to run up to me. The last thing I need in
> production is a too clever an application that decides to kill itself based
> on its local (perhaps confused) state.
>
> Usually SRE teams build all sorts of technology-specific tools to monitor
> health of the applications and they like to be as much in control as
> possible when it comes to killing processes.
>
> I guess what I'm saying is this: keep things simple. Do not over engineer.
> In real production environments the companies will most likely have this
> feature disabled (I know I would) and instead rely on their own tooling for
> handling failures.
>
>
Andrey, our priority should be to keep the cluster operational. If a frozen
Ignite node is kept around, the whole cluster becomes un-operational. I bet
this is not what you would prefer in production either. However, if we kill
the process, then the cluster should continue to operate.

We are talking about a distributed system in which a failure of one node
should not matter. If we want to keep this promise to the users, then we
must kill the process if Ignite node freezes.

Also, keep in mind that we are talking about the "default" behavior. If you
are not happy with the "default" mode, then you will be able to configure
other behaviors, like keeping the frozen Ignite node around, if you like.

D.

[jira] [Created] (IGNITE-1898) Make it possible to obtain the local Ignite instance statically

[Andrey Kornev (JIRA)]

Andrey Kornev created IGNITE-1898:
-

 Summary: Make it possible to obtain the local Ignite instance 
statically
 Key: IGNITE-1898
 URL: https://issues.apache.org/jira/browse/IGNITE-1898
 Project: Ignite
  Issue Type: Improvement
  Components: general
Reporter: Andrey Kornev
Priority: Minor


Currently the ways to obtain the local instance of Ignite make use of the grid 
name or the node UUID with the corresponding {{Ignition.ignite()}} methods. 
However, during deserialization if would sometime be desirable to be able to 
obtain the local instance of Ignite (the one on which the given class is being 
deserialized) without relying on the grid name. Such instance can then be used 
to look up the node-local resources and any other context. The grid name is not 
reliable way to serialize/deserialize {{IgniteKernal}} in a situation when a 
single JVM has multiple Ignite nodes started (as is often the case in unit 
tests).

The proposed solution is to
# implement a custom ThreadFactory to be used with the public, system, utility 
and so on thread pools. For all newly created thread, the factory will create a 
thread local and initialize it with the current instance of Ignite.
# modify the classes that create their own threads, to do the same.
# provide a public static method on the Ignition class
{{public static Ignite getLocalNode();}}
that returns the value of the thread local. It throws {{IllegalStateException}} 
if the method is invoked from a non-Ignite thread.

This will guarantee that all threads started by a particular Ignite instance 
will have the instance reference available in their thread locals.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-1899) Incorrect Ignite (IgniteKernal) instance deserialized when multiple Ignite nodes started in the same JVM

[Andrey Kornev (JIRA)]

Andrey Kornev created IGNITE-1899:
-

 Summary: Incorrect Ignite (IgniteKernal) instance deserialized 
when multiple Ignite nodes started in the same JVM
 Key: IGNITE-1899
 URL: https://issues.apache.org/jira/browse/IGNITE-1899
 Project: Ignite
  Issue Type: Bug
  Components: general
Reporter: Andrey Kornev


The current approach to serialization/deserialization of {{IgniteKernal}} is 
flawed as it relies on the grid name. However, in a situation when a single JVM 
instance has multiple Ignite instances started (as is often the case in unit 
tests) this approach doesn't work. Ignite requires that each Ignite instance 
started within the same JVM be assigned different names. Thus, during 
deserialization on another node a wrong instance of {{IgniteKernal}} gets 
looked up.

Say, the JVM instance has 2 nodes started: node A and node B. Then, when the 
node A serializes {{IgniteKernal}} it'll write the grid name "A" to the wire. 
Then, during deserialization of the {{IgniteKernal}} instance on the node B, 
the name "A" will be read from the wire and used to look up the Ignite instance 
uisng {{Ignition.ignite("A");}}. Clearly the class (holding a reference to 
{{IgniteKernal}}) ends up with a wrong instance of Ignite. If the class also 
retrieves the node-local resources by doing 
{{Ignition.ignite(deserializedGridName).cluster().nodeLocalMap()}} then it may 
end up with incorrect data.

As of Ignite 1.4, {{ClusterGroupAdapter}} class or any other class that 
serialize the instance of {{IgniteKernal}} suffer from this issue. The fix 
would be to simply stop serializing the grid name altogether and during 
deserialization rely on the thread local instead, as per IGNITE-1898.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-3306) Extend IgniteCluster interface with the methods to send and receive custom discovery events

[Andrey Kornev (JIRA)]

Andrey Kornev created IGNITE-3306:
-

 Summary: Extend IgniteCluster interface with the methods to send 
and receive custom discovery events
 Key: IGNITE-3306
 URL: https://issues.apache.org/jira/browse/IGNITE-3306
 Project: Ignite
  Issue Type: Improvement
Reporter: Andrey Kornev


{{GridDiscoveryManager}} already provides the methods for sending/receiving 
custom discovery messages: {{GridDiscoveryManager.sendCustomEvent(...)}} and 
{{GridDiscoveryManager.setCustomEventListener(...)}} methods correspondingly. 
This API is very powerful as it provides reliable delivery guarantees that are 
totally ordered with respect to the rest of discovery events. It's essential 
for implementing so-called view-synchronous group communication primitives.

Unfortunately, {{GridDiscoveryManager}} is not part of the public API. 

This ticket proposes to extend {{IgniteCluster}} interface to expose those 
methods. {{DiscoveryCustomMessage}} class should also be moved out of the 
{{internal}} package to a public package.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-7156) Disabling readFromBackup for a cache effectively disables near cache on non-affinity nodes

[Andrey Kornev (JIRA)]

Andrey Kornev created IGNITE-7156:
-

 Summary: Disabling readFromBackup for a cache effectively disables 
near cache on non-affinity nodes
 Key: IGNITE-7156
 URL: https://issues.apache.org/jira/browse/IGNITE-7156
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.3
Reporter: Andrey Kornev


This line in GridNearGetFuture:

{{boolean allowLocRead = !forcePrimary || 
cctx.localNode().equals(affNodes.get(0));}} [(github 
link)|https://github.com/apache/ignite/blob/1e8b5501ee9fde61f979c104b4d271a1a458eb1d/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/near/GridNearGetFuture.java#L434]

effectively disables Near Cache if "read from backup" is not enabled in the 
cache config and the local node is not one of the affinity nodes. Disabling 
backup reads should not affect near cache usage.

If this is by design (which would be unfortunate), then the javadocs of 
{{CacheConfiguration.setReadFromBackup(boolean)}} should be updated to reflect 
the behavior.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7368) IgniteCache.getAll() throws high volume of GridDhtInvalidPartitionException if on-heap is enabled

[Andrey Kornev (JIRA)]

Andrey Kornev created IGNITE-7368:
-

 Summary: IgniteCache.getAll() throws high volume of 
GridDhtInvalidPartitionException if on-heap is enabled
 Key: IGNITE-7368
 URL: https://issues.apache.org/jira/browse/IGNITE-7368
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.3
Reporter: Andrey Kornev
 Attachments: Screen Shot 2018-01-09 at 3.37.53 PM.png

With on-heap option enabled, {{IgniteCache.getAll()}} throws a 
GridDhtInvalidPartitionException if the input set of keys contains at least one 
non-local key (a key for which the node executing {{getAll}} is not the 
primary). In general case, the input set always contains such keys which, given 
sufficiently high request rate, results in thousands of exceptions thrown per 
second. Attached is a screenshot of a JFR recording showing the exceptions and 
their stack traces.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)