date:20171113

Re: GA Grid: Request to contribute GA library to Apache Ignite

2017-11-13 Thread techbysample

Denis,

Ok.  It's not clear what stage we are in this process. 

Do I need to fill out the IP Clearance form here?
http://incubator.apache.org/ip-clearance/ip-clearance-template.html
If so, I will simply model what was done for Ignite Persistence Store.

Also, What about the software grant form? When will it be provided?

Will I follow steps/guidelines here for pull request?:
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-
(Section: 1. Create GitHub pull-request)

Please advise on general order of steps in this whole process..

Regards,
Turik





--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: Ignite Enhancement Proposal #6 (Metrics)

2017-11-13 Thread Valentin Kulichenko

Here is the link:

[1] https://issues.apache.org/jira/browse/IGNITE-6846

-Val

On Mon, Nov 13, 2017 at 4:53 PM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> One of the users also brought up the issue that we don't have any metrics
> for entry processors [1]. Probably, makes sense to include this into IEP as
> well.
>
> -Val
>
> On Mon, Nov 13, 2017 at 4:48 PM, Denis Magda  wrote:
>
>> Hi Anton,
>>
>> Thanks for brining this out!
>>
>> The memory metrics really need a serious overhaul. For instance we a
>> missing basic things like:
>> - Bytes used per data region.
>> - Bytes used per cache.
>> - Size of checkpointing buffer.
>>
>> Plus advanced memory metrics will not be harmful:
>> - Space occupied by indexes.
>>
>> The ticket for the memory metrics improved is anxiously waiting to be
>> solved: https://issues.apache.org/jira/browse/IGNITE-5796 <
>> https://issues.apache.org/jira/browse/IGNITE-5796>
>>
>> —
>> Denis
>> > On Nov 13, 2017, at 9:14 AM, Anton Vinogradov  wrote:
>> >
>> > Igniters,
>> >
>> > As you may know Ignite have a lot of JMX based metric, but to perform
>> more
>> > effective grid monitoring some new JMX metrics needs to be implemented.
>> >
>> > Here's the accumulated list of metrics I'd like to see at nearest Ignite
>> > version:
>> > Topology
>> > - Current topology version
>> > - Total server nodes count
>> > - Total client nodes count
>> > - Method to count nodes filtered by some node attribute
>> > - Method to count nodes grouped by some node attribute
>> >
>> > Communication SPI
>> > - Received messages count grouped by message type
>> > - Received messages count grouped by sender node
>> > - Sent messages count grouped by message type
>> > - Sent messages count grouped by receiver node
>> >
>> > Partitions allocation (for cache groups)
>> > - Total primary partitions count located on the current node
>> > - Total backup partitions count located on the current node
>> > - Min/max partition backups left in the cluster for cache group
>> > - Maybe some methods to show partitions map/partition distribution
>> > statistics in the cluster
>> >
>> > Jobs execution
>> > - Total jobs execution time (now job execution statistics gathered since
>> > node started and can't be used to calculate average execution time
>> between
>> > probes, implementation of this metric can solve this problem)
>> >
>> > Cache
>> > - Topology validation status for cache
>> >
>> > IEP-6 [1] was prepared and ready to be discussed.
>> >
>> > Please, feel free to suggect or decline some metric.
>> >
>> > Anton.
>> >
>> > [1]
>> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-6%3A+
>> Metrics+improvements
>>
>>
>

Re: Ignite Enhancement Proposal #6 (Metrics)

2017-11-13 Thread Valentin Kulichenko

One of the users also brought up the issue that we don't have any metrics
for entry processors [1]. Probably, makes sense to include this into IEP as
well.

-Val

On Mon, Nov 13, 2017 at 4:48 PM, Denis Magda  wrote:

> Hi Anton,
>
> Thanks for brining this out!
>
> The memory metrics really need a serious overhaul. For instance we a
> missing basic things like:
> - Bytes used per data region.
> - Bytes used per cache.
> - Size of checkpointing buffer.
>
> Plus advanced memory metrics will not be harmful:
> - Space occupied by indexes.
>
> The ticket for the memory metrics improved is anxiously waiting to be
> solved: https://issues.apache.org/jira/browse/IGNITE-5796 <
> https://issues.apache.org/jira/browse/IGNITE-5796>
>
> —
> Denis
> > On Nov 13, 2017, at 9:14 AM, Anton Vinogradov  wrote:
> >
> > Igniters,
> >
> > As you may know Ignite have a lot of JMX based metric, but to perform
> more
> > effective grid monitoring some new JMX metrics needs to be implemented.
> >
> > Here's the accumulated list of metrics I'd like to see at nearest Ignite
> > version:
> > Topology
> > - Current topology version
> > - Total server nodes count
> > - Total client nodes count
> > - Method to count nodes filtered by some node attribute
> > - Method to count nodes grouped by some node attribute
> >
> > Communication SPI
> > - Received messages count grouped by message type
> > - Received messages count grouped by sender node
> > - Sent messages count grouped by message type
> > - Sent messages count grouped by receiver node
> >
> > Partitions allocation (for cache groups)
> > - Total primary partitions count located on the current node
> > - Total backup partitions count located on the current node
> > - Min/max partition backups left in the cluster for cache group
> > - Maybe some methods to show partitions map/partition distribution
> > statistics in the cluster
> >
> > Jobs execution
> > - Total jobs execution time (now job execution statistics gathered since
> > node started and can't be used to calculate average execution time
> between
> > probes, implementation of this metric can solve this problem)
> >
> > Cache
> > - Topology validation status for cache
> >
> > IEP-6 [1] was prepared and ready to be discussed.
> >
> > Please, feel free to suggect or decline some metric.
> >
> > Anton.
> >
> > [1]
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> 6%3A+Metrics+improvements
>
>

[jira] [Created] (IGNITE-6897) Visor doesn't show the list of caches after persistence enabled cluster restart

2017-11-13 Thread Valentin Kulichenko (JIRA)

Valentin Kulichenko created IGNITE-6897:
---

 Summary: Visor doesn't show the list of caches after persistence 
enabled cluster restart
 Key: IGNITE-6897
 URL: https://issues.apache.org/jira/browse/IGNITE-6897
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
  Components: cache, visor
Affects Versions: 2.3
Reporter: Valentin Kulichenko
 Fix For: 2.4


# Start a cluster with persistence enabled.
# Create a cache dynamically, put some data.
# Restart the cluster.
# Connect with Visor and execute {{cache}} command to get list of caches.
# No caches are shown.
# Try to access the cache from code (e.g. execute a query).
# Now caches are shown in Visor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Ignite Enhancement Proposal #6 (Metrics)

2017-11-13 Thread Denis Magda

Hi Anton,

Thanks for brining this out!

The memory metrics really need a serious overhaul. For instance we a missing 
basic things like:
- Bytes used per data region.
- Bytes used per cache.
- Size of checkpointing buffer.

Plus advanced memory metrics will not be harmful:
- Space occupied by indexes.

The ticket for the memory metrics improved is anxiously waiting to be solved: 
https://issues.apache.org/jira/browse/IGNITE-5796 


—
Denis
> On Nov 13, 2017, at 9:14 AM, Anton Vinogradov  wrote:
> 
> Igniters,
> 
> As you may know Ignite have a lot of JMX based metric, but to perform more
> effective grid monitoring some new JMX metrics needs to be implemented.
> 
> Here's the accumulated list of metrics I'd like to see at nearest Ignite
> version:
> Topology
> - Current topology version
> - Total server nodes count
> - Total client nodes count
> - Method to count nodes filtered by some node attribute
> - Method to count nodes grouped by some node attribute
> 
> Communication SPI
> - Received messages count grouped by message type
> - Received messages count grouped by sender node
> - Sent messages count grouped by message type
> - Sent messages count grouped by receiver node
> 
> Partitions allocation (for cache groups)
> - Total primary partitions count located on the current node
> - Total backup partitions count located on the current node
> - Min/max partition backups left in the cluster for cache group
> - Maybe some methods to show partitions map/partition distribution
> statistics in the cluster
> 
> Jobs execution
> - Total jobs execution time (now job execution statistics gathered since
> node started and can't be used to calculate average execution time between
> probes, implementation of this metric can solve this problem)
> 
> Cache
> - Topology validation status for cache
> 
> IEP-6 [1] was prepared and ready to be discussed.
> 
> Please, feel free to suggect or decline some metric.
> 
> Anton.
> 
> [1]
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-6%3A+Metrics+improvements

Re: GA Grid: Request to contribute GA library to Apache Ignite

2017-11-13 Thread Denis Magda

In addition to that, we need to push GA grid through the IP clearance process:
http://incubator.apache.org/ip-clearance/ip-clearance-template.html

This is how an IP clearance form looked like and process happened when GridGain 
was donating Ignite persistence:
http://incubator.apache.org/ip-clearance/persistent-distributed-store-ignite.html

I’ll help with the formalities.

—
Denis

> On Nov 13, 2017, at 10:03 AM, Yury Babak  wrote:
> 
> Turik,
> 
> From my point of view our first step is add GA Grid as is into package
> org.apache.ignite.ml.genetic in ML module.
> 
> It shouldn't be a problem, but before this we should check that GA Grid fits
> to our codestyle.
> 
> So please prepare pull-request with GA Grid.
> 
> Also if nobody object I will create ticket in our JIRA for this first step.
> 
> And also we have few formal steps, I hope Denis could help with them.
> 
> Regards,
> Yury
> 
> 
> 
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: Problem in Ignite Server When using it with hiredis to Store and retrieve C++ Class Object

2017-11-13 Thread FBMPOC Vzprobe

Hello,

Below is my redis client code through which i'm trying to store multiple
C++ Object to Ignite Server after serializing the object in to string using
protobuf serialization library.

The same code works fine in storing and retrieving for 10lakshs objects
with redis server. But the same client gives invalid data from 127 th
object while retrieval.

Look at he sample output Below:


=
person.proto

syntax = "proto2";

package tutorial;

message  Myself{
  required int32 orgId = 1;
  required string firstName = 2;
  required string lastName = 3;
  required string resume = 4;
  required string other =6;
}
=


#include 
#include "person.pb.h"


auto redis_context = redisConnect("127.0.0.1", 6379);
int _set(int32_t key, tutorial::Myself p){

std::string oss;
p.SerializeToString();
const auto set_reply =
redisCommand(redis_context, "SET %ld
%b",(long)key,oss.c_str(), oss.length());
freeReplyObject(set_reply);

}
int _get(int32_t key){
const auto get_reply =
static_cast(redisCommand(redis_context, "GET
%ld",(long)key));
std::string repr{get_reply->str, static_cast(get_reply->len)};

std::cout<< "\nLength: " << get_reply->len << std::endl;
if(static_cast(get_reply->len) <= 0) {
std::cout << "No key is matching" << std::endl;
return -1;
}

freeReplyObject(get_reply);
  tutorial::Myself person1;
person1.ParseFromString(repr);
   std::cout<<"\n" <<  person1.orgid() << "---" << person1.firstname() <<
"--" << person1.lastname() << "--" << person1.resume() << " --" <<
 person1.other() << std::endl ;


}
int main(int argc, char **argv) {

int start = atoi(argv[1]);
int range = atoi(argv[2]);

tutorial::Myself person;
for (int32_t i = start ; i < range ; ++i)
{
person.clear_orgid();
person.set_orgid(i);
person.set_firstname("John");
person.set_lastname("Cena");
person.set_resume("Analyst");
person.set_other("Summa");

_set(i,person);

}
for (int32_t i = start; i < range; ++i)
{
_get(i);
}

redisFree(redis_context);
}




*OUTPUT: === *

./a.out 125 129

125---John--Cena--Analyst --Summa

126---John--Cena--Analyst --Summa

127---John--Cena--Analyst --Summa

*3104751*---John--Cena--Analyst --Summa  //* in 127th object i could see
some junk value. *


Kindly let me know is there any configuration change to be done at ignite
server side.


NOTE: Included the connector Configuration already in the ignite server
configuration file.

Re: Fair affinity resurrection with baseline topology

2017-11-13 Thread Dmitriy Setrakyan

On Mon, Nov 13, 2017 at 7:49 AM, Alexei Scherbakov <
alexey.scherbak...@gmail.com> wrote:

> +1
>
> In fact, after implementing this, we will no longer need rendezvous
> affinity.
>
> Why do we need other affinity if we already provided ideal assignment to
> user ?
>

Because rendezvous is completely stateless and fair affinity requires
maintaining state between topology changes. I would keep both.

> I suggest hiding affinity function from public API and make available
> publicly only things like partition count and backup filter, because
> implementing correct affinity function is error prone and rarely needed.
>

But what if you do to provide custom partition assignment? Some complex use
cases do require it.

> It's even be possible to store/calculate single partition map for all
> caches if they share same partition count.
>

Don't we already do that with cache groups?

Re: Ignite Enhancement Proposal #6 (Metrics)

2017-11-13 Thread Dmitriy Setrakyan

Thanks, Anton! All makes sense. I was actually under impression that Ignite
already provides some of these metrics.

Can you please create relevant tickets and add them to the IEP page?
Perhaps someone in the community could pick these up. This would be a very
nice way to get started with the project.

D.

On Mon, Nov 13, 2017 at 9:14 AM, Anton Vinogradov  wrote:

> Igniters,
>
> As you may know Ignite have a lot of JMX based metric, but to perform more
> effective grid monitoring some new JMX metrics needs to be implemented.
>
> Here's the accumulated list of metrics I'd like to see at nearest Ignite
> version:
> Topology
> - Current topology version
> - Total server nodes count
> - Total client nodes count
> - Method to count nodes filtered by some node attribute
> - Method to count nodes grouped by some node attribute
>
> Communication SPI
> - Received messages count grouped by message type
> - Received messages count grouped by sender node
> - Sent messages count grouped by message type
> - Sent messages count grouped by receiver node
>
> Partitions allocation (for cache groups)
> - Total primary partitions count located on the current node
> - Total backup partitions count located on the current node
> - Min/max partition backups left in the cluster for cache group
> - Maybe some methods to show partitions map/partition distribution
> statistics in the cluster
>
> Jobs execution
> - Total jobs execution time (now job execution statistics gathered since
> node started and can't be used to calculate average execution time between
> probes, implementation of this metric can solve this problem)
>
> Cache
> - Topology validation status for cache
>
> IEP-6 [1] was prepared and ready to be discussed.
>
> Please, feel free to suggect or decline some metric.
>
> Anton.
>
> [1]
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> 6%3A+Metrics+improvements
>

[GitHub] ignite pull request #3025: Ignite-2.4.1

2017-11-13 Thread DmitriyGovorukhin

GitHub user DmitriyGovorukhin opened a pull request:

https://github.com/apache/ignite/pull/3025

Ignite-2.4.1



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.4.1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3025.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3025


commit e7ca9b65a68de7752195c8f4d2b5180f3c77d19f
Author: Dmitriy Govorukhin 
Date:   2017-11-13T18:52:47Z

ignite-blt-merge -> ignite-2.4.1

commit cc8168fc184bb7f5e3cc3bbb0743397097f78bfb
Author: Dmitriy Govorukhin 
Date:   2017-11-13T19:13:01Z

merge ignite-pitr-rc1 -> ignite-2.4.1




---

Re: GA Grid: Request to contribute GA library to Apache Ignite

2017-11-13 Thread Yury Babak

Turik,

>From my point of view our first step is add GA Grid as is into package
org.apache.ignite.ml.genetic in ML module.

It shouldn't be a problem, but before this we should check that GA Grid fits
to our codestyle.

So please prepare pull-request with GA Grid.

Also if nobody object I will create ticket in our JIRA for this first step.

And also we have few formal steps, I hope Denis could help with them.

Regards,
Yury



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Ignite Enhancement Proposal #6 (Metrics)

2017-11-13 Thread Anton Vinogradov

Igniters,

As you may know Ignite have a lot of JMX based metric, but to perform more
effective grid monitoring some new JMX metrics needs to be implemented.

Here's the accumulated list of metrics I'd like to see at nearest Ignite
version:
Topology
- Current topology version
- Total server nodes count
- Total client nodes count
- Method to count nodes filtered by some node attribute
- Method to count nodes grouped by some node attribute

Communication SPI
- Received messages count grouped by message type
- Received messages count grouped by sender node
- Sent messages count grouped by message type
- Sent messages count grouped by receiver node

Partitions allocation (for cache groups)
- Total primary partitions count located on the current node
- Total backup partitions count located on the current node
- Min/max partition backups left in the cluster for cache group
- Maybe some methods to show partitions map/partition distribution
statistics in the cluster

Jobs execution
- Total jobs execution time (now job execution statistics gathered since
node started and can't be used to calculate average execution time between
probes, implementation of this metric can solve this problem)

Cache
- Topology validation status for cache

IEP-6 [1] was prepared and ready to be discussed.

Please, feel free to suggect or decline some metric.

Anton.

[1]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-6%3A+Metrics+improvements

[jira] [Created] (IGNITE-6896) .NET: support Multidimensional Arrays in binary serializer

2017-11-13 Thread Alexey Popov (JIRA)

Alexey Popov created IGNITE-6896:


 Summary: .NET: support Multidimensional Arrays in binary serializer
 Key: IGNITE-6896
 URL: https://issues.apache.org/jira/browse/IGNITE-6896
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
  Components: platforms
Affects Versions: 2.3
Reporter: Alexey Popov


It is found that legacy 2D, 3D, etc. are not working in BinarySerializer.

Sample reproducer:
{code:java}
[Test]
public void TestXX()
{
var array2D = new float[32, 32];
var res = TestUtils.SerializeDeserialize(array2D);
Assert.AreEqual(array2D, res);
}
{code}

BTW, please note that 2D array in Java (a[2][2]) is just a jugged array:
obj = {byte[2][]@1928}
 0 = {byte[2]@1974}
 1 = {byte[2]@1975}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Fair affinity resurrection with baseline topology

2017-11-13 Thread Alexei Scherbakov

+1

In fact, after implementing this, we will no longer need rendezvous
affinity.

Why do we need other affinity if we already provided ideal assignment to
user ?

I suggest hiding affinity function from public API and make available
publicly only things like partition count and backup filter, because
implementing correct affinity function is error prone and rarely needed.

It's even be possible to store/calculate single partition map for all
caches if they share same partition count.



2017-11-09 21:44 GMT+03:00 Vladimir Ozerov :

> Huge +1.
>
> Related question - is it possible to use affinity comparison to "merge"
> partition maps of caches with similar affinity functions? If we do that, we
> will be able to get rid of these nasty "cache groups".
>
> On Thu, Nov 9, 2017 at 7:42 PM, Dmitriy Setrakyan 
> wrote:
>
> > I think it definitely makes sense to add the Fair affinity function back
> to
> > Ignite.
> >
> > On Thu, Nov 9, 2017 at 11:46 PM, Alexey Goncharuk <
> > alexey.goncha...@gmail.com> wrote:
> >
> > > Igniters,
> > >
> > > After spending some time on baseline topology review, it came to my
> mind
> > > that we can return stateful affinity functions for persistent caches.
> > Since
> > > now we will have a topology which is persisted to the cluster
> metastore,
> > > why not save the partition affinity distribution together with it?
> > >
> > > If we have this distribution saved, we can use it after cluster
> restart,
> > so
> > > the scenario which was broken with fair affinity is no longer valid. We
> > > will always have an old partition distribution and we will be able to
> > > consistently restore it after the cluster restart.
> > >
> > > As for the old colocation issue, we only need to define a way to
> > 'compare'
> > > affinity functions for equality, then we can simply choose another
> > cache's
> > > distribution.
> > >
> > > Do you think this makes sense and worth adding it to Ignite? If so, I
> > will
> > > extends the baseline topology IEP with another phase.
> > >
> > > -- AG
> > >
> >
>



-- 

Best regards,
Alexei Scherbakov

[jira] [Created] (IGNITE-6895) Provide metric to monitor any TX deadlock

2017-11-13 Thread Anton Vinogradov (JIRA)

Anton Vinogradov created IGNITE-6895:


 Summary: Provide metric to monitor any TX deadlock
 Key: IGNITE-6895
 URL: https://issues.apache.org/jira/browse/IGNITE-6895
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
Reporter: Anton Vinogradov






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6894) Provide metric to monitor Internal threads stuck

2017-11-13 Thread Anton Vinogradov (JIRA)

Anton Vinogradov created IGNITE-6894:


 Summary: Provide metric to monitor Internal threads stuck
 Key: IGNITE-6894
 URL: https://issues.apache.org/jira/browse/IGNITE-6894
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
Reporter: Anton Vinogradov






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6893) Provide metric to monitor Java Deadlocks

2017-11-13 Thread Anton Vinogradov (JIRA)

Anton Vinogradov created IGNITE-6893:


 Summary: Provide metric to monitor Java Deadlocks 
 Key: IGNITE-6893
 URL: https://issues.apache.org/jira/browse/IGNITE-6893
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
Reporter: Anton Vinogradov






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6892) Proper behavior on OOM/IgniteOutOfMemoryException

2017-11-13 Thread Anton Vinogradov (JIRA)

Anton Vinogradov created IGNITE-6892:


 Summary: Proper behavior on OOM/IgniteOutOfMemoryException
 Key: IGNITE-6892
 URL: https://issues.apache.org/jira/browse/IGNITE-6892
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
Reporter: Anton Vinogradov


In case of any OOM node should be stopped anyway, what we can provide is user 
callback, something like 'beforeNodeStop'.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6891) Proper behavior on Persistence errors

2017-11-13 Thread Anton Vinogradov (JIRA)

Anton Vinogradov created IGNITE-6891:


 Summary: Proper behavior on Persistence errors 
 Key: IGNITE-6891
 URL: https://issues.apache.org/jira/browse/IGNITE-6891
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
Reporter: Anton Vinogradov


Node should be stopped anyway, what we can provide is user callback, something 
like beforeNodeStop'.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6890) Proper behavior on ExchangeWorker exits with error

2017-11-13 Thread Anton Vinogradov (JIRA)

Anton Vinogradov created IGNITE-6890:


 Summary: Proper behavior on ExchangeWorker exits with error 
 Key: IGNITE-6890
 URL: https://issues.apache.org/jira/browse/IGNITE-6890
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
Reporter: Anton Vinogradov


Node should be stopped anyway, what we can provide is user callback, something 
like beforeNodeStop'.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6889) ignite.queue() returns null when queue configuration is null

2017-11-13 Thread Alexey Kukushkin (JIRA)

Alexey Kukushkin created IGNITE-6889:


 Summary: ignite.queue() returns null when queue configuration is 
null
 Key: IGNITE-6889
 URL: https://issues.apache.org/jira/browse/IGNITE-6889
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
Affects Versions: 2.3, 2.2
Reporter: Alexey Kukushkin


ignite.queue() returns null when queue configuration is null.

Reproducer: assertTrue(ignite.queue("a_queue", 1000, null) != null)

We should either allow specifying null as a queue configuration (meaning 
default configuration) or raise an IllegalArgumentException if collection 
configuration is null



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6888) Cache involved tables set for statement

2017-11-13 Thread Igor Seliverstov (JIRA)

Igor Seliverstov created IGNITE-6888:


 Summary: Cache involved tables set for statement
 Key: IGNITE-6888
 URL: https://issues.apache.org/jira/browse/IGNITE-6888
 Project: Ignite
  Issue Type: Sub-task
  Security Level: Public (Viewable by anyone)
  Components: sql
Reporter: Igor Seliverstov


Currently we have to create AST from {{PreparedStatement}} to collect and 
process all involved 
in the query caches (See IgniteH2Indexing#mvccTracker}}). It makes sense to 
cache parse results per statements by analogy with statements itself.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6886) .NET: Performance: Pre-calculate fieldId in reflective serializer

2017-11-13 Thread Pavel Tupitsyn (JIRA)

Pavel Tupitsyn created IGNITE-6886:
--

 Summary: .NET: Performance: Pre-calculate fieldId in reflective 
serializer
 Key: IGNITE-6886
 URL: https://issues.apache.org/jira/browse/IGNITE-6886
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
  Components: platforms
Reporter: Pavel Tupitsyn
Priority: Minor


See {{BinaryWriter.WriteFieldId}} - we have to calculate id from string on each 
write.
In reflective serializer we could cache these field ids.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6885) .NET: Binary serialization performance: eliminate virtual calls on hot path

2017-11-13 Thread Pavel Tupitsyn (JIRA)

Pavel Tupitsyn created IGNITE-6885:
--

 Summary: .NET: Binary serialization performance: eliminate virtual 
calls on hot path
 Key: IGNITE-6885
 URL: https://issues.apache.org/jira/browse/IGNITE-6885
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
  Components: platforms
Reporter: Pavel Tupitsyn
Priority: Minor


{{BinaryReader}} and {{BinaryWriter}} use {{IBinaryStream}} interface, which 
causes lots of virtual calls on every primitive read/write.

Find out if we can force the JIT to specialize generated code for different 
implementations by making all {{IBinaryStream}} implementations {{struct}} and 
making reader/writer generic.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2927: IGNITE-6752 JDBC thin: connection property refact...

2017-11-13 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2927


---

Re: GA Grid: Request to contribute GA library to Apache Ignite

2017-11-13 Thread techbysample

Yury,

Thanks for feedback.

I reviewed the Trainer API at:
https://github.com/apache/ignite/blob/db7697b17cf6eb94754edb2b5e200655a3610dc1/modules/ml/src/main/java/org/apache/ignite/ml/Trainer.java
 
and also recommend approach that new "GA trainers" should be implemented
that will use GA Grid".


Denis/Yury,

Please advise on next steps based on most recent posts.

Best,
Turik





--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[GitHub] ignite pull request #3024: ignite-6866: remove offheap allocation on client ...

2017-11-13 Thread abeliak

GitHub user abeliak opened a pull request:

https://github.com/apache/ignite/pull/3024

ignite-6866: remove offheap allocation on client nodes

Completely remove offheap from client nodes

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6866

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3024.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3024






---

[jira] [Created] (IGNITE-6884) Implement of tensorFold() and tensorProduct() for vectors and matrices

2017-11-13 Thread Yury Babak (JIRA)

Yury Babak created IGNITE-6884:
--

 Summary: Implement of tensorFold() and tensorProduct() for vectors 
and matrices
 Key: IGNITE-6884
 URL: https://issues.apache.org/jira/browse/IGNITE-6884
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
Reporter: Yury Babak


We want to implement tensor fold and map for matrices and vectors.

Also we must take into consideration the different types of matrices vectors, 
including distribution versions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6883) Distributed sub-query support

2017-11-13 Thread Alin Andrei Corodescu (JIRA)

Alin Andrei Corodescu created IGNITE-6883:
-

 Summary: Distributed sub-query support
 Key: IGNITE-6883
 URL: https://issues.apache.org/jira/browse/IGNITE-6883
 Project: Ignite
  Issue Type: New Feature
  Security Level: Public (Viewable by anyone)
Reporter: Alin Andrei Corodescu
Priority: Critical


Include behaviour similar to the distributedJoins flag for inner queries as 
well (gather data from multiple nodes). 
At the moment sub queries are run on remote nodes without being split into map 
and reduce parts and without taking into account data residing in other nodes 
(sometimes collocation is not possible).
One use case when results are affected by this behaviour is when the subquery 
contains a {code:sql}HAVING{code} clause, since the conditions can filter out 
groups for which not all the entries are on the same node.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #3023: Ignite-gg-12943

2017-11-13 Thread AMashenkov

GitHub user AMashenkov opened a pull request:

https://github.com/apache/ignite/pull/3023

Ignite-gg-12943



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-12943

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3023.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3023






---

[jira] [Created] (IGNITE-6882) Introduction of computation graph

2017-11-13 Thread Yury Babak (JIRA)

Yury Babak created IGNITE-6882:
--

 Summary: Introduction of computation graph
 Key: IGNITE-6882
 URL: https://issues.apache.org/jira/browse/IGNITE-6882
 Project: Ignite
  Issue Type: New Feature
  Security Level: Public (Viewable by anyone)
  Components: ml
Reporter: Yury Babak
Assignee: Yury Babak
 Fix For: 2.4


We want to implement a computation graph for NNs because this should helps us 
achieve for neural networks not only data parallelism but model parallelism too.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #3022: Ignite-1.9.8.b1

2017-11-13 Thread AMashenkov

GitHub user AMashenkov opened a pull request:

https://github.com/apache/ignite/pull/3022

Ignite-1.9.8.b1



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-1.9.8.b1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3022.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3022


commit cfbe8da934741e76c8964af87671a38ec7b6c9a3
Author: dkarachentsev 
Date:   2017-06-06T16:15:59Z

IGNITE-5103 Rolled back due to test failings.

commit 83307da08289c873c5c2eb02d5eb314018bc5c13
Author: Ivan Veselovskiy 
Date:   2017-06-06T13:56:09Z

IGNITE-5410: Fixed assertion in HadoopDataOutStream. This closes #2084.

(cherry picked from commit d2bf961)

commit e95626d609ee225918b49653b7981b180e5d4e49
Author: Evgenii Zhuravlev 
Date:   2017-06-01T16:56:34Z

SSL fix

(cherry picked from commit 95d5595)

commit 340204637a03e5533685f1b11ca65c9121f6e193
Author: Alexei Kaigorodov 
Date:   2017-06-08T16:37:40Z

IGNITE-5103 Rolled back due to test failings. (#69)

commit f3f726e9059e492573dc5125fd5edb5d2f71e9d3
Author: Andrey V. Mashenkov 
Date:   2017-06-13T11:11:17Z

IGNITE-4196: Added means to specify port for H2 debug console. This closes 
#1486.

(cherry picked from commit b246260)

commit c2c237d1222557d3e6b35d9a51a61a4c78e56782
Author: Sergey Kalashnikov 
Date:   2017-02-03T08:41:14Z

IGNITE-4196: Added means to specify port for H2 debug console. This closes 
#1486.

(cherry picked from commit b246260)

commit 4a8f295f2f2f34e8472b1d1320f03744135b2504
Author: Igor Sapego 
Date:   2017-06-13T16:47:00Z

IGNITE-5478: ODBC: SQLNumParams now returns number of required parameters.

(cherry picked from commit b1c56a1)

commit a2a4ec1ee9794cb542f146a07c6c67002cad444e
Author: Igor Sapego 
Date:   2017-06-14T09:16:43Z

IGNITE-5478: Fix for cherry pick

commit d268b32cb252a5f06887d2b803d27ddc20ded95f
Author: Igor Sapego 
Date:   2017-06-16T09:27:35Z

IGNITE-4370: Implemented writing of batch of parameters for ODBC.

(cherry picked from commit c10be5780589cc84e7929e234e4411d515166e0b)

commit 7fbaecc67f1b204162bda4595d6c118ddd45f963
Author: Andrey V. Mashenkov 
Date:   2017-06-16T17:01:49Z

IGNITE-5527: Prevent starvation in stripe pool on unstable topology.

commit f81964f59b0ea5b8dfdc8eb2acc34d2a5b8fee07
Author: sboikov 
Date:   2017-01-10T13:59:17Z

Do not evict removed entries, otherwise removes can be lost.

(cherry picked from commit 55ac6e7)

commit 38f0ea80f3d95be16b38b621b3bcc2910c463997
Author: sboikov 
Date:   2017-01-10T13:59:17Z

Do not evict removed entries, otherwise removes can be lost.

(cherry picked from commit 55ac6e7)

commit 5dd74ff635de50ff9561ccdb51bdeb620f60c3db
Author: sboikov 
Date:   2017-01-10T13:59:17Z

Do not evict removed entries, otherwise removes can be lost.

(cherry picked from commit 55ac6e7)

commit 799ef99b512fffb90b97d926532bc6b9404efbff
Author: Evgenii Zhuravlev 
Date:   2017-06-21T08:56:53Z

Merge remote-tracking branch 'remotes/origin/ignite-1.9.3' into ignite-1.9.4

commit c802b478ef47271f5b8864e0b0ae29925107e75f
Author: agura 
Date:   2017-06-21T15:52:17Z

Compilation is fixed

commit 5fb5c7e3b54ae4efb7a6a1832ba647677d93e0cd
Author: Evgenii Zhuravlev 
Date:   2017-06-22T06:43:03Z

IGNITE-5399 Manual cache rebalancing feature is broken

commit 01d41b72ecc3e81dfc8966cc0e395c247037241c
Author: Evgenii Zhuravlev 
Date:   2017-06-21T10:48:15Z

GG-12256 H2Indexes are not deleted if key class implements Externalizable

commit 5ac9afc719138e37a7d97d9d9db05243eee9a942
Author: Evgenii Zhuravlev 
Date:   2017-06-22T09:36:14Z

IGNITE-5399 add test to testsuite

commit a935d40a80e2f928a84a145aba540a45b156687f
Author: Evgenii Zhuravlev 
Date:   2017-06-22T12:10:32Z

GG-12256 Minor fixes

commit 7e2468770a4eb47a4f61204d8c2000b6ab67c967
Author: nikolay_tikhonov 
Date:   2017-06-22T13:13:01Z

IGNITE-GG-12197 Fixed "Ignore events for discarded update in CLOCK mode".

Signed-off-by: nikolay_tikhonov 

commit 5858efd406bb54352de14a0a7e7f21c2ac7bf899
Author: sboikov 
Date:   2016-12-16T16:23:29Z

IGNITE-2412 - Do not acquire asyncSemaphore for synchronous operations

[jira] [Created] (IGNITE-6881) More flexible metrics aggregation

2017-11-13 Thread Alin Andrei Corodescu (JIRA)

Alin Andrei Corodescu created IGNITE-6881:
-

 Summary: More flexible metrics aggregation
 Key: IGNITE-6881
 URL: https://issues.apache.org/jira/browse/IGNITE-6881
 Project: Ignite
  Issue Type: Wish
  Security Level: Public (Viewable by anyone)
Reporter: Alin Andrei Corodescu
Priority: Minor


The current implementation of ClusterMetrics, while powerful, is not very 
easily extensible. Through a single interface, it can provide both aggregated 
and current metrics, and the aggregated ones are defined through interface 
methods. Suppose I want to aggregate the metrics in some other way (compute 
p90, median etc); I would have to record my own "metric history", which Ignite 
already does under the hood, and compute the aggregates on that object. I 
propose the following solution to this problem:
Expose the history of the metrics through an interface which can provide a List 
of data points and the last recorded value for each metric. The last recorded 
values could be used for current metric querying, while the list of data points 
could be used for aggregation. Current aggregations could come implemented in 
the Ignite distribution, but this design would allow for the user to implement 
his/her own aggregations on the data.

Please feel free to correct me if I was wrong in any part about the current 
implementation, as I did not have a lot of time allocated to this particular 
task.

Another problem this would solve is the overhead of maintaining an interface 
with so many methods. At the time I tested my implementation, I discovered that 
some of the functions either returned 0 all the time or their documentation was 
unclear on how the aggregation is made.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2951: Ignite-gg13021: NPE on node stop when SSL is used...

2017-11-13 Thread AMashenkov

Github user AMashenkov closed the pull request at:

https://github.com/apache/ignite/pull/2951


---

[jira] [Created] (IGNITE-6880) KNN(k nearest neighbor) algorithm

2017-11-13 Thread Yury Babak (JIRA)

Yury Babak created IGNITE-6880:
--

 Summary: KNN(k nearest neighbor) algorithm
 Key: IGNITE-6880
 URL: https://issues.apache.org/jira/browse/IGNITE-6880
 Project: Ignite
  Issue Type: New Feature
  Security Level: Public (Viewable by anyone)
  Components: ml
Reporter: Yury Babak


We want to add KNN to Apache Ignite ML module.

Our implementation should support two modes:

* KNN-classifier(in this mode the output is a class membership. An object is 
classified by a majority vote of its neighbors, with the object being assigned 
to the class most common among its k nearest neighbors (k is a positive 
integer, typically small). If k = 1, then the object is simply assigned to the 
class of that single nearest neighbor)
* KNN-regression(the output is the property value for the object. This value is 
the average of the values of its k nearest neighbors.)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2998: Ignite-gg-13021

2017-11-13 Thread AMashenkov

Github user AMashenkov closed the pull request at:

https://github.com/apache/ignite/pull/2998


---

[jira] [Created] (IGNITE-6879) Support Spring 2.0

2017-11-13 Thread Alexey Kukushkin (JIRA)

Alexey Kukushkin created IGNITE-6879:


 Summary: Support Spring 2.0
 Key: IGNITE-6879
 URL: https://issues.apache.org/jira/browse/IGNITE-6879
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
  Components: spring
Affects Versions: 2.3
Reporter: Alexey Kukushkin


Ignite-spring and ignite-spring-data now use Spring 1.1 and cannot be rebuilt 
with Spring 2.0. Trying to change the Spring dependency version to 
2.0.0.release results in compile errors like below and requires regression in 
general. 
This improvement was created to either migrate from Spring 1.1 to 2.0 or create 
another set of modules ignite-spring-xxx-2 to have backward compatibility with 
Spring 1.1.

[ERROR] 
/Users/kukushal/Dev/incubator-ignite/modules/spring-data/src/main/java/org/apache/ignite/springdata/repository/IgniteRepository.java:[57,10]
 name clash: deleteAll(java.lang.Iterable) in 
org.apache.ignite.springdata.repository.IgniteRepository and 
deleteAll(java.lang.Iterable) in 
org.springframework.data.repository.CrudRepository have the same erasure, yet 
neither overrides the other




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6878) Naive Bayes classifier for ML module

2017-11-13 Thread Yury Babak (JIRA)

Yury Babak created IGNITE-6878:
--

 Summary: Naive Bayes classifier for ML module
 Key: IGNITE-6878
 URL: https://issues.apache.org/jira/browse/IGNITE-6878
 Project: Ignite
  Issue Type: New Feature
  Security Level: Public (Viewable by anyone)
  Components: ml
Reporter: Yury Babak


Naive Bayes classifiers are a family of simple probabilistic classifiers based 
on applying Bayes' theorem with strong (naive) independence assumptions between 
the features.

So we want to add this algorithm to Apache Ignite ML module.

Ideally, implementation should support both multinomial naive Bayes and 
Bernoulli naive Bayes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6877) Improve behaviour for non-correlated subqueries

2017-11-13 Thread Alin Andrei Corodescu (JIRA)

Alin Andrei Corodescu created IGNITE-6877:
-

 Summary: Improve behaviour for non-correlated subqueries
 Key: IGNITE-6877
 URL: https://issues.apache.org/jira/browse/IGNITE-6877
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
  Components: sql
Affects Versions: 2.3
Reporter: Alin Andrei Corodescu
Priority: Minor


Ignite behaves poorly in terms of performance when given queries which contain 
IN or = operators followed by a non-correlated subquery. My guess is that the 
query is actually run for each row of the table in order to test the condition. 
A possible solution is to store the results of the subquery in a temporary 
table, and use it from there.
A workaround at the moment is to use joins instead of IN or = operators, but 
this makes the query much more complicated.

Example:
SELECT name
FROM Employees
WHERE age IN (SELECT age FROM Customers)

The performance of this query is very slow for small amounts of data (about 20 
seconds for 4000 rows in each table last time I tried)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6876) ODBC: Add support for SQL_ATTR_CONNECTION_TIMEOUT

2017-11-13 Thread Alexey Popov (JIRA)

Alexey Popov created IGNITE-6876:


 Summary: ODBC: Add support for SQL_ATTR_CONNECTION_TIMEOUT
 Key: IGNITE-6876
 URL: https://issues.apache.org/jira/browse/IGNITE-6876
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
  Components: odbc
Affects Versions: 2.1
Reporter: Alexey Popov


Remote ODBC client should be able to request a timeout for socket send/receive 
operations. It should be done with SQL_ATTR_CONNECTION_TIMEOUT attribute.
If an application with ODBC driver experiences a timeout for some query, it can 
continue to work after closing the connection and establishing a new one.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

40 matches

Mail list logo