date:20170119

[jira] [Created] (IGNITE-4577) Ensure that certain interface addresses can be excluded form node attributes

2017-01-19 Thread Vladimir Ozerov (JIRA)

Vladimir Ozerov created IGNITE-4577:
---

 Summary: Ensure that certain interface addresses can be excluded 
form node attributes
 Key: IGNITE-4577
 URL: https://issues.apache.org/jira/browse/IGNITE-4577
 Project: Ignite
  Issue Type: Task
  Components: general
Affects Versions: 1.8
Reporter: Vladimir Ozerov
 Fix For: 2.0


*Problem*
Consider a case when node has some network interface which is not accessible 
from the outside (e.g. in Docker container). Ignite adds this address to 
attributes, which are shared with other nodes. Now if remote want to 
communicate with local node chances that he will try to establish connection 
with invalid address. 

In the worst case connection will be impossible. We use {{AddressResolver}} to 
handle this situation.

However, it appears that {{AddressResolver}} cannot prevent certain address to 
appear in address list. As a result users may experience communication delays 
as we establish peer-to-peer connection in one thread, iterating over all 
available addresses.

*Proposed solution*
We need to examine what happens when address resolver is set. May be it is 
necessary to rethink how we handle returned object. E.g. {{null}} or empty 
collection might mean that this address should not be included into the list of 
address. However, it may break existing applications, so chances that other 
solution is needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: ApacheCon CFP closing soon (11 February)

2017-01-19 Thread Denis Magda

+ user list


> On Jan 19, 2017, at 3:07 PM, Denis Magda  wrote:
> 
> Igniters,
> 
> Is there anyone of you who want to attend this conference as a speaker 
> presenting Apache Ignite in some form or sharing your working experience with 
> it?
> 
> —
> Denis
> 
>> Begin forwarded message:
>> 
>> From: Rich Bowen 
>> Subject: ApacheCon CFP closing soon (11 February)
>> Date: January 18, 2017 at 8:45:41 AM PST
>> To: comdev 
>> Reply-To: dev@ignite.apache.org
>> Reply-To: comdev 
>> 
>> Hello, fellow Apache enthusiast. Thanks for your participation, and
>> interest in, the projects of the Apache Software Foundation.
>> 
>> I wanted to remind you that the Call For Papers (CFP) for ApacheCon
>> North America, and Apache: Big Data North America, closes in less than a
>> month. If you've been putting it off because there was lots of time
>> left, it's time to dig for that inspiration and get those talk proposals in.
>> 
>> It's also time to discuss with your developer and user community whether
>> there's a track of talks that you might want to propose, so that you
>> have more complete coverage of your project than a talk or two.
>> 
>> We're looking for talks directly, and indirectly, related to projects at
>> the Apache Software Foundation. These can be anything from in-depth
>> technical discussions of the projects you work with, to talks about
>> community, documentation, legal issues, marketing, and so on. We're also
>> very interested in talks about projects and services built on top of
>> Apache projects, and case studies of how you use Apache projects to
>> solve real-world problems.
>> 
>> We are particularly interested in presentations from Apache projects
>> either in the Incubator, or recently graduated. ApacheCon is where
>> people come to find out what technology they'll be using this time next
>> year.
>> 
>> Important URLs are:
>> 
>> To submit a talk for Apache: Big Data -
>> http://events.linuxfoundation.org/events/apache-big-data-north-america/program/cfp
>> To submit a talk for ApacheCon -
>> http://events.linuxfoundation.org/events/apachecon-north-america/program/cfp
>> 
>> To register for Apache: Big Data -
>> http://events.linuxfoundation.org/events/apache-big-data-north-america/attend/register-
>> To register for ApacheCon -
>> http://events.linuxfoundation.org/events/apachecon-north-america/attend/register-
>> 
>> Early Bird registration rates end March 12th, but if you're a committer
>> on an Apache project, you get the low committer rate, which is less than
>> half of the early bird rate!
>> 
>> For further updated about ApacheCon, follow us on Twitter, @ApacheCon,
>> or drop by our IRC channel, #apachecon on the Freenode IRC network. Or
>> contact me - rbo...@apache.org - with any questions or concerns.
>> 
>> Thanks!
>> 
>> Rich Bowen, VP Conferences, Apache Software Foundation
>> 
>> -- 
>> (You've received this email because you're on a dev@ or users@ mailing
>> list of an Apache Software Foundation project. For subscription and
>> unsubscription information, consult the headers of this email message,
>> as this varies from one list to another.)
>

Upcoming Apache Ignite Webinars

2017-01-19 Thread Denis Magda

Igniters,

Don’t miss a chance to learn more about Apache Ignite deployments best 
practices by attending a webinar of two experienced Apache Ignite solutions 
architects:
https://www.gridgain.com/resources/webinars/deploying-apacher-ignitetm-and-gridgain-top-7-faqs
 


Next, if you want to discover how to automate RDBMS and Ignite integration then 
this webinar is for you:
https://www.gridgain.com/resources/webinars/apacher-ignitetm-web-console-automating-rdbms-integration
 


Prachi, could you add two these webinars to “Latest News” section on the site?

—
Denis

Fwd: ApacheCon CFP closing soon (11 February)

2017-01-19 Thread Denis Magda

Igniters,

Is there anyone of you who want to attend this conference as a speaker 
presenting Apache Ignite in some form or sharing your working experience with 
it?

—
Denis

> Begin forwarded message:
> 
> From: Rich Bowen 
> Subject: ApacheCon CFP closing soon (11 February)
> Date: January 18, 2017 at 8:45:41 AM PST
> To: comdev 
> Reply-To: dev@ignite.apache.org
> Reply-To: comdev 
> 
> Hello, fellow Apache enthusiast. Thanks for your participation, and
> interest in, the projects of the Apache Software Foundation.
> 
> I wanted to remind you that the Call For Papers (CFP) for ApacheCon
> North America, and Apache: Big Data North America, closes in less than a
> month. If you've been putting it off because there was lots of time
> left, it's time to dig for that inspiration and get those talk proposals in.
> 
> It's also time to discuss with your developer and user community whether
> there's a track of talks that you might want to propose, so that you
> have more complete coverage of your project than a talk or two.
> 
> We're looking for talks directly, and indirectly, related to projects at
> the Apache Software Foundation. These can be anything from in-depth
> technical discussions of the projects you work with, to talks about
> community, documentation, legal issues, marketing, and so on. We're also
> very interested in talks about projects and services built on top of
> Apache projects, and case studies of how you use Apache projects to
> solve real-world problems.
> 
> We are particularly interested in presentations from Apache projects
> either in the Incubator, or recently graduated. ApacheCon is where
> people come to find out what technology they'll be using this time next
> year.
> 
> Important URLs are:
> 
> To submit a talk for Apache: Big Data -
> http://events.linuxfoundation.org/events/apache-big-data-north-america/program/cfp
> To submit a talk for ApacheCon -
> http://events.linuxfoundation.org/events/apachecon-north-america/program/cfp
> 
> To register for Apache: Big Data -
> http://events.linuxfoundation.org/events/apache-big-data-north-america/attend/register-
> To register for ApacheCon -
> http://events.linuxfoundation.org/events/apachecon-north-america/attend/register-
> 
> Early Bird registration rates end March 12th, but if you're a committer
> on an Apache project, you get the low committer rate, which is less than
> half of the early bird rate!
> 
> For further updated about ApacheCon, follow us on Twitter, @ApacheCon,
> or drop by our IRC channel, #apachecon on the Freenode IRC network. Or
> contact me - rbo...@apache.org - with any questions or concerns.
> 
> Thanks!
> 
> Rich Bowen, VP Conferences, Apache Software Foundation
> 
> -- 
> (You've received this email because you're on a dev@ or users@ mailing
> list of an Apache Software Foundation project. For subscription and
> unsubscription information, consult the headers of this email message,
> as this varies from one list to another.)

Re: Apache Ignite SEO Audit

2017-01-19 Thread Dmitriy Setrakyan

Hi Terry, this should not be a problem. I have sent you the credentials
privately. Let me know if you need more help.

D.

On Thu, Jan 19, 2017 at 11:35 AM, Terry Erisman 
wrote:

> Hi All,
>
>
>
> In order to proceed effectively with the SEO audit of the Apache Ignite
> website, we need admin access to the Google Analytics account for the
> website. Once we have the Google Analytics account access, we will setup
> Google Webmaster Tools for the website (it isn't setup currently) which
> will
> give us insights into a variety of characteristics of the website.
>
>
>
> If anyone has any concerns about this, please let me know. I look forward
> to
> getting started on this audit.
>
>
>
> Thanks,
>
>
>
> Terry
>
>
>
>
>
>
>
>

Re: Hibernate OGM for Ignite. I need a help with documentation

2017-01-19 Thread Denis Magda

Done. Please check.

Victor, could you address Prachi’s comments in the doc and mine below in this 
discussion?

—
Denis

> On Jan 19, 2017, at 10:43 AM, Dmitriy Setrakyan  wrote:
> 
> I don't have permission to access this doc. Can it be granted?
> 
> On Wed, Jan 18, 2017 at 6:28 PM, Prachi Garg  wrote:
> 
>> Hi Victor,
>> 
>> I have tried to fix the document. However, I do have some questions. Please
>> see my comments in the document.
>> 
>> https://docs.google.com/document/d/1ZSFt9NPnN8EyOpNnNguc5-
>> 9FIDr4RfQLfAZMH0VgBPc/edit#
>> 
>> Thanks,
>> -Prachi
>> 
>> On Tue, Jan 17, 2017 at 7:22 PM, Denis Magda  wrote:
>> 
>>> Hi Victor,
>>> 
>>> This seems to be a duplicate message. Please join the conversation in the
>>> original message
>>> http://apache-ignite-developers.2346864.n4.nabble.
>>> com/Hibernate-OGM-for-Ignite-I-need-a-help-with-
>> documentation-tp13762.html
>>> >> com/Hibernate-OGM-for-Ignite-I-need-a-help-with-
>> documentation-tp13762.html
 
>>> 
>>> —
>>> Denis
>>> 
 On Jan 17, 2017, at 12:12 PM, Victor Z  wrote:
 
 Hi everybody!
 
 Hibernate OGM  is a JPA implementation for
>>> NoSQL
 datastores. I'm creating module for Apache Ignite.
 Here is my work
 ,
>>> and pull
 request >> ApacheIgnite>
 to Hibernate OGM
 I've almost done, but there is one issue with documentation. I have to
 write it.
 But I'm programmer, not a techwriter, and my English is not very good.
 I made a draft of documentation
 https://drive.google.com/open?id=0B2da3AGK_cO6TWpvVzFrWWF0dEk
 
 Could you please take a look at this to correct mistakes or to add some
 useful information?
 
 Here is existing documentation
  that
>> I
 used as an example.
 
 Thank you in advance,
 Victor
>>> 
>>> 
>>

Apache Ignite SEO Audit

2017-01-19 Thread Terry Erisman

Hi All,

 

In order to proceed effectively with the SEO audit of the Apache Ignite
website, we need admin access to the Google Analytics account for the
website. Once we have the Google Analytics account access, we will setup
Google Webmaster Tools for the website (it isn't setup currently) which will
give us insights into a variety of characteristics of the website.

 

If anyone has any concerns about this, please let me know. I look forward to
getting started on this audit.

 

Thanks,

 

Terry

Re: Hibernate OGM for Ignite. I need a help with documentation

2017-01-19 Thread Dmitriy Setrakyan

I don't have permission to access this doc. Can it be granted?

On Wed, Jan 18, 2017 at 6:28 PM, Prachi Garg  wrote:

> Hi Victor,
>
> I have tried to fix the document. However, I do have some questions. Please
> see my comments in the document.
>
> https://docs.google.com/document/d/1ZSFt9NPnN8EyOpNnNguc5-
> 9FIDr4RfQLfAZMH0VgBPc/edit#
>
> Thanks,
> -Prachi
>
> On Tue, Jan 17, 2017 at 7:22 PM, Denis Magda  wrote:
>
> > Hi Victor,
> >
> > This seems to be a duplicate message. Please join the conversation in the
> > original message
> > http://apache-ignite-developers.2346864.n4.nabble.
> > com/Hibernate-OGM-for-Ignite-I-need-a-help-with-
> documentation-tp13762.html
> >  > com/Hibernate-OGM-for-Ignite-I-need-a-help-with-
> documentation-tp13762.html
> > >
> >
> > —
> > Denis
> >
> > > On Jan 17, 2017, at 12:12 PM, Victor Z  wrote:
> > >
> > > Hi everybody!
> > >
> > > Hibernate OGM  is a JPA implementation for
> > NoSQL
> > > datastores. I'm creating module for Apache Ignite.
> > > Here is my work
> > > ,
> > and pull
> > > request  > ApacheIgnite>
> > > to Hibernate OGM
> > > I've almost done, but there is one issue with documentation. I have to
> > > write it.
> > > But I'm programmer, not a techwriter, and my English is not very good.
> > > I made a draft of documentation
> > > https://drive.google.com/open?id=0B2da3AGK_cO6TWpvVzFrWWF0dEk
> > >
> > > Could you please take a look at this to correct mistakes or to add some
> > > useful information?
> > >
> > > Here is existing documentation
> > >  that
> I
> > > used as an example.
> > >
> > > Thank you in advance,
> > > Victor
> >
> >
>

Re: Sort nodes in the ring in order to minimize the number of reconnections

2017-01-19 Thread Dmitriy Setrakyan

I agree with Yakov. Having an integer as region ID should be sufficient to
support all the use cases.

On Thu, Jan 19, 2017 at 4:37 AM, Yakov Zhdanov  wrote:

> Alexander, as far as I remember we talked about having cluster id set via
> TcpDiscoverySpi configuration and also via system property.
>
> What do you mean by "existence of sufficient comparator"? If we require
> that attribute is integer then we don't have any problems. If it is not
> integer then you should throw exception on start.
>
> I repeat this once again - we need to (1)prevent our users from falling
> into terrible discovery issues (which are hard to debug) if users provide
> inconsistent comparators on different nodes, but we also need to have
> (2)full flexibility and control over nodes ordering. I think if we don't
> have comparator on public API then we are still OK with both points above.
> Therefore, I would like you to implement this feature using the approach we
> agreed on before. Let me know if you want me to take a look at the code
> before you proceed.
>
> --Yakov
>

Re: Partition data lost event

2017-01-19 Thread Denis Magda

Val, Alex, could anyone of you document this on readme.io ?

—
Denis

> On Jan 19, 2017, at 9:49 AM, Valentin Kulichenko 
>  wrote:
> 
> Alex,
> 
> Thanks for response!
> 
> -Val
> 
> On Wed, Jan 18, 2017 at 10:17 PM, Alexey Goncharuk <
> alexey.goncha...@gmail.com> wrote:
> 
>> Val et al,
>> 
>> Currently partition lost event is fired on nodes which should become
>> partition owners, but did not find other partition owners to rebalance data
>> from. In your example one of the node should have became an owner for
>> partitions 1, 3, the other - for partitions 2, 4 (according to the affinity
>> function assignment), thus this set of events is fired.
>> 
>> In Ignite 2.0 lost partitions will be handled on coordinator and events
>> will be fired for all partitions on all cache nodes.
>> 
>> --AG
>> 
>> 2017-01-18 23:39 GMT+03:00 Denis Magda :
>> 
>>> Alex G. and Yakov should be able to clarify this since they took part in
>>> the creation of the following tickets related to partitions lost
>> consistency
>>> https://issues.apache.org/jira/browse/IGNITE-1605 <
>>> https://issues.apache.org/jira/browse/IGNITE-1605>
>>> https://issues.apache.org/jira/browse/IGNITE-2378 <
>>> https://issues.apache.org/jira/browse/IGNITE-2378>
>>> 
>>> —
>>> Denis
>>> 
 On Jan 18, 2017, at 12:16 PM, Valentin Kulichenko <
>>> valentin.kuliche...@gmail.com> wrote:
 
 Folks,
 
 Does anyone know how the EVT_CACHE_REBALANCE_PART_DATA_LOST event is
>>> fired?
 I just create a small test and the result confused me. This is what I
>> do:
 
 1. Start several server nodes, all listen to
 EVT_CACHE_REBALANCE_PART_DATA_LOST event.
 2. Create a cache with no backups.
 3. Put some values.
 4. Stop one of the nodes (lose some partitions).
 
 What I noticed is that the event is fired on all nodes that are still
>> in
 topology, but for different partitions. For example, if there are 4
 partitions lost (1,2,3,4), I can get notifications for 1,3 on first
>> node
 and 2,4 on second node. This seems weird and it's not clear to me how
>> to
 use this event.
 
 Can anyone explain this behavior? Is this expected?
 
 Here is the test code:
 
 public class PartitionTest {
   public static void main(String[] args) throws InterruptedException {
   for (int i = 0; i < 4; i++)
   Ignition.start(config("server-" + i));
 
   Ignition.setClientMode(true);
 
   Ignite ignite = Ignition.start();
 
   CacheConfiguration cfg = new
 CacheConfiguration<>("test");
 
   cfg.setAffinity(new RendezvousAffinityFunction(20, null));
 
   IgniteCache cache = ignite.createCache(cfg);
 
   for (int i = 0; i < 100; i++)
   cache.put(i, i);
 
   System.out.println("Populated.");
 
   Thread.sleep(2000);
 
   Ignition.stop("server-3", true);
   }
 
   private static IgniteConfiguration config(String name) {
   IgniteConfiguration cfg = new IgniteConfiguration();
 
   cfg.setGridName(name);
 
 cfg.setIncludeEventTypes(EventType.EVT_CACHE_REBALANCE_
>> PART_DATA_LOST);
   cfg.setLocalEventListeners(Collections. extends
 Event>, int[]>singletonMap(
   new IgnitePredicate() {
   @Override public boolean apply(Event event) {
   U.debug("EVENT: " + event);
 
   return true;
   }
   },
   new int[] {EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST}
   ));
 
   return cfg;
   }
 }
 
 -Val
>>> 
>>> 
>>

Re: Partition data lost event

2017-01-19 Thread Valentin Kulichenko

Alex,

Thanks for response!

-Val

On Wed, Jan 18, 2017 at 10:17 PM, Alexey Goncharuk <
alexey.goncha...@gmail.com> wrote:

> Val et al,
>
> Currently partition lost event is fired on nodes which should become
> partition owners, but did not find other partition owners to rebalance data
> from. In your example one of the node should have became an owner for
> partitions 1, 3, the other - for partitions 2, 4 (according to the affinity
> function assignment), thus this set of events is fired.
>
> In Ignite 2.0 lost partitions will be handled on coordinator and events
> will be fired for all partitions on all cache nodes.
>
> --AG
>
> 2017-01-18 23:39 GMT+03:00 Denis Magda :
>
> > Alex G. and Yakov should be able to clarify this since they took part in
> > the creation of the following tickets related to partitions lost
> consistency
> > https://issues.apache.org/jira/browse/IGNITE-1605 <
> > https://issues.apache.org/jira/browse/IGNITE-1605>
> > https://issues.apache.org/jira/browse/IGNITE-2378 <
> > https://issues.apache.org/jira/browse/IGNITE-2378>
> >
> > —
> > Denis
> >
> > > On Jan 18, 2017, at 12:16 PM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> > >
> > > Folks,
> > >
> > > Does anyone know how the EVT_CACHE_REBALANCE_PART_DATA_LOST event is
> > fired?
> > > I just create a small test and the result confused me. This is what I
> do:
> > >
> > > 1. Start several server nodes, all listen to
> > > EVT_CACHE_REBALANCE_PART_DATA_LOST event.
> > > 2. Create a cache with no backups.
> > > 3. Put some values.
> > > 4. Stop one of the nodes (lose some partitions).
> > >
> > > What I noticed is that the event is fired on all nodes that are still
> in
> > > topology, but for different partitions. For example, if there are 4
> > > partitions lost (1,2,3,4), I can get notifications for 1,3 on first
> node
> > > and 2,4 on second node. This seems weird and it's not clear to me how
> to
> > > use this event.
> > >
> > > Can anyone explain this behavior? Is this expected?
> > >
> > > Here is the test code:
> > >
> > > public class PartitionTest {
> > >public static void main(String[] args) throws InterruptedException {
> > >for (int i = 0; i < 4; i++)
> > >Ignition.start(config("server-" + i));
> > >
> > >Ignition.setClientMode(true);
> > >
> > >Ignite ignite = Ignition.start();
> > >
> > >CacheConfiguration cfg = new
> > > CacheConfiguration<>("test");
> > >
> > >cfg.setAffinity(new RendezvousAffinityFunction(20, null));
> > >
> > >IgniteCache cache = ignite.createCache(cfg);
> > >
> > >for (int i = 0; i < 100; i++)
> > >cache.put(i, i);
> > >
> > >System.out.println("Populated.");
> > >
> > >Thread.sleep(2000);
> > >
> > >Ignition.stop("server-3", true);
> > >}
> > >
> > >private static IgniteConfiguration config(String name) {
> > >IgniteConfiguration cfg = new IgniteConfiguration();
> > >
> > >cfg.setGridName(name);
> > >
> > > cfg.setIncludeEventTypes(EventType.EVT_CACHE_REBALANCE_
> PART_DATA_LOST);
> > >cfg.setLocalEventListeners(Collections. > > Event>, int[]>singletonMap(
> > >new IgnitePredicate() {
> > >@Override public boolean apply(Event event) {
> > >U.debug("EVENT: " + event);
> > >
> > >return true;
> > >}
> > >},
> > >new int[] {EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST}
> > >));
> > >
> > >return cfg;
> > >}
> > > }
> > >
> > > -Val
> >
> >
>

Re: IGNITE-2894 - Binary object inside of Externalizable still serialized with OptimizedMarshaller

2017-01-19 Thread Valentin Kulichenko

Nikita,

In my view we just need to support Externalizable and
writeObject/readObject in BinaryMarshaller and get rid of delegation to
optimized marshaller. Once such classes also go through BinaryMarshaller
streams, they will be aware of binary configuration and will share the same
set of handles as well. This should take care of all the issues we have
here.

-Val

On Thu, Jan 19, 2017 at 7:26 AM, Nikita Amelchev 
wrote:

> I have some questions about single Marshaller.
> It seems not easy to merge OptimizedMarshaller with BinaryMarshaller and is
> there any sense in it?
> When Binary object inside Externalizable serialized with optimized it
> losing all benefits.
> Will OptimizedMarshaller be supported at 2.0 version? Or to merge they is
> better?
> What do you think about it?
>
> In addition, Vladimir Ozerov, I would like to hear your opinion.
>
> 2017-01-17 23:32 GMT+03:00 Denis Magda :
>
> > Someone else added you to the contributors list in JIRA. This is why I
> > couldn’t add you for the second time. Ignite committers, please reply on
> > the dev list if you add someone to the list.
> >
> > Nikita, yes, this ticket is still relevant. Go ahead and assign it on
> > yourself.
> >
> > Also please you may want to help with approaching 2.0 release and take
> > care of one of the sub-tasks that must be included in 2.0:
> > https://issues.apache.org/jira/browse/IGNITE-4547 <
> > https://issues.apache.org/jira/browse/IGNITE-4547>
> >
> > —
> > Denis
> >
> > > On Jan 15, 2017, at 9:02 PM, Nikita Amelchev 
> > wrote:
> > >
> > > This issue was created long ago. Is still relevant?
> > >
> > > JIRA account:
> > > Username: NSAmelchev
> > > Full Name: Amelchev Nikita
> > >
> > >
> > > 2017-01-14 1:52 GMT+03:00 Denis Magda :
> > >
> > >> Hi Nikita,
> > >>
> > >> I can’t find provided account in Ignite JIRA
> > >> https://issues.apache.org/jira/browse/IGNITE <
> > https://issues.apache.org/
> > >> jira/browse/IGNITE>
> > >>
> > >> Please create an account there and share with me.
> > >>
> > >> This information might be useful for you as well.
> > >>
> > >> Subscribe to both dev and user lists:
> > >> https://ignite.apache.org/community/resources.html#mail-lists
> > >>
> > >> Get familiar with Ignite development process described here:
> > >> https://cwiki.apache.org/confluence/display/IGNITE/
> Development+Process
> > >>
> > >> Instructions on how to contribute can be found here:
> > >> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
> > >>
> > >> Project setup in Intellij IDEAL
> > >> https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup
> > >>
> > >> Regards,
> > >> Denis
> > >>
> > >>> On Jan 13, 2017, at 1:37 AM, Nikita Amelchev 
> > >> wrote:
> > >>>
> > >>> Hello everyone.
> > >>>
> > >>> I'd like to take IGNITE-2894. Can you assign to me?
> > >>>
> > >>> Username: NSAmelchev
> > >>>
> > >>> --
> > >>> Best wishes,
> > >>> Amelchev Nikita
> > >>
> > >>
> > >
> > >
> > > --
> > > Best wishes,
> > > Amelchev Nikita
> >
> >
>
>
> --
> Best wishes,
> Amelchev Nikita
>

Re: Allow distributed SQL query execution over explicit set of partitions

2017-01-19 Thread Dmitriy Setrakyan

Alexey S,

My comments are below. I would like to ask you again to provide all the API
changes explicitly in the Jira ticket without asking the community to
download the whole GIT repo.

D.

On Thu, Jan 19, 2017 at 6:27 AM, Alexei Scherbakov <
alexey.scherbak...@gmail.com> wrote:

> 1. OK.
>

Agree.


>
> 2. Agreed. In future we might split query execution between nodes, but for
> now query is routed to random node in grid,
>

If you are talking about REPLICATED caches, then sending query to a random
node when a user explicitly specifies the partitions is just deceiving. I
would throw an exception in case if a user specifies partitions for
REPLICATED caches.


>
> 3. OK, let's mark getter/setter as deprecated.
>

I still do not know the proposed new API and why we are deprecating methods
on the old API. I have asked many times to post all the API changes in the
ticket.

Alexey S., can you please do it, so the community members can review them
without installing the project on their mobile devices?

4. Query must be executed locally only for defined partitions. Currently
> this setting is ignored for local queries.
>

This is again the wrong behavior. We should not "ignore" anything. Let's
throw an exception with a correct error message.


> 5. I have the same understanding. Distributed joins will ignore the
> setting.
> This is not implemented yet..
>

And again, this will be very confusing to users. Any chance we can throw an
exception with a proper error message here?


>
>
> 2017-01-19 15:39 GMT+03:00 Sergi Vladykin :
>
> > Agree, lets remove everything related to partition ranges. Looks like
> > unnecessary complication.
> >
> > Sergi
> >
> > 2017-01-19 10:01 GMT+03:00 Vladimir Ozerov :
> >
> > > Several side notes about API.
> > >
> > > 1) I would avoid ranges even in this form.for the sake of simplicity.
> > > Ignite do not have any notion of "partition range" in affinity API, so
> I
> > do
> > > not understand how users are going to work on ranges unless they have
> > some
> > > very special custom affinity function, which is rather unlikely case.
> > >
> > > 2) The fact that this property is ignored in REPLICATED cache is
> > confusing.
> > > REPLICATED cache still divides partitions into primaries and backups.
> If
> > I
> > > have very large data set and want to execute some query, I would
> > definitely
> > > expect that Ignite will take advantage of distributed computing and
> > spread
> > > the load between nodes. I understand that currently SQL queries do not
> > work
> > > this way, but this is clear disadvantage for certain scenarios, which
> we
> > > may improve in future. I would remove this paragraph from docs.
> > >
> > > 3) We already have ScanQuery.partition getter/setter. We need to make
> > sure
> > > that they are "merged" somehow. For instance, we may deprecate two
> > methods
> > > in ScanQuery class, and advise users to use Query.partitions, with
> > > clarification - only single partition is supported for ScanQuery at the
> > > moment.
> > >
> > > 4) What should happen if "partitions" are defined and "local" flag is
> > set?
> > >
> > > As per distributed joins - how are we going to execute them when
> > partitions
> > > are set explicitly? As far as I understand, partitions should apply
> only
> > to
> > > map step and only for the cache query was created from, This way
> > > distributed join execution should effectively ignore partitions?
> > >
> > > Vladimir.
> > >
> > >
> > > On Thu, Jan 19, 2017 at 1:04 AM, Alexei Scherbakov <
> > > alexey.scherbak...@gmail.com> wrote:
> > >
> > > > I mean distributed joins.
> > > >
> > > > 2017-01-19 0:10 GMT+03:00 Alexei Scherbakov <
> > > alexey.scherbak...@gmail.com>
> > > > :
> > > >
> > > > > Guys,
> > > > >
> > > > > I've finished adding API changes and implemented proper nodes
> > routing.
> > > > >
> > > > > Currently it doesn't work with distributed queries.But I think this
> > > > > feature should be compatible with it.
> > > > >
> > > > > Could anyone take a look at current branch state while I'm looking
> > > deeper
> > > > > into dsitributed queries code?
> > > > >
> > > > > Issue: https://issues.apache.org/jira/browse/IGNITE-4523
> > > > > PR: https://github.com/apache/ignite/pull/1418
> > > > >
> > > > >
> > > > >
> > > > > 2017-01-13 15:55 GMT+03:00 Alexei Scherbakov <
> > > > alexey.scherbak...@gmail.com
> > > > > >:
> > > > >
> > > > >> OK, let's do it this way.
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >> 2017-01-13 13:27 GMT+03:00 Sergi Vladykin <
> sergi.vlady...@gmail.com
> > >:
> > > > >>
> > > > >>> Internally we still use int[] when we send partitions (see
> > > > >>> GridH2QueryRequest.parts). It looks like we only do more work
> with
> > > > >>> PartitionSet.
> > > > >>>
> > > > >>> I like the idea of bitset for partitions, but
> > > > >>>
> > > > >>> 1. We have to change internals first to use it, otherwise the
> > > > >>> optimization
> > >

[jira] [Created] (IGNITE-4576) .NET: Rename IgniteConfiguration.gridName

2017-01-19 Thread Alexandr Fedotov (JIRA)

Alexandr Fedotov created IGNITE-4576:


 Summary: .NET: Rename IgniteConfiguration.gridName
 Key: IGNITE-4576
 URL: https://issues.apache.org/jira/browse/IGNITE-4576
 Project: Ignite
  Issue Type: Improvement
  Components: general
Affects Versions: 1.1.4
Reporter: Alexandr Fedotov
 Fix For: 2.0


The same as [IGNITE-3207|https://issues.apache.org/jira/browse/IGNITE-3207] for 
.NET



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: IgniteConfiguration.gridName is very confusing

2017-01-19 Thread Alexander Fedotov

Hi,

I've completed working on IGNITE-3207
https://issues.apache.org/jira/browse/IGNITE-3207

Looks like TC test results don't have problems related to my changes
http://ci.ignite.apache.org/viewLog.html?buildId=423955=buildResultsDiv=IgniteTests_RunAll

Kindly take a look at PR https://github.com/apache/ignite/pull/1435/

On Thu, Jan 12, 2017 at 1:16 AM, Denis Magda  wrote:

> Support Pavel’s point of view.
>
> Also Alexander please make sure that your changes are merged into
> ignite-2.0 branch rather than to the master. I think this functionality
> has to be available in 2.0 first. Finally, please update 2.0 Migration
> Guide once you’ve finished with this task:
> https://cwiki.apache.org/confluence/display/IGNITE/
> Apache+Ignite+2.0+Migration+Guide  confluence/display/IGNITE/Apache+Ignite+2.0+Migration+Guide>
>
> —
> Denis
>
> > On Jan 10, 2017, at 1:58 AM, Pavel Tupitsyn 
> wrote:
> >
> > I think we should fix log output as well and replace all "grid"
> occurences
> > with "instance".
> >
> > On Tue, Jan 10, 2017 at 12:55 PM, Alexander Fedotov <
> > alexander.fedot...@gmail.com> wrote:
> >
> >> Hi,
> >>
> >> I think we should leave null as a default value for unnamed Ignite
> >> instances. At least that change should be considered out of the current
> >> scope.
> >>
> >> What about naming, I'm also renaming log occurrences of "grid" and "grid
> >> name" where it stands reasonable.
> >> Are there places in the logging logic where we should prefer name
> "grid" or
> >> "grid name" instead of "Ignite instance name" or "Ignite instance name"
> can
> >> be used without any semantic impact?
> >>
> >> On Sat, Dec 31, 2016 at 11:23 AM, Alexander Fedotov <
> >> alexander.fedot...@gmail.com> wrote:
> >>
> >>> Okay. From the all said above I suppose "instanceName" should work for
> >>> IgniteConfiguration and "igniteInstanceName" in all other places.
> >>>
> >>> Regards,
> >>> Alexander
> >>>
> >>> 31 дек. 2016 г. 3:43 AM пользователь "Dmitriy Setrakyan" <
> >>> dsetrak...@apache.org> написал:
> >>>
> >>> It sounds like it must be unique then. I would propose the following:
> >>>
> >>>   1. If user defines the instanceName, then we assign it to the node.
> >>>   2. If user does not define the instance name, then we have to give it
> >>>   some unique value, like node ID or PID.
> >>>
> >>> Will this change be backward compatible, or should we leave it as null
> if
> >>> user does not define it?
> >>>
> >>> D.
> >>>
> >>> On Fri, Dec 30, 2016 at 4:19 PM, Denis Magda 
> >> wrote:
> >>>
>  Sounds reasonable. Agree that 'instanceName' suits better considering
> >>> your
>  explanation.
> 
>  --
>  Denis
> 
>  On Friday, December 30, 2016, Valentin Kulichenko <
>  valentin.kuliche...@gmail.com> wrote:
> > This name identifies instance of Ignite, in case there are more than
> >>> one
> > within an application. Here are our API methods around this:
> >
> > // We provide a name and get newly started *Ignite* instance.
> > Ignite ignite = Ignition.start(new
>  IgniteConfiguration().setGridName(name));
> >
> > // We provide a name and get existing *Ignite* instance.
> > Ignite ignite = Ignition.ignite(name);
> >
> > This has nothing to do with nodes. For node representation we have
> > ClusterNode API, which already has nodeId() method for
> >> identification.
> >
> > In other words, if we choose nodeName, we will have both nodeName and
> > nodeId in the product, but with absolutely different meaning and used
> >>> in
> > different parts of API. How user is going to understand the
> >> difference
> > between them? In my view, this is even more confusing than current
>  gridName.
> >
> > -Val
> >
> > On Fri, Dec 30, 2016 at 2:42 PM, Denis Magda 
>  wrote:
> >
> >> Alexander, frankly speaking I'm still for your original proposal -
> >> nodeName. The uniqueness specificities can be set in the doc.
> >>
> >> --
> >> Denis
> >>
> >> On Friday, December 30, 2016, Alexander Fedotov <
> >> alexander.fedot...@gmail.com> wrote:
> >>> Well, then may be we should go with one of the below names:
> >>>
> >>> processNodeName
> >>> jvmNodeName
> >>> runtimeNodeName
> >>> processScopedNodeName
> >>> jvmScopedNodeName
> >>> runtimeScopedNodeName
> >>> processWideNodeName
> >>> jvmWideNodeName
> >>> runtimeWideNodeName
> >>>
> >>> Regards,
> >>> Alexander
> >>>
> >>> 31 дек. 2016 г. 12:37 AM пользователь "Denis Magda" <
>  dma...@apache.org>
> >>> написал:
> >>>
> >>> The parameter specifies a node name which has to be unique per JVM
> >> process
> >>> (if you start multiple nodes in a single process). In my
> >>> understanding
>  it
> >>> was mainly introduced to handle these

[jira] [Created] (IGNITE-4575) Implement in Ignite wrapper for enums based on H2 user value type

2017-01-19 Thread Alexander Paschenko (JIRA)

Alexander Paschenko created IGNITE-4575:
---

 Summary: Implement in Ignite wrapper for enums based on H2 user 
value type
 Key: IGNITE-4575
 URL: https://issues.apache.org/jira/browse/IGNITE-4575
 Project: Ignite
  Issue Type: Sub-task
  Components: SQL
Reporter: Alexander Paschenko
Assignee: Alexander Paschenko
 Fix For: 1.9






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4574) Introduce user value types to H2 w/custom conversion logic

2017-01-19 Thread Alexander Paschenko (JIRA)

Alexander Paschenko created IGNITE-4574:
---

 Summary: Introduce user value types to H2 w/custom conversion logic
 Key: IGNITE-4574
 URL: https://issues.apache.org/jira/browse/IGNITE-4574
 Project: Ignite
  Issue Type: Sub-task
Reporter: Alexander Paschenko






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4573) Optimize H2 comparisons w/constant

2017-01-19 Thread Alexander Paschenko (JIRA)

Alexander Paschenko created IGNITE-4573:
---

 Summary: Optimize H2 comparisons w/constant
 Key: IGNITE-4573
 URL: https://issues.apache.org/jira/browse/IGNITE-4573
 Project: Ignite
  Issue Type: Sub-task
  Components: SQL
Reporter: Alexander Paschenko
Assignee: Alexander Paschenko


Currently H2 performs constant conversions for each row - say, for {{WHERE 
person.id = '1'}} the {{'1'}} will be converted to {{int}} for each row which 
is clearly redundant.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4572) Develop distributed algebra support for dense and sparse data sets.

2017-01-19 Thread Nikita Ivanov (JIRA)

Nikita Ivanov created IGNITE-4572:
-

 Summary: Develop distributed algebra support for dense and sparse 
data sets.
 Key: IGNITE-4572
 URL: https://issues.apache.org/jira/browse/IGNITE-4572
 Project: Ignite
  Issue Type: Task
  Components: general
Reporter: Nikita Ivanov
Assignee: Nikita Ivanov


This is the base functionality for adding support for future ML capabilities.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] ignite pull request #1444: IGNITE-4106: SQL: parallelize sql queries over ca...

2017-01-19 Thread AMashenkov

GitHub user AMashenkov opened a pull request:

https://github.com/apache/ignite/pull/1444

IGNITE-4106: SQL: parallelize sql queries over cache local partitions

Implemented

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4106

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1444.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1444


commit 6fffa4afb6699fdb54fc5e3ca02797547c4d09ea
Author: Andrey V. Mashenkov 
Date:   2017-01-19T15:24:03Z

Implemented




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1386: IGNITE-4106: SQL: parallelize sql queries over ca...

2017-01-19 Thread AMashenkov

Github user AMashenkov closed the pull request at:

https://github.com/apache/ignite/pull/1386


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: IGNITE-2894 - Binary object inside of Externalizable still serialized with OptimizedMarshaller

2017-01-19 Thread Nikita Amelchev

I have some questions about single Marshaller.
It seems not easy to merge OptimizedMarshaller with BinaryMarshaller and is
there any sense in it?
When Binary object inside Externalizable serialized with optimized it
losing all benefits.
Will OptimizedMarshaller be supported at 2.0 version? Or to merge they is
better?
What do you think about it?

In addition, Vladimir Ozerov, I would like to hear your opinion.

2017-01-17 23:32 GMT+03:00 Denis Magda :

> Someone else added you to the contributors list in JIRA. This is why I
> couldn’t add you for the second time. Ignite committers, please reply on
> the dev list if you add someone to the list.
>
> Nikita, yes, this ticket is still relevant. Go ahead and assign it on
> yourself.
>
> Also please you may want to help with approaching 2.0 release and take
> care of one of the sub-tasks that must be included in 2.0:
> https://issues.apache.org/jira/browse/IGNITE-4547 <
> https://issues.apache.org/jira/browse/IGNITE-4547>
>
> —
> Denis
>
> > On Jan 15, 2017, at 9:02 PM, Nikita Amelchev 
> wrote:
> >
> > This issue was created long ago. Is still relevant?
> >
> > JIRA account:
> > Username: NSAmelchev
> > Full Name: Amelchev Nikita
> >
> >
> > 2017-01-14 1:52 GMT+03:00 Denis Magda :
> >
> >> Hi Nikita,
> >>
> >> I can’t find provided account in Ignite JIRA
> >> https://issues.apache.org/jira/browse/IGNITE <
> https://issues.apache.org/
> >> jira/browse/IGNITE>
> >>
> >> Please create an account there and share with me.
> >>
> >> This information might be useful for you as well.
> >>
> >> Subscribe to both dev and user lists:
> >> https://ignite.apache.org/community/resources.html#mail-lists
> >>
> >> Get familiar with Ignite development process described here:
> >> https://cwiki.apache.org/confluence/display/IGNITE/Development+Process
> >>
> >> Instructions on how to contribute can be found here:
> >> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
> >>
> >> Project setup in Intellij IDEAL
> >> https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup
> >>
> >> Regards,
> >> Denis
> >>
> >>> On Jan 13, 2017, at 1:37 AM, Nikita Amelchev 
> >> wrote:
> >>>
> >>> Hello everyone.
> >>>
> >>> I'd like to take IGNITE-2894. Can you assign to me?
> >>>
> >>> Username: NSAmelchev
> >>>
> >>> --
> >>> Best wishes,
> >>> Amelchev Nikita
> >>
> >>
> >
> >
> > --
> > Best wishes,
> > Amelchev Nikita
>
>


-- 
Best wishes,
Amelchev Nikita

[GitHub] ignite pull request #1443: avoid to run resolveIgniteUrl twice

2017-01-19 Thread wbmark

GitHub user wbmark opened a pull request:

https://github.com/apache/ignite/pull/1443

avoid to run resolveIgniteUrl twice



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/wbmark/ignite master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1443.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1443


commit 90618cc8b55b26004cf455f770e028d95699035b
Author: weibin 
Date:   2017-01-19T15:01:12Z

avoid to run resolveIgniteUrl twice




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Allow distributed SQL query execution over explicit set of partitions

2017-01-19 Thread Alexei Scherbakov

Vladimir, do you have any idea how specific partition settings will work
for queries over replicated caches, when every node will have full data
set. ?

The query will only be parallelized on nodes defined by passed partitions,
or something else ?



2017-01-19 17:27 GMT+03:00 Alexei Scherbakov :

> 1. OK.
>
> 2. Agreed. In future we might split query execution between nodes, but for
> now query is routed to random node in grid,
>
> 3. OK, let's mark getter/setter as deprecated.
>
> 4. Query must be executed locally only for defined partitions. Currently
> this setting is ignored for local queries.
>
> 5. I have the same understanding. Distributed joins will ignore the
> setting.
> This is not implemented yet..
>
>
> 2017-01-19 15:39 GMT+03:00 Sergi Vladykin :
>
>> Agree, lets remove everything related to partition ranges. Looks like
>> unnecessary complication.
>>
>> Sergi
>>
>> 2017-01-19 10:01 GMT+03:00 Vladimir Ozerov :
>>
>> > Several side notes about API.
>> >
>> > 1) I would avoid ranges even in this form.for the sake of simplicity.
>> > Ignite do not have any notion of "partition range" in affinity API, so
>> I do
>> > not understand how users are going to work on ranges unless they have
>> some
>> > very special custom affinity function, which is rather unlikely case.
>> >
>> > 2) The fact that this property is ignored in REPLICATED cache is
>> confusing.
>> > REPLICATED cache still divides partitions into primaries and backups.
>> If I
>> > have very large data set and want to execute some query, I would
>> definitely
>> > expect that Ignite will take advantage of distributed computing and
>> spread
>> > the load between nodes. I understand that currently SQL queries do not
>> work
>> > this way, but this is clear disadvantage for certain scenarios, which we
>> > may improve in future. I would remove this paragraph from docs.
>> >
>> > 3) We already have ScanQuery.partition getter/setter. We need to make
>> sure
>> > that they are "merged" somehow. For instance, we may deprecate two
>> methods
>> > in ScanQuery class, and advise users to use Query.partitions, with
>> > clarification - only single partition is supported for ScanQuery at the
>> > moment.
>> >
>> > 4) What should happen if "partitions" are defined and "local" flag is
>> set?
>> >
>> > As per distributed joins - how are we going to execute them when
>> partitions
>> > are set explicitly? As far as I understand, partitions should apply
>> only to
>> > map step and only for the cache query was created from, This way
>> > distributed join execution should effectively ignore partitions?
>> >
>> > Vladimir.
>> >
>> >
>> > On Thu, Jan 19, 2017 at 1:04 AM, Alexei Scherbakov <
>> > alexey.scherbak...@gmail.com> wrote:
>> >
>> > > I mean distributed joins.
>> > >
>> > > 2017-01-19 0:10 GMT+03:00 Alexei Scherbakov <
>> > alexey.scherbak...@gmail.com>
>> > > :
>> > >
>> > > > Guys,
>> > > >
>> > > > I've finished adding API changes and implemented proper nodes
>> routing.
>> > > >
>> > > > Currently it doesn't work with distributed queries.But I think this
>> > > > feature should be compatible with it.
>> > > >
>> > > > Could anyone take a look at current branch state while I'm looking
>> > deeper
>> > > > into dsitributed queries code?
>> > > >
>> > > > Issue: https://issues.apache.org/jira/browse/IGNITE-4523
>> > > > PR: https://github.com/apache/ignite/pull/1418
>> > > >
>> > > >
>> > > >
>> > > > 2017-01-13 15:55 GMT+03:00 Alexei Scherbakov <
>> > > alexey.scherbak...@gmail.com
>> > > > >:
>> > > >
>> > > >> OK, let's do it this way.
>> > > >>
>> > > >>
>> > > >>
>> > > >>
>> > > >>
>> > > >> 2017-01-13 13:27 GMT+03:00 Sergi Vladykin <
>> sergi.vlady...@gmail.com>:
>> > > >>
>> > > >>> Internally we still use int[] when we send partitions (see
>> > > >>> GridH2QueryRequest.parts). It looks like we only do more work with
>> > > >>> PartitionSet.
>> > > >>>
>> > > >>> I like the idea of bitset for partitions, but
>> > > >>>
>> > > >>> 1. We have to change internals first to use it, otherwise the
>> > > >>> optimization
>> > > >>> makes no sense.
>> > > >>> 2. We will need to have a method SqlQuery.setPartitions(int...
>> parts)
>> > > for
>> > > >>> usability reasons anyways.
>> > > >>>
>> > > >>> Thus I suggest for now to go the straightforward way with int[]
>> and
>> > > >>> create
>> > > >>> a separate ticket describing the optimization with bitset.
>> > > >>>
>> > > >>> Sergi
>> > > >>>
>> > > >>> 2017-01-13 13:06 GMT+03:00 Alexei Scherbakov <
>> > > >>> alexey.scherbak...@gmail.com>:
>> > > >>>
>> > > >>> > PartitionSet hides internal implementation of int array.
>> > > >>> >
>> > > >>> > This allows as to efficiently represent contiguous range of
>> > > partitions
>> > > >>> and
>> > > >>> > defines clear API for ordered iteration over partitions and
>> > > containment
>> > > >>> > check.
>> > > >>> >
>> > > >>> > Even better to go with compressed bitmap, as

Re: Allow distributed SQL query execution over explicit set of partitions

2017-01-19 Thread Alexei Scherbakov

1. OK.

2. Agreed. In future we might split query execution between nodes, but for
now query is routed to random node in grid,

3. OK, let's mark getter/setter as deprecated.

4. Query must be executed locally only for defined partitions. Currently
this setting is ignored for local queries.

5. I have the same understanding. Distributed joins will ignore the setting.
This is not implemented yet..


2017-01-19 15:39 GMT+03:00 Sergi Vladykin :

> Agree, lets remove everything related to partition ranges. Looks like
> unnecessary complication.
>
> Sergi
>
> 2017-01-19 10:01 GMT+03:00 Vladimir Ozerov :
>
> > Several side notes about API.
> >
> > 1) I would avoid ranges even in this form.for the sake of simplicity.
> > Ignite do not have any notion of "partition range" in affinity API, so I
> do
> > not understand how users are going to work on ranges unless they have
> some
> > very special custom affinity function, which is rather unlikely case.
> >
> > 2) The fact that this property is ignored in REPLICATED cache is
> confusing.
> > REPLICATED cache still divides partitions into primaries and backups. If
> I
> > have very large data set and want to execute some query, I would
> definitely
> > expect that Ignite will take advantage of distributed computing and
> spread
> > the load between nodes. I understand that currently SQL queries do not
> work
> > this way, but this is clear disadvantage for certain scenarios, which we
> > may improve in future. I would remove this paragraph from docs.
> >
> > 3) We already have ScanQuery.partition getter/setter. We need to make
> sure
> > that they are "merged" somehow. For instance, we may deprecate two
> methods
> > in ScanQuery class, and advise users to use Query.partitions, with
> > clarification - only single partition is supported for ScanQuery at the
> > moment.
> >
> > 4) What should happen if "partitions" are defined and "local" flag is
> set?
> >
> > As per distributed joins - how are we going to execute them when
> partitions
> > are set explicitly? As far as I understand, partitions should apply only
> to
> > map step and only for the cache query was created from, This way
> > distributed join execution should effectively ignore partitions?
> >
> > Vladimir.
> >
> >
> > On Thu, Jan 19, 2017 at 1:04 AM, Alexei Scherbakov <
> > alexey.scherbak...@gmail.com> wrote:
> >
> > > I mean distributed joins.
> > >
> > > 2017-01-19 0:10 GMT+03:00 Alexei Scherbakov <
> > alexey.scherbak...@gmail.com>
> > > :
> > >
> > > > Guys,
> > > >
> > > > I've finished adding API changes and implemented proper nodes
> routing.
> > > >
> > > > Currently it doesn't work with distributed queries.But I think this
> > > > feature should be compatible with it.
> > > >
> > > > Could anyone take a look at current branch state while I'm looking
> > deeper
> > > > into dsitributed queries code?
> > > >
> > > > Issue: https://issues.apache.org/jira/browse/IGNITE-4523
> > > > PR: https://github.com/apache/ignite/pull/1418
> > > >
> > > >
> > > >
> > > > 2017-01-13 15:55 GMT+03:00 Alexei Scherbakov <
> > > alexey.scherbak...@gmail.com
> > > > >:
> > > >
> > > >> OK, let's do it this way.
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> 2017-01-13 13:27 GMT+03:00 Sergi Vladykin  >:
> > > >>
> > > >>> Internally we still use int[] when we send partitions (see
> > > >>> GridH2QueryRequest.parts). It looks like we only do more work with
> > > >>> PartitionSet.
> > > >>>
> > > >>> I like the idea of bitset for partitions, but
> > > >>>
> > > >>> 1. We have to change internals first to use it, otherwise the
> > > >>> optimization
> > > >>> makes no sense.
> > > >>> 2. We will need to have a method SqlQuery.setPartitions(int...
> parts)
> > > for
> > > >>> usability reasons anyways.
> > > >>>
> > > >>> Thus I suggest for now to go the straightforward way with int[] and
> > > >>> create
> > > >>> a separate ticket describing the optimization with bitset.
> > > >>>
> > > >>> Sergi
> > > >>>
> > > >>> 2017-01-13 13:06 GMT+03:00 Alexei Scherbakov <
> > > >>> alexey.scherbak...@gmail.com>:
> > > >>>
> > > >>> > PartitionSet hides internal implementation of int array.
> > > >>> >
> > > >>> > This allows as to efficiently represent contiguous range of
> > > partitions
> > > >>> and
> > > >>> > defines clear API for ordered iteration over partitions and
> > > containment
> > > >>> > check.
> > > >>> >
> > > >>> > Even better to go with compressed bitmap, as I mentioned in
> ticket
> > > >>> comment.
> > > >>> > This will allow us to minimize heap footprint for this object.
> > > >>> >
> > > >>> > Moreover, it will be useful to create reusable compressed bitmap
> > > >>> > implementation in Ignite and use it in other cases, on example,
> for
> > > >>> > replacing H2's IntArray and Set.
> > > >>> >
> > > >>> > Should I create a ticket for this ?
> > > >>> >
> > > >>> > .
> > > >>> >
> > > >>> > 2017-01-13 1:01 GMT+03:00 Dmitriy

Re: Blogs updates: 1.8 release, Redis, Microservices and more

2017-01-19 Thread 李玉珏


great！


在 2017/1/19 10:24, Denis Magda 写道:

Community,

Some of us published new blog posts this month. Please go ahead and enjoy:
https://ignite.apache.org/blogs.html 

—
Denis

[GitHub] ignite pull request #1442: IGNITE-4475 Simplify async API

2017-01-19 Thread tledkov-gridgain

GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/1442

IGNITE-4475 Simplify async API



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4475

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1442.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1442


commit c893da70a9757b16b0799adc8eaa29fa1b03d06e
Author: tledkov-gridgain 
Date:   2016-12-21T11:54:33Z

IGNITE-4399: IGFS: Merged IgfsSecondaryFileSystem and 
IgfsSecondaryFileSystemV2 interfaces. This closes #1346.

commit c5882a85f4e3a1f61723ac54fd92f087684df6da
Author: devozerov 
Date:   2016-12-26T11:15:42Z

Merge branch 'master' into ignite-2.0

commit 7e73d0223a3f09cbe0b7094a2c04bdf9d63ca9be
Author: devozerov 
Date:   2016-12-28T09:54:47Z

Merge branch 'master' into ignite-2.0

commit 7d82d6a06b5e9f1f8cd2909b865e37d46b8da03f
Author: devozerov 
Date:   2016-12-28T09:58:11Z

IGNITE-3875: Introduced separate thread pool for data streamer. This closes 
#1173. This closes #1383.

commit a61b0eaff1817d84c0659e8a7e095f29e22800e1
Author: tledkov-gridgain 
Date:   2016-12-28T11:09:38Z

IGNITE-4405: Hadoop: implemented "readLine" method for HadoopDataInStream 
and HadoopDirectDataInput classes. This closes #1358.

commit 2df39a80d80e2575be61a902ccd48615796fcde9
Author: tledkov-gridgain 
Date:   2016-12-28T13:47:24Z

IGNITE-3961: IGFS: Added IgfsSecondaryFileSystem.affintiy() method. This 
closes #1114. This closes #1252.

commit 2e691d80ea4870c3e7b5b127792b66c920f72c39
Author: tledkov-gridgain 
Date:   2016-12-29T08:00:01Z

IGNITE-4462: IGFS: removed grid name from HadoopIgfsEndpoint. This closes 
#1368.

commit a9b1fc2b3840d47d7c978d9296e8ae6bdeb10be5
Author: tledkov-gridgain 
Date:   2016-12-29T08:07:22Z

IGNITE-4459: Hadoop: weighted planned is default one from now on. This 
closes #1391.

commit 1f743465d6875ef48b1835d03a78a0dbaf339bf6
Author: tledkov-gridgain 
Date:   2016-12-29T08:14:10Z

IGNITE-4458: Hadoop: "striped" shuffle mode is default from now on. This 
closes #1390.

commit 6090ebdfcd0ea3840b0d32cb10197b43615e1e89
Author: devozerov 
Date:   2017-01-05T09:23:06Z

Merge branch 'master' into ignite-2.0

commit 77ca2e636c73e464f833f227c4894df0785ae9e2
Author: devozerov 
Date:   2017-01-16T13:07:49Z

Merge branch 'master' into ignite-2.0

commit d14e0727b3dd61ab5ec2957133d77dbc25e9ba68
Author: tledkov-gridgain 
Date:   2017-01-16T13:36:25Z

IGNITE-4428: Hadoop: moved HadoopMapReducePlanner and dependent classes to 
public space. This closes #1389. This closes #1394.

commit f1365421c299b754a10edf8b6f156aeeb5ff0ce1
Author: tledkov-gridgain 
Date:   2017-01-16T13:57:27Z

IGNITE-4503: Hadoop: added boundary checks to HadoopDirectDataInput. This 
closes # 1416.

commit e2ef47553c8b351722c92ce62d6ca9b0b70b96d1
Author: tledkov-gridgain 
Date:   2017-01-19T13:42:38Z

IGNITE-4475: change async API at the IgniteCompute, change examples, add 
tests




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-4570) Handle CREATE INDEX statements

2017-01-19 Thread Alexander Paschenko (JIRA)

Alexander Paschenko created IGNITE-4570:
---

 Summary: Handle CREATE INDEX statements
 Key: IGNITE-4570
 URL: https://issues.apache.org/jira/browse/IGNITE-4570
 Project: Ignite
  Issue Type: Sub-task
  Components: SQL
Reporter: Alexander Paschenko
Assignee: Alexander Paschenko
 Fix For: 1.9


Parse and execute CREATE INDEX boiling down to IgniteCacheEx.createQueryIndex 
introduced by IGNITE-4566



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4569) Create local portion of index w/table locking

2017-01-19 Thread Alexander Paschenko (JIRA)

Alexander Paschenko created IGNITE-4569:
---

 Summary: Create local portion of index w/table locking
 Key: IGNITE-4569
 URL: https://issues.apache.org/jira/browse/IGNITE-4569
 Project: Ignite
  Issue Type: Sub-task
  Components: cache, SQL
Reporter: Alexander Paschenko
Assignee: Alexander Paschenko
 Fix For: 1.9


Actual index data modification - initially will render whole SQL table 
inaccessible during initial index buildup



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4568) Dynamically create distributed index

2017-01-19 Thread Alexander Paschenko (JIRA)

Alexander Paschenko created IGNITE-4568:
---

 Summary: Dynamically create distributed index
 Key: IGNITE-4568
 URL: https://issues.apache.org/jira/browse/IGNITE-4568
 Project: Ignite
  Issue Type: Sub-task
  Components: messaging, SQL
Reporter: Alexander Paschenko
Assignee: Alexander Paschenko
 Fix For: 1.9


Handle index creation messaging using means introduced by IGNITE-4567



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4567) Design and implement means for distributed DDL statements

2017-01-19 Thread Alexander Paschenko (JIRA)

Alexander Paschenko created IGNITE-4567:
---

 Summary: Design and implement means for distributed DDL statements
 Key: IGNITE-4567
 URL: https://issues.apache.org/jira/browse/IGNITE-4567
 Project: Ignite
  Issue Type: Sub-task
  Components: messaging, SQL
Reporter: Alexander Paschenko
Assignee: Alexander Paschenko


We need some kind of common way to handle stuff like CREATE INDEX/CREATE TABLE 
statements - in other words, run potentially long-running (and possibly cache 
locking) DDL operations on cluster nodes that would allow using new 
tables/indexes only when all nodes have performed requested operations and data 
modifications.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Allow distributed SQL query execution over explicit set of partitions

2017-01-19 Thread Sergi Vladykin

Agree, lets remove everything related to partition ranges. Looks like
unnecessary complication.

Sergi

2017-01-19 10:01 GMT+03:00 Vladimir Ozerov :

> Several side notes about API.
>
> 1) I would avoid ranges even in this form.for the sake of simplicity.
> Ignite do not have any notion of "partition range" in affinity API, so I do
> not understand how users are going to work on ranges unless they have some
> very special custom affinity function, which is rather unlikely case.
>
> 2) The fact that this property is ignored in REPLICATED cache is confusing.
> REPLICATED cache still divides partitions into primaries and backups. If I
> have very large data set and want to execute some query, I would definitely
> expect that Ignite will take advantage of distributed computing and spread
> the load between nodes. I understand that currently SQL queries do not work
> this way, but this is clear disadvantage for certain scenarios, which we
> may improve in future. I would remove this paragraph from docs.
>
> 3) We already have ScanQuery.partition getter/setter. We need to make sure
> that they are "merged" somehow. For instance, we may deprecate two methods
> in ScanQuery class, and advise users to use Query.partitions, with
> clarification - only single partition is supported for ScanQuery at the
> moment.
>
> 4) What should happen if "partitions" are defined and "local" flag is set?
>
> As per distributed joins - how are we going to execute them when partitions
> are set explicitly? As far as I understand, partitions should apply only to
> map step and only for the cache query was created from, This way
> distributed join execution should effectively ignore partitions?
>
> Vladimir.
>
>
> On Thu, Jan 19, 2017 at 1:04 AM, Alexei Scherbakov <
> alexey.scherbak...@gmail.com> wrote:
>
> > I mean distributed joins.
> >
> > 2017-01-19 0:10 GMT+03:00 Alexei Scherbakov <
> alexey.scherbak...@gmail.com>
> > :
> >
> > > Guys,
> > >
> > > I've finished adding API changes and implemented proper nodes routing.
> > >
> > > Currently it doesn't work with distributed queries.But I think this
> > > feature should be compatible with it.
> > >
> > > Could anyone take a look at current branch state while I'm looking
> deeper
> > > into dsitributed queries code?
> > >
> > > Issue: https://issues.apache.org/jira/browse/IGNITE-4523
> > > PR: https://github.com/apache/ignite/pull/1418
> > >
> > >
> > >
> > > 2017-01-13 15:55 GMT+03:00 Alexei Scherbakov <
> > alexey.scherbak...@gmail.com
> > > >:
> > >
> > >> OK, let's do it this way.
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> 2017-01-13 13:27 GMT+03:00 Sergi Vladykin :
> > >>
> > >>> Internally we still use int[] when we send partitions (see
> > >>> GridH2QueryRequest.parts). It looks like we only do more work with
> > >>> PartitionSet.
> > >>>
> > >>> I like the idea of bitset for partitions, but
> > >>>
> > >>> 1. We have to change internals first to use it, otherwise the
> > >>> optimization
> > >>> makes no sense.
> > >>> 2. We will need to have a method SqlQuery.setPartitions(int... parts)
> > for
> > >>> usability reasons anyways.
> > >>>
> > >>> Thus I suggest for now to go the straightforward way with int[] and
> > >>> create
> > >>> a separate ticket describing the optimization with bitset.
> > >>>
> > >>> Sergi
> > >>>
> > >>> 2017-01-13 13:06 GMT+03:00 Alexei Scherbakov <
> > >>> alexey.scherbak...@gmail.com>:
> > >>>
> > >>> > PartitionSet hides internal implementation of int array.
> > >>> >
> > >>> > This allows as to efficiently represent contiguous range of
> > partitions
> > >>> and
> > >>> > defines clear API for ordered iteration over partitions and
> > containment
> > >>> > check.
> > >>> >
> > >>> > Even better to go with compressed bitmap, as I mentioned in ticket
> > >>> comment.
> > >>> > This will allow us to minimize heap footprint for this object.
> > >>> >
> > >>> > Moreover, it will be useful to create reusable compressed bitmap
> > >>> > implementation in Ignite and use it in other cases, on example, for
> > >>> > replacing H2's IntArray and Set.
> > >>> >
> > >>> > Should I create a ticket for this ?
> > >>> >
> > >>> > .
> > >>> >
> > >>> > 2017-01-13 1:01 GMT+03:00 Dmitriy Setrakyan  >:
> > >>> >
> > >>> > > On Thu, Jan 12, 2017 at 6:12 AM, Sergi Vladykin <
> > >>> > sergi.vlady...@gmail.com>
> > >>> > > wrote:
> > >>> > >
> > >>> > > > I looked at the code. The PartitionSet concept looks
> > >>> overengineered to
> > >>> > > me,
> > >>> > > > why wouldn't we just go with int[]?
> > >>> > > >
> > >>> > >
> > >>> > > Agree.
> > >>> > >
> > >>> > >
> > >>> > > >
> > >>> > > > Sergi
> > >>> > > >
> > >>> > > > 2017-01-12 15:18 GMT+03:00 Alexei Scherbakov <
> > >>> > > alexey.scherbak...@gmail.com
> > >>> > > > >:
> > >>> > > >
> > >>> > > > > Done.
> > >>> > > > >
> > >>> > > > > 2017-01-11 20:39 GMT+03:00 Dmitriy Setrakyan <
> > >>> dsetrak...@apache.org
> > >>> > >:
> > >>> > > > >

Re: Sort nodes in the ring in order to minimize the number of reconnections

2017-01-19 Thread Yakov Zhdanov

Alexander, as far as I remember we talked about having cluster id set via
TcpDiscoverySpi configuration and also via system property.

What do you mean by "existence of sufficient comparator"? If we require
that attribute is integer then we don't have any problems. If it is not
integer then you should throw exception on start.

I repeat this once again - we need to (1)prevent our users from falling
into terrible discovery issues (which are hard to debug) if users provide
inconsistent comparators on different nodes, but we also need to have
(2)full flexibility and control over nodes ordering. I think if we don't
have comparator on public API then we are still OK with both points above.
Therefore, I would like you to implement this feature using the approach we
agreed on before. Let me know if you want me to take a look at the code
before you proceed.

--Yakov

[jira] [Created] (IGNITE-4566) Introduce IgniteCacheEx.createQueryIndex

2017-01-19 Thread Alexander Paschenko (JIRA)

Alexander Paschenko created IGNITE-4566:
---

 Summary: Introduce IgniteCacheEx.createQueryIndex
 Key: IGNITE-4566
 URL: https://issues.apache.org/jira/browse/IGNITE-4566
 Project: Ignite
  Issue Type: Sub-task
  Components: cache, SQL
Reporter: Alexander Paschenko
Assignee: Alexander Paschenko
 Fix For: 1.9






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4565) Support CREATE INDEX DDL statements

2017-01-19 Thread Alexander Paschenko (JIRA)

Alexander Paschenko created IGNITE-4565:
---

 Summary: Support CREATE INDEX DDL statements
 Key: IGNITE-4565
 URL: https://issues.apache.org/jira/browse/IGNITE-4565
 Project: Ignite
  Issue Type: New Feature
  Components: SQL
Reporter: Alexander Paschenko
Assignee: Alexander Paschenko
 Fix For: 1.9






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4564) Ensure that builder approach is used for all setters in public API

2017-01-19 Thread Vladimir Ozerov (JIRA)

Vladimir Ozerov created IGNITE-4564:
---

 Summary: Ensure that builder approach is used for all setters in 
public API
 Key: IGNITE-4564
 URL: https://issues.apache.org/jira/browse/IGNITE-4564
 Project: Ignite
  Issue Type: Task
  Components: general
Affects Versions: 1.8
Reporter: Vladimir Ozerov
 Fix For: 2.0


*Problem*
We employed "builder" approach for some configuration classes:
{code}
class Configuration {
Configuration setSomething(Something);
}
{code}

This is very convenient for users. However, only part of our configs employ 
this approach.

*Task*
Let's make sure that all other parts of our API follow this rule.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: IGNITE-4157 improvement patch available

2017-01-19 Thread Alexey Goncharuk

Sergey, thanks for the careful work! I've merged your contribution to
ignite-2.0 branch.

On Dec 16, 2016 11:50, "Sergey Chugunov"  wrote:

> Folks,
>
> I finished development on IGNITE-4157
>  *Use discovery custom
> messages instead of marshaller cache.*
>
> Latest CI build
>  tab=buildResultsDiv=IgniteTests_RunAll>
> in team city looks very good, there are no tests failures related to my
> refactoring (only flaky tests are broken, this set varies from build to
> build) nor build hangs.
>
> I believe the code is stable enough and ready to be reviewed and merged to
> ignite-2.0 branch. Please take a look at pull request pull/1271
> .
>
> Thanks,
> Sergey.
>

[GitHub] ignite pull request #1422: IGNITE-3837: ODBC: Support for CONVERT function e...

2017-01-19 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1422


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1054: IGNITE-3376 IGFS: Allow direct PROXY mode invocat...

2017-01-19 Thread tledkov-gridgain

Github user tledkov-gridgain closed the pull request at:

https://github.com/apache/ignite/pull/1054


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1358: IGNITE-4405: Hadoop: support "readLine" method in...

2017-01-19 Thread tledkov-gridgain

Github user tledkov-gridgain closed the pull request at:

https://github.com/apache/ignite/pull/1358


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-4563) .NET: ICache.LoadCache fails on non-primitive arguments

2017-01-19 Thread Pavel Tupitsyn (JIRA)

Pavel Tupitsyn created IGNITE-4563:
--

 Summary: .NET: ICache.LoadCache fails on non-primitive arguments
 Key: IGNITE-4563
 URL: https://issues.apache.org/jira/browse/IGNITE-4563
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Affects Versions: 1.8, 1.7, 1.6, 1.5.0.final
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
Priority: Critical
 Fix For: 1.9


{{ICache.LoadCache}} arguments are unnecessarily deserialized on Java side (see 
{{readObjectArray}} call in {{PlatformCache.loadCache0}}). So any arguments 
that are not primitives or primitive collections cause an exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4562) .NET: Add BinaryObjectException

2017-01-19 Thread Pavel Tupitsyn (JIRA)

Pavel Tupitsyn created IGNITE-4562:
--

 Summary: .NET: Add BinaryObjectException
 Key: IGNITE-4562
 URL: https://issues.apache.org/jira/browse/IGNITE-4562
 Project: Ignite
  Issue Type: Improvement
  Components: platforms
Reporter: Pavel Tupitsyn
Priority: Trivial
 Fix For: 2.1


Add mapping for Java BinaryObjectException in .NET. This exception can occur 
due to misconfiguration or deployment issues, we should map it to a .NET class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

42 matches

Mail list logo