[jira] [Created] (IGNITE-1766) Zooming of charts

[Pavel Konstantinov (JIRA)]

Pavel Konstantinov created IGNITE-1766:
--

 Summary: Zooming of charts
 Key: IGNITE-1766
 URL: https://issues.apache.org/jira/browse/IGNITE-1766
 Project: Ignite
  Issue Type: Sub-task
Reporter: Pavel Konstantinov


Currently the minimum results page size is 50, but in case of several values on 
Y-axis viewing for example bar chart becomes uncomfortable



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Ignite Web Console - Loading metadata from MySQL

[Dmitriy Setrakyan]

On Wed, Oct 21, 2015 at 5:08 PM, Prachi Garg  wrote:

> I am trying to load metadata from MySQL database, but the web console
> doesn't allow me to provide the MySQL driver JAR. It is set to H2, by
> default, and doesn't let me change. I tried running the Ignite web agent in
> both test and non-test modes. How can I provide the MySQL driver JAR?
>

The question (?) helper popup next to the driver field actually explains
what to do. You need to copy the driver JAR into the “jdbc-drivers” folder
in the agent.


> Thanks,
> -Prachi
>

[jira] [Created] (IGNITE-1768) .Net: Review portable configuration on Java side.

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-1768:
---

 Summary: .Net: Review portable configuration on Java side.
 Key: IGNITE-1768
 URL: https://issues.apache.org/jira/browse/IGNITE-1768
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: ignite-1.4
Reporter: Vladimir Ozerov
Priority: Critical
 Fix For: 1.5


I am concerned why for some objects we pass assembly + type, while for others 
type only.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-1764) Insufficient width for label on Y-axis in case of large values

[Pavel Konstantinov (JIRA)]

Pavel Konstantinov created IGNITE-1764:
--

 Summary: Insufficient width for label on Y-axis in case of large 
values
 Key: IGNITE-1764
 URL: https://issues.apache.org/jira/browse/IGNITE-1764
 Project: Ignite
  Issue Type: Sub-task
Reporter: Pavel Konstantinov
 Attachments: ig-1764.png

Please see attachment



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: TC patch validation

[Konstantin Boudnik]

On Thu, Oct 22, 2015 at 01:28PM, Artem Shutak wrote:
> Yes, automatic runs were disabled for patches too.
> 
> To run patches you can use "Run All for patch" with specified jira number
> at "Paramethers / Jira number*".

Right, that presumes that a user has an account. I am sure I have one, but I
lost the password, because of the changes of the laptop and whatnot. There's
no obvious name to reset it though; Artem could you help me to reset it and
drop me a password separately or via skype?

Thank you very much in advance!
  Cos

> 
> -- Artem --
> 
> On Thu, Oct 22, 2015 at 4:11 AM, Dmitriy Setrakyan 
> wrote:
> 
> > I think Cos’ question also was why the patches attached to Jira do not get
> > automatically run on TC. To my knowledge, the automatic runs were disabled,
> > but I am not sure if it includes patches as well.
> >
> > Can someone clarify how to run patches on TC?
> >
> > D.
> >
> > On Fri, Oct 16, 2015 at 2:16 AM, Ivan Veselovskiy <
> > iveselovs...@gridgain.com
> > > wrote:
> >
> > > Hi, Konstantin,
> > > Tests are grouped in suites by subsystems. You can run particular test
> > > suite -- e.g. "Ignite Hadoop".
> > > To run the tests with particular changes push the "Run..." button near
> > that
> > > test suite and chose "pull//head" in "Build
> > > branch" field on the "Changes" tab.
> > >
> > > On Fri, Oct 16, 2015 at 1:01 AM, Konstantin Boudnik 
> > > wrote:
> > >
> > > > Gents,
> > > >
> > > > is TC still picking up the JIRAs in PA state? Somehow it seems that
> > only
> > > > the
> > > > latest changes from master are being picked.
> > > >
> > > > On a somewhat similar note: can I run tests for just a single module?
> > > E.g.
> > > > in
> > > > IGNITE-1701 I made some changes in ignite-spark pom.xml and want to
> > > quickly
> > > > verify that this doesn't affect anything (which it shouldn't, clearly).
> > > > Running the whole testset on my tiny laptop will probably take a few
> > > hours.
> > > >
> > > > Appreciate any pointers, thanks in advance!
> > > >   Cos
> > > >
> > > >
> > >
> >


signature.asc
Description: Digital signature

Re: MQTT Streamer Code Style and Javadoc fixes (IGNITE-1747)

[Dmitriy Setrakyan]

Raul,

Thanks for your contribution!

I want to address the coding guidelines comments, as they keep coming up
again and again. I do agree that coding guidelines may not be ideal, and
may seem redundant at times. However, they make Ignite code look
consistent. If you try to change them now, even if it seems in a better
direction, the overall impact on the code will be negative, as the old code
will remain inconsistent with new one.

I am also noticing that every discussion about coding guidelines is about a
matter of taste. We will never all agree on the taste, so my preference
would be just to follow the guidelines as they are.

I also think that we should relax the coding guidelines for the
contributors (non-committers) as long as they get fixed by the committer
who does the final merge. This will make contributions easier (I will start
a separate thread about it).

D.

On Thu, Oct 22, 2015 at 12:22 PM, Raul Kripalani  wrote:

> Hi Yakov,
>
> Thanks for the review.
>
> Comments inline.
>
> Aside from them: being honest, do you think Javadoc like the following
> snippets are useful and provide any value? More so in a test case...
>
> Snippet #1:
>
> /**
>  * @param topics Topics.
>  * @param fromIdx From index.
>  * @param cnt Count.
>  * @param singleMsg Single message flag.
>  * @throws MqttException If failed.
>  */
>
> Snippet #2:
>
> /**
>  * @throws Exception If failed.
>  */
>
> To me, they are redundant. And I'm concerned about imposing superfluous
> Javadoc "just because", or due to aesthetics, rather than value.
>
> Regards,
>
> *Raúl Kripalani*
> PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
> Messaging Engineer
> http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
> http://blog.raulkr.net | twitter: @raulvk
>
> On Thu, Oct 22, 2015 at 11:52 AM, Yakov Zhdanov 
> wrote:
>
> > Hi Raul!
> >
> > Thank you very much for addressing my comments!
> >
> > I like the code however there are still a couple of points (I committed
> > everything to ignite-1747):
> >
> > 1. Please take a look at
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines#CodingGuidelines-StringOutput
>
>
> Yeah, I'm familiar with this rule and I thought I was complying with it.
> Could you give me an example where the code is not compliant?
>
>
> > 2. org.apache.ignite.stream.mqtt.MqttStreamer#connected - volatile write
> > then read of most likely the same value - I changed it to return true
> > literal.
> > connected = true;
> > return connected;
> >
>
> Agree.
>
>
> > 3. blockUntilConnected - I would name this property syncConn
> >
>
> Why? To me "blockUntilConnected" is clearer and more explicit than
> "syncConn", which sounds mystic to an ordinary user who's not familiar with
> the community's abbreviation table. It's ok for a private member, but not
> for a property. In fact, syncConn requires the user to have the Javadoc
> handy, blockUntilConnected doesn't. So I prefer the latter.
>
>
> > 4. Abbreviation rules violations - connected - conn, executor - exec,
> > values - vals, count - cnt, message - msg
> >
>
> Ok, this is a rule... yeah. But you can imagine it's extremely hard to
> memorise someone else's glossary. In my opinion, abbreviations should be
> used only when needed, not by default. An abbreviation subtracts
> readability for the general user – long-term Ignite developers are probably
> used to these by now.
>
>
> > 5. What is the point of "connected" member? Client has "isConnected()"?
> Is
> > it similar to what you want to achieve?
> >
>
> I'll look at the implementation of MqttClient.isConnected(), but off the
> top of my head, it seems it could be replaced, yes.
>
>
> > 6. Race on stop - 1 thread calls stop and successfully exits the method,
> > retrier is not stopped (awaitTermination has not been called - should it
> > be?) and can connect client back - is this the case?
> >
>
> I'll have a look.
>
>
> > 7. I think we use {@code } instead of ...
> >
>
> Correct. Thanks.
>
> On more point community should agree on  - imports ordering and grouping. I
> > will post another email.
> >
>
> This was already added a few weeks ago in the Coding rules.
>
>
> >
> > --Yakov
> >
> > 2015-10-20 20:39 GMT+03:00 Raul Kripalani :
> >
> > > Hi Yakov,
> > >
> > > I've had a few free cycles to fix the code style and Javadoc issues you
> > > reported in the MQTT Streamer and I've pushed my changes to branch
> > > ignite-1747.
> > >
> > > Would you mind having a look to see if it adapts better now?
> > >
> > > Thanks,
> > >
> > > *Raúl Kripalani*
> > > PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data
> and
> > > Messaging Engineer
> > > http://about.me/raulkripalani |
> http://www.linkedin.com/in/raulkripalani
> > > http://blog.raulkr.net | twitter: @raulvk
> > >
> >
>

[jira] [Created] (IGNITE-1778) IGFS: implement rollback procedure: cleanup the "reserved" data.

[Ivan Veselovsky (JIRA)]

Ivan Veselovsky created IGNITE-1778:
---

 Summary: IGFS: implement rollback procedure: cleanup the 
"reserved" data.
 Key: IGNITE-1778
 URL: https://issues.apache.org/jira/browse/IGNITE-1778
 Project: Ignite
  Issue Type: Sub-task
Reporter: Ivan Veselovsky
Assignee: Ivan Veselovsky


The following procedure is applied if the file is locked:
1) take Node id from the lock Id.
2) see via discovery service if this node is alive.
3) if yes, return (we cannot lock the file).
4) if not: do a rollback:
 -  delete all the blocks in "reserved" range from the data cache.
 -  set reserved range to zero.
 -  remove the lock from the FileInfo. 

The above procedure should be performed upon every attempt to take a lock, and 
(may be) periodically while traversing the file system.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: IGNITE-1371 patch is ready

[Artem Shutak]

Hi Igor,

Please, see my comments in the Jira.

-- Artem --

On Wed, Oct 21, 2015 at 3:50 AM, Igor Rudyak  wrote:

> Hi guys,
>
> I attached patch for IGNITE-1371 on a previous week, but looks like it
> wasn't taken into consideration.
>
> I am assuming that may be I forgot to do some extra steps with the Jira
> ticket. As far as I am rather new to Ignite Jira process, could somebody
> help me with this?
>
>
> Thanks,
> Igor Rudyak
>

Re: MQTT Streamer Code Style and Javadoc fixes (IGNITE-1747)

[Raul Kripalani]

Hi Yakov,

Thanks for the review.

Comments inline.

Aside from them: being honest, do you think Javadoc like the following
snippets are useful and provide any value? More so in a test case...

Snippet #1:

/**
 * @param topics Topics.
 * @param fromIdx From index.
 * @param cnt Count.
 * @param singleMsg Single message flag.
 * @throws MqttException If failed.
 */

Snippet #2:

/**
 * @throws Exception If failed.
 */

To me, they are redundant. And I'm concerned about imposing superfluous
Javadoc "just because", or due to aesthetics, rather than value.

Regards,

*Raúl Kripalani*
PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
Messaging Engineer
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
http://blog.raulkr.net | twitter: @raulvk

On Thu, Oct 22, 2015 at 11:52 AM, Yakov Zhdanov  wrote:

> Hi Raul!
>
> Thank you very much for addressing my comments!
>
> I like the code however there are still a couple of points (I committed
> everything to ignite-1747):
>
> 1. Please take a look at
>
> https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines#CodingGuidelines-StringOutput


Yeah, I'm familiar with this rule and I thought I was complying with it.
Could you give me an example where the code is not compliant?


> 2. org.apache.ignite.stream.mqtt.MqttStreamer#connected - volatile write
> then read of most likely the same value - I changed it to return true
> literal.
> connected = true;
> return connected;
>

Agree.


> 3. blockUntilConnected - I would name this property syncConn
>

Why? To me "blockUntilConnected" is clearer and more explicit than
"syncConn", which sounds mystic to an ordinary user who's not familiar with
the community's abbreviation table. It's ok for a private member, but not
for a property. In fact, syncConn requires the user to have the Javadoc
handy, blockUntilConnected doesn't. So I prefer the latter.


> 4. Abbreviation rules violations - connected - conn, executor - exec,
> values - vals, count - cnt, message - msg
>

Ok, this is a rule... yeah. But you can imagine it's extremely hard to
memorise someone else's glossary. In my opinion, abbreviations should be
used only when needed, not by default. An abbreviation subtracts
readability for the general user – long-term Ignite developers are probably
used to these by now.


> 5. What is the point of "connected" member? Client has "isConnected()"? Is
> it similar to what you want to achieve?
>

I'll look at the implementation of MqttClient.isConnected(), but off the
top of my head, it seems it could be replaced, yes.


> 6. Race on stop - 1 thread calls stop and successfully exits the method,
> retrier is not stopped (awaitTermination has not been called - should it
> be?) and can connect client back - is this the case?
>

I'll have a look.


> 7. I think we use {@code } instead of ...
>

Correct. Thanks.

On more point community should agree on  - imports ordering and grouping. I
> will post another email.
>

This was already added a few weeks ago in the Coding rules.


>
> --Yakov
>
> 2015-10-20 20:39 GMT+03:00 Raul Kripalani :
>
> > Hi Yakov,
> >
> > I've had a few free cycles to fix the code style and Javadoc issues you
> > reported in the MQTT Streamer and I've pushed my changes to branch
> > ignite-1747.
> >
> > Would you mind having a look to see if it adapts better now?
> >
> > Thanks,
> >
> > *Raúl Kripalani*
> > PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
> > Messaging Engineer
> > http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
> > http://blog.raulkr.net | twitter: @raulvk
> >
>

Re: MQTT Streamer Code Style and Javadoc fixes (IGNITE-1747)

[Denis Magda]

Raul,

It wasn't easy for me to get into the way of following Ignite coding 
guidelines. So you're not alone :)


However one day I found "ignite/idea/ignite_codeStyle.xml" file and set 
it in Intellij IDEA settings. Since that time coding guidelines has 
never worried me again. Probably this will help you as well.



--
Denis

On 10/22/2015 10:50 PM, Dmitriy Setrakyan wrote:

Raul,

Thanks for your contribution!

I want to address the coding guidelines comments, as they keep coming up
again and again. I do agree that coding guidelines may not be ideal, and
may seem redundant at times. However, they make Ignite code look
consistent. If you try to change them now, even if it seems in a better
direction, the overall impact on the code will be negative, as the old code
will remain inconsistent with new one.

I am also noticing that every discussion about coding guidelines is about a
matter of taste. We will never all agree on the taste, so my preference
would be just to follow the guidelines as they are.

I also think that we should relax the coding guidelines for the
contributors (non-committers) as long as they get fixed by the committer
who does the final merge. This will make contributions easier (I will start
a separate thread about it).

D.

On Thu, Oct 22, 2015 at 12:22 PM, Raul Kripalani  wrote:


Hi Yakov,

Thanks for the review.

Comments inline.

Aside from them: being honest, do you think Javadoc like the following
snippets are useful and provide any value? More so in a test case...

Snippet #1:

 /**
  * @param topics Topics.
  * @param fromIdx From index.
  * @param cnt Count.
  * @param singleMsg Single message flag.
  * @throws MqttException If failed.
  */

Snippet #2:

 /**
  * @throws Exception If failed.
  */

To me, they are redundant. And I'm concerned about imposing superfluous
Javadoc "just because", or due to aesthetics, rather than value.

Regards,

*Raúl Kripalani*
PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
Messaging Engineer
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
http://blog.raulkr.net | twitter: @raulvk

On Thu, Oct 22, 2015 at 11:52 AM, Yakov Zhdanov 
wrote:


Hi Raul!

Thank you very much for addressing my comments!

I like the code however there are still a couple of points (I committed
everything to ignite-1747):

1. Please take a look at



https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines#CodingGuidelines-StringOutput


Yeah, I'm familiar with this rule and I thought I was complying with it.
Could you give me an example where the code is not compliant?



2. org.apache.ignite.stream.mqtt.MqttStreamer#connected - volatile write
then read of most likely the same value - I changed it to return true
literal.
connected = true;
return connected;


Agree.



3. blockUntilConnected - I would name this property syncConn


Why? To me "blockUntilConnected" is clearer and more explicit than
"syncConn", which sounds mystic to an ordinary user who's not familiar with
the community's abbreviation table. It's ok for a private member, but not
for a property. In fact, syncConn requires the user to have the Javadoc
handy, blockUntilConnected doesn't. So I prefer the latter.



4. Abbreviation rules violations - connected - conn, executor - exec,
values - vals, count - cnt, message - msg


Ok, this is a rule... yeah. But you can imagine it's extremely hard to
memorise someone else's glossary. In my opinion, abbreviations should be
used only when needed, not by default. An abbreviation subtracts
readability for the general user – long-term Ignite developers are probably
used to these by now.



5. What is the point of "connected" member? Client has "isConnected()"?

Is

it similar to what you want to achieve?


I'll look at the implementation of MqttClient.isConnected(), but off the
top of my head, it seems it could be replaced, yes.



6. Race on stop - 1 thread calls stop and successfully exits the method,
retrier is not stopped (awaitTermination has not been called - should it
be?) and can connect client back - is this the case?


I'll have a look.



7. I think we use {@code } instead of ...


Correct. Thanks.

On more point community should agree on  - imports ordering and grouping. I

will post another email.


This was already added a few weeks ago in the Coding rules.



--Yakov

2015-10-20 20:39 GMT+03:00 Raul Kripalani :


Hi Yakov,

I've had a few free cycles to fix the code style and Javadoc issues you
reported in the MQTT Streamer and I've pushed my changes to branch
ignite-1747.

Would you mind having a look to see if it adapts better now?

Thanks,

*Raúl Kripalani*
PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data

and

Messaging Engineer
http://about.me/raulkripalani |

http://www.linkedin.com/in/raulkripalani

http://blog.raulkr.net | twitter: @raulvk

[jira] [Created] (IGNITE-1777) IGFS: write files with fail-safe logic: "lock" -> "reserve space" -> "write" -> "size update, unlock"

[Ivan Veselovsky (JIRA)]

Ivan Veselovsky created IGNITE-1777:
---

 Summary: IGFS: write files with fail-safe logic: "lock" -> 
"reserve space" -> "write" -> "size update, unlock"
 Key: IGNITE-1777
 URL: https://issues.apache.org/jira/browse/IGNITE-1777
 Project: Ignite
  Issue Type: Sub-task
Reporter: Ivan Veselovsky
Assignee: Ivan Veselovsky






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Ignite Web Console - Loading metadata from MySQL

[Dmitriy Setrakyan]

On Thu, Oct 22, 2015 at 11:06 AM, Prachi Garg  wrote:

> Thanks!
>
> I think we should also provide an option on the client side for the user to
> browse to the local folder of his system.
>

I really like this idea. I think it fits into the current architecture
perfectly.


>
> -P
>
> On Thu, Oct 22, 2015 at 12:39 AM, Dmitriy Setrakyan  >
> wrote:
>
> > On Wed, Oct 21, 2015 at 5:08 PM, Prachi Garg  wrote:
> >
> > > I am trying to load metadata from MySQL database, but the web console
> > > doesn't allow me to provide the MySQL driver JAR. It is set to H2, by
> > > default, and doesn't let me change. I tried running the Ignite web
> agent
> > in
> > > both test and non-test modes. How can I provide the MySQL driver JAR?
> > >
> >
> > The question (?) helper popup next to the driver field actually explains
> > what to do. You need to copy the driver JAR into the “jdbc-drivers”
> folder
> > in the agent.
> >
> >
> > > Thanks,
> > > -Prachi
> > >
> >
>

Re: MQTT Streamer Code Style and Javadoc fixes (IGNITE-1747)

[Raul Kripalani]

Hey Denis,

Yep, I've been using the Ignite code style for IntelliJ since the
beginning.

It's ok for indentation, whitespacing, line breaks, etc. But things too
specific like abbreviations, string outputs, Javadoc conventions like using
{@code ...}, etc. are not covered.

checkstyle is a potential solution to automate the audit. We'll definitely
need to implement custom checks, but it's worth it. Earlier I saw a comment
in a ticket from another user asking for a Sonar-like report of their
mistakes... So at least I'm not alone in my frustration!

Regards,

*Raúl Kripalani*
PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
Messaging Engineer
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
http://blog.raulkr.net | twitter: @raulvk

On Thu, Oct 22, 2015 at 8:59 PM, Denis Magda  wrote:

> Raul,
>
> It wasn't easy for me to get into the way of following Ignite coding
> guidelines. So you're not alone :)
>
> However one day I found "ignite/idea/ignite_codeStyle.xml" file and set it
> in Intellij IDEA settings. Since that time coding guidelines has never
> worried me again. Probably this will help you as well.
>
>
> --
> Denis
>
>
> On 10/22/2015 10:50 PM, Dmitriy Setrakyan wrote:
>
>> Raul,
>>
>> Thanks for your contribution!
>>
>> I want to address the coding guidelines comments, as they keep coming up
>> again and again. I do agree that coding guidelines may not be ideal, and
>> may seem redundant at times. However, they make Ignite code look
>> consistent. If you try to change them now, even if it seems in a better
>> direction, the overall impact on the code will be negative, as the old
>> code
>> will remain inconsistent with new one.
>>
>> I am also noticing that every discussion about coding guidelines is about
>> a
>> matter of taste. We will never all agree on the taste, so my preference
>> would be just to follow the guidelines as they are.
>>
>> I also think that we should relax the coding guidelines for the
>> contributors (non-committers) as long as they get fixed by the committer
>> who does the final merge. This will make contributions easier (I will
>> start
>> a separate thread about it).
>>
>> D.
>>
>> On Thu, Oct 22, 2015 at 12:22 PM, Raul Kripalani 
>> wrote:
>>
>> Hi Yakov,
>>>
>>> Thanks for the review.
>>>
>>> Comments inline.
>>>
>>> Aside from them: being honest, do you think Javadoc like the following
>>> snippets are useful and provide any value? More so in a test case...
>>>
>>> Snippet #1:
>>>
>>>  /**
>>>   * @param topics Topics.
>>>   * @param fromIdx From index.
>>>   * @param cnt Count.
>>>   * @param singleMsg Single message flag.
>>>   * @throws MqttException If failed.
>>>   */
>>>
>>> Snippet #2:
>>>
>>>  /**
>>>   * @throws Exception If failed.
>>>   */
>>>
>>> To me, they are redundant. And I'm concerned about imposing superfluous
>>> Javadoc "just because", or due to aesthetics, rather than value.
>>>
>>> Regards,
>>>
>>> *Raúl Kripalani*
>>> PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
>>> Messaging Engineer
>>> http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
>>> http://blog.raulkr.net | twitter: @raulvk
>>>
>>> On Thu, Oct 22, 2015 at 11:52 AM, Yakov Zhdanov 
>>> wrote:
>>>
>>> Hi Raul!

 Thank you very much for addressing my comments!

 I like the code however there are still a couple of points (I committed
 everything to ignite-1747):

 1. Please take a look at



>>> https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines#CodingGuidelines-StringOutput
>>>
>>>
>>> Yeah, I'm familiar with this rule and I thought I was complying with it.
>>> Could you give me an example where the code is not compliant?
>>>
>>>
>>> 2. org.apache.ignite.stream.mqtt.MqttStreamer#connected - volatile write
 then read of most likely the same value - I changed it to return true
 literal.
 connected = true;
 return connected;

 Agree.
>>>
>>>
>>> 3. blockUntilConnected - I would name this property syncConn

 Why? To me "blockUntilConnected" is clearer and more explicit than
>>> "syncConn", which sounds mystic to an ordinary user who's not familiar
>>> with
>>> the community's abbreviation table. It's ok for a private member, but not
>>> for a property. In fact, syncConn requires the user to have the Javadoc
>>> handy, blockUntilConnected doesn't. So I prefer the latter.
>>>
>>>
>>> 4. Abbreviation rules violations - connected - conn, executor - exec,
 values - vals, count - cnt, message - msg

 Ok, this is a rule... yeah. But you can imagine it's extremely hard to
>>> memorise someone else's glossary. In my opinion, abbreviations should be
>>> used only when needed, not by default. An abbreviation subtracts
>>> readability for the general user – long-term Ignite developers are
>>> probably
>>> used 

[GitHub] ignite pull request: IGNITE-1371 implemented

[irudyak]

GitHub user irudyak opened a pull request:

https://github.com/apache/ignite/pull/172

IGNITE-1371 implemented



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/irudyak/ignite ignite-1371

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/172.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #172


commit 3b5c293894446a3013804e8a475d4a687e8b56f3
Author: Igor 
Date:   2015-10-16T05:39:26Z

ignite 1371: Implemented.

commit a89daea495a32a7bd3ea3211f4376bda5d7b6e8d
Author: root 
Date:   2015-10-16T05:45:14Z

Merge branch 'master' into ignite-1371




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-1769) We must close the current query on server-side when user start new one

[Pavel Konstantinov (JIRA)]

Pavel Konstantinov created IGNITE-1769:
--

 Summary: We must close the current query on server-side when user 
start new one
 Key: IGNITE-1769
 URL: https://issues.apache.org/jira/browse/IGNITE-1769
 Project: Ignite
  Issue Type: Sub-task
Reporter: Pavel Konstantinov


Otherwise we could get 'out of memory'



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-1770) Portables: implement constant-time field lookup.

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-1770:
---

 Summary: Portables: implement constant-time field lookup.
 Key: IGNITE-1770
 URL: https://issues.apache.org/jira/browse/IGNITE-1770
 Project: Ignite
  Issue Type: Task
  Components: general, interop
Affects Versions: ignite-1.4
Reporter: Vladimir Ozerov
Priority: Blocker
 Fix For: 1.5


See 
https://cwiki.apache.org/confluence/display/IGNITE/Portable+object+constant-time+field+lookup



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-1771) Ignite should support Apache Spark 1.5.1

[Artem Shutak (JIRA)]

Artem Shutak created IGNITE-1771:


 Summary: Ignite should support Apache Spark 1.5.1
 Key: IGNITE-1771
 URL: https://issues.apache.org/jira/browse/IGNITE-1771
 Project: Ignite
  Issue Type: Bug
Reporter: Artem Shutak


User can start Ignite with Spark 1.3.1 and 1.4.1, but cannot do the same with 
1.5.1. It should be fixed.

User list: 
http://apache-ignite-users.70518.x6.nabble.com/Question-about-getting-started-with-spark-shell-td1663.html.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] ignite pull request: IGNITE-1759 .Net: Guid read/write is not plat...

[ptupitsyn]

GitHub user ptupitsyn opened a pull request:

https://github.com/apache/ignite/pull/171

IGNITE-1759 .Net: Guid read/write is not platform-safe.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ptupitsyn/ignite ignite-1759

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/171.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #171


commit 52698b17068475536752c9ec9e6cc0a36b18ebaa
Author: ptupitsyn 
Date:   2015-10-21T17:28:47Z

wip todos

commit 77132979501ba32f27cc31f35bdedee1ddaa3486
Author: ptupitsyn 
Date:   2015-10-22T09:04:10Z

Endianness issue fixed

commit d5173706035a45936993902156198c41f88b8ce6
Author: ptupitsyn 
Date:   2015-10-22T09:41:20Z

WriteGuidBytewise

commit 3996dc01e3fbf05ceebc25d5a8a04ec977bc5ec6
Author: ptupitsyn 
Date:   2015-10-22T10:07:12Z

wip

commit 022300a1421359d57a598189f91787fd62532b47
Author: ptupitsyn 
Date:   2015-10-22T10:09:34Z

GetIsGuidSequentialPacked improved check

commit 6733894c879bf139002a4340858f7979ca48d16b
Author: ptupitsyn 
Date:   2015-10-22T10:11:24Z

wip

commit ddc5fc4c3f2f1425c39f0692c2d29c43d7239ef4
Author: ptupitsyn 
Date:   2015-10-22T10:11:59Z

Merge remote-tracking branch 'remotes/upstream/ignite-1282' into ignite-1759




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: TC patch validation

[Artem Shutak]

Yes, automatic runs were disabled for patches too.

To run patches you can use "Run All for patch" with specified jira number
at "Paramethers / Jira number*".

-- Artem --

On Thu, Oct 22, 2015 at 4:11 AM, Dmitriy Setrakyan 
wrote:

> I think Cos’ question also was why the patches attached to Jira do not get
> automatically run on TC. To my knowledge, the automatic runs were disabled,
> but I am not sure if it includes patches as well.
>
> Can someone clarify how to run patches on TC?
>
> D.
>
> On Fri, Oct 16, 2015 at 2:16 AM, Ivan Veselovskiy <
> iveselovs...@gridgain.com
> > wrote:
>
> > Hi, Konstantin,
> > Tests are grouped in suites by subsystems. You can run particular test
> > suite -- e.g. "Ignite Hadoop".
> > To run the tests with particular changes push the "Run..." button near
> that
> > test suite and chose "pull//head" in "Build
> > branch" field on the "Changes" tab.
> >
> > On Fri, Oct 16, 2015 at 1:01 AM, Konstantin Boudnik 
> > wrote:
> >
> > > Gents,
> > >
> > > is TC still picking up the JIRAs in PA state? Somehow it seems that
> only
> > > the
> > > latest changes from master are being picked.
> > >
> > > On a somewhat similar note: can I run tests for just a single module?
> > E.g.
> > > in
> > > IGNITE-1701 I made some changes in ignite-spark pom.xml and want to
> > quickly
> > > verify that this doesn't affect anything (which it shouldn't, clearly).
> > > Running the whole testset on my tiny laptop will probably take a few
> > hours.
> > >
> > > Appreciate any pointers, thanks in advance!
> > >   Cos
> > >
> > >
> >
>

[jira] [Created] (IGNITE-1776) [Test Failed] TcpClientDiscoverySpiSelfTest.testClientNodeFailOneServer fails sometimes

[Artem Shutak (JIRA)]

Artem Shutak created IGNITE-1776:


 Summary: [Test Failed] 
TcpClientDiscoverySpiSelfTest.testClientNodeFailOneServer fails sometimes
 Key: IGNITE-1776
 URL: https://issues.apache.org/jira/browse/IGNITE-1776
 Project: Ignite
  Issue Type: Bug
Reporter: Artem Shutak
Priority: Blocker
 Fix For: 1.5


TcpClientDiscoverySpiSelfTest.testClientNodeFailOneServer fails on TC sometimes.

Looks like a test bug, need just increase awaitTime(). I see Node FAILED 
message exactly after 10 seconds. But it's strange that it's require too long 
time. May be something wrong on agent.

Logs:
{noformat}
junit.framework.AssertionFailedError: Latch count: 1
at junit.framework.Assert.fail(Assert.java:57)
at junit.framework.Assert.assertTrue(Assert.java:22)
at junit.framework.TestCase.assertTrue(TestCase.java:192)
at 
org.apache.ignite.spi.discovery.tcp.TcpClientDiscoverySpiSelfTest.await(TcpClientDiscoverySpiSelfTest.java:2030)
at 
org.apache.ignite.spi.discovery.tcp.TcpClientDiscoverySpiSelfTest.testClientNodeFailOneServer(TcpClientDiscoverySpiSelfTest.java:811)
--- Stdout: ---
[03:18:46,228][INFO ][main][root] >>> Starting test: 
testClientNodeFailOneServer <<<
[03:18:46,233][INFO ][test-runner][IgniteKernal%server-0] 

>>>__    
>>>   /  _/ ___/ |/ /  _/_  __/ __/  
>>>  _/ // (7 7// /  / / / _/
>>> /___/\___/_/|_/___/ /_/ /___/   
>>> 
>>> ver. 1.5.0-SNAPSHOT#19700101-sha1:DEV
>>> 2015 Copyright(C) Apache Software Foundation
>>> 
>>> Ignite documentation: http://ignite.apache.org

[03:18:46,233][INFO ][test-runner][IgniteKernal%server-0] Config URL: n/a
[03:18:46,233][INFO ][test-runner][IgniteKernal%server-0] Daemon mode: off
[03:18:46,233][INFO ][test-runner][IgniteKernal%server-0] OS: Mac OS X 10.7.5 
x86_64
[03:18:46,233][INFO ][test-runner][IgniteKernal%server-0] OS user: teamcity
[03:18:46,233][INFO ][test-runner][IgniteKernal%server-0] Language runtime: 
Java Platform API Specification ver. 1.7
[03:18:46,233][INFO ][test-runner][IgniteKernal%server-0] VM information: 
Java(TM) SE Runtime Environment 1.7.0_67-b01 Oracle Corporation Java 
HotSpot(TM) 64-Bit Server VM 24.65-b04
[03:18:46,233][INFO ][test-runner][IgniteKernal%server-0] VM total memory: 2.7GB
[03:18:46,233][INFO ][test-runner][IgniteKernal%server-0] Remote Management 
[restart: off, REST: off, JMX (remote: off)]
[03:18:46,233][INFO ][test-runner][IgniteKernal%server-0] 
IGNITE_HOME=/Users/teamcity/buildAgent/work/871ff4a46e450b13
[03:18:46,233][INFO ][test-runner][IgniteKernal%server-0] VM arguments: 
[-DJAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home, 
-DMCAST_GRP=229.116.1.2, -Dagent.home.dir=/Users/teamcity/buildAgent, 
-Dagent.name=ip_192.168.1.116, -Dagent.ownPort=9090, 
-Dagent.work.dir=/Users/teamcity/buildAgent/work, -Dbuild.number=3345, 
-Dbuild.vcs.number=91e31e99c2574f7cede8d80ec9b8d981ccc34bcb, 
-Dbuild.vcs.number.1=91e31e99c2574f7cede8d80ec9b8d981ccc34bcb, 
-Dbuild.vcs.number.ApacheIgniteMirrorOnGitHub=91e31e99c2574f7cede8d80ec9b8d981ccc34bcb,
 -Dclassworlds.conf=/Users/teamcity/buildAgent/temp/buildTmp/teamcity.m2.conf, 
-Dcom.jetbrains.maven.watcher.report.file=/Users/teamcity/buildAgent/temp/buildTmp/maven-build-info.xml,
 -Djava.io.tmpdir=/Users/teamcity/buildAgent/temp/buildTmp, 
-Dmaven.home=/Users/teamcity/buildAgent/tools/maven3, 
-Dmaven.repo.local=/Users/teamcity/.m2/repository, 
-Dteamcity.agent.cpuBenchmark=605, 
-Dteamcity.agent.dotnet.agent_url=http://localhost:9090/RPC2, 
-Dteamcity.agent.dotnet.build_id=555612, 
-Dteamcity.auth.password=j1gKlVPBOa3H6Wnrjp3Q3sK7ZqR4biuf, 
-Dteamcity.auth.userId=TeamCityBuildId=555612, 
-Dteamcity.build.changedFiles.file=/Users/teamcity/buildAgent/temp/buildTmp/changedFiles5940678572218707636.txt,
 -Dteamcity.build.checkoutDir=/Users/teamcity/buildAgent/work/871ff4a46e450b13, 
-Dteamcity.build.id=555612, 
-Dteamcity.build.properties.file=/Users/teamcity/buildAgent/temp/buildTmp/teamcity.build3844607694634646095.properties,
 -Dteamcity.build.tempDir=/Users/teamcity/buildAgent/temp/buildTmp, 
-Dteamcity.build.workingDir=/Users/teamcity/buildAgent/work/871ff4a46e450b13, 
-Dteamcity.buildConfName=Ignite SPI, -Dteamcity.buildType.id=Ignite_Spi, 
-Dteamcity.configuration.properties.file=/Users/teamcity/buildAgent/temp/buildTmp/teamcity.config8915518964213838573.properties,
 -Dteamcity.idea.home=/Users/teamcity/buildAgent/plugins/idea, 
-Dteamcity.maven.watcher.home=/Users/teamcity/buildAgent/plugins/mavenPlugin/maven-watcher,
 -Dteamcity.projectName=Ignite Tests, 
-Dteamcity.runner.properties.file=/Users/teamcity/buildAgent/temp/buildTmp/teamcity.runner225244050948827951.properties,
 
-Dteamcity.tests.recentlyFailedTests.file=/Users/teamcity/buildAgent/temp/buildTmp/testsToRunFirst1548973897998565136.txt,
 -Dteamcity.version=8.1.4 (build 30168), -ea, -XX:MaxPermSize=1024m, 

[jira] [Created] (IGNITE-1775) Show Y-columns in chart settings with same color as chart

[Alexey Kuznetsov (JIRA)]

Alexey Kuznetsov created IGNITE-1775:


 Summary: Show Y-columns in chart settings with same color as chart
 Key: IGNITE-1775
 URL: https://issues.apache.org/jira/browse/IGNITE-1775
 Project: Ignite
  Issue Type: Sub-task
  Components: wizards
Affects Versions: 1.5
Reporter: Alexey Kuznetsov
Assignee: Alexey Kuznetsov
 Fix For: 1.5


# Show synthetic columns TIME_LINE, ROW_IDX with different color.
# Show Y-columns with corresponding chart colors.
# Made chart colors more distinct.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Data Snapshots in Ignite

[M G]

I have a specific use-case for snapshots that my current client would want
to make use of, it may be helpful if I share it with you.

At the start of day we make a batch load from a reference data system, and
run a set of Start-Of-Day (SOD) reports.  Those reports must be on a
consistent view of the data - no updates from external sources are
permitted whilst the reports are running.  However, once the SOD reports
have run, we then want to receive updates from our databases and apply
those to the cache.  Here is the use-case: we want to have the original SOD
data available, snapshotted, so that we can re-run reports if they fail, or
compare what was used in the SOD reports with what those reports now
produce.

At present we are going to build an extra layer around my Ignite-based
library that provides this snapshot functionality.

On Thu, Oct 22, 2015 at 1:38 PM, Raul Kripalani  wrote:

> Hey Andre,
>
> I think I answered some of your questions in my response to Dmitriy [1].
> Could you please have a look and tell me if it answers your questions?
>
> N.B.: My idea is based around the typical use case for LevelDb Snapshots,
> but we might create something entirely different in Ignite if the community
> wants to.
>
> [1]
>
> http://apache-ignite-developers.2346864.n4.nabble.com/Data-Snapshots-in-Ignite-tp4183p4220.html
>
> *Raúl Kripalani*
> PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
> Messaging Engineer
> http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
> http://blog.raulkr.net | twitter: @raulvk
>
> On Thu, Oct 22, 2015 at 12:49 PM, Andrey Kornev 
> wrote:
>
> > Hello,
> >
> > Just a few questions.
> >
> > 1) It's not clear from the proposed API how to capture/retrieve a
> > consistent snapshot of multiple caches. If my query involves a join I'd
> > like to ensure consistency across all join participants.
> > 2) Implementation wise, is the snapshot just a physical copy of all cache
> > entries and their indexes? Or some other mechanism is being considered?
> > 3) Isolation: is the snapshot isolated with respect to concurrent
> > modifications?
> > 4) Serialization: what are my options to ensure that I can still read the
> > data from the old snapshots as my key/value class definitions change over
> > time?
> >
> >  I feel I do not quite understand the specific use case this feature is
> > expected to be applicable to. Why keeping a snapshot for 2 weeks is
> > unimaginable, but 1 or 2 hours is ok?
> >
> > Also, I think forcing people to set a TTL on a snapshot is pointless and
> > will be abused by setting it to an unreasonably large value, just in
> case.
> >
> > Thanks
> > Andrey
> >
> > > From: ra...@apache.org
> > > Date: Wed, 21 Oct 2015 10:06:25 +0100
> > > Subject: Data Snapshots in Ignite
> > > To: dev@ignite.apache.org
> > >
> > > Hey guys,
> > >
> > > LevelDb has a functionality called Snapshots which provides a
> consistent
> > > read-only view of the DB at a given point in time, against which
> queries
> > > can be executed.
> > >
> > > To my knowledge, this functionality doesn't exist in the world of open
> > > source In-Memory Computing. Ignite could be an innovator here.
> > >
> > > Ignite Snapshots would allow queries, distributed closures, map-reduce
> > > jobs, etc. It could be useful for Spark RDDs to avoid data shift while
> > the
> > > computation is taking place (not sure if there's already some form of
> > > snapshotting, though). Same for IGFS.
> > >
> > > Example usage:
> > >
> > > IgniteCacheSnapshot snapshot =
> > > ignite.cache("mycache").snapshots().create();
> > >
> > > // all three queries are executed against a view of the cache at
> the
> > > point in time where it was snapshotted
> > > snapshot.query("select ...");
> > > snapshot.query("select ...");
> > > snapshot.query("select ...");
> > >
> > > In fact, it would be awesome to be able to logically save this snapshot
> > > with a name so that later jobs, queries, etc. can run on top of it,
> e.g.:
> > >
> > > IgniteCacheSnapshot snapshot =
> > > ignite.cache("mycache").snapshots().create("abc");
> > >
> > > // ...
> > > // in another module of a distributed system, or in another thread
> in
> > > parallel, use the saved snapshot
> > > IgniteCacheSnapshot snapshot =
> > > ignite.cache("mycache").snapshots().get("abc");
> > > 
> > >
> > > Named snapshotting can be dangerous due to data retention, e.g. imagine
> > > keeping a snapshot for 2 weeks! So we should force the user to specify
> a
> > > TTL:
> > >
> > > IgniteCacheSnapshot snapshot =
> > > ignite.cache("mycache").snapshots().create("abc", 2, TimeUnit.HOURS);
> > >
> > > Such functionality would allow for "reporting checkpoints" and "time
> > > travel", for example, where you want users to be able to query the data
> > as
> > > it stood 1 hour ago, 2 hours ago, etc.
> > >
> > > What do you think?
> > >
> > > P.S.: We do 

[jira] [Created] (IGNITE-1774) REST API should be implemented using Jersey

[Michael Griggs (JIRA)]

Michael Griggs created IGNITE-1774:
--

 Summary: REST API should be implemented using Jersey
 Key: IGNITE-1774
 URL: https://issues.apache.org/jira/browse/IGNITE-1774
 Project: Ignite
  Issue Type: Improvement
Affects Versions: ignite-1.4
Reporter: Michael Griggs


Jersey+Jetty is a well established method for implementing RESTful interfaces.  

The REST API should implement non-modifying cache methods (e.g., get()) via 
HTTP GET methods, and modifying cache methods (e.g., put(), replace()) via HTTP 
POST methods.  This allows the JSON data to be correctly formatted in the body 
of the message, instead of needing to be URL-encoded on the URI.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Ignite Web Console - Loading metadata from MySQL

[Alexey Kuznetsov]

Prachi,

If this issue: https://issues.apache.org/jira/browse/IGNITE-1761

Is the same as you propose?

P.S.
 Did you managed to load metadata from your MySQL database?

On Fri, Oct 23, 2015 at 1:06 AM, Prachi Garg  wrote:

> Thanks!
>
> I think we should also provide an option on the client side for the user to
> browse to the local folder of his system.
>
> -P
>
> On Thu, Oct 22, 2015 at 12:39 AM, Dmitriy Setrakyan  >
> wrote:
>
> > On Wed, Oct 21, 2015 at 5:08 PM, Prachi Garg  wrote:
> >
> > > I am trying to load metadata from MySQL database, but the web console
> > > doesn't allow me to provide the MySQL driver JAR. It is set to H2, by
> > > default, and doesn't let me change. I tried running the Ignite web
> agent
> > in
> > > both test and non-test modes. How can I provide the MySQL driver JAR?
> > >
> >
> > The question (?) helper popup next to the driver field actually explains
> > what to do. You need to copy the driver JAR into the “jdbc-drivers”
> folder
> > in the agent.
> >
> >
> > > Thanks,
> > > -Prachi
> > >
> >
>



-- 
Alexey Kuznetsov
GridGain Systems
www.gridgain.com

Re: Data Snapshots in Ignite

[Raul Kripalani]

Hey Andre,

I think I answered some of your questions in my response to Dmitriy [1].
Could you please have a look and tell me if it answers your questions?

N.B.: My idea is based around the typical use case for LevelDb Snapshots,
but we might create something entirely different in Ignite if the community
wants to.

[1]
http://apache-ignite-developers.2346864.n4.nabble.com/Data-Snapshots-in-Ignite-tp4183p4220.html

*Raúl Kripalani*
PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
Messaging Engineer
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
http://blog.raulkr.net | twitter: @raulvk

On Thu, Oct 22, 2015 at 12:49 PM, Andrey Kornev 
wrote:

> Hello,
>
> Just a few questions.
>
> 1) It's not clear from the proposed API how to capture/retrieve a
> consistent snapshot of multiple caches. If my query involves a join I'd
> like to ensure consistency across all join participants.
> 2) Implementation wise, is the snapshot just a physical copy of all cache
> entries and their indexes? Or some other mechanism is being considered?
> 3) Isolation: is the snapshot isolated with respect to concurrent
> modifications?
> 4) Serialization: what are my options to ensure that I can still read the
> data from the old snapshots as my key/value class definitions change over
> time?
>
>  I feel I do not quite understand the specific use case this feature is
> expected to be applicable to. Why keeping a snapshot for 2 weeks is
> unimaginable, but 1 or 2 hours is ok?
>
> Also, I think forcing people to set a TTL on a snapshot is pointless and
> will be abused by setting it to an unreasonably large value, just in case.
>
> Thanks
> Andrey
>
> > From: ra...@apache.org
> > Date: Wed, 21 Oct 2015 10:06:25 +0100
> > Subject: Data Snapshots in Ignite
> > To: dev@ignite.apache.org
> >
> > Hey guys,
> >
> > LevelDb has a functionality called Snapshots which provides a consistent
> > read-only view of the DB at a given point in time, against which queries
> > can be executed.
> >
> > To my knowledge, this functionality doesn't exist in the world of open
> > source In-Memory Computing. Ignite could be an innovator here.
> >
> > Ignite Snapshots would allow queries, distributed closures, map-reduce
> > jobs, etc. It could be useful for Spark RDDs to avoid data shift while
> the
> > computation is taking place (not sure if there's already some form of
> > snapshotting, though). Same for IGFS.
> >
> > Example usage:
> >
> > IgniteCacheSnapshot snapshot =
> > ignite.cache("mycache").snapshots().create();
> >
> > // all three queries are executed against a view of the cache at the
> > point in time where it was snapshotted
> > snapshot.query("select ...");
> > snapshot.query("select ...");
> > snapshot.query("select ...");
> >
> > In fact, it would be awesome to be able to logically save this snapshot
> > with a name so that later jobs, queries, etc. can run on top of it, e.g.:
> >
> > IgniteCacheSnapshot snapshot =
> > ignite.cache("mycache").snapshots().create("abc");
> >
> > // ...
> > // in another module of a distributed system, or in another thread in
> > parallel, use the saved snapshot
> > IgniteCacheSnapshot snapshot =
> > ignite.cache("mycache").snapshots().get("abc");
> > 
> >
> > Named snapshotting can be dangerous due to data retention, e.g. imagine
> > keeping a snapshot for 2 weeks! So we should force the user to specify a
> > TTL:
> >
> > IgniteCacheSnapshot snapshot =
> > ignite.cache("mycache").snapshots().create("abc", 2, TimeUnit.HOURS);
> >
> > Such functionality would allow for "reporting checkpoints" and "time
> > travel", for example, where you want users to be able to query the data
> as
> > it stood 1 hour ago, 2 hours ago, etc.
> >
> > What do you think?
> >
> > P.S.: We do have some form of snapshotting in the Compute checkpointing
> > functionality – but my proposal is to generalise the notion.
> >
> > Regards,
> >
> > *Raúl Kripalani*
> > PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
> > Messaging Engineer
> > http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
> > http://blog.raulkr.net | twitter: @raulvk
>
>

Re: Data Snapshots in Ignite

[Raul Kripalani]

Hey Dmitry,

Actually, there are so many possibilities around snapshotting that we're
thinking about what I feel are two distinct functionalities ;-)

While persistent snapshotting is indeed useful, what you describe is a
mechanism somewhere in the spectrum between archiving and backups, right? I
think this may be a nice to have, but not a priority. Reason being that
Ignite would typically be part of a Lambda Architecture where
recent/actionable data is in cache + storage, and historical data (entire
dataset) only in storage. So the data ingestion layer (e.g. glued by Kafka)
would take care of feeding the data into both a persistent store (e.g.
Cassandra) indexed by time and into Ignite. I believe most users already
have some degree of persistence backing Ignite, in order to allow them to
recover from an integral Ignite disaster, right?

What I had in mind is a functionality that Ignite currently lacks (unless
I'm mistaken): the possibility of executing multiple read-only actions
against a consistent view of (paused) cache data. If I understand
correctly, there's currently no way to tell Ignite: "hey! I want to launch
3 compute jobs, one after another, each taking 5 minutes, against an
*identical* set of data, i.e. against a snapshot of data; I don't want
these jobs to see any data changes even if they occur in the underlying
cache during this time".

This type of snapshots would be short-lived, hence persisting the entire
snapshot is questionable. But retaining entries throughout the snapshot's
lifespan can also be dangerous due to memory constraints. So... how would
be solve this dilemma? Ideas:

* Move only evicted / outdated entries that are still active in the scope
of a snapshot to persistent medium. We would need an indexing mechanism
that addresses the location of the data item (e.g. memory or offset N in
persistent file X). As data changes in the underlying cache, Ignite would
keep filling up a disk file with the previous state of the updated /
evicted items as they stood within a snapshot.

* Keep the snapshot only in memory and allow the user to specify a policy
on how to handle memory repletion while snapshots are active:

* Cancel and discard the snapshot when memory usage reaches a certain
threshold. Interrupt any jobs / queries, etc. that were running and return
an exception.
* Throttle cache operations while the snapshot is active and memory is
getting full (reliant on a threshold).

To me, the TTL is important if we're retaining entries in memory...

Regards,

*Raúl Kripalani*
PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
Messaging Engineer
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
http://blog.raulkr.net | twitter: @raulvk

On Thu, Oct 22, 2015 at 1:58 AM, Dmitriy Setrakyan 
wrote:

> On Wed, Oct 21, 2015 at 4:48 PM, Konstantin Boudnik 
> wrote:
>
> > I like it quite a bit, as well! Ticket would make the most sense as well,
> > so
> > there will be a single place to collect the design docs (if needed), etc.
> >
> > On Wed, Oct 21, 2015 at 04:45PM, Dmitriy Setrakyan wrote:
> > > I also really like the idea. One potential use case is fraud analysis
> in
> > > financial institutions. Rarely it makes sense to perform such analysis
> > on a
> > > life system, but rather a snapshot of some data needs to be taken and
> > > analyzed offline.
> > >
> > > I think snapshots should be saved to disk, so users could load them for
> > > analysis on a totally different cluster.
> >
> > I think disk persistence should be optional, not mandatory.
> >
>
> I would actually prefer to support disk-only snapshots. I think it will be
> difficult (double-the-work) to support both, in-memory and disk formats.
> Also, storing snapshots in-memory would require extra memory (a lot of it)
> for something that gets saved mainly for historic purposes or offline
> analysis.
>
>
> >
> > Cos
> >
> > > Raul, if you don’t mind, can you file a ticket and see if anyone in the
> > > community wants to pick it up?
> > >
> > > D.
> > >
> > > On Wed, Oct 21, 2015 at 5:51 AM, Sergi Vladykin <
> > sergi.vlady...@gmail.com>
> > > wrote:
> > >
> > > > Raul,
> > > >
> > > > Actually SQL indexes are already snapshotable. I'm not sure if it
> does
> > make
> > > > sense to make
> > > > the whole cache (with full cache API support) snapshotable, but I
> like
> > your
> > > > idea
> > > > about running multiple SQL statements against the same snapshot.
> > > >
> > > > Also I don't think that it is a good idea to keep snapshots for a
> long
> > > > time,
> > > > so I'd prefer to have typical AutoClosable API like:
> > > >
> > > > try (Snapshot s = ...) {
> > > > s.query(...);
> > > > s.query(...);
> > > > s.query(...);
> > > > }
> > > >
> > > > Though I'm not sure when we will be able to get down to this.
> > > >
> > > > Sergi
> > > >
> > > > 2015-10-21 12:06 GMT+03:00 Raul Kripalani :
> > > >
> > > > > 

[jira] [Created] (IGNITE-1763) We need somehow handle long field name in metadata dialog

[Pavel Konstantinov (JIRA)]

Pavel Konstantinov created IGNITE-1763:
--

 Summary: We need somehow handle long field name in metadata dialog
 Key: IGNITE-1763
 URL: https://issues.apache.org/jira/browse/IGNITE-1763
 Project: Ignite
  Issue Type: Sub-task
Reporter: Pavel Konstantinov


Please see attachment



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)