Re: Move documentation from readme.io to GitHub pages

[Alexey Kuznetsov]

Pavel,

Do I correctly understand, that if I find a typo in docs, I should commit
fix to master and cherry-pick to all relevant branches?

On Fri, Feb 2, 2018 at 2:28 PM, Pavel Tupitsyn  wrote:

> I did some research for my blog some time ago, and proposal is:
>
> * Export markdown from Readme.io (tweak as necessary, code samples etc)
> * Store markdown in git. Separate branch for each Ignite version
> (ignite-2.3-docs, ignite-2.4-docs, etc). Future version is in master.
> * Use Jekyll to generate HTML from markdown
> * Host documentation on ignite.apache.org (upload is via SVN, any
> committer
> has rights, *changes are instant*)
>
> On Fri, Feb 2, 2018 at 2:05 AM, Denis Magda  wrote:
>
> > Dmitriy,
> >
> > We will do a research before jumping off the readme.io <
> http://readme.io/>
> > train.
> >
> > In addition, to the technical issues outlined in the ticket, readme.io <
> > http://readme.io/> doesn’t give a way to tell Google “Hey, Google, index
> > only the latest docs' version! Ignore specific versions”. As a result,
> the
> > Internet is flooded with Ignite 1.1, 1.7, 1.9 versions of the pages that
> > pop up at the top of the search. You know this annoying thing better than
> > me :)
> >
> > —
> > Denis
> >
> > > On Feb 1, 2018, at 1:39 PM, Dmitriy Setrakyan 
> > wrote:
> > >
> > > Guys, as I have been saying all along, we need to identify a
> > documentation
> > > system we want to move to. Once we do that, we can plan how to move off
> > of
> > > readme.
> > >
> > > Again, the most important benefit readme has is instantaneous
> > documentation
> > > update. However, on the downside, they do not integrate with Git well,
> > and
> > > ASF wants all its projects to keep the documentation in Apache Git.
> > >
> > > Does anyone have suggestions?
> > >
> > > D.
> > >
> > > On Thu, Feb 1, 2018 at 5:57 AM, Pavel Tupitsyn 
> > wrote:
> > >
> > >> Good news, Denis.
> > >>
> > >> By the way, readme.io uses some kind of markdown dialect, and they
> have
> > >> "export" feature,
> > >> so it should be possible to move to some markdown-based system like
> > Jekyll
> > >> without too much hassle.
> > >>
> > >> On Thu, Feb 1, 2018 at 2:18 AM, Denis Magda 
> wrote:
> > >>
> > >>> Created a ticket. I think the time to move from readme.io <
> > >>> http://readme.io/> has come. It’s becoming too challenging to live
> > with
> > >>> it:
> > >>> https://issues.apache.org/jira/browse/IGNITE-7595 <
> > >>> https://issues.apache.org/jira/browse/IGNITE-7595>
> > >>>
> > >>> —
> > >>> Denis
> > >>>
> >  On Apr 13, 2017, at 8:25 AM, Dmitriy Setrakyan <
> dsetrak...@apache.org
> > >
> > >>> wrote:
> > 
> >  On Wed, Apr 12, 2017 at 8:52 PM, Konstantin Boudnik  >
> > >>> wrote:
> > 
> > > I hate to be that guy, but mentors warned this very project no to
> do
> > > this in the first place. At least once [1] on the dev@, and a few
> > > times off-line
> > >
> > 
> >  Cos, no doubt this had come up before (after all, we did have very
> > good
> >  mentors during the incubation process).
> > 
> >  Unfortunately, for tasks like this it is never a good time. I hope
> > >>> someone
> >  in the community picks it up, or at least gets started on it.
> > >>>
> > >>>
> > >>
> >
> >
>



-- 
Alexey Kuznetsov

Re: Move documentation from readme.io to GitHub pages

[Pavel Tupitsyn]

I did some research for my blog some time ago, and proposal is:

* Export markdown from Readme.io (tweak as necessary, code samples etc)
* Store markdown in git. Separate branch for each Ignite version
(ignite-2.3-docs, ignite-2.4-docs, etc). Future version is in master.
* Use Jekyll to generate HTML from markdown
* Host documentation on ignite.apache.org (upload is via SVN, any committer
has rights, *changes are instant*)

On Fri, Feb 2, 2018 at 2:05 AM, Denis Magda  wrote:

> Dmitriy,
>
> We will do a research before jumping off the readme.io 
> train.
>
> In addition, to the technical issues outlined in the ticket, readme.io <
> http://readme.io/> doesn’t give a way to tell Google “Hey, Google, index
> only the latest docs' version! Ignore specific versions”. As a result, the
> Internet is flooded with Ignite 1.1, 1.7, 1.9 versions of the pages that
> pop up at the top of the search. You know this annoying thing better than
> me :)
>
> —
> Denis
>
> > On Feb 1, 2018, at 1:39 PM, Dmitriy Setrakyan 
> wrote:
> >
> > Guys, as I have been saying all along, we need to identify a
> documentation
> > system we want to move to. Once we do that, we can plan how to move off
> of
> > readme.
> >
> > Again, the most important benefit readme has is instantaneous
> documentation
> > update. However, on the downside, they do not integrate with Git well,
> and
> > ASF wants all its projects to keep the documentation in Apache Git.
> >
> > Does anyone have suggestions?
> >
> > D.
> >
> > On Thu, Feb 1, 2018 at 5:57 AM, Pavel Tupitsyn 
> wrote:
> >
> >> Good news, Denis.
> >>
> >> By the way, readme.io uses some kind of markdown dialect, and they have
> >> "export" feature,
> >> so it should be possible to move to some markdown-based system like
> Jekyll
> >> without too much hassle.
> >>
> >> On Thu, Feb 1, 2018 at 2:18 AM, Denis Magda  wrote:
> >>
> >>> Created a ticket. I think the time to move from readme.io <
> >>> http://readme.io/> has come. It’s becoming too challenging to live
> with
> >>> it:
> >>> https://issues.apache.org/jira/browse/IGNITE-7595 <
> >>> https://issues.apache.org/jira/browse/IGNITE-7595>
> >>>
> >>> —
> >>> Denis
> >>>
>  On Apr 13, 2017, at 8:25 AM, Dmitriy Setrakyan  >
> >>> wrote:
> 
>  On Wed, Apr 12, 2017 at 8:52 PM, Konstantin Boudnik 
> >>> wrote:
> 
> > I hate to be that guy, but mentors warned this very project no to do
> > this in the first place. At least once [1] on the dev@, and a few
> > times off-line
> >
> 
>  Cos, no doubt this had come up before (after all, we did have very
> good
>  mentors during the incubation process).
> 
>  Unfortunately, for tasks like this it is never a good time. I hope
> >>> someone
>  in the community picks it up, or at least gets started on it.
> >>>
> >>>
> >>
>
>

Re: Thin client failover mechanism (+ODBC, JDBC)

[Pavel Tupitsyn]

Dmitriy, yes, that's what I'm implementing as part of IGNITE-7282:
* List of hosts in config
* Pick random index (basic load balancing), connect
* When connection is lost, all current requests throw an exception
* Next request causes reconnect attempt to the next host
* If all hosts fail, throw exception


On Fri, Feb 2, 2018 at 12:44 AM, Dmitriy Setrakyan 
wrote:

> On Thu, Feb 1, 2018 at 5:55 AM, Pavel Tupitsyn 
> wrote:
>
> > Ok, let's add simple reconnect logic and see what will come of it.
> >
>
> Just to clarify, a list of IP addresses a client should connect to needs to
> be provided on startup. Once a connection is lost, a client needs to try to
> connect to all other IPs in the list before failing. Agree, this should be
> fairly simple to implement.
>

[jira] [Created] (IGNITE-7609) .NET: FieldsQueryCursor should expose data types too

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-7609:
--

 Summary: .NET: FieldsQueryCursor should expose data types too
 Key: IGNITE-7609
 URL: https://issues.apache.org/jira/browse/IGNITE-7609
 Project: Ignite
  Issue Type: Improvement
  Components: platforms
Affects Versions: 2.4
Reporter: Pavel Tupitsyn


See IGNITE-7607 for Java, add same thing to .NET:
{code}
public interface IFieldsQueryCursor
{
...
IList FieldTypes
}
{code}

T could be string or Type, needs investigation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7608) Sort keys in putAll/removeAll methods

[Alexander Belyak (JIRA)]

Alexander Belyak created IGNITE-7608:


 Summary: Sort keys in putAll/removeAll methods
 Key: IGNITE-7608
 URL: https://issues.apache.org/jira/browse/IGNITE-7608
 Project: Ignite
  Issue Type: Improvement
  Components: general
Affects Versions: 2.2, 2.1, 2.0
 Environment: We need to sort keys in cache putAll/removeAll operations 
to avoid deadlocks there.
Reporter: Alexander Belyak






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7607) FieldsQueryCursor should expose data types too

[Mikhail Cherkasov (JIRA)]

Mikhail Cherkasov created IGNITE-7607:
-

 Summary: FieldsQueryCursor should expose data types too
 Key: IGNITE-7607
 URL: https://issues.apache.org/jira/browse/IGNITE-7607
 Project: Ignite
  Issue Type: Improvement
  Components: sql
Reporter: Mikhail Cherkasov


FieldsQueryCursor should expose data types too, this will simple users life in 
some case.

 

This feature was requested by user list:

http://apache-ignite-users.70518.x6.nabble.com/Optional-meta-schema-in-cache-level-tt19635.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Ignite 3rd party support and employment page

[Denis Magda]

Igniters,

After clicking through various Apache sites and wikis, I discovered that some 
projects such as Hadoop [2] and Cassandra [3] maintain the pages of 3rd parties 
that employ Apache experts and/or provide professional support.

Eventually, I dared to assemble a similar page for Ignite [1]. Why? This is how 
those who gained Ignite expertise can find their next dream job, brag about 
Ignite there and keep coming to the community contribute to the code base.

I would listen for your opinion on this and then send the page off to the user 
list calling for companies who need Ignite experts or provide technical support.
 
[1] 
https://cwiki.apache.org/confluence/display/IGNITE/Third+Party+Support+and+Employment
[2] https://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Support
[3] https://wiki.apache.org/cassandra/ThirdPartySupport

—
Denis

Re: Upcoming Apache Ignite events this month

[Dmitriy Setrakyan]

Great to see such a busy schedule!

Ignite community is unstoppable :)

D.

On Thu, Feb 1, 2018 at 3:19 PM, Tom Diederich 
wrote:

> Igniters,
>
> The following is a list of upcoming events in February. To view this list
> from the Ignite events page, click here
> .
>
> *Tokyo*
>
> *February 1:*  *Meetup*: Meet Apache Ignite In-Memory Computing Platform
>
>  Join Roman Shtykh at the Tech it Easy- Tokyo Meetup for an introductory
> talk on Apache Ignite.
>
>  In this talk you will learn about Apache Ignite memory-centric
> distributed database, caching, and processing platform. Roman will explain
> how one can do distributed computing, and use SQL with horizontal
> scalability and high availability of NoSQL systems with Apache Ignite.
>
>  Only six spots left so RSVP now! http://bit.ly/2nygyRI
>
>  *San Francisco Bay Area*
>
>  *February 7*: *Conference talk:* Apache Ignite Service Grid: Foundation
> of Your Microservices-Based Solution
>
>  Denis Magda will be attending DeveloperWeek 2018 in San Francisco to
> deliver presentation that provides a step-by-step guide on how to build a
> fault-tolerant and scalable microservices-based solution using Apache
> Ignite's Service Grid and other components to resolve these aforementioned
> issues.
>
>  Details here: http://bit.ly/2BHwFBr
>
>
>
> *London*
>
>  *February 7:* *Meetup:* Building consistent and highly available
> distributed systems with Apache Ignite
>
>  Akmal Chaudhri will speak at the inaugural gathering of the London
> In-Memory Computing Meetup.
>
>  He'll explain that while it is well known that there is a tradeoff
> between data consistency and high availability, there are many applications
> that require very strong consistency guarantees. Making such applications
> highly available can be a significant challenge. Akmal will explain how to
> overcome these challenges.
>
> This will be an outstanding event with free food and beverages. Space is
> limited, however. RSVP now to reserve your spot (you may also include 2
> guests).
>
> http://bit.ly/2BH893c
>
>
> *Boston*
>
> *February 12: Meetup*: Turbocharge your MySQL queries in-memory with
> Apache Ignite
>
> Fotios Filacouris will be the featured speaker at the Boston MySQL Meetup
> Group.
>
> The abstract of his talk: Apache Ignite is a unique data management
> platform that is built on top of a distributed key-value storage and
> provides full-fledged MySQL support.Attendees will learn how Apache Ignite
> handles auto-loading of a MySQL schema and data from PostgreSQL, supports
> MySQL indexes, supports compound indexes, and various forms of MySQL
> queries including distributed MySQL joins.
>
> Space is limited so RSVP today! http://bit.ly/2DP8W44
>
>
> *Boston*
>
> *February 13: Meetup* -- Java and In-Memory Computing: Apache Ignite
>
>  Fotios Filacouris will speak at the Boston Java Meetup Group
>
> In his talk, Foti will introduce the many components of the open-source
> Apache Ignite. Meetup members, as Java professionals, will learn how to
> solve some of the most demanding scalability and performance challenges.
> He’ll also cover a few typical use cases and work through some code
> examples. Attendees would leave ready to fire up their own database
> deployments!
>
> RSVP here: http://bit.ly/2BJ1nde
>
>
>
> *Sydney, Australia  *
>
> *February 13: Meetup:* Ignite your Cassandra Love Story: Caching
> Cassandra with Apache Ignite
>
> Rachel Pedreschi will be the guest speaker at the Sydney Cassandra Users
> Meetup. In this session attendees will learn how Apache Ignite can
> turbocharge a Cassandra cluster without sacrificing availability
> guarantees. In this talk she'll cover:
>
>
>
>- An overview of the Apache Ignite architecture
>- How to deploy Apache Ignite in minutes on top of Cassandra
>- How companies use this powerful combination to handle extreme OLTP
>workloads
>
>
>  RSVP now to secure your spot: http://bit.ly/2sydneytalk
>
>
>
> * February 14: Webinar:*  Getting Started with Apache® Ignite™ as a
> Distributed Database
>
> Join presenter Valentin Kulichenko in this live webinar featuring Apache
> Ignite native persistence --  a distributed ACID and SQL-compliant store
> that turns Apache Ignite into a full-fledged distributed SQL database.
>
>  In this webinar, Valentin will:
>
>
>
>-  Explain what native persistence is, and how it works
>- Show step-by-step how to set up Apache Ignite with native persistence
>- Explain the best practices for configuration and tuning
>
>
> RSVP now to reserve your spot: http://bit.ly/2E0SWiS
>
>
>
> *Copenhagen*
>
> *February 14: Meetup: *Apache Ignite: the in-memory hammer in your data
> science toolkit
>
> Akmal Chaudhri will be the guest speaker at the Symbion IoT Meetup
> (Copenhagen, Denmark). In this presentation, Akmal will explain some of the
> main components of Apache Ignite, such as the Compute Grid, Data Grid and
> the Machine 

Upcoming Apache Ignite events this month

[Tom Diederich]

Igniters, 

The following is a list of upcoming events in February. To view this list from 
the Ignite events page, click here . 

Tokyo

February 1:  Meetup: Meet Apache Ignite In-Memory Computing Platform 

 Join Roman Shtykh at the Tech it Easy- Tokyo Meetup for an introductory talk 
on Apache Ignite.


 In this talk you will learn about Apache Ignite memory-centric distributed 
database, caching, and processing platform. Roman will explain how one can do 
distributed computing, and use SQL with horizontal scalability and high 
availability of NoSQL systems with Apache Ignite.


 Only six spots left so RSVP now! http://bit.ly/2nygyRI  


 San Francisco Bay Area

 February 7: Conference talk: Apache Ignite Service Grid: Foundation of Your 
Microservices-Based Solution


 Denis Magda will be attending DeveloperWeek 2018 in San Francisco to deliver 
presentation that provides a step-by-step guide on how to build a 
fault-tolerant and scalable microservices-based solution using Apache Ignite's 
Service Grid and other components to resolve these aforementioned issues.


 Details here: http://bit.ly/2BHwFBr  


 

London

 February 7: Meetup: Building consistent and highly available distributed 
systems with Apache Ignite


 Akmal Chaudhri will speak at the inaugural gathering of the London In-Memory 
Computing Meetup.


 He'll explain that while it is well known that there is a tradeoff between 
data consistency and high availability, there are many applications that 
require very strong consistency guarantees. Making such applications highly 
available can be a significant challenge. Akmal will explain how to overcome 
these challenges.


This will be an outstanding event with free food and beverages. Space is 
limited, however. RSVP now to reserve your spot (you may also include 2 guests).


http://bit.ly/2BH893c  



Boston

February 12: Meetup: Turbocharge your MySQL queries in-memory with Apache 
Ignite 

Fotios Filacouris will be the featured speaker at the Boston MySQL Meetup Group.


The abstract of his talk: Apache Ignite is a unique data management platform 
that is built on top of a distributed key-value storage and provides 
full-fledged MySQL support.Attendees will learn how Apache Ignite handles 
auto-loading of a MySQL schema and data from PostgreSQL, supports MySQL 
indexes, supports compound indexes, and various forms of MySQL queries 
including distributed MySQL joins.


Space is limited so RSVP today! http://bit.ly/2DP8W44  




Boston

February 13: Meetup -- Java and In-Memory Computing: Apache Ignite


 Fotios Filacouris will speak at the Boston Java Meetup Group


In his talk, Foti will introduce the many components of the open-source Apache 
Ignite. Meetup members, as Java professionals, will learn how to solve some of 
the most demanding scalability and performance challenges. He’ll also cover a 
few typical use cases and work through some code examples. Attendees would 
leave ready to fire up their own database deployments!


RSVP here: http://bit.ly/2BJ1nde  


 
Sydney, Australia  

February 13: Meetup: Ignite your Cassandra Love Story: Caching Cassandra with 
Apache Ignite


Rachel Pedreschi will be the guest speaker at the Sydney Cassandra Users 
Meetup. In this session attendees will learn how Apache Ignite can turbocharge 
a Cassandra cluster without sacrificing availability guarantees. In this talk 
she'll cover:



An overview of the Apache Ignite architecture
How to deploy Apache Ignite in minutes on top of Cassandra
How companies use this powerful combination to handle extreme OLTP workloads


 RSVP now to secure your spot: http://bit.ly/2sydneytalk 
 


 

 February 14: Webinar:  Getting Started with Apache® Ignite™ as a Distributed 
Database


Join presenter Valentin Kulichenko in this live webinar featuring Apache Ignite 
native persistence --  a distributed ACID and SQL-compliant store that turns 
Apache Ignite into a full-fledged distributed SQL database.


 In this webinar, Valentin will:



 Explain what native persistence is, and how it works
Show step-by-step how to set up Apache Ignite with native persistence
Explain the best practices for configuration and tuning


RSVP now to reserve your spot: http://bit.ly/2E0SWiS  


 

Copenhagen

February 14: Meetup: Apache Ignite: the in-memory hammer in your data science 
toolkit


Akmal Chaudhri will be the guest speaker at the Symbion IoT Meetup (Copenhagen, 
Denmark). In this presentation, Akmal will explain some of the main components 
of Apache Ignite, such as the Compute Grid, Data Grid and the Machine Learning 
Grid. Through examples, attendees will learn how Apache Ignite can be used for 
data analysis.


This meetup is free but an RSVP is required to secure your spot: 
http://bit.ly/2DSO1Bi 

Re: Move documentation from readme.io to GitHub pages

[Denis Magda]

Dmitriy,

We will do a research before jumping off the readme.io  
train.

In addition, to the technical issues outlined in the ticket, readme.io 
 doesn’t give a way to tell Google “Hey, Google, index only 
the latest docs' version! Ignore specific versions”. As a result, the Internet 
is flooded with Ignite 1.1, 1.7, 1.9 versions of the pages that pop up at the 
top of the search. You know this annoying thing better than me :)

—
Denis

> On Feb 1, 2018, at 1:39 PM, Dmitriy Setrakyan  wrote:
> 
> Guys, as I have been saying all along, we need to identify a documentation
> system we want to move to. Once we do that, we can plan how to move off of
> readme.
> 
> Again, the most important benefit readme has is instantaneous documentation
> update. However, on the downside, they do not integrate with Git well, and
> ASF wants all its projects to keep the documentation in Apache Git.
> 
> Does anyone have suggestions?
> 
> D.
> 
> On Thu, Feb 1, 2018 at 5:57 AM, Pavel Tupitsyn  wrote:
> 
>> Good news, Denis.
>> 
>> By the way, readme.io uses some kind of markdown dialect, and they have
>> "export" feature,
>> so it should be possible to move to some markdown-based system like Jekyll
>> without too much hassle.
>> 
>> On Thu, Feb 1, 2018 at 2:18 AM, Denis Magda  wrote:
>> 
>>> Created a ticket. I think the time to move from readme.io <
>>> http://readme.io/> has come. It’s becoming too challenging to live with
>>> it:
>>> https://issues.apache.org/jira/browse/IGNITE-7595 <
>>> https://issues.apache.org/jira/browse/IGNITE-7595>
>>> 
>>> —
>>> Denis
>>> 
 On Apr 13, 2017, at 8:25 AM, Dmitriy Setrakyan 
>>> wrote:
 
 On Wed, Apr 12, 2017 at 8:52 PM, Konstantin Boudnik 
>>> wrote:
 
> I hate to be that guy, but mentors warned this very project no to do
> this in the first place. At least once [1] on the dev@, and a few
> times off-line
> 
 
 Cos, no doubt this had come up before (after all, we did have very good
 mentors during the incubation process).
 
 Unfortunately, for tasks like this it is never a good time. I hope
>>> someone
 in the community picks it up, or at least gets started on it.
>>> 
>>> 
>> 

Re: Thin client failover mechanism (+ODBC, JDBC)

[Dmitriy Setrakyan]

On Thu, Feb 1, 2018 at 5:55 AM, Pavel Tupitsyn  wrote:

> Ok, let's add simple reconnect logic and see what will come of it.
>

Just to clarify, a list of IP addresses a client should connect to needs to
be provided on startup. Once a connection is lost, a client needs to try to
connect to all other IPs in the list before failing. Agree, this should be
fairly simple to implement.

Re: Move documentation from readme.io to GitHub pages

[Dmitriy Setrakyan]

Guys, as I have been saying all along, we need to identify a documentation
system we want to move to. Once we do that, we can plan how to move off of
readme.

Again, the most important benefit readme has is instantaneous documentation
update. However, on the downside, they do not integrate with Git well, and
ASF wants all its projects to keep the documentation in Apache Git.

Does anyone have suggestions?

D.

On Thu, Feb 1, 2018 at 5:57 AM, Pavel Tupitsyn  wrote:

> Good news, Denis.
>
> By the way, readme.io uses some kind of markdown dialect, and they have
> "export" feature,
> so it should be possible to move to some markdown-based system like Jekyll
> without too much hassle.
>
> On Thu, Feb 1, 2018 at 2:18 AM, Denis Magda  wrote:
>
> > Created a ticket. I think the time to move from readme.io <
> > http://readme.io/> has come. It’s becoming too challenging to live with
> > it:
> > https://issues.apache.org/jira/browse/IGNITE-7595 <
> > https://issues.apache.org/jira/browse/IGNITE-7595>
> >
> > —
> > Denis
> >
> > > On Apr 13, 2017, at 8:25 AM, Dmitriy Setrakyan 
> > wrote:
> > >
> > > On Wed, Apr 12, 2017 at 8:52 PM, Konstantin Boudnik 
> > wrote:
> > >
> > >> I hate to be that guy, but mentors warned this very project no to do
> > >> this in the first place. At least once [1] on the dev@, and a few
> > >> times off-line
> > >>
> > >
> > > Cos, no doubt this had come up before (after all, we did have very good
> > > mentors during the incubation process).
> > >
> > > Unfortunately, for tasks like this it is never a good time. I hope
> > someone
> > > in the community picks it up, or at least gets started on it.
> >
> >
>

Re: Orphaned, duplicate, and main-class tests!

[Dmitry Pavlov]

Hi Ilya,

Thank you for this research. I think it is useful for community to identify
and remove obsolete tests (if any), and include lost test into CI run chain
(if applicable).

For test with main() methods I suggest to ask authors (git annotate) and if
there is no response probably we should remove such code.

Since I am not sure all tests in this lost suite are quite stable I
suggest to create standalone TC Run configuration for such tests.

Earlier I've removed most of tests causing timeouts from basic suite.
Ideally Basic suite should contain fast run quite stable tests ( and 0
flaky ) because it is included into RunAllBasic sub set to brief commit
check  (
https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_RunBasicTests
 ).

Sincerely,
Dmitriy Pavlov

чт, 1 февр. 2018 г. в 20:22, Ilya Kasnacheev :

> Hello!
>
> While working on Ignite, I have noticed that not all tests are in any test
> suite, hence I expect they are ignored. I have also noticed some files in
> src/test and named *Test.java are actually runnable main-classes and not
> tests. I think they're ignored to. Also I've noticed that 6 tests repeat
> twice.
>
> I have tried to fix it by introducing "lost and found" test suite. Not sure
> what to do with main-classes. I have also renamed abstract test classes to
> *AbstractTest.
>
> Please consider pull request https://github.com/apache/ignite/pull/3464
>
> I have started this suite on TC but I expect it to hang or worse.
>
> https://ci.ignite.apache.org/viewLog.html?buildId=1071504=queuedBuildOverviewTab
>
> Regards,
> --
> Ilya Kasnacheev
>

Re: LOCAL cache on client

[Valentin Kulichenko]

I meant "they should *explicitly* provide data region configuration", of
course.

-Val

On Thu, Feb 1, 2018 at 10:58 AM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Agree with Mike. I don't think it's a good idea to implicitly create data
> regions on client node. If one wants to have a local there, they should
> implicitly provide data region configuration (default or otherwise). If
> data region is not configured, then we should throw proper exception
> instead of NPE.
>
> -Val
>
> On Thu, Feb 1, 2018 at 10:54 AM, Michael Cherkasov <
> michael.cherka...@gmail.com> wrote:
>
>> Hi Dmitry,
>>
>> I think we should make a user to explicitly create data region on a
>> client.
>>
>> Client nodes aren't supposed to be used for data storing, so if someone
>> what to use a local cache on a client node, let's make him/her create data
>> region explicitly. Just to make sure that user knows what he/she is doing.
>>
>> Thanks,
>> Mike.
>>
>>
>> 2018-02-01 7:36 GMT-08:00 Dmitry Karachentsev > >:
>>
>> > Hello everyone!
>> >
>> > We have an erroneous use case when client tries to create LOCAL cache,
>> but
>> > by default it does not initiates default data region. So client gets
>> NPE.
>> > [1]
>> >
>> > I think it should be a lazy data region initialization on client. Do you
>> > have any concerns about this approach or other proposals?
>> >
>> > [1] https://issues.apache.org/jira/browse/IGNITE-7572
>> >
>> > Thanks!
>> >
>> >
>>
>
>

Re: Eviction policies with persistence

[Valentin Kulichenko]

Guys,

Thanks for responses. But I still fail to understand how it affects a user.

Are you saying that with persistence enabled Random-LRU is always used
regardless of what is specified in pageEvictionMode, while in in-memory
only scenario we can also utilize Random-2-LRU for eviction? Is this
accurate? Are there any other differences?

-Val

On Thu, Feb 1, 2018 at 5:01 AM, Dmitry Pavlov  wrote:

> Hi Vyacheslav,
>
> Yes, this is right, but for now PDS page based eviction uses Random-LRU,
> not Random-2-LRU. Something may be changed in future Ignite releases, but
> now Random-LRU is used. I am going to research if there is some room for
> improvements in this aspect (e.g. IGNITE-7507 recenly found).
>
> If clean page is evicted, nothing really happends. If dirty page is evicted
> from region during checkpointing process run, page is stored in file before
> eviction
>
> Some info about this can be found in
> https://cwiki.apache.org/confluence/display/IGNITE/
> Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-
> underthehood-Pagebasedeviction
>
> Sincerely,
> Dmitriy Pavlov
>
> чт, 1 февр. 2018 г. в 9:53, Vyacheslav Daradur :
>
> > Hi Valentin,
> >
> > As far as I know, when persistence is enabled and Ignite can't
> > allocate new page-memory in the memory segment, then Ignite
> > automatically evict some data from RAM to PDS using Random-2-LRU
> > eviction algorithm. And there are no mechanisms to evict entries out
> > when PDS is enabled.
> >
> > I'll be glad if I am not right and someone will correct me because, as
> > Ignite user, I can't use PDS only as RAM disk replication for the use
> > case in my project.
> >
> >
> > On Thu, Feb 1, 2018 at 3:53 AM, Valentin Kulichenko
> >  wrote:
> > > Folks,
> > >
> > > On "Eviction Policies" documentation page [1] we have the following
> > callout:
> > >
> > >> Configured eviction policy has no effect if Ignite persistence is
> > enabled
> > >> Note that if Ignite Persistence is enabled, then the page-based
> > evictions
> > > have no effect because the oldest pages will be purged from memory
> > > automatically.
> > >
> > > This really confuses me. Why is there a difference in how data is
> evicted
> > > from memory depending on having disk enabled or not? And how does
> > eviction
> > > work with persistence enabled then?
> > >
> > > Does anyone can clarify?
> > >
> > > [1] https://apacheignite.readme.io/docs/evictions
> > >
> > > -Val
> >
> >
> >
> > --
> > Best Regards, Vyacheslav D.
> >
>

Re: LOCAL cache on client

[Valentin Kulichenko]

Agree with Mike. I don't think it's a good idea to implicitly create data
regions on client node. If one wants to have a local there, they should
implicitly provide data region configuration (default or otherwise). If
data region is not configured, then we should throw proper exception
instead of NPE.

-Val

On Thu, Feb 1, 2018 at 10:54 AM, Michael Cherkasov <
michael.cherka...@gmail.com> wrote:

> Hi Dmitry,
>
> I think we should make a user to explicitly create data region on a client.
>
> Client nodes aren't supposed to be used for data storing, so if someone
> what to use a local cache on a client node, let's make him/her create data
> region explicitly. Just to make sure that user knows what he/she is doing.
>
> Thanks,
> Mike.
>
>
> 2018-02-01 7:36 GMT-08:00 Dmitry Karachentsev  >:
>
> > Hello everyone!
> >
> > We have an erroneous use case when client tries to create LOCAL cache,
> but
> > by default it does not initiates default data region. So client gets NPE.
> > [1]
> >
> > I think it should be a lazy data region initialization on client. Do you
> > have any concerns about this approach or other proposals?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-7572
> >
> > Thanks!
> >
> >
>

Re: LOCAL cache on client

[Michael Cherkasov]

Hi Dmitry,

I think we should make a user to explicitly create data region on a client.

Client nodes aren't supposed to be used for data storing, so if someone
what to use a local cache on a client node, let's make him/her create data
region explicitly. Just to make sure that user knows what he/she is doing.

Thanks,
Mike.


2018-02-01 7:36 GMT-08:00 Dmitry Karachentsev :

> Hello everyone!
>
> We have an erroneous use case when client tries to create LOCAL cache, but
> by default it does not initiates default data region. So client gets NPE.
> [1]
>
> I think it should be a lazy data region initialization on client. Do you
> have any concerns about this approach or other proposals?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-7572
>
> Thanks!
>
>

Orphaned, duplicate, and main-class tests!

[Ilya Kasnacheev]

Hello!

While working on Ignite, I have noticed that not all tests are in any test
suite, hence I expect they are ignored. I have also noticed some files in
src/test and named *Test.java are actually runnable main-classes and not
tests. I think they're ignored to. Also I've noticed that 6 tests repeat
twice.

I have tried to fix it by introducing "lost and found" test suite. Not sure
what to do with main-classes. I have also renamed abstract test classes to
*AbstractTest.

Please consider pull request https://github.com/apache/ignite/pull/3464

I have started this suite on TC but I expect it to hang or worse.
https://ci.ignite.apache.org/viewLog.html?buildId=1071504=queuedBuildOverviewTab

Regards,
-- 
Ilya Kasnacheev

[GitHub] ignite pull request #3464: Lost and Found suite for orphaned tests

[alamar]

GitHub user alamar opened a pull request:

https://github.com/apache/ignite/pull/3464

Lost and Found suite for orphaned tests

Remove duplicate tests.
Mark abstract Abstract.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-orphan

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3464.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3464


commit 14aa3b0b915412581be9c86aee8f0f3e4d6dd6eb
Author: Ilya Kasnacheev 
Date:   2018-02-01T17:15:33Z

Lost and Found suite for orphaned tests

Remove duplicate tests.
Mark abstract Abstract.




---

[jira] [Created] (IGNITE-7606) Write evicted dirty page during eviction without holding segment write lock

[Dmitriy Pavlov (JIRA)]

Dmitriy Pavlov created IGNITE-7606:
--

 Summary: Write evicted dirty page during eviction without holding 
segment write lock
 Key: IGNITE-7606
 URL: https://issues.apache.org/jira/browse/IGNITE-7606
 Project: Ignite
  Issue Type: Improvement
  Components: persistence
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov
 Fix For: 2.5
 Attachments: putdumpAt17second.txt

If a dirty page under the checkpoint is found, following is suggested
- copy it to the local thread buffer, 
- and then after performing all actions in region for evicting the page
- finish execution allocatePage()/acquirePage()
- unlock segment to allow other workers to operate
- perform the pwrite() call based on the data from local buffer

Now if page eviction started there is possible drops to 0 put/seconds in case a 
lot of threads are watiting for same segment lock.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Ignite SQL: How to drop table without explicit cache name

[Valentin Kulichenko]

Nikolay,

I believe the first option is the way to go. However, I'd also like to hear
from other community members.

Vladimir, Alex P, can you comment on this? I know we currently don't have
public API to execute DDL statement without having a cache. However, looks
like we use private API of GridQueryProcessor to overcome this in JDBC
drivers and Web Console. We now need a similar solution for data frames
integration, can you confirm that we're moving in the right direction?

-Val

On Thu, Feb 1, 2018 at 4:04 AM, Nikolay Izhikov  wrote:

> Hello, guys.
>
> I working on support of saving Spark DataFrame to Ignite [1].
>
> I want to execute "DROP TABLE XXX" query.
> To execute some SQL query in regulary way one need some cache.
> That cache has to differs from table cache to execute "DROP TABLE"
> successfully.
>
> I founded 3 different ways to execute SQL query without explicit cache
> name.
> Please, tell me which way is right?
> Do we need all 3 way to make a query?
>
> 1. JdbcRequestHandler - [2]
>
> ```
>
> private final GridKernalContext ctx;
> ...
> List> results = ctx.query().querySqlFields(qry,
> true, protocolVer.compareTo(VER_2_3_0) < 0);
>
> ```
>
> 2. GridCacheProcessor - [3]
>
> ```
>
> /**
> * Gets public cache that can be used for query execution.
> * If cache isn't created on current node it will be started.
> */
> public IgniteCacheProxy getOrStartPublicCache(boolean start,
> boolean inclLoc) throws IgniteCheckedException {
>
> ```
>
> 3. QueryCommandHandler - [4]
>
> ```
>
> protected static final String DFLT_CACHE_NAME = "default";
> ...
> String cacheName = req.cacheName() == null ? DFLT_CACHE_NAME :
> req.cacheName();
>
> ```
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-7337
> [2] https://github.com/apache/ignite/blob/master/modules/
> core/src/main/java/org/apache/ignite/internal/processors/
> odbc/jdbc/JdbcRequestHandler.java#L310
> [3] https://github.com/apache/ignite/blob/master/modules/
> core/src/main/java/org/apache/ignite/internal/processors/
> cache/GridCacheProcessor.java#L1648
> [4] https://github.com/apache/ignite/blob/master/modules/
> core/src/main/java/org/apache/ignite/internal/processors/
> rest/handlers/query/QueryCommandHandler.java#L318

[GitHub] ignite pull request #3460: IGNITE-7599: Missed licenses in ignite-direct-io ...

[dspavlov]

Github user dspavlov closed the pull request at:

https://github.com/apache/ignite/pull/3460


---

[jira] [Created] (IGNITE-7605) SQL COPY: add more SQL parser tests for positive scenarios

[Kirill Shirokov (JIRA)]

Kirill Shirokov created IGNITE-7605:
---

 Summary: SQL COPY: add more SQL parser tests for positive scenarios
 Key: IGNITE-7605
 URL: https://issues.apache.org/jira/browse/IGNITE-7605
 Project: Ignite
  Issue Type: Bug
  Components: sql
Affects Versions: 2.4
Reporter: Kirill Shirokov






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

LOCAL cache on client

[Dmitry Karachentsev]

Hello everyone!

We have an erroneous use case when client tries to create LOCAL cache, 
but by default it does not initiates default data region. So client gets 
NPE. [1]


I think it should be a lazy data region initialization on client. Do you 
have any concerns about this approach or other proposals?


[1] https://issues.apache.org/jira/browse/IGNITE-7572

Thanks!

[jira] [Created] (IGNITE-7604) SQL TX: Allow DML operations with reducer

[Igor Seliverstov (JIRA)]

Igor Seliverstov created IGNITE-7604:


 Summary: SQL TX: Allow DML operations with reducer
 Key: IGNITE-7604
 URL: https://issues.apache.org/jira/browse/IGNITE-7604
 Project: Ignite
  Issue Type: Task
  Components: sql
Reporter: Igor Seliverstov


Allow DML operations with reducer



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Transport compression (not store compression)

[Nikita Amelchev]

Thanks for your comments,

1. At first, I would define final design and then I will make it pluggable.

2. We cannot prioritize messages at this network communication level. If
compression utilizes processor poorly then we can increase count of
selectors (make more threads when compression enabled).

These benchmarks show how much bytes and time will be saved on transfer
messages with different length and regularity. Tests provide data which
have a simple physical meaning. The use of "real" operations will give the
unstable result and will change as you change the implementation of these
"real" operations. Test that generated a lot of puts will also be
synthetic, just like any other.
However, I can try to come up with yardstick test with operations like a
real case. What will we test? Do we test putAll + getAll with text data?

2018-01-25 13:23 GMT+03:00 gvvinblade :

> Nikita,
>
> I took a look at the changes and see two issues.
>
> The first one is additional dependencies in core module. That means you
> should whether make the filter plugable and provide some additional module
> for it or put all the necessary classes to ignite-core.
>
> The second is possible bottleneck in NIO threads. Since each worker serves
> several connections we may face troubles on big clusters (for example one
> or
> two ignite server nodes and a few dozens client nodes) because
> compression/decompression is an expensive operation and happens (in your
> implementation) in NIO threads where message processing becames sequential.
>
> The provided benchmarks are quite synthetic (they are local, they involve
> only two nodes, they don't use "real" operations like put, get, putAll etc
> and don't show a real picture)
>
> We have to be careful with such changes and check all possible scenarios.
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>



-- 
Best wishes,
Amelchev Nikita

[jira] [Created] (IGNITE-7603) Waiting on reconnect future doesn't lead to working instance when client reconnects

[Ilya Kasnacheev (JIRA)]

Ilya Kasnacheev created IGNITE-7603:
---

 Summary: Waiting on reconnect future doesn't lead to working 
instance when client reconnects
 Key: IGNITE-7603
 URL: https://issues.apache.org/jira/browse/IGNITE-7603
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Ilya Kasnacheev
Assignee: Ilya Kasnacheev


See attached modified test. It will fail on
{code:java}
icde.reconnectFuture().get();
assertNull(client.getOrCreateCache(CACHE_PARAMS).get(1L));{code}
about 20% of time. I expect that I can get cache after waiting on reconnect 
future.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Move documentation from readme.io to GitHub pages

[Pavel Tupitsyn]

Good news, Denis.

By the way, readme.io uses some kind of markdown dialect, and they have
"export" feature,
so it should be possible to move to some markdown-based system like Jekyll
without too much hassle.

On Thu, Feb 1, 2018 at 2:18 AM, Denis Magda  wrote:

> Created a ticket. I think the time to move from readme.io <
> http://readme.io/> has come. It’s becoming too challenging to live with
> it:
> https://issues.apache.org/jira/browse/IGNITE-7595 <
> https://issues.apache.org/jira/browse/IGNITE-7595>
>
> —
> Denis
>
> > On Apr 13, 2017, at 8:25 AM, Dmitriy Setrakyan 
> wrote:
> >
> > On Wed, Apr 12, 2017 at 8:52 PM, Konstantin Boudnik 
> wrote:
> >
> >> I hate to be that guy, but mentors warned this very project no to do
> >> this in the first place. At least once [1] on the dev@, and a few
> >> times off-line
> >>
> >
> > Cos, no doubt this had come up before (after all, we did have very good
> > mentors during the incubation process).
> >
> > Unfortunately, for tasks like this it is never a good time. I hope
> someone
> > in the community picks it up, or at least gets started on it.
>
>

Re: Thin client failover mechanism (+ODBC, JDBC)

[Pavel Tupitsyn]

Ok, let's add simple reconnect logic and see what will come of it.

On Thu, Feb 1, 2018 at 12:49 AM, Dmitriy Setrakyan 
wrote:

> Pavel,
>
> I disagree. I think automatic reconnect is a very useful feature. For
> example, all client-side operation can throw exception anyway, so if you
> throw an exception due to a client reconnect, it will not require any
> additional exception-handling logic.
>
> On the other hand, after a few failed operations in case of a reconnect,
> the client will continue to operate normally. This will make our clients
> resilient to failures and make it way more powerful.
>
> I strongly vote to add this behavior.
>
> D.
>
>
> On Wed, Jan 31, 2018 at 3:15 AM, Pavel Tupitsyn 
> wrote:
>
> > Alexey, retrieving addresses from topology makes sense, but in this
> thread
> > I'm trying to understand whether any kind of built-in failover
> > makes sense at all at the Ignite API level.
> >
> > I mean, on the business logic level failover certainly makes sense:
> > if Web Agent has failed to execute some operation, it can show an error,
> > automatically reconnect to another node and continue working.
> >
> > But on the Ignite API level it gets questionable. We can implement some
> > failover/reconnect logic, but users still has to handle failed operations
> > themselves.
> >
> > Pavel
> >
> > On Wed, Jan 31, 2018 at 2:08 PM, Alexey Kuznetsov  >
> > wrote:
> >
> > > Pavel,
> > >
> > > I hope, that at some point Web agent (connector to Web Console) will be
> > > refactored from REST to thin client.
> > >
> > > It will be nice if thin client will support following modes:
> > > 1) Specify several addresses in thin client connection config. Thin
> > client
> > > will use ONLY this addresses (hardcoded list).
> > > 2) Same as #1, but in addition to specified list of addresses thin
> client
> > > collect list of "connectable" nodes from topology (extendable list).
> > >
> > > What do you think?
> > >
> > >
> > > On Wed, Jan 31, 2018 at 5:14 PM, Pavel Tupitsyn 
> > > wrote:
> > >
> > > > Igniters,
> > > >
> > > > I'm working on client-side failover logic for .NET Thin Client.
> > > > This will probably apply to ODBC and JDBC thin clients as well in
> > future.
> > > >
> > > > Currently all thin clients connect to a single specified Ignite node.
> > > > The idea is to have multiple known nodes (host:port pairs) and
> > reconnect
> > > > to another node if current one goes down.
> > > >
> > > > Problems:
> > > > - Protocol is stateful, server keeps track of query cursors for the
> > > session
> > > > - Many operations are not idempotent, so retry is not an option
> > > > - Async operations and multithreading are supported in .NET thin
> client
> > > >
> > > > So while we can detect socket connection failure and reconnect to a
> > > > different node,
> > > > all currently executing client operations and query cursors will
> still
> > > fail
> > > > with an exception.
> > > >
> > > > I'm not sure how useful this behavior will be.
> > > > Any thoughts, ideas?
> > > >
> > > > Thanks,
> > > > Pavel
> > > >
> > >
> > >
> > >
> > > --
> > > Alexey Kuznetsov
> > >
> >
>

[GitHub] ignite pull request #3463: IGNITE-7489 Factor total allocated pages in data ...

[alamar]

GitHub user alamar opened a pull request:

https://github.com/apache/ignite/pull/3463

IGNITE-7489 Factor total allocated pages in data region's fillFactor.

… add test.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7489

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3463.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3463


commit aa35af86b3f06257add6d5a8b1e3098e721729d0
Author: Ilya Kasnacheev 
Date:   2018-02-01T13:45:12Z

IGNITE-7489 Factor total allocated pages in data region's fillFactor, add 
test.




---

[jira] [Created] (IGNITE-7602) SQL COPY: add tests for file reading failures

[Kirill Shirokov (JIRA)]

Kirill Shirokov created IGNITE-7602:
---

 Summary: SQL COPY: add tests for file reading failures
 Key: IGNITE-7602
 URL: https://issues.apache.org/jira/browse/IGNITE-7602
 Project: Ignite
  Issue Type: Bug
  Components: sql
Affects Versions: 2.4
Reporter: Kirill Shirokov






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Stop nodes after test by default - IGNITE-6842

[Dmitry Pavlov]

Hi Maxim,

I would be happy if this issue is completed.

I think best solultion is
1) to validate remained up and running nodes
2) and fail test if not-stopped nodes remained

instead of silently stop node. If nodes will be stopped silently, probable
errors may be missed both in test and in Ignite code.

Sincerely,
Dmitriy Pavlov

чт, 1 февр. 2018 г. в 3:54, Denis Magda :

> Hello Maxim,
>
> Granted you all the required permissions in JIRA. Feel free to assign the
> ticket to your account.
>
> In general, review how the testing framework works in Ignite and you’ll
> get why we stop all the nodes manually. It will help to get answers on most
> of the questions. Please propose your solution or ask for advice here after
> that.
>
> —
> Denis
>
> > On Jan 31, 2018, at 8:01 AM, Maxim Muzafarov  wrote:
> >
> > Hello everyone!
> >
> > I would like to take this one issue for implicating myself into Ignite
> > community.
> >
> > https://issues.apache.org/jira/browse/IGNITE-6842
> >
> > Can you help me with granting contributor permissons?
> > My JIRA ID: mmuzaf
> > e-mail: maxmu...@gmail.com
> > So I can assign this ticket for myself, If you don't mind of course.
> >
> > Also, it will be great to have something like general roadmap for fixing
> > this issue. What should I look at on my first step?
>
>

Re: Eviction policies with persistence

[Dmitry Pavlov]

Hi Vyacheslav,

Yes, this is right, but for now PDS page based eviction uses Random-LRU,
not Random-2-LRU. Something may be changed in future Ignite releases, but
now Random-LRU is used. I am going to research if there is some room for
improvements in this aspect (e.g. IGNITE-7507 recenly found).

If clean page is evicted, nothing really happends. If dirty page is evicted
from region during checkpointing process run, page is stored in file before
eviction

Some info about this can be found in
https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-underthehood-Pagebasedeviction

Sincerely,
Dmitriy Pavlov

чт, 1 февр. 2018 г. в 9:53, Vyacheslav Daradur :

> Hi Valentin,
>
> As far as I know, when persistence is enabled and Ignite can't
> allocate new page-memory in the memory segment, then Ignite
> automatically evict some data from RAM to PDS using Random-2-LRU
> eviction algorithm. And there are no mechanisms to evict entries out
> when PDS is enabled.
>
> I'll be glad if I am not right and someone will correct me because, as
> Ignite user, I can't use PDS only as RAM disk replication for the use
> case in my project.
>
>
> On Thu, Feb 1, 2018 at 3:53 AM, Valentin Kulichenko
>  wrote:
> > Folks,
> >
> > On "Eviction Policies" documentation page [1] we have the following
> callout:
> >
> >> Configured eviction policy has no effect if Ignite persistence is
> enabled
> >> Note that if Ignite Persistence is enabled, then the page-based
> evictions
> > have no effect because the oldest pages will be purged from memory
> > automatically.
> >
> > This really confuses me. Why is there a difference in how data is evicted
> > from memory depending on having disk enabled or not? And how does
> eviction
> > work with persistence enabled then?
> >
> > Does anyone can clarify?
> >
> > [1] https://apacheignite.readme.io/docs/evictions
> >
> > -Val
>
>
>
> --
> Best Regards, Vyacheslav D.
>

[jira] [Created] (IGNITE-7601) SQL COPY: add tests for implicit columns (_key, _val, _ver)

[Kirill Shirokov (JIRA)]

Kirill Shirokov created IGNITE-7601:
---

 Summary: SQL COPY: add tests for implicit columns (_key, _val, 
_ver)
 Key: IGNITE-7601
 URL: https://issues.apache.org/jira/browse/IGNITE-7601
 Project: Ignite
  Issue Type: Bug
  Components: sql
Affects Versions: 2.4
Reporter: Kirill Shirokov






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3462: Ignite 2.4.3

[ilantukh]

GitHub user ilantukh opened a pull request:

https://github.com/apache/ignite/pull/3462

Ignite 2.4.3



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.4.3

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3462.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3462


commit 4925ffc10ce8e8287980eaec38b587512568a302
Author: Alexey Goncharuk 
Date:   2018-01-17T12:26:03Z

IGNITE-7453 Use GridUnsafe.cleanDirectBuffer in PageSnapshot

commit bcd68a058683b2f17b7ac60471b6e7aab3e4f6de
Author: Pavel Tupitsyn 
Date:   2018-01-17T12:38:20Z

IGNITE-7301 .NET: Baseline topology

This closes #3352

commit 66b96316a7775ce8a6e2ff4475185d5929e4998b
Author: devozerov 
Date:   2018-01-17T12:54:17Z

Merge branch 'master' into ignite-2.4

commit 268481c1cf7fe57df24be130eb67c3e3a13afe01
Author: Alexey Goncharuk 
Date:   2018-01-17T13:50:34Z

IGNITE-7453 Use GridUnsafe.cleanDirectBuffer in WalStat

commit db0cd105719c8ae713b13b34d9dca0a8cd45d377
Author: Pavel Tupitsyn 
Date:   2018-01-17T14:05:25Z

IGNITE-6776 .NET: Thin client: Add SQL & LINQ example

This closes #3390

commit c214db879101aa5660e2a50b11cd20964c0bc114
Author: Andrey Gura 
Date:   2018-01-17T12:42:41Z

ignite-7450 FileWriteAheadLogManager always uses RandomAccessFileIOFactory 
now

commit 75c27d5e49d7458e46eb46e6f87a445c3f1320ea
Author: Alexey Kuznetsov 
Date:   2018-01-18T02:25:19Z

IGNITE-7274 Web Console: Support multiple statements on Queries screen.
(cherry picked from commit 1926783)

commit 36cc822935387b2150e690c29bc6992dca0563f7
Author: Dmitriy Shabalin 
Date:   2018-01-18T04:49:08Z

IGNITE-7306 Web Console: Fixed export data from tables.
(cherry picked from commit 1bb60ec)

commit d753298b4012894b56f5c9218325211cd84a21d5
Author: Peter Ivanov 
Date:   2018-01-18T06:18:53Z

IGNITE-7107 Apache Ignite RPM packages

* added changelog to package specification

This closes #3396

commit 63445893f1bc75dc9777184499f7eabc1d4e51b1
Author: Denis Mekhanikov 
Date:   2018-01-18T08:36:18Z

IGNITE-3935 Use PeerDeployAware for streamer transformer - Fixes #3378.

Signed-off-by: Alexey Goncharuk 

commit f3f9f2a24b23027cf0c835140322e41a788932ae
Author: Pavel Tupitsyn 
Date:   2018-01-18T09:05:12Z

IGNITE-7413 Fix SqlDmlExample

This closes #3389

commit 1daa7c41bf1460a4d9a2b0c26a7a317f2fca3fb7
Author: Alexey Kuznetsov 
Date:   2018-01-18T10:14:53Z

IGNITE-7461 UI tools: Actualized data storage configuration.
(cherry picked from commit 577e632)

commit cf0080210d24d9dd8b057f959446fac5f8a4ca01
Author: dpavlov 
Date:   2018-01-18T10:53:29Z

IGNITE-7380 Implemented pluggable Direct IO - Fixes #3226.

Signed-off-by: Alexey Goncharuk 

commit dd06d0bd7ef266bfbe156e858b312d1ac86e8982
Author: Pavel Tupitsyn 
Date:   2018-01-18T12:55:49Z

IGNITE-7465 .NET: Fix SqlDdlExample failure with standalone node

commit 57479ec564e1761716da3d5f9feb7a64c396a9f9
Author: Pavel Tupitsyn 
Date:   2018-01-18T13:45:54Z

.NET: Fix CacheLocalTest.TestTxDeadlockDetection

commit bd6be8a4653322905a3b63850c7e033ce3801ce5
Author: Pavel Tupitsyn 
Date:   2018-01-18T18:25:05Z

.NET: Thin client: Fix OP_BINARY_TYPE_GET schema passing format

commit 97564d160586d6d57d300937e6b8877994e35fc7
Author: rkondakov 
Date:   2018-01-19T08:24:51Z

IGNITE-6456: Ability to separately enable or disable JDBC, ODBC and thin 
client endpoints. This closes #3309.

commit d50274ca8875c9680c12e8786ac355a787ba95e0
Author: Yakov Zhdanov 
Date:   2018-01-18T17:57:17Z

Javadoc enhancements - added @see

commit cb2d3cf22388ab19fb2d34ae5bdfc8f1b608db75
Author: Dmitriy Govorukhin 
Date:   2018-01-18T14:14:26Z

IGNITE-7471 Use soft reference for checkpoint entry contents to avoid 
excessive memory usage

commit 3965923369870bb4e8e57e3332c1a1eb1e5f5ed3
Author: rkondakov 
Date:   2018-01-19T09:00:55Z

IGNITE-6772: SQL exception messages became more informative. This closes 
#3342.

commit ba68cb0fa87f776fcd0499d030c333f182611f41
Author: devozerov 
Date:   2018-01-19T09:03:52Z

Merge remote-tracking branch 'origin/ignite-2.4' into ignite-2.4

commit b54c0c8786bd74aa0abb208f537c29f0c4be4b1e
Author: tledkov-gridgain 
Date:   2018-01-19T09:09:34Z

IGNITE-7248: JDBC: fixed schema resolution for streaming mode. This closes 
#3384.

Ignite SQL: How to drop table without explicit cache name

[Nikolay Izhikov]

Hello, guys.

I working on support of saving Spark DataFrame to Ignite [1].

I want to execute "DROP TABLE XXX" query.
To execute some SQL query in regulary way one need some cache.
That cache has to differs from table cache to execute "DROP TABLE" successfully.

I founded 3 different ways to execute SQL query without explicit cache name.
Please, tell me which way is right?
Do we need all 3 way to make a query?

1. JdbcRequestHandler - [2]

```

private final GridKernalContext ctx;
...
List> results = ctx.query().querySqlFields(qry, 
true, protocolVer.compareTo(VER_2_3_0) < 0);

```

2. GridCacheProcessor - [3]

```

/**
* Gets public cache that can be used for query execution.
* If cache isn't created on current node it will be started.
*/
public IgniteCacheProxy getOrStartPublicCache(boolean start, boolean 
inclLoc) throws IgniteCheckedException {

```

3. QueryCommandHandler - [4]

```

protected static final String DFLT_CACHE_NAME = "default";
... 
String cacheName = req.cacheName() == null ? DFLT_CACHE_NAME : 
req.cacheName();

```


[1] https://issues.apache.org/jira/browse/IGNITE-7337
[2] 
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/odbc/jdbc/JdbcRequestHandler.java#L310
[3] 
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheProcessor.java#L1648
 
[4] 
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/rest/handlers/query/QueryCommandHandler.java#L318

signature.asc
Description: This is a digitally signed message part

[GitHub] ignite pull request #3457: IGNITE-7561 .NET: Add IServices.GetDynamicService...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3457


---

[GitHub] ignite pull request #3418: Ignite 6899 check

[oignatenko]

Github user oignatenko closed the pull request at:

https://github.com/apache/ignite/pull/3418


---

[GitHub] ignite pull request #3461: IGNITE-7309 Throw NodeStoppingException if marsha...

[gromtech]

GitHub user gromtech opened a pull request:

https://github.com/apache/ignite/pull/3461

IGNITE-7309 Throw NodeStoppingException if marshaling fails while node 
stopping

* Wrapped error log (U.error) at GridJobWorker.onJobFinished to check node 
stopping
* Add GridJobWorkerTest

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7309

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3461.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3461


commit d9ec315c62c542b13844a35617caaa3a4e58ec73
Author: Roman Guseinov 
Date:   2018-01-31T10:25:58Z

IGNITE-7309 Throw NodeStoppingException if marshaling fail while node 
stopping

commit 4f3fd63eddfc76322e38c4bbbc6e6014a3012102
Author: Roman Guseinov 
Date:   2018-02-01T10:25:25Z

IGNITE-7309 Added GridJobWorkerTest




---

[jira] [Created] (IGNITE-7600) visorcmd: add column 'type' for 'mlist' result

[Pavel Konstantinov (JIRA)]

Pavel Konstantinov created IGNITE-7600:
--

 Summary: visorcmd: add column 'type' for 'mlist' result
 Key: IGNITE-7600
 URL: https://issues.apache.org/jira/browse/IGNITE-7600
 Project: Ignite
  Issue Type: Improvement
Reporter: Pavel Konstantinov


It should print type of variable. F.e. n0..nX - node, h0..hX - host



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3460: IGNITE-7599: Missed licenses in ignite-direct-io ...

[dspavlov]

GitHub user dspavlov opened a pull request:

https://github.com/apache/ignite/pull/3460

IGNITE-7599: Missed licenses in ignite-direct-io module



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7599

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3460.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3460


commit 0088404eab5455b1436ac55b8361997429e196b1
Author: dpavlov 
Date:   2018-02-01T09:52:30Z

IGNITE-7599: Missed licenses in ignite-direct-io module




---

Re: Annoying extra steps for enabling metrics

[Anton Vinogradov]

Folks,

.setEventsDisabled looks to be a good solution, since we will always call
it like .setEventsDisabled(true).

Calling .setEventsEnabled(false), since true is a default, looks odd to me.

On Wed, Jan 31, 2018 at 11:02 PM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Alex,
>
> As a workaround, you can use boxed Boolean as a field type, and then return
> true from getEventsEnabled in case it's null. Will this work?
>
> -Val
>
> On Wed, Jan 31, 2018 at 7:31 AM, Alex Plehanov 
> wrote:
>
> > Denis, there is a question about IGNITE-7346.
> >
> > CacheConfiguration fields are set to they default values when cache
> > configuration is created by constructor. When CacheConfiguration is
> created
> > by deserialization (from another node or from PDS), constructor is not
> > called. If it was serialized by older version of Ignite, a new added
> > boolean fields are set to false after deserialization by a new Ignite
> > versions. We can't change this behavior without methods for custom
> > serialization/deserialization (which are not implemented now in
> > CacheConfiguration class). It will differ from requirements for "Eve
> > ntsEnabled" property (events must be enabled for caches by default).
> >
> > I think we can reverse the logic and rename method
> > CacheConfiguration.setEventsEnabled
> > to CacheConfiguration.setEventsDisabled, in this case the field value
> will
> > always be "false" by default.
> >
> > What do you think about it?
> >
> > 2017-12-30 10:12 GMT+03:00 Alex Plehanov :
> >
> > > Due to holidays I can start work on this ticket only after 8 jan 2018
> > >
> > > 2017-12-30 2:12 GMT+03:00 Denis Magda :
> > >
> > >> Good, closed the original ticket.
> > >>
> > >> Alex P, do you have time to work on IGNITE-7346 instead to address the
> > >> issue with the cache events per cache in 2.4 release?
> > >>
> > >> —
> > >> Denis
> > >>
> > >> > On Dec 29, 2017, at 3:10 PM, Valentin Kulichenko <
> > >> valentin.kuliche...@gmail.com> wrote:
> > >> >
> > >> > Agree.
> > >> >
> > >> > -Val
> > >> >
> > >> > On Fri, Dec 29, 2017 at 3:08 PM, Denis Magda 
> > wrote:
> > >> >
> > >> >> Now I see. Seems I was doing something wrong in my initial
> > reproducer.
> > >> >>
> > >> >> Updated cache metrics readme doc by purging any events related
> > >> parameters
> > >> >> from there:
> > >> >> https://apacheignite.readme.io/v2.3/docs/cache-metrics <
> > >> >> https://apacheignite.readme.io/v2.3/docs/cache-metrics>
> > >> >>
> > >> >> The events readme doc looks good to me. Just updated the javadoc of
> > >> >> IgniteEvents#*Query methods to bring MemoryEventStorageSpi to users
> > >> >> attention.
> > >> >>
> > >> >> As for the cache events, it’s an oversight that there is no
> parameter
> > >> that
> > >> >> enables/disables the events per cache. Let’s add
> > >> CacheConfiguration.setEventsEnabled
> > >> >> method and have it enabled by default (current behavior). If the
> user
> > >> >> deploys hundreds of caches or just interested in the events of
> > specific
> > >> >> ones then he can always set this property to ‘false’ in
> configuration
> > >> of
> > >> >> the caches of no interest:
> > >> >> https://issues.apache.org/jira/browse/IGNITE-7346 <
> > >> >> https://issues.apache.org/jira/browse/IGNITE-7346>
> > >> >>
> > >> >> Agree?
> > >> >>
> > >> >> —
> > >> >> Denis
> > >> >>
> > >> >>
> > >> >>
> > >> >>> On Dec 29, 2017, at 11:10 AM, Valentin Kulichenko <
> > >> >> valentin.kuliche...@gmail.com> wrote:
> > >> >>>
> > >> >>> Guys,
> > >> >>>
> > >> >>> I'm not sure what issue we're trying to solve. Basically, we have
> > >> three
> > >> >>> different functionality parts here:
> > >> >>>
> > >> >>> 1. Cache metrics exposed via CacheMetrics interface and MBeans
> > >> (number of
> > >> >>> puts, average put time, this kind of stuff). These are controlled
> on
> > >> per
> > >> >>> cache level by CacheConfiguration#statisticsEnabled property.
> > >> >>> 2. Listening to cache events. To achieve this you only need to
> > enable
> > >> >>> particular event types and register listeners, this does not
> depend
> > on
> > >> >>> CacheConfiguration#statisticsEnabled. Here is the test confirming
> > >> this:
> > >> >>> https://gist.github.com/vkulichenko/
> 54043c7b75c69eca5515e7d7692b52
> > 76
> > >> >>> 3. Querying cache events via IgniteEvents#*Query methods. This is
> > the
> > >> >> ONLY
> > >> >>> piece that requires MemoryEventStorageSpi to be configured. Since
> > >> >> querying
> > >> >>> is not very frequently used, and event storage introduces
> > significant
> > >> >>> memory overhead, I don't think we should ever implicitly enable
> it.
> > We
> > >> >>> deliberately made no-op implementation the default one not so long
> > >> ago.
> > >> >>>
> > >> >>> All three points above seem to work without any issues. The only
> > >> useful
> > >> >>> addition I see here is ability to enable cache events on 

[GitHub] ignite pull request #3459: IGNITE-7590: Change example for decision trees

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3459


---

[jira] [Created] (IGNITE-7599) Missed licenses in apache-ignite-io module

[Sergey Kozlov (JIRA)]

Sergey Kozlov created IGNITE-7599:
-

 Summary: Missed licenses in apache-ignite-io module 
 Key: IGNITE-7599
 URL: https://issues.apache.org/jira/browse/IGNITE-7599
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Sergey Kozlov
 Fix For: 2.4


{\{libs/optional/apache-ignite-io/licenses}} has no Apache Ignite licenses



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7598) SQL: INSERT requires you to specify the column names even if you are adding values for all the columns of the table

[Roman Shtykh (JIRA)]

Roman Shtykh created IGNITE-7598:


 Summary: SQL: INSERT requires you to specify the column names even 
if you are adding values for all the columns of the table
 Key: IGNITE-7598
 URL: https://issues.apache.org/jira/browse/IGNITE-7598
 Project: Ignite
  Issue Type: Bug
  Components: sql
Reporter: Roman Shtykh


Checked via JDBC thin.
{noformat}
Caused by: org.h2.jdbc.JdbcSQLException: Column count does not match; SQL 
statement:
INSERT INTO MYTABLE VALUES (?,?,?,?,?,?,?,?,?,?,?) [21002-195]
    at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
    at org.h2.message.DbException.get(DbException.java:179)
    at org.h2.message.DbException.get(DbException.java:155)
    at org.h2.message.DbException.get(DbException.java:144)
    at org.h2.command.dml.Insert.prepare(Insert.java:265){noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)