date:20171019

On Thursday, October 19, 2017, Dmitriy Setrakyan 
wrote:

> Denis,
>
> Awesome work and great contribution!
>
> A minor comment. Does SELECT belong under DML?


Thanks for feedback!

It turned out that yes, SELECT belongs to DML. Personally, I deciphered ‘M’
as ‘modification’ before while in fact it stands for ‘manipulation’.

Some proof links:
https://en.m.wikipedia.org/wiki/Data_manipulation_language
http://www.h2database.com/html/grammar.html

Denis

>
> D.
>
>
> On Thu, Oct 19, 2017 at 6:36 PM, Denis Magda  > wrote:
>
> > Igniters,
> >
> > I’m glad to share that the new SQL doc is ready for proof read:
> > https://apacheignite-sql.readme.io/docs
> >
> > Main sections:
> > - Overview - SQL capabilities outline and getting started guide.
> > - SQL Reference - SQL itself with all the commands and operations.
> > - Architecture - insights on how SQL engine works and some of its
> features.
> > - JDBC
> > - ODBC
> > - Tools and Analytics - tools to work with Ignite as with SQL database.
> > -  Java Dev Guide - Java specific features
> > - .NET Dev Guide - .NET specific features
> > - C++ Dev Guide - C++ specific features
> >
> > Note, that all the SQL related pages will be hidden (consider removed) on
> > Java, .NET and C++ readme.io sites in 2.3. All the SQL related
> > capabilities have to be documented on this new specific doc.
> >
> > Here is we’re tracking minor documentation tasks to close before the
> > release:
> > https://issues.apache.org/jira/browse/IGNITE-6685
> >
> > Many thanks to Sergey Puchnin for the help with SQL reference
> > documentation!
> >
> > —
> > Denis
> >
> >
> > > On Sep 14, 2017, at 5:00 AM, Serge Puchnin  >
> > wrote:
> > >
> > > Denis,
> > > Thanks a lot for the link.
> > > It's very helpful to split "function part" to more specific like
> > 'string',
> > > 'numeric', etc.
> > >
> > > the main reason to separate "Java Developer" as an independent part
> > because
> > > it's about a mix between Java and SQL code.
> > > It's about classes, methods, annotation, GC, etc.
> > > It doesn't fully belong to the SQL specification but IMHO should be in
> > SQL
> > > category.
> > >
> > > BR,
> > > Serge
> > >
> > > On Wed, 13 Sep 2017 at 22:39, Denis Magda  > wrote:
> > >
> > >> Serge,
> > >>
> > >> I like the proposed format. It reminds me the one chosen by MemSQL
> which
> > >> is crystal clear:
> > >> https://docs.memsql.com/sql-reference/v5.8/sql-reference-overview/
> > >>
> > >> Not 100% sure about “Java Developer Guides” section. Let me
> contemplate
> > a
> > >> bit and propose other solutions.
> > >>
> > >> —
> > >> Denis
> > >>
> > >> On Sep 13, 2017, at 3:41 AM, Serge Puchnin  >
> > >> wrote:
> > >>
> > >> I have done some reorganizing for the SQL documentation.
> > >> Thanks a lot for the review and proof read.
> > >>
> > >>
> > >>
> > >> On Wed, 13 Sep 2017 at 01:22, Dmitriy Setrakyan <
> dsetrak...@apache.org >
> > >> wrote:
> > >>
> > >>> Thanks Sergey, I think we should add SQL Syntax section on the left.
> > Also,
> > >>> please let us know when we can proof read it. I would like to fix
> some
> > >>> phrasing on the page.
> > >>>
> > >>> On Tue, Sep 12, 2017 at 8:26 AM, Serge Puchnin <
> > sergey.puch...@gmail.com >
> > >>> wrote:
> > >>>
> >  I've added a page for SQL language, create index statement and Upper
> >  function as an example for the new format:
> > 
> >  https://apacheignite-sql.readme.io/v2.1/docs/create-index
> > 
> > 
> > 
> >  For me, it'd be OK to add a separate page for each SQL statement.
> > 
> > 
> >  BR,
> > 
> >  Serge
> > 
> >  On Sat, 9 Sep 2017 at 02:19, Denis Magda  > wrote:
> > 
> > > Please check you inbox.
> > >
> > > —
> > > Denis
> > >
> > >> On Sep 8, 2017, at 3:38 PM, Dmitriy Setrakyan <
> > >>> dsetrak...@apache.org >
> > > wrote:
> > >>
> > >> Denis,
> > >>
> > >> Can you please add me as an admin to that site?
> > >>
> > >> D.
> > >>
> > >> On Fri, Sep 8, 2017 at 3:23 PM, Denis Magda  >
> > >>> wrote:
> > >>
> > >>> Igniter (especially those who are willing to work on the new SQL
> > >>> doc),
> > >>>
> > >>> The doc for SQL capabilities has been set up. Copy pasted all the
> > >>> SQL
> > >>> materials we have there:
> > >>> https://apacheignite-sql.readme.io/docs <
> https://apacheignite-sql.
> > >>> readme.io/docs>
> > >>>
> > >>> Sergey P., feel free to start reworking the doc according to the
> > >>> new
> > >>> proposed format. Don’t hesitate removing outdating or misleading
> > > content.
> > >>> If you’re uncertain about something just send us a message here.
> > >>>
> >

[GitHub] ignite pull request #2845: IGNITE-6519 Race in SplitAwareTopologyValidator o...

2017-10-19 Thread akuramshingg

Github user akuramshingg closed the pull request at:

https://github.com/apache/ignite/pull/2845


---

Re: sql documentation

Denis,

Awesome work and great contribution!

A minor comment. Does SELECT belong under DML?

D.


On Thu, Oct 19, 2017 at 6:36 PM, Denis Magda  wrote:

> Igniters,
>
> I’m glad to share that the new SQL doc is ready for proof read:
> https://apacheignite-sql.readme.io/docs
>
> Main sections:
> - Overview - SQL capabilities outline and getting started guide.
> - SQL Reference - SQL itself with all the commands and operations.
> - Architecture - insights on how SQL engine works and some of its features.
> - JDBC
> - ODBC
> - Tools and Analytics - tools to work with Ignite as with SQL database.
> -  Java Dev Guide - Java specific features
> - .NET Dev Guide - .NET specific features
> - C++ Dev Guide - C++ specific features
>
> Note, that all the SQL related pages will be hidden (consider removed) on
> Java, .NET and C++ readme.io sites in 2.3. All the SQL related
> capabilities have to be documented on this new specific doc.
>
> Here is we’re tracking minor documentation tasks to close before the
> release:
> https://issues.apache.org/jira/browse/IGNITE-6685
>
> Many thanks to Sergey Puchnin for the help with SQL reference
> documentation!
>
> —
> Denis
>
>
> > On Sep 14, 2017, at 5:00 AM, Serge Puchnin 
> wrote:
> >
> > Denis,
> > Thanks a lot for the link.
> > It's very helpful to split "function part" to more specific like
> 'string',
> > 'numeric', etc.
> >
> > the main reason to separate "Java Developer" as an independent part
> because
> > it's about a mix between Java and SQL code.
> > It's about classes, methods, annotation, GC, etc.
> > It doesn't fully belong to the SQL specification but IMHO should be in
> SQL
> > category.
> >
> > BR,
> > Serge
> >
> > On Wed, 13 Sep 2017 at 22:39, Denis Magda  wrote:
> >
> >> Serge,
> >>
> >> I like the proposed format. It reminds me the one chosen by MemSQL which
> >> is crystal clear:
> >> https://docs.memsql.com/sql-reference/v5.8/sql-reference-overview/
> >>
> >> Not 100% sure about “Java Developer Guides” section. Let me contemplate
> a
> >> bit and propose other solutions.
> >>
> >> —
> >> Denis
> >>
> >> On Sep 13, 2017, at 3:41 AM, Serge Puchnin 
> >> wrote:
> >>
> >> I have done some reorganizing for the SQL documentation.
> >> Thanks a lot for the review and proof read.
> >>
> >>
> >>
> >> On Wed, 13 Sep 2017 at 01:22, Dmitriy Setrakyan 
> >> wrote:
> >>
> >>> Thanks Sergey, I think we should add SQL Syntax section on the left.
> Also,
> >>> please let us know when we can proof read it. I would like to fix some
> >>> phrasing on the page.
> >>>
> >>> On Tue, Sep 12, 2017 at 8:26 AM, Serge Puchnin <
> sergey.puch...@gmail.com>
> >>> wrote:
> >>>
>  I've added a page for SQL language, create index statement and Upper
>  function as an example for the new format:
> 
>  https://apacheignite-sql.readme.io/v2.1/docs/create-index
> 
> 
> 
>  For me, it'd be OK to add a separate page for each SQL statement.
> 
> 
>  BR,
> 
>  Serge
> 
>  On Sat, 9 Sep 2017 at 02:19, Denis Magda  wrote:
> 
> > Please check you inbox.
> >
> > —
> > Denis
> >
> >> On Sep 8, 2017, at 3:38 PM, Dmitriy Setrakyan <
> >>> dsetrak...@apache.org>
> > wrote:
> >>
> >> Denis,
> >>
> >> Can you please add me as an admin to that site?
> >>
> >> D.
> >>
> >> On Fri, Sep 8, 2017 at 3:23 PM, Denis Magda 
> >>> wrote:
> >>
> >>> Igniter (especially those who are willing to work on the new SQL
> >>> doc),
> >>>
> >>> The doc for SQL capabilities has been set up. Copy pasted all the
> >>> SQL
> >>> materials we have there:
> >>> https://apacheignite-sql.readme.io/docs  >>> readme.io/docs>
> >>>
> >>> Sergey P., feel free to start reworking the doc according to the
> >>> new
> >>> proposed format. Don’t hesitate removing outdating or misleading
> > content.
> >>> If you’re uncertain about something just send us a message here.
> >>>
> >>> Presently the doc is not indexed by searching engines. Let’s
> >>> finish it
> >>> first and announce afterwards.
> >>>
> >>> —
> >>> Denis
> >>>
>  On Aug 30, 2017, at 5:43 PM, Dmitriy Setrakyan <
>  dsetrak...@apache.org>
> >>> wrote:
> 
>  On Wed, Aug 30, 2017 at 10:04 PM, Denis Magda 
> > wrote:
> 
> > Serge,
> >
> > Undoubtedly, I like this format. I’ve created a new documentation
> > domain
> > for all our SQL capabilities:
> > https://apacheignite-sql.readme.io/docs <
> >>> https://apacheignite-sql.
> > readme.io/docs>
> >
> > Once I get an open source license for it we can start moving the
> >>> existing
> > docs and create the new there.
>

Re: sql documentation

Igniters,

I’m glad to share that the new SQL doc is ready for proof read:
https://apacheignite-sql.readme.io/docs

Main sections:
- Overview - SQL capabilities outline and getting started guide.
- SQL Reference - SQL itself with all the commands and operations.
- Architecture - insights on how SQL engine works and some of its features.
- JDBC
- ODBC
- Tools and Analytics - tools to work with Ignite as with SQL database.
-  Java Dev Guide - Java specific features
- .NET Dev Guide - .NET specific features
- C++ Dev Guide - C++ specific features

Note, that all the SQL related pages will be hidden (consider removed) on Java, 
.NET and C++ readme.io sites in 2.3. All the SQL related capabilities have to 
be documented on this new specific doc.

Here is we’re tracking minor documentation tasks to close before the release:
https://issues.apache.org/jira/browse/IGNITE-6685

Many thanks to Sergey Puchnin for the help with SQL reference documentation!

—
Denis


> On Sep 14, 2017, at 5:00 AM, Serge Puchnin  wrote:
> 
> Denis,
> Thanks a lot for the link.
> It's very helpful to split "function part" to more specific like 'string',
> 'numeric', etc.
> 
> the main reason to separate "Java Developer" as an independent part because
> it's about a mix between Java and SQL code.
> It's about classes, methods, annotation, GC, etc.
> It doesn't fully belong to the SQL specification but IMHO should be in SQL
> category.
> 
> BR,
> Serge
> 
> On Wed, 13 Sep 2017 at 22:39, Denis Magda  wrote:
> 
>> Serge,
>> 
>> I like the proposed format. It reminds me the one chosen by MemSQL which
>> is crystal clear:
>> https://docs.memsql.com/sql-reference/v5.8/sql-reference-overview/
>> 
>> Not 100% sure about “Java Developer Guides” section. Let me contemplate a
>> bit and propose other solutions.
>> 
>> —
>> Denis
>> 
>> On Sep 13, 2017, at 3:41 AM, Serge Puchnin 
>> wrote:
>> 
>> I have done some reorganizing for the SQL documentation.
>> Thanks a lot for the review and proof read.
>> 
>> 
>> 
>> On Wed, 13 Sep 2017 at 01:22, Dmitriy Setrakyan 
>> wrote:
>> 
>>> Thanks Sergey, I think we should add SQL Syntax section on the left. Also,
>>> please let us know when we can proof read it. I would like to fix some
>>> phrasing on the page.
>>> 
>>> On Tue, Sep 12, 2017 at 8:26 AM, Serge Puchnin 
>>> wrote:
>>> 
 I've added a page for SQL language, create index statement and Upper
 function as an example for the new format:
 
 https://apacheignite-sql.readme.io/v2.1/docs/create-index
 
 
 
 For me, it'd be OK to add a separate page for each SQL statement.
 
 
 BR,
 
 Serge
 
 On Sat, 9 Sep 2017 at 02:19, Denis Magda  wrote:
 
> Please check you inbox.
> 
> —
> Denis
> 
>> On Sep 8, 2017, at 3:38 PM, Dmitriy Setrakyan <
>>> dsetrak...@apache.org>
> wrote:
>> 
>> Denis,
>> 
>> Can you please add me as an admin to that site?
>> 
>> D.
>> 
>> On Fri, Sep 8, 2017 at 3:23 PM, Denis Magda 
>>> wrote:
>> 
>>> Igniter (especially those who are willing to work on the new SQL
>>> doc),
>>> 
>>> The doc for SQL capabilities has been set up. Copy pasted all the
>>> SQL
>>> materials we have there:
>>> https://apacheignite-sql.readme.io/docs >> readme.io/docs>
>>> 
>>> Sergey P., feel free to start reworking the doc according to the
>>> new
>>> proposed format. Don’t hesitate removing outdating or misleading
> content.
>>> If you’re uncertain about something just send us a message here.
>>> 
>>> Presently the doc is not indexed by searching engines. Let’s
>>> finish it
>>> first and announce afterwards.
>>> 
>>> —
>>> Denis
>>> 
 On Aug 30, 2017, at 5:43 PM, Dmitriy Setrakyan <
 dsetrak...@apache.org>
>>> wrote:
 
 On Wed, Aug 30, 2017 at 10:04 PM, Denis Magda 
> wrote:
 
> Serge,
> 
> Undoubtedly, I like this format. I’ve created a new documentation
> domain
> for all our SQL capabilities:
> https://apacheignite-sql.readme.io/docs <
>>> https://apacheignite-sql.
> readme.io/docs>
> 
> Once I get an open source license for it we can start moving the
>>> existing
> docs and create the new there.
> 
 
 Sounds good. We should make sure that we do not "move" the
> documentation,
 we should "copy" it. Otherwise we will get 404 errors in google.
 
 
> 
> —
> Denis
> 
>> On Aug 30, 2017, at 4:38 AM, Serge Puchnin n <
> sergey.puch...@gmail.com
 
> wrote:
>> 
>> Sorry for inconvenience!
>> there is a link

Re: SQL error codes

Yes, we should document as much as we can here. Created a ticket in advance so 
that this is solved before 2.3 release:
https://issues.apache.org/jira/browse/IGNITE-6687 


—
Denis

> On Oct 19, 2017, at 3:51 PM, Alexander Paschenko  
> wrote:
> 
> I meant «that “commonization”», sorry.
> 
> пт, 20 окт. 2017 г. в 1:49, Alexander Paschenko  >:
> I doubt that codes that are common between ODBC and JDBC really need that 
> “communication”. Those are different sets of codes documented independently.
> On the other hand, Ignite specific codes thrown when queries are run via 
> Ignite API are effectively of a third kind. Vladimir, what do you think, 
> should we document those?
> 
> - Alex
> 
> пт, 20 окт. 2017 г. в 1:03, Dmitriy Setrakyan  >:
> On Thu, Oct 19, 2017 at 2:48 PM, Alexander Paschenko  > wrote:
> Dmitriy,
> 
> There are Ignite specific codes, they are returned when queries are run via 
> Ignite API. There are no codes explicitly shared by JDBC and ODBC, although 
> some of them happen to be the same.
> 
> Thanks, Alex. Now it sounds that we propagate these codes to JDBC and ODBC 
> drivers right? In that case, we should have Generic Error Codes pages in 
> documentation which lists all the common error codes, right?
>

[jira] [Created] (IGNITE-6687) SQL Error Codes Documentation

Denis Magda created IGNITE-6687:
---

 Summary: SQL Error Codes Documentation
 Key: IGNITE-6687
 URL: https://issues.apache.org/jira/browse/IGNITE-6687
 Project: Ignite
  Issue Type: Task
  Security Level: Public (Viewable by anyone)
Reporter: Denis Magda
Assignee: Vladimir Ozerov
 Fix For: 2.3


Presently we have JDBC error codes documented:
https://apacheignite-sql.readme.io/docs/jdbc-error-codes

Two things have to be done:
* If there are similar error codes shared between the ODBC and JDBC drivers, 
and native APIs then let's put them under SQL reference category below Data 
Types: https://apacheignite-sql.readme.io/docs/sql-reference-overview
* ODBC specific error codes with an example of an error handling need to be 
listed in ODBC category [~isapego], please assist with this: 
https://apacheignite-sql.readme.io/docs/odbc-driver

Discussion on the dev list:
http://apache-ignite-developers.2346864.n4.nabble.com/SQL-error-codes-td23415.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6686) Document SQL Data Types

Denis Magda created IGNITE-6686:
---

 Summary: Document SQL Data Types
 Key: IGNITE-6686
 URL: https://issues.apache.org/jira/browse/IGNITE-6686
 Project: Ignite
  Issue Type: Task
  Security Level: Public (Viewable by anyone)
Reporter: Denis Magda
Assignee: Vladimir Ozerov


The data types page is empty for now:
https://apacheignite-sql.readme.io/docs/data-types

It should include references to language or driver specific pages like the one 
we have for ODBC:
https://apacheignite-sql.readme.io/docs/odbc-data-types

Guys please help with the following (create a separate page like we have for 
ODBC):
# [~ptupitsyn] - .NET data types.
# [~isapego] - C++ data type.
# [~al.psc] - JDBC and JAVA data types.

[~vozerov], please coordinate the activity. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: CacheConfiguration reusage issues with EvictionPolicy enabled.

2017-10-19 Thread Valentin Kulichenko

Andrey,

Using factory instead of policy instance in configuration makes sense to
me, I think we should do this change.

As for Hadoop module issue, is it possible to do the mentioned validation
after creating the policy? I think that should fix the problem.

-Val

On Thu, Oct 19, 2017 at 7:50 AM Andrey Mashenkov 
wrote:

> Hi Igniters,
>
> I'm working on a ticket IGNITE-6649 [1] and got stuck.
> Currently, we allow user to set EvictionPolicy instance into configuration
> instead of factory.
> The leads to some isses when user tries to reuse EvictionPolicy instance,
> e.g. it doesn't clean its queue on cache stop.
>
> Seems, we should replace EvictionPolicy with its factory as policy objec is
> a statefull object and current approach is error prone.
>
> The issue I faced is that we make checks for EvicitonPolicy derives some
> known class in several code places.
> E.g. in HadoopModule you can find IgfsPerBlockLruEvictionPolicy usages.
> What will be correct way to overwork this with using facrories?
>
>
> Also, I've noticed that we check EvictionPolicy instances if they implement
> LifecycleAware interface and no one policy available out-of-box really
> implements it.
> The second way here is to make these EvictionPolicies implements
> LifecycleAware interface and gracefully clean their queues.
> This will resolve the issue for case when user recreate cache with same
> configuration instance,
> but seem will not help in case when user share EvictionPolicy instance for
> several caches.
>
>
> Also`EvictionPolicies has non-transient "queue" field, so queue may leak to
> another node.
> I'm not sure it can happens, I'll check.
>
> Thoughts?
>
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-6649
>
> --
> Best regards,
> Andrey V. Mashenkov
>

Re: SQL error codes

On Thu, Oct 19, 2017 at 2:48 PM, Alexander Paschenko <
apasche...@gridgain.com> wrote:

> Dmitriy,
>
> There are Ignite specific codes, they are returned when queries are run
> via Ignite API. There are no codes explicitly shared by JDBC and ODBC,
> although some of them happen to be the same.
>

Thanks, Alex. Now it sounds that we propagate these codes to JDBC and ODBC
drivers right? In that case, we should have Generic Error Codes pages in
documentation which lists all the common error codes, right?

Re: SQL error codes

On Thu, Oct 19, 2017 at 9:48 AM, Vladimir Ozerov 
wrote:

> No, they are not re-used. SQLSTATE is JDBC/ODBC-specific thing (and both
> driver types have different codes).
>

Vladimir, are there any error codes that come from the Ignite engine that
are shared between JDBC and ODBC drivers? What if the SQL execution is
triggered from Ignite API directly, what error codes are used then?

Re: Ignite Durable Memory & Persistent Store introduction for developers

2017-10-19 Thread Dmitry Pavlov

Hi Denis,

I've updated the section in relation to your questions, please see
https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-underthehood-IgniteDurableMemory

I’ve also moved paragraph related to page offset calculation and usage in
files into second article (PDS)

Sincerely,
Dmitriy Pavlov

пт, 1 сент. 2017 г. в 19:50, Denis Magda :

> Dmitriy,
>
> First of all, thanks for putting this low level details on paper! Current
> and future community members will be grateful to you.
>
> Second, let me be a reviewer of the page and suggest some improvements.
> Until the review is over I would encourage us to hold on with any public
> announcements such as blog post.
>
> I’ve reviewed and edited Ignite Durable Memory section (
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-underthehood-IgniteDurableMemory
> <
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-underthehood-IgniteDurableMemory>).
> Please address the questions below and I’ll move on with the new sections
> afterwards.
>
> 1) "Different caches and its partitions bring more complex addressing
> scheme but it is still possible to map from page ID to position in
> particular file: pageId => file + offset in this file”.
>
> Probably this appears too early in the doc because above we just
> introduced page ID notion with durable memory and now we’re talking about
> some files? Are these files of ignite persistence? What if the persistence
> is disabled? Honestly, the sentence gets out of the story context.
>
>
> 2) "Also, the tree pages can be merged into one if < 50% of space is used.”
>
> 50% of the whole memory available? Please clarify.
>
>
> 3) "There is one page ID table per memory segment.”
>
> It’s the same segment as it’s described in the main doc, right?
> https://apacheignite.readme.io/docs/memory-architecture#section-memory-segment
> <
> https://apacheignite.readme.io/docs/memory-architecture#section-memory-segment
> >
>
>
> 4) "within unsafe segment”
>
> If we want to mention “usafe” here we need to explain what that is.
>
> 5) "If memory amount less than whole index size, Ignite still can operate.”
>
> How do we define “index size”? Is this the page size?
>
>
> —
> Denis
>
>
> > On Aug 30, 2017, at 8:24 AM, Dmitry Pavlov 
> wrote:
> >
> > Hi Igniters,
> >
> >
> > I am glad to introduce two new wiki articles. These articles contain
> > high-level description of
> >
> > - Ignite Durable Memory (
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood
> > )
> >
> > - and Ignite Persistent Store (
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood
> > )
> >
> > These articles cover basic principles and internal design. Pages may be
> > useful for new contributors during start of working with persistent store
> > code base.
> >
> >
> > I would like to thank all contributors who helped me with my questions.
> > Their answers and support made it possible to start this description.
> >
> >
> > I will appreciate any feedback and questions. Suggestions and edits are
> > welcome, of course.
> >
> >
> > Sincerely,
> >
> > Dmitriy Pavlov
>
>

[GitHub] ignite pull request #2884: IGNITE-6529: fix metadata result set

2017-10-19 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2884


---

[jira] [Created] (IGNITE-6684) Rename "ignitesql" to "sqlline"

2017-10-19 Thread Vladimir Ozerov (JIRA)

Vladimir Ozerov created IGNITE-6684:
---

 Summary: Rename "ignitesql" to "sqlline"
 Key: IGNITE-6684
 URL: https://issues.apache.org/jira/browse/IGNITE-6684
 Project: Ignite
  Issue Type: Task
  Security Level: Public (Viewable by anyone)
  Components: general
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
 Fix For: 2.3






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6683) Move WAL and Checkpointing sections to separate pages

Denis Magda created IGNITE-6683:
---

 Summary: Move WAL and Checkpointing sections to separate pages
 Key: IGNITE-6683
 URL: https://issues.apache.org/jira/browse/IGNITE-6683
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
Reporter: Denis Magda
Assignee: Prachi Garg
Priority: Critical
 Fix For: 2.3


Please move WAL and checkpointing sections from this page [1] to dedicated ones.

Page [1] should mention and refer to the new pages.

Make the new pages invisible until 2.3 release.

[1] https://dash.readme.io/project/apacheignite/v2.2/docs/ignite-persistence-23



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6682) Document manual cluster activation feature

Denis Magda created IGNITE-6682:
---

Summary: Document manual cluster activation feature
Key: IGNITE-6682
URL: https://issues.apache.org/jira/browse/IGNITE-6682
Project: Ignite
Issue Type: Bug
Security Level: Public (Viewable by anyone)
Reporter: Denis Magda
Assignee: Prachi Garg

Once Ignite persistence is enabled, the cluster has to be activated manually
before any operation can be processed by it. That's needed to ensure that all
the nodes that have data on disk have joined the cluster after a restart and
that the topology is considered to be stable.

Some details can be found in this discussion:
* Usage section of this doc [1]. Look for a paragraph that starts with _"Next,
when Ignite sees that the persistence is enabled it moves the cluster from
active to inactive state making sure that applications cannot modify the data
until allowed."_
*
http://apache-ignite-developers.2346864.n4.nabble.com/Control-sh-script-and-cluster-activation-td20821.html

Document the manual cluster activation on a dedicated page [2] at least
considering the following:
* Overview. Why it's required? Move the paragraph from [1] and expand if
required.
* Show an exception that happens if you try to execute a SQL or key-value query
or run a computation against an activated cluster.
* Explain how to activate the cluster with:
* From command line with control.sh script.
* From web console.
* From source code with {{Ignite.activate()}} method.
* Add a callout that the automatic cluster activation is to be added in the
future. You may refer to this discussion [3].

[1] https://apacheignite.readme.io/v2.2/docs/ignite-persistence-23#section-usage
[2] https://apacheignite.readme.io/v2.2/docs/cluster-activation
[3]
http://apache-ignite-developers.2346864.n4.nabble.com/Cluster-auto-activation-design-proposal-td20295.html

--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Adding sqlline tool to Apache Ignite project

>> How about renaming it to "sqlline”?

No objections from my side. Who will do the renaming? Please send a note once 
it happens so that Prachi can update the doc and screencast.


Prachi, 

The doc looks good, thanks! Please keep an eye on the discussion and rename the 
script both in the docs and screencasts once the change is applied.


Anton,

All the cosmetic changes (2.1 to 2.3) will be done before the doc release. 
Presently it’s not exposed anywhere and invisible for Google crawlers.

—
Denis

> On Oct 19, 2017, at 2:27 AM, Anton Vinogradov  
> wrote:
> 
> Prachi,
> Seems, documentation should point to 2.3 instead of 2.1?
> 
> On Thu, Oct 19, 2017 at 9:30 AM, Vladimir Ozerov 
> wrote:
> 
>> Igniters,
>> 
>> We named the script "Ignitesql.sh" because initially we thought that it
>> would have additional logic. But now it is merely a thin wrapper around
>> sqlline which only contains classpath creation logic and some convenient
>> defaults. How about renaming it to "sqlline"?
>> 
>> чт, 19 окт. 2017 г. в 2:27, Prachi Garg :
>> 
>>> Here is the link for SQLLine documentation -
>>> https://apacheignite-sql.readme.io/v2.1/docs/sqlline
>>> 
>>> -P
>>> 
>>> On Tue, Oct 17, 2017 at 11:54 AM, Denis Magda  wrote:
>>> 
 Looks good to me. Prachi will help us documenting the tool usage:
 https://issues.apache.org/jira/browse/IGNITE-6656 <
 https://issues.apache.org/jira/browse/IGNITE-6656>
 
 However, I can’t conceive how to see a table structure (columns and
>> their
 types, indexes with names and types) using SQLLine. I’ve tried
>> !metadata
 with a variety of parameters but no luck. As for !indexes and !tables
 commands they just print out table names and secondary indexes omitting
 columns, indexes types and *primary indexes*. Considering that Ignite
 doesn’t support standard *describe* command I assumed SQLLine would
>> help
>>> us
 out. But how do I do this with SQLLine?
 
 —
 Denis
 
> On Oct 17, 2017, at 4:33 AM, Oleg Ostanin 
>>> wrote:
> 
> New example build with sqlline:
> 
> https://ci.ignite.apache.org/viewLog.html?buildId=894407;
 tab=artifacts=IgniteRelease_XxxFromMirrorIgniteRelease3Pre
 pareVote#!1rrb2,1esn4zrslm4po,-h8h0hn9vvvxp
> 
> 
> On Wed, Oct 11, 2017 at 1:00 AM, Denis Magda 
>>> wrote:
> 
>> Oleg,
>> 
>> Looks good to me. Please consider the notes left in the ticket. I
>> want
 us
>> to prepare a script for Windows, review the language for help notice
>>> and
>> errors, put together documentation. Prachi will be able to help with
>>> the
>> editing and documentation.
>> 
>> —
>> Denis
>> 
>>> On Oct 9, 2017, at 10:13 AM, Oleg Ostanin 
 wrote:
>>> 
>>> New build with fixed argument parsing:
>>> https://ci.ignite.apache.org/viewLog.html?buildId=882282;
>> tab=artifacts=IgniteRelease_
>> XxxFromMirrorIgniteRelease3Pre
>> pareVote#!1rrb2,1esn4zrslm4po,-h8h0hn9vvvxp
>>> 
>>> On Mon, Oct 9, 2017 at 5:38 PM, Denis Magda 
 wrote:
>>> 
 I think it’s a must have for the ticket resolution.
 
 Denis
 
 On Monday, October 9, 2017, Anton Vinogradov <
 avinogra...@gridgain.com>
 wrote:
 
> Any plans to have ignitesql.bat?
> 
> On Mon, Oct 9, 2017 at 5:29 PM, Oleg Ostanin <
>>> oosta...@gridgain.com
> > wrote:
> 
>> Another build with sqlline included:
>> https://ci.ignite.apache.org/viewLog.html?buildId=881120;
>> tab=artifacts=IgniteRelease_
>> XxxFromMirrorIgniteRelease3Pre
>> pareVote#!1rrb2,-wpvx2aopzexz,1esn4zrslm4po,-h8h0hn9vvvxp
>> 
>> On Sun, Oct 8, 2017 at 5:11 PM, Denis Magda <
>> dma...@gridgain.com
> > wrote:
>> 
>>> No more doubts on my side. +1 for Vladimir’s suggestion.
>>> 
>>> Denis
>>> 
>>> On Saturday, October 7, 2017, Dmitriy Setrakyan <
 dsetrak...@apache.org
> >
>>> wrote:
>>> 
 I now tend to agree with Vladimir. We should always require
>> that
 some
 address is specified. The help menu should clearly state how
>> to
> connect
>>> to
 a localhost.
 
 D.
 
 On Sat, Oct 7, 2017 at 12:44 AM, Vladimir Ozerov <
> voze...@gridgain.com 
 >
 wrote:
 
> Denis,
> 
> Default Ignite configuration uses multicast, this is why you
>> do
 not
>>> need
 to
> change anything. Ignite node is always both a

Re: SQL error codes

2017-10-19 Thread Vladimir Ozerov

No, they are not re-used. SQLSTATE is JDBC/ODBC-specific thing (and both
driver types have different codes).

On Thu, Oct 19, 2017 at 7:39 PM, Denis Magda  wrote:

> Igor,
>
> Do you have them documented somewhere? Couldn’t find in the existing docs.
>
> Vladimir, Alex P.,
>
> Do we reuse the error codes developed for JDBC on the native APIs side
> (Java, .NET, C++)? On the JDBC page we say that the codes are ANSI SQL
> compliant and my guess that they have to be the same across all SQL APIs.
>
> —
> Denis
>
> > On Oct 19, 2017, at 3:16 AM, Igor Sapego  wrote:
> >
> > Error codes are different for ODBC and JDBC.
> >
> > Best Regards,
> > Igor
> >
> > On Thu, Oct 19, 2017 at 3:23 AM, Denis Magda  wrote:
> >
> >> Alex P, Vladimir,
> >>
> >> Please confirm that the error codes documented for JDBC [1] driver are
> >> shared by all SQL APIs we have. If it’s so I would create Error Codes
> page
> >> under SQL Reference section and will provide error handling code
> snippets
> >> for JDBC, ODBC and the native APIs.
> >>
> >> [1] https://apacheignite-sql.readme.io/v2.1/docs/error-codes
> >>
> >> —
> >> Denis
>
>

Re: SQL error codes

Igor,

Do you have them documented somewhere? Couldn’t find in the existing docs.

Vladimir, Alex P., 

Do we reuse the error codes developed for JDBC on the native APIs side (Java, 
.NET, C++)? On the JDBC page we say that the codes are ANSI SQL compliant and 
my guess that they have to be the same across all SQL APIs.

—
Denis

> On Oct 19, 2017, at 3:16 AM, Igor Sapego  wrote:
> 
> Error codes are different for ODBC and JDBC.
> 
> Best Regards,
> Igor
> 
> On Thu, Oct 19, 2017 at 3:23 AM, Denis Magda  wrote:
> 
>> Alex P, Vladimir,
>> 
>> Please confirm that the error codes documented for JDBC [1] driver are
>> shared by all SQL APIs we have. If it’s so I would create Error Codes page
>> under SQL Reference section and will provide error handling code snippets
>> for JDBC, ODBC and the native APIs.
>> 
>> [1] https://apacheignite-sql.readme.io/v2.1/docs/error-codes
>> 
>> —
>> Denis

[jira] [Created] (IGNITE-6681) .NET: QueryMetrics

2017-10-19 Thread Pavel Tupitsyn (JIRA)

Pavel Tupitsyn created IGNITE-6681:
--

 Summary: .NET: QueryMetrics
 Key: IGNITE-6681
 URL: https://issues.apache.org/jira/browse/IGNITE-6681
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
  Components: platforms
Reporter: Pavel Tupitsyn
 Fix For: 2.4


Propagate query metrics to .NET. See {{IgniteCache.queryMetrics}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6680) .NET: QueryDetailMetrics

2017-10-19 Thread Pavel Tupitsyn (JIRA)

Pavel Tupitsyn created IGNITE-6680:
--

 Summary: .NET: QueryDetailMetrics
 Key: IGNITE-6680
 URL: https://issues.apache.org/jira/browse/IGNITE-6680
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
  Components: platforms
Reporter: Pavel Tupitsyn
Priority: Minor
 Fix For: 2.4


Propagate query metrics to .NET:
* {{CacheConfiguration.QueryDetailMetricsSize}} (default 0, positive enables 
metrics)
* {{ICache.GetQueryDetailMetrics}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6679) Clean up some deprecated cache metrics

2017-10-19 Thread Sergey Puchnin (JIRA)

Sergey Puchnin created IGNITE-6679:
--

 Summary: Clean up some deprecated cache metrics 
 Key: IGNITE-6679
 URL: https://issues.apache.org/jira/browse/IGNITE-6679
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
  Components: cache
Reporter: Sergey Puchnin
Priority: Trivial


An old optimistic serializable mode implementation was removed in 2.0 but some 
cache metrics still present in CacheMetrics interface. 
Need to clean up and remove these metrics:
*TxCommitQueueSize*
*TxPrepareQueueSize*
*TxStartVersionCountsSize*
*TxDhtCommitQueueSize*
*TxDhtPrepareQueueSize*
*TxDhtStartVersionCountsSize*

An algorithm for page eviction was changed and metric 
*DhtEvictQueueCurrentSize* should be removed also.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2886: IGNITE-6675 .NET: Fix ignored IgniteConfiguration...

2017-10-19 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2886


---

[jira] [Created] (IGNITE-6678) Web agent: Implement web-agent as cluster singleton

2017-10-19 Thread Alexey Kuznetsov (JIRA)

Alexey Kuznetsov created IGNITE-6678:


 Summary: Web agent: Implement web-agent as cluster singleton
 Key: IGNITE-6678
 URL: https://issues.apache.org/jira/browse/IGNITE-6678
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
  Components: wizards
Affects Versions: 2.2
Reporter: Alexey Kuznetsov
Assignee: Alexey Kuznetsov
 Fix For: 2.4


It may be very useful to start agent not as separate JVM program, but as 
cluster singleton (for example in cloud).
This will reduce data transfer between cluster and web agent greatly (no-need 
to transform result to JSON string) and also will allow to handle self-healing 
of web-agent by killing hanging node by managment tools (Zookeeper for example).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6677) Collect QueryDetailMetrics for cache-less SQL queries

2017-10-19 Thread Alexey Kuznetsov (JIRA)

Alexey Kuznetsov created IGNITE-6677:


 Summary: Collect QueryDetailMetrics for cache-less SQL queries
 Key: IGNITE-6677
 URL: https://issues.apache.org/jira/browse/IGNITE-6677
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
  Components: sql
Affects Versions: 2.2
Reporter: Alexey Kuznetsov
Assignee: Vladimir Ozerov
 Fix For: 2.4


We have logic that collect query history per caches.
We need:
# Add history size param on IgniteConfiguration
# Implement logic that will collect history for SQL queries executed directly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6676) .NET: Use FsCheck for tests

2017-10-19 Thread Pavel Tupitsyn (JIRA)

Pavel Tupitsyn created IGNITE-6676:
--

 Summary: .NET: Use FsCheck for tests
 Key: IGNITE-6676
 URL: https://issues.apache.org/jira/browse/IGNITE-6676
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
  Components: platforms
Reporter: Pavel Tupitsyn
Priority: Minor


FsCheck provides a way to auto-generate test function inputs: 
https://github.com/fscheck/FsCheck

It has integration with NUnit and can help us improve tests (especially 
serialization tests).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2889: Ignite 1.9.8

2017-10-19 Thread dkarachentsev

GitHub user dkarachentsev opened a pull request:

https://github.com/apache/ignite/pull/2889

Ignite 1.9.8



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-1.9.8

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2889.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2889


commit 72882126c047b937bd5ed93c85e28262530e4977
Author: Vasiliy Sisko 
Date:   2017-03-30T04:08:10Z

IGNITE-4838 Fixed internal task detection logic. Added tests.
(cherry picked from commit ba68c6c)

commit 0d3d93cd4eeb589f1b6a11b48e429defad01c82f
Author: dkarachentsev 
Date:   2017-05-18T16:11:08Z

IGNITE-4842 Now containsKey() respects isReadFromBackup() flag.

(cherry picked from commit d84fd29)

commit 4bf9123a6cff3b1fd17e8772bb0496108fce3f5e
Author: dkarachentsev 
Date:   2017-05-18T16:13:47Z

Merge remote-tracking branch 'origin/ignite-1.8.7' into ignite-1.8.7

commit 2a818d36395dd1af23acf444adf396b2e2edbede
Author: Konstantin Dudkov 
Date:   2017-05-22T13:28:07Z

Fixed "IGNITE-4205 CassandraCacheStore should start IiteThread threads in 
loadCache() method"

Signed-off-by: nikolay_tikhonov 

commit 04fadd4a499239176ba21c390d93e30809abb4c1
Author: dkarachentsev 
Date:   2017-05-23T12:42:20Z

IGNITE-5223 Allow use local binary metadata cache if it's possible

commit b2040b7a95e421609bcf7ae05b56dc623310b409
Author: dkarachentsev 
Date:   2017-05-23T13:14:08Z

IGNITE-5259 Minor serialization fix

commit b77428d12658b3ab2cdd43ca61ed71d329e83283
Author: sboikov 
Date:   2017-01-10T13:59:17Z

Do not evict removed entries, otherwise removes can be lost.

(cherry picked from commit 55ac6e7)

commit 29187ef6b663eafe67eaaaf38e4c09fc244ac7aa
Author: dkarachentsev 
Date:   2017-05-24T14:31:27Z

Do not evict removed entries, otherwise removes can be lost.

Rollback due to test failings.

commit 442aac2507210d39b7f30ab8f8d9a3dbe2610cae
Author: Andrey V. Mashenkov 
Date:   2017-05-24T15:32:11Z

IGNITE-5225: Fix NPE caused by changes in IGNITE-4577.

(cherry picked from commit d463840)

commit b1736c0bd87d6cfb65f9ef422241e0f1aba04c8d
Author: Andrey V. Mashenkov 
Date:   2017-05-24T15:48:52Z

Fixed thread pools incorrect shutdown.

(cherry picked from commit 66cef22)

commit 15d94b432fdfe458a826df6ad3c30a0408a93f49
Author: Andrey V. Mashenkov 
Date:   2017-05-25T11:27:08Z

Backport of IGNITE-4336: Manual rebalance can't be requested twice.
(cherry picked from commit 9a691c4)

commit 3a12fee29625de8d75a291e39b7d52c5f5111fb4
Author: Andrey V. Mashenkov 
Date:   2017-05-25T16:38:59Z

Minors fix segmented indices snapshots.

commit 7362d3c692c28245b193658d727b20caa62ffd38
Author: dkarachentsev 
Date:   2017-05-25T17:13:01Z

Merge compilation fix

commit 26072dffb8f5b28693731f8367872a8e1e6dfe7e
Author: agura 
Date:   2017-05-18T16:40:09Z

ignite-5203 Simple BLOB support added

commit e105b4e5c9c04f966fa4ffcff0e49dc253f4f050
Author: Andrey V. Mashenkov 
Date:   2017-05-30T13:40:24Z

Merge remote-tracking branch 'origin/ignite-1.7.11' into ignite-1.8.7

# Conflicts:
#   
modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/datasource/DataSource.java
#   
modules/cassandra/store/src/test/java/org/apache/ignite/tests/IgnitePersistentStoreTest.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheAdapter.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/service/GridServiceProcessor.java
#   
modules/core/src/test/java/org/apache/ignite/internal/processors/service/GridServiceProcessorMultiNodeConfigSelfTest.java
#   
modules/core/src/test/java/org/apache/ignite/internal/processors/service/GridServiceProcessorMultiNodeSelfTest.java
#   
modules/core/src/test/java/org/apache/ignite/testsuites/IgniteBinaryObjectsTestSuite.java

Merge remote-tracking branch 'origin/ignite-1.7.11' into ignite-1.8.7

# Conflicts:
#   docs/RELEASE_NOTES.txt
#   docs/community/RELEASE_NOTES.txt
#   
modules/compatibility/src/test/java/org/gridgain/grid/compatibility/tests/GridCompatibilityAbstractTest.java
#   modules/core/src/main/resources/gridgain.properties

commit d77a134fffee431cd7fa0bae2349419bc97ec1cf
Author: dkarachentsev

[GitHub] ignite pull request #2888: Ignite 1.9.8

2017-10-19 Thread dkarachentsev

GitHub user dkarachentsev opened a pull request:

https://github.com/apache/ignite/pull/2888

Ignite 1.9.8



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-1.9.8

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2888.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2888


commit 72882126c047b937bd5ed93c85e28262530e4977
Author: Vasiliy Sisko 
Date:   2017-03-30T04:08:10Z

IGNITE-4838 Fixed internal task detection logic. Added tests.
(cherry picked from commit ba68c6c)

commit 0d3d93cd4eeb589f1b6a11b48e429defad01c82f
Author: dkarachentsev 
Date:   2017-05-18T16:11:08Z

IGNITE-4842 Now containsKey() respects isReadFromBackup() flag.

(cherry picked from commit d84fd29)

commit 4bf9123a6cff3b1fd17e8772bb0496108fce3f5e
Author: dkarachentsev 
Date:   2017-05-18T16:13:47Z

Merge remote-tracking branch 'origin/ignite-1.8.7' into ignite-1.8.7

commit 2a818d36395dd1af23acf444adf396b2e2edbede
Author: Konstantin Dudkov 
Date:   2017-05-22T13:28:07Z

Fixed "IGNITE-4205 CassandraCacheStore should start IiteThread threads in 
loadCache() method"

Signed-off-by: nikolay_tikhonov 

commit 04fadd4a499239176ba21c390d93e30809abb4c1
Author: dkarachentsev 
Date:   2017-05-23T12:42:20Z

IGNITE-5223 Allow use local binary metadata cache if it's possible

commit b2040b7a95e421609bcf7ae05b56dc623310b409
Author: dkarachentsev 
Date:   2017-05-23T13:14:08Z

IGNITE-5259 Minor serialization fix

commit b77428d12658b3ab2cdd43ca61ed71d329e83283
Author: sboikov 
Date:   2017-01-10T13:59:17Z

Do not evict removed entries, otherwise removes can be lost.

(cherry picked from commit 55ac6e7)

commit 29187ef6b663eafe67eaaaf38e4c09fc244ac7aa
Author: dkarachentsev 
Date:   2017-05-24T14:31:27Z

Do not evict removed entries, otherwise removes can be lost.

Rollback due to test failings.

commit 442aac2507210d39b7f30ab8f8d9a3dbe2610cae
Author: Andrey V. Mashenkov 
Date:   2017-05-24T15:32:11Z

IGNITE-5225: Fix NPE caused by changes in IGNITE-4577.

(cherry picked from commit d463840)

commit b1736c0bd87d6cfb65f9ef422241e0f1aba04c8d
Author: Andrey V. Mashenkov 
Date:   2017-05-24T15:48:52Z

Fixed thread pools incorrect shutdown.

(cherry picked from commit 66cef22)

commit 15d94b432fdfe458a826df6ad3c30a0408a93f49
Author: Andrey V. Mashenkov 
Date:   2017-05-25T11:27:08Z

Backport of IGNITE-4336: Manual rebalance can't be requested twice.
(cherry picked from commit 9a691c4)

commit 3a12fee29625de8d75a291e39b7d52c5f5111fb4
Author: Andrey V. Mashenkov 
Date:   2017-05-25T16:38:59Z

Minors fix segmented indices snapshots.

commit 7362d3c692c28245b193658d727b20caa62ffd38
Author: dkarachentsev 
Date:   2017-05-25T17:13:01Z

Merge compilation fix

commit 26072dffb8f5b28693731f8367872a8e1e6dfe7e
Author: agura 
Date:   2017-05-18T16:40:09Z

ignite-5203 Simple BLOB support added

commit e105b4e5c9c04f966fa4ffcff0e49dc253f4f050
Author: Andrey V. Mashenkov 
Date:   2017-05-30T13:40:24Z

Merge remote-tracking branch 'origin/ignite-1.7.11' into ignite-1.8.7

# Conflicts:
#   
modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/datasource/DataSource.java
#   
modules/cassandra/store/src/test/java/org/apache/ignite/tests/IgnitePersistentStoreTest.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheAdapter.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/service/GridServiceProcessor.java
#   
modules/core/src/test/java/org/apache/ignite/internal/processors/service/GridServiceProcessorMultiNodeConfigSelfTest.java
#   
modules/core/src/test/java/org/apache/ignite/internal/processors/service/GridServiceProcessorMultiNodeSelfTest.java
#   
modules/core/src/test/java/org/apache/ignite/testsuites/IgniteBinaryObjectsTestSuite.java

Merge remote-tracking branch 'origin/ignite-1.7.11' into ignite-1.8.7

# Conflicts:
#   docs/RELEASE_NOTES.txt
#   docs/community/RELEASE_NOTES.txt
#   
modules/compatibility/src/test/java/org/gridgain/grid/compatibility/tests/GridCompatibilityAbstractTest.java
#   modules/core/src/main/resources/gridgain.properties

commit d77a134fffee431cd7fa0bae2349419bc97ec1cf
Author: dkarachentsev

CacheConfiguration reusage issues with EvictionPolicy enabled.

2017-10-19 Thread Andrey Mashenkov

Hi Igniters,

I'm working on a ticket IGNITE-6649 [1] and got stuck.
Currently, we allow user to set EvictionPolicy instance into configuration
instead of factory.
The leads to some isses when user tries to reuse EvictionPolicy instance,
e.g. it doesn't clean its queue on cache stop.

Seems, we should replace EvictionPolicy with its factory as policy objec is
a statefull object and current approach is error prone.

The issue I faced is that we make checks for EvicitonPolicy derives some
known class in several code places.
E.g. in HadoopModule you can find IgfsPerBlockLruEvictionPolicy usages.
What will be correct way to overwork this with using facrories?


Also, I've noticed that we check EvictionPolicy instances if they implement
LifecycleAware interface and no one policy available out-of-box really
implements it.
The second way here is to make these EvictionPolicies implements
LifecycleAware interface and gracefully clean their queues.
This will resolve the issue for case when user recreate cache with same
configuration instance,
but seem will not help in case when user share EvictionPolicy instance for
several caches.


Also`EvictionPolicies has non-transient "queue" field, so queue may leak to
another node.
I'm not sure it can happens, I'll check.

Thoughts?



[1] https://issues.apache.org/jira/browse/IGNITE-6649

-- 
Best regards,
Andrey V. Mashenkov

[GitHub] ignite pull request #2887: Ignite gg 12887 1.9.8

2017-10-19 Thread dkarachentsev

GitHub user dkarachentsev opened a pull request:

https://github.com/apache/ignite/pull/2887

Ignite gg 12887 1.9.8



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-12887-1.9.8

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2887.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2887


commit 0d3d93cd4eeb589f1b6a11b48e429defad01c82f
Author: dkarachentsev 
Date:   2017-05-18T16:11:08Z

IGNITE-4842 Now containsKey() respects isReadFromBackup() flag.

(cherry picked from commit d84fd29)

commit 4bf9123a6cff3b1fd17e8772bb0496108fce3f5e
Author: dkarachentsev 
Date:   2017-05-18T16:13:47Z

Merge remote-tracking branch 'origin/ignite-1.8.7' into ignite-1.8.7

commit 2a818d36395dd1af23acf444adf396b2e2edbede
Author: Konstantin Dudkov 
Date:   2017-05-22T13:28:07Z

Fixed "IGNITE-4205 CassandraCacheStore should start IiteThread threads in 
loadCache() method"

Signed-off-by: nikolay_tikhonov 

commit 04fadd4a499239176ba21c390d93e30809abb4c1
Author: dkarachentsev 
Date:   2017-05-23T12:42:20Z

IGNITE-5223 Allow use local binary metadata cache if it's possible

commit b2040b7a95e421609bcf7ae05b56dc623310b409
Author: dkarachentsev 
Date:   2017-05-23T13:14:08Z

IGNITE-5259 Minor serialization fix

commit b77428d12658b3ab2cdd43ca61ed71d329e83283
Author: sboikov 
Date:   2017-01-10T13:59:17Z

Do not evict removed entries, otherwise removes can be lost.

(cherry picked from commit 55ac6e7)

commit 29187ef6b663eafe67eaaaf38e4c09fc244ac7aa
Author: dkarachentsev 
Date:   2017-05-24T14:31:27Z

Do not evict removed entries, otherwise removes can be lost.

Rollback due to test failings.

commit 442aac2507210d39b7f30ab8f8d9a3dbe2610cae
Author: Andrey V. Mashenkov 
Date:   2017-05-24T15:32:11Z

IGNITE-5225: Fix NPE caused by changes in IGNITE-4577.

(cherry picked from commit d463840)

commit b1736c0bd87d6cfb65f9ef422241e0f1aba04c8d
Author: Andrey V. Mashenkov 
Date:   2017-05-24T15:48:52Z

Fixed thread pools incorrect shutdown.

(cherry picked from commit 66cef22)

commit 15d94b432fdfe458a826df6ad3c30a0408a93f49
Author: Andrey V. Mashenkov 
Date:   2017-05-25T11:27:08Z

Backport of IGNITE-4336: Manual rebalance can't be requested twice.
(cherry picked from commit 9a691c4)

commit 3a12fee29625de8d75a291e39b7d52c5f5111fb4
Author: Andrey V. Mashenkov 
Date:   2017-05-25T16:38:59Z

Minors fix segmented indices snapshots.

commit 7362d3c692c28245b193658d727b20caa62ffd38
Author: dkarachentsev 
Date:   2017-05-25T17:13:01Z

Merge compilation fix

commit 26072dffb8f5b28693731f8367872a8e1e6dfe7e
Author: agura 
Date:   2017-05-18T16:40:09Z

ignite-5203 Simple BLOB support added

commit e105b4e5c9c04f966fa4ffcff0e49dc253f4f050
Author: Andrey V. Mashenkov 
Date:   2017-05-30T13:40:24Z

Merge remote-tracking branch 'origin/ignite-1.7.11' into ignite-1.8.7

# Conflicts:
#   
modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/datasource/DataSource.java
#   
modules/cassandra/store/src/test/java/org/apache/ignite/tests/IgnitePersistentStoreTest.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheAdapter.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/service/GridServiceProcessor.java
#   
modules/core/src/test/java/org/apache/ignite/internal/processors/service/GridServiceProcessorMultiNodeConfigSelfTest.java
#   
modules/core/src/test/java/org/apache/ignite/internal/processors/service/GridServiceProcessorMultiNodeSelfTest.java
#   
modules/core/src/test/java/org/apache/ignite/testsuites/IgniteBinaryObjectsTestSuite.java

Merge remote-tracking branch 'origin/ignite-1.7.11' into ignite-1.8.7

# Conflicts:
#   docs/RELEASE_NOTES.txt
#   docs/community/RELEASE_NOTES.txt
#   
modules/compatibility/src/test/java/org/gridgain/grid/compatibility/tests/GridCompatibilityAbstractTest.java
#   modules/core/src/main/resources/gridgain.properties

commit d77a134fffee431cd7fa0bae2349419bc97ec1cf
Author: dkarachentsev 
Date:   2017-05-30T16:00:47Z

IGNITE-5342 - Skip permission check for TASK_EXECUTE for service jobs

commit c33504d4d51682dfb27ee3d38b4625f0946f4719
Author: dkarachentsev

[GitHub] ignite pull request #2886: IGNITE-6675 .NET: Fix ignored IgniteConfiguration...

2017-10-19 Thread ptupitsyn

GitHub user ptupitsyn opened a pull request:

https://github.com/apache/ignite/pull/2886

IGNITE-6675 .NET: Fix ignored IgniteConfiguration.IgniteHome



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ptupitsyn/ignite ignite-6675

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2886.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2886


commit f3e22cdc43df1b2d134b8edd296edcdf1e86bb9d
Author: Pavel Tupitsyn 
Date:   2017-10-19T14:34:53Z

IGNITE-6675 .NET: Fix ignored IgniteConfiguration.IgniteHome




---

Re: Integration of Spark and Ignite. Prototype.

2017-10-19 Thread Николай Ижиков

Hello.

> 3. IgniteCatalog vs. IgniteExternalCatalog. Why do we have two Catalog
implementations and what is the difference?

IgniteCatalog removed.

> 5. I don't like that IgniteStrategy and IgniteOptimization have to be set
manually on SQLContext each time it's createdIs there any way to
automate this and improve usability?

IgniteStrategy and IgniteOptimization are removed as it empty now.

> Actually, I think it makes sense to create a builder similar to
SparkSession.builder()...

IgniteBuilder added.
Syntax looks like:

```
val igniteSession = IgniteSparkSession.builder()
.appName("Spark Ignite catalog example")
.master("local")
.config("spark.executor.instances", "2")
.igniteConfig(CONFIG)
.getOrCreate()

igniteSession.catalog.listTables().show()
```

Please, see updated PR - https://github.com/apache/ignite/pull/2742

2017-10-18 20:02 GMT+03:00 Николай Ижиков :

> Hello, Valentin.
>
> My answers is below.
> Dmitry, do we need to move discussion to Jira?
>
> > 1. Why do we have org.apache.spark.sql.ignite package in our codebase?
>
> As I mentioned earlier, to implement and override Spark Catalog one have
> to use internal(private) Spark API.
> So I have to use package `org.spark.sql.***` to have access to private
> class and variables.
>
> For example, SharedState class that stores link to ExternalCatalog
> declared as `private[sql] class SharedState` - i.e. package private.
>
> > Can these classes reside under org.apache.ignite.spark instead?
>
> No, as long as we want to have our own implementation of ExternalCatalog.
>
> > 2. IgniteRelationProvider contains multiple constants which I guess are
> some king of config options. Can you describe the purpose of each of them?
>
> I extend comments for this options.
> Please, see my commit [1] or PR HEAD:
>
> > 3. IgniteCatalog vs. IgniteExternalCatalog. Why do we have two Catalog
> implementations and what is the difference?
>
> Good catch, thank you!
> After additional research I founded that only IgniteExternalCatalog
> required.
> I will update PR with IgniteCatalog remove in a few days.
>
> > 4. IgniteStrategy and IgniteOptimization are currently no-op. What are
> our plans on implementing them? Also, what exactly is planned in
> IgniteOptimization and what is its purpose?
>
> Actually, this is very good question :)
> And I need advice from experienced community members here:
>
> `IgniteOptimization` purpose is to modify query plan created by Spark.
> Currently, we have one optimization described in IGNITE-3084 [2] by you,
> Valentin :) :
>
> “If there are non-Ignite relations in the plan, we should fall back to
> native Spark strategies“
>
> I think we can go little further and reduce join of two Ignite backed Data
> Frames into single Ignite SQL query. Currently, this feature is
> unimplemented.
>
> *Do we need it now? Or we can postpone it and concentrates on basic Data
> Frame and Catalog implementation?*
>
> `Strategy` purpose, as you correctly mentioned in [2], is transform
> LogicalPlan into physical operators.
> I don’t have ideas how to use this opportunity. So I think we don’t need
> IgniteStrategy.
>
> Can you or anyone else suggest some optimization strategy to speed up SQL
> query execution?
>
> > 5. I don't like that IgniteStrategy and IgniteOptimization have to be
> set manually on SQLContext each time it's createdIs there any way to
> automate this and improve usability?
>
> These classes added to `extraOptimizations` when one using
> IgniteSparkSession.
> As far as I know, there is no way to automatically add these classes to
> regular SparkSession.
>
> > 6. What is the purpose of IgniteSparkSession? I see it's used in
> IgniteCatalogExample but not in IgniteDataFrameExample, which is Confusing.
>
> DataFrame API is *public* Spark API. So anyone can provide implementation
> and plug it into Spark. That’s why IgniteDataFrameExample doesn’t need any
> Ignite specific session.
>
> Catalog API is *internal* Spark API. There is no way to plug custom
> catalog implementation into Spark [3]. So we have to use
> `IgniteSparkSession` that extends regular SparkSession and overrides links
> to `ExternalCatalog`.
>
> > 7. To create IgniteSparkSession we first create IgniteContext. Is it
> really needed? It looks like we can directly provide the configuration
> file; if IgniteSparkSession really requires IgniteContext, it can create it
> by itself under the hood.
>
> Actually, IgniteContext is base class for Ignite <-> Spark integration for
> now. So I tried to reuse it here. I like the idea to remove explicit usage
> of IgniteContext.
> Will implement it in a few days.
>
> > Actually, I think it makes sense to create a builder similar to
> SparkSession.builder()...
>
> Great idea! I will implement such builder in a few days.
>
> > 9. Do I understand correctly that IgniteCacheRelation is for the case
> when we don't have SQL configured on Ignite side?
>
> Yes, IgniteCacheRelation is Data Frame

[GitHub] ignite pull request #2874: IGNITE-6658 Add a version of Ignite instance to l...

2017-10-19 Thread daradurvs

Github user daradurvs closed the pull request at:

https://github.com/apache/ignite/pull/2874


---

[GitHub] ignite pull request #2885: GG-12974 Don't use org.apache.spark.Logging in Ig...

2017-10-19 Thread apopovgg

GitHub user apopovgg opened a pull request:

https://github.com/apache/ignite/pull/2885

GG-12974 Don't use org.apache.spark.Logging in Ignite-spark



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-12974

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2885.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2885


commit b7ab27301b59bf93fc73b52fdf8e0bcf124fec1d
Author: Andrey V. Mashenkov 
Date:   2017-04-06T11:43:50Z

IGNITE-4832: Prevent service deployment on client by default when 
configuration is provided on startup. This closes #1748.

commit b214211eb3461746b6931c0623e086bb208e5dda
Author: Anton Vinogradov 
Date:   2017-04-06T12:00:12Z

IGNITE-4644 Value from IgniteQueue in atomic mode could be lost

commit 014161427fb603b6df7c8ecc3c0904f5df47a21d
Author: Denis Magda 
Date:   2017-02-14T01:33:32Z

IGNITE-4159: Kubernetes IP finder.
(cherry picked from commit 37c0a22)

commit 1db238402f11c67d2b28bfb7ff47955415f00c25
Author: Denis Magda 
Date:   2017-02-16T04:37:26Z

IGNITE-4159: fixing logging
(cherry picked from commit 06908d2)
(cherry picked from commit fa27ee3)

commit 5dfe16f7e91374008b9f6dfbb899364f5b2e1164
Author: Denis Magda 
Date:   2017-02-14T06:03:30Z

IGNITE-4159: using logger instead of system.out.println
(cherry picked from commit b9bf77c)

commit 6e596d1ef426b66abd866d011a8f5cf5c5c25124
Author: Andrey V. Mashenkov 
Date:   2017-04-06T11:43:50Z

IGNITE-4832: Prevent service deployment on client by default when 
configuration is provided on startup. This closes #1748.

(cherry picked from commit b7ab273)

commit 443ac9a7aa82af1359a03bcfc8f9212b108300e4
Author: Andrey V. Mashenkov 
Date:   2017-04-05T12:01:02Z

IGNITE-4917: Fixed failure when accessing BinaryObjectBuilder field value 
serialized with OptimizedMarshaller . This closes #1736.

commit 05f3c747921aed6838804d2f5f2c8d2bd7985337
Author: Andrey V. Mashenkov 
Date:   2017-04-05T12:01:02Z

IGNITE-4917: Fixed failure when accessing BinaryObjectBuilder field value 
serialized with OptimizedMarshaller . This closes #1736.

(cherry picked from commit 443ac9a)

commit 3be4e00373ec5a2b49788d70eb0aebccc3cb6ccf
Author: Alexander Fedotov 
Date:   2017-04-07T11:59:00Z

Merge branch ignite-1.6.5 into ignite-1.8.5-p1

commit d81548d3a4e384e1a9b4adacf1fb487444bbfd33
Author: Alexander Fedotov 
Date:   2017-04-07T12:33:08Z

Merge branch ignite-1.6.6-p1 into ignite-1.8.5-p1

commit 6954ff0c85f2f75507ee0bd306c879f490b4201a
Author: Alexander Fedotov 
Date:   2017-04-07T12:44:48Z

Merge branch ignite-1.6.12 into ignite-1.8.5-p1

commit 62dbba81c009170ff6243a28d3ef12fa75b96225
Author: Alexander Fedotov 
Date:   2017-04-07T12:46:11Z

Merge branch ignite-1.7.4-p1 into ignite-1.8.5-p1

commit 4fce28054bc325741f65035ae384f9b4b9c3fee8
Author: Alexander Fedotov 
Date:   2017-04-07T13:06:34Z

Merge branch ignite-1.8.4-p1 into ignite-1.8.5-p1

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/internal/managers/discovery/GridDiscoveryManager.java

commit 3d616799efb472227b3b313516e6b40729654631
Author: dkarachentsev 
Date:   2017-04-10T07:36:11Z

IGNITE-2466 - Use current NIO back pressure mechanism to limit received 
messages. Mark them process only when backups acknowledged.

(backport from 1.9.2)

(cherry picked from commit 220db882b466c03eadd148b3a19a0bf70d82d4a6)

commit 2a88a7a7581465ff0a6f8733550e6d42d7f71a6c
Author: dkarachentsev 
Date:   2017-04-10T07:54:37Z

IGNITE-4667 - Throw exception on starting client cache when indexed types 
cannot be loaded

commit ba6227be49c8a395a5632e9841a6acb65ae340b6
Author: dkarachentsev 
Date:   2017-04-10T08:40:17Z

IGNITE-2466 - Disable back-pressure for sender data nodes to avoid deadlock.

commit 315ff38eeef96f12954d6ff39c84d58b2b959667
Author: Andrey V. Mashenkov 
Date:   2017-04-06T11:43:50Z

IGNITE-4879: Fixed System pool starvation while partition evicting.

commit 89e9dbe484312c251f02c9fbe9698c3ac2e03df8
Author: Alexander Fedotov 
Date:   2017-04-10T13:36:33Z

Fix org.apache.ignite.internal.processors.cache.expiry
.IgniteCacheExpiryPolicyAbstractTest#testNearExpiresWithCacheStore

commit

[jira] [Created] (IGNITE-6675) IGNITE_HOME is not detected automatically when using IgniteOutputCacheProvider

2017-10-19 Thread Irina Zaporozhtseva (JIRA)

Irina Zaporozhtseva created IGNITE-6675:
---

 Summary: IGNITE_HOME is not detected automatically when using 
IgniteOutputCacheProvider
 Key: IGNITE-6675
 URL: https://issues.apache.org/jira/browse/IGNITE-6675
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
  Components: platforms
Affects Versions: 2.3
 Environment: Visual Studio Community 2015 version 14.0.25431.01 update 
3
.NET Framework version 4.7.02046
64bit version of IIS
Reporter: Irina Zaporozhtseva
 Fix For: 2.3


Steps to reproduce:
1) Create a new C# Web application in the MS Visual Studio:
File -> New -> Project -> Visual C# -> Web -> ASP.NET Web Application

2) Install Apache.Ignite.AspNet NuGet package

3) Open Web.config file and add following code into beginning of 
"configuration" section:

{code:java}









{code}

And following code into beginning of "system.web" section:


{code:java}













{code}

Open Default.aspx.cs file and add the following to Page_Load method:

{code:java}
Session["test"] = "abc";
{code}

4) Run project

Following exception appears in browser:


{code}
Configuration Error

Description: An error occurred during the processing of a configuration file 
required to service this request. Please review the specific error details 
below and modify your configuration file appropriately. 

Parser Error Message: Failed to initialize 
Apache.Ignite.AspNet.IgniteSessionStateStoreProvider: 
Apache.Ignite.Core.Common.IgniteException: Failed to initialize JVM. ---> 
Apache.Ignite.Core.Common.IgniteException: Java class is not found (did you set 
IGNITE_HOME environment variable?): 
org/apache/ignite/internal/processors/platform/utils/PlatformUtils ---> 
Apache.Ignite.Core.Common.JavaException
  --- End of inner exception stack trace ---
  --- End of inner exception stack trace ---
  at Apache.Ignite.Core.Impl.Unmanaged.UnmanagedCallbacks.Error(Void* target, 
Int32 errType, SByte* errClsChars, Int32 errClsCharsLen, SByte* errMsgChars, 
Int32 errMsgCharsLen, SByte* stackTraceChars, Int32 stackTraceCharsLen, Void* 
errData, Int32 errDataLen)
  at 
Apache.Ignite.Core.Impl.Unmanaged.IgniteJniNativeMethods.CreateContext(Void* 
opts, Int32 optsLen, Void* cbs)
  at Apache.Ignite.Core.Impl.IgniteManager.CreateJvm(IgniteConfiguration cfg, 
UnmanagedCallbacks cbs)
  at Apache.Ignite.Core.Impl.IgniteManager.CreateJvmContext(IgniteConfiguration 
cfg, UnmanagedCallbacks cbs, ILogger log)
  at Apache.Ignite.Core.Ignition.Start(IgniteConfiguration cfg)
  at 
Apache.Ignite.AspNet.Impl.ConfigUtil.InitializeCache[TK,TV](NameValueCollection 
config, Type callerType, String defaultCacheName)

Source Error: 


Line 29: 
Line 30:   
Line 31: 
Line 32:   
Line 33: 

Source File: c:\visual studio 
2015\Projects\WebApplication10\WebApplication10\web.configLine: 31 
{code}




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Adding sqlline tool to Apache Ignite project