Re: Apache Ignite 2.8.1 RELEASE [Time, Scope, Manager]

2020-04-16 Thread Alex Plehanov
Hello guys,

I propose to cherry-pick to 2.8.1 following bug-fixes too, which are
already in master:
Scan query over an evicted partition can cause node failure (IGNITE-12734
[1])
Java thin client: There were problems with deserialization of some types on
the client-side, these types can't be used (IGNITE-12624
[2], IGNITE-12468 [3])
Java thin client: Thread doesn't stop properly on client close when
partition awareness is enabled, this prevents main() method from exiting
(IGNITE-12743 [4])

Also, there is a performance fix for checkpoint read lock, which I propose
to cherry-pick too (IGNITE-12491 [5]). This fix brings significant
performance boost on environments with a large number of CPUs (there was
some drop on such environments introduced in 2.8.0 for all transactional
operations after IGNITE-12593 fixing)

WDYT?

[1]: https://issues.apache.org/jira/browse/IGNITE-12734
[2]: https://issues.apache.org/jira/browse/IGNITE-12624
[3]: https://issues.apache.org/jira/browse/IGNITE-12468
[4]: https://issues.apache.org/jira/browse/IGNITE-12743
[5]: https://issues.apache.org/jira/browse/IGNITE-12491

чт, 16 апр. 2020 г. в 18:48, Maxim Muzafarov :

> Nikolay,
>
> Probably, we should not wait for all blocker issues in minor bug-fix
> releases except very special cases. I think we should release all
> accumulated bug-fixes `as is` and schedule the next 2.8.2 release.
> This will allow as to have shorten minor releases.
>
> On Thu, 16 Apr 2020 at 18:17, Nikolay Izhikov  wrote:
> >
> > Hello, Igniters.
> >
> > I’m started to work on this 2.8.1 release [1]
> >
> > Resolved issues for release(28) - [2]
> > Unresolved issues for release(30) - [3]
> >
> > My next step:
> >
> > 1. I want to double-check that all commits for the tickets with the
> fixVersion=2.8.1 presented in corresponding release branch.
> > And cherry-pick losted changes.
> >
> > 2. I want to reduce the scope of the release and exclude tickets that is
> not ready for now.
> >
> > As you may know, 2.8.1 is a bug fix release.
> > Therefore, I think we can wait only for a blocker issues.
> >
> > What do you think?
> >
> > [1]
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.8.1
> > [2]
> https://issues.apache.org/jira/issues/?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20is%20not%20empty%20AND%20fixVersion%20in%20(%272.8.1%27))%20AND%20(component%20is%20EMPTY%20OR%20component%20not%20in%20(documentation))%20and%20status%20in%20(%27CLOSED%27%2C%20%27RESOLVED%27)%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20
> > [3]
> https://issues.apache.org/jira/issues/?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20is%20not%20empty%20AND%20fixVersion%20in%20(%272.8.1%27))%20AND%20(component%20is%20EMPTY%20OR%20component%20not%20in%20(documentation))%20%20and%20status%20not%20in%20(%27CLOSED%27%2C%20%27RESOLVED%27)%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20
> >
> >
> > > 8 апр. 2020 г., в 20:15, Вячеслав Коптилин 
> написал(а):
> > >
> > > Folks,
> > >
> > > I'd like to add ticket IGNITE-12805 "NullPointerException on node
> restart
> > > when 3rd party persistence and Ignite native persistence are used" to
> > > ignite-2.8.1 scope.
> > >
> > > [1]  https://issues.apache.org/jira/browse/IGNITE-12805
> > >
> > > Thanks,
> > > S.
> > >
> > > вт, 7 апр. 2020 г. в 19:57, Ilya Kasnacheev  >:
> > >
> > >> Hello!
> > >>
> > >> Done!
> > >>
> > >> Regards,
> > >> --
> > >> Ilya Kasnacheev
> > >>
> > >>
> > >> вт, 7 апр. 2020 г. в 12:31, Sergey :
> > >>
> > >>> Hi,
> > >>>
> > >>> I'm proposing to add
> > >>> https://issues.apache.org/jira/browse/IGNITE-12549  (fix
> iterators/scan
> > >>> queries for replicated caches)
> > >>> to 2.8.1.
> > >>>
> > >>> Best regards,
> > >>> Sergey Kosarev.
> > >>>
> > >>>
> > >>> вс, 5 апр. 2020 г. в 01:22, Saikat Maitra :
> > >>>
> >  Hi,
> > 
> >  I observed that we already have release 2.8.1 branch
> >  https://github.com/apache/ignite/tree/ignite-2.8.1
> > 
> >  In that case we should be ok to merge these 2 open PRs in master to
> > >> make
> > >>> it
> >  available for 2.9.0 release.
> > 
> >  https://github.com/apache/ignite/pull/7240
> >  https://github.com/apache/ignite/pull/7227
> > 
> >  Can you please review and confirm?
> > 
> >  Regards,
> >  Saikat
> > 
> >  On Fri, Mar 20, 2020 at 8:19 AM Maxim Muzafarov 
> > >>> wrote:
> > 
> > > Igniters,
> > >
> > > I support Nikolay Izhikov as the release manager of 2.8.1 Apache
> > > Ignite release. Since no one else of committers, PMCs expressed a
> > > desire to lead this release I think we can close this question and
> > > focus on the release scope and dates.
> > >
> > >
> > > Ivan,
> > >
> > > You helped me configuring TC.Bot that time, can you please help
> again
> > > and set `ignite-2.8.1` branch for guard under TC.Bot [1]? We should
> > > start collecting TC statistics 

[jira] [Created] (IGNITE-12908) Confusding check for presence of SqlViewExporterSpi class in classpath during initialization

2020-04-16 Thread Andrey N. Gura (Jira)
Andrey N. Gura created IGNITE-12908:
---

 Summary: Confusding check for presence of SqlViewExporterSpi class 
in classpath during initialization
 Key: IGNITE-12908
 URL: https://issues.apache.org/jira/browse/IGNITE-12908
 Project: Ignite
  Issue Type: Improvement
Reporter: Andrey N. Gura


{{IgniteEx.initializeDefaultSpi()}} method checks for presence of 
{{SqlViewExporterSpi}} class in classpath whilst actual intention is to check 
for presence of indexing in classpath. It could be done just using 
{{IgniteComponentType.INDEXING.inClassPath()}} method.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-12907) GridSystemViewManager is not thread safe

2020-04-16 Thread Andrey N. Gura (Jira)
Andrey N. Gura created IGNITE-12907:
---

 Summary: GridSystemViewManager is not thread safe
 Key: IGNITE-12907
 URL: https://issues.apache.org/jira/browse/IGNITE-12907
 Project: Ignite
  Issue Type: Bug
Reporter: Andrey N. Gura


{{GridSystemViewManager}} is not thread safe because it allows to registers 
walkers using public  {{registerWalker()}} that just adds walker to a 
{{HashMap}}.

It seems that simplest solution here: make {{registerWalker()}} method private 
because it has usages only from {{GridSystemViewManager}} constructor.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: Apache Ignite 2.8.1 RELEASE [Time, Scope, Manager]

2020-04-16 Thread Maxim Muzafarov
Nikolay,

Probably, we should not wait for all blocker issues in minor bug-fix
releases except very special cases. I think we should release all
accumulated bug-fixes `as is` and schedule the next 2.8.2 release.
This will allow as to have shorten minor releases.

On Thu, 16 Apr 2020 at 18:17, Nikolay Izhikov  wrote:
>
> Hello, Igniters.
>
> I’m started to work on this 2.8.1 release [1]
>
> Resolved issues for release(28) - [2]
> Unresolved issues for release(30) - [3]
>
> My next step:
>
> 1. I want to double-check that all commits for the tickets with the 
> fixVersion=2.8.1 presented in corresponding release branch.
> And cherry-pick losted changes.
>
> 2. I want to reduce the scope of the release and exclude tickets that is not 
> ready for now.
>
> As you may know, 2.8.1 is a bug fix release.
> Therefore, I think we can wait only for a blocker issues.
>
> What do you think?
>
> [1] https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.8.1
> [2] 
> https://issues.apache.org/jira/issues/?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20is%20not%20empty%20AND%20fixVersion%20in%20(%272.8.1%27))%20AND%20(component%20is%20EMPTY%20OR%20component%20not%20in%20(documentation))%20and%20status%20in%20(%27CLOSED%27%2C%20%27RESOLVED%27)%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20
> [3] 
> https://issues.apache.org/jira/issues/?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20is%20not%20empty%20AND%20fixVersion%20in%20(%272.8.1%27))%20AND%20(component%20is%20EMPTY%20OR%20component%20not%20in%20(documentation))%20%20and%20status%20not%20in%20(%27CLOSED%27%2C%20%27RESOLVED%27)%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20
>
>
> > 8 апр. 2020 г., в 20:15, Вячеслав Коптилин  
> > написал(а):
> >
> > Folks,
> >
> > I'd like to add ticket IGNITE-12805 "NullPointerException on node restart
> > when 3rd party persistence and Ignite native persistence are used" to
> > ignite-2.8.1 scope.
> >
> > [1]  https://issues.apache.org/jira/browse/IGNITE-12805
> >
> > Thanks,
> > S.
> >
> > вт, 7 апр. 2020 г. в 19:57, Ilya Kasnacheev :
> >
> >> Hello!
> >>
> >> Done!
> >>
> >> Regards,
> >> --
> >> Ilya Kasnacheev
> >>
> >>
> >> вт, 7 апр. 2020 г. в 12:31, Sergey :
> >>
> >>> Hi,
> >>>
> >>> I'm proposing to add
> >>> https://issues.apache.org/jira/browse/IGNITE-12549  (fix iterators/scan
> >>> queries for replicated caches)
> >>> to 2.8.1.
> >>>
> >>> Best regards,
> >>> Sergey Kosarev.
> >>>
> >>>
> >>> вс, 5 апр. 2020 г. в 01:22, Saikat Maitra :
> >>>
>  Hi,
> 
>  I observed that we already have release 2.8.1 branch
>  https://github.com/apache/ignite/tree/ignite-2.8.1
> 
>  In that case we should be ok to merge these 2 open PRs in master to
> >> make
> >>> it
>  available for 2.9.0 release.
> 
>  https://github.com/apache/ignite/pull/7240
>  https://github.com/apache/ignite/pull/7227
> 
>  Can you please review and confirm?
> 
>  Regards,
>  Saikat
> 
>  On Fri, Mar 20, 2020 at 8:19 AM Maxim Muzafarov 
> >>> wrote:
> 
> > Igniters,
> >
> > I support Nikolay Izhikov as the release manager of 2.8.1 Apache
> > Ignite release. Since no one else of committers, PMCs expressed a
> > desire to lead this release I think we can close this question and
> > focus on the release scope and dates.
> >
> >
> > Ivan,
> >
> > You helped me configuring TC.Bot that time, can you please help again
> > and set `ignite-2.8.1` branch for guard under TC.Bot [1]? We should
> > start collecting TC statistics for the release branch as early as
> > possible.
> >
> >
> > [1] https://mtcga.gridgain.com/guard.html
> >
> > On Fri, 20 Mar 2020 at 14:48, Taras Ledkov 
> >>> wrote:
> >>
> >> Hi,
> >>
> >> I propose to add the issue [1] related to SQL query execution to
> >> this
> > scope.
> >>
> >> We had omitted this case and Ignite 2.8 contains serious SQL issue:
> >> cursor of a local query is not thread-safe.
> >> It is root cause of several SQL issue, e.g. JDBC thin client cannot
> >> execute query  from replicated cache,
> >> PME may hang after execute such queries from JDBC thin, etc.
> >>
> >> [1]. https://issues.apache.org/jira/browse/IGNITE-12800
> >>
> >> On 19.03.2020 17:52, Denis Magda wrote:
> >>> Igniters,
> >>>
> >>> As long as 2.8.1 is inevitable and we already keep adding
> >> critical
> > issues
> >>> to the working queue, let's settle on the release time frames and
> > decide
> >>> who will be a release manager. This is the time proposed by Maxim
>  and,
> >>> personally, I concur with such a schedule:
> >>>
> >>>- Scope Freeze: April 15, 2020
> >>>- Code Freeze: April 22, 2020
> >>>- Voting Date: April 27, 2020
> >>>- Release Date: May 1, 2020
> >>>
> >>> Do we agree on this time? Is 

Re: Apache Ignite 2.8.1 RELEASE [Time, Scope, Manager]

2020-04-16 Thread Nikolay Izhikov
Hello, Igniters.

I’m started to work on this 2.8.1 release [1] 

Resolved issues for release(28) - [2]
Unresolved issues for release(30) - [3]

My next step: 

1. I want to double-check that all commits for the tickets with the 
fixVersion=2.8.1 presented in corresponding release branch.
And cherry-pick losted changes.

2. I want to reduce the scope of the release and exclude tickets that is not 
ready for now.

As you may know, 2.8.1 is a bug fix release.
Therefore, I think we can wait only for a blocker issues.

What do you think?

[1] https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.8.1
[2] 
https://issues.apache.org/jira/issues/?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20is%20not%20empty%20AND%20fixVersion%20in%20(%272.8.1%27))%20AND%20(component%20is%20EMPTY%20OR%20component%20not%20in%20(documentation))%20and%20status%20in%20(%27CLOSED%27%2C%20%27RESOLVED%27)%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20
[3] 
https://issues.apache.org/jira/issues/?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20is%20not%20empty%20AND%20fixVersion%20in%20(%272.8.1%27))%20AND%20(component%20is%20EMPTY%20OR%20component%20not%20in%20(documentation))%20%20and%20status%20not%20in%20(%27CLOSED%27%2C%20%27RESOLVED%27)%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20


> 8 апр. 2020 г., в 20:15, Вячеслав Коптилин  
> написал(а):
> 
> Folks,
> 
> I'd like to add ticket IGNITE-12805 "NullPointerException on node restart
> when 3rd party persistence and Ignite native persistence are used" to
> ignite-2.8.1 scope.
> 
> [1]  https://issues.apache.org/jira/browse/IGNITE-12805
> 
> Thanks,
> S.
> 
> вт, 7 апр. 2020 г. в 19:57, Ilya Kasnacheev :
> 
>> Hello!
>> 
>> Done!
>> 
>> Regards,
>> --
>> Ilya Kasnacheev
>> 
>> 
>> вт, 7 апр. 2020 г. в 12:31, Sergey :
>> 
>>> Hi,
>>> 
>>> I'm proposing to add
>>> https://issues.apache.org/jira/browse/IGNITE-12549  (fix iterators/scan
>>> queries for replicated caches)
>>> to 2.8.1.
>>> 
>>> Best regards,
>>> Sergey Kosarev.
>>> 
>>> 
>>> вс, 5 апр. 2020 г. в 01:22, Saikat Maitra :
>>> 
 Hi,
 
 I observed that we already have release 2.8.1 branch
 https://github.com/apache/ignite/tree/ignite-2.8.1
 
 In that case we should be ok to merge these 2 open PRs in master to
>> make
>>> it
 available for 2.9.0 release.
 
 https://github.com/apache/ignite/pull/7240
 https://github.com/apache/ignite/pull/7227
 
 Can you please review and confirm?
 
 Regards,
 Saikat
 
 On Fri, Mar 20, 2020 at 8:19 AM Maxim Muzafarov 
>>> wrote:
 
> Igniters,
> 
> I support Nikolay Izhikov as the release manager of 2.8.1 Apache
> Ignite release. Since no one else of committers, PMCs expressed a
> desire to lead this release I think we can close this question and
> focus on the release scope and dates.
> 
> 
> Ivan,
> 
> You helped me configuring TC.Bot that time, can you please help again
> and set `ignite-2.8.1` branch for guard under TC.Bot [1]? We should
> start collecting TC statistics for the release branch as early as
> possible.
> 
> 
> [1] https://mtcga.gridgain.com/guard.html
> 
> On Fri, 20 Mar 2020 at 14:48, Taras Ledkov 
>>> wrote:
>> 
>> Hi,
>> 
>> I propose to add the issue [1] related to SQL query execution to
>> this
> scope.
>> 
>> We had omitted this case and Ignite 2.8 contains serious SQL issue:
>> cursor of a local query is not thread-safe.
>> It is root cause of several SQL issue, e.g. JDBC thin client cannot
>> execute query  from replicated cache,
>> PME may hang after execute such queries from JDBC thin, etc.
>> 
>> [1]. https://issues.apache.org/jira/browse/IGNITE-12800
>> 
>> On 19.03.2020 17:52, Denis Magda wrote:
>>> Igniters,
>>> 
>>> As long as 2.8.1 is inevitable and we already keep adding
>> critical
> issues
>>> to the working queue, let's settle on the release time frames and
> decide
>>> who will be a release manager. This is the time proposed by Maxim
 and,
>>> personally, I concur with such a schedule:
>>> 
>>>- Scope Freeze: April 15, 2020
>>>- Code Freeze: April 22, 2020
>>>- Voting Date: April 27, 2020
>>>- Release Date: May 1, 2020
>>> 
>>> Do we agree on this time? Is there anybody who ready to drive the
> release
>>> as a release manager?
>>> 
>>> -
>>> Denis
>>> 
>>> 
>>> On Thu, Mar 19, 2020 at 5:50 AM Sergey Antonov <
> antonovserge...@gmail.com>
>>> wrote:
>>> 
 Folks,
 
 I'd like to add ticket IGNITE-12774 Transaction hangs after too
>>> many
> open
 files NIO exception [1] to ignite-2.8.1 scope.
 
 [1] https://issues.apache.org/jira/browse/IGNITE-12774
 
 ср, 18 мар. 2020 г. в 16:53, Maxim 

[jira] [Created] (IGNITE-12906) Add to IgniteWalConverter possibility output only hashes instead real data

2020-04-16 Thread Kirill Tkalenko (Jira)
Kirill Tkalenko created IGNITE-12906:


 Summary: Add to IgniteWalConverter possibility output only hashes 
instead real data
 Key: IGNITE-12906
 URL: https://issues.apache.org/jira/browse/IGNITE-12906
 Project: Ignite
  Issue Type: Improvement
Reporter: Kirill Tkalenko
Assignee: Kirill Tkalenko


Add flag to be able to hide sensitive data in WAL reader output. For example, 
we can use cryptographic hash to hide data and see some representation of it to 
know is it unique or not, with high probability.

If define binaryMetadata then output field name and hash field value, else 
output hash of source binary value

This change is required for user security reasons



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-12905) QueryCursorImpl missing custom spliterator() implementation

2020-04-16 Thread Johnny Galatikitis (Jira)
Johnny Galatikitis created IGNITE-12905:
---

 Summary: QueryCursorImpl missing custom spliterator() 
implementation
 Key: IGNITE-12905
 URL: https://issues.apache.org/jira/browse/IGNITE-12905
 Project: Ignite
  Issue Type: Bug
  Components: cache, general
Affects Versions: 2.8
 Environment: Windows 10
JDK 1.8.0_172
ignite-core 2.8.0
reactor-core 3.3.3
Reporter: Johnny Galatikitis



We are using apache ignite with reactor-core and since reactors upgrade from 
3.2.12 to 3.3.3 {code:java}
org.apache.ignite.internal.processors.cache.QueryCursorImpl.iterator
{code}
is called multiple times. It starts with:
1. calling iterable.spliterator().hasCharacteristics(Spliterator.SIZED), where 
iterable is instanceof QueryCursorImpl
2. calls default implementation Spliterators.spliteratorUnknownSize(iterator(), 
0)
3. which in turn calls ignite's QueryCursorImpl.iterator() call and that "uses 
it up" for subsequent calls, i.e. throw IgniteException "Iterator is already 
fetched or query was cancelled."



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-12904) ШптшеуЫусгкшен

2020-04-16 Thread Denis Garus (Jira)
Denis Garus created IGNITE-12904:


 Summary: ШптшеуЫусгкшен
 Key: IGNITE-12904
 URL: https://issues.apache.org/jira/browse/IGNITE-12904
 Project: Ignite
  Issue Type: Improvement
Reporter: Denis Garus






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-12903) Fix ML + SQL examples

2020-04-16 Thread Taras Ledkov (Jira)
Taras Ledkov created IGNITE-12903:
-

 Summary: Fix ML + SQL examples
 Key: IGNITE-12903
 URL: https://issues.apache.org/jira/browse/IGNITE-12903
 Project: Ignite
  Issue Type: Task
  Components: examples
Reporter: Taras Ledkov
Assignee: Taras Ledkov


The examples
{{DecisionTreeClassificationTrainerSQLInferenceExample}}
{{DecisionTreeClassificationTrainerSQLTableExample}}
are used CSVREAD function to initial load data into cluster.

Must be changed because this function is disabled by default



--
This message was sent by Atlassian Jira
(v8.3.4#803005)