Re: Unit testing of persistence & durable memory

[Dmitriy Setrakyan]

Thanks Dmitriy! Is it possible to start Ignite instance in the tests if
needed or is it only needed for basic isolated method tests?

D.

On Wed, Feb 7, 2018 at 6:24 AM, Dmitry Pavlov  wrote:

> Hi Igniters,
>
> Recently new unit tests suite was added to TC run chain, it is PDS unit
> test suite. See
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_
> IgnitePdsUnitTests
> and corresponding suite in code
> org.apache.ignite.testsuites.IgnitePdsUnitTestSuite
>
> This suite contain JUnit4 styled unit tests. Such tests never start Ignite
> instance, but checks only one aspect/unit behaviour of PDS/Durable
> memory/etc.
>
> If it is required. tests may use Mockito to stub unnecessary Ignite context
> elements and provide required behaviour. There are no real cluster building
> and/or real HDD disk usage, and such tests are quite fast (each <5s to run)
> and each test is quite small.
>
> Suite code was recently merged to master with throttling changes. Now it
> contains only 9 tests, but I am sure it will became full of tests. Please
> feel free to add yours.
>
> Sincerely,
> Dmitriy Pavlov
>

Re: Next Steps: GA Grid: Request to contribute GA library to Apache Ignite

[techbysample]

Denis,

Thank's for following up.. Please let me know once the paperwork has been
completed..

In addition, I agree with releasing GA Grid as part of ML framework in
Ignite 2.5.

Just let me know if you require  additional information beyond what I have
provided.

Regards,
Turik Campbell





--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: Eviction policies with persistence

[Denis Magda]

Thanks folks. I’ll improve the doc by 2.4 release:
IGNITE-7645 - Clarify eviction policy documentation 


—
Denis

> On Feb 6, 2018, at 11:25 PM, Alexey Goncharuk  
> wrote:
> 
> Denis,
> 
> Yes, if there are no concurrent updates and there are no ongoing
> transactions preventing the entries to be evicted, then the whole page will
> be purged.
> 
> 2018-02-07 0:23 GMT+03:00 Denis Magda :
> 
>> Alexey,
>> 
>> Let me understand this part because it’s still not crystal clear to me:
>> 
>>> Now as per the
>>> eviction mechanism, it is both per-page and per-entry: first, we choose a
>>> page which fits most for eviction, however, we cannot simply erase the
>> page
>>> because quite a lot of data structures are referring to that page. To
>> deal
>>> with it, we read keys that the chosen page contains and then clear
>>> individual cache entries, which in turn will clear up all necessary
>> links.
>> 
>> 
>> However will all the keys-values from a chosen page be removed eventually
>> during the eviction phase? If it’s true then it’s still a page based
>> eviction - we select 5 random oldest pages and purge all the key-values
>> from them.
>> 
>> —
>> Denis
>> 
>>> On Feb 4, 2018, at 12:05 PM, Alexey Goncharuk <
>> alexey.goncha...@gmail.com> wrote:
>>> 
>>> Guys,
>>> 
>>> To clarify the questions, I would like to reiterate over the mechanics of
>>> evictions and then suggest a renaming that should resolve such things in
>>> the future.
>>> 
>>> First of all, Lucas is right - eviction policy only makes sense when
>> native
>>> persistence is disabled because what it does is actually freeing up
>> memory
>>> when a user hits the memory limit. The only way to do this is to destroy
>>> inserted data because there is no other way to free memory. Now as per
>> the
>>> eviction mechanism, it is both per-page and per-entry: first, we choose a
>>> page which fits most for eviction, however, we cannot simply erase the
>> page
>>> because quite a lot of data structures are referring to that page. To
>> deal
>>> with it, we read keys that the chosen page contains and then clear
>>> individual cache entries, which in turn will clear up all necessary
>> links.
>>> If there are no concurrent updates, the page becomes empty and will be
>>> reused for other user data. This is exactly what is explained on the wiki
>>> page (at least in my reading of the page).
>>> 
>>> Second, at this point, I would rename page management mechanism with
>>> enabled persistence from 'eviction' to 'page replacement' or 'page
>>> swapping', because it does not destroy any data, but rather replaces a
>>> chosen buffer in memory from one page to another. The content of neither
>>> pages does not change during page replacement. This mechanism is unlikely
>>> to be selected by a user because the effectiveness of page replacements
>>> heavily depends on internal data structures and may change from version
>> to
>>> version, and may even be adaptive depending on the load pattern.
>>> 
>>> Hope this resolves the confusion.
>>> 
>>> --AG
>>> 
>>> 2018-02-03 1:03 GMT+03:00 Denis Magda :
>>> 
 Dmitriy,
 
 I’m surprised to hear that Random-LRU works at the entry level when
>> Ignite
 persistence is off. Frankly, this section confuses a lot:
 https://cwiki.apache.org/confluence/display/IGNITE/
 Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-
 underthehood-Entryeviction 
 
 While it was assumed that the entry-based eviction works only for
>> on-heap
 caches:
 https://apacheignite.readme.io/docs/evictions <
 https://apacheignite.readme.io/docs/evictions>
 
 Alex G., please chime in and clarify the misunderstanding.
 
 —
 Denis
 
> On Feb 2, 2018, at 4:01 AM, Dmitry Pavlov 
>> wrote:
> 
> Hi Val,
> 
> I think it is quite accurate because eviction in case PDS enabled or
>> not
> has quite different purposes.
> 
> 1. Let us consider PDS enabled and page eviction occurs. First of all
>> it
 is
> page based eviction, but not entry-based. Actually data is not removed,
 but
> only written to disk. We can address this page later by ID.
> PDS eviction is primarily the replacement of pages from memory to page
>> on
> disk. This eviction policy should ensure the maximum performance of the
> solution in the future. There is no data removal from grid in this
>> case.
> And Ignite does not allow users to configure the policy. If benchmarks
 show
> that a change in policy results in increased performance, then we can
> switch policy.
> 
> 2. Entry-based eviction (if there is no persistence is available) is
> slithly 

[jira] [Created] (IGNITE-7645) Clarify eviction policy documentation

[Denis Magda (JIRA)]

Denis Magda created IGNITE-7645:
---

 Summary: Clarify eviction policy documentation
 Key: IGNITE-7645
 URL: https://issues.apache.org/jira/browse/IGNITE-7645
 Project: Ignite
  Issue Type: Task
Reporter: Denis Magda
Assignee: Denis Magda
 Fix For: 2.4


Eviction policies work differently depending on the configuration that might be 
one of the following:
 * Just off-heap memory w/o Ignite persistence
 * off-heap memory + on-heap cache
 * off-heap memory + Ignite persistence
 * off-heap memory + swap or cache store

Cover all these scenarios on the main eviction doc 
page:https://apacheignite.readme.io/docs/evictions

More details: 
http://apache-ignite-developers.2346864.n4.nabble.com/Eviction-policies-with-persistence-td26588.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: HelloWorld!

[Denis Magda]

Hello Yaroslav,

Welcome to the Ignite community!

Please subscribe to both dev and user lists:
https://ignite.apache.org/community/resources.html#mail-lists

Get familiar with Ignite development process described here:
https://cwiki.apache.org/confluence/display/IGNITE/Development+Process

Instructions on how to contribute can be found here:
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute

Project setup in Intellij IDEA:
https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup

Once you got familiar and were able to run a few examples, you should pick
a Jira ticket you would like to start on. Send an email to the dev list sharing 
your JIRA id, so we can add you as a contributor in Jira.

These are the easy tickets to start with:
https://issues.apache.org/jira/browse/IGNITE-4549?jql=project%20%3D%20IGNITE%20AND%20labels%20in%20(newbie)%20and%20status%20%3D%20OPEN

While those are more advanced but appealing:
https://ignite.apache.org/community/contribute.html#pick-tickets

Looking forward to your contributions!

—
Denis

> On Feb 7, 2018, at 7:35 AM, Yaroslav Lymar  wrote:
> 
> I would like to get acquainted with the product and contribute to its
> development. I am middle-level developer and I would take some simple task
> to join the community.
> 
> 2018-02-07 18:22 GMT+03:00 Alexey Kuznetsov :
> 
>> Hello!
>> 
>> Do you have questions about Ignite?
>> Or willing to contribute?
>> 
>> :)
>> 
>> 
>> On Wed, Feb 7, 2018 at 7:46 PM, Yaroslav Lymar 
>> wrote:
>> 
>>> Hello anybody!
>>> 
>> 
>> 
>> 
>> --
>> Alexey Kuznetsov
>> 

Re: Removing "fabric" from Ignite binary package name

[Denis Magda]

Anton,

I don’t get how the hadoop editions are related to this task. The project is 
not named as “data fabric” for a while. Check up the site or docs.

The “fabric” word is being removed from all over the places and needs to be 
removed from the editions’ names.

As for the hadoop future, my personal position is to retire this component and 
forget about it. I would restart the conversation again after we done with 2.4.

—
Denis

> On Feb 7, 2018, at 2:13 AM, Anton Vinogradov  wrote:
> 
> Denis, Petr,
> 
> I checked PR and found we have *overcomplicated* logic with "fabric" and
> "hadoop" postfixs.
> 
> Do we really need to assembly 2 editions?
> "Hadoop" edition still valued?
> 
> My proposal is to get rid of "hadoop" edition and replace it with
> instruction of how to use "fabric" edition instead.
> Instruction will be pretty easy -> move "hadoop" folder from "optional" to
> root directory :)
> 
> In that case we can just remove all postfix logic from maven poms and
> simplify release process.
> 
> On Thu, Dec 28, 2017 at 9:20 PM, Denis Magda  wrote:
> 
>> Petr, thanks for solving it!
>> 
>> Hope that Anton V. or some other build master will double-check the
>> changes and merge them.
>> 
>> —
>> Denis
>> 
>>> On Dec 28, 2017, at 8:29 AM, Petr Ivanov  wrote:
>>> 
>>> IGNITE-7251 is done, needs review and some additional tests. See PR
>> #3315 [1].
>>> 
>>> 
>>> [1] https://github.com/apache/ignite/pull/3315 <
>> https://github.com/apache/ignite/pull/3315>
>>> 
>>> 
>>> 
 On 20 Dec 2017, at 23:15, Denis Magda  wrote:
 
 Petr, thanks, such a swift turnaround!
 
 Have you found the one who can asses and review the changes?
 
 Maintainers label might be helpful. Just ping them directly:
 https://cwiki.apache.org/confluence/display/IGNITE/How+
>> to+Contribute#HowtoContribute-ReviewProcessandMaintainers <
>> https://cwiki.apache.org/confluence/display/IGNITE/How+
>> to+Contribute#HowtoContribute-ReviewProcessandMaintainers>
 
 
 —
 Denis
 
> On Dec 20, 2017, at 12:24 AM, Petr Ivanov  wrote:
> 
> Assigned myself — done the same work while preparing RPM package.
> But for fixing DEVNOTES.txt waiting for review and merge of
>> IGNITE-7107 [1].
> 
> 
> [1] https://issues.apache.org/jira/browse/IGNITE-7107
> 
> 
> 
>> On 19 Dec 2017, at 22:55, Denis Magda  wrote:
>> 
>> All the bids were accepted and the verdict is executed:
>> https://issues.apache.org/jira/browse/IGNITE-7251 <
>> https://issues.apache.org/jira/browse/IGNITE-7251>
>> 
>> Who is ready to pick this up?
>> 
>> —
>> Denis
>> 
>>> On Dec 19, 2017, at 5:35 AM, Anton Vinogradov <
>> avinogra...@gridgain.com> wrote:
>>> 
>>> +1б фо шур
>>> 
>>> On Tue, Dec 19, 2017 at 9:59 AM, Vladimir Ozerov <
>> voze...@gridgain.com>
>>> wrote:
>>> 
 +1б вуаштшеудн
 
 On Tue, Dec 19, 2017 at 2:34 AM, Valentin Kulichenko <
 valentin.kuliche...@gmail.com> wrote:
 
> +1
> 
> On Mon, Dec 18, 2017 at 12:24 PM, Dmitriy Setrakyan <
 dsetrak...@apache.org
>> 
> wrote:
> 
>> +1 (completely agree)
>> 
>> On Mon, Dec 18, 2017 at 10:21 AM, Denis Magda 
 wrote:
>> 
>>> Igniters,
>>> 
>>> Apache Ignite binary releases still include “fabric” word in
>> their
> names:
>>> https://ignite.apache.org/download.cgi#binaries <
>>> https://ignite.apache.org/download.cgi#binaries>
>>> 
>>> For instance, this is a full name of the previous release -
>>> apache-ignite-fabric-2.3.0-bin.
>>> It’s a little oversight on our side because the project has not
>> been
>>> positioned as a fabric for a while.
>>> 
>>> Proposal! Remove “fabric” from the name and have the binary
>> releases
>> named
>>> as - apache-ignite-{version}-bin.
>>> 
>>> If we’re in consensus then let’s make the change in 2.4.
>>> 
>>> —
>>> Denis
>>> 
>> 
> 
 
>> 
> 
 
>>> 
>> 
>> 

Re: Abandoned Patch Available JIRA tickets

[Denis Magda]

Anyway, your suggestion requires to have self-disciplined committers that will 
be keeping track of the tickets they promised to review. 

I’m ready to propose a guideline here but the committers have to be committed 
to that. Otherwise, I’ll just waste my time.

—
Denis

> On Feb 7, 2018, at 1:41 AM, Andrey Kuznetsov  wrote:
> 
> Periodic punches can frustrate committers, and also it's unpleasant for
> contributors. Most IP->PA transitions are commented with something like
> "John Doe, this awesome feature is ready and needs your review." Maybe it's
> better to adopt following rule of thumb? If the change is clean and
> straightforward it should be reviewed is a day, otherwise committer puts a
> comment in Jira issue about planned review time.
> 
> Is this acceptable?
> 
> 2018-02-07 0:14 GMT+03:00 Denis Magda  >:
> 
>> I guess it’s all about discipline.
>> 
>> Committers need to walk-through a list of the pull-request regularly while
>> contributors have to remind of a pending pull-request periodically. So both
>> parts have to be proactive.
>> 
>> Another approach is to find a volunteer from the community who will keep
>> an eye on the contributions and spread them out among committers.
>> 
>> Not sure I like the latter approach and would rather go for the one when
>> both the committers and contributors are proactive and disciplined. But
>> guess what, if you want to make the contributors proactive then the
>> committers have to be an example.
>> 
>> —
>> Denis
>> 
>> 
>> 
> 
> 
> -- 
> Best regards,
>  Andrey Kuznetsov.

Re: Apache Ignite 2.4 release

[Dmitriy Setrakyan]

Agree on both, the performance fix and the spark data frames. Let's get
them into the release.

However, Raymond is right. We should know how long the performance fix will
take. If it adds another month to the development, we should include it
into the next release. I am hoping that it can be done faster though.


Alexey Goncharuk, Dmitriy Pavlov, any ideas?

D.

On Wed, Feb 7, 2018 at 9:07 AM, Nikolay Izhikov  wrote:

> Hello, Igniters.
>
> Please, consider including IGNITE-7337 - Spark Data Frames: support saving
> a data frame in Ignite [1] in the 2.4 release.
> It seems we can merge it into the master in a day or few.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-7337
>
>
> В Ср, 07/02/2018 в 08:35 -0800, Denis Magda пишет:
> > I’m voting for the blocker addition into the release. Sergey K. how will
> it affect your testing cycles? Do you need to re-run everything from
> scratch and how many days you need?
> >
> > —
> > Denis
> >
> > > On Feb 6, 2018, at 11:29 PM, Alexey Goncharuk <
> alexey.goncha...@gmail.com> wrote:
> > >
> > > Guys,
> > >
> > > Thanks to Dmitriy Pavlov we found the ticket [1] which causes a major
> > > slowdown when page replacement starts. Even though it's not a
> regression, I
> > > suggest we consider it a blocker for 2.4 because this is a huge
> performance
> > > issue which can make it virtually impossible to use native persistence
> when
> > > data size is significantly larger than memory size.
> > >
> > > Any objections?
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-7638
> > >
> > > 2018-01-30 17:10 GMT+03:00 Pavel Tupitsyn :
> > >
> > > > Igniters, I will handle 2.4 release if there are no objections.
> > > > Let's take a bit more time for testing and start vote by the end of
> this
> > > > week.
> > > >
> > > > Pavel
> > > >
> > > > On Sat, Jan 27, 2018 at 3:32 AM, Denis Magda 
> wrote:
> > > >
> > > > > Hi Vyacheslav,
> > > > >
> > > > > According to the previous review notes the impact of the changes
> might be
> > > > > significant, thus, I would recommend us to move the changes to the
> next
> > > > > release.
> > > > >
> > > > > BTW, don’t hesitate to ping reviewers more frequently if there is a
> > > > > pending/abandon review. We are all the people who are tend to
> forget or
> > > > > miss notifications ;)
> > > > >
> > > > > > On Jan 26, 2018, at 2:04 AM, Vyacheslav Daradur <
> daradu...@gmail.com>
> > > > >
> > > > > wrote:
> > > > > >
> > > > > > Hi, Vladimir, it's good news. I'm looking forward to new Ignite
> > > >
> > > > release!
> > > > > >
> > > > > > Could you please share a release schedule for 'varint'
> optimizations?
> > > > > >
> > > > > > The task [1] is waiting for review for 5 months.
> > > > > >
> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-5097
> > > > > >
> > > > > >
> > > > > > On Fri, Jan 26, 2018 at 12:51 PM, Vladimir Ozerov <
> > > >
> > > > voze...@gridgain.com>
> > > > > wrote:
> > > > > > > Hi Igniters,
> > > > > > >
> > > > > > > As far as I can see all required tasks and fixes were merged. I
> > > >
> > > > propose
> > > > > to
> > > > > > > take several days of silence to test what we've done and start
> vote at
> > > > >
> > > > > the
> > > > > > > beginning of the next week.
> > > > > > >
> > > > > > > Makes sense?
> > > > > > >
> > > > > > > On Mon, Jan 22, 2018 at 8:39 PM, Denis Magda <
> dma...@apache.org>
> > > >
> > > > wrote:
> > > > > > >
> > > > > > > > Ok, let’s target Wednesday as a code freeze date.
> > > > > > > >
> > > > > > > > Community members who are involved in 2.4 release please
> merge you
> > > > >
> > > > > fixes
> > > > > > > > and optimizations by that time.
> > > > > > > >
> > > > > > > > —
> > > > > > > > Denis
> > > > > > > >
> > > > > > > > > On Jan 22, 2018, at 8:24 AM, Anton Vinogradov <
> a...@apache.org>
> > > >
> > > > wrote:
> > > > > > > > >
> > > > > > > > > Issues
> > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-6711
> > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-6902
> > > > > > > > >
> > > > > > > > > are in master and ignite-2.4
> > > > > > > > >
> > > > > > > > > On Mon, Jan 22, 2018 at 6:43 PM, Denis Magda <
> dma...@apache.org>
> > > > >
> > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Igniters,
> > > > > > > > > >
> > > > > > > > > > What’s left and what prevents us from passing 2.4
> through the QA
> > > > >
> > > > > phase?
> > > > > > > > > >
> > > > > > > > > > Petr, Anton, Vladimir, Alex G., others please chime in.
> > > > > > > > > >
> > > > > > > > > > —
> > > > > > > > > > Denis
> > > > > > > > > >
> > > > > > > > > > > On Jan 17, 2018, at 7:05 PM, Dmitriy Setrakyan <
> > > > >
> > > > > dsetrak...@apache.org>
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > Great news, indeed! Looking forward to 2.4 release.
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Jan 17, 2018 at 2:55 AM, Vladimir Ozerov <
> > > > >
> > > > > 

Re: Apache Ignite 2.4 release

[Nikolay Izhikov]

Hello, Igniters.

Please, consider including IGNITE-7337 - Spark Data Frames: support saving a 
data frame in Ignite [1] in the 2.4 release.
It seems we can merge it into the master in a day or few.

[1] https://issues.apache.org/jira/browse/IGNITE-7337


В Ср, 07/02/2018 в 08:35 -0800, Denis Magda пишет:
> I’m voting for the blocker addition into the release. Sergey K. how will it 
> affect your testing cycles? Do you need to re-run everything from scratch and 
> how many days you need?
> 
> —
> Denis
> 
> > On Feb 6, 2018, at 11:29 PM, Alexey Goncharuk  
> > wrote:
> > 
> > Guys,
> > 
> > Thanks to Dmitriy Pavlov we found the ticket [1] which causes a major
> > slowdown when page replacement starts. Even though it's not a regression, I
> > suggest we consider it a blocker for 2.4 because this is a huge performance
> > issue which can make it virtually impossible to use native persistence when
> > data size is significantly larger than memory size.
> > 
> > Any objections?
> > 
> > [1] https://issues.apache.org/jira/browse/IGNITE-7638
> > 
> > 2018-01-30 17:10 GMT+03:00 Pavel Tupitsyn :
> > 
> > > Igniters, I will handle 2.4 release if there are no objections.
> > > Let's take a bit more time for testing and start vote by the end of this
> > > week.
> > > 
> > > Pavel
> > > 
> > > On Sat, Jan 27, 2018 at 3:32 AM, Denis Magda  wrote:
> > > 
> > > > Hi Vyacheslav,
> > > > 
> > > > According to the previous review notes the impact of the changes might 
> > > > be
> > > > significant, thus, I would recommend us to move the changes to the next
> > > > release.
> > > > 
> > > > BTW, don’t hesitate to ping reviewers more frequently if there is a
> > > > pending/abandon review. We are all the people who are tend to forget or
> > > > miss notifications ;)
> > > > 
> > > > > On Jan 26, 2018, at 2:04 AM, Vyacheslav Daradur 
> > > > 
> > > > wrote:
> > > > > 
> > > > > Hi, Vladimir, it's good news. I'm looking forward to new Ignite
> > > 
> > > release!
> > > > > 
> > > > > Could you please share a release schedule for 'varint' optimizations?
> > > > > 
> > > > > The task [1] is waiting for review for 5 months.
> > > > > 
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-5097
> > > > > 
> > > > > 
> > > > > On Fri, Jan 26, 2018 at 12:51 PM, Vladimir Ozerov <
> > > 
> > > voze...@gridgain.com>
> > > > wrote:
> > > > > > Hi Igniters,
> > > > > > 
> > > > > > As far as I can see all required tasks and fixes were merged. I
> > > 
> > > propose
> > > > to
> > > > > > take several days of silence to test what we've done and start vote 
> > > > > > at
> > > > 
> > > > the
> > > > > > beginning of the next week.
> > > > > > 
> > > > > > Makes sense?
> > > > > > 
> > > > > > On Mon, Jan 22, 2018 at 8:39 PM, Denis Magda 
> > > 
> > > wrote:
> > > > > > 
> > > > > > > Ok, let’s target Wednesday as a code freeze date.
> > > > > > > 
> > > > > > > Community members who are involved in 2.4 release please merge you
> > > > 
> > > > fixes
> > > > > > > and optimizations by that time.
> > > > > > > 
> > > > > > > —
> > > > > > > Denis
> > > > > > > 
> > > > > > > > On Jan 22, 2018, at 8:24 AM, Anton Vinogradov 
> > > 
> > > wrote:
> > > > > > > > 
> > > > > > > > Issues
> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-6711
> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-6902
> > > > > > > > 
> > > > > > > > are in master and ignite-2.4
> > > > > > > > 
> > > > > > > > On Mon, Jan 22, 2018 at 6:43 PM, Denis Magda 
> > > > 
> > > > wrote:
> > > > > > > > 
> > > > > > > > > Igniters,
> > > > > > > > > 
> > > > > > > > > What’s left and what prevents us from passing 2.4 through the 
> > > > > > > > > QA
> > > > 
> > > > phase?
> > > > > > > > > 
> > > > > > > > > Petr, Anton, Vladimir, Alex G., others please chime in.
> > > > > > > > > 
> > > > > > > > > —
> > > > > > > > > Denis
> > > > > > > > > 
> > > > > > > > > > On Jan 17, 2018, at 7:05 PM, Dmitriy Setrakyan <
> > > > 
> > > > dsetrak...@apache.org>
> > > > > > > > > wrote:
> > > > > > > > > > 
> > > > > > > > > > Great news, indeed! Looking forward to 2.4 release.
> > > > > > > > > > 
> > > > > > > > > > On Wed, Jan 17, 2018 at 2:55 AM, Vladimir Ozerov <
> > > > 
> > > > voze...@gridgain.com
> > > > > > > > 
> > > > > > > > > > wrote:
> > > > > > > > > > 
> > > > > > > > > > > Igniters,
> > > > > > > > > > > 
> > > > > > > > > > > Great news - two very important tickets - baseline 
> > > > > > > > > > > topology and
> > > > 
> > > > Java 8
> > > > > > > > > > > support - were merged to master. At this point it makes 
> > > > > > > > > > > sense to
> > > > > > > 
> > > > > > > create
> > > > > > > > > a
> > > > > > > > > > > branch to stabilize the release and finalize outstanding
> > > 
> > > tickets. I
> > > > > > > > > created
> > > > > > > > > > > the branch *ignite-2.4*. Please follow the rules below 

Re: Apache Ignite 2.4 release

[Denis Magda]

I’m voting for the blocker addition into the release. Sergey K. how will it 
affect your testing cycles? Do you need to re-run everything from scratch and 
how many days you need?

—
Denis

> On Feb 6, 2018, at 11:29 PM, Alexey Goncharuk  
> wrote:
> 
> Guys,
> 
> Thanks to Dmitriy Pavlov we found the ticket [1] which causes a major
> slowdown when page replacement starts. Even though it's not a regression, I
> suggest we consider it a blocker for 2.4 because this is a huge performance
> issue which can make it virtually impossible to use native persistence when
> data size is significantly larger than memory size.
> 
> Any objections?
> 
> [1] https://issues.apache.org/jira/browse/IGNITE-7638
> 
> 2018-01-30 17:10 GMT+03:00 Pavel Tupitsyn :
> 
>> Igniters, I will handle 2.4 release if there are no objections.
>> Let's take a bit more time for testing and start vote by the end of this
>> week.
>> 
>> Pavel
>> 
>> On Sat, Jan 27, 2018 at 3:32 AM, Denis Magda  wrote:
>> 
>>> Hi Vyacheslav,
>>> 
>>> According to the previous review notes the impact of the changes might be
>>> significant, thus, I would recommend us to move the changes to the next
>>> release.
>>> 
>>> BTW, don’t hesitate to ping reviewers more frequently if there is a
>>> pending/abandon review. We are all the people who are tend to forget or
>>> miss notifications ;)
>>> 
 On Jan 26, 2018, at 2:04 AM, Vyacheslav Daradur 
>>> wrote:
 
 Hi, Vladimir, it's good news. I'm looking forward to new Ignite
>> release!
 
 Could you please share a release schedule for 'varint' optimizations?
 
 The task [1] is waiting for review for 5 months.
 
 [1] https://issues.apache.org/jira/browse/IGNITE-5097
 
 
 On Fri, Jan 26, 2018 at 12:51 PM, Vladimir Ozerov <
>> voze...@gridgain.com>
>>> wrote:
> Hi Igniters,
> 
> As far as I can see all required tasks and fixes were merged. I
>> propose
>>> to
> take several days of silence to test what we've done and start vote at
>>> the
> beginning of the next week.
> 
> Makes sense?
> 
> On Mon, Jan 22, 2018 at 8:39 PM, Denis Magda 
>> wrote:
> 
>> Ok, let’s target Wednesday as a code freeze date.
>> 
>> Community members who are involved in 2.4 release please merge you
>>> fixes
>> and optimizations by that time.
>> 
>> —
>> Denis
>> 
>>> On Jan 22, 2018, at 8:24 AM, Anton Vinogradov 
>> wrote:
>>> 
>>> Issues
>>> https://issues.apache.org/jira/browse/IGNITE-6711
>>> https://issues.apache.org/jira/browse/IGNITE-6902
>>> 
>>> are in master and ignite-2.4
>>> 
>>> On Mon, Jan 22, 2018 at 6:43 PM, Denis Magda 
>>> wrote:
>>> 
 Igniters,
 
 What’s left and what prevents us from passing 2.4 through the QA
>>> phase?
 
 Petr, Anton, Vladimir, Alex G., others please chime in.
 
 —
 Denis
 
> On Jan 17, 2018, at 7:05 PM, Dmitriy Setrakyan <
>>> dsetrak...@apache.org>
 wrote:
> 
> Great news, indeed! Looking forward to 2.4 release.
> 
> On Wed, Jan 17, 2018 at 2:55 AM, Vladimir Ozerov <
>>> voze...@gridgain.com
>>> 
> wrote:
> 
>> Igniters,
>> 
>> Great news - two very important tickets - baseline topology and
>>> Java 8
>> support - were merged to master. At this point it makes sense to
>> create
 a
>> branch to stabilize the release and finalize outstanding
>> tickets. I
 created
>> the branch *ignite-2.4*. Please follow the rules below when
>> merging
>> new
>> commits to master:
>> 1) If ticket is targeted for 2.4 release, please merge to master,
>>> then
>> cherry-pick to ignite-2.4
>> 2) Otherwise just merge to master.
>> 
>> Vladimir.
>> 
>> On Tue, Jan 16, 2018 at 12:32 PM, Anton Vinogradov <
>> avinogra...@gridgain.com
>>> wrote:
>> 
>>> Denis,
>>> 
>>> I'll review https://issues.apache.org/jira/browse/IGNITE-6902
>>> 
>>> On Fri, Jan 12, 2018 at 11:45 PM, Denis Magda <
>> dma...@apache.org>
 wrote:
>>> 
 Here is the page create before:
 https://cwiki.apache.org/confluence/display/IGNITE/
>> Apache+Ignite+2.4
 <
 https://cwiki.apache.org/confluence/display/IGNITE/
>> Apache+Ignite+2.4>
 
 This is good news. It will be perfect if we also release as
>> many
>>> "required
 tickets" as we can:
 https://cwiki.apache.org/confluence/display/IGNITE/
>> Apache+Ignite+2.4
 <
 https://cwiki.apache.org/confluence/display/IGNITE/
>> 

[jira] [Created] (IGNITE-7644) Add an utility to export all key-value data from a persisted partition

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-7644:


 Summary: Add an utility to export all key-value data from a 
persisted partition
 Key: IGNITE-7644
 URL: https://issues.apache.org/jira/browse/IGNITE-7644
 Project: Ignite
  Issue Type: Improvement
  Components: persistence
Affects Versions: 2.1
Reporter: Alexey Goncharuk
 Fix For: 2.5


We need an emergency utility analogous to pgdump which will be able to 
full-scan all PDS partition pages and extract all survived data in some form 
that later can be uploaded back to Ignite cluster



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3488: IGNITE-7605: Add more SQL parser tests for COPY c...

[gg-shq]

GitHub user gg-shq opened a pull request:

https://github.com/apache/ignite/pull/3488

IGNITE-7605: Add more SQL parser tests for COPY command



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7605

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3488.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3488


commit 3175214a8f12fe9a945b3020dabb5b19dee5edd3
Author: gg-shq 
Date:   2018-02-07T16:00:12Z

IGNITE-7606: Added BATCH_SIZE tests




---

Re: HelloWorld!

[Alexey Kuznetsov]

Hi!

Please get familiar with Ignite development process described here:
https://cwiki.apache.org/confluence/display/IGNITE/Development+Process

Subscribe to both dev and user lists:
https://ignite.apache.org/community/resources.html#mail-lists <
https://ignite.apache.org/community/resources.html#mail-lists>

Instructions on how to contribute can be found here:
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute

Project setup in Intellij IDEA:
https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup

Once you got familiar and were able to run a few examples, you should pick
a Jira ticket you would like to start on.

These are the easy tickets to start with:
https://issues.apache.org/jira/browse/IGNITE-4549?jql=
project%20%3D%20IGNITE%20AND%20labels%20in%20(newbie)%
20and%20status%20%3D%20OPEN

While those are more advanced but appealing:
https://ignite.apache.org/community/contribute.html#pick-ticket

Looking forward to your contributions!


On Wed, Feb 7, 2018 at 10:35 PM, Yaroslav Lymar 
wrote:

> I would like to get acquainted with the product and contribute to its
> development. I am middle-level developer and I would take some simple task
> to join the community.
>
> 2018-02-07 18:22 GMT+03:00 Alexey Kuznetsov :
>
> > Hello!
> >
> > Do you have questions about Ignite?
> > Or willing to contribute?
> >
> > :)
> >
> >
> > On Wed, Feb 7, 2018 at 7:46 PM, Yaroslav Lymar <
> slavalymar.w...@gmail.com>
> > wrote:
> >
> > > Hello anybody!
> > >
> >
> >
> >
> > --
> > Alexey Kuznetsov
> >
>



-- 
Alexey Kuznetsov

[GitHub] ignite pull request #3487: IGNITE-7535: Implement CHARSET option in COPY ......

[gg-shq]

GitHub user gg-shq opened a pull request:

https://github.com/apache/ignite/pull/3487

IGNITE-7535: Implement CHARSET option in COPY ... FORMAT CSV command



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7535

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3487.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3487


commit deb7994f0fd4233e3e0b699794b9066af87195c7
Author: gg-shq 
Date:   2018-01-19T12:13:37Z

IGNITE-6917: Intermediate commit

commit e7747a58c2cdacc6987d625a46d1f79a81863cd3
Author: gg-shq 
Date:   2018-01-19T17:21:50Z

IGNITE-6917: Intermediate commit

commit 6f37e6751285a96bdf757b392e1d4113bb47ee48
Author: gg-shq 
Date:   2018-01-19T17:30:22Z

IGNITE-6917: Intermediate commit

commit 49f0324c77d0bb3b4ec87317b1ecbde1bd6f34b1
Author: gg-shq 
Date:   2018-01-22T10:27:34Z

IGNITE-6917: Intermediate commit

commit a5bec61d41d8dc242cfbf11a7cf03c23bbbcd7c3
Author: gg-shq 
Date:   2018-01-22T12:25:04Z

IGNITE-6917: Intermediate commit

commit e18e18696fc92b93b17decf087721c693625ac36
Author: gg-shq 
Date:   2018-01-22T12:35:56Z

IGNITE-6917: Intermediate commit

commit 990c04919e181535e57290ee2516a9603657c160
Author: gg-shq 
Date:   2018-01-22T16:18:18Z

IGNITE-6917: Intermediate commit

commit 8b163410845a6e6233fa8a2746402651ccea3f69
Author: gg-shq 
Date:   2018-01-22T17:16:20Z

IGNITE-6917: Intermediate commit

commit faf762815ef58865c560d6de722be446e429c61d
Author: gg-shq 
Date:   2018-01-23T12:37:56Z

IGNITE-6917: Intermediate commit

commit f79343f04360b913b32403d5aa0defaf5d04b357
Author: gg-shq 
Date:   2018-01-23T12:40:34Z

IGNITE-6917: Intermediate commit

commit efc5d7ab9bad52aaad0872977495a158b0e47770
Author: gg-shq 
Date:   2018-01-23T13:50:04Z

IGNITE-6917: Added BATCH_SIZE parameter to COPY SQL command for internal 
testing. Adding tests.

commit b61db5de48d9a91a658f6133a2ad2544f358ebbf
Author: gg-shq 
Date:   2018-01-24T11:18:55Z

IGNITE-6917: Adding tests. Clarifying default columns set.

commit aa31488b74c74f881c247339a4b2bd31bf45b849
Author: gg-shq 
Date:   2018-01-24T19:00:17Z

IGNITE-6917: More tests, more logging, cleanups, streaming CSV decoder

commit 01125f4bb68bc4ae958cae1d2f8f7dee493fa55e
Author: gg-shq 
Date:   2018-01-25T11:54:29Z

IGNITE-6917: Javadoc, added BulkLoadCacheWriter.

commit 1a21cd91b3571a23d21bab7cb653478312178bb0
Author: gg-shq 
Date:   2018-01-25T12:29:03Z

IGNITE-6917: Javadoc, javadoc, javadoc.

commit a34060392bd5d86b0118fbd26127460d54f918c3
Author: gg-shq 
Date:   2018-01-25T12:29:57Z

IGNITE-6917: Fixed a syntax error added involuntarily in the previous 
commit.

commit ccaef2e349a728145c77c50d96d24e8a38ac35e1
Author: gg-shq 
Date:   2018-01-25T13:31:40Z

IGNITE-6917: Fixed charset decoder bugs, tests, handling of empty lines

commit b4cf0a4a4fb6cf3c6c35f09fe99ac5954541a679
Author: gg-shq 
Date:   2018-01-26T10:54:01Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-6917-1

# Conflicts:
#   modules/core/src/main/java/org/apache/ignite/internal/sql/SqlParser.java
#   
modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/IgniteH2Indexing.java

commit 26694bc00b76895d4f22e8416af29000197230ec
Author: gg-shq 
Date:   2018-01-26T12:45:32Z

IGNITE-6917: Moved syntax tests to a separate file, moved truncated rows 
handling from UpdatePlan.processRow() to a different place, minor changes

commit 1d7a9f8818dff4c4bc6a8f9d509a23be394b3e59
Author: gg-shq 
Date:   2018-01-26T12:55:47Z

IGNITE-6917: Find input files by using IgniteUtils.resolveIgnitePath(), 
test fixes.

commit 58c9b2dc7190c149e9c9fa377a2581849bb41420
Author: gg-shq 
Date:   2018-01-26T12:58:58Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-6917-1

commit 47eace53c82770b076b26bbba87944c872a941ad
Author: gg-shq 
Date:   2018-01-26T13:06:11Z

IGNITE-6917: Javadoc, tidying up.

commit ba0f9c822a0873662a5505892956ca4e68d87e56
Author: gg-shq 
Date:   2018-01-26T14:53:10Z

IGNITE-6917: Added error reporting and tests for batch mode and into jdbc2 
driver.

commit 6e550ef3f0de02ceb0a24e4787906cacb320e54e
Author: gg-shq 
Date:   2018-01-26T14:55:37Z

IGNITE-6917: Minor fixes in testMultipleStatement()

commit 8902a6c061fbd5ea0a6ac203f03e079b3499bd54
Author: gg-shq 

Re: Removing "fabric" from Ignite binary package name

[Valentin Kulichenko]

Anton,

We already had a discussion about this:
http://apache-ignite-developers.2346864.n4.nabble.com/ignite-spark-module-in-Hadoop-Accelerator-td12343.html

While I'm personally very much for getting rid of two separate editions, we
didn't come to consensus there.

-Val

On Wed, Feb 7, 2018 at 2:13 AM, Anton Vinogradov  wrote:

> Denis, Petr,
>
> I checked PR and found we have *overcomplicated* logic with "fabric" and
> "hadoop" postfixs.
>
> Do we really need to assembly 2 editions?
> "Hadoop" edition still valued?
>
> My proposal is to get rid of "hadoop" edition and replace it with
> instruction of how to use "fabric" edition instead.
> Instruction will be pretty easy -> move "hadoop" folder from "optional" to
> root directory :)
>
> In that case we can just remove all postfix logic from maven poms and
> simplify release process.
>
> On Thu, Dec 28, 2017 at 9:20 PM, Denis Magda  wrote:
>
> > Petr, thanks for solving it!
> >
> > Hope that Anton V. or some other build master will double-check the
> > changes and merge them.
> >
> > —
> > Denis
> >
> > > On Dec 28, 2017, at 8:29 AM, Petr Ivanov  wrote:
> > >
> > > IGNITE-7251 is done, needs review and some additional tests. See PR
> > #3315 [1].
> > >
> > >
> > > [1] https://github.com/apache/ignite/pull/3315 <
> > https://github.com/apache/ignite/pull/3315>
> > >
> > >
> > >
> > >> On 20 Dec 2017, at 23:15, Denis Magda  wrote:
> > >>
> > >> Petr, thanks, such a swift turnaround!
> > >>
> > >> Have you found the one who can asses and review the changes?
> > >>
> > >> Maintainers label might be helpful. Just ping them directly:
> > >> https://cwiki.apache.org/confluence/display/IGNITE/How+
> > to+Contribute#HowtoContribute-ReviewProcessandMaintainers <
> > https://cwiki.apache.org/confluence/display/IGNITE/How+
> > to+Contribute#HowtoContribute-ReviewProcessandMaintainers>
> > >>
> > >>
> > >> —
> > >> Denis
> > >>
> > >>> On Dec 20, 2017, at 12:24 AM, Petr Ivanov 
> wrote:
> > >>>
> > >>> Assigned myself — done the same work while preparing RPM package.
> > >>> But for fixing DEVNOTES.txt waiting for review and merge of
> > IGNITE-7107 [1].
> > >>>
> > >>>
> > >>> [1] https://issues.apache.org/jira/browse/IGNITE-7107
> > >>>
> > >>>
> > >>>
> >  On 19 Dec 2017, at 22:55, Denis Magda  wrote:
> > 
> >  All the bids were accepted and the verdict is executed:
> >  https://issues.apache.org/jira/browse/IGNITE-7251 <
> > https://issues.apache.org/jira/browse/IGNITE-7251>
> > 
> >  Who is ready to pick this up?
> > 
> >  —
> >  Denis
> > 
> > > On Dec 19, 2017, at 5:35 AM, Anton Vinogradov <
> > avinogra...@gridgain.com> wrote:
> > >
> > > +1б фо шур
> > >
> > > On Tue, Dec 19, 2017 at 9:59 AM, Vladimir Ozerov <
> > voze...@gridgain.com>
> > > wrote:
> > >
> > >> +1б вуаштшеудн
> > >>
> > >> On Tue, Dec 19, 2017 at 2:34 AM, Valentin Kulichenko <
> > >> valentin.kuliche...@gmail.com> wrote:
> > >>
> > >>> +1
> > >>>
> > >>> On Mon, Dec 18, 2017 at 12:24 PM, Dmitriy Setrakyan <
> > >> dsetrak...@apache.org
> > 
> > >>> wrote:
> > >>>
> >  +1 (completely agree)
> > 
> >  On Mon, Dec 18, 2017 at 10:21 AM, Denis Magda <
> dma...@apache.org>
> > >> wrote:
> > 
> > > Igniters,
> > >
> > > Apache Ignite binary releases still include “fabric” word in
> > their
> > >>> names:
> > > https://ignite.apache.org/download.cgi#binaries <
> > > https://ignite.apache.org/download.cgi#binaries>
> > >
> > > For instance, this is a full name of the previous release -
> > > apache-ignite-fabric-2.3.0-bin.
> > > It’s a little oversight on our side because the project has not
> > been
> > > positioned as a fabric for a while.
> > >
> > > Proposal! Remove “fabric” from the name and have the binary
> > releases
> >  named
> > > as - apache-ignite-{version}-bin.
> > >
> > > If we’re in consensus then let’s make the change in 2.4.
> > >
> > > —
> > > Denis
> > >
> > 
> > >>>
> > >>
> > 
> > >>>
> > >>
> > >
> >
> >
>

[GitHub] ignite pull request #3485: IGNITE-7586: Added COPY command into the JDBC exa...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3485


---

Re: HelloWorld!

[Yaroslav Lymar]

I would like to get acquainted with the product and contribute to its
development. I am middle-level developer and I would take some simple task
to join the community.

2018-02-07 18:22 GMT+03:00 Alexey Kuznetsov :

> Hello!
>
> Do you have questions about Ignite?
> Or willing to contribute?
>
> :)
>
>
> On Wed, Feb 7, 2018 at 7:46 PM, Yaroslav Lymar 
> wrote:
>
> > Hello anybody!
> >
>
>
>
> --
> Alexey Kuznetsov
>

Re: Stop nodes after test by default - IGNITE-6842

[Maxim Muzafarov]

Thank you all,

I'll add this comment's for JIRA ticket, if you don't mind.

ср, 7 февр. 2018 г. в 15:16, Dmitry Pavlov :

> Yes, this solution allows to cover both cases:
> a) not stopped node from previous test and
> b) allows to remove useless code that stops Ignite nodes from each test.
>
> ср, 7 февр. 2018 г. в 15:13, Anton Vinogradov :
>
> > Maxim,
> >
> > We discussed with Dima privately, and decided
> >
> > 1) We have to assert that there is no alive nodes at GridAbstractTest's
> > beforeTestsStarted
> > 2) We have to kill all alive nodes (without force) at GridAbstractTest's
> > afterTestsStopped
> > 3) In case of any exceptions at #2 we have to see test error
> > 4) We can get rid of all useless stopAllGrids at GridAbstractTest's
> > subclasses.
> >
> >
> >
> > On Wed, Feb 7, 2018 at 2:32 PM, Dmitry Pavlov 
> > wrote:
> >
> > > > Let's just add stopAllGrids(flase) to GridAbstractTest 's
> > > afterTestsStopped
> > > method body.
> > >
> > > Can't agree with it becase implicit silent shutdown of nodes from test
> > > framework may cause a lot of bugs introduced to Ignite.
> > >
> > > I think stop+test fail should occur.
> > > In that case author of incorrect test or not consistent Ignite
> shutdown
> > > will see problem.
> > >
> > > 'Fail fast' if always better than hidding real problem under automatic
> > > framework feature.
> > >
> > > ср, 7 февр. 2018 г. в 14:05, Anton Vinogradov <
> avinogra...@gridgain.com
> > >:
> > >
> > > > Hi all,
> > > >
> > > > > I've made some research about this problem and i think that in
> > general
> > > we
> > > > > should move stopAllGrids method in GridAbstractTest class to
> > > > > afterTestsStopped method with some changes. Am I right?
> > > > Let's just add stopAllGrids(flase) to GridAbstractTest 's
> > > > afterTestsStopped method
> > > > body.
> > > >
> > > > > I have a question about stopAllGrids(boolean cancel) this "cancel"
> > > > That's just a flag means "do not wait for any operations finish"
> > > > See no reason to set it true in that case.
> > > >
> > > > > If you delegate closing to afterTestsStopped this will affect only
> > > > > last test (method).
> > > > The idea is to stop all nodes at last test's method finish.
> > > >
> > > > >  Nodes that survive between tests can affect successive
> > > > tests.
> > > > Thats ok. we have a lot tests where nodes reusable between test's
> > > methods.
> > > >
> > > >
> > > > On Wed, Feb 7, 2018 at 1:20 PM, Dmitry Pavlov  >
> > > > wrote:
> > > >
> > > > > Hi Igniters,
> > > > >
> > > > > IMO nodes that survive between tests is not general practice and
> I'm
> > > not
> > > > > sure is a best practice. I suggest to mark such tests with some
> > method
> > > > > overriden from AbstractTest.
> > > > >
> > > > > About cancel flag, please note stopAllGrids(boolean cancel)
> > > cancel=false
> > > > > allows to wait of checkpoint ends in case persistence enabled.
> > > > >
> > > > > I still suggest to avoid stopping any nodes by test, but validate
> not
> > > > > stopped node exist and fail test instead of siltent implicit
> actions.
> > > > >
> > > > > Sincerely,
> > > > > Dmitriy Pavlov
> > > > >
> > > > > ср, 7 февр. 2018 г. в 13:04, Andrey Kuznetsov :
> > > > >
> > > > > > Hi Maxim,
> > > > > >
> > > > > > Regarding your first question, the use of afterTestsStopped is
> not
> > > > enough
> > > > > > to stop all nodes, since each individual test (method) can start
> > > custom
> > > > > set
> > > > > > of notes during its operation, and this very test should stop all
> > > those
> > > > > > nodes. If you delegate closing to afterTestsStopped this will
> > affect
> > > > only
> > > > > > last test (method). Nodes that survive between tests can affect
> > > > > successive
> > > > > > tests.
> > > > > >
> > > > > > 2018-02-07 1:10 GMT+03:00 Maxim Muzafarov :
> > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > I've made some research about this problem and i think that in
> > > > general
> > > > > we
> > > > > > > should move stopAllGrids method in GridAbstractTest class to
> > > > > > > afterTestsStopped method with some changes. Am I right?
> > > > > > >
> > > > > > > Also, I have a question about stopAllGrids(boolean cancel) this
> > > > > "cancel"
> > > > > > > argument. Why in some cases we should interrupt ComputeJob and
> in
> > > > some
> > > > > > > cases shouldn't? For example here:
> > > > > > > IgniteBaselineAffinityTopologyActivationTest#afterTest
> > > > > > > we call method stopAllGrids(false) this way. Why not "true"
> > > argument
> > > > > > > instead?
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > Best regards,
> > > > > >   Andrey Kuznetsov.
> > > > > >
> > > > >
> > > >
> > >
> >
>

[GitHub] ignite pull request #3468: IGNITE-7508: Fix contention on system property ac...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3468


---

Re: HelloWorld!

[Alexey Kuznetsov]

Hello!

Do you have questions about Ignite?
Or willing to contribute?

:)


On Wed, Feb 7, 2018 at 7:46 PM, Yaroslav Lymar 
wrote:

> Hello anybody!
>



-- 
Alexey Kuznetsov

[GitHub] ignite pull request #3486: Ignite 7263

[pvinokurov]

GitHub user pvinokurov opened a pull request:

https://github.com/apache/ignite/pull/3486

Ignite 7263



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7263

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3486.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3486


commit a311697c4881efe0eb44da27ff0fc309e0e1a846
Author: Dmitriy Govorukhin 
Date:   2018-01-31T08:43:38Z

GG-13407 Skip apply procedure PITR if initial wal pointer is null. Case 
handling when we have some cache for rebalancing but we do not have wal point 
for recovery.

commit a98479b9192bc1e6cedeb9607aff6e5f7334d996
Author: pvinokurov 
Date:   2018-02-07T08:59:48Z

IGNITE-7263 Ignore client configuration for daemon node




---

Unit testing of persistence & durable memory

[Dmitry Pavlov]

Hi Igniters,

Recently new unit tests suite was added to TC run chain, it is PDS unit
test suite. See
https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_IgnitePdsUnitTests
and corresponding suite in code
org.apache.ignite.testsuites.IgnitePdsUnitTestSuite

This suite contain JUnit4 styled unit tests. Such tests never start Ignite
instance, but checks only one aspect/unit behaviour of PDS/Durable
memory/etc.

If it is required. tests may use Mockito to stub unnecessary Ignite context
elements and provide required behaviour. There are no real cluster building
and/or real HDD disk usage, and such tests are quite fast (each <5s to run)
and each test is quite small.

Suite code was recently merged to master with throttling changes. Now it
contains only 9 tests, but I am sure it will became full of tests. Please
feel free to add yours.

Sincerely,
Dmitriy Pavlov

[GitHub] ignite pull request #3485: IGNITE-7586: Added COPY command into the JDBC exa...

[gg-shq]

GitHub user gg-shq opened a pull request:

https://github.com/apache/ignite/pull/3485

IGNITE-7586: Added COPY command into the JDBC example.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7586

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3485.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3485


commit 530756813411d51e731ffd5f145c7c9f57adbb31
Author: gg-shq 
Date:   2018-02-07T13:38:59Z

IGNITE-7586: Added COPY command into the JDBC example.




---

HelloWorld!

[Yaroslav Lymar]

Hello anybody!

[GitHub] ignite pull request #2558: IGNITE-6217 Add benchmark to compare JDBC drivers...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2558


---

Re: Stop nodes after test by default - IGNITE-6842

[Dmitry Pavlov]

Yes, this solution allows to cover both cases:
a) not stopped node from previous test and
b) allows to remove useless code that stops Ignite nodes from each test.

ср, 7 февр. 2018 г. в 15:13, Anton Vinogradov :

> Maxim,
>
> We discussed with Dima privately, and decided
>
> 1) We have to assert that there is no alive nodes at GridAbstractTest's
> beforeTestsStarted
> 2) We have to kill all alive nodes (without force) at GridAbstractTest's
> afterTestsStopped
> 3) In case of any exceptions at #2 we have to see test error
> 4) We can get rid of all useless stopAllGrids at GridAbstractTest's
> subclasses.
>
>
>
> On Wed, Feb 7, 2018 at 2:32 PM, Dmitry Pavlov 
> wrote:
>
> > > Let's just add stopAllGrids(flase) to GridAbstractTest 's
> > afterTestsStopped
> > method body.
> >
> > Can't agree with it becase implicit silent shutdown of nodes from test
> > framework may cause a lot of bugs introduced to Ignite.
> >
> > I think stop+test fail should occur.
> > In that case author of incorrect test or not consistent Ignite  shutdown
> > will see problem.
> >
> > 'Fail fast' if always better than hidding real problem under automatic
> > framework feature.
> >
> > ср, 7 февр. 2018 г. в 14:05, Anton Vinogradov  >:
> >
> > > Hi all,
> > >
> > > > I've made some research about this problem and i think that in
> general
> > we
> > > > should move stopAllGrids method in GridAbstractTest class to
> > > > afterTestsStopped method with some changes. Am I right?
> > > Let's just add stopAllGrids(flase) to GridAbstractTest 's
> > > afterTestsStopped method
> > > body.
> > >
> > > > I have a question about stopAllGrids(boolean cancel) this "cancel"
> > > That's just a flag means "do not wait for any operations finish"
> > > See no reason to set it true in that case.
> > >
> > > > If you delegate closing to afterTestsStopped this will affect only
> > > > last test (method).
> > > The idea is to stop all nodes at last test's method finish.
> > >
> > > >  Nodes that survive between tests can affect successive
> > > tests.
> > > Thats ok. we have a lot tests where nodes reusable between test's
> > methods.
> > >
> > >
> > > On Wed, Feb 7, 2018 at 1:20 PM, Dmitry Pavlov 
> > > wrote:
> > >
> > > > Hi Igniters,
> > > >
> > > > IMO nodes that survive between tests is not general practice and I'm
> > not
> > > > sure is a best practice. I suggest to mark such tests with some
> method
> > > > overriden from AbstractTest.
> > > >
> > > > About cancel flag, please note stopAllGrids(boolean cancel)
> > cancel=false
> > > > allows to wait of checkpoint ends in case persistence enabled.
> > > >
> > > > I still suggest to avoid stopping any nodes by test, but validate not
> > > > stopped node exist and fail test instead of siltent implicit actions.
> > > >
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > > >
> > > > ср, 7 февр. 2018 г. в 13:04, Andrey Kuznetsov :
> > > >
> > > > > Hi Maxim,
> > > > >
> > > > > Regarding your first question, the use of afterTestsStopped is not
> > > enough
> > > > > to stop all nodes, since each individual test (method) can start
> > custom
> > > > set
> > > > > of notes during its operation, and this very test should stop all
> > those
> > > > > nodes. If you delegate closing to afterTestsStopped this will
> affect
> > > only
> > > > > last test (method). Nodes that survive between tests can affect
> > > > successive
> > > > > tests.
> > > > >
> > > > > 2018-02-07 1:10 GMT+03:00 Maxim Muzafarov :
> > > > >
> > > > > > Hello,
> > > > > >
> > > > > > I've made some research about this problem and i think that in
> > > general
> > > > we
> > > > > > should move stopAllGrids method in GridAbstractTest class to
> > > > > > afterTestsStopped method with some changes. Am I right?
> > > > > >
> > > > > > Also, I have a question about stopAllGrids(boolean cancel) this
> > > > "cancel"
> > > > > > argument. Why in some cases we should interrupt ComputeJob and in
> > > some
> > > > > > cases shouldn't? For example here:
> > > > > > IgniteBaselineAffinityTopologyActivationTest#afterTest
> > > > > > we call method stopAllGrids(false) this way. Why not "true"
> > argument
> > > > > > instead?
> > > > > >
> > > > > >
> > > > > > --
> > > > > Best regards,
> > > > >   Andrey Kuznetsov.
> > > > >
> > > >
> > >
> >
>

Re: Stop nodes after test by default - IGNITE-6842

[Anton Vinogradov]

Maxim,

We discussed with Dima privately, and decided

1) We have to assert that there is no alive nodes at GridAbstractTest's
beforeTestsStarted
2) We have to kill all alive nodes (without force) at GridAbstractTest's
afterTestsStopped
3) In case of any exceptions at #2 we have to see test error
4) We can get rid of all useless stopAllGrids at GridAbstractTest's
subclasses.



On Wed, Feb 7, 2018 at 2:32 PM, Dmitry Pavlov  wrote:

> > Let's just add stopAllGrids(flase) to GridAbstractTest 's
> afterTestsStopped
> method body.
>
> Can't agree with it becase implicit silent shutdown of nodes from test
> framework may cause a lot of bugs introduced to Ignite.
>
> I think stop+test fail should occur.
> In that case author of incorrect test or not consistent Ignite  shutdown
> will see problem.
>
> 'Fail fast' if always better than hidding real problem under automatic
> framework feature.
>
> ср, 7 февр. 2018 г. в 14:05, Anton Vinogradov :
>
> > Hi all,
> >
> > > I've made some research about this problem and i think that in general
> we
> > > should move stopAllGrids method in GridAbstractTest class to
> > > afterTestsStopped method with some changes. Am I right?
> > Let's just add stopAllGrids(flase) to GridAbstractTest 's
> > afterTestsStopped method
> > body.
> >
> > > I have a question about stopAllGrids(boolean cancel) this "cancel"
> > That's just a flag means "do not wait for any operations finish"
> > See no reason to set it true in that case.
> >
> > > If you delegate closing to afterTestsStopped this will affect only
> > > last test (method).
> > The idea is to stop all nodes at last test's method finish.
> >
> > >  Nodes that survive between tests can affect successive
> > tests.
> > Thats ok. we have a lot tests where nodes reusable between test's
> methods.
> >
> >
> > On Wed, Feb 7, 2018 at 1:20 PM, Dmitry Pavlov 
> > wrote:
> >
> > > Hi Igniters,
> > >
> > > IMO nodes that survive between tests is not general practice and I'm
> not
> > > sure is a best practice. I suggest to mark such tests with some method
> > > overriden from AbstractTest.
> > >
> > > About cancel flag, please note stopAllGrids(boolean cancel)
> cancel=false
> > > allows to wait of checkpoint ends in case persistence enabled.
> > >
> > > I still suggest to avoid stopping any nodes by test, but validate not
> > > stopped node exist and fail test instead of siltent implicit actions.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > ср, 7 февр. 2018 г. в 13:04, Andrey Kuznetsov :
> > >
> > > > Hi Maxim,
> > > >
> > > > Regarding your first question, the use of afterTestsStopped is not
> > enough
> > > > to stop all nodes, since each individual test (method) can start
> custom
> > > set
> > > > of notes during its operation, and this very test should stop all
> those
> > > > nodes. If you delegate closing to afterTestsStopped this will affect
> > only
> > > > last test (method). Nodes that survive between tests can affect
> > > successive
> > > > tests.
> > > >
> > > > 2018-02-07 1:10 GMT+03:00 Maxim Muzafarov :
> > > >
> > > > > Hello,
> > > > >
> > > > > I've made some research about this problem and i think that in
> > general
> > > we
> > > > > should move stopAllGrids method in GridAbstractTest class to
> > > > > afterTestsStopped method with some changes. Am I right?
> > > > >
> > > > > Also, I have a question about stopAllGrids(boolean cancel) this
> > > "cancel"
> > > > > argument. Why in some cases we should interrupt ComputeJob and in
> > some
> > > > > cases shouldn't? For example here:
> > > > > IgniteBaselineAffinityTopologyActivationTest#afterTest
> > > > > we call method stopAllGrids(false) this way. Why not "true"
> argument
> > > > > instead?
> > > > >
> > > > >
> > > > > --
> > > > Best regards,
> > > >   Andrey Kuznetsov.
> > > >
> > >
> >
>

[GitHub] ignite pull request #3484: IGNITE-7451: Make Linear SVM for multi-classifica...

[zaleslaw]

GitHub user zaleslaw opened a pull request:

https://github.com/apache/ignite/pull/3484

IGNITE-7451: Make Linear SVM for multi-classification

Added model, trainer and tests for them

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7451

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3484.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3484


commit bb2450fe2a69233bbff557710ca87ed09a5d95bd
Author: Zinoviev Alexey 
Date:   2018-02-02T14:22:06Z

First version of model and trainer

commit a2cc1938d2bc8c78341f67d02a01099e0f6a8293
Author: zaleslaw 
Date:   2018-02-06T14:23:57Z

Merge branch 'master' into ignite-7451

commit 83d7488766db3afe8edd8f969d98c02131a79c24
Author: zaleslaw 
Date:   2018-02-06T15:53:17Z

Added tests




---

[GitHub] ignite pull request #3483: IGNITE-7436 Username/password authentication for ...

[tledkov-gridgain]

GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/3483

IGNITE-7436  Username/password authentication for thin clients



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7436

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3483.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3483


commit 48d383be2c86f632f4394f21681e6e20233c67fb
Author: tledkov-gridgain 
Date:   2018-01-16T15:46:18Z

IGNITE-7439: save the progress

commit a6394d0dbe3be38e65025e3de1f5dbe0b0f0b3f9
Author: tledkov-gridgain 
Date:   2018-01-17T09:02:24Z

IGNITE-7439: save the progress

commit 5f816e0eeec35e3b714887c4342a4d6159e8ae30
Author: tledkov-gridgain 
Date:   2018-01-17T13:16:45Z

Merge branch '_master' into ignite-7439

commit 2c78e7d1a9be9ad858c6369f5033a9236781c99f
Author: tledkov-gridgain 
Date:   2018-01-18T13:21:13Z

IGNITE-7439: save the progress

commit b484b50865acf1b996e3d85713c2c7e6d246bbb5
Author: tledkov-gridgain 
Date:   2018-01-18T13:45:19Z

Merge branch '_master' into ignite-7439

commit 4bcb6e4ea4ccba9eea2a06111302015ecaab7846
Author: tledkov-gridgain 
Date:   2018-01-19T12:08:29Z

IGNITE-7439: save the progress

commit 7957e19a95d8a3ab6046d38bd117f65834009985
Author: tledkov-gridgain 
Date:   2018-01-19T12:16:12Z

Merge branch 'master' into ignite-7439

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/configuration/ClientConnectorConfiguration.java

commit 1006b666bc5e570a20afd54d6ecf455183ce1dbb
Author: tledkov-gridgain 
Date:   2018-01-19T16:52:22Z

IGNITE-7439: save the progress

commit 9e12b872177da6f9ea1d21b4464b22ae822b9c9a
Author: tledkov-gridgain 
Date:   2018-01-23T10:25:32Z

IGNITE-7439: save the progress

commit 7edc494c9899266143496e57ab9a4cdee6ef804f
Author: tledkov-gridgain 
Date:   2018-01-23T10:57:03Z

Merge branch 'master' into ignite-7439

# Conflicts:
#   modules/core/src/main/java/org/apache/ignite/internal/GridTopic.java
#   
modules/core/src/main/java/org/apache/ignite/internal/managers/communication/GridIoMessageFactory.java

commit bba2f71aa5c916c65bdb9ff315afeb3cfe0b8e57
Author: tledkov-gridgain 
Date:   2018-01-24T10:45:25Z

IGNITE-7439: save the progress

commit af83ecd0a2c44ac5d9d394b7b099d46eefd9b813
Author: tledkov-gridgain 
Date:   2018-01-25T14:21:27Z

IGNITE-7439: save the progress

commit 83809e9473f832fdc472bdceab8233a9824fbaac
Author: tledkov-gridgain 
Date:   2018-01-25T14:21:36Z

Merge branch '_master' into ignite-7439

commit 27f428b7a2d72d298745c828815fee3c1882968d
Author: tledkov-gridgain 
Date:   2018-01-26T10:49:53Z

Merge branch '_master' into ignite-7439

commit f15b405fc3bf9c62999661c627bd690519c78394
Author: tledkov-gridgain 
Date:   2018-01-29T09:09:27Z

IGNITE-7439: save the progress

commit 0087b2ce8c5887a301000bd1558bfdd7493930ff
Author: tledkov-gridgain 
Date:   2018-01-29T15:24:20Z

IGNITE-7439: save the progress

commit c9a7ac9a23ab32b40ed380920304cbce9390e84d
Author: tledkov-gridgain 
Date:   2018-01-30T09:07:33Z

IGNITE-7557: save the progress

commit d2e2bf96f50a75ee43d7c02a70f9363a95e4373b
Author: tledkov-gridgain 
Date:   2018-01-30T09:42:55Z

Merge branch '_master' into ignite-7439

commit ea897bc5756bcada73be703a538ce09901782230
Author: tledkov-gridgain 
Date:   2018-02-01T15:06:06Z

IGNITE-7439: save the progress

commit 664eb5aeb5525072b353ffc0dc2427ec517747a7
Author: tledkov-gridgain 
Date:   2018-02-01T15:08:53Z

IGNITE-7439: save the progress

commit 41c6cfef7227aa1b4092efc23bf888c3016638d2
Author: tledkov-gridgain 
Date:   2018-02-01T15:13:19Z

Merge branch 'master' into ignite-7439

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/configuration/ClientConnectorConfiguration.java

commit a52e46cca6a8d506206a76377f43f649737ea73d
Author: tledkov-gridgain 
Date:   2018-02-01T15:17:43Z

Merge branch 'ignite-7439' into ignite-7557

commit 24e0f3cda82d1386d3bdcecc72338a365f862102
Author: tledkov-gridgain 
Date:   2018-02-02T09:29:29Z

IGNITE-7557: save the progress

commit 0a233d05b8859674a17f4398e57e412fbb303aeb
Author: tledkov-gridgain 
Date:   2018-02-02T11:52:25Z

IGNITE-7557: save the progress

commit 2e60575e431920eb5c75efb0cd305f9cc4307405
Author: tledkov-gridgain 
Date:   2018-02-02T11:53:45Z

Merge branch '_master' into ignite-7439

commit 

[GitHub] ignite pull request #3482: IGNITE-7538 Introduce new test project for Ignite...

[vveider]

GitHub user vveider opened a pull request:

https://github.com/apache/ignite/pull/3482

IGNITE-7538 Introduce new test project for Ignite 2.0+ with Java 8 / Java 9 
compatibility



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7538

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3482.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3482


commit f3b4e8bf9e65350d6c35b787e173f21d08794d79
Author: Ivanov Petr 
Date:   2018-02-06T08:22:23Z

IGNITE-7538 Introduce new test project for Ignite 2.0+ with Java 8 / Java 9 
compatibility
 * updated versions of flatten-maven-plugin and apache-rat-plugin for RAT 
licenses check task

commit d72815bf86bbaf4bef53e8bba40d3beaf18611bb
Author: Ivanov Petr 
Date:   2018-02-07T11:44:20Z

IGNITE-7538 Introduce new test project for Ignite 2.0+ with Java 8 / Java 9 
compatibility
 * update maven-javadoc-plugin version




---

[GitHub] ignite pull request #3481: IGNITE-7638: Test created to reproduce FullPageId...

[dspavlov]

GitHub user dspavlov opened a pull request:

https://github.com/apache/ignite/pull/3481

IGNITE-7638: Test created to reproduce FullPageIdTable



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7638

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3481.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3481


commit 90befa9d3bc89833eeed42a8814e8a7f51a11448
Author: dpavlov 
Date:   2018-01-25T18:11:07Z

IGNITE-7533: Throttle writing threads according fsync progress and 
checkpoint write speed

commit 74411e7aea9744df7f7656006807aa0403ae921f
Author: dpavlov 
Date:   2018-01-29T15:51:23Z

IGNITE-7533: Throttle writing threads according fsync progress and 
checkpoint write speed

commit 4be02ec3444596ee0bc95bd55ece9ba741e729a1
Author: dpavlov 
Date:   2018-01-29T15:55:48Z

Merge branch 'master' into ignite-7533

commit f5d383ddef1b2a66470ec110f781a98aa78c5d03
Author: dpavlov 
Date:   2018-01-29T17:08:35Z

IGNITE-7533: Throttle writing threads according fsync progress and 
checkpoint write speed

commit 7c0afa374907202e45d6dcbfae88af1c3a27687f
Author: dpavlov 
Date:   2018-01-30T14:04:52Z

IGNITE-7533: Option to enable old implementation of throttling

commit 9f9c1e7955d894bbfd8a8572362d2c13177a60c6
Author: dpavlov 
Date:   2018-01-30T14:13:42Z

IGNITE-7380: Flaky test reproduction

commit 8d8aecd55a4d94c091e970ccf3bbbd72272cf325
Author: dpavlov 
Date:   2018-01-30T14:20:50Z

IGNITE-7380: Flaky test reproduction

commit cf9d42ba77133c4c6e37b1d8a7ac6d54054d1bc1
Author: dpavlov 
Date:   2018-01-30T14:40:26Z

IGNITE-7533: Preserve order of writing in fsync

commit b37f27275446a8cccafbd231c35e3605d9fd7089
Author: dpavlov 
Date:   2018-01-30T14:43:10Z

IGNITE-7380: Increase of timeout of checkpoint

commit b05ef5dae3d4e5deef6482844989077aba6f1bf2
Author: dpavlov 
Date:   2018-01-30T16:23:51Z

IGNITE-7533: Too much pages written case, no throttling in case too long 
wait. Added more delay in case low space left

commit 62685bcb363add269930af99e59b74d396870b55
Author: dpavlov 
Date:   2018-01-30T16:37:35Z

IGNITE-7380: Flaky test reproduction

commit c7ba24580199a238506ea00176aaf7ae229aa135
Author: dpavlov 
Date:   2018-01-30T17:57:15Z

IGNITE-7533: Sandbox test with progress gaps detection was added.

commit d32654d902ca2fe0ec4e3fa5327afe84f949cdaf
Author: dpavlov 
Date:   2018-01-31T15:52:12Z

IGNITE-7175: fix compatible with speed based throttling

commit 018ed3c0de21cbe1ddd9f7558a417369740bd2cc
Author: dpavlov 
Date:   2018-01-31T17:28:23Z

IGNITE-7533: recurrent warning of throttling if significant pressure.

commit 687d1ffd3d7e6ccc9a0c609ffda6762e7707d788
Author: dpavlov 
Date:   2018-01-31T17:38:43Z

IGNITE-7533: recurrent warning of throttling if significant pressure: cp 
pages added

commit 94dac70231c52a915717ca444e7aba8e4b816003
Author: dpavlov 
Date:   2018-01-31T18:15:03Z

IGNITE-7533: Test suite added

commit 25f4774c1990cb956d2d01eb1c261da762a8798e
Author: dpavlov 
Date:   2018-01-31T18:15:46Z

IGNITE-7533: Test suite added

commit c0a078d45caf08e5f4d9b28f901176155b26c8a4
Author: dpavlov 
Date:   2018-02-01T11:59:21Z

Merge branch 'master' into ignite-7533

commit eab2b06c776e6e4a5f7ee6d1b8a9aa7596831660
Author: dpavlov 
Date:   2018-02-01T12:27:31Z

IGNITE-7533: Message updated to be more clear

commit 45845535cf42b2d86b7f2ff967713baf3c4c430b
Author: dpavlov 
Date:   2018-02-01T17:34:33Z

IGNITE-7533: data streamer test

commit 8c78c888e565f90d60ed2cd7ba426128b22a9c8a
Author: dpavlov 
Date:   2018-02-06T17:52:07Z

IGNITE-7638: Problem demonstrated with FullPageIdTable

commit aff846cd60fc8e3f4d901ed8e865709bcc81518b
Author: dpavlov 
Date:   2018-02-07T10:27:19Z

Merge branch 'master' into ignite-7638

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheDatabaseSharedManager.java
#   
modules/core/src/test/java/org/apache/ignite/testsuites/IgnitePdsUnitTestSuite.java




---

[GitHub] ignite pull request #3419: IGNITE-6917: SQL: implement COPY command for effi...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3419


---

Re: Stop nodes after test by default - IGNITE-6842

[Dmitry Pavlov]

> Let's just add stopAllGrids(flase) to GridAbstractTest 's afterTestsStopped
method body.

Can't agree with it becase implicit silent shutdown of nodes from test
framework may cause a lot of bugs introduced to Ignite.

I think stop+test fail should occur.
In that case author of incorrect test or not consistent Ignite  shutdown
will see problem.

'Fail fast' if always better than hidding real problem under automatic
framework feature.

ср, 7 февр. 2018 г. в 14:05, Anton Vinogradov :

> Hi all,
>
> > I've made some research about this problem and i think that in general we
> > should move stopAllGrids method in GridAbstractTest class to
> > afterTestsStopped method with some changes. Am I right?
> Let's just add stopAllGrids(flase) to GridAbstractTest 's
> afterTestsStopped method
> body.
>
> > I have a question about stopAllGrids(boolean cancel) this "cancel"
> That's just a flag means "do not wait for any operations finish"
> See no reason to set it true in that case.
>
> > If you delegate closing to afterTestsStopped this will affect only
> > last test (method).
> The idea is to stop all nodes at last test's method finish.
>
> >  Nodes that survive between tests can affect successive
> tests.
> Thats ok. we have a lot tests where nodes reusable between test's methods.
>
>
> On Wed, Feb 7, 2018 at 1:20 PM, Dmitry Pavlov 
> wrote:
>
> > Hi Igniters,
> >
> > IMO nodes that survive between tests is not general practice and I'm not
> > sure is a best practice. I suggest to mark such tests with some method
> > overriden from AbstractTest.
> >
> > About cancel flag, please note stopAllGrids(boolean cancel)  cancel=false
> > allows to wait of checkpoint ends in case persistence enabled.
> >
> > I still suggest to avoid stopping any nodes by test, but validate not
> > stopped node exist and fail test instead of siltent implicit actions.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > ср, 7 февр. 2018 г. в 13:04, Andrey Kuznetsov :
> >
> > > Hi Maxim,
> > >
> > > Regarding your first question, the use of afterTestsStopped is not
> enough
> > > to stop all nodes, since each individual test (method) can start custom
> > set
> > > of notes during its operation, and this very test should stop all those
> > > nodes. If you delegate closing to afterTestsStopped this will affect
> only
> > > last test (method). Nodes that survive between tests can affect
> > successive
> > > tests.
> > >
> > > 2018-02-07 1:10 GMT+03:00 Maxim Muzafarov :
> > >
> > > > Hello,
> > > >
> > > > I've made some research about this problem and i think that in
> general
> > we
> > > > should move stopAllGrids method in GridAbstractTest class to
> > > > afterTestsStopped method with some changes. Am I right?
> > > >
> > > > Also, I have a question about stopAllGrids(boolean cancel) this
> > "cancel"
> > > > argument. Why in some cases we should interrupt ComputeJob and in
> some
> > > > cases shouldn't? For example here:
> > > > IgniteBaselineAffinityTopologyActivationTest#afterTest
> > > > we call method stopAllGrids(false) this way. Why not "true" argument
> > > > instead?
> > > >
> > > >
> > > > --
> > > Best regards,
> > >   Andrey Kuznetsov.
> > >
> >
>

Re: Stop nodes after test by default - IGNITE-6842

[Anton Vinogradov]

Hi all,

> I've made some research about this problem and i think that in general we
> should move stopAllGrids method in GridAbstractTest class to
> afterTestsStopped method with some changes. Am I right?
Let's just add stopAllGrids(flase) to GridAbstractTest 's
afterTestsStopped method
body.

> I have a question about stopAllGrids(boolean cancel) this "cancel"
That's just a flag means "do not wait for any operations finish"
See no reason to set it true in that case.

> If you delegate closing to afterTestsStopped this will affect only
> last test (method).
The idea is to stop all nodes at last test's method finish.

>  Nodes that survive between tests can affect successive
tests.
Thats ok. we have a lot tests where nodes reusable between test's methods.


On Wed, Feb 7, 2018 at 1:20 PM, Dmitry Pavlov  wrote:

> Hi Igniters,
>
> IMO nodes that survive between tests is not general practice and I'm not
> sure is a best practice. I suggest to mark such tests with some method
> overriden from AbstractTest.
>
> About cancel flag, please note stopAllGrids(boolean cancel)  cancel=false
> allows to wait of checkpoint ends in case persistence enabled.
>
> I still suggest to avoid stopping any nodes by test, but validate not
> stopped node exist and fail test instead of siltent implicit actions.
>
> Sincerely,
> Dmitriy Pavlov
>
> ср, 7 февр. 2018 г. в 13:04, Andrey Kuznetsov :
>
> > Hi Maxim,
> >
> > Regarding your first question, the use of afterTestsStopped is not enough
> > to stop all nodes, since each individual test (method) can start custom
> set
> > of notes during its operation, and this very test should stop all those
> > nodes. If you delegate closing to afterTestsStopped this will affect only
> > last test (method). Nodes that survive between tests can affect
> successive
> > tests.
> >
> > 2018-02-07 1:10 GMT+03:00 Maxim Muzafarov :
> >
> > > Hello,
> > >
> > > I've made some research about this problem and i think that in general
> we
> > > should move stopAllGrids method in GridAbstractTest class to
> > > afterTestsStopped method with some changes. Am I right?
> > >
> > > Also, I have a question about stopAllGrids(boolean cancel) this
> "cancel"
> > > argument. Why in some cases we should interrupt ComputeJob and in some
> > > cases shouldn't? For example here:
> > > IgniteBaselineAffinityTopologyActivationTest#afterTest
> > > we call method stopAllGrids(false) this way. Why not "true" argument
> > > instead?
> > >
> > >
> > > --
> > Best regards,
> >   Andrey Kuznetsov.
> >
>

Re: Apache Ignite 2.4 release

[Raymond Wilson]

Thanks for the clarification Dmitry. 

Sent from my iPhone

> On 7/02/2018, at 11:13 PM, Dmitry Pavlov  wrote:
> 
> Hi Raymond,
> 
> Minor releases were discussed already at dev list. Community does not
> support minor releases. In case of emergency fix is required, full release
> process has to be followed for next release, e.g. for 2.5. Exactly that
> type of release was 2.2, it was emergency release for fix persistence
> usability issues.
> 
> Early access build may being build using DEVNOTES.txt instruction from
> Ignite source code.
> 
> IMO this fix is mandatory because problem occurs every time when Durable
> Memory pages replacement is started (rotation of pages with RAM and disk).
> It is general case for most of databases, when there is plenty of HDD space
> but not so much RAM.
> 
> Full fix with hashing algoritm change will require around a week, but in
> the same time I will try to find out workaround to provide fast fix for 2.4.
> 
> Sincerely,
> Dmitriy Pavlov
> 
> ср, 7 февр. 2018 г. в 12:07, Raymond Wilson :
> 
>> I guess this depends on how long it will take to implement.
>> 
>> Is it acceptable to release 2.4 without this fix (which allows integrators
>> earlier access to elements such as thin client protocol etc), but follow it
>> up with a 2.4.1 release containing the performance fix?
>> 
>> What is the community policy for minor point releases? Does the
>> build/release process allow for a more agile release cadence?
>> 
>> Thanks,
>> Raymond.
>> 
>> Sent from my iPhone
>> 
>>> On 7/02/2018, at 8:29 PM, Alexey Goncharuk 
>> wrote:
>>> 
>>> Guys,
>>> 
>>> Thanks to Dmitriy Pavlov we found the ticket [1] which causes a major
>>> slowdown when page replacement starts. Even though it's not a
>> regression, I
>>> suggest we consider it a blocker for 2.4 because this is a huge
>> performance
>>> issue which can make it virtually impossible to use native persistence
>> when
>>> data size is significantly larger than memory size.
>>> 
>>> Any objections?
>>> 
>>> [1] https://issues.apache.org/jira/browse/IGNITE-7638
>>> 
>>> 2018-01-30 17:10 GMT+03:00 Pavel Tupitsyn :
>>> 
 Igniters, I will handle 2.4 release if there are no objections.
 Let's take a bit more time for testing and start vote by the end of this
 week.
 
 Pavel
 
> On Sat, Jan 27, 2018 at 3:32 AM, Denis Magda 
>> wrote:
> 
> Hi Vyacheslav,
> 
> According to the previous review notes the impact of the changes might
>> be
> significant, thus, I would recommend us to move the changes to the next
> release.
> 
> BTW, don’t hesitate to ping reviewers more frequently if there is a
> pending/abandon review. We are all the people who are tend to forget or
> miss notifications ;)
> 
>> On Jan 26, 2018, at 2:04 AM, Vyacheslav Daradur 
> wrote:
>> 
>> Hi, Vladimir, it's good news. I'm looking forward to new Ignite
 release!
>> 
>> Could you please share a release schedule for 'varint' optimizations?
>> 
>> The task [1] is waiting for review for 5 months.
>> 
>> [1] https://issues.apache.org/jira/browse/IGNITE-5097
>> 
>> 
>> On Fri, Jan 26, 2018 at 12:51 PM, Vladimir Ozerov <
 voze...@gridgain.com>
> wrote:
>>> Hi Igniters,
>>> 
>>> As far as I can see all required tasks and fixes were merged. I
 propose
> to
>>> take several days of silence to test what we've done and start vote
>> at
> the
>>> beginning of the next week.
>>> 
>>> Makes sense?
>>> 
>>> On Mon, Jan 22, 2018 at 8:39 PM, Denis Magda 
 wrote:
>>> 
 Ok, let’s target Wednesday as a code freeze date.
 
 Community members who are involved in 2.4 release please merge you
> fixes
 and optimizations by that time.
 
 —
 Denis
 
> On Jan 22, 2018, at 8:24 AM, Anton Vinogradov 
 wrote:
> 
> Issues
> https://issues.apache.org/jira/browse/IGNITE-6711
> https://issues.apache.org/jira/browse/IGNITE-6902
> 
> are in master and ignite-2.4
> 
> On Mon, Jan 22, 2018 at 6:43 PM, Denis Magda 
> wrote:
> 
>> Igniters,
>> 
>> What’s left and what prevents us from passing 2.4 through the QA
> phase?
>> 
>> Petr, Anton, Vladimir, Alex G., others please chime in.
>> 
>> —
>> Denis
>> 
>>> On Jan 17, 2018, at 7:05 PM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
>> wrote:
>>> 
>>> Great news, indeed! Looking forward to 2.4 release.
>>> 
>>> On Wed, Jan 17, 2018 at 2:55 AM, Vladimir Ozerov <
> voze...@gridgain.com
> 
>>> 

Re: Stop nodes after test by default - IGNITE-6842

[Dmitry Pavlov]

Hi Igniters,

IMO nodes that survive between tests is not general practice and I'm not
sure is a best practice. I suggest to mark such tests with some method
overriden from AbstractTest.

About cancel flag, please note stopAllGrids(boolean cancel)  cancel=false
allows to wait of checkpoint ends in case persistence enabled.

I still suggest to avoid stopping any nodes by test, but validate not
stopped node exist and fail test instead of siltent implicit actions.

Sincerely,
Dmitriy Pavlov

ср, 7 февр. 2018 г. в 13:04, Andrey Kuznetsov :

> Hi Maxim,
>
> Regarding your first question, the use of afterTestsStopped is not enough
> to stop all nodes, since each individual test (method) can start custom set
> of notes during its operation, and this very test should stop all those
> nodes. If you delegate closing to afterTestsStopped this will affect only
> last test (method). Nodes that survive between tests can affect successive
> tests.
>
> 2018-02-07 1:10 GMT+03:00 Maxim Muzafarov :
>
> > Hello,
> >
> > I've made some research about this problem and i think that in general we
> > should move stopAllGrids method in GridAbstractTest class to
> > afterTestsStopped method with some changes. Am I right?
> >
> > Also, I have a question about stopAllGrids(boolean cancel) this "cancel"
> > argument. Why in some cases we should interrupt ComputeJob and in some
> > cases shouldn't? For example here:
> > IgniteBaselineAffinityTopologyActivationTest#afterTest
> > we call method stopAllGrids(false) this way. Why not "true" argument
> > instead?
> >
> >
> > --
> Best regards,
>   Andrey Kuznetsov.
>

Re: Apache Ignite 2.4 release

[Dmitry Pavlov]

Hi Raymond,

Minor releases were discussed already at dev list. Community does not
support minor releases. In case of emergency fix is required, full release
process has to be followed for next release, e.g. for 2.5. Exactly that
type of release was 2.2, it was emergency release for fix persistence
usability issues.

Early access build may being build using DEVNOTES.txt instruction from
Ignite source code.

IMO this fix is mandatory because problem occurs every time when Durable
Memory pages replacement is started (rotation of pages with RAM and disk).
It is general case for most of databases, when there is plenty of HDD space
but not so much RAM.

Full fix with hashing algoritm change will require around a week, but in
the same time I will try to find out workaround to provide fast fix for 2.4.

Sincerely,
Dmitriy Pavlov

ср, 7 февр. 2018 г. в 12:07, Raymond Wilson :

> I guess this depends on how long it will take to implement.
>
> Is it acceptable to release 2.4 without this fix (which allows integrators
> earlier access to elements such as thin client protocol etc), but follow it
> up with a 2.4.1 release containing the performance fix?
>
> What is the community policy for minor point releases? Does the
> build/release process allow for a more agile release cadence?
>
> Thanks,
> Raymond.
>
> Sent from my iPhone
>
> > On 7/02/2018, at 8:29 PM, Alexey Goncharuk 
> wrote:
> >
> > Guys,
> >
> > Thanks to Dmitriy Pavlov we found the ticket [1] which causes a major
> > slowdown when page replacement starts. Even though it's not a
> regression, I
> > suggest we consider it a blocker for 2.4 because this is a huge
> performance
> > issue which can make it virtually impossible to use native persistence
> when
> > data size is significantly larger than memory size.
> >
> > Any objections?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-7638
> >
> > 2018-01-30 17:10 GMT+03:00 Pavel Tupitsyn :
> >
> >> Igniters, I will handle 2.4 release if there are no objections.
> >> Let's take a bit more time for testing and start vote by the end of this
> >> week.
> >>
> >> Pavel
> >>
> >>> On Sat, Jan 27, 2018 at 3:32 AM, Denis Magda 
> wrote:
> >>>
> >>> Hi Vyacheslav,
> >>>
> >>> According to the previous review notes the impact of the changes might
> be
> >>> significant, thus, I would recommend us to move the changes to the next
> >>> release.
> >>>
> >>> BTW, don’t hesitate to ping reviewers more frequently if there is a
> >>> pending/abandon review. We are all the people who are tend to forget or
> >>> miss notifications ;)
> >>>
>  On Jan 26, 2018, at 2:04 AM, Vyacheslav Daradur 
> >>> wrote:
> 
>  Hi, Vladimir, it's good news. I'm looking forward to new Ignite
> >> release!
> 
>  Could you please share a release schedule for 'varint' optimizations?
> 
>  The task [1] is waiting for review for 5 months.
> 
>  [1] https://issues.apache.org/jira/browse/IGNITE-5097
> 
> 
>  On Fri, Jan 26, 2018 at 12:51 PM, Vladimir Ozerov <
> >> voze...@gridgain.com>
> >>> wrote:
> > Hi Igniters,
> >
> > As far as I can see all required tasks and fixes were merged. I
> >> propose
> >>> to
> > take several days of silence to test what we've done and start vote
> at
> >>> the
> > beginning of the next week.
> >
> > Makes sense?
> >
> > On Mon, Jan 22, 2018 at 8:39 PM, Denis Magda 
> >> wrote:
> >
> >> Ok, let’s target Wednesday as a code freeze date.
> >>
> >> Community members who are involved in 2.4 release please merge you
> >>> fixes
> >> and optimizations by that time.
> >>
> >> —
> >> Denis
> >>
> >>> On Jan 22, 2018, at 8:24 AM, Anton Vinogradov 
> >> wrote:
> >>>
> >>> Issues
> >>> https://issues.apache.org/jira/browse/IGNITE-6711
> >>> https://issues.apache.org/jira/browse/IGNITE-6902
> >>>
> >>> are in master and ignite-2.4
> >>>
> >>> On Mon, Jan 22, 2018 at 6:43 PM, Denis Magda 
> >>> wrote:
> >>>
>  Igniters,
> 
>  What’s left and what prevents us from passing 2.4 through the QA
> >>> phase?
> 
>  Petr, Anton, Vladimir, Alex G., others please chime in.
> 
>  —
>  Denis
> 
> > On Jan 17, 2018, at 7:05 PM, Dmitriy Setrakyan <
> >>> dsetrak...@apache.org>
>  wrote:
> >
> > Great news, indeed! Looking forward to 2.4 release.
> >
> > On Wed, Jan 17, 2018 at 2:55 AM, Vladimir Ozerov <
> >>> voze...@gridgain.com
> >>>
> > wrote:
> >
> >> Igniters,
> >>
> >> Great news - two very important tickets - baseline topology and
> >>> Java 8
> >> support - were merged to master. At this point it makes sense to
> >> create
> 

Re: Removing "fabric" from Ignite binary package name

[Anton Vinogradov]

Denis, Petr,

I checked PR and found we have *overcomplicated* logic with "fabric" and
"hadoop" postfixs.

Do we really need to assembly 2 editions?
"Hadoop" edition still valued?

My proposal is to get rid of "hadoop" edition and replace it with
instruction of how to use "fabric" edition instead.
Instruction will be pretty easy -> move "hadoop" folder from "optional" to
root directory :)

In that case we can just remove all postfix logic from maven poms and
simplify release process.

On Thu, Dec 28, 2017 at 9:20 PM, Denis Magda  wrote:

> Petr, thanks for solving it!
>
> Hope that Anton V. or some other build master will double-check the
> changes and merge them.
>
> —
> Denis
>
> > On Dec 28, 2017, at 8:29 AM, Petr Ivanov  wrote:
> >
> > IGNITE-7251 is done, needs review and some additional tests. See PR
> #3315 [1].
> >
> >
> > [1] https://github.com/apache/ignite/pull/3315 <
> https://github.com/apache/ignite/pull/3315>
> >
> >
> >
> >> On 20 Dec 2017, at 23:15, Denis Magda  wrote:
> >>
> >> Petr, thanks, such a swift turnaround!
> >>
> >> Have you found the one who can asses and review the changes?
> >>
> >> Maintainers label might be helpful. Just ping them directly:
> >> https://cwiki.apache.org/confluence/display/IGNITE/How+
> to+Contribute#HowtoContribute-ReviewProcessandMaintainers <
> https://cwiki.apache.org/confluence/display/IGNITE/How+
> to+Contribute#HowtoContribute-ReviewProcessandMaintainers>
> >>
> >>
> >> —
> >> Denis
> >>
> >>> On Dec 20, 2017, at 12:24 AM, Petr Ivanov  wrote:
> >>>
> >>> Assigned myself — done the same work while preparing RPM package.
> >>> But for fixing DEVNOTES.txt waiting for review and merge of
> IGNITE-7107 [1].
> >>>
> >>>
> >>> [1] https://issues.apache.org/jira/browse/IGNITE-7107
> >>>
> >>>
> >>>
>  On 19 Dec 2017, at 22:55, Denis Magda  wrote:
> 
>  All the bids were accepted and the verdict is executed:
>  https://issues.apache.org/jira/browse/IGNITE-7251 <
> https://issues.apache.org/jira/browse/IGNITE-7251>
> 
>  Who is ready to pick this up?
> 
>  —
>  Denis
> 
> > On Dec 19, 2017, at 5:35 AM, Anton Vinogradov <
> avinogra...@gridgain.com> wrote:
> >
> > +1б фо шур
> >
> > On Tue, Dec 19, 2017 at 9:59 AM, Vladimir Ozerov <
> voze...@gridgain.com>
> > wrote:
> >
> >> +1б вуаштшеудн
> >>
> >> On Tue, Dec 19, 2017 at 2:34 AM, Valentin Kulichenko <
> >> valentin.kuliche...@gmail.com> wrote:
> >>
> >>> +1
> >>>
> >>> On Mon, Dec 18, 2017 at 12:24 PM, Dmitriy Setrakyan <
> >> dsetrak...@apache.org
> 
> >>> wrote:
> >>>
>  +1 (completely agree)
> 
>  On Mon, Dec 18, 2017 at 10:21 AM, Denis Magda 
> >> wrote:
> 
> > Igniters,
> >
> > Apache Ignite binary releases still include “fabric” word in
> their
> >>> names:
> > https://ignite.apache.org/download.cgi#binaries <
> > https://ignite.apache.org/download.cgi#binaries>
> >
> > For instance, this is a full name of the previous release -
> > apache-ignite-fabric-2.3.0-bin.
> > It’s a little oversight on our side because the project has not
> been
> > positioned as a fabric for a while.
> >
> > Proposal! Remove “fabric” from the name and have the binary
> releases
>  named
> > as - apache-ignite-{version}-bin.
> >
> > If we’re in consensus then let’s make the change in 2.4.
> >
> > —
> > Denis
> >
> 
> >>>
> >>
> 
> >>>
> >>
> >
>
>

Re: Stop nodes after test by default - IGNITE-6842

[Andrey Kuznetsov]

Hi Maxim,

Regarding your first question, the use of afterTestsStopped is not enough
to stop all nodes, since each individual test (method) can start custom set
of notes during its operation, and this very test should stop all those
nodes. If you delegate closing to afterTestsStopped this will affect only
last test (method). Nodes that survive between tests can affect successive
tests.

2018-02-07 1:10 GMT+03:00 Maxim Muzafarov :

> Hello,
>
> I've made some research about this problem and i think that in general we
> should move stopAllGrids method in GridAbstractTest class to
> afterTestsStopped method with some changes. Am I right?
>
> Also, I have a question about stopAllGrids(boolean cancel) this "cancel"
> argument. Why in some cases we should interrupt ComputeJob and in some
> cases shouldn't? For example here:
> IgniteBaselineAffinityTopologyActivationTest#afterTest
> we call method stopAllGrids(false) this way. Why not "true" argument
> instead?
>
>
> --
Best regards,
  Andrey Kuznetsov.

[GitHub] ignite pull request #3480: IGNITE-7643: broken javadoc.

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3480


---

Re: Abandoned Patch Available JIRA tickets

[Andrey Kuznetsov]

Periodic punches can frustrate committers, and also it's unpleasant for
contributors. Most IP->PA transitions are commented with something like
"John Doe, this awesome feature is ready and needs your review." Maybe it's
better to adopt following rule of thumb? If the change is clean and
straightforward it should be reviewed is a day, otherwise committer puts a
comment in Jira issue about planned review time.

Is this acceptable?

2018-02-07 0:14 GMT+03:00 Denis Magda :

> I guess it’s all about discipline.
>
> Committers need to walk-through a list of the pull-request regularly while
> contributors have to remind of a pending pull-request periodically. So both
> parts have to be proactive.
>
> Another approach is to find a volunteer from the community who will keep
> an eye on the contributions and spread them out among committers.
>
> Not sure I like the latter approach and would rather go for the one when
> both the committers and contributors are proactive and disciplined. But
> guess what, if you want to make the contributors proactive then the
> committers have to be an example.
>
> —
> Denis
>
>
>


-- 
Best regards,
  Andrey Kuznetsov.

Re: Text query with persistence

[Dmitry Karachentsev]

Thanks Alexey, I'll take a look on it!

07.02.2018 11:48, Alexey Goncharuk пишет:

Hi Dmitriy,

We do have a ticket for this [1], however, I am not sure it will be
completed any time soon.

As for the rebuilding indexes at the node startup, we already have an
internal routine for this (see GridQueryIndexing#rebuildIndexesFromHash).
The only thing we need to implement is skipping of regular indexes update
during the full-text index rebuild and maybe provide an API so that a user
can trigger this.

--Alexey

[1] https://issues.apache.org/jira/browse/IGNITE-5371

2018-02-07 11:30 GMT+03:00 Dmitry Karachentsev :


Hi guys,

We have Apache Lucene based full text search that creates in-memory
indexes. It's a nice feature, but it seems absolutely useless with
persistence as indexes are dropped on node restart.

Probably we need a configuration flag and\or API call to rebuild them, or
persist somehow.

Do we have plans/ticket for that? Or maybe some tricky way to recreate
indexes?

Thanks!

-Dmitry

Re: Text query with persistence

[Alexey Goncharuk]

Hi Dmitriy,

We do have a ticket for this [1], however, I am not sure it will be
completed any time soon.

As for the rebuilding indexes at the node startup, we already have an
internal routine for this (see GridQueryIndexing#rebuildIndexesFromHash).
The only thing we need to implement is skipping of regular indexes update
during the full-text index rebuild and maybe provide an API so that a user
can trigger this.

--Alexey

[1] https://issues.apache.org/jira/browse/IGNITE-5371

2018-02-07 11:30 GMT+03:00 Dmitry Karachentsev :

> Hi guys,
>
> We have Apache Lucene based full text search that creates in-memory
> indexes. It's a nice feature, but it seems absolutely useless with
> persistence as indexes are dropped on node restart.
>
> Probably we need a configuration flag and\or API call to rebuild them, or
> persist somehow.
>
> Do we have plans/ticket for that? Or maybe some tricky way to recreate
> indexes?
>
> Thanks!
>
> -Dmitry
>
>

[GitHub] ignite pull request #3480: IGNITE-7643: broken javadoc.

[ybabak]

GitHub user ybabak opened a pull request:

https://github.com/apache/ignite/pull/3480

IGNITE-7643: broken javadoc.

fixed

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7643

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3480.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3480


commit 075c51a98ca2bcdacd81d4055e4ccb23d4f52f76
Author: YuriBabak 
Date:   2018-02-07T08:30:33Z

IGNITE-7643: broken javadoc.

fixed




---

Text query with persistence

[Dmitry Karachentsev]

Hi guys,

We have Apache Lucene based full text search that creates in-memory 
indexes. It's a nice feature, but it seems absolutely useless with 
persistence as indexes are dropped on node restart.


Probably we need a configuration flag and\or API call to rebuild them, 
or persist somehow.


Do we have plans/ticket for that? Or maybe some tricky way to recreate 
indexes?


Thanks!

-Dmitry

[jira] [Created] (IGNITE-7643) Broken javadoc in partitioned dataset

[Yury Babak (JIRA)]

Yury Babak created IGNITE-7643:
--

 Summary: Broken javadoc in partitioned dataset 
 Key: IGNITE-7643
 URL: https://issues.apache.org/jira/browse/IGNITE-7643
 Project: Ignite
  Issue Type: Task
  Components: ml
Affects Versions: 2.5
Reporter: Yury Babak
Assignee: Yury Babak
 Fix For: 2.5


[22:25:12][Step 7/7] [WARNING] Javadoc Warnings
[22:25:12][Step 7/7] [WARNING] 
/data/teamcity/work/bd85361428dcdb1/examples/src/main/java/org/apache/ignite/examples/ml/dataset/AlgorithmSpecificDatasetExample.java:51:
 warning - Tag @link: reference not found: AlgorithmSpecificDataset
[22:25:12][Step 7/7] [WARNING] 
/data/teamcity/work/bd85361428dcdb1/examples/src/main/java/org/apache/ignite/examples/ml/dataset/AlgorithmSpecificDatasetExample.java:51:
 warning - Tag @link: reference not found: AlgorithmSpecificPartitionContext




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)