Re: 2.9.1 release proposal

[Nikolay Izhikov]

Hello, Yaroslav.

Looks like we have fixVersion 2.9.1 in Jira.
Let’s mark all tickets with label «2.9.1-rc1» with fixVersion=2.9.1 and use 
fixVersion instead of label further.

https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%202.9.1

> 2 нояб. 2020 г., в 19:42, Yaroslav Molochkov  
> написал(а):
> 
> Guys,
> 
> should you agree that issues with the 2.9.1-rc tag are good enough
> for a maintenance release, i'd like to give it a go.
> 
> On Thu, Oct 29, 2020 at 5:33 PM Alexey Zinoviev 
> wrote:
> 
>> Let's discuss the possible planning dates for feature freeze for 2.10, for
>> example? Do you have any plans or ideas?
>> 
>> чт, 29 окт. 2020 г. в 17:24, Andrey Gura :
>> 
>>> Hi,
>>> 
>>> I agree with Alexey. We should release 2.9.1 and start preparing for
>>> the 2.10 release.
>>> 
>>> 2.x releases usually take a lot of time. So we can release 2.9.1, and
>>> even 2.9.2 before 2.10.
>>> 
>>> On Thu, Oct 29, 2020 at 12:02 PM Alexey Goncharuk
>>>  wrote:
 
 Hello folks,
 
 I think we should start both 2.9.1 and 2.10. In practice, maintenance
 release contains only critical and usability bugfixes (for example, I
>>> would
 include this ticket [1] to include in 2.9.1 as it prevents users from
>>> using
 tracing) and is released much faster than a minor release.
 
 [1] https://issues.apache.org/jira/browse/IGNITE-13640
 
 вт, 27 окт. 2020 г. в 21:41, Guru Stron :
 
> Hello,
> 
> Agree with Pavel.
> 
> On Tue, 27 Oct 2020 at 19:22, Pavel Tupitsyn 
>>> wrote:
> 
>> Igniters,
>> 
>> I think we should plan 2.10 instead of 2.9.1.
>> ignite-2.9 branch was cut 4 months ago, a bunch of new features are
> waiting
>> to be released.
>> 
>> On Tue, Oct 27, 2020 at 4:23 AM 18624049226 <18624049...@163.com>
>>> wrote:
>> 
>>> Hello,
>>> 
>>> I suggest that the remaining document issue in version 2.9.0 can
>> be
>>> solved in version 2.9.1, and it is not a good practice to
>> postpone
>>> to
>>> version 2.10.
>>> 
>>> 在 2020/10/27 上午2:00, Maxim Muzafarov 写道:
 Hello,
 
 +1
 Should we start the discussion about the release leader and
>>> release
>>> dates?
 
 On Tue, 20 Oct 2020 at 10:12, Anton Vinogradov 
> wrote:
> +1 to start the 2.9.1 release process once as soon as 2.9
>>> released.
> 
> On Mon, Oct 19, 2020 at 8:49 PM Nikolay Izhikov <
> nizhi...@apache.org>
>>> wrote:
> 
>> Hello, Yaroslav.
>> 
>> I support the idea.
>> But, we should carefully work with the release scope.
>> 
>> IGNITE-13477 - fix for a SQL tracing that will be available
>>> only in
>>> 2.10
>> IGNITE-13427 - fix for a new system view that exists in
>> master
> only,
>> we
>> need to include IGNITE-13409
>> 
>> Other tickets should be checked, also.
>> Is this a real fix of a released bug or we fix some issue we
>>> bring
>> with
>> the brand new contribution.
>> 
>> 
>> I propose to include the following tickets, also:
>> 
>> CMD tools improvements:
>> 
>> IGNITE-13488 - Command to print metric value
>> IGNITE-13426 - Command to print system view content
>> IGNITE-13422 - Parameter to explicitly enable experimental
>>> commands
>> 
>> IGNITE-13380 - Output IgniteSystemProperties via ignite.sh
>> 
>> New system views:
>> 
>> IGNITE-13409 Metastorage and DistributedMetastorage viewы.
>> IGNITE-13408 BinaryMetadata view.
>> 
>> 
>>> 19 окт. 2020 г., в 18:20, Yaroslav Molochkov <
> molochko...@gmail.com
>>> 
>> написал(а):
>>> Hello, Igniters!
>>> 
>>> I've compiled a list of tickets that, i think, deserve to be
>> released
>>> in
>> a
>>> minor Ignite release (meaning in 2.9.1) after 2.9. Here they
>>> are:
>>> 
>>> IGNITE-13569
>>> disable archiving + walCompactionEnabled probably broke
>>> reading
> from
>>> wal
>> on
>>> server restart
>>> 
>>> IGNITE-13418
>>> Deadlock on multiple cache delete
>>> 
>>> IGNITE-13563
>>> Deserializing IBinaryObject containing an IBinaryObject
>> field
> fails
>>> 
>>> IGNITE-13575
>>> Invalid blocking section in GridNioWorker and
>>> GridNioClientWorker
>>> leads
>> to
>>> false positive blocking thread detection
>>> 
>>> IGNITE-13458
>>> RebalancingPartitionsTotal metrics
>>> 
>>> IGNITE-13536
>>> .NET: Child processes become zombies when persistence is
>> used
>>> with
>>> direct-io on Linux
>>> 
>>> IGNITE-13500

Can't build new documentation with Jekyll

[Pavel Tupitsyn]

Igniters,

I tried to build the docs today using the instructions from README.adoc [1]

Bare Jekyll:

/usr/lib/ruby/vendor_ruby/rouge.rb:55:in `block in ':
asciidoctor: FAILED:
/home/pavel/w/ignite/docs/_docs/SQL/JDBC/error-codes.adoc: Failed to load
AsciiDoc document - uninitialized constant Rouge::Lexers

Docker:

Downloading listen-3.1.5 revealed dependencies not in the API or the
lockfile
(rb-fsevent (>= 0.9.4, ~> 0.9), ruby_dep (~> 1.2)).
Either installing with `--full-index` or running `bundle update listen`
should
fix the problem.

Any idea what may be wrong?

[1] https://github.com/apache/ignite/blob/master/docs/README.adoc

Re: [DISCUSS] Ignite 3.0 development approach

[Nikita Ivanov]

+1 on having a separate repo. Make the work cleaner and more effective.

--
Nikita Ivanov



On Mon, Nov 2, 2020 at 11:09 PM Nikolay Izhikov  wrote:

> Igniters, should we have a call for this topic?
>
> > 2 нояб. 2020 г., в 18:53, Pavel Tupitsyn 
> написал(а):
> >
> >> not intend to rewrite everything from scratch
> >
> >> Every single test from Ignite 2.x should be moved to Ignite 3
> >> regardless of how we choose to proceed.
> >
> > Alexey, thank you for the explanation, this addresses all of my concerns.
> >
> >
> >
> >
> >
> > On Mon, Nov 2, 2020 at 6:43 PM Andrey Mashenkov <
> andrey.mashen...@gmail.com>
> > wrote:
> >
> >> Hi, Igniters.
> >>
> >> * AFAIU, we need a new repo if we want to apply different restrictions
> to
> >> pull requests,
> >> otherwise I see no difference for myself.
> >> E.g. make static analysis (do we have?), compile, styles, and javadoc
> >> checks mandatory.
> >>
> >> I think that relaxed requirements here will lead to bad product quality.
> >>
> >> * Agree with Pavel, we should 'keep' integrations tests somehow.
> >> During active development tests will be broken most of time, so,
> >> I'd port them e.g. suite-by-suite once we will have a stable and
> featured
> >> environment to run them and of course make test's code clear and avoid
> >> bad/non-relevant ones.
> >>
> >> * I like bottom-up approach.
> >> With it we could make a better framework. I mean clear component
> lifecycle,
> >> component wiring mechanics, general methods to approach core components
> >> such as exchange/communication
> >> to avoid code mess like we have in ExchangeFuture with all these custom
> >> callbacks for each component, interfaces like
> >> PartitionsExchangeAware, IgniteChangeGlobalStateSupport and
> >> a pack of
> start/stop/onKernalStart/onPluginStart/onActivate/onDisconnected
> >> and so on in various unexpected places.
> >> Hope, we will be able to port most of the good code to the new framework
> >> version.
> >>
> >>
> >>
> >> On Mon, Nov 2, 2020 at 6:18 PM Alexey Goncharuk <
> >> alexey.goncha...@gmail.com>
> >> wrote:
> >>
> >>> Nikolay, Pavel,
> >>>
> >>> Thanks for the feedback! First of all, I wanted to stress that I do not
> >>> intend to rewrite everything from scratch (I never used this phrase).
> >> There
> >>> are significant parts of code that would be moved with minimal
> >>> modifications. Second, I never said that we will get rid of the old
> tests
> >>> codebase. Every single test from Ignite 2.x should be moved to Ignite 3
> >>> regardless of how we choose to proceed.
> >>>
> >>> My point is that for some parts of the code a clean bottom-up
> >>> implementation will be cheaper in many ways. Let me give you a few
> >> concrete
> >>> examples:
> >>>
> >>>   - I think no one can object that we need a cleanly separated
> >> persistence
> >>>   layer for Ignite. There is a very raw draft IEP for this already. On
> >> the
> >>>   other hand, I think we also can agree that we need a split-brain
> >>> resistant
> >>>   replication protocol for caches. There is also an IEP for this.
> >> Neither
> >>> of
> >>>   the changes is a good fit for 2.x because they are likely to
> introduce
> >>>   breaking changes in the persistence layer, configuration and
> behavior.
> >>>   Additionally, these components are now tightly coupled, so there is
> no
> >>> way
> >>>   these two changes can be implemented in parallel and then merged
> >>> together
> >>>   easily. So what we will end up with is having to implement these
> >> changes
> >>>   sequentially, fixing all existing tests twice, and essentially
> >> throwing
> >>>   away half of the work done because the other part of the change is
> >>>   re-implemented
> >>>   - Similar example goes with getting rid of IgniteInternalFuture and
> >>>   replacing it with CompletableFuture, and any other change that
> touches
> >>> the
> >>>   asynchronous part of the code.
> >>>
> >>> Third, I do not see how this choice affects the UX of Ignite. The end
> >> user
> >>> experience must be fixed in the IEP regardless of the development
> process
> >>> and the fact that we have gaps in this area in Ignite 2.x just confirms
> >>> that.
> >>>
> >>> Pavel, agree that a repo/branch is a technicality, I guess if
> >> reformulate,
> >>> my point is that we might agree to have a single development master
> >> branch
> >>> with 'disassembled' end-to-end functionality for some period of time to
> >>> speed up development, and re-assemble the core features after having
> >>> submodules tested independently.
> >>>
> >>> Nikolay,
>  We have many features that have to evolve.
>  Snapshots, rebalance, tooling, tracing, zookeeper support, etc.
> >>> This is not very specific. In the end, resources are limited and we
> will
> >>> not be able to drive both tracks simultaneously, especially after a
> >> couple
> >>> of features having been implemented for Ignite 3.0. If there are indeed
> >>> some major changes that we want to do in Ignite 2.x instead of 

Re: [DISCUSS] Ignite 3.0 development approach

[Nikolay Izhikov]

Igniters, should we have a call for this topic?

> 2 нояб. 2020 г., в 18:53, Pavel Tupitsyn  написал(а):
> 
>> not intend to rewrite everything from scratch
> 
>> Every single test from Ignite 2.x should be moved to Ignite 3
>> regardless of how we choose to proceed.
> 
> Alexey, thank you for the explanation, this addresses all of my concerns.
> 
> 
> 
> 
> 
> On Mon, Nov 2, 2020 at 6:43 PM Andrey Mashenkov 
> wrote:
> 
>> Hi, Igniters.
>> 
>> * AFAIU, we need a new repo if we want to apply different restrictions to
>> pull requests,
>> otherwise I see no difference for myself.
>> E.g. make static analysis (do we have?), compile, styles, and javadoc
>> checks mandatory.
>> 
>> I think that relaxed requirements here will lead to bad product quality.
>> 
>> * Agree with Pavel, we should 'keep' integrations tests somehow.
>> During active development tests will be broken most of time, so,
>> I'd port them e.g. suite-by-suite once we will have a stable and featured
>> environment to run them and of course make test's code clear and avoid
>> bad/non-relevant ones.
>> 
>> * I like bottom-up approach.
>> With it we could make a better framework. I mean clear component lifecycle,
>> component wiring mechanics, general methods to approach core components
>> such as exchange/communication
>> to avoid code mess like we have in ExchangeFuture with all these custom
>> callbacks for each component, interfaces like
>> PartitionsExchangeAware, IgniteChangeGlobalStateSupport and
>> a pack of start/stop/onKernalStart/onPluginStart/onActivate/onDisconnected
>> and so on in various unexpected places.
>> Hope, we will be able to port most of the good code to the new framework
>> version.
>> 
>> 
>> 
>> On Mon, Nov 2, 2020 at 6:18 PM Alexey Goncharuk <
>> alexey.goncha...@gmail.com>
>> wrote:
>> 
>>> Nikolay, Pavel,
>>> 
>>> Thanks for the feedback! First of all, I wanted to stress that I do not
>>> intend to rewrite everything from scratch (I never used this phrase).
>> There
>>> are significant parts of code that would be moved with minimal
>>> modifications. Second, I never said that we will get rid of the old tests
>>> codebase. Every single test from Ignite 2.x should be moved to Ignite 3
>>> regardless of how we choose to proceed.
>>> 
>>> My point is that for some parts of the code a clean bottom-up
>>> implementation will be cheaper in many ways. Let me give you a few
>> concrete
>>> examples:
>>> 
>>>   - I think no one can object that we need a cleanly separated
>> persistence
>>>   layer for Ignite. There is a very raw draft IEP for this already. On
>> the
>>>   other hand, I think we also can agree that we need a split-brain
>>> resistant
>>>   replication protocol for caches. There is also an IEP for this.
>> Neither
>>> of
>>>   the changes is a good fit for 2.x because they are likely to introduce
>>>   breaking changes in the persistence layer, configuration and behavior.
>>>   Additionally, these components are now tightly coupled, so there is no
>>> way
>>>   these two changes can be implemented in parallel and then merged
>>> together
>>>   easily. So what we will end up with is having to implement these
>> changes
>>>   sequentially, fixing all existing tests twice, and essentially
>> throwing
>>>   away half of the work done because the other part of the change is
>>>   re-implemented
>>>   - Similar example goes with getting rid of IgniteInternalFuture and
>>>   replacing it with CompletableFuture, and any other change that touches
>>> the
>>>   asynchronous part of the code.
>>> 
>>> Third, I do not see how this choice affects the UX of Ignite. The end
>> user
>>> experience must be fixed in the IEP regardless of the development process
>>> and the fact that we have gaps in this area in Ignite 2.x just confirms
>>> that.
>>> 
>>> Pavel, agree that a repo/branch is a technicality, I guess if
>> reformulate,
>>> my point is that we might agree to have a single development master
>> branch
>>> with 'disassembled' end-to-end functionality for some period of time to
>>> speed up development, and re-assemble the core features after having
>>> submodules tested independently.
>>> 
>>> Nikolay,
 We have many features that have to evolve.
 Snapshots, rebalance, tooling, tracing, zookeeper support, etc.
>>> This is not very specific. In the end, resources are limited and we will
>>> not be able to drive both tracks simultaneously, especially after a
>> couple
>>> of features having been implemented for Ignite 3.0. If there are indeed
>>> some major changes that we want to do in Ignite 2.x instead of putting
>>> effort into 3.0 - let's discuss them. I am just not aware of any, that's
>>> why I am eager to move forward with Ignite 3.0.
>>> 
 We have bugs and issues that can be fixed in 2.x without breaking
>> backward
>>> compatibility.
 We have many users who are happy with the 2.x with all it’s issues.
>>> These changes will be covered by end-to-end tests and migrated to Ignite
>>> 3.0, so I see 

[jira] [Created] (IGNITE-13656) Getting Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException: in ignite 2.9.0

[Dipak jadhav (Jira)]

Dipak jadhav created IGNITE-13656:
-

 Summary: Getting Caused by: class 
org.apache.ignite.binary.BinaryInvalidTypeException: in ignite 2.9.0
 Key: IGNITE-13656
 URL: https://issues.apache.org/jira/browse/IGNITE-13656
 Project: Ignite
  Issue Type: Bug
  Components: binary
Affects Versions: 2.9.1
Reporter: Dipak jadhav


I am getting Caused by: class 
org.apache.ignite.binary.BinaryInvalidTypeException:

 
{code:java}
Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException: id_key at 
org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:689)
 at 
org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:686)
 at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1757)
 at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
 at 
org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:796)
 at 
org.apache.ignite.internal.binary.BinaryObjectImpl.value(BinaryObjectImpl.java:142)
 at 
org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateProps(QueryTypeDescriptorImpl.java:607)
 at 
org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateKeyAndValue(QueryTypeDescriptorImpl.java:587)
 at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.validateKeyAndValue(GridQueryProcessor.java:3542)
 at 
org.apache.ignite.internal.processors.cache.GridCacheContext.validateKeyAndValue(GridCacheContext.java:1907)
 ... 32 more Caused by: java.lang.ClassNotFoundException: id_key at 
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
 at 
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) at 
java.base/java.lang.Class.forName0(Native Method) at 
java.base/java.lang.Class.forName(Class.java:398) at 
org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8916) at 
org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:376)
 at 
org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:680)
 ... 41 more
{code}
 

I have upgraded from 2.8.1 to 2.9.0. I am using binary cache and cache.putAll() 
operation is throwing exception. It is working fine with 2.8.1.

The reason which we found is when we set Queryentity in cacacheconfiguration 
without indexing then above exception is thrown.

 

Instead of throwing exception on cache operation the exception should be throw 
while setting QueryEntity without indexing.

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Interoperable Ignite.NET Dates

[Pavel Tupitsyn]

Alexey,

Just to clarify before we start the discussion:
this proposal seems to introduce some breaking changes, so we are talking
about Ignite 3.0, correct?

Pavel

On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin 
wrote:

> Igniters,
>
> What do you think about changing .NET API to read/write portable dates by
> default and making that really portable?
>
> *The Problem*
> Presently .NET API writes dates as composite Ignite objects. Only .NET
> clients can read such dates: any other client (JDBC, Java, etc) does not
> understand it without custom deserialization.
>
> It is still possible to configure .NET serialization to write dates as
> Ignite dates - see DateTime Serialization note
> <
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> >.
> But then Ignite accepts only UTC dates, requiring the application
> developers to convert local dates to UTC dates and back. This task is not
> trivial: DateTime.ToUniversalTime
> <
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> >
> uses
> calendars different from Java (and the .NET calendars are the invalid ones
> - especially for pre-1990 dates).
>
> The motivation for the current default behavior was probably the desire to
> keep the Time Zone information: Ignite dates do not store time zones.
>
> In our experience interoperability is more important than storing time zone
> info.
>
> *The Proposal*
>
>1. Always write .NET dates as portable Ignite dates: get rid of the
>BinaryReflectiveSerializer.ForceTimestamp flag that currently triggers
> this
>behavior.
>   - We could still keep the ForceTimestamp flag if saving .NET dates as
>   transparent objects seems a useful case. We do not think it is
> useful.
>2. Automatically convert Local dates to UTC and back *inside*
>Ignite.NET.
>   - In this case we lose the DateTime.Kind of UTC dates: we write a UTC
>   date but we would read a Local date since Ignite would always
> convert UTC
>   to Local when reading.
>   We could add a UtcDate date flag to QuerySqlFieldAttribute
>   and BinaryReflectiveSerializer to control the deserialization
> behavior if
>   keeping dates in UTC format use case seems important.
>3. Use NodaTime  for UTC<->Local conversions.
>Noda time uses Java calendars making the conversion truely portable.
>
> *The Benefits*
>
>1. We think portable dates are much more important than storing time
>zone info. Why do we store time zones for every date on the server
> anyway?
>Time zone is client-side info.
>2. Simpler application code: no more manual configuration to trigger the
>portable behavior.
>3. Non-trivial code to make the truly portable UTC<->Local conversion is
>implemented once inside Ignite instead of having every Ignite.NET
>application implementing it.
>
> Thank you!
>
> --
> Best regards,
> Alexey
>

Interoperable Ignite.NET Dates

[Alexey Kukushkin]

Igniters,

What do you think about changing .NET API to read/write portable dates by
default and making that really portable?

*The Problem*
Presently .NET API writes dates as composite Ignite objects. Only .NET
clients can read such dates: any other client (JDBC, Java, etc) does not
understand it without custom deserialization.

It is still possible to configure .NET serialization to write dates as
Ignite dates - see DateTime Serialization note
.
But then Ignite accepts only UTC dates, requiring the application
developers to convert local dates to UTC dates and back. This task is not
trivial: DateTime.ToUniversalTime

uses
calendars different from Java (and the .NET calendars are the invalid ones
- especially for pre-1990 dates).

The motivation for the current default behavior was probably the desire to
keep the Time Zone information: Ignite dates do not store time zones.

In our experience interoperability is more important than storing time zone
info.

*The Proposal*

   1. Always write .NET dates as portable Ignite dates: get rid of the
   BinaryReflectiveSerializer.ForceTimestamp flag that currently triggers this
   behavior.
  - We could still keep the ForceTimestamp flag if saving .NET dates as
  transparent objects seems a useful case. We do not think it is useful.
   2. Automatically convert Local dates to UTC and back *inside*
   Ignite.NET.
  - In this case we lose the DateTime.Kind of UTC dates: we write a UTC
  date but we would read a Local date since Ignite would always convert UTC
  to Local when reading.
  We could add a UtcDate date flag to QuerySqlFieldAttribute
  and BinaryReflectiveSerializer to control the deserialization behavior if
  keeping dates in UTC format use case seems important.
   3. Use NodaTime  for UTC<->Local conversions.
   Noda time uses Java calendars making the conversion truely portable.

*The Benefits*

   1. We think portable dates are much more important than storing time
   zone info. Why do we store time zones for every date on the server anyway?
   Time zone is client-side info.
   2. Simpler application code: no more manual configuration to trigger the
   portable behavior.
   3. Non-trivial code to make the truly portable UTC<->Local conversion is
   implemented once inside Ignite instead of having every Ignite.NET
   application implementing it.

Thank you!

-- 
Best regards,
Alexey

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Hi!

I've updated PR: https://github.com/apache/ignite/pull/8367. Anton, Ivan,
Ivan could you please review it?

Some moments to mention:
1. I've added new suites: SerializerSuite (ignite-cassandra-serializers),
DistanceTestSuite, NaiveBayesTestSuite (ignite-ml). Should we configure a
TeamCity to run them?

2. Some tests marked as failed, I'll create corresponding tickets for them
after PR approved:
- IgnitePKIndexesMigrationToUnwrapPkTest
- P2PGridifySelfTest
- GridCacheMultithreadedFailoverAbstractTest
- WalCompactionAfterRestartTest
- GridTcpCommunicationSpiLogTest
- ComplexSecondaryKeyUnwrapSelfTest
- SqlTransactionsSelfTest

3. Add docs to DEVNOTES.txt

On Mon, Nov 2, 2020 at 11:44 AM Anton Vinogradov  wrote:

> > As I understand we
> > can't just move suites between modules, as TeamCity may depend on the
> path
> > to them.
> See no problem to update TC as well.
>
> On Fri, Oct 30, 2020 at 4:32 PM Ivan Daschinsky 
> wrote:
>
> > I suggests to mark these tests with @Ignore and file tickets to fix them.
> >
> > пт, 30 окт. 2020 г. в 16:26, Ivan Daschinsky :
> >
> > > Hi
> > >
> > > WalCompactionAfterRestartTest -- yes we need it. This test failed
> because
> > > of race (test shold be rewritten a little bit)
> > >
> > > пт, 30 окт. 2020 г. в 16:15, Max Timonin :
> > >
> > >> Hi!
> > >>
> > >> Yes, you're correct. I've developed the check and started to clean
> tests
> > >> (move them to suites, mark some tests with Ignore, etc.). I finish
> work
> > on
> > >> the core module. I hope it was the biggest one, and others are less.
> If
> > >> so,
> > >> I think I will finish the work on other modules in 1 or 2 weeks, as I
> do
> > >> this activity in the background (~10% of my work time). Actually I've
> > >> found
> > >> 3 failed tests in the core module that aren't in any suite, so I need
> > time
> > >> to discover reason of failures and if we actually need those tests:
> > >>
> > >> GridCacheMultithreadedFailoverAbstractTest
> > >> WalCompactionAfterRestartTest
> > >> GridTcpCommunicationSpiLogTest
> > >>
> > >> Also we should decide how to be with wrongly located es. As I
> understand
> > >> we
> > >> can't just move suites between modules, as TeamCity may depend on the
> > path
> > >> to them. So, for such cases I've just created suites in the right
> > module,
> > >> and replaced the test list with the new class suite. It does not look
> > >> pretty enough, but I think It's a path of least resistance. WDYT?
> > >>
> > >> BEFORE:
> > >> Module A -> SuiteA -> testA1, testA2, testB1, testB2
> > >> Module B -> testB1, testB2
> > >>
> > >> AFTER:
> > >> Module A -> SuiteA, SuiteB
> > >> Module B -> SuiteB -> testB1, testB2
> > >>
> > >> On Fri, Oct 30, 2020 at 3:38 PM Anton Vinogradov 
> wrote:
> > >>
> > >> > Folks,
> > >> > What's the current state of this thread?
> > >> > AFAIU, we found unused and wrongly located tests and developed some
> > >> > checker, could we split this to some PRs?
> > >> > Let's merge tests usage fix and location fixes first, this will
> > provide
> > >> us
> > >> > an ability to automate check using Travis.
> > >> >
> > >> > On Tue, Oct 20, 2020 at 12:06 PM Ivan Pavlukhin <
> vololo...@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Max, Ivan,
> > >> > >
> > >> > > Using explicit @Ignore and the automated check sounds good to me.
> If
> > >> > > nobody has arguments against it I think we should do it.
> > >> > >
> > >> > > 2020-10-19 19:30 GMT+03:00, Max Timonin  >:
> > >> > > > Hi Ivan,
> > >> > > >
> > >> > > > I've checked the ticket you provide. It contains subtasks to
> > >> uncomment
> > >> > or
> > >> > > > to remove some unused tests. It definitely describes some cases
> > I've
> > >> > > found.
> > >> > > > So what do you think if I uncomment them in suites, add @Ignore
> > >> > > annotation
> > >> > > > for those tests while the tickets are open? This will help to
> find
> > >> out
> > >> > > > tests that were forgiven in a recent time.
> > >> > > >
> > >> > > > Also I believe that this check must be automated. I didn't find
> a
> > >> way
> > >> > how
> > >> > > > uncomment / unused tests are found in the ticket. If there is no
> > >> any -
> > >> > I
> > >> > > > propose mine PR for this purpose.
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On Mon, Oct 19, 2020 at 5:24 PM Ivan Daschinsky <
> > >> ivanda...@gmail.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > >> Ivan, as far as I understand, Max also created verification
> check
> > >> for
> > >> > > not
> > >> > > >> included test and found a few tests, that have never been
> > included
> > >> in
> > >> > > any
> > >> > > >> testsuites.
> > >> > > >>
> > >> > > >> Also, I suppose, that even if we cannot run some tests, these
> > tests
> > >> > > >> should
> > >> > > >> be ignored using annotation, but not commented.
> > >> > > >>
> > >> > > >> пн, 19 окт. 2020 г. в 16:33, Ivan Pavlukhin <
> vololo...@gmail.com
> > >:
> > >> > > >>
> > >> > > >> > Hi Max,
> > >> > > >> >
> > >> > > >> > There is an 

[jira] [Created] (IGNITE-13655) Implement readiness probe REST endpoint

[Alexander Korenshteyn (Jira)]

Alexander Korenshteyn created IGNITE-13655:
--

 Summary: Implement readiness probe REST endpoint
 Key: IGNITE-13655
 URL: https://issues.apache.org/jira/browse/IGNITE-13655
 Project: Ignite
  Issue Type: New Feature
  Components: rest
Affects Versions: 2.9.1
Reporter: Alexander Korenshteyn
 Fix For: 2.9.1


Create a REST command (PROBE) which returns a success code (200) if kernal has 
fully started 

and 503 otherwise

 

implemented by attaching to the join future.

It looks pretty similar to warmup machinery (in terms of implementation).

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Custom Affinity Functions proposed for removal?

[Alexei Scherbakov]

Thanks for the clarification.

There was no intention to remove the customizable key to partition mapping.

Difficulties arise when mapping partitions to nodes, so it's desirable to
have internally tested implementation with a way to customize it's behavior
without additional coding on the user side.

пн, 2 нояб. 2020 г. в 23:01, Raymond Wilson :

> Just to be clear, the affinity functions we are using convert keys to
> partitions, we do not map partitions to nodes and leave that to Ignite.
>
> On Tue, Nov 3, 2020 at 8:48 AM Alexei Scherbakov <
> alexey.scherbak...@gmail.com> wrote:
>
> > Hello.
> >
> > Custom affinity functions can cause weird bugs and data loss if
> implemented
> > wrongly.
> > There is an intention  to keep a backup filter based on user attributes
> > (with additional validation logic to ensure correctness) for controllable
> > data placement.
> >
> > Can you describe more precisely why you had to implement custom affinity
> > functions and not resort to default rendezvous affinity + backup filter ?
> >
> >
> > пн, 2 нояб. 2020 г. в 21:45, Raymond Wilson  >:
> >
> > > We also use custom affinity functions (vis the C# client).
> > >
> > > The wish list mentions use of a particular annotation
> > > (@CentralizedAffinityFunction):
> > > Is the wish to remove just this annotation, or the ability to define
> > custom
> > > affinity functions at all?
> > >
> > > In our case we use affinity functions to ensure particular distribution
> > of
> > > spatial data across a processing cluster to ensure even load
> management.
> > >
> > > On Tue, Nov 3, 2020 at 5:31 AM Moti Nisenson 
> > > wrote:
> > >
> > > > I saw at
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0+Wishlist
> > > > that custom affinity functions are on the potential wishlist for
> > removal.
> > > > The way we're using it's very critical that we be able to control the
> > > > placement of data quite precisely - as part of that we specify
> > explicitly
> > > > the partition we want in the key, and then our affinity function uses
> > > that
> > > > (else delegating to default rendezvous). We don't need all the
> > > > abilities there, although I think that often others do.
> > > >
> > > > This seems to me to be a case that the benefit of removing this is
> > > minimal
> > > > and could cause quite a lot of disruption to users.
> > > >
> > > > Thanks!
> > > >
> > >
> > >
> > > --
> > > 
> > > Raymond Wilson
> > > Solution Architect, Civil Construction Software Systems (CCSS)
> > > 11 Birmingham Drive | Christchurch, New Zealand
> > > +64-21-2013317 Mobile
> > > raymond_wil...@trimble.com
> > >
> > > <
> > >
> >
> https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch
> > > >
> > >
> >
> >
> > --
> >
> > Best regards,
> > Alexei Scherbakov
> >
>
>
> --
> 
> Raymond Wilson
> Solution Architect, Civil Construction Software Systems (CCSS)
> 11 Birmingham Drive | Christchurch, New Zealand
> +64-21-2013317 Mobile
> raymond_wil...@trimble.com
>
> <
> https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch
> >
>


-- 

Best regards,
Alexei Scherbakov

Re: Custom Affinity Functions proposed for removal?

[Raymond Wilson]

Just to be clear, the affinity functions we are using convert keys to
partitions, we do not map partitions to nodes and leave that to Ignite.

On Tue, Nov 3, 2020 at 8:48 AM Alexei Scherbakov <
alexey.scherbak...@gmail.com> wrote:

> Hello.
>
> Custom affinity functions can cause weird bugs and data loss if implemented
> wrongly.
> There is an intention  to keep a backup filter based on user attributes
> (with additional validation logic to ensure correctness) for controllable
> data placement.
>
> Can you describe more precisely why you had to implement custom affinity
> functions and not resort to default rendezvous affinity + backup filter ?
>
>
> пн, 2 нояб. 2020 г. в 21:45, Raymond Wilson :
>
> > We also use custom affinity functions (vis the C# client).
> >
> > The wish list mentions use of a particular annotation
> > (@CentralizedAffinityFunction):
> > Is the wish to remove just this annotation, or the ability to define
> custom
> > affinity functions at all?
> >
> > In our case we use affinity functions to ensure particular distribution
> of
> > spatial data across a processing cluster to ensure even load management.
> >
> > On Tue, Nov 3, 2020 at 5:31 AM Moti Nisenson 
> > wrote:
> >
> > > I saw at
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0+Wishlist
> > > that custom affinity functions are on the potential wishlist for
> removal.
> > > The way we're using it's very critical that we be able to control the
> > > placement of data quite precisely - as part of that we specify
> explicitly
> > > the partition we want in the key, and then our affinity function uses
> > that
> > > (else delegating to default rendezvous). We don't need all the
> > > abilities there, although I think that often others do.
> > >
> > > This seems to me to be a case that the benefit of removing this is
> > minimal
> > > and could cause quite a lot of disruption to users.
> > >
> > > Thanks!
> > >
> >
> >
> > --
> > 
> > Raymond Wilson
> > Solution Architect, Civil Construction Software Systems (CCSS)
> > 11 Birmingham Drive | Christchurch, New Zealand
> > +64-21-2013317 Mobile
> > raymond_wil...@trimble.com
> >
> > <
> >
> https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch
> > >
> >
>
>
> --
>
> Best regards,
> Alexei Scherbakov
>


-- 

Raymond Wilson
Solution Architect, Civil Construction Software Systems (CCSS)
11 Birmingham Drive | Christchurch, New Zealand
+64-21-2013317 Mobile
raymond_wil...@trimble.com

Re: Custom Affinity Functions proposed for removal?

[Alexei Scherbakov]

Hello.

Custom affinity functions can cause weird bugs and data loss if implemented
wrongly.
There is an intention  to keep a backup filter based on user attributes
(with additional validation logic to ensure correctness) for controllable
data placement.

Can you describe more precisely why you had to implement custom affinity
functions and not resort to default rendezvous affinity + backup filter ?


пн, 2 нояб. 2020 г. в 21:45, Raymond Wilson :

> We also use custom affinity functions (vis the C# client).
>
> The wish list mentions use of a particular annotation
> (@CentralizedAffinityFunction):
> Is the wish to remove just this annotation, or the ability to define custom
> affinity functions at all?
>
> In our case we use affinity functions to ensure particular distribution of
> spatial data across a processing cluster to ensure even load management.
>
> On Tue, Nov 3, 2020 at 5:31 AM Moti Nisenson 
> wrote:
>
> > I saw at
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0+Wishlist
> > that custom affinity functions are on the potential wishlist for removal.
> > The way we're using it's very critical that we be able to control the
> > placement of data quite precisely - as part of that we specify explicitly
> > the partition we want in the key, and then our affinity function uses
> that
> > (else delegating to default rendezvous). We don't need all the
> > abilities there, although I think that often others do.
> >
> > This seems to me to be a case that the benefit of removing this is
> minimal
> > and could cause quite a lot of disruption to users.
> >
> > Thanks!
> >
>
>
> --
> 
> Raymond Wilson
> Solution Architect, Civil Construction Software Systems (CCSS)
> 11 Birmingham Drive | Christchurch, New Zealand
> +64-21-2013317 Mobile
> raymond_wil...@trimble.com
>
> <
> https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch
> >
>


-- 

Best regards,
Alexei Scherbakov

Re: Custom Affinity Functions proposed for removal?

[Raymond Wilson]

We also use custom affinity functions (vis the C# client).

The wish list mentions use of a particular annotation
(@CentralizedAffinityFunction):
Is the wish to remove just this annotation, or the ability to define custom
affinity functions at all?

In our case we use affinity functions to ensure particular distribution of
spatial data across a processing cluster to ensure even load management.

On Tue, Nov 3, 2020 at 5:31 AM Moti Nisenson 
wrote:

> I saw at
>
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0+Wishlist
> that custom affinity functions are on the potential wishlist for removal.
> The way we're using it's very critical that we be able to control the
> placement of data quite precisely - as part of that we specify explicitly
> the partition we want in the key, and then our affinity function uses that
> (else delegating to default rendezvous). We don't need all the
> abilities there, although I think that often others do.
>
> This seems to me to be a case that the benefit of removing this is minimal
> and could cause quite a lot of disruption to users.
>
> Thanks!
>


-- 

Raymond Wilson
Solution Architect, Civil Construction Software Systems (CCSS)
11 Birmingham Drive | Christchurch, New Zealand
+64-21-2013317 Mobile
raymond_wil...@trimble.com

Re: [DISCUSS] Disable socket linger by default in TCP discovery SPI.

[Ivan Daschinsky]

Ilya, yes, there is an option TcpDiscoverySpi#soLinger. The main question
is why the default value is true, 5 instead of false,0

пн, 2 нояб. 2020 г., 20:14 Ilya Kasnacheev :

> Hello!
>
> Is there any option to re-enable linger on SSL sockets?
>
> Telling people to re-configure does not help if they can't.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 30 окт. 2020 г. в 15:21, Anton Vinogradov :
>
> > > When TLS 1.3 is introduced, whole sun.security.ssl.SSLSocketImpl was
> > > rewritten.
> > Correct, I meant rewritten TLSv1.3, the good news that 1.2- also were
> > fixed.
> > so,
> > -- brand new TLS with any linger
> > -- plain old TLS with linger>0
> >
> > On Fri, Oct 30, 2020 at 3:10 PM Ivan Daschinsky 
> > wrote:
> >
> > > Ilya, Anton.
> > > It means that not if TLS 1.3 is worked ok and with TLS < 1.2 is not ok.
> > >
> > > When TLS 1.3 is introduced, whole sun.security.ssl.SSLSocketImpl was
> > > rewritten.
> > > There is not any code anymore that could cause a deadlock.
> > > Therefore, in JDK, that supports TLS 1.3, this option is unnecessary,
> > even
> > > if you use TLS 1.2
> > >
> > >
> > > пт, 30 окт. 2020 г. в 14:46, Anton Vinogradov :
> > >
> > > > Ilya
> > > > > I think we should still keep setting linger if SSL is enabled
> > > > Modern (updated) JVMs do not require this.
> > > > AFAIK, Problem caused this workaround already fixed everywhere,
> > including
> > > > JDK 8.
> > > >
> > > > > If SSL only works with TLSv1.3 and no linger
> > > > SSL works if
> > > > -- TLSv1.3 with any linger
> > > > -- TLSv1.2- with linger>0
> > > >
> > > > > we should make TLSv1.3 a
> > > > > default. If JVM does not support it, users will have to reconfigure
> > > > > explicitly.
> > > > I don't think it's a good idea to reconfigure production environments
> > > this
> > > > way.
> > > >
> > > > P.s.
> > > > My +1 to zero linger as default + warning on SSL enabled on JVM
> before
> > > the
> > > > fix + warning at documentation + migration notes
> > > >
> > > > On Fri, Oct 30, 2020 at 2:19 PM Ilya Kasnacheev <
> > > ilya.kasnach...@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hello!
> > > > >
> > > > > I think we should still keep setting linger if SSL is enabled, and
> > not
> > > > > expect user to enable it (or face consequences).
> > > > >
> > > > > If SSL only works with TLSv1.3 and no linger, we should make
> TLSv1.3
> > a
> > > > > default. If JVM does not support it, user will have to reconfigure
> > > > > explicitly.
> > > > >
> > > > > Regards,
> > > > > --
> > > > > Ilya Kasnacheev
> > > > >
> > > > >
> > > > > пт, 30 окт. 2020 г. в 14:05, Steshin Vladimir  >:
> > > > >
> > > > > > *
> > > > > >
> > > > > > Hi, Igniters.
> > > > > >
> > > > > > We’ve found that enabled by default socket linger causes
> unexpected
> > > > > > delay in detection of node failure.
> > > > > >
> > > > > >
> > > > > > Moreover, long closing of socket works as Thread.sleep() within
> > > > > > algorithms of failure detection and connection recovery in TCP
> > > > > > discovery. These time gaps lead to hardly predictable behavior of
> > the
> > > > > > discovery. When the socket linger is enabled, it’s hard or even
> > > > > > impossible to figure out what time is taken to detect node
> failure
> > > and
> > > > > > restore connections with the provided settings.
> > > > > >
> > > > > > Socket linger was enabled only as a workaround for SSL bugs (i.e.
> > > [2],
> > > > > > [3]). It was enabled without including in failure processing
> > routines
> > > > in
> > > > > > TCP discovery SPI as described above. SSL bugs, mentioned above,
> > were
> > > > > > fixed and backported to various JDK, supporting TLS 1.3 ([4] and
> > > [5]).
> > > > > >
> > > > > >
> > > > > > I’d suggest to disable socket linger by default, because enabled
> > > socket
> > > > > > linger prolongs detection of node failure. The ticket is [1]. In
> > case
> > > > of
> > > > > > SSL issues the linger could be enabled. Or one may just update
> JDK.
> > > > > > We'll provide the documentation.
> > > > > >
> > > > > > WDYT?
> > > > > >
> > > > > >
> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-13643
> > > > > >
> > > > > > [2] https://bugs.openjdk.java.net/browse/JDK-8219658
> > > > > >
> > > > > > [3]https://issues.apache.org/jira/browse/IGNITE-12818
> > > > > >
> > > > > > [4]https://bugs.openjdk.java.net/browse/JDK-8245468
> > > > > >
> > > > > > [5]
> > > > https://www.oracle.com/java/technologies/javase/8u261-relnotes.html
> > > > > >
> > > > > > *
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > > Sincerely yours, Ivan Daschinskiy
> > >
> >
>

[jira] [Created] (IGNITE-13654) Add snapshot commands to REST API

[Maxim Muzafarov (Jira)]

Maxim Muzafarov created IGNITE-13654:


 Summary: Add snapshot commands to REST API
 Key: IGNITE-13654
 URL: https://issues.apache.org/jira/browse/IGNITE-13654
 Project: Ignite
  Issue Type: Task
Reporter: Maxim Muzafarov
Assignee: Maxim Muzafarov
 Fix For: 2.10


Snapshot create/cancel commands must be added to the Apache Ignite REST API.
Documentation must be also updated.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Disable socket linger by default in TCP discovery SPI.

[Ilya Kasnacheev]

Hello!

Is there any option to re-enable linger on SSL sockets?

Telling people to re-configure does not help if they can't.

Regards,
-- 
Ilya Kasnacheev


пт, 30 окт. 2020 г. в 15:21, Anton Vinogradov :

> > When TLS 1.3 is introduced, whole sun.security.ssl.SSLSocketImpl was
> > rewritten.
> Correct, I meant rewritten TLSv1.3, the good news that 1.2- also were
> fixed.
> so,
> -- brand new TLS with any linger
> -- plain old TLS with linger>0
>
> On Fri, Oct 30, 2020 at 3:10 PM Ivan Daschinsky 
> wrote:
>
> > Ilya, Anton.
> > It means that not if TLS 1.3 is worked ok and with TLS < 1.2 is not ok.
> >
> > When TLS 1.3 is introduced, whole sun.security.ssl.SSLSocketImpl was
> > rewritten.
> > There is not any code anymore that could cause a deadlock.
> > Therefore, in JDK, that supports TLS 1.3, this option is unnecessary,
> even
> > if you use TLS 1.2
> >
> >
> > пт, 30 окт. 2020 г. в 14:46, Anton Vinogradov :
> >
> > > Ilya
> > > > I think we should still keep setting linger if SSL is enabled
> > > Modern (updated) JVMs do not require this.
> > > AFAIK, Problem caused this workaround already fixed everywhere,
> including
> > > JDK 8.
> > >
> > > > If SSL only works with TLSv1.3 and no linger
> > > SSL works if
> > > -- TLSv1.3 with any linger
> > > -- TLSv1.2- with linger>0
> > >
> > > > we should make TLSv1.3 a
> > > > default. If JVM does not support it, users will have to reconfigure
> > > > explicitly.
> > > I don't think it's a good idea to reconfigure production environments
> > this
> > > way.
> > >
> > > P.s.
> > > My +1 to zero linger as default + warning on SSL enabled on JVM before
> > the
> > > fix + warning at documentation + migration notes
> > >
> > > On Fri, Oct 30, 2020 at 2:19 PM Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hello!
> > > >
> > > > I think we should still keep setting linger if SSL is enabled, and
> not
> > > > expect user to enable it (or face consequences).
> > > >
> > > > If SSL only works with TLSv1.3 and no linger, we should make TLSv1.3
> a
> > > > default. If JVM does not support it, user will have to reconfigure
> > > > explicitly.
> > > >
> > > > Regards,
> > > > --
> > > > Ilya Kasnacheev
> > > >
> > > >
> > > > пт, 30 окт. 2020 г. в 14:05, Steshin Vladimir :
> > > >
> > > > > *
> > > > >
> > > > > Hi, Igniters.
> > > > >
> > > > > We’ve found that enabled by default socket linger causes unexpected
> > > > > delay in detection of node failure.
> > > > >
> > > > >
> > > > > Moreover, long closing of socket works as Thread.sleep() within
> > > > > algorithms of failure detection and connection recovery in TCP
> > > > > discovery. These time gaps lead to hardly predictable behavior of
> the
> > > > > discovery. When the socket linger is enabled, it’s hard or even
> > > > > impossible to figure out what time is taken to detect node failure
> > and
> > > > > restore connections with the provided settings.
> > > > >
> > > > > Socket linger was enabled only as a workaround for SSL bugs (i.e.
> > [2],
> > > > > [3]). It was enabled without including in failure processing
> routines
> > > in
> > > > > TCP discovery SPI as described above. SSL bugs, mentioned above,
> were
> > > > > fixed and backported to various JDK, supporting TLS 1.3 ([4] and
> > [5]).
> > > > >
> > > > >
> > > > > I’d suggest to disable socket linger by default, because enabled
> > socket
> > > > > linger prolongs detection of node failure. The ticket is [1]. In
> case
> > > of
> > > > > SSL issues the linger could be enabled. Or one may just update JDK.
> > > > > We'll provide the documentation.
> > > > >
> > > > > WDYT?
> > > > >
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-13643
> > > > >
> > > > > [2] https://bugs.openjdk.java.net/browse/JDK-8219658
> > > > >
> > > > > [3]https://issues.apache.org/jira/browse/IGNITE-12818
> > > > >
> > > > > [4]https://bugs.openjdk.java.net/browse/JDK-8245468
> > > > >
> > > > > [5]
> > > https://www.oracle.com/java/technologies/javase/8u261-relnotes.html
> > > > >
> > > > > *
> > > > >
> > > >
> > >
> >
> >
> > --
> > Sincerely yours, Ivan Daschinskiy
> >
>

Re: 2.9.1 release proposal

[Yaroslav Molochkov]

Guys,

should you agree that issues with the 2.9.1-rc tag are good enough
for a maintenance release, i'd like to give it a go.

On Thu, Oct 29, 2020 at 5:33 PM Alexey Zinoviev 
wrote:

> Let's discuss the possible planning dates for feature freeze for 2.10, for
> example? Do you have any plans or ideas?
>
> чт, 29 окт. 2020 г. в 17:24, Andrey Gura :
>
> > Hi,
> >
> > I agree with Alexey. We should release 2.9.1 and start preparing for
> > the 2.10 release.
> >
> > 2.x releases usually take a lot of time. So we can release 2.9.1, and
> > even 2.9.2 before 2.10.
> >
> > On Thu, Oct 29, 2020 at 12:02 PM Alexey Goncharuk
> >  wrote:
> > >
> > > Hello folks,
> > >
> > > I think we should start both 2.9.1 and 2.10. In practice, maintenance
> > > release contains only critical and usability bugfixes (for example, I
> > would
> > > include this ticket [1] to include in 2.9.1 as it prevents users from
> > using
> > > tracing) and is released much faster than a minor release.
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-13640
> > >
> > > вт, 27 окт. 2020 г. в 21:41, Guru Stron :
> > >
> > > > Hello,
> > > >
> > > > Agree with Pavel.
> > > >
> > > > On Tue, 27 Oct 2020 at 19:22, Pavel Tupitsyn 
> > wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > I think we should plan 2.10 instead of 2.9.1.
> > > > > ignite-2.9 branch was cut 4 months ago, a bunch of new features are
> > > > waiting
> > > > > to be released.
> > > > >
> > > > > On Tue, Oct 27, 2020 at 4:23 AM 18624049226 <18624049...@163.com>
> > wrote:
> > > > >
> > > > > > Hello,
> > > > > >
> > > > > > I suggest that the remaining document issue in version 2.9.0 can
> be
> > > > > > solved in version 2.9.1, and it is not a good practice to
> postpone
> > to
> > > > > > version 2.10.
> > > > > >
> > > > > > 在 2020/10/27 上午2:00, Maxim Muzafarov 写道:
> > > > > > > Hello,
> > > > > > >
> > > > > > > +1
> > > > > > > Should we start the discussion about the release leader and
> > release
> > > > > > dates?
> > > > > > >
> > > > > > > On Tue, 20 Oct 2020 at 10:12, Anton Vinogradov 
> > > > wrote:
> > > > > > >> +1 to start the 2.9.1 release process once as soon as 2.9
> > released.
> > > > > > >>
> > > > > > >> On Mon, Oct 19, 2020 at 8:49 PM Nikolay Izhikov <
> > > > nizhi...@apache.org>
> > > > > > wrote:
> > > > > > >>
> > > > > > >>> Hello, Yaroslav.
> > > > > > >>>
> > > > > > >>> I support the idea.
> > > > > > >>> But, we should carefully work with the release scope.
> > > > > > >>>
> > > > > > >>> IGNITE-13477 - fix for a SQL tracing that will be available
> > only in
> > > > > > 2.10
> > > > > > >>> IGNITE-13427 - fix for a new system view that exists in
> master
> > > > only,
> > > > > we
> > > > > > >>> need to include IGNITE-13409
> > > > > > >>>
> > > > > > >>> Other tickets should be checked, also.
> > > > > > >>> Is this a real fix of a released bug or we fix some issue we
> > bring
> > > > > with
> > > > > > >>> the brand new contribution.
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> I propose to include the following tickets, also:
> > > > > > >>>
> > > > > > >>> CMD tools improvements:
> > > > > > >>>
> > > > > > >>> IGNITE-13488 - Command to print metric value
> > > > > > >>> IGNITE-13426 - Command to print system view content
> > > > > > >>> IGNITE-13422 - Parameter to explicitly enable experimental
> > commands
> > > > > > >>>
> > > > > > >>> IGNITE-13380 - Output IgniteSystemProperties via ignite.sh
> > > > > > >>>
> > > > > > >>> New system views:
> > > > > > >>>
> > > > > > >>> IGNITE-13409 Metastorage and DistributedMetastorage viewы.
> > > > > > >>> IGNITE-13408 BinaryMetadata view.
> > > > > > >>>
> > > > > > >>>
> > > > > >  19 окт. 2020 г., в 18:20, Yaroslav Molochkov <
> > > > molochko...@gmail.com
> > > > > >
> > > > > > >>> написал(а):
> > > > > >  Hello, Igniters!
> > > > > > 
> > > > > >  I've compiled a list of tickets that, i think, deserve to be
> > > > > released
> > > > > > in
> > > > > > >>> a
> > > > > >  minor Ignite release (meaning in 2.9.1) after 2.9. Here they
> > are:
> > > > > > 
> > > > > >  IGNITE-13569
> > > > > >  disable archiving + walCompactionEnabled probably broke
> > reading
> > > > from
> > > > > > wal
> > > > > > >>> on
> > > > > >  server restart
> > > > > > 
> > > > > >  IGNITE-13418
> > > > > >  Deadlock on multiple cache delete
> > > > > > 
> > > > > >  IGNITE-13563
> > > > > >  Deserializing IBinaryObject containing an IBinaryObject
> field
> > > > fails
> > > > > > 
> > > > > >  IGNITE-13575
> > > > > >  Invalid blocking section in GridNioWorker and
> > GridNioClientWorker
> > > > > > leads
> > > > > > >>> to
> > > > > >  false positive blocking thread detection
> > > > > > 
> > > > > >  IGNITE-13458
> > > > > >  RebalancingPartitionsTotal metrics
> > > > > > 
> > > > > >  IGNITE-13536
> > > > > >  .NET: Child processes become zombies when persistence is
> used
> > with
> > > > > > 

Custom Affinity Functions proposed for removal?

[Moti Nisenson]

I saw at
https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0+Wishlist
that custom affinity functions are on the potential wishlist for removal.
The way we're using it's very critical that we be able to control the
placement of data quite precisely - as part of that we specify explicitly
the partition we want in the key, and then our affinity function uses that
(else delegating to default rendezvous). We don't need all the
abilities there, although I think that often others do.

This seems to me to be a case that the benefit of removing this is minimal
and could cause quite a lot of disruption to users.

Thanks!

Re: [DISCUSS] Ignite 3.0 development approach

[Pavel Tupitsyn]

> not intend to rewrite everything from scratch

> Every single test from Ignite 2.x should be moved to Ignite 3
> regardless of how we choose to proceed.

Alexey, thank you for the explanation, this addresses all of my concerns.





On Mon, Nov 2, 2020 at 6:43 PM Andrey Mashenkov 
wrote:

> Hi, Igniters.
>
> * AFAIU, we need a new repo if we want to apply different restrictions to
> pull requests,
> otherwise I see no difference for myself.
> E.g. make static analysis (do we have?), compile, styles, and javadoc
> checks mandatory.
>
> I think that relaxed requirements here will lead to bad product quality.
>
> * Agree with Pavel, we should 'keep' integrations tests somehow.
> During active development tests will be broken most of time, so,
> I'd port them e.g. suite-by-suite once we will have a stable and featured
> environment to run them and of course make test's code clear and avoid
> bad/non-relevant ones.
>
> * I like bottom-up approach.
> With it we could make a better framework. I mean clear component lifecycle,
> component wiring mechanics, general methods to approach core components
> such as exchange/communication
> to avoid code mess like we have in ExchangeFuture with all these custom
> callbacks for each component, interfaces like
> PartitionsExchangeAware, IgniteChangeGlobalStateSupport and
> a pack of start/stop/onKernalStart/onPluginStart/onActivate/onDisconnected
> and so on in various unexpected places.
> Hope, we will be able to port most of the good code to the new framework
> version.
>
>
>
> On Mon, Nov 2, 2020 at 6:18 PM Alexey Goncharuk <
> alexey.goncha...@gmail.com>
> wrote:
>
> > Nikolay, Pavel,
> >
> > Thanks for the feedback! First of all, I wanted to stress that I do not
> > intend to rewrite everything from scratch (I never used this phrase).
> There
> > are significant parts of code that would be moved with minimal
> > modifications. Second, I never said that we will get rid of the old tests
> > codebase. Every single test from Ignite 2.x should be moved to Ignite 3
> > regardless of how we choose to proceed.
> >
> > My point is that for some parts of the code a clean bottom-up
> > implementation will be cheaper in many ways. Let me give you a few
> concrete
> > examples:
> >
> >- I think no one can object that we need a cleanly separated
> persistence
> >layer for Ignite. There is a very raw draft IEP for this already. On
> the
> >other hand, I think we also can agree that we need a split-brain
> > resistant
> >replication protocol for caches. There is also an IEP for this.
> Neither
> > of
> >the changes is a good fit for 2.x because they are likely to introduce
> >breaking changes in the persistence layer, configuration and behavior.
> >Additionally, these components are now tightly coupled, so there is no
> > way
> >these two changes can be implemented in parallel and then merged
> > together
> >easily. So what we will end up with is having to implement these
> changes
> >sequentially, fixing all existing tests twice, and essentially
> throwing
> >away half of the work done because the other part of the change is
> >re-implemented
> >- Similar example goes with getting rid of IgniteInternalFuture and
> >replacing it with CompletableFuture, and any other change that touches
> > the
> >asynchronous part of the code.
> >
> > Third, I do not see how this choice affects the UX of Ignite. The end
> user
> > experience must be fixed in the IEP regardless of the development process
> > and the fact that we have gaps in this area in Ignite 2.x just confirms
> > that.
> >
> > Pavel, agree that a repo/branch is a technicality, I guess if
> reformulate,
> > my point is that we might agree to have a single development master
> branch
> > with 'disassembled' end-to-end functionality for some period of time to
> > speed up development, and re-assemble the core features after having
> > submodules tested independently.
> >
> > Nikolay,
> >   >We have many features that have to evolve.
> >   >Snapshots, rebalance, tooling, tracing, zookeeper support, etc.
> > This is not very specific. In the end, resources are limited and we will
> > not be able to drive both tracks simultaneously, especially after a
> couple
> > of features having been implemented for Ignite 3.0. If there are indeed
> > some major changes that we want to do in Ignite 2.x instead of putting
> > effort into 3.0 - let's discuss them. I am just not aware of any, that's
> > why I am eager to move forward with Ignite 3.0.
> >
> > >We have bugs and issues that can be fixed in 2.x without breaking
> backward
> > compatibility.
> > >We have many users who are happy with the 2.x with all it’s issues.
> > These changes will be covered by end-to-end tests and migrated to Ignite
> > 3.0, so I see no issues here.
> >
> > Finally, Anton & Nikolay
> > I do not have an estimate for this simply because the activity is
> > community-driven and it depends on the number of 

Re: [DISCUSS] Ignite 3.0 development approach

[Andrey Mashenkov]

Hi, Igniters.

* AFAIU, we need a new repo if we want to apply different restrictions to
pull requests,
otherwise I see no difference for myself.
E.g. make static analysis (do we have?), compile, styles, and javadoc
checks mandatory.

I think that relaxed requirements here will lead to bad product quality.

* Agree with Pavel, we should 'keep' integrations tests somehow.
During active development tests will be broken most of time, so,
I'd port them e.g. suite-by-suite once we will have a stable and featured
environment to run them and of course make test's code clear and avoid
bad/non-relevant ones.

* I like bottom-up approach.
With it we could make a better framework. I mean clear component lifecycle,
component wiring mechanics, general methods to approach core components
such as exchange/communication
to avoid code mess like we have in ExchangeFuture with all these custom
callbacks for each component, interfaces like
PartitionsExchangeAware, IgniteChangeGlobalStateSupport and
a pack of start/stop/onKernalStart/onPluginStart/onActivate/onDisconnected
and so on in various unexpected places.
Hope, we will be able to port most of the good code to the new framework
version.



On Mon, Nov 2, 2020 at 6:18 PM Alexey Goncharuk 
wrote:

> Nikolay, Pavel,
>
> Thanks for the feedback! First of all, I wanted to stress that I do not
> intend to rewrite everything from scratch (I never used this phrase). There
> are significant parts of code that would be moved with minimal
> modifications. Second, I never said that we will get rid of the old tests
> codebase. Every single test from Ignite 2.x should be moved to Ignite 3
> regardless of how we choose to proceed.
>
> My point is that for some parts of the code a clean bottom-up
> implementation will be cheaper in many ways. Let me give you a few concrete
> examples:
>
>- I think no one can object that we need a cleanly separated persistence
>layer for Ignite. There is a very raw draft IEP for this already. On the
>other hand, I think we also can agree that we need a split-brain
> resistant
>replication protocol for caches. There is also an IEP for this. Neither
> of
>the changes is a good fit for 2.x because they are likely to introduce
>breaking changes in the persistence layer, configuration and behavior.
>Additionally, these components are now tightly coupled, so there is no
> way
>these two changes can be implemented in parallel and then merged
> together
>easily. So what we will end up with is having to implement these changes
>sequentially, fixing all existing tests twice, and essentially throwing
>away half of the work done because the other part of the change is
>re-implemented
>- Similar example goes with getting rid of IgniteInternalFuture and
>replacing it with CompletableFuture, and any other change that touches
> the
>asynchronous part of the code.
>
> Third, I do not see how this choice affects the UX of Ignite. The end user
> experience must be fixed in the IEP regardless of the development process
> and the fact that we have gaps in this area in Ignite 2.x just confirms
> that.
>
> Pavel, agree that a repo/branch is a technicality, I guess if reformulate,
> my point is that we might agree to have a single development master branch
> with 'disassembled' end-to-end functionality for some period of time to
> speed up development, and re-assemble the core features after having
> submodules tested independently.
>
> Nikolay,
>   >We have many features that have to evolve.
>   >Snapshots, rebalance, tooling, tracing, zookeeper support, etc.
> This is not very specific. In the end, resources are limited and we will
> not be able to drive both tracks simultaneously, especially after a couple
> of features having been implemented for Ignite 3.0. If there are indeed
> some major changes that we want to do in Ignite 2.x instead of putting
> effort into 3.0 - let's discuss them. I am just not aware of any, that's
> why I am eager to move forward with Ignite 3.0.
>
> >We have bugs and issues that can be fixed in 2.x without breaking backward
> compatibility.
> >We have many users who are happy with the 2.x with all it’s issues.
> These changes will be covered by end-to-end tests and migrated to Ignite
> 3.0, so I see no issues here.
>
> Finally, Anton & Nikolay
> I do not have an estimate for this simply because the activity is
> community-driven and it depends on the number of people willing to
> contribute. With the current pace, I would hope to have an RC of Ignite 3.0
> to be ready by the end of 2021. My gut feeling is that by moving with
> incremental changes, we will not be able to implement even half of the
> wishlist by that time.
> I doubt that releasing several major releases with breaking changes will
> make Ignite users happy either because each upgrade will cost Ignite users
> money, so the fewer major versions we release, the better. Thus my wish to
> include all breaking changes in one 

Re: [DISCUSS] Ignite 3.0 development approach

[Ilya Kasnacheev]

Hello!

In my opinion, what you are actually proposing is writing a new
IMDG/distributed database.

I'm not sure why one would assume that this new product will be
particularly successful with users. We have some very good developers out
there now, but some of the people who actually wrote Ignite 2.x are no
longer with us. Maybe it will be a new and good IMDG, maybe not. Time has
passed, the bar is now higher than it was before. People might as well
consider moving to a different IMDG not called Ignite, than moving to a
different, experimental IMDG named Ignite 3.0.

Maybe there are some bad parts that we have (such as
persistence format/separation, split-brain protocol, etc), but we have 95
out of 100 parts which are very good and battle tested, and I would prefer
them to be kept intact instead of cherry-picking them one by one into the
new grid.

We have very good distributed compute/peer class loading support, people
don't realize how good it is, but they will notice if it is gone/rewritten
from scratch to "just ok" state instead of "very good".

I think that we should just do a deep refactoring/API change in 3.0, and
leave the rest of stuff to 3.x or 4.0. I also happen to think that the code
we already have is very valuable, and it would be a pity to see it replaced
with cleaner, worse offering.

Regards,
-- 
Ilya Kasnacheev


пн, 2 нояб. 2020 г. в 18:18, Alexey Goncharuk :

> Nikolay, Pavel,
>
> Thanks for the feedback! First of all, I wanted to stress that I do not
> intend to rewrite everything from scratch (I never used this phrase). There
> are significant parts of code that would be moved with minimal
> modifications. Second, I never said that we will get rid of the old tests
> codebase. Every single test from Ignite 2.x should be moved to Ignite 3
> regardless of how we choose to proceed.
>
> My point is that for some parts of the code a clean bottom-up
> implementation will be cheaper in many ways. Let me give you a few concrete
> examples:
>
>- I think no one can object that we need a cleanly separated persistence
>layer for Ignite. There is a very raw draft IEP for this already. On the
>other hand, I think we also can agree that we need a split-brain
> resistant
>replication protocol for caches. There is also an IEP for this. Neither
> of
>the changes is a good fit for 2.x because they are likely to introduce
>breaking changes in the persistence layer, configuration and behavior.
>Additionally, these components are now tightly coupled, so there is no
> way
>these two changes can be implemented in parallel and then merged
> together
>easily. So what we will end up with is having to implement these changes
>sequentially, fixing all existing tests twice, and essentially throwing
>away half of the work done because the other part of the change is
>re-implemented
>- Similar example goes with getting rid of IgniteInternalFuture and
>replacing it with CompletableFuture, and any other change that touches
> the
>asynchronous part of the code.
>
> Third, I do not see how this choice affects the UX of Ignite. The end user
> experience must be fixed in the IEP regardless of the development process
> and the fact that we have gaps in this area in Ignite 2.x just confirms
> that.
>
> Pavel, agree that a repo/branch is a technicality, I guess if reformulate,
> my point is that we might agree to have a single development master branch
> with 'disassembled' end-to-end functionality for some period of time to
> speed up development, and re-assemble the core features after having
> submodules tested independently.
>
> Nikolay,
>   >We have many features that have to evolve.
>   >Snapshots, rebalance, tooling, tracing, zookeeper support, etc.
> This is not very specific. In the end, resources are limited and we will
> not be able to drive both tracks simultaneously, especially after a couple
> of features having been implemented for Ignite 3.0. If there are indeed
> some major changes that we want to do in Ignite 2.x instead of putting
> effort into 3.0 - let's discuss them. I am just not aware of any, that's
> why I am eager to move forward with Ignite 3.0.
>
> >We have bugs and issues that can be fixed in 2.x without breaking backward
> compatibility.
> >We have many users who are happy with the 2.x with all it’s issues.
> These changes will be covered by end-to-end tests and migrated to Ignite
> 3.0, so I see no issues here.
>
> Finally, Anton & Nikolay
> I do not have an estimate for this simply because the activity is
> community-driven and it depends on the number of people willing to
> contribute. With the current pace, I would hope to have an RC of Ignite 3.0
> to be ready by the end of 2021. My gut feeling is that by moving with
> incremental changes, we will not be able to implement even half of the
> wishlist by that time.
> I doubt that releasing several major releases with breaking changes will
> make Ignite users happy either because each 

Re: [DISCUSS] Ignite 3.0 development approach

[Alexey Goncharuk]

Nikolay, Pavel,

Thanks for the feedback! First of all, I wanted to stress that I do not
intend to rewrite everything from scratch (I never used this phrase). There
are significant parts of code that would be moved with minimal
modifications. Second, I never said that we will get rid of the old tests
codebase. Every single test from Ignite 2.x should be moved to Ignite 3
regardless of how we choose to proceed.

My point is that for some parts of the code a clean bottom-up
implementation will be cheaper in many ways. Let me give you a few concrete
examples:

   - I think no one can object that we need a cleanly separated persistence
   layer for Ignite. There is a very raw draft IEP for this already. On the
   other hand, I think we also can agree that we need a split-brain resistant
   replication protocol for caches. There is also an IEP for this. Neither of
   the changes is a good fit for 2.x because they are likely to introduce
   breaking changes in the persistence layer, configuration and behavior.
   Additionally, these components are now tightly coupled, so there is no way
   these two changes can be implemented in parallel and then merged together
   easily. So what we will end up with is having to implement these changes
   sequentially, fixing all existing tests twice, and essentially throwing
   away half of the work done because the other part of the change is
   re-implemented
   - Similar example goes with getting rid of IgniteInternalFuture and
   replacing it with CompletableFuture, and any other change that touches the
   asynchronous part of the code.

Third, I do not see how this choice affects the UX of Ignite. The end user
experience must be fixed in the IEP regardless of the development process
and the fact that we have gaps in this area in Ignite 2.x just confirms
that.

Pavel, agree that a repo/branch is a technicality, I guess if reformulate,
my point is that we might agree to have a single development master branch
with 'disassembled' end-to-end functionality for some period of time to
speed up development, and re-assemble the core features after having
submodules tested independently.

Nikolay,
  >We have many features that have to evolve.
  >Snapshots, rebalance, tooling, tracing, zookeeper support, etc.
This is not very specific. In the end, resources are limited and we will
not be able to drive both tracks simultaneously, especially after a couple
of features having been implemented for Ignite 3.0. If there are indeed
some major changes that we want to do in Ignite 2.x instead of putting
effort into 3.0 - let's discuss them. I am just not aware of any, that's
why I am eager to move forward with Ignite 3.0.

>We have bugs and issues that can be fixed in 2.x without breaking backward
compatibility.
>We have many users who are happy with the 2.x with all it’s issues.
These changes will be covered by end-to-end tests and migrated to Ignite
3.0, so I see no issues here.

Finally, Anton & Nikolay
I do not have an estimate for this simply because the activity is
community-driven and it depends on the number of people willing to
contribute. With the current pace, I would hope to have an RC of Ignite 3.0
to be ready by the end of 2021. My gut feeling is that by moving with
incremental changes, we will not be able to implement even half of the
wishlist by that time.
I doubt that releasing several major releases with breaking changes will
make Ignite users happy either because each upgrade will cost Ignite users
money, so the fewer major versions we release, the better. Thus my wish to
include all breaking changes in one release.

I'll be now quiet for a while, let's see what other community members think.

пн, 2 нояб. 2020 г. в 15:52, Pavel Tupitsyn :

> 1. Rewriting from scratch is never a good idea.
> We don't want to follow the path of Netscape and lose all our users
> by the time we have a working 3.0 [1]
>
> 2. Not sure about new repo - seems like some pain and no gain, what's the
> problem with a branch?
>
> 3. We should keep existing integration tests when possible.
> We have accumulated a lot of edge case knowledge over the years,
> it is not a good idea to send all of that down the drain.
> Yes, integration tests are slow, but they are the most valuable.
> I think we can move more stuff into nightly runs and have a fast and modern
> basic suite.
>
>
> Alexey, you are much more familiar with the Ignite core codebase than most
> of us,
> can you please explain in more detail which particular feature, in your
> opinion,
> mandates this "start from scratch" approach?
> Is it really not possible at all to follow a less radical way?
>
>
> [1]
>
> https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/
>
> On Mon, Nov 2, 2020 at 2:25 PM Nikolay Izhikov 
> wrote:
>
> > Hello, Alexey.
> >
> > I think that «rewriting from scratch» approach has a high risk to make
> new
> > features unusable.
> > At the time Ignite2 was started no-one wants to do bad UX or bad
> 

Re: Client App Object Allocation Rate

[Ilya Kasnacheev]

Hello!

Okay, that's not very cool. I hope to get some response from development
side at this point.

Sans reaction, I will file a ticket.

Regards,
-- 
Ilya Kasnacheev


пн, 2 нояб. 2020 г. в 15:03, ssansoy :

> Apologies I may have spoken to soon (I was looking at the wrong process).
>
> It looks like we can't turn EVT_NODE_METRICS_UPDATED off as it is
> designated
> as an internal event GridEventStorageManager.disableEvents line 441 (ignite
> 2.8.1), this checks to see if the event that is being disabled is part of
> EVTS_DISCOVERY_ALL, which it is, so this isn't set to false...
>
> Are there any workarounds to this?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: [DISCUSS] Ignite 3.0 development approach

[Pavel Tupitsyn]

1. Rewriting from scratch is never a good idea.
We don't want to follow the path of Netscape and lose all our users
by the time we have a working 3.0 [1]

2. Not sure about new repo - seems like some pain and no gain, what's the
problem with a branch?

3. We should keep existing integration tests when possible.
We have accumulated a lot of edge case knowledge over the years,
it is not a good idea to send all of that down the drain.
Yes, integration tests are slow, but they are the most valuable.
I think we can move more stuff into nightly runs and have a fast and modern
basic suite.


Alexey, you are much more familiar with the Ignite core codebase than most
of us,
can you please explain in more detail which particular feature, in your
opinion,
mandates this "start from scratch" approach?
Is it really not possible at all to follow a less radical way?


[1]
https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/

On Mon, Nov 2, 2020 at 2:25 PM Nikolay Izhikov  wrote:

> Hello, Alexey.
>
> I think that «rewriting from scratch» approach has a high risk to make new
> features unusable.
> At the time Ignite2 was started no-one wants to do bad UX or bad features.
> Nevertheless, it happen.
>
> I think we can avoid it with the Ignite3 and successors if we will move
> step by step without keeping backward compatibility
> With the step by step approach, we can focus on each component separately.
>
> > What new features are we planning to implement for Ignite 2.x?
>
> We have many features that have to evolve.
> Snapshots, rebalance, tooling, tracing, zookeeper support, etc.
> We have bugs and issues that can be fixed in 2.x without breaking backward
> compatibility.
> We have many users who are happy with the 2.x with all it’s issues.
>
> > 2 нояб. 2020 г., в 14:09, Anton Vinogradov  написал(а):
> >
> > Alexey,
> >
> > Do we have any estimates of how fast we'll be able to gain
> production-ready
> > AI 3.0 in case of a "new repo" choice?
> >
> > On Mon, Nov 2, 2020 at 2:01 PM Alexey Goncharuk <
> alexey.goncha...@gmail.com>
> > wrote:
> >
> >> Nikolay,
> >>
> >> What new features are we planning to implement for Ignite 2.x? I think
> once
> >> we commence working on Ignite 3.0, we should gradually cease the
> activity
> >> on Ignite 2.x to mere bugfixes because such parallel development will be
> >> overwhelming regardless of how we choose to proceed.
> >>
> >> пн, 2 нояб. 2020 г. в 13:38, Nikolay Izhikov :
> >>
> >>> To be clear:
> >>>
>  I would suggest creating a new repository for Ignite 3.0 (perhaps, a
> >> new
> >>> clean branch, but a new repo looks nicer to me) and a new Ignite 3.0
> >>> TeamCity project.
> >>>
> >>> +1 for new Team City project.
> >>> +1 for new branch for Ignite3.
> >>> -1 for new repo.
> >>>
> >>>
>  2 нояб. 2020 г., в 13:35, Nikolay Izhikov 
> >>> написал(а):
> 
>  Hello, Alexey.
> 
>  I think it will hurt our project more than help.
>  Developing new features for 2 separate branches with the different
> APIs
> >>> and internal structure is overwhelming
> 
>  Maybe we should relax a bit requirements for Ignite3?
>  Maybe we should move step by step and make Ignite3 with new
> >>> configuration than Ignite4 with new transactions, etc?
> 
> > 2 нояб. 2020 г., в 13:14, Alexey Goncharuk <
> >> alexey.goncha...@gmail.com>
> >>> написал(а):
> >
> > Igniters,
> >
> > I wanted to pitch a rather radical idea regarding the Ignite 3.0
> > development which has occurred to me some time ago.
> >
> > We already have several IEPs targeted to Ignite 3.0 which imply major
> > changes to the codebase (the change in replication protocol and thus
> > transactions, change in binary format, updated metastorage, etc). We
> >>> also
> > planned significant changes in public APIs: configuration format
> >> change,
> > improvements in cache APIs, SQL APIs, transaction mode rework. The
> >>> wishlist
> > of changes for Ignite 3.0 is huge.
> >
> > So, I was wondering whether it makes sense to try to change the old
> > codebase, or start with a new baseline and move old pieces of code
> >> that
> >>> do
> > not require significant rework. Personally, I would go with the
> second
> > option for the following reasons:
> >
> > - We have a chance to shift the development paradigm in the project
> >> and
> > introduce the practice of true unit-tests. In the new baseline at the
> > beginning there will be no ability to run an end-to-end scenario,
> >> thus
> >>> we
> > will be forced to write unit-tests. So far, such practice was hard to
> > implement because of tight coupling between Ignite components and
> >>> inability
> > to instantiate components without an instance of KernalContext. For
> > example, we should be able to thoroughly test internal primitives,
> >>> such as
> > replication protocol (without actual communication), distributed
> 

[jira] [Created] (IGNITE-13653) Don't print warning if unordered map used for bulk update operation on atomic cache

[Aleksey Plekhanov (Jira)]

Aleksey Plekhanov created IGNITE-13653:
--

 Summary: Don't print warning if unordered map used for bulk update 
operation on atomic cache
 Key: IGNITE-13653
 URL: https://issues.apache.org/jira/browse/IGNITE-13653
 Project: Ignite
  Issue Type: Improvement
 Environment: Since IGNITE-12451 is resolved there no more deadlock 
possible for atomic caches and it's safe to use HashMap and other unordered 
maps as argument of putAll/removeAll/invokeAll operations on atomic caches. 
Warning, introduced by IGNITE-6804, is not required anymore.
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Ignite 3.0 development approach

[Nikolay Izhikov]

Hello, Alexey.

I think that «rewriting from scratch» approach has a high risk to make new 
features unusable.
At the time Ignite2 was started no-one wants to do bad UX or bad features. 
Nevertheless, it happen.

I think we can avoid it with the Ignite3 and successors if we will move step by 
step without keeping backward compatibility
With the step by step approach, we can focus on each component separately.

> What new features are we planning to implement for Ignite 2.x? 

We have many features that have to evolve.
Snapshots, rebalance, tooling, tracing, zookeeper support, etc.
We have bugs and issues that can be fixed in 2.x without breaking backward 
compatibility.
We have many users who are happy with the 2.x with all it’s issues.

> 2 нояб. 2020 г., в 14:09, Anton Vinogradov  написал(а):
> 
> Alexey,
> 
> Do we have any estimates of how fast we'll be able to gain production-ready
> AI 3.0 in case of a "new repo" choice?
> 
> On Mon, Nov 2, 2020 at 2:01 PM Alexey Goncharuk 
> wrote:
> 
>> Nikolay,
>> 
>> What new features are we planning to implement for Ignite 2.x? I think once
>> we commence working on Ignite 3.0, we should gradually cease the activity
>> on Ignite 2.x to mere bugfixes because such parallel development will be
>> overwhelming regardless of how we choose to proceed.
>> 
>> пн, 2 нояб. 2020 г. в 13:38, Nikolay Izhikov :
>> 
>>> To be clear:
>>> 
 I would suggest creating a new repository for Ignite 3.0 (perhaps, a
>> new
>>> clean branch, but a new repo looks nicer to me) and a new Ignite 3.0
>>> TeamCity project.
>>> 
>>> +1 for new Team City project.
>>> +1 for new branch for Ignite3.
>>> -1 for new repo.
>>> 
>>> 
 2 нояб. 2020 г., в 13:35, Nikolay Izhikov 
>>> написал(а):
 
 Hello, Alexey.
 
 I think it will hurt our project more than help.
 Developing new features for 2 separate branches with the different APIs
>>> and internal structure is overwhelming
 
 Maybe we should relax a bit requirements for Ignite3?
 Maybe we should move step by step and make Ignite3 with new
>>> configuration than Ignite4 with new transactions, etc?
 
> 2 нояб. 2020 г., в 13:14, Alexey Goncharuk <
>> alexey.goncha...@gmail.com>
>>> написал(а):
> 
> Igniters,
> 
> I wanted to pitch a rather radical idea regarding the Ignite 3.0
> development which has occurred to me some time ago.
> 
> We already have several IEPs targeted to Ignite 3.0 which imply major
> changes to the codebase (the change in replication protocol and thus
> transactions, change in binary format, updated metastorage, etc). We
>>> also
> planned significant changes in public APIs: configuration format
>> change,
> improvements in cache APIs, SQL APIs, transaction mode rework. The
>>> wishlist
> of changes for Ignite 3.0 is huge.
> 
> So, I was wondering whether it makes sense to try to change the old
> codebase, or start with a new baseline and move old pieces of code
>> that
>>> do
> not require significant rework. Personally, I would go with the second
> option for the following reasons:
> 
> - We have a chance to shift the development paradigm in the project
>> and
> introduce the practice of true unit-tests. In the new baseline at the
> beginning there will be no ability to run an end-to-end scenario,
>> thus
>>> we
> will be forced to write unit-tests. So far, such practice was hard to
> implement because of tight coupling between Ignite components and
>>> inability
> to instantiate components without an instance of KernalContext. For
> example, we should be able to thoroughly test internal primitives,
>>> such as
> replication protocol (without actual communication), distributed
> metastorage contracts, persistence layer, etc.
> - We will significantly reduce the development cycle in the beginning
> (right now the RunAll takes two hours of astronomical time with empty
>>> TC;
> in the new approach developer will be able to run ALL tests locally
>> in
>>> a
> matter of minutes)
> - We can get rid of TC bot and enforce green TC by integrating TC
>> build
> results with GitHub PRs (the same way Travis is currently integrated
>>> to PR
> check). We should restrict PR merge without a TC check
> - We will still have to re-write all tests, but only once. If we try
>> to
> modify the old codebase, we would need to modify all the tests for
>>> every
> major change (public API change, configuration change)
> - We will have fewer conflicts when working together. For example, I
> cannot imagine how one would merge two changes of getting rid of
> IgniteFuture and changes in replication protocol, for example
> 
> Technically, I would suggest creating a new repository for Ignite 3.0
> (perhaps, a new clean branch, but a new repo looks nicer to me) and a
>>> new
> Ignite 3.0 TeamCity project.
> 
> While it may seem 

Re: [DISCUSS] Ignite 3.0 development approach

[Anton Vinogradov]

Alexey,

Do we have any estimates of how fast we'll be able to gain production-ready
AI 3.0 in case of a "new repo" choice?

On Mon, Nov 2, 2020 at 2:01 PM Alexey Goncharuk 
wrote:

> Nikolay,
>
> What new features are we planning to implement for Ignite 2.x? I think once
> we commence working on Ignite 3.0, we should gradually cease the activity
> on Ignite 2.x to mere bugfixes because such parallel development will be
> overwhelming regardless of how we choose to proceed.
>
> пн, 2 нояб. 2020 г. в 13:38, Nikolay Izhikov :
>
> > To be clear:
> >
> > > I would suggest creating a new repository for Ignite 3.0 (perhaps, a
> new
> > clean branch, but a new repo looks nicer to me) and a new Ignite 3.0
> > TeamCity project.
> >
> > +1 for new Team City project.
> > +1 for new branch for Ignite3.
> > -1 for new repo.
> >
> >
> > > 2 нояб. 2020 г., в 13:35, Nikolay Izhikov 
> > написал(а):
> > >
> > > Hello, Alexey.
> > >
> > > I think it will hurt our project more than help.
> > > Developing new features for 2 separate branches with the different APIs
> > and internal structure is overwhelming
> > >
> > > Maybe we should relax a bit requirements for Ignite3?
> > > Maybe we should move step by step and make Ignite3 with new
> > configuration than Ignite4 with new transactions, etc?
> > >
> > >> 2 нояб. 2020 г., в 13:14, Alexey Goncharuk <
> alexey.goncha...@gmail.com>
> > написал(а):
> > >>
> > >> Igniters,
> > >>
> > >> I wanted to pitch a rather radical idea regarding the Ignite 3.0
> > >> development which has occurred to me some time ago.
> > >>
> > >> We already have several IEPs targeted to Ignite 3.0 which imply major
> > >> changes to the codebase (the change in replication protocol and thus
> > >> transactions, change in binary format, updated metastorage, etc). We
> > also
> > >> planned significant changes in public APIs: configuration format
> change,
> > >> improvements in cache APIs, SQL APIs, transaction mode rework. The
> > wishlist
> > >> of changes for Ignite 3.0 is huge.
> > >>
> > >> So, I was wondering whether it makes sense to try to change the old
> > >> codebase, or start with a new baseline and move old pieces of code
> that
> > do
> > >> not require significant rework. Personally, I would go with the second
> > >> option for the following reasons:
> > >>
> > >>  - We have a chance to shift the development paradigm in the project
> and
> > >>  introduce the practice of true unit-tests. In the new baseline at the
> > >>  beginning there will be no ability to run an end-to-end scenario,
> thus
> > we
> > >>  will be forced to write unit-tests. So far, such practice was hard to
> > >>  implement because of tight coupling between Ignite components and
> > inability
> > >>  to instantiate components without an instance of KernalContext. For
> > >>  example, we should be able to thoroughly test internal primitives,
> > such as
> > >>  replication protocol (without actual communication), distributed
> > >>  metastorage contracts, persistence layer, etc.
> > >>  - We will significantly reduce the development cycle in the beginning
> > >>  (right now the RunAll takes two hours of astronomical time with empty
> > TC;
> > >>  in the new approach developer will be able to run ALL tests locally
> in
> > a
> > >>  matter of minutes)
> > >>  - We can get rid of TC bot and enforce green TC by integrating TC
> build
> > >>  results with GitHub PRs (the same way Travis is currently integrated
> > to PR
> > >>  check). We should restrict PR merge without a TC check
> > >>  - We will still have to re-write all tests, but only once. If we try
> to
> > >>  modify the old codebase, we would need to modify all the tests for
> > every
> > >>  major change (public API change, configuration change)
> > >>  - We will have fewer conflicts when working together. For example, I
> > >>  cannot imagine how one would merge two changes of getting rid of
> > >>  IgniteFuture and changes in replication protocol, for example
> > >>
> > >> Technically, I would suggest creating a new repository for Ignite 3.0
> > >> (perhaps, a new clean branch, but a new repo looks nicer to me) and a
> > new
> > >> Ignite 3.0 TeamCity project.
> > >>
> > >> While it may seem quite radical, I do believe that this approach will
> > give
> > >> us more benefits than trying to make such major changes in the
> existing
> > >> codebase. If needed, let's schedule a discord chat like before to
> > discuss
> > >> this.
> > >>
> > >> WDYT?
> > >
> >
> >
>

Re: [DISCUSS] Ignite 3.0 development approach

[Alexey Goncharuk]

Nikolay,

What new features are we planning to implement for Ignite 2.x? I think once
we commence working on Ignite 3.0, we should gradually cease the activity
on Ignite 2.x to mere bugfixes because such parallel development will be
overwhelming regardless of how we choose to proceed.

пн, 2 нояб. 2020 г. в 13:38, Nikolay Izhikov :

> To be clear:
>
> > I would suggest creating a new repository for Ignite 3.0 (perhaps, a new
> clean branch, but a new repo looks nicer to me) and a new Ignite 3.0
> TeamCity project.
>
> +1 for new Team City project.
> +1 for new branch for Ignite3.
> -1 for new repo.
>
>
> > 2 нояб. 2020 г., в 13:35, Nikolay Izhikov 
> написал(а):
> >
> > Hello, Alexey.
> >
> > I think it will hurt our project more than help.
> > Developing new features for 2 separate branches with the different APIs
> and internal structure is overwhelming
> >
> > Maybe we should relax a bit requirements for Ignite3?
> > Maybe we should move step by step and make Ignite3 with new
> configuration than Ignite4 with new transactions, etc?
> >
> >> 2 нояб. 2020 г., в 13:14, Alexey Goncharuk 
> написал(а):
> >>
> >> Igniters,
> >>
> >> I wanted to pitch a rather radical idea regarding the Ignite 3.0
> >> development which has occurred to me some time ago.
> >>
> >> We already have several IEPs targeted to Ignite 3.0 which imply major
> >> changes to the codebase (the change in replication protocol and thus
> >> transactions, change in binary format, updated metastorage, etc). We
> also
> >> planned significant changes in public APIs: configuration format change,
> >> improvements in cache APIs, SQL APIs, transaction mode rework. The
> wishlist
> >> of changes for Ignite 3.0 is huge.
> >>
> >> So, I was wondering whether it makes sense to try to change the old
> >> codebase, or start with a new baseline and move old pieces of code that
> do
> >> not require significant rework. Personally, I would go with the second
> >> option for the following reasons:
> >>
> >>  - We have a chance to shift the development paradigm in the project and
> >>  introduce the practice of true unit-tests. In the new baseline at the
> >>  beginning there will be no ability to run an end-to-end scenario, thus
> we
> >>  will be forced to write unit-tests. So far, such practice was hard to
> >>  implement because of tight coupling between Ignite components and
> inability
> >>  to instantiate components without an instance of KernalContext. For
> >>  example, we should be able to thoroughly test internal primitives,
> such as
> >>  replication protocol (without actual communication), distributed
> >>  metastorage contracts, persistence layer, etc.
> >>  - We will significantly reduce the development cycle in the beginning
> >>  (right now the RunAll takes two hours of astronomical time with empty
> TC;
> >>  in the new approach developer will be able to run ALL tests locally in
> a
> >>  matter of minutes)
> >>  - We can get rid of TC bot and enforce green TC by integrating TC build
> >>  results with GitHub PRs (the same way Travis is currently integrated
> to PR
> >>  check). We should restrict PR merge without a TC check
> >>  - We will still have to re-write all tests, but only once. If we try to
> >>  modify the old codebase, we would need to modify all the tests for
> every
> >>  major change (public API change, configuration change)
> >>  - We will have fewer conflicts when working together. For example, I
> >>  cannot imagine how one would merge two changes of getting rid of
> >>  IgniteFuture and changes in replication protocol, for example
> >>
> >> Technically, I would suggest creating a new repository for Ignite 3.0
> >> (perhaps, a new clean branch, but a new repo looks nicer to me) and a
> new
> >> Ignite 3.0 TeamCity project.
> >>
> >> While it may seem quite radical, I do believe that this approach will
> give
> >> us more benefits than trying to make such major changes in the existing
> >> codebase. If needed, let's schedule a discord chat like before to
> discuss
> >> this.
> >>
> >> WDYT?
> >
>
>

Re: [DISCUSS] Ignite 3.0 development approach

[Nikolay Izhikov]

To be clear: 

> I would suggest creating a new repository for Ignite 3.0 (perhaps, a new 
> clean branch, but a new repo looks nicer to me) and a new Ignite 3.0 TeamCity 
> project.

+1 for new Team City project.
+1 for new branch for Ignite3.
-1 for new repo.


> 2 нояб. 2020 г., в 13:35, Nikolay Izhikov  написал(а):
> 
> Hello, Alexey.
> 
> I think it will hurt our project more than help.
> Developing new features for 2 separate branches with the different APIs and 
> internal structure is overwhelming
> 
> Maybe we should relax a bit requirements for Ignite3?
> Maybe we should move step by step and make Ignite3 with new configuration 
> than Ignite4 with new transactions, etc?
> 
>> 2 нояб. 2020 г., в 13:14, Alexey Goncharuk  
>> написал(а):
>> 
>> Igniters,
>> 
>> I wanted to pitch a rather radical idea regarding the Ignite 3.0
>> development which has occurred to me some time ago.
>> 
>> We already have several IEPs targeted to Ignite 3.0 which imply major
>> changes to the codebase (the change in replication protocol and thus
>> transactions, change in binary format, updated metastorage, etc). We also
>> planned significant changes in public APIs: configuration format change,
>> improvements in cache APIs, SQL APIs, transaction mode rework. The wishlist
>> of changes for Ignite 3.0 is huge.
>> 
>> So, I was wondering whether it makes sense to try to change the old
>> codebase, or start with a new baseline and move old pieces of code that do
>> not require significant rework. Personally, I would go with the second
>> option for the following reasons:
>> 
>>  - We have a chance to shift the development paradigm in the project and
>>  introduce the practice of true unit-tests. In the new baseline at the
>>  beginning there will be no ability to run an end-to-end scenario, thus we
>>  will be forced to write unit-tests. So far, such practice was hard to
>>  implement because of tight coupling between Ignite components and inability
>>  to instantiate components without an instance of KernalContext. For
>>  example, we should be able to thoroughly test internal primitives, such as
>>  replication protocol (without actual communication), distributed
>>  metastorage contracts, persistence layer, etc.
>>  - We will significantly reduce the development cycle in the beginning
>>  (right now the RunAll takes two hours of astronomical time with empty TC;
>>  in the new approach developer will be able to run ALL tests locally in a
>>  matter of minutes)
>>  - We can get rid of TC bot and enforce green TC by integrating TC build
>>  results with GitHub PRs (the same way Travis is currently integrated to PR
>>  check). We should restrict PR merge without a TC check
>>  - We will still have to re-write all tests, but only once. If we try to
>>  modify the old codebase, we would need to modify all the tests for every
>>  major change (public API change, configuration change)
>>  - We will have fewer conflicts when working together. For example, I
>>  cannot imagine how one would merge two changes of getting rid of
>>  IgniteFuture and changes in replication protocol, for example
>> 
>> Technically, I would suggest creating a new repository for Ignite 3.0
>> (perhaps, a new clean branch, but a new repo looks nicer to me) and a new
>> Ignite 3.0 TeamCity project.
>> 
>> While it may seem quite radical, I do believe that this approach will give
>> us more benefits than trying to make such major changes in the existing
>> codebase. If needed, let's schedule a discord chat like before to discuss
>> this.
>> 
>> WDYT?
> 

Re: [DISCUSS] Ignite 3.0 development approach

[Nikolay Izhikov]

Hello, Alexey.

I think it will hurt our project more than help.
Developing new features for 2 separate branches with the different APIs and 
internal structure is overwhelming

Maybe we should relax a bit requirements for Ignite3?
Maybe we should move step by step and make Ignite3 with new configuration than 
Ignite4 with new transactions, etc?

> 2 нояб. 2020 г., в 13:14, Alexey Goncharuk  
> написал(а):
> 
> Igniters,
> 
> I wanted to pitch a rather radical idea regarding the Ignite 3.0
> development which has occurred to me some time ago.
> 
> We already have several IEPs targeted to Ignite 3.0 which imply major
> changes to the codebase (the change in replication protocol and thus
> transactions, change in binary format, updated metastorage, etc). We also
> planned significant changes in public APIs: configuration format change,
> improvements in cache APIs, SQL APIs, transaction mode rework. The wishlist
> of changes for Ignite 3.0 is huge.
> 
> So, I was wondering whether it makes sense to try to change the old
> codebase, or start with a new baseline and move old pieces of code that do
> not require significant rework. Personally, I would go with the second
> option for the following reasons:
> 
>   - We have a chance to shift the development paradigm in the project and
>   introduce the practice of true unit-tests. In the new baseline at the
>   beginning there will be no ability to run an end-to-end scenario, thus we
>   will be forced to write unit-tests. So far, such practice was hard to
>   implement because of tight coupling between Ignite components and inability
>   to instantiate components without an instance of KernalContext. For
>   example, we should be able to thoroughly test internal primitives, such as
>   replication protocol (without actual communication), distributed
>   metastorage contracts, persistence layer, etc.
>   - We will significantly reduce the development cycle in the beginning
>   (right now the RunAll takes two hours of astronomical time with empty TC;
>   in the new approach developer will be able to run ALL tests locally in a
>   matter of minutes)
>   - We can get rid of TC bot and enforce green TC by integrating TC build
>   results with GitHub PRs (the same way Travis is currently integrated to PR
>   check). We should restrict PR merge without a TC check
>   - We will still have to re-write all tests, but only once. If we try to
>   modify the old codebase, we would need to modify all the tests for every
>   major change (public API change, configuration change)
>   - We will have fewer conflicts when working together. For example, I
>   cannot imagine how one would merge two changes of getting rid of
>   IgniteFuture and changes in replication protocol, for example
> 
> Technically, I would suggest creating a new repository for Ignite 3.0
> (perhaps, a new clean branch, but a new repo looks nicer to me) and a new
> Ignite 3.0 TeamCity project.
> 
> While it may seem quite radical, I do believe that this approach will give
> us more benefits than trying to make such major changes in the existing
> codebase. If needed, let's schedule a discord chat like before to discuss
> this.
> 
> WDYT?

[DISCUSS] Ignite 3.0 development approach

[Alexey Goncharuk]

Igniters,

I wanted to pitch a rather radical idea regarding the Ignite 3.0
development which has occurred to me some time ago.

We already have several IEPs targeted to Ignite 3.0 which imply major
changes to the codebase (the change in replication protocol and thus
transactions, change in binary format, updated metastorage, etc). We also
planned significant changes in public APIs: configuration format change,
improvements in cache APIs, SQL APIs, transaction mode rework. The wishlist
of changes for Ignite 3.0 is huge.

So, I was wondering whether it makes sense to try to change the old
codebase, or start with a new baseline and move old pieces of code that do
not require significant rework. Personally, I would go with the second
option for the following reasons:

   - We have a chance to shift the development paradigm in the project and
   introduce the practice of true unit-tests. In the new baseline at the
   beginning there will be no ability to run an end-to-end scenario, thus we
   will be forced to write unit-tests. So far, such practice was hard to
   implement because of tight coupling between Ignite components and inability
   to instantiate components without an instance of KernalContext. For
   example, we should be able to thoroughly test internal primitives, such as
   replication protocol (without actual communication), distributed
   metastorage contracts, persistence layer, etc.
   - We will significantly reduce the development cycle in the beginning
   (right now the RunAll takes two hours of astronomical time with empty TC;
   in the new approach developer will be able to run ALL tests locally in a
   matter of minutes)
   - We can get rid of TC bot and enforce green TC by integrating TC build
   results with GitHub PRs (the same way Travis is currently integrated to PR
   check). We should restrict PR merge without a TC check
   - We will still have to re-write all tests, but only once. If we try to
   modify the old codebase, we would need to modify all the tests for every
   major change (public API change, configuration change)
   - We will have fewer conflicts when working together. For example, I
   cannot imagine how one would merge two changes of getting rid of
   IgniteFuture and changes in replication protocol, for example

Technically, I would suggest creating a new repository for Ignite 3.0
(perhaps, a new clean branch, but a new repo looks nicer to me) and a new
Ignite 3.0 TeamCity project.

While it may seem quite radical, I do believe that this approach will give
us more benefits than trying to make such major changes in the existing
codebase. If needed, let's schedule a discord chat like before to discuss
this.

WDYT?

[jira] [Created] (IGNITE-13652) Wrong GitHub link for Apache Ignite With Spring Data/Example

[Denis Garus (Jira)]

Denis Garus created IGNITE-13652:


 Summary: Wrong GitHub link for Apache Ignite With Spring 
Data/Example
 Key: IGNITE-13652
 URL: https://issues.apache.org/jira/browse/IGNITE-13652
 Project: Ignite
  Issue Type: Bug
  Components: documentation
Affects Versions: 2.9
Reporter: Denis Garus


Wrong GihHub link in
https://ignite.apache.org/docs/latest/extensions-and-integrations/spring/spring-data#example



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Missed (non-suited) tests

[Anton Vinogradov]

> As I understand we
> can't just move suites between modules, as TeamCity may depend on the path
> to them.
See no problem to update TC as well.

On Fri, Oct 30, 2020 at 4:32 PM Ivan Daschinsky  wrote:

> I suggests to mark these tests with @Ignore and file tickets to fix them.
>
> пт, 30 окт. 2020 г. в 16:26, Ivan Daschinsky :
>
> > Hi
> >
> > WalCompactionAfterRestartTest -- yes we need it. This test failed because
> > of race (test shold be rewritten a little bit)
> >
> > пт, 30 окт. 2020 г. в 16:15, Max Timonin :
> >
> >> Hi!
> >>
> >> Yes, you're correct. I've developed the check and started to clean tests
> >> (move them to suites, mark some tests with Ignore, etc.). I finish work
> on
> >> the core module. I hope it was the biggest one, and others are less. If
> >> so,
> >> I think I will finish the work on other modules in 1 or 2 weeks, as I do
> >> this activity in the background (~10% of my work time). Actually I've
> >> found
> >> 3 failed tests in the core module that aren't in any suite, so I need
> time
> >> to discover reason of failures and if we actually need those tests:
> >>
> >> GridCacheMultithreadedFailoverAbstractTest
> >> WalCompactionAfterRestartTest
> >> GridTcpCommunicationSpiLogTest
> >>
> >> Also we should decide how to be with wrongly located es. As I understand
> >> we
> >> can't just move suites between modules, as TeamCity may depend on the
> path
> >> to them. So, for such cases I've just created suites in the right
> module,
> >> and replaced the test list with the new class suite. It does not look
> >> pretty enough, but I think It's a path of least resistance. WDYT?
> >>
> >> BEFORE:
> >> Module A -> SuiteA -> testA1, testA2, testB1, testB2
> >> Module B -> testB1, testB2
> >>
> >> AFTER:
> >> Module A -> SuiteA, SuiteB
> >> Module B -> SuiteB -> testB1, testB2
> >>
> >> On Fri, Oct 30, 2020 at 3:38 PM Anton Vinogradov  wrote:
> >>
> >> > Folks,
> >> > What's the current state of this thread?
> >> > AFAIU, we found unused and wrongly located tests and developed some
> >> > checker, could we split this to some PRs?
> >> > Let's merge tests usage fix and location fixes first, this will
> provide
> >> us
> >> > an ability to automate check using Travis.
> >> >
> >> > On Tue, Oct 20, 2020 at 12:06 PM Ivan Pavlukhin 
> >> > wrote:
> >> >
> >> > > Max, Ivan,
> >> > >
> >> > > Using explicit @Ignore and the automated check sounds good to me. If
> >> > > nobody has arguments against it I think we should do it.
> >> > >
> >> > > 2020-10-19 19:30 GMT+03:00, Max Timonin :
> >> > > > Hi Ivan,
> >> > > >
> >> > > > I've checked the ticket you provide. It contains subtasks to
> >> uncomment
> >> > or
> >> > > > to remove some unused tests. It definitely describes some cases
> I've
> >> > > found.
> >> > > > So what do you think if I uncomment them in suites, add @Ignore
> >> > > annotation
> >> > > > for those tests while the tickets are open? This will help to find
> >> out
> >> > > > tests that were forgiven in a recent time.
> >> > > >
> >> > > > Also I believe that this check must be automated. I didn't find a
> >> way
> >> > how
> >> > > > uncomment / unused tests are found in the ticket. If there is no
> >> any -
> >> > I
> >> > > > propose mine PR for this purpose.
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Mon, Oct 19, 2020 at 5:24 PM Ivan Daschinsky <
> >> ivanda...@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > >> Ivan, as far as I understand, Max also created verification check
> >> for
> >> > > not
> >> > > >> included test and found a few tests, that have never been
> included
> >> in
> >> > > any
> >> > > >> testsuites.
> >> > > >>
> >> > > >> Also, I suppose, that even if we cannot run some tests, these
> tests
> >> > > >> should
> >> > > >> be ignored using annotation, but not commented.
> >> > > >>
> >> > > >> пн, 19 окт. 2020 г. в 16:33, Ivan Pavlukhin  >:
> >> > > >>
> >> > > >> > Hi Max,
> >> > > >> >
> >> > > >> > There is an existing effort about "abandoned" tests
> >> > > >> > https://issues.apache.org/jira/browse/IGNITE-9210
> >> > > >> >
> >> > > >> > 2020-10-19 16:25 GMT+03:00, Max Timonin <
> timonin.ma...@gmail.com
> >> >:
> >> > > >> > > Hi Igniters!
> >> > > >> > >
> >> > > >> > > I made a research into tests that aren't included in any test
> >> > suite.
> >> > > >> > > As
> >> > > >> > > TeamCity runs tests by suites so there could be tests that
> >> never
> >> > run
> >> > > >> > > on
> >> > > >> > TC.
> >> > > >> > > So I tried implementing a simple check for such tests and
> >> include
> >> > it
> >> > > >> > > in
> >> > > >> > > Ignite's travis config.
> >> > > >> > >
> >> > > >> > > The check runs while "mvn test" command and piggy-backs on
> the
> >> > maven
> >> > > >> > > surefire plugin. I replaced the junit provider with a custom
> >> one
> >> > > that
> >> > > >> > > checks if a class is a test or a suite (there are some Ignite
> >> > > >> > > specific
> >> > > >> > > stuff), marks tests that are in suites and raises an
> exception
> 

[jira] [Created] (IGNITE-13651) Ignite Docs: Port Apache Zeppelin docs from readme.io

[YuJue Li (Jira)]

YuJue Li created IGNITE-13651:
-

 Summary: Ignite Docs: Port Apache Zeppelin docs from readme.io
 Key: IGNITE-13651
 URL: https://issues.apache.org/jira/browse/IGNITE-13651
 Project: Ignite
  Issue Type: Task
  Components: documentation
Affects Versions: 2.9
Reporter: YuJue Li
 Fix For: 2.10


The content in the link below is missing from the new version of the document:

[https://apacheignite-sql.readme.io/docs/apache-zeppelin]

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)