Re: I want to contribute to Apache ignite

[Andrey Gura]

Hi, Anton

As I can see you already have contributor permissions. Do you
experience any access problems?

On Mon, Mar 18, 2024 at 7:45 PM Anton Potemkin
 wrote:
>
> Hello, Everyone!
>
> My name is Anton. I have experience using Ignite as a client. We are
> building a distribution system at work, of course it isn’t as advanced and
> complex as Ignite, but I am sure, I benefit Ignite and the community.
> I would like to start from IGNITE-15735
> . Is this ticket still
> relevant? What changes need to be discussed?
> Please, help me to start contributing.
>
> My ASF JIRA username is anton.potemkin
>
> Regards,
>
> Anton

[DISCUSSION] IEP-117: Pluggable Storage Support (Apache Ignite 3)

[Andrey Gura]

Igniters,

please, take a look at IEP-117: Pluggable Storages Support [1] which
is a proposal for
Apache Ignite 3.

The main idea is to have the possibility to support various storages
for different tables in order to provide the best suitable storage
engine for a user's data.

1. 
https://cwiki.apache.org/confluence/display/IGNITE/IEP-117%3A+Pluggable+Storages+Support

[DISCUSSION] IEP-116: Metrics (Apache Ignite 3)

[Andrey Gura]

Igniters,

please, take a look at IEP-116: Metrics [1] which is a proposal for
Apache Ignite 3.

While we already implement metrics in the proposed way any comments
are still welcome.

1. https://cwiki.apache.org/confluence/display/IGNITE/IEP-116%3A+Metrics

Re: Adding ARM64 Dockerfile

[Andrey Gura]

Looks good to me. No objections at all.

Thanks a lot, Pavel!

On Fri, Aug 18, 2023 at 2:30 PM Pavel Tupitsyn  wrote:
>
> Igniters,
>
> I've added another Dockerfile for ARM64 (to support native execution on
> Apple Silicon, AWS Graviton, etc).
>
> This will require another command in section 6.3.9 of Release Process [2]
>
> Ideally, we could publish a multi-arch image [1] under a single tag,
> however, this would require switching from Alpine base to Ubuntu, which I'm
> not sure about. Also, we follow the existing approach with a separate
> Dockerfile for s390x.
>
> PR is ready for review: [3]
>
> Thoughts, objections?
>
>
> [1] https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/
> [2] https://cwiki.apache.org/confluence/display/IGNITE/Release+Process
> [3] https://github.com/apache/ignite/pull/10877
> [4] https://github.com/apache/ignite/issues/10876

Re: [DISCUSSION] IEP-110: Schema synchronization: basic schema changes

[Andrey Gura]

Hi, Roman.

Thanks for IEP. Could you please also add tickets to the corresponding
section (see otghert IEP's for example).

On Mon, Jul 24, 2023 at 9:28 AM Roman Puchkovskiy
 wrote:
>
> Hi Igniters!
>
> Please take a look at a proposal about the application of the Schema
> Synchronization framework to basic schema change types in Ignite 3
> [1].
>
> 1. 
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-110%3A+Schema+synchronization%3A+basic+schema+changes

Re: [DISCUSSION] IEP-103: Code Deployment in Apache Ignite 3

[Andrey Gura]

One more change to the original IEP. The "Dependency resolving and
class loading" section [1] proposes to throw an instance of
ClassNotFoundException for cases where a deployment unit is not found
or can't be used. The same exception should be thrown in cases where a
class loader is not able to load a required class. The problem here is
existing non-Java clients which should be able to distinguish a
regular standard Java exception from Ignite specific exception and
provide to a user a descriptive error message with an error code. Also
a compute job/task could throw a task specific or an unexpected
exception. It also should be taken into account and a separate public
exception should be added for this purpose.

Given this I propose to add two public exceptions:

- DeploymentUnitUnavailableException - have to be thrown if a required
deployment unit is not found (actually it is not deployed) or if a
required deployment unit is obsolete and can't be used in a code
execution.
- ComputeExecutionException - a CompletableFuture instance returned
from the Compute API method has to be completed with an instance of
this exception in case of any other error raised during code
execution.

Corresponding changes have been made to the IEP.

1. 
https://cwiki.apache.org/confluence/display/IGNITE/IEP-103%3A+Code+Deployment#IEP103:CodeDeployment-Dependencyresolvingandclassloading

On Tue, May 23, 2023 at 8:19 PM Andrey Gura  wrote:
>
> Hi, Igniters!
>
> Please take a look at the proposal for Code Deployment functionality
> in Apache Ignite 3 [1].
>
> Thanks for any feedback!
>
> 1. 
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-103%3A+Code+Deployment

Re: [DISCUSSION] IEP-103: Code Deployment in Apache Ignite 3

[Andrey Gura]

Folks,

due to the impossibility of timely detection of changing of a leader
in a replication group I've changed the section "Unit deployment
process" [1]. Now it is enough to deploy a deployment unit to a
replica group majority without mandatory inclusion of a leader.

1. 
https://cwiki.apache.org/confluence/display/IGNITE/IEP-103:+Code+Deployment#IEP103:CodeDeployment-Unitdeploymentprocess

On Tue, May 23, 2023 at 8:19 PM Andrey Gura  wrote:
>
> Hi, Igniters!
>
> Please take a look at the proposal for Code Deployment functionality
> in Apache Ignite 3 [1].
>
> Thanks for any feedback!
>
> 1. 
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-103%3A+Code+Deployment

Re: [DISCUSSION] IEP-103: Code Deployment in Apache Ignite 3

[Andrey Gura]

Vadim,

> Likewise, the `node unit list` command will list only node statuses so the
`nodeId` is redundant because it will be called on the particular node
endpoint.

I still don't understand about the `node unit list`. The suggested syntax is:

node unit list [nodeId [--version version]] [--status ...]

If nodeId is redundant so how will the CLI know a target node? It
means that we have to specify the node. But `nodeId` is not mandatory.
I'm confused. It seems that the suggested syntax is not correct.

On Thu, May 25, 2023 at 6:40 PM Vadim Pakhnushev
 wrote:
>
> The `nodeId` parameter is not required for the `cluster unit list` because
> we will list only cluster statuses.
> Likewise, the `node unit list` command will list only node statuses so the
> `nodeId` is redundant because it will be called on the particular node
> endpoint.
>
> On Thu, May 25, 2023 at 6:18 PM Andrey Gura  wrote:
>
> > Hi, Vadim!
> >
> > Why is the `nodeId` parameter not required in the suggested syntax of
> > `cluster unit list` and `node unit list` commands?
> >
> > What does the `nodeId` parameter mean in the `cluster unit list` command?
> >
> >
> > On Thu, May 25, 2023 at 12:34 PM Vadim Pakhnushev
> >  wrote:
> > >
> > > Hi Andrey!
> > >
> > > I have a small suggestion regarding the design of the CLI commands.
> > > Since we already have a cluster/node namespaces in the CLI, I'd suggest
> > > moving the deploy/undeploy/list commands to the cluster subcommand, and
> > > rather than using --node option to list the units on the particular node,
> > > use node namespace for the list command which will be operating on the
> > node
> > > level.
> > > So the whole list of commands would be:
> > > cluster unit deploy/undeploy - stay the same
> > > cluster unit list [nodeId [--version version]] [--status ...] - without
> > the
> > > --node option
> > > node unit list [nodeId [--version version]] [--status ...] - same
> > filters,
> > > but will show the node unit statuses rather than cluster statuses.
> > >
> > > WDYT?
> > >
> > > Thanks,
> > > Vadim
> > >
> > > On Tue, May 23, 2023 at 8:19 PM Andrey Gura  wrote:
> > >
> > > > Hi, Igniters!
> > > >
> > > > Please take a look at the proposal for Code Deployment functionality
> > > > in Apache Ignite 3 [1].
> > > >
> > > > Thanks for any feedback!
> > > >
> > > > 1.
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-103%3A+Code+Deployment
> > > >
> >

Re: [DISCUSSION] IEP-103: Code Deployment in Apache Ignite 3

[Andrey Gura]

Hi, Vadim!

Why is the `nodeId` parameter not required in the suggested syntax of
`cluster unit list` and `node unit list` commands?

What does the `nodeId` parameter mean in the `cluster unit list` command?


On Thu, May 25, 2023 at 12:34 PM Vadim Pakhnushev
 wrote:
>
> Hi Andrey!
>
> I have a small suggestion regarding the design of the CLI commands.
> Since we already have a cluster/node namespaces in the CLI, I'd suggest
> moving the deploy/undeploy/list commands to the cluster subcommand, and
> rather than using --node option to list the units on the particular node,
> use node namespace for the list command which will be operating on the node
> level.
> So the whole list of commands would be:
> cluster unit deploy/undeploy - stay the same
> cluster unit list [nodeId [--version version]] [--status ...] - without the
> --node option
> node unit list [nodeId [--version version]] [--status ...] - same filters,
> but will show the node unit statuses rather than cluster statuses.
>
> WDYT?
>
> Thanks,
> Vadim
>
> On Tue, May 23, 2023 at 8:19 PM Andrey Gura  wrote:
>
> > Hi, Igniters!
> >
> > Please take a look at the proposal for Code Deployment functionality
> > in Apache Ignite 3 [1].
> >
> > Thanks for any feedback!
> >
> > 1.
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-103%3A+Code+Deployment
> >

Re: IgniteTxStateImpl's non-threadsafe fields may cause crashes and/or data loss

[Andrey Gura]

Please, run benchmarks after fixing the problem. E.g. replacing HashMap to
ConcurrentHashMap can significantly affect performance.

See for example comments to IGNITE-2968 issue (
https://issues.apache.org/jira/browse/IGNITE-2968?focusedCommentId=15415170=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15415170
).

I believe that mentioned invariants were broken later but in general I
agree with Alexey, this state should be accessed mostly from one thread.
Exceptional cases should be synchronized or redesigned. E.g. if metrics
read a transaction's state I prefer remove these metrics or ignore some
inaccuracy then performance reducing.




On Fri, May 19, 2023 at 7:32 PM Ivan Daschinsky  wrote:

> >> Tx processing is supposed to be thread bound by hashing the version to a
> partition
> This invariant is violated in many places. The most notorious example is tx
> recovery.
>
> Another example: I just added an assertion that checks tId of a creator
> thread with tId of an accessor thread.
> TxMultiCacheAsyncOpsTest fails immediately on processing of a tx prepare
> request. Looks like a big issue, IMO
>
>
> пт, 19 мая 2023 г. в 19:11, Alexei Scherbakov <
> alexey.scherbak...@gmail.com
> >:
>
> > Tx processing is supposed to be thread bound by hashing the version to a
> > partition, see methods like [1]
> > If for some cases this invariant is broken, this should be fixed.
> >
> > [1]
> >
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareRequest#partition
> >
> > пт, 19 мая 2023 г. в 15:57, Anton Vinogradov :
> >
> > > Igniters,
> > >
> > > My team was faced with node failure [1] because of non-threadsafe
> > > collections usage.
> > >
> > > IgniteTxStateImpl's fields
> > > - activeCacheIds
> > > - txMap
> > > are not thread safe, but are widely used from different threads without
> > the
> > > proper sync.
> > >
> > > The main question is ... why?
> > >
> > > According to the research, we have no guarantee that tx will be
> processed
> > > at the single thread.
> > > It may be processed at the several! threads at the striped pool and at
> > the
> > > tx recovery thread as well.
> > >
> > > Thread at the striped pool will be selected by the message's
> partition()
> > > method, which can be calculated like this:
> > > - return keys != null && !keys.isEmpty() ? keys.get(0).partition() :
> -1;
> > > - return U.safeAbs(version().hashCode());
> > > - ...,
> > > so, no guarantee it is processed at the same thread (proven by tests).
> > >
> > > Seems, we MAY lose the data.
> > > For example, ignoring some or all keys from txMap at commit.
> > >
> > > If anyone knows why this is not a problem (I mean sync lack, not data
> > loss)
> > > or how to fix this properly, please give me a hint, or correct my
> > > conclusions if necessary.
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-19445
> > >
> >
> >
> > --
> >
> > Best regards,
> > Alexei Scherbakov
> >
>
>
> --
> Sincerely yours, Ivan Daschinskiy
>

[DISCUSSION] IEP-103: Code Deployment in Apache Ignite 3

[Andrey Gura]

Hi, Igniters!

Please take a look at the proposal for Code Deployment functionality
in Apache Ignite 3 [1].

Thanks for any feedback!

1. https://cwiki.apache.org/confluence/display/IGNITE/IEP-103%3A+Code+Deployment

Re: [DISCUSSION] IEP-100: Compute API: Phase 1 - Simple remote job execution

[Andrey Gura]

> Creating a compute job for each record would quickly get unmanageable and 
> broadcast isn’t equivalent as it doesn’t take into account the possibility of 
> partitions migrating during calculations.

Of course creating a compute job for each record is a waste of
resources. Even in current really simple API a user can execute bulk
operation, it is the responsibility of the job's code. But we also
should provide API which simplifies some WAs which user should use in
current design.
Thanks for highlighting these important aspects of Compute API.

On Mon, May 15, 2023 at 5:39 PM Stephen Darlington
 wrote:
>
> Right, it doesn’t need to be the same — exposing partitions in this manner is 
> weird — but I think we need to support bulk operations in some capacity. 
> Creating a compute job for each record would quickly get unmanageable and 
> broadcast isn’t equivalent as it doesn’t take into account the possibility of 
> partitions migrating during calculations.
>
> > On 15 May 2023, at 14:30, Andrey Gura  wrote:
> >
> >> In Ignite 2, for bulk operations there’s an affinityRun method that takes 
> >> a partition as a parameter. I don’t see an equivalent here. Is one in the 
> >> plan?
> >
> > Ideally users don't need to know a partition for execution of
> > computation and the right tool for it is colocation. Method with
> > partitions was introduced in order to execute scan queries on specific
> > partitions and it is some kind of hack. I believe this approach should
> > be revised if possible.
> >
> >> Are other APIs like apply, async methods, map-reduce being considered in 
> >> later phases?
> >
> > All methods are async and return CompletableFuture. Yes, Map/reduce,
> > etc will be designed in later phases.
> >
> > On Mon, May 15, 2023 at 4:08 PM Stephen Darlington
> >  wrote:
> >>
> >> In Ignite 2, for bulk operations there’s an affinityRun method that takes 
> >> a partition as a parameter. I don’t see an equivalent here. Is one in the 
> >> plan?
> >>
> >> Are other APIs like apply, async methods, map-reduce being considered in 
> >> later phases?
> >>
> >>> On 15 May 2023, at 12:52, Andrey Gura  wrote:
> >>>
> >>> Hi, Igniters!
> >>>
> >>> Please take a look at the first phase of proposal for Apache ignite 3
> >>> Compute API: Simple remote job execution [1].
> >>>
> >>> While most issues are already resolved due to simplicity of the proposal
> >>> any feedback and ideas will be really useful for the next phases.
> >>>
> >>> Thanks!
> >>>
> >>> [1]
> >>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-100%3A+Compute+API%3A+Phase+1+-+Simple+remote+job+execution
> >>
>

Re: [DISCUSSION] IEP-100: Compute API: Phase 1 - Simple remote job execution

[Andrey Gura]

> In Ignite 2, for bulk operations there’s an affinityRun method that takes a 
> partition as a parameter. I don’t see an equivalent here. Is one in the plan?

Ideally users don't need to know a partition for execution of
computation and the right tool for it is colocation. Method with
partitions was introduced in order to execute scan queries on specific
partitions and it is some kind of hack. I believe this approach should
be revised if possible.

> Are other APIs like apply, async methods, map-reduce being considered in 
> later phases?

All methods are async and return CompletableFuture. Yes, Map/reduce,
etc will be designed in later phases.

On Mon, May 15, 2023 at 4:08 PM Stephen Darlington
 wrote:
>
> In Ignite 2, for bulk operations there’s an affinityRun method that takes a 
> partition as a parameter. I don’t see an equivalent here. Is one in the plan?
>
> Are other APIs like apply, async methods, map-reduce being considered in 
> later phases?
>
> > On 15 May 2023, at 12:52, Andrey Gura  wrote:
> >
> > Hi, Igniters!
> >
> > Please take a look at the first phase of proposal for Apache ignite 3
> > Compute API: Simple remote job execution [1].
> >
> > While most issues are already resolved due to simplicity of the proposal
> > any feedback and ideas will be really useful for the next phases.
> >
> > Thanks!
> >
> > [1]
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-100%3A+Compute+API%3A+Phase+1+-+Simple+remote+job+execution
>

[DISCUSSION] IEP-100: Compute API: Phase 1 - Simple remote job execution

[Andrey Gura]

Hi, Igniters!

Please take a look at the first phase of proposal for Apache ignite 3
Compute API: Simple remote job execution [1].

While most issues are already resolved due to simplicity of the proposal
any feedback and ideas will be really useful for the next phases.

Thanks!

[1]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-100%3A+Compute+API%3A+Phase+1+-+Simple+remote+job+execution

Re: [VOTE] Release Extensions Ignite Spark 2.0.0 RC1, Ignite Spark 3.0.0 RC1

[Andrey Gura]

+1

On Fri, Dec 9, 2022 at 10:36 AM Nikita Amelchev  wrote:
>
> +1
>
> Checked compilation, ran some examples.
>
> пт, 9 дек. 2022 г. в 03:53, Alexandr Shapkin :
> >
> > Dear Community,
> >
> >
> > The release candidates are ready.
> >
> > These are new Ignite Spark integrations based on Spark 2.4 and 3.2 
> > dependencies respectively.
> >
> >
> > See the links below.
> >
> >
> > == Ignite Spark Extension Module 2.0.0 ==
> >
> > The release candidate is uploaded to:
> > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-spark-ext-2.0.0-rc1/
> >
> > The following Maven staging can be used:
> > https://repository.apache.org/content/repositories/orgapacheignite-1557/org/apache/ignite/ignite-spark-ext/
> >
> > Tag:
> > https://github.com/apache/ignite-extensions/releases/tag/ignite-spark-ext-2.0.0-rc1
> >
> > RELEASE_NOTES:
> > https://github.com/apache/ignite-extensions/blob/release/ignite-spark-ext-2.0.0/modules/spark-ext/spark/RELEASE_NOTES.txt
> >
> > DEVNOTES:
> > https://github.com/apache/ignite-extensions/blob/release/ignite-spark-ext-2.0.0/DEVNOTES.md
> >
> > Release Checker Results:
> > https://github.com/apache/ignite-extensions/actions/runs/3652976657
> >
> >
> >
> > == Ignite Spark Extension Module 3.0.0 ==
> >
> > The release candidate is uploaded to:
> > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-spark-ext-3.0.0-rc1/
> >
> > The following Maven staging can be used:
> > https://repository.apache.org/content/repositories/orgapacheignite-1557/org/apache/ignite/ignite-spark-ext/
> >
> > Tag:
> > https://github.com/apache/ignite-extensions/releases/tag/ignite-spark-ext-3.0.0-rc1
> >
> > RELEASE_NOTES:
> > https://github.com/apache/ignite-extensions/blob/release/ignite-spark-ext-3.0.0/modules/spark-ext/spark/RELEASE_NOTES.txt
> >
> > DEVNOTES:
> > https://github.com/apache/ignite-extensions/blob/release/ignite-spark-ext-3.0.0/DEVNOTES.md
> >
> > Release Checker Results:
> > https://github.com/apache/ignite-extensions/actions/runs/3634524603
> >
> >
> >
> > The vote is formal, see voting guidelines
> > https://www.apache.org/foundation/voting.html
> >
> > +1 - to accept (Ignite Spark 3.0.0 RC1, Ignite Spark 2.0.0 RC1)
> > 0 - don't care either way
> > -1 - DO NOT accept (explain why)
> >
> > See notes on how to verify release here
> > https://www.apache.org/info/verification.html
> > and
> > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification
> >
> >
> > This vote will be open for at least 72 hours till Mon Dec, 12 2022, 02:00 
> > CET TZ.
> > Please, write down the thread if you need additional time to check the 
> > release.
> >
>
>
> --
> Best wishes,
> Amelchev Nikita

[ANNOUNCE] Release managers for Apache Ignite 3.0.0 beta 1 RELEASE

[Andrey Gura]

Igniters,

Vyacheslav Koptilin () is a release manager
for the Apache Ignite 3.0.0 beta 1 release.

Stan Lukyanov () will assist Vyacheslav.

Slava, Stan, thanks a lot for your help!

[ANNOUNCE] SCOPE FREEZE for Apache Ignite 3.0.0 beta 1 RELEASE

[Andrey Gura]

Igniters,

The 'ignite-3.0.0-beta1' branch was created (the latest commit is
8160ef31ecf8d49f227562b6f0ab090c6b4438c1).

The scope for the release is frozen.

It means the following:

- Any issue could be added to the release (fixVersion == 3.0.0-beta1)
only after discussion with the community and a release manager in this
thread.
- Any commit to the release branch must be also applied to the 'main' branch.

Re: [WANTED A NEW RELEASE MANAGER] Apache Ignite 3.0.0 beta 1 RELEASE

[Andrey Gura]

Stan, Slava,

I'll announce the scope freeze very soon. After that I'll announce
changes in the release management.

Thank you for your help!

On Thu, Oct 13, 2022 at 5:46 PM Kseniya Romanova  wrote:
>
> That's unusual, but doesn't seem to break any formal rules as I can see.
>
> чт, 13 окт. 2022 г. в 16:34, Stanislav Lukyanov :
>
> > Thanks Slava. For me, that's a dream team of two! :)
> > Happy to work in this way if there are no objections.
> >
> > Thanks,
> > Stan
> >
> > > On 13 Oct 2022, at 17:08, Andrey Gura  wrote:
> > >
> > > Slava, Stan,
> > >
> > > formally Stan can't be a release manager because he isn't a committer. So
> > > it would be better if Slava will be a release manager and Stan will
> > assist
> > > him.
> > >
> > > It also means that all emails from Slava or Stan (announces, requests,
> > etc)
> > > related to the release will be valid.
> > >
> > > чт, 13 окт. 2022 г., 15:36 Вячеслав Коптилин :
> > >
> > >> Hi Stan,
> > >>
> > >>> I’ll need help from a PMC to do the steps requiring PMC permissions
> > but I
> > >> can do most of the work.
> > >> I'm ready to assist you with this.
> > >>
> > >> Thanks,
> > >> Slava.
> > >>
> > >>
> > >> чт, 13 окт. 2022 г. в 15:08, :
> > >>
> > >>> Hi,
> > >>>
> > >>> I’ll be happy to do this.
> > >>>
> > >>> I’ll need help from a PMC to do the steps requiring PMC permissions
> > but I
> > >>> can do most of the work.
> > >>>
> > >>> Any PMC who is ready to support me in this?
> > >>>
> > >>> Thanks,
> > >>> Stan
> > >>>
> > >>>> On 13 Oct 2022, at 15:54, Andrey Gura  wrote:
> > >>>>
> > >>>> Igniters,
> > >>>>
> > >>>> Due to personal reasons I need to take a pause so we need a new
> > >>>> release manager for the Apache Ignite 3 beta 1 release.
> > >>>>
> > >>>> The best option is a PMC member. Committer is also a good option, but
> > >>>> it will need some help from PMC members.
> > >>>>
> > >>>> Please feel free to submit your candidacy in this thread.
> > >>>
> > >>
> >
> >

Re: [WANTED A NEW RELEASE MANAGER] Apache Ignite 3.0.0 beta 1 RELEASE

[Andrey Gura]

Slava, Stan,

formally Stan can't be a release manager because he isn't a committer. So
it would be better if Slava will be a release manager and Stan will assist
him.

It also means that all emails from Slava or Stan (announces, requests, etc)
related to the release will be valid.

чт, 13 окт. 2022 г., 15:36 Вячеслав Коптилин :

> Hi Stan,
>
> > I’ll need help from a PMC to do the steps requiring PMC permissions but I
> can do most of the work.
> I'm ready to assist you with this.
>
> Thanks,
> Slava.
>
>
> чт, 13 окт. 2022 г. в 15:08, :
>
> > Hi,
> >
> > I’ll be happy to do this.
> >
> > I’ll need help from a PMC to do the steps requiring PMC permissions but I
> > can do most of the work.
> >
> > Any PMC who is ready to support me in this?
> >
> > Thanks,
> > Stan
> >
> > > On 13 Oct 2022, at 15:54, Andrey Gura  wrote:
> > >
> > > Igniters,
> > >
> > > Due to personal reasons I need to take a pause so we need a new
> > > release manager for the Apache Ignite 3 beta 1 release.
> > >
> > > The best option is a PMC member. Committer is also a good option, but
> > > it will need some help from PMC members.
> > >
> > > Please feel free to submit your candidacy in this thread.
> >
>

Re: Apache Ignite 3.0.0 beta 1 RELEASE [Time, Scope, Manager]

[Andrey Gura]

Igniters,

I removed the "3.0.0-alpha6" version and created the "3.0.0-beta1"
version. All issues were rescheduled to the "3.0.0-beta1" version.

Despite the fact that formally was on 12th october there is still
possibility to add issues to the release scope. Deadline is 14th
october. Tomorrow I will announce the scope freeze officially. It
means that any issue could be added to the release only after
discussion with the community and a release manager.

On Mon, Oct 10, 2022 at 7:41 AM Aleksandr Pakhomov  wrote:
>
> +1
>
> > On 7 Oct 2022, at 23:05, Andrey Gura  wrote:
> >
> > Hi, Igniters!
> >
> > It's time for a new release of Apache Ignite 3 beta 1. The expected
> > feature list consists of:
> >
> > - RPM and DEB packages: simplified installation and node management
> > with system services.
> > - Client's Partition Awareness: Clients are now aware of data
> > distribution over the cluster nodes which helps avoid additional
> > network transmissions and lowers operations latency.
> > - C++ client:  Basic C++ client, able to perform operations on data.
> > - Autogenerated values: now a function can be specified as a default
> > value generator during a table creation. Currently only
> > gen_random_uuid is supported.
> > - SQL Transactions.
> > - Transactional Protocol: improved locking model, multi-version based
> > lock-free read-only transactions.
> > - Storage: A number of improvements to memory-only and on-disk engines
> > based on Page Memory.
> > - Indexes: Basic functionality, hash and sorted indexes.
> > - Client logging: A LoggerFactory may be provided during client
> > creation to specify a custom logger for logs generated by the client.
> > - Metrics framework: Collection and export of cluster metrics.
> >
> > I want to propose myself to be the release manager of the Apache
> > Ignite 3 beta 1.
> >
> > Also I propose the following milestones for the release:
> >
> > Scope Freeze: October 12, 2022
> > Code Freeze: October 20, 2022
> > Voting Date: October 31, 2022
> > Release Date: November 5, 2022
> >
> > WDYT?
>

[WANTED A NEW RELEASE MANAGER] Apache Ignite 3.0.0 beta 1 RELEASE

[Andrey Gura]

Igniters,

Due to personal reasons I need to take a pause so we need a new
release manager for the Apache Ignite 3 beta 1 release.

The best option is a PMC member. Committer is also a good option, but
it will need some help from PMC members.

Please feel free to submit your candidacy in this thread.

Apache Ignite 3.0.0 beta 1 RELEASE [Time, Scope, Manager]

[Andrey Gura]

Hi, Igniters!

It's time for a new release of Apache Ignite 3 beta 1. The expected
feature list consists of:

- RPM and DEB packages: simplified installation and node management
with system services.
- Client's Partition Awareness: Clients are now aware of data
distribution over the cluster nodes which helps avoid additional
network transmissions and lowers operations latency.
- C++ client:  Basic C++ client, able to perform operations on data.
- Autogenerated values: now a function can be specified as a default
value generator during a table creation. Currently only
gen_random_uuid is supported.
- SQL Transactions.
- Transactional Protocol: improved locking model, multi-version based
lock-free read-only transactions.
- Storage: A number of improvements to memory-only and on-disk engines
based on Page Memory.
- Indexes: Basic functionality, hash and sorted indexes.
- Client logging: A LoggerFactory may be provided during client
creation to specify a custom logger for logs generated by the client.
- Metrics framework: Collection and export of cluster metrics.

I want to propose myself to be the release manager of the Apache
Ignite 3 beta 1.

Also I propose the following milestones for the release:

Scope Freeze: October 12, 2022
Code Freeze: October 20, 2022
Voting Date: October 31, 2022
Release Date: November 5, 2022

WDYT?

Re: [VOTE] Ignite Packaging IEP

[Andrey Gura]

Igniters,

What is the purpose of this vote? There is no problem during
discussion of the IEP. Silent consensus is enough here.

As I understand, aforementioned dependencies don't affect production
code, so this item doesn't require a voting process.

+1 from me. But it seems that we can go forward without any voting.


On Mon, Aug 29, 2022 at 10:01 AM Вячеслав Коптилин
 wrote:
>
> +1
>
> пт, 26 авг. 2022 г. в 18:07, Mikhail Pochatkin :
>
> > Hi! No, this doesn't affect Ignite 2.x. This IEP is applicable only for
> > Apache Ignite 3.
> >
> > On Fri, Aug 26, 2022 at 5:47 PM Ilya Kasnacheev  > >
> > wrote:
> >
> > > Hello!
> > >
> > > Please clarify whether this is Apache Ignite 3 only and that it does not
> > > affect Apache Ignite 2.x (or does it?)
> > >
> > > Thanks,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > пн, 22 авг. 2022 г. в 16:07, Mikhail Pochatkin :
> > >
> > > > Hi Igniters,
> > > >
> > > > I want to start a voting process about several points from IEP-93 [1].
> > > >
> > > > 1. Switch Apache Ignite build system to Gradle as default.
> > > > 2. Consequently, after changing the build system to Gradle we need to
> > add
> > > > new third party dependencies to several Gradle plugins. This is list of
> > > new
> > > > complete time deps:
> > > > openapiGenerator = "org.openapi.generator:5.4.0"
> > > > javacc = "com.intershop.gradle.javacc:4.0.1"
> > > > launch4j = "edu.sc.seis.launch4j:2.5.3"
> > > > shadow = "com.github.johnrengelman.shadow:7.1.2"
> > > > cmake = "io.github.tomtzook.gradle-cmake:1.0.0"
> > > > jreleaser =  "org.jreleaser:1.1.0"
> > > > 3. Agreement on all target package formats for Apache Ignite
> > > distributions.
> > > >
> > > > The vote is formal, see voting guidelines [3].
> > > >
> > > > +1 - to accept all points above
> > > > 0 - don’t care either way
> > > > -1 - DO NOT accept (explain why)
> > > >
> > > > This vote will be open for at least 4 days till Friday Aug 26, 2022,
> > > > 22:00 Moscow TZ.
> > > >
> > > > [1]
> > > >
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-93%3A+Ignite+Packaging
> > > > [2] https://www.apache.org/foundation/voting.html
> > > >
> > >
> >

Re: URL parameters in CLI

[Andrey Gura]

The main question here is : what is the node URL? How will the user
know a port from a wide port range?

On Tue, Jul 26, 2022 at 3:01 PM Vadim Pakhnushev
 wrote:
>
> Hi Igniters!
> After implementing commands for CLI tool listed in the IEP-88
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-88%3A+CLI+Tool some
> early users raised a concern about confusion between --node-url and
> --cluster-url arguments.
> Initially --node-url was intended to be used in commands dealing with the
> particular node (i.e. node config show) and --cluster-url for commands
> dealing with the cluster as a whole (i.e. cluster config show).
> Although cluster URL is basically a URL of any node in the cluster.
> So the question for discussion is - would it make sense to unify parameters
> to both cluster- and node-related commands to be --node-url?
> Or it would be better to keep the distinction?

Re: Questions about SQL transactions

[Andrey Gura]

Hi,

> 1. Is MVCC the only atomicity mode that supports SQL transactions?

Yes, only TRANSACTIONAL_SNAPSHOT atomicity mode is applicable to SQL.

> What if
I create a table with TRANSACTIONAL atomicity? Will it fall back to like
optimistic/pessimistic transactions (OCC/PCC), or just no transaction
supports at all? If OCC/PCC is applied, what are the limitations?

TRANSACTIONAL atomicity mode works only with KV API.

> 2 (1-yes). Is that possible to implement an external cache store that
provides some sort of transaction support that could work for SQL OCC/PCC,
any suggestions on supplemental materials that are helpful for the goal?

No, it is impossible now. Moreover, SQL is not intended for queries
over cache store storages.

> 3 (1-false). What are the plans for SQL transaction support later on? Will
it likely be supported in the following 2.x versions?

Currently there is no (at least I don't know about it) the
transactional SQL in road map for 2.x.

> I've noticed Ignite
3.x initiative that regards tables as first-class citizens so that SQL
transactions and K-V transactions would be the same thing and natively be
supported?

Yes, Apache Ignite 3 goal is to provide transactions with strong
serializable consistency. KV API and SQL will use the same
transactional protocol and will not provide weak atomic guaranties.
Nevertheless, single KV read/write operations will have minimal
transactional overhead for most cases.

On Tue, Aug 2, 2022 at 3:26 PM Yijie Shen  wrote:
>
> Hi,
>
> I notice that "Caches with the TRANSACTIONAL_SNAPSHOT atomicity mode
> support SQL transactions"[1], but there are also notes on deprecation of `
> TRANSACTIONAL_SNAPSHOT` since Ignite 2.12 due to lots of limitations and
> are not considered production ready [2].
>
> I have a few questions about SQL transactions in Ignite:
>
> 1. Is MVCC the only atomicity mode that supports SQL transactions? What if
> I create a table with TRANSACTIONAL atomicity? Will it fall back to like
> optimistic/pessimistic transactions (OCC/PCC), or just no transaction
> supports at all? If OCC/PCC is applied, what are the limitations?
>
> 2 (1-yes). Is that possible to implement an external cache store that
> provides some sort of transaction support that could work for SQL OCC/PCC,
> any suggestions on supplemental materials that are helpful for the goal?
>
> 3 (1-false). What are the plans for SQL transaction support later on? Will
> it likely be supported in the following 2.x versions? I've noticed Ignite
> 3.x initiative that regards tables as first-class citizens so that SQL
> transactions and K-V transactions would be the same thing and natively be
> supported?
>
> Best,
> Yijie
>
> [1] https://ignite.apache.org/docs/latest/transactions/mvcc
> [2] https://issues.apache.org/jira/browse/IGNITE-15757

[ANNOUNCE] Apache Ignite 3.0.0-alpha5 is released!

[Andrey Gura]

Igniters,

I'm happy to announce that the 5th alpha version of Ignite 3 is out!

On top of the functionality that was previously released, Alpha 5 adds
the following major features:

 - Pluggable storages: ability to choose a specific storage for a
table (RocksDB based storage, Page memory persistent and in-memory
storage) with some known limitations.
  - Compute API (A simple remote job execution): The first phase of
Compute API design and implementation. Of course, with known
limitations.
  - Data colocation: The colocation key concept replaces the affinity
key concept. DDL introduces COLOCATE BY clause. Colocated job
execution.
  - Open API for the Ignite REST endpoints: A Specification to
generate a client for any language + auto-generated docs for REST API.
  - Ignite REPL: The Ignite CLI as a REPL with autocompletion and improved UX.
  - Cluster lifecycle: It introduces cluster initialization logic and
allows to specify cluster management and meta storage groups. Improved
node join protocol.
  - Local and distributed recovery: Now it is possible to restart a
cluster/node without data loss.
  - Data rebalance improvements (in progress and could be excluded
from the release), including dynamically changing the number of
partition replicas.
  - Robust client connection with seamless reconnection support and
retry policies.
  - Java API for SQL: A simplified API for executing SQL
queries on a cluster.

Code examples have been added for the new features, you can check them out
here: https://github.com/apache/ignite-3/tree/3.0.0-alpha5/examples

The best way to try the release out is to go through the Getting Started
Guide: https://ignite.apache.org/docs/3.0.0-alpha

If there are any questions, issues, or thoughts, please do not hesitate to
reply to this email. Ignite Community is welcoming any feedback and will
consider it in future development.

[RESULT][VOTE] Release Apache Ignite 3.0.0-alpha5 RC1

[Andrey Gura]

Igniters,

Apache Ignite 3.0.0-alpha5 RC1 has been accepted.

7 "+1" votes received:
- Semyon Danilov
- Kirill Tkalenko
- Alexander Polovtcev
- Valentin Kulichenko (binding)
- Igor Sapego (binding)
- Pavel Tupitsyn (binding)
    - Andrey Gura (binding)


No "0" or "-1" votes.

Vote thread:
https://lists.apache.org/thread/df97213fjw4t0hhsn5lz9ktvc6ropxyk

Re: [VOTE] Release Apache Ignite 3.0.0-alpha5 RC1

[Andrey Gura]

+1 (binding)

Signatures - checked.
Examples - checked.
Build - checked.

On Tue, Jun 14, 2022 at 11:56 AM Pavel Tupitsyn  wrote:
>
> +1 (binding)
>
> On Fri, Jun 10, 2022 at 4:35 PM Igor Sapego  wrote:
>
> > +1
> >
> > Best Regards,
> > Igor
> >
> >
> > On Fri, Jun 10, 2022 at 1:37 AM Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> > > +1
> > >
> > > On Thu, Jun 9, 2022 at 9:17 AM Alexander Polovtcev <
> > > alexpolovt...@gmail.com>
> > > wrote:
> > >
> > > > Looks good, so many great features! +1
> > > >
> > > > On Thu, Jun 9, 2022 at 6:42 PM Andrey Gura  wrote:
> > > >
> > > > > Dear Community,
> > > > >
> > > > > In the last few month the following major features have been added:
> > > > >
> > > > >   - Pluggable storages: ability to choose a specific storage for a
> > > > > table (RocksDB based storage, Page memory persistent and in-memory
> > > > > storage) with some known limitations.
> > > > >   - Compute API (A simple remote job execution): The first phase of
> > > > > Compute API design and implementation. Of course, with known
> > > > > limitations.
> > > > >   - Data colocation: The colocation key concept replaces the affinity
> > > > > key concept. DDL introduces COLOCATE BY clause. Colocated job
> > > > > execution.
> > > > >   - Open API for the Ignite REST endpoints: A Specification to
> > > > > generate a client for any language + auto-generated docs for REST
> > API.
> > > > >   - Ignite REPL: The Ignite CLI as a REPL with autocompletion and
> > > > improved
> > > > > UX.
> > > > >   - Cluster lifecycle: It introduces cluster initialization logic and
> > > > > allows to specify cluster management and meta storage groups.
> > Improved
> > > > > node join protocol.
> > > > >   - Local and distributed recovery: Now it is possible to restart a
> > > > > cluster/node without data loss.
> > > > >   - Data rebalance improvements (in progress and could be excluded
> > > > > from the release), including dynamically changing the number of
> > > > > partition replicas.
> > > > >   - Robust client connection with seamless reconnection support and
> > > > > retry policies.
> > > > >   - Java API for SQL: A simplified API for executing SQL
> > > > > queries on a cluster.
> > > > >
> > > > > This is an important and interesting milestone and we should share
> > the
> > > > > current state
> > > > > with the broader community. I propose to release 3.0.0 Alpha5 with
> > the
> > > > > features listed above.
> > > > >
> > > > > Please vote.
> > > > >
> > > > > ---
> > > > >
> > > > > Release candidate:
> > > > > https://dist.apache.org/repos/dist/dev/ignite/3.0.0-alpha5-rc1/
> > > > > Maven staging:
> > > > >
> > > https://repository.apache.org/content/repositories/orgapacheignite-1551/
> > > > > Git tag: https://github.com/apache/ignite-3/tree/3.0.0-alpha5-rc1
> > > > >
> > > > > +1 - accept Apache Ignite 3.0.0-alpha5 RC1
> > > > >  0 - don't care either way
> > > > > -1 - DO NOT accept Apache Ignite 3.0.0-alpha5 RC1 (explain why)
> > > > >
> > > > > Voting guidelines: https://www.apache.org/foundation/voting.html
> > > > > How to verify the release:
> > > https://www.apache.org/info/verification.html
> > > > >
> > > > > The vote will be open for 96 hours and will close on June 13th at
> > > 6:45pm
> > > > > https://www.timeanddate.com/countdown/to?iso=20220613T1845=352
> > > > >
> > > >
> > > >
> > > > --
> > > > With regards,
> > > > Aleksandr Polovtcev
> > > >
> > >
> >

[VOTE] Release Apache Ignite 3.0.0-alpha5 RC1

[Andrey Gura]

Dear Community,

In the last few month the following major features have been added:

  - Pluggable storages: ability to choose a specific storage for a
table (RocksDB based storage, Page memory persistent and in-memory
storage) with some known limitations.
  - Compute API (A simple remote job execution): The first phase of
Compute API design and implementation. Of course, with known
limitations.
  - Data colocation: The colocation key concept replaces the affinity
key concept. DDL introduces COLOCATE BY clause. Colocated job
execution.
  - Open API for the Ignite REST endpoints: A Specification to
generate a client for any language + auto-generated docs for REST API.
  - Ignite REPL: The Ignite CLI as a REPL with autocompletion and improved UX.
  - Cluster lifecycle: It introduces cluster initialization logic and
allows to specify cluster management and meta storage groups. Improved
node join protocol.
  - Local and distributed recovery: Now it is possible to restart a
cluster/node without data loss.
  - Data rebalance improvements (in progress and could be excluded
from the release), including dynamically changing the number of
partition replicas.
  - Robust client connection with seamless reconnection support and
retry policies.
  - Java API for SQL: A simplified API for executing SQL
queries on a cluster.

This is an important and interesting milestone and we should share the
current state
with the broader community. I propose to release 3.0.0 Alpha5 with the
features listed above.

Please vote.

---

Release candidate:
https://dist.apache.org/repos/dist/dev/ignite/3.0.0-alpha5-rc1/
Maven staging:
https://repository.apache.org/content/repositories/orgapacheignite-1551/
Git tag: https://github.com/apache/ignite-3/tree/3.0.0-alpha5-rc1

+1 - accept Apache Ignite 3.0.0-alpha5 RC1
 0 - don't care either way
-1 - DO NOT accept Apache Ignite 3.0.0-alpha5 RC1 (explain why)

Voting guidelines: https://www.apache.org/foundation/voting.html
How to verify the release: https://www.apache.org/info/verification.html

The vote will be open for 96 hours and will close on June 13th at 6:45pm
https://www.timeanddate.com/countdown/to?iso=20220613T1845=352

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Thanks a lot to all! We are ready to vote.

On Thu, Jun 9, 2022 at 2:28 PM Andrey Gura  wrote:
>
> Targeted to the release.
>
> Thanks, Alexander.
>
> On Thu, Jun 9, 2022 at 2:09 PM Alexander Polovtcev
>  wrote:
> >
> > Hello again, there's another issue that blocks building the release branch:
> > https://issues.apache.org/jira/browse/IGNITE-17144. Fix will be provided
> > shortly
> >
> > On Mon, Jun 6, 2022 at 5:56 PM Andrey Gura  wrote:
> >
> > > Igniters,
> > >
> > > our release schedule has shifted a bit. But it is time for a code
> > > freeze and a new branch creation.
> > >
> > > The following issues is still in progress (not an issues status, but
> > > work state):
> > >
> > > Data rebalancing
> > > https://issues.apache.org/jira/browse/IGNITE-14209
> > >
> > > CLI MVP
> > > https://issues.apache.org/jira/browse/IGNITE-16971
> > >
> > > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > > https://issues.apache.org/jira/browse/IGNITE-17107
> > >
> > > SQL API: Implement query metadata
> > > https://issues.apache.org/jira/browse/IGNITE-16962
> > >
> > > SQL API: Add batched DML queries support.
> > > https://issues.apache.org/jira/browse/IGNITE-16963
> > >
> > > SQL API: Examples.
> > > https://issues.apache.org/jira/browse/IGNITE-17088
> > >
> > > Please, give some planned timelines for these issues. I would like to
> > > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > > Otherwise, extra steps will be required to include the PR's to the
> > > release branch.
> > >
> > > Thanks!
> > >
> >
> >
> > --
> > With regards,
> > Aleksandr Polovtcev

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Targeted to the release.

Thanks, Alexander.

On Thu, Jun 9, 2022 at 2:09 PM Alexander Polovtcev
 wrote:
>
> Hello again, there's another issue that blocks building the release branch:
> https://issues.apache.org/jira/browse/IGNITE-17144. Fix will be provided
> shortly
>
> On Mon, Jun 6, 2022 at 5:56 PM Andrey Gura  wrote:
>
> > Igniters,
> >
> > our release schedule has shifted a bit. But it is time for a code
> > freeze and a new branch creation.
> >
> > The following issues is still in progress (not an issues status, but
> > work state):
> >
> > Data rebalancing
> > https://issues.apache.org/jira/browse/IGNITE-14209
> >
> > CLI MVP
> > https://issues.apache.org/jira/browse/IGNITE-16971
> >
> > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > https://issues.apache.org/jira/browse/IGNITE-17107
> >
> > SQL API: Implement query metadata
> > https://issues.apache.org/jira/browse/IGNITE-16962
> >
> > SQL API: Add batched DML queries support.
> > https://issues.apache.org/jira/browse/IGNITE-16963
> >
> > SQL API: Examples.
> > https://issues.apache.org/jira/browse/IGNITE-17088
> >
> > Please, give some planned timelines for these issues. I would like to
> > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > Otherwise, extra steps will be required to include the PR's to the
> > release branch.
> >
> > Thanks!
> >
>
>
> --
> With regards,
> Aleksandr Polovtcev

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Alexander,

It looks like a blocker. The issue has been targeted to alpha5 release.

On Wed, Jun 8, 2022 at 7:30 PM Alexander Polovtcev
 wrote:
>
> Hello, dear Igniters.
> I've discovered a bug regarding the Pluggable Storage Engines feature.
> Please consider adding it to the release scope:
> https://issues.apache.org/jira/browse/IGNITE-17139
>
> On Mon, Jun 6, 2022 at 5:56 PM Andrey Gura  wrote:
>
> > Igniters,
> >
> > our release schedule has shifted a bit. But it is time for a code
> > freeze and a new branch creation.
> >
> > The following issues is still in progress (not an issues status, but
> > work state):
> >
> > Data rebalancing
> > https://issues.apache.org/jira/browse/IGNITE-14209
> >
> > CLI MVP
> > https://issues.apache.org/jira/browse/IGNITE-16971
> >
> > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > https://issues.apache.org/jira/browse/IGNITE-17107
> >
> > SQL API: Implement query metadata
> > https://issues.apache.org/jira/browse/IGNITE-16962
> >
> > SQL API: Add batched DML queries support.
> > https://issues.apache.org/jira/browse/IGNITE-16963
> >
> > SQL API: Examples.
> > https://issues.apache.org/jira/browse/IGNITE-17088
> >
> > Please, give some planned timelines for these issues. I would like to
> > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > Otherwise, extra steps will be required to include the PR's to the
> > release branch.
> >
> > Thanks!
> >
>
>
> --
> With regards,
> Aleksandr Polovtcev

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Thanks, Yuri!

On Wed, Jun 8, 2022 at 1:48 PM Юрий  wrote:
>
> Seems  SQL API: Add batched DML queries support [1] will be moved to next
> alpha, so pleas don't waiting for it.
>
> In the same time SQL API: Examples [2] already done and merged to alpha 5
> branch.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-16963
> [2] https://issues.apache.org/jira/browse/IGNITE-17088
>
> пн, 6 июн. 2022 г. в 22:52, Andrey Gura :
>
> > Igniters,
> >
> > ignite-3.0.0-alpha5 release branch has been created. But the following
> > issues are still in progress:
> >
> > Data rebalancing
> > https://issues.apache.org/jira/browse/IGNITE-14209
> >
> > SQL API: Add batched DML queries support.
> > https://issues.apache.org/jira/browse/IGNITE-16963
> >
> > SQL API: Examples.
> > https://issues.apache.org/jira/browse/IGNITE-17088
> >
> > Please, make sure these commits will be merged to the main and to the
> > release branch.
> >
> > Thanks!
> >
> > On Mon, Jun 6, 2022 at 6:02 PM Aleksandr Pakhomov 
> > wrote:
> > >
> > > Hi Andrey,
> > >
> > > As for CLI MVP, the planned timeline is today till 21:00.
> > > Probably, will be ready in an hour. Just waiting for CI build.
> > >
> > > Best regards,
> > > Aleksandr
> > >
> > > > On 6 Jun 2022, at 17:56, Andrey Gura  wrote:
> > > >
> > > > Igniters,
> > > >
> > > > our release schedule has shifted a bit. But it is time for a code
> > > > freeze and a new branch creation.
> > > >
> > > > The following issues is still in progress (not an issues status, but
> > > > work state):
> > > >
> > > > Data rebalancing
> > > > https://issues.apache.org/jira/browse/IGNITE-14209
> > > >
> > > > CLI MVP
> > > > https://issues.apache.org/jira/browse/IGNITE-16971
> > > >
> > > > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > > > https://issues.apache.org/jira/browse/IGNITE-17107
> > > >
> > > > SQL API: Implement query metadata
> > > > https://issues.apache.org/jira/browse/IGNITE-16962
> > > >
> > > > SQL API: Add batched DML queries support.
> > > > https://issues.apache.org/jira/browse/IGNITE-16963
> > > >
> > > > SQL API: Examples.
> > > > https://issues.apache.org/jira/browse/IGNITE-17088
> > > >
> > > > Please, give some planned timelines for these issues. I would like to
> > > > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > > > Otherwise, extra steps will be required to include the PR's to the
> > > > release branch.
> > > >
> > > > Thanks!
> > >
> >
>
>
> --
> Живи с улыбкой! :D

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Pavel,

The issue is targeted to the alpha5 release.

On Wed, Jun 8, 2022 at 11:13 AM Pavel Tupitsyn  wrote:
>
> Igniters, I'd like to include the following ticket as well:
> IGNITE-17057 Thin 3.0: Implement synchronous SQL API for java thin client
> [1]
>
> Would be great to have the same API available in server and client modes.
> It'll be in main in a couple hours.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-17057
>
> On Wed, Jun 8, 2022 at 9:27 AM Андрей Хитрин 
> wrote:
>
> > Looks reasonable to me. IGNITE-17129 could wait for the next release. Thank
> > you!
> >
> > вт, 7 июн. 2022 г. в 20:05, Andrey Gura :
> >
> > > Andrey,
> > >
> > > while IGNITE-17128 is okay just because it demonstrates some
> > > functionality, IGNITE-17129 doesn't look critical and could be fixed
> > > in some future release. WDYT?
> > >
> > > On Tue, Jun 7, 2022 at 5:58 PM Андрей Хитрин 
> > > wrote:
> > > >
> > > > Hello! Could you please add the following issues to the release?
> > > >
> > > > * https://issues.apache.org/jira/browse/IGNITE-17128:
> > > > PersistentPageMemoryStorageExample cannot be run twice
> > > > * https://issues.apache.org/jira/browse/IGNITE-17129: cli tool doesn't
> > > > expand tilde in a config path
> > > >
> > > > пн, 6 июн. 2022 г. в 19:56, Andrey Gura :
> > > >
> > > > > Igniters,
> > > > >
> > > > > our release schedule has shifted a bit. But it is time for a code
> > > > > freeze and a new branch creation.
> > > > >
> > > > > The following issues is still in progress (not an issues status, but
> > > > > work state):
> > > > >
> > > > > Data rebalancing
> > > > > https://issues.apache.org/jira/browse/IGNITE-14209
> > > > >
> > > > > CLI MVP
> > > > > https://issues.apache.org/jira/browse/IGNITE-16971
> > > > >
> > > > > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > > > > https://issues.apache.org/jira/browse/IGNITE-17107
> > > > >
> > > > > SQL API: Implement query metadata
> > > > > https://issues.apache.org/jira/browse/IGNITE-16962
> > > > >
> > > > > SQL API: Add batched DML queries support.
> > > > > https://issues.apache.org/jira/browse/IGNITE-16963
> > > > >
> > > > > SQL API: Examples.
> > > > > https://issues.apache.org/jira/browse/IGNITE-17088
> > > > >
> > > > > Please, give some planned timelines for these issues. I would like to
> > > > > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > > > > Otherwise, extra steps will be required to include the PR's to the
> > > > > release branch.
> > > > >
> > > > > Thanks!
> > > > >
> > > >
> > > >
> > > > --
> > > > Андрей Хитрин
> > >
> >
> >
> > --
> > Андрей Хитрин
> >

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

On Wed, Jun 8, 2022 at 11:13 AM Pavel Tupitsyn  wrote:
>
> Igniters, I'd like to include the following ticket as well:
> IGNITE-17057 Thin 3.0: Implement synchronous SQL API for java thin client
> [1]
>
> Would be great to have the same API available in server and client modes.
> It'll be in main in a couple hours.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-17057
>
> On Wed, Jun 8, 2022 at 9:27 AM Андрей Хитрин 
> wrote:
>
> > Looks reasonable to me. IGNITE-17129 could wait for the next release. Thank
> > you!
> >
> > вт, 7 июн. 2022 г. в 20:05, Andrey Gura :
> >
> > > Andrey,
> > >
> > > while IGNITE-17128 is okay just because it demonstrates some
> > > functionality, IGNITE-17129 doesn't look critical and could be fixed
> > > in some future release. WDYT?
> > >
> > > On Tue, Jun 7, 2022 at 5:58 PM Андрей Хитрин 
> > > wrote:
> > > >
> > > > Hello! Could you please add the following issues to the release?
> > > >
> > > > * https://issues.apache.org/jira/browse/IGNITE-17128:
> > > > PersistentPageMemoryStorageExample cannot be run twice
> > > > * https://issues.apache.org/jira/browse/IGNITE-17129: cli tool doesn't
> > > > expand tilde in a config path
> > > >
> > > > пн, 6 июн. 2022 г. в 19:56, Andrey Gura :
> > > >
> > > > > Igniters,
> > > > >
> > > > > our release schedule has shifted a bit. But it is time for a code
> > > > > freeze and a new branch creation.
> > > > >
> > > > > The following issues is still in progress (not an issues status, but
> > > > > work state):
> > > > >
> > > > > Data rebalancing
> > > > > https://issues.apache.org/jira/browse/IGNITE-14209
> > > > >
> > > > > CLI MVP
> > > > > https://issues.apache.org/jira/browse/IGNITE-16971
> > > > >
> > > > > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > > > > https://issues.apache.org/jira/browse/IGNITE-17107
> > > > >
> > > > > SQL API: Implement query metadata
> > > > > https://issues.apache.org/jira/browse/IGNITE-16962
> > > > >
> > > > > SQL API: Add batched DML queries support.
> > > > > https://issues.apache.org/jira/browse/IGNITE-16963
> > > > >
> > > > > SQL API: Examples.
> > > > > https://issues.apache.org/jira/browse/IGNITE-17088
> > > > >
> > > > > Please, give some planned timelines for these issues. I would like to
> > > > > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > > > > Otherwise, extra steps will be required to include the PR's to the
> > > > > release branch.
> > > > >
> > > > > Thanks!
> > > > >
> > > >
> > > >
> > > > --
> > > > Андрей Хитрин
> > >
> >
> >
> > --
> > Андрей Хитрин
> >

Re: Re[2]: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Igor,

Thanks for your efforts. All changes have been merged.

On Tue, Jun 7, 2022 at 5:41 PM Игорь Гусев  wrote:
>
>
>
> I fixed issues with the PRs mentioned earlier, so hopefully they can be 
> merged.
>
> >Вторник, 7 июня 2022, 16:45 +04:00 от Andrey Gura :
> >
> >Great! Thanks, Slava!
> >
> >On Tue, Jun 7, 2022 at 2:50 PM Вячеслав Коптилин
> >< slava.kopti...@gmail.com > wrote:
> >>
> >> Hello Andrey,
> >>
> >> Data rebalancing (https://issues.apache.org/jira/browse/IGNITE-14209) is
> >> already done and merged to main & ignite-3.0.0-alpha5 branches.
> >>
> >> Thanks,
> >> S.
> >>
> >>
> >> пн, 6 июн. 2022 г. в 22:52, Andrey Gura < ag...@apache.org >:
> >>
> >> > Igniters,
> >> >
> >> > ignite-3.0.0-alpha5 release branch has been created. But the following
> >> > issues are still in progress:
> >> >
> >> > Data rebalancing
> >> >  https://issues.apache.org/jira/browse/IGNITE-14209
> >> >
> >> > SQL API: Add batched DML queries support.
> >> >  https://issues.apache.org/jira/browse/IGNITE-16963
> >> >
> >> > SQL API: Examples.
> >> >  https://issues.apache.org/jira/browse/IGNITE-17088
> >> >
> >> > Please, make sure these commits will be merged to the main and to the
> >> > release branch.
> >> >
> >> > Thanks!
> >> >
> >> > On Mon, Jun 6, 2022 at 6:02 PM Aleksandr Pakhomov < apk...@gmail.com >
> >> > wrote:
> >> > >
> >> > > Hi Andrey,
> >> > >
> >> > > As for CLI MVP, the planned timeline is today till 21:00.
> >> > > Probably, will be ready in an hour. Just waiting for CI build.
> >> > >
> >> > > Best regards,
> >> > > Aleksandr
> >> > >
> >> > > > On 6 Jun 2022, at 17:56, Andrey Gura < ag...@apache.org > wrote:
> >> > > >
> >> > > > Igniters,
> >> > > >
> >> > > > our release schedule has shifted a bit. But it is time for a code
> >> > > > freeze and a new branch creation.
> >> > > >
> >> > > > The following issues is still in progress (not an issues status, but
> >> > > > work state):
> >> > > >
> >> > > > Data rebalancing
> >> > > >  https://issues.apache.org/jira/browse/IGNITE-14209
> >> > > >
> >> > > > CLI MVP
> >> > > >  https://issues.apache.org/jira/browse/IGNITE-16971
> >> > > >
> >> > > > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> >> > > >  https://issues.apache.org/jira/browse/IGNITE-17107
> >> > > >
> >> > > > SQL API: Implement query metadata
> >> > > >  https://issues.apache.org/jira/browse/IGNITE-16962
> >> > > >
> >> > > > SQL API: Add batched DML queries support.
> >> > > >  https://issues.apache.org/jira/browse/IGNITE-16963
> >> > > >
> >> > > > SQL API: Examples.
> >> > > >  https://issues.apache.org/jira/browse/IGNITE-17088
> >> > > >
> >> > > > Please, give some planned timelines for these issues. I would like to
> >> > > > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> >> > > > Otherwise, extra steps will be required to include the PR's to the
> >> > > > release branch.
> >> > > >
> >> > > > Thanks!
> >> > >
> >> >
>
>
> --
> Igor Gusev
>

Re: Re[2]: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Igor,

Documentation issues have been targeted to the current release. I'm
waiting for "ready for merge" notification.

Thanks!

On Tue, Jun 7, 2022 at 6:20 PM Игорь Гусев  wrote:
>
>
> Please include these 2 issues into release:
> https://issues.apache.org/jira/browse/IGNITE-17127
> https://issues.apache.org/jira/browse/IGNITE-17106
>
>
> >Вторник, 7 июня 2022, 19:07 +04:00 от Andrey Gura :
> >
> >Aleksandr,
> >
> >okay, looks like a harmless change with documentation fixes. Targeted
> >to the release. Notify me when it will be ready for merge.
> >
> >On Tue, Jun 7, 2022 at 6:04 PM Aleksandr Pakhomov < apk...@gmail.com > wrote:
> >>
> >> Hi Andrey,
> >>
> >> Could you please include this ticket into release?
> >>  https://issues.apache.org/jira/browse/IGNITE-17126 < 
> >> https://issues.apache.org/jira/browse/IGNITE-17126 >
> >>
> >> Thanks
> >>
> >> > On 6 Jun 2022, at 17:56, Andrey Gura < ag...@apache.org > wrote:
> >> >
> >> > Igniters,
> >> >
> >> > our release schedule has shifted a bit. But it is time for a code
> >> > freeze and a new branch creation.
> >> >
> >> > The following issues is still in progress (not an issues status, but
> >> > work state):
> >> >
> >> > Data rebalancing
> >> >  https://issues.apache.org/jira/browse/IGNITE-14209
> >> >
> >> > CLI MVP
> >> >  https://issues.apache.org/jira/browse/IGNITE-16971
> >> >
> >> > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> >> >  https://issues.apache.org/jira/browse/IGNITE-17107
> >> >
> >> > SQL API: Implement query metadata
> >> >  https://issues.apache.org/jira/browse/IGNITE-16962
> >> >
> >> > SQL API: Add batched DML queries support.
> >> >  https://issues.apache.org/jira/browse/IGNITE-16963
> >> >
> >> > SQL API: Examples.
> >> >  https://issues.apache.org/jira/browse/IGNITE-17088
> >> >
> >> > Please, give some planned timelines for these issues. I would like to
> >> > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> >> > Otherwise, extra steps will be required to include the PR's to the
> >> > release branch.
> >> >
> >> > Thanks!
> >>
>
>
> --
> Igor Gusev
>

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Aleksandr,

okay, looks like a harmless change with documentation fixes. Targeted
to the release. Notify me when it will be ready for merge.

On Tue, Jun 7, 2022 at 6:04 PM Aleksandr Pakhomov  wrote:
>
> Hi Andrey,
>
> Could you please include this ticket into release?
> https://issues.apache.org/jira/browse/IGNITE-17126 
> <https://issues.apache.org/jira/browse/IGNITE-17126>
>
> Thanks
>
> > On 6 Jun 2022, at 17:56, Andrey Gura  wrote:
> >
> > Igniters,
> >
> > our release schedule has shifted a bit. But it is time for a code
> > freeze and a new branch creation.
> >
> > The following issues is still in progress (not an issues status, but
> > work state):
> >
> > Data rebalancing
> > https://issues.apache.org/jira/browse/IGNITE-14209
> >
> > CLI MVP
> > https://issues.apache.org/jira/browse/IGNITE-16971
> >
> > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > https://issues.apache.org/jira/browse/IGNITE-17107
> >
> > SQL API: Implement query metadata
> > https://issues.apache.org/jira/browse/IGNITE-16962
> >
> > SQL API: Add batched DML queries support.
> > https://issues.apache.org/jira/browse/IGNITE-16963
> >
> > SQL API: Examples.
> > https://issues.apache.org/jira/browse/IGNITE-17088
> >
> > Please, give some planned timelines for these issues. I would like to
> > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > Otherwise, extra steps will be required to include the PR's to the
> > release branch.
> >
> > Thanks!
>

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Kirill, thanks for help!

I've already targeted this issue to the current release. Make me know
when the issue will be ready to merge.

On Tue, Jun 7, 2022 at 5:59 PM ткаленко кирилл  wrote:
>
> Hello everyone, I would like to fix the identified bug when destroying the 
> table for persistent PageMemory, I could fix it tomorrow, can we add it to 
> the scope?
>
> https://issues.apache.org/jira/browse/IGNITE-17128

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Andrey,

while IGNITE-17128 is okay just because it demonstrates some
functionality, IGNITE-17129 doesn't look critical and could be fixed
in some future release. WDYT?

On Tue, Jun 7, 2022 at 5:58 PM Андрей Хитрин  wrote:
>
> Hello! Could you please add the following issues to the release?
>
> * https://issues.apache.org/jira/browse/IGNITE-17128:
> PersistentPageMemoryStorageExample cannot be run twice
> * https://issues.apache.org/jira/browse/IGNITE-17129: cli tool doesn't
> expand tilde in a config path
>
> пн, 6 июн. 2022 г. в 19:56, Andrey Gura :
>
> > Igniters,
> >
> > our release schedule has shifted a bit. But it is time for a code
> > freeze and a new branch creation.
> >
> > The following issues is still in progress (not an issues status, but
> > work state):
> >
> > Data rebalancing
> > https://issues.apache.org/jira/browse/IGNITE-14209
> >
> > CLI MVP
> > https://issues.apache.org/jira/browse/IGNITE-16971
> >
> > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > https://issues.apache.org/jira/browse/IGNITE-17107
> >
> > SQL API: Implement query metadata
> > https://issues.apache.org/jira/browse/IGNITE-16962
> >
> > SQL API: Add batched DML queries support.
> > https://issues.apache.org/jira/browse/IGNITE-16963
> >
> > SQL API: Examples.
> > https://issues.apache.org/jira/browse/IGNITE-17088
> >
> > Please, give some planned timelines for these issues. I would like to
> > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > Otherwise, extra steps will be required to include the PR's to the
> > release branch.
> >
> > Thanks!
> >
>
>
> --
> Андрей Хитрин

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Great! Thanks, Slava!

On Tue, Jun 7, 2022 at 2:50 PM Вячеслав Коптилин
 wrote:
>
> Hello Andrey,
>
> Data rebalancing (https://issues.apache.org/jira/browse/IGNITE-14209) is
> already done and merged to main & ignite-3.0.0-alpha5 branches.
>
> Thanks,
> S.
>
>
> пн, 6 июн. 2022 г. в 22:52, Andrey Gura :
>
> > Igniters,
> >
> > ignite-3.0.0-alpha5 release branch has been created. But the following
> > issues are still in progress:
> >
> > Data rebalancing
> > https://issues.apache.org/jira/browse/IGNITE-14209
> >
> > SQL API: Add batched DML queries support.
> > https://issues.apache.org/jira/browse/IGNITE-16963
> >
> > SQL API: Examples.
> > https://issues.apache.org/jira/browse/IGNITE-17088
> >
> > Please, make sure these commits will be merged to the main and to the
> > release branch.
> >
> > Thanks!
> >
> > On Mon, Jun 6, 2022 at 6:02 PM Aleksandr Pakhomov 
> > wrote:
> > >
> > > Hi Andrey,
> > >
> > > As for CLI MVP, the planned timeline is today till 21:00.
> > > Probably, will be ready in an hour. Just waiting for CI build.
> > >
> > > Best regards,
> > > Aleksandr
> > >
> > > > On 6 Jun 2022, at 17:56, Andrey Gura  wrote:
> > > >
> > > > Igniters,
> > > >
> > > > our release schedule has shifted a bit. But it is time for a code
> > > > freeze and a new branch creation.
> > > >
> > > > The following issues is still in progress (not an issues status, but
> > > > work state):
> > > >
> > > > Data rebalancing
> > > > https://issues.apache.org/jira/browse/IGNITE-14209
> > > >
> > > > CLI MVP
> > > > https://issues.apache.org/jira/browse/IGNITE-16971
> > > >
> > > > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > > > https://issues.apache.org/jira/browse/IGNITE-17107
> > > >
> > > > SQL API: Implement query metadata
> > > > https://issues.apache.org/jira/browse/IGNITE-16962
> > > >
> > > > SQL API: Add batched DML queries support.
> > > > https://issues.apache.org/jira/browse/IGNITE-16963
> > > >
> > > > SQL API: Examples.
> > > > https://issues.apache.org/jira/browse/IGNITE-17088
> > > >
> > > > Please, give some planned timelines for these issues. I would like to
> > > > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > > > Otherwise, extra steps will be required to include the PR's to the
> > > > release branch.
> > > >
> > > > Thanks!
> > >
> >

Re: Re[2]: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Thanks, Igor.

These issues are included in the release. Please, move forward and let
me know when PR's will be ready to be merged (all checks done, review
done, tickets have corresponding comments from reviewrs).

On Tue, Jun 7, 2022 at 2:19 PM Игорь Гусев  wrote:
>
>
> Hi, I did some PRs with docs for new features. Can you include them in 
> release?
>
> https://github.com/apache/ignite-3/pull/850
> https://github.com/apache/ignite-3/pull/856
> https://github.com/apache/ignite-3/pull/857
>
>
> >Вторник, 7 июня 2022, 0:09 +04:00 от Andrey Gura :
> >
> >And, of course, there is a code freeze for the release branch. So,
> >permission is required to make any changes to the release branch.
> >Don't hesitate to discuss such changes in this topic (except three
> >issues from the previous post).
> >
> >On Mon, Jun 6, 2022 at 10:52 PM Andrey Gura < ag...@apache.org > wrote:
> >>
> >> Igniters,
> >>
> >> ignite-3.0.0-alpha5 release branch has been created. But the following
> >> issues are still in progress:
> >>
> >> Data rebalancing
> >>  https://issues.apache.org/jira/browse/IGNITE-14209
> >>
> >> SQL API: Add batched DML queries support.
> >>  https://issues.apache.org/jira/browse/IGNITE-16963
> >>
> >> SQL API: Examples.
> >>  https://issues.apache.org/jira/browse/IGNITE-17088
> >>
> >> Please, make sure these commits will be merged to the main and to the
> >> release branch.
> >>
> >> Thanks!
> >>
> >> On Mon, Jun 6, 2022 at 6:02 PM Aleksandr Pakhomov < apk...@gmail.com > 
> >> wrote:
> >> >
> >> > Hi Andrey,
> >> >
> >> > As for CLI MVP, the planned timeline is today till 21:00.
> >> > Probably, will be ready in an hour. Just waiting for CI build.
> >> >
> >> > Best regards,
> >> > Aleksandr
> >> >
> >> > > On 6 Jun 2022, at 17:56, Andrey Gura < ag...@apache.org > wrote:
> >> > >
> >> > > Igniters,
> >> > >
> >> > > our release schedule has shifted a bit. But it is time for a code
> >> > > freeze and a new branch creation.
> >> > >
> >> > > The following issues is still in progress (not an issues status, but
> >> > > work state):
> >> > >
> >> > > Data rebalancing
> >> > >  https://issues.apache.org/jira/browse/IGNITE-14209
> >> > >
> >> > > CLI MVP
> >> > >  https://issues.apache.org/jira/browse/IGNITE-16971
> >> > >
> >> > > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> >> > >  https://issues.apache.org/jira/browse/IGNITE-17107
> >> > >
> >> > > SQL API: Implement query metadata
> >> > >  https://issues.apache.org/jira/browse/IGNITE-16962
> >> > >
> >> > > SQL API: Add batched DML queries support.
> >> > >  https://issues.apache.org/jira/browse/IGNITE-16963
> >> > >
> >> > > SQL API: Examples.
> >> > >  https://issues.apache.org/jira/browse/IGNITE-17088
> >> > >
> >> > > Please, give some planned timelines for these issues. I would like to
> >> > > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> >> > > Otherwise, extra steps will be required to include the PR's to the
> >> > > release branch.
> >> > >
> >> > > Thanks!
> >> >
>
>
> --
> Igor Gusev
>

Re: Request for contributors permissions

[Andrey Gura]

Aleksey,

contributor role has been granted.

On Mon, Jun 6, 2022 at 1:56 PM Aleksey Demakov  wrote:
>
> Hi Ignite community,
>
> My name is Aleksey Demakov, I'd like to contribute to the Apache
> Ignite project. Could you please grant me contributor permissions to
> jira and wiki. My username is ademakov.
>
> Thanks,
> Aleksey

Re: Request for contributors permissions

[Andrey Gura]

Vadim,

Contributor role has been granted.

On Mon, Jun 6, 2022 at 11:36 AM Vadim Pakhnushev
 wrote:
>
> Hi!
>
> Could you please grant contributor access to JIRA and Confluence?
> My username is vpakhnushev.
>
> Thanks!

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

And, of course, there is a code freeze for the release branch. So,
permission is required to make any changes to the release branch.
Don't hesitate to discuss such changes in this topic (except three
issues from the previous post).

On Mon, Jun 6, 2022 at 10:52 PM Andrey Gura  wrote:
>
> Igniters,
>
> ignite-3.0.0-alpha5 release branch has been created. But the following
> issues are still in progress:
>
> Data rebalancing
> https://issues.apache.org/jira/browse/IGNITE-14209
>
> SQL API: Add batched DML queries support.
> https://issues.apache.org/jira/browse/IGNITE-16963
>
> SQL API: Examples.
> https://issues.apache.org/jira/browse/IGNITE-17088
>
> Please, make sure these commits will be merged to the main and to the
> release branch.
>
> Thanks!
>
> On Mon, Jun 6, 2022 at 6:02 PM Aleksandr Pakhomov  wrote:
> >
> > Hi Andrey,
> >
> > As for CLI MVP, the planned timeline is today till 21:00.
> > Probably, will be ready in an hour. Just waiting for CI build.
> >
> > Best regards,
> > Aleksandr
> >
> > > On 6 Jun 2022, at 17:56, Andrey Gura  wrote:
> > >
> > > Igniters,
> > >
> > > our release schedule has shifted a bit. But it is time for a code
> > > freeze and a new branch creation.
> > >
> > > The following issues is still in progress (not an issues status, but
> > > work state):
> > >
> > > Data rebalancing
> > > https://issues.apache.org/jira/browse/IGNITE-14209
> > >
> > > CLI MVP
> > > https://issues.apache.org/jira/browse/IGNITE-16971
> > >
> > > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > > https://issues.apache.org/jira/browse/IGNITE-17107
> > >
> > > SQL API: Implement query metadata
> > > https://issues.apache.org/jira/browse/IGNITE-16962
> > >
> > > SQL API: Add batched DML queries support.
> > > https://issues.apache.org/jira/browse/IGNITE-16963
> > >
> > > SQL API: Examples.
> > > https://issues.apache.org/jira/browse/IGNITE-17088
> > >
> > > Please, give some planned timelines for these issues. I would like to
> > > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > > Otherwise, extra steps will be required to include the PR's to the
> > > release branch.
> > >
> > > Thanks!
> >

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Igniters,

ignite-3.0.0-alpha5 release branch has been created. But the following
issues are still in progress:

Data rebalancing
https://issues.apache.org/jira/browse/IGNITE-14209

SQL API: Add batched DML queries support.
https://issues.apache.org/jira/browse/IGNITE-16963

SQL API: Examples.
https://issues.apache.org/jira/browse/IGNITE-17088

Please, make sure these commits will be merged to the main and to the
release branch.

Thanks!

On Mon, Jun 6, 2022 at 6:02 PM Aleksandr Pakhomov  wrote:
>
> Hi Andrey,
>
> As for CLI MVP, the planned timeline is today till 21:00.
> Probably, will be ready in an hour. Just waiting for CI build.
>
> Best regards,
> Aleksandr
>
> > On 6 Jun 2022, at 17:56, Andrey Gura  wrote:
> >
> > Igniters,
> >
> > our release schedule has shifted a bit. But it is time for a code
> > freeze and a new branch creation.
> >
> > The following issues is still in progress (not an issues status, but
> > work state):
> >
> > Data rebalancing
> > https://issues.apache.org/jira/browse/IGNITE-14209
> >
> > CLI MVP
> > https://issues.apache.org/jira/browse/IGNITE-16971
> >
> > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > https://issues.apache.org/jira/browse/IGNITE-17107
> >
> > SQL API: Implement query metadata
> > https://issues.apache.org/jira/browse/IGNITE-16962
> >
> > SQL API: Add batched DML queries support.
> > https://issues.apache.org/jira/browse/IGNITE-16963
> >
> > SQL API: Examples.
> > https://issues.apache.org/jira/browse/IGNITE-17088
> >
> > Please, give some planned timelines for these issues. I would like to
> > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > Otherwise, extra steps will be required to include the PR's to the
> > release branch.
> >
> > Thanks!
>

[ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Igniters,

our release schedule has shifted a bit. But it is time for a code
freeze and a new branch creation.

The following issues is still in progress (not an issues status, but
work state):

Data rebalancing
https://issues.apache.org/jira/browse/IGNITE-14209

CLI MVP
https://issues.apache.org/jira/browse/IGNITE-16971

[Native Persistence 3.0] End-to-end test for persistent PageMemory
https://issues.apache.org/jira/browse/IGNITE-17107

SQL API: Implement query metadata
https://issues.apache.org/jira/browse/IGNITE-16962

SQL API: Add batched DML queries support.
https://issues.apache.org/jira/browse/IGNITE-16963

SQL API: Examples.
https://issues.apache.org/jira/browse/IGNITE-17088

Please, give some planned timelines for these issues. I would like to
create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
Otherwise, extra steps will be required to include the PR's to the
release branch.

Thanks!

Re: [RESULT][VOTE] Add swagger dependency to Ignite 3

[Andrey Gura]

Aleksandr,

could you please update
https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries

On Mon, May 30, 2022 at 12:33 PM Aleksandr Pakhomov  wrote:
>
> Hello Igniters,
>
> The swagger dependency has been accepted.
>
> 3 - “+1” votes received.
> 1 - “0.5” vote received.
>
> Here are +1 votes received:
>
> - Alexander Polovtcev
> - Andrey Gura
> - Ivan Pavlukhin
>
> Here is -0.5 vote received:
>
> - Ilya Kasnacheev (binding)
>
> Link to the voting thread -
> https://lists.apache.org/thread/dntnxcyzpddjg5yc5vkd8lxjtd5z54rc 
> <https://lists.apache.org/thread/dntnxcyzpddjg5yc5vkd8lxjtd5z54rc>
>
> Best regards,
> Aleksandr

Re: [VOTE] Add micronaut dependency to Ignite 3 (reopened)

[Andrey Gura]

+1

On Mon, May 30, 2022 at 12:13 PM Alexander Polovtcev
 wrote:
>
> Aleksandr,
> Thanks for the update, but can you please explain what security
> capabilities are planned to be used? Maybe it should be included in the IEP
> as well.
>
> On Sun, May 29, 2022 at 11:03 PM Aleksandr Pakhomov 
> wrote:
>
> > Dear community,
> >
> > We’ve had a productive discussion about
> > a micronaut dependency [1]. As a result,
> > we’ve came to the conclusion that the
> > security support in micronaut is a good
> > reason to use this library. I am reopening the
> > vote for adding the micronaut dependency to
> > the rest module. Also, micronaut-serde is
> > dropped from the list because Ignite 3
> > already uses jackson as a serialisation lib.
> >
> > List of dependencies:
> > io.micronaut:micronaut-inject:jar:3.4.1
> > io.micronaut:micronaut-core:jar:3.4.1
> > io.micronaut:micronaut-http-server-netty:jar:3.4.1
> > io.micronaut:micronaut-http-server:jar:3.4.1
> > io.micronaut:micronaut-router:jar:3.4.1
> > io.micronaut:micronaut-http-netty:jar:3.4.1
> > io.micronaut:micronaut-buffer-netty:jar:3.4.1
> > io.micronaut:micronaut-runtime:jar:3.4.1
> > io.micronaut:micronaut-http:jar:3.4.1
> > io.micronaut:micronaut-aop:jar:3.4.1
> > io.micronaut:micronaut-context:jar:3.4.1
> > io.micronaut:micronaut-core-reactive:jar:3.4.1
> >
> > More information about motivation and
> > implementation details could be found
> > in IEP-87 [2].
> >
> > The vote is formal, see voting guidelines [3].
> >
> > +1 - to accept additional dependencies to be included to Java code
> > Guidelines [4]
> > 0 - don’t care either way
> > -1 - DO NOT accept (explain why)
> >
> > This vote will be open for at leas 3 days till Wed June 1, 2022,
> > 23:00 Moscow TZ [5].
> >
> >
> > [1] https://lists.apache.org/thread/0nq6wx8t9r036mrjsk6n592gwnvvqbhs <
> > https://lists.apache.org/thread/0nq6wx8t9r036mrjsk6n592gwnvvqbhs>
> > [2]
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST
> > <
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-87:+Open+API+support+for+REST>
> >
> > [3] https://www.apache.org/foundation/voting.html <
> > https://www.apache.org/foundation/voting.html>
> > [4]
> > https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries
> > <
> > https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries
> > >
> > [5]
> > https://www.timeanddate.com/countdown/generic?iso=20220601T23=166=%5BVOTE%5D+Add+micronaut+dependency+to+Ignite+3+%28reopened%29=cursive
> > <
> > https://www.timeanddate.com/countdown/generic?iso=20220601T23=166=[VOTE]+Add+micronaut+dependency+to+Ignite+3+(reopened)=cursive>
> >
>
>
>
> --
> With regards,
> Aleksandr Polovtcev

Re: [VOTE] Add micronaut dependency to Ignite 3

[Andrey Gura]

Aleksandr,

please, start a new vote after updating the IEP-87. I think that
Micronaut Security is a good reason for using this framework in the
Apache Ignite.



On Wed, May 25, 2022 at 5:41 PM Andrey Gura  wrote:
>
> -1 (binding) from me.
>
> Because (if I understood correctly) the main value of the IEP-87 is
> the possibility to generate API specification and Swagger annotations
> is enough for this purpose I don't see reasons for these dependencies.
> We already have our own controllers for REST-like API's
> implementation. Why can't we just use Swagger annotations only in
> addition to our rest-api module?
>
> On Mon, May 23, 2022 at 8:08 PM Aleksandr Pakhomov  wrote:
> >
> > Dear community,
> >
> > Micronaut-based REST server implementation was a hot
> > topic we discussed in the previous week. So, I've separeted
> > votes about swagger and micronaut. This vote is about
> > adding micronaut to the Ignite 3.
> >
> > The exact list of dependencies could be fined in IEP-87 [1]
> > io.micronaut.serde:micronaut-serde
> > io.micronaut:micronaut-context
> > io.micronaut:micronaut-http
> > io.micronaut:micronaut-inject
> > io.micronaut:micronaut-http-server
> > io.micronaut:micronaut-runtime
> > io.micronaut:micronaut-core
> > io.micronaut:micronaut-http-server-netty
> > io.micronaut:micronaut-http-netty
> > io.micronaut:micronaut-buffer-netty
> > io.micronaut:micronaut-aop
> > io.micronaut:micronaut-core-reactive
> > Io.micronaut:micronaut-json-core
> > io.micronaut:micronaut-jackson-core
> >
> > Swagger is out of the scope of this voting.
> >
> > The vote is formal, see voting guidelines [2]
> >
> > +1 - to accept additional dependencies to be included to Java code 
> > Guidelines [3]
> > 0 - don't care either way
> > -1 - DO NOT accept (explain why)
> >
> > This vote will be open for at least 4 days till Fri May 27, 2022,
> > 21:00 Moscow TZ.
> >
> > [1] 
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST#IEP87:OpenAPIsupportforREST-Additionaldependencies
> > [2] https://www.apache.org/foundation/voting.html 
> > <https://www.apache.org/foundation/voting.html>
> > [3] 
> > https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries
> >  
> > <https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries>
> > [4] 
> > https://www.timeanddate.com/countdown/generic?iso=20220527T21=166=%5BVOTE%5D+Add+micronaut+dependency+to+Ignite+3=cursive

Re: [DISCUSSION] Add additional 3rd party libraries to Ignite 3 Code style and practices

[Andrey Gura]

Hi,

Hmmm... It seems that [2] is just a marketing page that doesn't give
any arguments about the micronaut.

Anyway, security is a good point. Aleksandr, could you please add some
note to the IEP about security related value of the micronaut
framework?





On Thu, May 26, 2022 at 9:48 AM Mikhail Pochatkin
 wrote:
>
> Hello Igniters.
>
> I see that the main argument against Micronaut is that we need
> Micronaut+Swagger to achieve IEP 87 if we can use only Swagger. I agree
> this is a valid point and logically. But I think we can take a look into it
> from another side, from future vision. Main idea to start using micronaut
> to simplify code writing in future just because Micronaut Framework really
> helps to develop REST services in practice. For example, I think it's
> obvious that Ignite will need to implement some authentication mechanism
> and Micronaut can help to achieve it [1]. Moreover Micronaut is based on
> the same technology which is currently used in Ignite, I mean Netty. It
> means that we should speak only about vulnerabilities inside of Micronaut
> implementation, but the Micronaut project has a very big community and a
> lot of clients. In context of it I would say that bugs and vulnerabilities
> in Micronaut implementation will be found more quickly than in internal
> Ignite implementation with pure Netty. Another point about cloud native
> support [2] which also can be useful in the context of Ignite.
>
> [1] https://micronaut-projects.github.io/micronaut-security/latest/guide/
> [2] https://objectcomputing.com/expertise/cloud-engineering
>
> On Wed, May 25, 2022 at 6:45 PM Andrey Gura  wrote:
>
> > Why couldn't the handlers be annotated?
> >
> > I think Kirill Gusakov (as the feature contributor) can give some
> > ideas about what and how should be annotated. This may require some
> > tweaking.
> >
> > On Wed, May 25, 2022 at 6:06 PM Aleksandr Pakhomov 
> > wrote:
> > >
> > > Hi Alexander,
> > >
> > > Yes, swagger allows to annotate classes
> > > and then the spec will be generated. But
> > > here is a question. What classes should we
> > > annotade?
> > >
> > > Web framework brings the convention about
> > > how to write an endpoint and what to annotate.
> > > Our own REST server implementation does not.
> > > It is just the number of handlers.
> > >
> > > > On 25 May 2022, at 17:46, Alexander Polovtcev 
> > wrote:
> > > >
> > > > Aleksandr,
> > > >
> > > > Can we use these annotations without the micronaut dependencies? If
> > yes,
> > > > why do we need micronaut at all?
> > > >
> > > > On Mon, May 23, 2022 at 5:34 PM Aleksandr Pakhomov 
> > wrote:
> > > >
> > > >> Yes, swagger-core has its own set of annotations [1]
> > > >>
> > > >> [1]
> > > >>
> > https://github.com/swagger-api/swagger-core/wiki/Swagger-2.X---Annotations#quick-annotation-overview
> > > >>
> > > >>> On 23 May 2022, at 12:37, Alexander Polovtcev <
> > alexpolovt...@gmail.com>
> > > >> wrote:
> > > >>>
> > > >>> I'm not that scared of having a big library, I just believe that it
> > might
> > > >>> be possible to avoid using it altogether. Don't swagger generators
> > have
> > > >>> their own annotations?
> > > >>
> > > >>
> > > >
> > >
> >

Re: [VOTE] Add swagger dependency to Ignite 3

[Andrey Gura]

+1 from me. Open API spec could be useful in the future for
implementing external cluster management tools.

On Thu, May 26, 2022 at 11:41 AM Aleksandr Pakhomov  wrote:
>
> Hi Ivan,
>
> Dependencies that are needed for annotating
> classes are going to be included. From those
> annotations Open API spec is generated by
> maven plugin. That’s it.
>
> If you are asking about swagger ui or any
> web stuff then the answer is no. We are
> not going to include this into production.
>
> > On 26 May 2022, at 07:34, Ivan Pavlukhin  wrote:
> >
> > Hi,
> >
> > Are we going to include swagger into production packages? I always
> > thought (I might be mistaken) that swagger should be used during
> > development. Worries are usual:
> > 1. Potential vulnerabilities.
> > 2. Unintentional use of transitive dependencies.
> >
> > Best regards,
> > Ivan Pavlukhin
> >
> > чт, 26 мая 2022 г. в 00:46, Mikhail Pochatkin :
> >>
> >> +1 from me, de facto swagger standard within OpenApi.
> >>
> >> On Mon, May 23, 2022 at 7:57 PM Aleksandr Pakhomov  
> >> wrote:
> >>
> >>> Dear community,
> >>>
> >>> Discussion about 3rd party dependencies took place
> >>> and I think it is time to vote if we agreed to include
> >>> swagger dependency to the Ignite 3 or not.
> >>>
> >>> The exact list of dependencies could be fined in IEP-87 [1]
> >>> (swagger-annotations, swagger-core,
> >>> swagger-codegen-maven-plugin)
> >>>
> >>> Micronaut is out of the scope of this voting. I will launch
> >>> a separate one.
> >>>
> >>> The vote is formal, see voting guidelines [2]
> >>>
> >>> +1 - to accept additional dependencies to be included to Java code
> >>> Guidelines [3]
> >>> 0 - don't care either way
> >>> -1 - DO NOT accept (explain why)
> >>>
> >>> [1]
> >>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST#IEP87:OpenAPIsupportforREST-Additionaldependencies
> >>> [2] https://www.apache.org/foundation/voting.html <
> >>> https://www.apache.org/foundation/voting.html>
> >>> [3]
> >>> https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries
>

Re: [DISCUSSION] Add additional 3rd party libraries to Ignite 3 Code style and practices

[Andrey Gura]

Why couldn't the handlers be annotated?

I think Kirill Gusakov (as the feature contributor) can give some
ideas about what and how should be annotated. This may require some
tweaking.

On Wed, May 25, 2022 at 6:06 PM Aleksandr Pakhomov  wrote:
>
> Hi Alexander,
>
> Yes, swagger allows to annotate classes
> and then the spec will be generated. But
> here is a question. What classes should we
> annotade?
>
> Web framework brings the convention about
> how to write an endpoint and what to annotate.
> Our own REST server implementation does not.
> It is just the number of handlers.
>
> > On 25 May 2022, at 17:46, Alexander Polovtcev  
> > wrote:
> >
> > Aleksandr,
> >
> > Can we use these annotations without the micronaut dependencies? If yes,
> > why do we need micronaut at all?
> >
> > On Mon, May 23, 2022 at 5:34 PM Aleksandr Pakhomov  wrote:
> >
> >> Yes, swagger-core has its own set of annotations [1]
> >>
> >> [1]
> >> https://github.com/swagger-api/swagger-core/wiki/Swagger-2.X---Annotations#quick-annotation-overview
> >>
> >>> On 23 May 2022, at 12:37, Alexander Polovtcev 
> >> wrote:
> >>>
> >>> I'm not that scared of having a big library, I just believe that it might
> >>> be possible to avoid using it altogether. Don't swagger generators have
> >>> their own annotations?
> >>
> >>
> >
>

Re: [DISCUSSION] Add additional 3rd party libraries to Ignite 3 Code style and practices

[Andrey Gura]

Agree with Alexander. I ask the same question in the vote thread.

On Wed, May 25, 2022 at 5:47 PM Alexander Polovtcev
 wrote:
>
> Aleksandr,
>
> Can we use these annotations without the micronaut dependencies? If yes,
> why do we need micronaut at all?
>
> On Mon, May 23, 2022 at 5:34 PM Aleksandr Pakhomov  wrote:
>
> > Yes, swagger-core has its own set of annotations [1]
> >
> > [1]
> > https://github.com/swagger-api/swagger-core/wiki/Swagger-2.X---Annotations#quick-annotation-overview
> >
> > > On 23 May 2022, at 12:37, Alexander Polovtcev 
> > wrote:
> > >
> > > I'm not that scared of having a big library, I just believe that it might
> > > be possible to avoid using it altogether. Don't swagger generators have
> > > their own annotations?
> >
> >
>
> --
> With regards,
> Aleksandr Polovtcev

Re: [VOTE] Add swagger dependency to Ignite 3

[Andrey Gura]

Ilya,

are there any alternatives to Swagger that you could recommend that
don't have the mentioned drawback?

It seems that OPen API itself doesn't define primitive and wrapped
types because such information is language/runtime/etc specific. Maybe
this problem will be addressed in the future.

On Mon, May 23, 2022 at 8:55 PM Ilya Kasnacheev
 wrote:
>
> Hello!
>
> Back when I looked at it, Swagger was very primitive, such as not
> supporting primitive types in generated models
> https://stackoverflow.com/a/45053804/36498
>
> I'm not sure it is the right tool, please clarify why it is needed.
>
> -0.5 from me (binding)
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пн, 23 мая 2022 г. в 19:57, Aleksandr Pakhomov :
>
> > Dear community,
> >
> > Discussion about 3rd party dependencies took place
> > and I think it is time to vote if we agreed to include
> > swagger dependency to the Ignite 3 or not.
> >
> > The exact list of dependencies could be fined in IEP-87 [1]
> > (swagger-annotations, swagger-core,
> > swagger-codegen-maven-plugin)
> >
> > Micronaut is out of the scope of this voting. I will launch
> > a separate one.
> >
> > The vote is formal, see voting guidelines [2]
> >
> > +1 - to accept additional dependencies to be included to Java code
> > Guidelines [3]
> > 0 - don't care either way
> > -1 - DO NOT accept (explain why)
> >
> > [1]
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST#IEP87:OpenAPIsupportforREST-Additionaldependencies
> > [2] https://www.apache.org/foundation/voting.html <
> > https://www.apache.org/foundation/voting.html>
> > [3]
> > https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries

Re: [VOTE] Add micronaut dependency to Ignite 3

[Andrey Gura]

-1 (binding) from me.

Because (if I understood correctly) the main value of the IEP-87 is
the possibility to generate API specification and Swagger annotations
is enough for this purpose I don't see reasons for these dependencies.
We already have our own controllers for REST-like API's
implementation. Why can't we just use Swagger annotations only in
addition to our rest-api module?

On Mon, May 23, 2022 at 8:08 PM Aleksandr Pakhomov  wrote:
>
> Dear community,
>
> Micronaut-based REST server implementation was a hot
> topic we discussed in the previous week. So, I've separeted
> votes about swagger and micronaut. This vote is about
> adding micronaut to the Ignite 3.
>
> The exact list of dependencies could be fined in IEP-87 [1]
> io.micronaut.serde:micronaut-serde
> io.micronaut:micronaut-context
> io.micronaut:micronaut-http
> io.micronaut:micronaut-inject
> io.micronaut:micronaut-http-server
> io.micronaut:micronaut-runtime
> io.micronaut:micronaut-core
> io.micronaut:micronaut-http-server-netty
> io.micronaut:micronaut-http-netty
> io.micronaut:micronaut-buffer-netty
> io.micronaut:micronaut-aop
> io.micronaut:micronaut-core-reactive
> Io.micronaut:micronaut-json-core
> io.micronaut:micronaut-jackson-core
>
> Swagger is out of the scope of this voting.
>
> The vote is formal, see voting guidelines [2]
>
> +1 - to accept additional dependencies to be included to Java code Guidelines 
> [3]
> 0 - don't care either way
> -1 - DO NOT accept (explain why)
>
> This vote will be open for at least 4 days till Fri May 27, 2022,
> 21:00 Moscow TZ.
>
> [1] 
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST#IEP87:OpenAPIsupportforREST-Additionaldependencies
> [2] https://www.apache.org/foundation/voting.html 
> 
> [3] 
> https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries
>  
> 
> [4] 
> https://www.timeanddate.com/countdown/generic?iso=20220527T21=166=%5BVOTE%5D+Add+micronaut+dependency+to+Ignite+3=cursive

Re: [DISCUSSION] Add additional 3rd party libraries to Ignite 3 Code style and practices

[Andrey Gura]

> The main
functionality of micronaut framework is REST, so this
library is scanned for vulnerabilities regularly and fixes
them asap (looking to [1] it takes about a week).
I don't  think that Ignite REST server implementation
will be scanned as regular as micronaut and issues
will be fixed as quickly as micronaut does.

The problem is not the scan frequency. The actual problem is the
release cycle: somebody should fix a vulnerability in some dependency,
then that dependency should be released and in the end Ignite should
refresh dependencies and then should be released.

In absence of dependencies we just fix the problem and build a
maintenance release.

> As for autogenerated spec, I would mention that manual
spec writing introduces the second source of truth for
the API. So, implementation declares one API, spec
declares another API and there is no prooved contract
between them.

>From my point of view this is a bogus problem. Of course the described
situation is possible and often happens. But the presence of a
generation step does not protect against problems on other levels. I
can describe the interface correctly but implementation of this
interface can be wrong. So such instruments give just an illusion of
correctness. Where is the truth? In fact, no one knows.

On Fri, May 20, 2022 at 3:39 PM Aleksandr Pakhomov  wrote:
>
> Hi Andrey,
>
> Thank you for the valuable arguments.
>
> Speaking about micronaut, it is a popular library that
> provides a lot of build-in features like error handling,
> auth, IoC, test infrastructure, and many more. The main
> functionality of micronaut framework is REST, so this
> library is scanned for vulnerabilities regularly and fixes
> them asap (looking to [1] it takes about a week).
> I don't  think that Ignite REST server implementation
> will be scanned as regular as micronaut and issues
> will be fixed as quickly as micronaut does.
>
> As for autogenerated spec, I would mention that manual
> spec writing introduces the second source of truth for
> the API. So, implementation declares one API, spec
> declares another API and there is no prooved contract
> between them. For example, a developer adds "name"
> parameter to the existing endpoint and goes to the
> spec and adds "names" parameter there (makes a mistake).
> What is the right parameter here "name" or "names"?
> Also, if there won't be a compatibility test this mistake could
> go to the production and API spec consumers could get
> a REST client that is not compatible with the server.
>
>
> > On 19 May 2022, at 00:32, Andrey Gura  wrote:
> >
> > I personally don't support any additional 3rd party dependencies as
> > well as I don't fully understand the value of autogenerated specs for
> > REST endpoints. In my opinion we have another option: writing spec
> > manually. This option doesn't require any of proposed dependencies and
> > allow to avoid possible vulnerabilities. Of course at the cost of
> > manual actions.
> >
> > I understand that my statement is arguable. So I'll just wait for
> > opinions of other community members.
>

Re: [DISCUSSION] IEP-88: CLI Tool

[Andrey Gura]

Aleksandr,

Both proposed options look good to me because both cases assume that a
user must express their intent explicitly.

On Thu, May 19, 2022 at 10:53 AM Aleksandr Pakhomov  wrote:
>
> I got it. What do you think about this proposal:
>
> -  “ignite”  prints help
> -  “ignite shell” enters REPL
>
> Or
>
> -  “ignite” prints help
> -  “ignite-shell” enters REPL and it is a separate application
>
> I prefer the first varian but I would like to hear opinions of other 
> community members.
>
>
> > On 19 May 2022, at 01:16, Andrey Gura  wrote:
> >
> > I can just have a mistake in my script, e.g. running ignite command
> > without any parameters. What will happen in such a case from the
> > script perspective? I think the script will wait for returning value
> > while the shell will wait for a user input. Due to a server-side
> > nature of the script it will hang forever because there is no user on
> > the server side.
>

Re: [DISCUSS] Query API for Ignite 3.0

[Andrey Gura]

Andrey,

there is no IEP for Java API for SQL and would be great to have it.
I'll create the IEP soon.

On Fri, May 20, 2022 at 1:56 PM Andrey Mashenkov
 wrote:
>
> JFYI. we have merged the initial version of SQL public API [1] and are
> going to implement it in epic [2] and I've created few tickets on this.
>
> Andey Gura, Taras Ledkov, Konstantine Orlov would you mind to share your
> thoughts on implementation part in some IEP?
> Is there IEP page created on the topic?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-15212
> [2] https://issues.apache.org/jira/browse/IGNITE-16952
>
> On Tue, Apr 6, 2021 at 4:23 PM Andrey Mashenkov 
> wrote:
>
> > Hi Taras.
> >
> > 1. AFAIK, only Thin clients will be available in 3.0.
> > However, yes, Native JDBC API is still "must have" on servers.
> >
> > 2. We won't have other projects in dependencies if possible. Unless
> > we/they are Jigsaw compatible though.
> >
> > 2.1  I think it makes sense to be an R2DBC-compatible as it is a widely
> > used framework. E.g. Spring supports R2DBC [1].
> > Having an extension with R2DBC support will make Spring integration with
> > Ignite easy in the future.
> >
> > If it will be enough to have a dependency on just reactive-streams API [2],
> > then adding this API to dependencies looks ok to me, as there are only a
> > few interfaces and converters to/from JDK Flow API.
> > And the question is would we go with Flow API or Reactive-Streams?
> >
> > If we need some more than reactive-streams API, then I would suggest to go
> > with Flow API.
> >
> > 2.2 ADBA is compatible with R2DBC but was discontinued [3].
> >
> > 2.3 I don't think custom API is ever useful. We will need
> > converters/adapters to Flow or Reactive-streams for easy integration with
> > other APIs/frameworks.
> > Otherwise, uses will have difficulties interacting with Ignite in a
> > reactive way.
> >
> > [1]
> > https://spring.io/blog/2018/12/07/reactive-programming-and-relational-databases
> > [2] https://github.com/reactive-streams/reactive-streams-jvm
> > [3]
> > https://mail.openjdk.java.net/pipermail/jdbc-spec-discuss/2019-September/000529.html
> >
> > On Tue, Apr 6, 2021 at 3:05 PM Taras Ledkov  wrote:
> >
> >> Hi,
> >>
> >> Ignite 3.0 requires a rethinking of the query API.
> >> We have 'cache.query()' and 'SqlFieldQuery' abstractions at the Ignite
> >> 2.x native API and several JDBC implementation for clients.
> >>
> >> I propose to think about new query/SQL API for the Ignite 3.0.
> >>
> >> My vision is something like this:
> >> Ignite will support two query APIs: standard JDBC (on native) and one of
> >> reactive DB API.
> >>
> >> 1. Native JDBC API, e.g.:
> >>  Connection conn = node.sql().connection(props);
> >>
> >> JDBC is the industrial standard of the DB access and we have to support
> >> one.
> >> Also:
> >> 1.1. Thin JDBC client will be really thin: provide network communication
> >> layer and transparently map to native API.
> >> 1.2. Thick JDBC client implementation will be trivial: start client node
> >> and open JDBC connection on the started node.
> >> 1.3. JDBC provides sufficient functionality to implement ODBC (need to
> >> investigate: may be thin protocol may be extended to unify JDBC and ODBC).
> >>
> >> 2. About reactive DB API.
> >> I don't know of any industrial standard API for DB reactive access now.
> >> There are several candidates:
> >> 2.1. R2DBC look like the popular and alive. See [1];
> >> 2.2. ADBA (java.sql2 [2]) looks like not alive. Not sure;
> >> 2.3. Custom async/reactive API.
> >> e.g. oracle DB use this way [3].
> >>
> >> Igniters, WDYT?
> >>
> >> [1]. https://github.com/r2dbc/r2dbc-spi
> >> [2].
> >> http://cr.openjdk.java.net/~lancea/apidoc/java/sql2/package-summary.html
> >> [3].
> >>
> >> https://docs.oracle.com/en/database/oracle/oracle-database/21/jjdbc/jdbc-reactive-extensions.html#GUID-1C40C43B-3823-4848-8B5A-D2F97A82F79B
> >>
> >> --
> >> Taras Ledkov
> >> Mail-To: tled...@gridgain.com
> >>
> >>
> >
> > --
> > Best regards,
> > Andrey V. Mashenkov
> >
>
>
> --
> Best regards,
> Andrey V. Mashenkov

Apache Ignite 3.0.0 alpha 5 RELEASE [Time, Scope, Manager]

[Andrey Gura]

Hi Igniters,

Four months have passed already since the Ignite 3 alpha 4 release. At
the moment we have a set of features that can be released in order to
give to a user an ability to try the features and share some feedback
with the community. The expected feature list consists of:

  - Pluggable storages: ability to choose a specific storage for a
table (LSM based storage, Page memory persistent and in-memory
storage) with some known limitations.
  - Compute API (A simple remote job execution): The first phase of
Compute API design and implementation. Of course, with known
limitations.
  - Data colocation: The colocation key concept replaces the affinity
key concept. DDL introduces COLOCATE BY clause. Colocated job
execution.
  - Open API for the Ignite REST endpoints: A Specification to
generate a client for any language + auto-generated docs for REST API.
  - Ignite REPL: The Ignite CLI as a REPL with autocompletion and improves UX.
  - Cluster lifecycle: It introduces cluster initialization logic and
allows to specify cluster management and meta storage groups. Improved
node join protocol.
  - Local and distributed recovery: Now it is possible to restart a
cluster/node without data loss.
  - Data rebalance improvements (in progress and could be excluded
from the release), including dynamically changing the number of
partition replicas.
  - Robust client connection with seamless reconnection support and
retry policies.
  - Java API for SQL: A simplified API (design only) for executing SQL
queries on a cluster.

I want to propose myself to be the release manager of the Ignite 3 alpha 5.

Also I propose the following milestones for the release:

Scope Freeze: June 1, 2022
Code Freeze: June 3, 2022
Voting Date: June 6, 2022
Release Date: June 10, 2012

Please, take into account that the proposed release is still alpha, so
we can afford to have such a compressed schedule.

WDYT?

Re: IEP-90 Ignite 3 Client Lifecycle

[Andrey Gura]

Thanks for the answers, Igor.

On Thu, May 19, 2022 at 10:55 PM Igor Sapego  wrote:
>
> Andrey,
>
> 1. If a server generates a UUID that already exists it can check and just
> re-generate it straight away
> as a check is just a simple map lookup.
>
> 2. Well, yes. This proposal does not consider monitoring, but with current
> approach it will be definitely
> easier to implement it properly.
>
> Pavel,
>
> Wow, that's really cool that you've already started working on it.
>
> Regarding your proposal I like it - it will really make a procedure easier
> and faster for a client. I'll change
> the IEP accordingly.
>
> Best Regards,
> Igor
>
>
> On Thu, May 19, 2022 at 11:40 PM Pavel Tupitsyn 
> wrote:
>
> > Igor,
> >
> > I've started working on the initial sessions implementation [1],
> > and I'd like to clarify the procedure of logical connection restore.
> >
> > > client in its turn tries to establish a new *node connection*
> > > and restore *logical connection *using ConnectionRestoreReq
> >
> > This implies either an additional request, or something that replaces
> > normal handshake.
> > However, we need to handle two cases here:
> > - Logical connection is restored
> > - Logical connection is not restored (timed out, server restarted, etc), in
> > which case we still establish the connection and use it.
> >
> > To account for the second case, we should always start with a regular
> > handshake.
> > I propose to add a nullable UUID field to the handshake request to cover
> > both cases:
> > - connectionId is null or not found on server: proceed with normal
> > handshake, return newly generated connectionId.
> > - connectionId is not null and found on server: restore logical connection,
> > return the same connectionId
> >
> > Client checks if returned connectionId equals to the original and
> > understands whether logical connection was restored or not.
> >
> > Thoughts?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-16928
> >
> > On Thu, May 19, 2022 at 10:27 PM Pavel Tupitsyn 
> > wrote:
> >
> > > Andrey,
> > >
> > > > different connections on different client instances theoretically
> > > > could generate an already existing connection ID
> > >
> > > Do you mean an UUID collision?
> > >
> > > On Thu, May 19, 2022 at 1:09 AM Andrey Gura  wrote:
> > >
> > >> Igor,
> > >>
> > >> Thanks for the proposal.
> > >>
> > >> I understand that such a situation is almost impossible but "if
> > >> anything can go wrong, it will". Does the protocol take into account
> > >> that different connections on different client instances theoretically
> > >> could generate an already existing connection ID?
> > >>
> > >> Also, do I understand correctly that a server has enough information
> > >> about client connections so it will be possible to observe a
> > >> connections list on the server? It would be useful for cluster
> > >> monitoring purposes.
> > >>
> > >> On Tue, May 17, 2022 at 3:11 PM Igor Sapego  wrote:
> > >> >
> > >> > 1. I've tried to clarify IDs part;
> > >> > 2. Maybe you are right, though in this case we'd need to use
> > >> authentication
> > >> > in ConnectionRestoreReq. Which sounds reasonable to me.
> > >> >
> > >> > Best Regards,
> > >> > Igor
> > >> >
> > >> >
> > >> > On Tue, May 17, 2022 at 10:47 AM Pavel Tupitsyn  > >
> > >> > wrote:
> > >> >
> > >> > > Igor,
> > >> > >
> > >> > > The proposal looks good to me. Very detailed!
> > >> > >
> > >> > > A couple comments:
> > >> > >
> > >> > > 1. There is a bit of a term mixup with "Connection ID", "Node ID",
> > >> "Token"
> > >> > > - can you please review those?
> > >> > >
> > >> > > 2. > The Connection ID should be generated using a proper secure
> > >> algorithm
> > >> > > (additional research is required here) to make sure an intruder can
> > >> not
> > >> > > generate an existing Connection ID
> > >> > > Not sure about the reasoning here. I think randomUUID() should be
> > >> eno

Re: [DISCUSSION] IEP-88: CLI Tool

[Andrey Gura]

> 1. Does “Ignite shell” sounds better?

I don't know. It could be Ignite REPL, IGnite Shell or something else.
Just an idea.

> But I did not get your point about scripting. What do you mean by saying “it 
> will just hang”?  What kind of error is going to be the cause of such 
> behaviour?

I can just have a mistake in my script, e.g. running ignite command
without any parameters. What will happen in such a case from the
script perspective? I think the script will wait for returning value
while the shell will wait for a user input. Due to a server-side
nature of the script it will hang forever because there is no user on
the server side.

On Thu, May 19, 2022 at 1:02 AM Aleksandr Pakhomov  wrote:
>
> Hi, Andrey
>
> Thanks for comments.
>
> 1. Does “Ignite shell” sounds better?
>
> 2. Yes, you are right. Most tools print help if they are executed without 
>   arguments. But I did not get your point about scripting. What do you mean 
> by saying “it will just hang”?  What kind of error is going to be the cause 
> of such behaviour?
>
> > On 18 May 2022, at 21:14, Andrey Gura  wrote:
> >
> > Hi,
> >
> > My two cents:
> >
> > 1. CLI Tool - looks like not the best name :) Shell?
> >
> > 2. The description says: "REPL mode is used by default and is
> > activated if the ignite command is executed without parameters."
> >
> > I think it is a bad idea. Firstly, it is usual CLI's behaviour to
> > print a help for a user. An exception if we would use a special CLI
> > for REPL. Shell? :)
> > Also such approach breaks my expectations related to a scripting. In
> > case of an error in a script it will just hang. So, a separate CLI
> > utility looks better than ignite CLI.
> >
> >
> > On Thu, May 12, 2022 at 11:43 PM Aleksandr Pakhomov  
> > wrote:
> >>
> >> Hello, Igniters.
> >>
> >> I’d like to start a discussion about Ignite 3 Command Line Tool [1]. The 
> >> main value is to develop a user-friendly command-line tool with advanced 
> >> completions and SQL REPL mode.
> >>
> >> The set of commands and parameters can be discussed. Questions and 
> >> comments are welcomed.
> >>
> >> [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-88%3A+CLI+Tool 
> >> <https://cwiki.apache.org/confluence/display/IGNITE/IEP-88:+CLI+Tool>
>

Re: IEP-90 Ignite 3 Client Lifecycle

[Andrey Gura]

Igor,

Thanks for the proposal.

I understand that such a situation is almost impossible but "if
anything can go wrong, it will". Does the protocol take into account
that different connections on different client instances theoretically
could generate an already existing connection ID?

Also, do I understand correctly that a server has enough information
about client connections so it will be possible to observe a
connections list on the server? It would be useful for cluster
monitoring purposes.

On Tue, May 17, 2022 at 3:11 PM Igor Sapego  wrote:
>
> 1. I've tried to clarify IDs part;
> 2. Maybe you are right, though in this case we'd need to use authentication
> in ConnectionRestoreReq. Which sounds reasonable to me.
>
> Best Regards,
> Igor
>
>
> On Tue, May 17, 2022 at 10:47 AM Pavel Tupitsyn 
> wrote:
>
> > Igor,
> >
> > The proposal looks good to me. Very detailed!
> >
> > A couple comments:
> >
> > 1. There is a bit of a term mixup with "Connection ID", "Node ID", "Token"
> > - can you please review those?
> >
> > 2. > The Connection ID should be generated using a proper secure algorithm
> > (additional research is required here) to make sure an intruder can not
> > generate an existing Connection ID
> > Not sure about the reasoning here. I think randomUUID() should be enough:
> > - In the case of an unsecured cluster it does not matter, because anyone
> > can do anything.
> > - In the case of a secured cluster it does not matter, because
> > authentication/authorization keeps intruders out.
> >
> >
> > On Mon, May 16, 2022 at 11:07 PM Igor Sapego  wrote:
> >
> > > Hi, Igniters
> > >
> > > I've prepared an IEP for Ignite 3 Client Lifecycle [1]. The main idea is
> > to
> > > define client lifecycle as well as core algorithms and mechanisms used by
> > > clients. This proposal can be used as a reference for implementation of a
> > > new client for Ignite when dealing with such problems as:
> > >
> > >- Resolving of user-provided addresses;
> > >- Initial connection to a cluster;
> > >- Maintaining cluster connection;
> > >- Connection recovery;
> > >- Connection break handling.
> > >
> > > So take a look and let me know what you think guys.
> > >
> > > [1] -
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-90+Client+Lifecycle
> > >
> > > Best Regards,
> > > Igor
> > >
> >

Re: [DISCUSSION] Add additional 3rd party libraries to Ignite 3 Code style and practices

[Andrey Gura]

I personally don't support any additional 3rd party dependencies as
well as I don't fully understand the value of autogenerated specs for
REST endpoints. In my opinion we have another option: writing spec
manually. This option doesn't require any of proposed dependencies and
allow to avoid possible vulnerabilities. Of course at the cost of
manual actions.

I understand that my statement is arguable. So I'll just wait for
opinions of other community members.

On Mon, May 16, 2022 at 7:45 PM Aleksandr Pakhomov  wrote:
>
>
> Hello, Igniters.
>
> I would like to start a discussion about Java Code Style Guide [1] changes 
> that are going to be a part of IEP-87 [2] implementation. The set of 
> libraries and frameworks that are going to be allowed to be used in 
> production:
> - Micronaut for REST Server [3]
> - Swagger for Open API annotations [4]
> - Micronaut serde for REST serialization [5]
>
> Any objections? Also, comments on IEP-87 are welcomed.
> [1] 
> https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries
>  
> 
> [2] 
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST#IEP87:OpenAPIsupportforREST-Additionaldependencies
>  
> 
> [3] https://micronaut.io/docs /
> [4] https://swagger.io 
> [5] 
> https://micronaut-projects.github.io/micronaut-serialization/snapshot/guide 
> /

Re: [DISCUSSION] IEP-88: CLI Tool

[Andrey Gura]

Hi,

My two cents:

1. CLI Tool - looks like not the best name :) Shell?

2. The description says: "REPL mode is used by default and is
activated if the ignite command is executed without parameters."

I think it is a bad idea. Firstly, it is usual CLI's behaviour to
print a help for a user. An exception if we would use a special CLI
for REPL. Shell? :)
Also such approach breaks my expectations related to a scripting. In
case of an error in a script it will just hang. So, a separate CLI
utility looks better than ignite CLI.


On Thu, May 12, 2022 at 11:43 PM Aleksandr Pakhomov  wrote:
>
> Hello, Igniters.
>
> I’d like to start a discussion about Ignite 3 Command Line Tool [1]. The main 
> value is to develop a user-friendly command-line tool with advanced 
> completions and SQL REPL mode.
>
> The set of commands and parameters can be discussed. Questions and comments 
> are welcomed.
>
> [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-88%3A+CLI+Tool 
> 

Re: [DISCUSSION] IEP-87: Open API support for REST

[Andrey Gura]

Hi,

I took a look at the proposal and have some questions and comments.

1. It is not clear what the main value of this proposal is. The
current implementation of REST is code-first. API specification could
be written manually. It seems that the main value is the possibility
to generate an API specification from code. If so, then it would be
great to point it out strongly. Otherwise, this proposal looks like an
attempt to replace one implementation by another one (may be more
popular) but with additional 3rd party dependencies. Also, it would be
great to propose a process of artefact management (what should we do
and when) in relation to specification (Where it should be published?
Should it be placed in a source code repository or not? Should we do
some version management?)

2. The "Modular architecture support" part says: "Ignite modules can
provide endpoints that will be included into the netty server by
RestComponent **at the build time**". If I understood correctly, we
talk only about endpoints here, but registration/deregistration of
handlers for known endpoints could be done at runtime. Am I right?

3. The "API" part refers to the meta storage nodes and CMG nodes.
Could you please refer to a document which introduces these concepts?

4. Also it would be great to state that the proposed API actually is
the current API which exists in Apache Ignite.

5. The task about developer documentation should be added to the
issues list. The documentation is a readme.md file which will help an
Ignite developer to understand how to add a new endpoint, how to
generate API specification, etc.

On Wed, May 11, 2022 at 11:17 AM Aleksandr Pakhomov  wrote:
>
> Hello, Igniters.
>
> I’d like to start a discussion about Open API support for REST [1]. The main 
> purpose of this improvement is to add the support of Open API specification 
> by generating it from the source code.
>
> [1] 
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST

Re: [DISCUSSION] Add additional 3rd party libraries to Ignite 3 Code style and practices

[Andrey Gura]

Aleksandr,

it would be better to finish the IEP draft and then start a discussion
thread on dev list. At the moment I see TBD sections and it makes
impossible to have a full view about the proposal.

Thanks!

On Fri, May 6, 2022 at 6:11 PM Aleksandr Pakhomov  wrote:
>
> Hello, Igniters.
>
> I'd like to add a few 3rd party libraries to Ignite 3 Java Code Style Guide 
> [1]. As I mentioned in IEP-87 [2], it will be:
> Micrionaut for REST [3]
> Swagger for Open API annotations [4]
> Micronaut serde for REST serialization [5]
>
> WDYT? Any objections? Also, comments on IEP-87 are welcomed.
>
> [1] 
> https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries
> [2] 
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST#IEP87:OpenAPIsupportforREST-Additionaldependencies
> [3] https://micronaut.io/docs/
> [4] https://swagger.io
> [5] 
> https://micronaut-projects.github.io/micronaut-serialization/snapshot/guide/
>
> --
> Best regards,
> Aleksandr Pakhomov

Re: Ignite 3 IEP

[Andrey Gura]

Permissions are granted. Please, check.

On Fri, May 6, 2022 at 1:32 PM Aleksandr Pakhomov  wrote:
>
> Sure,
>
> apkhmv
>
> > On 6 May 2022, at 13:05, Andrey Gura  wrote:
> >
> > Aleksandr,
> >
> > could you please create an Apache Confluence account [1] and provide
> > your user name?
> >
> > 1. https://cwiki.apache.org/confluence/signup.action
> >
> > On Thu, May 5, 2022 at 7:24 PM Aleksandr Pakhomov  wrote:
> >>
> >> Hello, Igniters. I am working on Ignite 3 REST server and a new CLI tool. 
> >> I’d like to develop IEPs both for REST and CLI. Could you please allow me 
> >> to add new pages to the Confluence page 
> >> https://cwiki.apache.org/confluence/display/IGNITE/Proposals+for+Ignite+3.0
> >>  ?  My email is apk...@gmail.com.
> >>
> >> Best regards,
> >> Aleksandr Pakhomov
>

Re: Ignite 3 IEP

[Andrey Gura]

Aleksandr,

could you please create an Apache Confluence account [1] and provide
your user name?

1. https://cwiki.apache.org/confluence/signup.action

On Thu, May 5, 2022 at 7:24 PM Aleksandr Pakhomov  wrote:
>
> Hello, Igniters. I am working on Ignite 3 REST server and a new CLI tool. I’d 
> like to develop IEPs both for REST and CLI. Could you please allow me to add 
> new pages to the Confluence page 
> https://cwiki.apache.org/confluence/display/IGNITE/Proposals+for+Ignite+3.0 ? 
>  My email is apk...@gmail.com.
>
> Best regards,
> Aleksandr Pakhomov

Re: [DISCUSSION] Update documentation with GitHub Actions

[Andrey Gura]

+1

Great! Thanks!

On Sun, May 1, 2022 at 8:51 AM Pavel Tupitsyn  wrote:
>
> +1, this is awesome
>
> On Fri, Apr 29, 2022 at 3:26 PM Nikita Safonov 
> wrote:
>
> > +1
> >
> > Cannot but support!
> >
> > Thanks,
> > Nikita
> >
> > сб, 23 апр. 2022 г. в 17:47, Maxim Muzafarov :
> >
> > > Hello,
> > >
> > > +1, great improvement.
> > >
> > > On Sat, 23 Apr 2022 at 17:26, Nikita Amelchev 
> > > wrote:
> > > >
> > > > Hello, Igniters.
> > > >
> > > > I propose to add a GitHub action to update documentation for released
> > > > Ignite versions by a click. [1]
> > > >
> > > > For now, this is a complex manual work that requires understanding all
> > > > the intermediate steps. [2]
> > > >
> > > > New process to update documentation:
> > > > 1. Commit documentation changes to a released branch.
> > > > 2. Run the action with a click.
> > > > 3. Check the result.
> > > >
> > > > Moreover, this will also simplify the process of publishing a new
> > > release.
> > > >
> > > > ASF's GitHub Actions Policy allows automated services to push changes
> > > > related to documentation. [3]
> > > > Write access is required to run the workflow. So, only committers can
> > > > run the action. [4]
> > > >
> > > > WDYT? Any objections?
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-16895
> > > > [2]
> > >
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=85461527#HowtoDocument-UpdatingPublishedDocs
> > > > [3] https://infra.apache.org/github-actions-policy.html
> > > > [4]
> > >
> > https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow
> > > >
> > > > --
> > > > Best wishes,
> > > > Amelchev Nikita
> > >
> >

Re: [VOTE] Release Ignite AWS 1.0.0 RC1, Ignite Azure 1.0.0 RC1, Ignite GCE 1.0.0 RC1

[Andrey Gura]

+1

On Fri, Apr 29, 2022 at 7:22 PM Nikita Amelchev  wrote:
>
> +1
>
> пт, 29 апр. 2022 г. в 15:51, Anton Vinogradov :
> >
> > +1
> >
> > On Fri, Apr 29, 2022 at 3:48 PM Maxim Muzafarov  wrote:
> >
> > > Dear Community,
> > >
> > >
> > > The release candidates are ready. See the links below.
> > >
> > >
> > > == Ignite AWS Extension Module 1.0.0 ==
> > >
> > > The release candidate is uploaded to:
> > >
> > > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-aws-ext-1.0.0-rc1
> > >
> > > The following Maven staging can be used:
> > >
> > > https://repository.apache.org/content/repositories/orgapacheignite-1543/org/apache/ignite/ignite-aws-ext/
> > >
> > > Tag:
> > >
> > > https://github.com/apache/ignite-extensions/releases/tag/ignite-aws-ext-1.0.0-rc1
> > >
> > > RELEASE_NOTES:
> > >
> > > https://github.com/apache/ignite-extensions/blob/release/ignite-aws-ext-1.0.0/modules/aws-ext/RELEASE_NOTES.txt
> > >
> > > DEVNOTES:
> > > https://github.com/apache/ignite-extensions/blob/master/DEVNOTES.md
> > >
> > > Release Checker Results:
> > >
> > > https://github.com/apache/ignite-extensions/runs/6227581198?check_suite_focus=true
> > >
> > >
> > > == Ignite Azure Extension Module 1.0.0 ==
> > >
> > > The release candidate is uploaded to:
> > >
> > > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-azure-ext-1.0.0-rc1/
> > >
> > > The following Maven staging can be used:
> > >
> > > https://repository.apache.org/content/repositories/orgapacheignite-1545/org/apache/ignite/ignite-azure-ext/
> > >
> > > Tag:
> > >
> > > https://github.com/apache/ignite-extensions/releases/tag/ignite-azure-ext-1.0.0-rc1
> > >
> > > RELEASE_NOTES:
> > >
> > > https://github.com/apache/ignite-extensions/blob/release/ignite-azure-ext-1.0.0/modules/azure-ext/RELEASE_NOTES.txt
> > >
> > > DEVNOTES:
> > > https://github.com/apache/ignite-extensions/blob/master/DEVNOTES.md
> > >
> > > Release Checker Results:
> > >
> > > https://github.com/apache/ignite-extensions/runs/6228019576?check_suite_focus=true
> > >
> > >
> > > == Ignite GCE Extension Module 1.0.0 ==
> > >
> > > The release candidate is uploaded to:
> > >
> > > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-gce-ext-1.0.0-rc1/
> > >
> > > The following Maven staging can be used:
> > >
> > > https://repository.apache.org/content/repositories/orgapacheignite-1546/org/apache/ignite/ignite-gce-ext/
> > >
> > > Tag:
> > >
> > > https://github.com/apache/ignite-extensions/releases/tag/ignite-gce-ext-1.0.0-rc1
> > >
> > > RELEASE_NOTES:
> > >
> > > https://github.com/apache/ignite-extensions/blob/release/ignite-gce-ext-1.0.0/modules/gce-ext/RELEASE_NOTES.txt
> > >
> > > DEVNOTES:
> > > https://github.com/apache/ignite-extensions/blob/master/DEVNOTES.md
> > >
> > > Release Checker Results:
> > >
> > > https://github.com/apache/ignite-extensions/runs/6228169980?check_suite_focus=true
> > >
> > >
> > > The vote is formal, see voting guidelines
> > > https://www.apache.org/foundation/voting.html
> > >
> > > +1 - to accept (Ignite AWS 1.0.0 RC1, Ignite Azure 1.0.0 RC1, Ignite
> > > GCE 1.0.0 RC1)
> > > 0 - don't care either way
> > > -1 - DO NOT accept (explain why)
> > >
> > > See notes on how to verify release here
> > > https://www.apache.org/info/verification.html
> > > and
> > >
> > > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification
> > >
> > >
> > > This vote will be open for at least 7 days till Fri May 7, 2022,
> > > 14:00 Moscow TZ. Please, write me down the thread if you need
> > > additional time to check the release.
> > >
> > >
> > > https://www.timeanddate.com/countdown/vote?iso=20220507T14=166=%5BVOTE%5D+Release+Ignite+AWS+1.0.0+RC1%2C+Ignite+Azure+1.0.0+RC1%2C+Ignite+GCE+1.0.0+RC1=sanserif
> > >
>
>
>
> --
> Best wishes,
> Amelchev Nikita

Re: [VOTE] Release Extensions Ignite Spring Data 2.2.0 RC1, Ignite Spring Session 1.0.0 RC1, Ignite Zookeeper IP Finder 1.0.0 RC1

[Andrey Gura]

+1

On Fri, Apr 29, 2022 at 7:22 PM Nikita Amelchev  wrote:
>
> +1
>
> пт, 29 апр. 2022 г. в 17:54, Anton Vinogradov :
> >
> > +1
> >
> > On Fri, Apr 29, 2022 at 5:28 PM Maxim Muzafarov  wrote:
> >
> > > Dear Community,
> > >
> > >
> > > The release candidates are ready. See the links below.
> > >
> > >
> > > == Ignite Spring Data Extension Module 2.2.0 ==
> > >
> > > The release candidate is uploaded to:
> > >
> > > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-spring-data-ext-2.2.0-rc1/
> > >
> > > The following Maven staging can be used:
> > >
> > > https://repository.apache.org/content/repositories/orgapacheignite-1547/org/apache/ignite/ignite-spring-data-ext/
> > >
> > > Tag:
> > >
> > > https://github.com/apache/ignite-extensions/releases/tag/ignite-spring-data-ext-2.2.0-rc1
> > >
> > > RELEASE_NOTES:
> > >
> > > https://github.com/apache/ignite-extensions/blob/release/ignite-spring-data-ext-2.2.0/modules/spring-data-ext/spring-data/RELEASE_NOTES.txt
> > >
> > > DEVNOTES:
> > > https://github.com/apache/ignite-extensions/blob/master/DEVNOTES.md
> > >
> > > Release Checker Results:
> > >
> > > https://github.com/apache/ignite-extensions/runs/6229346844?check_suite_focus=true
> > >
> > >
> > >
> > > == Ignite Spring Session Extension Module 1.0.0 ==
> > >
> > > The release candidate is uploaded to:
> > >
> > > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-spring-session-ext-1.0.0-rc1/
> > >
> > > The following Maven staging can be used:
> > >
> > > https://repository.apache.org/content/repositories/orgapacheignite-1548/org/apache/ignite/ignite-spring-session-ext/
> > >
> > > Tag:
> > >
> > > https://github.com/apache/ignite-extensions/releases/tag/ignite-spring-session-ext-1.0.0-rc1
> > >
> > > RELEASE_NOTES:
> > >
> > > https://github.com/apache/ignite-extensions/blob/release/ignite-spring-session-ext-1.0.0/modules/spring-session-ext/RELEASE_NOTES.txt
> > >
> > > DEVNOTES:
> > > https://github.com/apache/ignite-extensions/blob/master/DEVNOTES.md
> > >
> > > Release Checker Results:
> > >
> > > https://github.com/apache/ignite-extensions/runs/6229613223?check_suite_focus=true
> > >
> > >
> > > == Ignite Zookeeper Ip Finder Extension Module 1.0.0 ==
> > >
> > > The release candidate is uploaded to:
> > >
> > > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-zookeeper-ip-finder-ext-1.0.0-rc1/
> > >
> > > The following Maven staging can be used:
> > >
> > > https://repository.apache.org/content/repositories/orgapacheignite-1549/org/apache/ignite/ignite-zookeeper-ip-finder-ext/
> > >
> > > Tag:
> > >
> > > https://github.com/apache/ignite-extensions/releases/tag/ignite-zookeeper-ip-finder-ext-1.0.0-rc1
> > >
> > > RELEASE_NOTES:
> > >
> > > https://github.com/apache/ignite-extensions/blob/release/ignite-zookeeper-ip-finder-ext-1.0.0/modules/zookeeper-ip-finder-ext/zookeeper-ip-finder/RELEASE_NOTES.txt
> > >
> > > DEVNOTES:
> > > https://github.com/apache/ignite-extensions/blob/master/DEVNOTES.md
> > >
> > > Release Checker Results:
> > >
> > > https://github.com/apache/ignite-extensions/runs/6210193576?check_suite_focus=true
> > >
> > >
> > >
> > > The vote is formal, see voting guidelines
> > > https://www.apache.org/foundation/voting.html
> > >
> > > +1 - to accept (Ignite Spring Data 2.2.0 RC1, Ignite Spring Session
> > > 1.0.0 RC1, Ignite Zookeeper IP Finder 1.0.0 RC1)
> > > 0 - don't care either way
> > > -1 - DO NOT accept (explain why)
> > >
> > > See notes on how to verify release here
> > > https://www.apache.org/info/verification.html
> > > and
> > >
> > > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification
> > >
> > >
> > > This vote will be open for at least 7 days till Sat May 7, 2022,
> > > 14:00 Moscow TZ. Please, write me down the thread if you need
> > > additional time to check the release.
> > >
> > >
> > > https://www.timeanddate.com/countdown/vote?iso=20220507T14=166=%5BVOTE%5D+Release+Extensions+Ignite+Spring+Data+2.2.0+RC1%2C+Ignite+Spring+Session+1.0.0+RC1%2C+Ignite+Zookeeper+IP+Finder+1.0.0+RC1=sanserif
> > >
>
>
>
> --
> Best wishes,
> Amelchev Nikita

[DISCUSSION] IEP-86: Colocation Key (Apache Ignite 3.0)

[Andrey Gura]

Igniters,

I'd like to start a discussion about API changes related to the data
colocation in Apache Ignite 3.0. [1]

The main purpose of the proposal is a definition of tables' colocation
from a user perspective including DDL syntax and changes related to
used terminology.

While proposed changes are already implemented partially we still can
discuss these changes and revisit some aspects.


[1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-86:+Colocation+Key

[ANNOUNCE] New wiki section for Apache Ignite 3 contributors

[Andrey Gura]

Igniters,

The new wiki section for Apache Ignite 3 contributors is created. This
page [1] is the main entry point for new and existing contributors and
the page provides information about differences in the development
process (e.g. code style, practices, etc), the product architecture,
technical details, etc.

At the moment the Java Code Style Guide page [2] contains information
about Apache Ignite 3 code style and some additional rules discussed
on dev-list.

Please, feel free to contribute appropriate content to this wiki
section. It would be great to have pages about project setup, best
coding practices for developers, etc.

1. https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3
2. https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide

Re: [VOTE] Release Apache Ignite 3.0.0-alpha4 RC1

[Andrey Gura]

+1 (binding)

On Wed, Jan 26, 2022 at 4:46 PM Denis Magda  wrote:
>
> +1 (binding)
>
> —
> Denis
>
> > On Jan 25, 2022, at 2:44 PM, Valentin Kulichenko 
> >  wrote:
> >
> > Dear Community,
> >
> > Ignite 3 is moving forward and I think we're in a good spot to release
> > another alpha version. In the last few month the following major features
> > have been added:
> >
> >   - Transactional API
> >   - Record and KeyValue views with POJO mapping support
> >   - DDL
> >
> > This is a significant milestone, so I think it will be valuable to share
> > the current state with the broader community. I propose to release 3.0.0
> > Alpha4 with the features listed above.
> >
> > Please vote.
> >
> > ---
> >
> > Release candidate:
> > https://dist.apache.org/repos/dist/dev/ignite/3.0.0-alpha4-rc1/
> > Maven staging:
> > https://repository.apache.org/content/repositories/orgapacheignite-1540/
> > Git tag: https://github.com/apache/ignite-3/tree/3.0.0-alpha4-rc1
> >
> > +1 - accept Apache Ignite 3.0.0-alpha4 RC1
> > 0 - don't care either way
> > -1 - DO NOT accept Apache Ignite 3.0.0-alpha4 RC1 (explain why)
> >
> > Voting guidelines: https://www.apache.org/foundation/voting.html
> > How to verify the release: https://www.apache.org/info/verification.html
> >
> > The vote will be open for 72 hours and will close on January 28th at 8pm
> > UTC: https://www.timeanddate.com/countdown/to?iso=20220128T12=224
> >
> > -Val

Re: [ANNOUNCE] Welcome Semyon Danilov as a new committer

[Andrey Gura]

I'm sorry for the mistake. Of course please join me in welcoming
**Semyon**, and congratulating him on the new role in
the Apache Ignite Community.

On Wed, Dec 1, 2021 at 8:51 AM Zhenya Stanilovsky
 wrote:
>
>
> Wellcome Semyon !
> Andrey what`s Ivan are you talking at the end of the message or this is some 
> kind of phraseologism that all russians are ivans ?:)
>
> >Igniters,
> >
> >The Apache Ignite Project Management Committee (PMC) has invited
> >Semyon Danilov to become a new committer and are happy to announce
> >that he
> >has accepted.
> >
> >Semyon contributes actively to the project's code base (AI 2 & 3) and
> >helps a lot with the development environment set up (you know, all
> >this Maven and plugins related issues).
> >
> >Being a committer enables easier contribution to the project since there is
> >no need to go via the patch submission process. This should enable better
> >productivity.
> >
> >Please join me in welcoming Ivan, and congratulating him on the new role in
> >the Apache Ignite Community.
> >
> >Best Regards,
> >Andrey Gura
>
>
>
>

[ANNOUNCE] Welcome Semyon Danilov as a new committer

[Andrey Gura]

Igniters,

The Apache Ignite Project Management Committee (PMC) has invited
Semyon Danilov  to become a new committer and are happy to announce
that he
has accepted.

Semyon contributes actively to the project's code base (AI  2 & 3) and
helps a lot with the development environment set up (you know, all
this Maven and plugins related issues).

Being a committer enables easier contribution to the project since there is
no need to go via the patch submission process. This should enable better
productivity.

Please join me in welcoming Ivan, and congratulating him on the new role in
the Apache Ignite Community.

Best Regards,
Andrey Gura

Re: [DISCUSSION] Add SystemRegionMetricsEnabled property on DataStorageConfiguration.

[Andrey Gura]

Hi,

Your proposal is consistent with the current interface of
DataStorageConfiguration class, so I', okay with the proposed change.

Alternatively, DataStorageConfiguration#systemDataRegionConfiguration
field could be introduced which could be DataRegionConfiguration type
or special SystemDataRegionConfiguration type in case if we want to
limit access to the system data region parameters.

On Tue, Nov 2, 2021 at 2:42 PM Eduard Rakhmankulov  wrote:
>
> Hello, Igniters!
>
> At the moment metrics on system region are disabled on node start.
> User-defined regions can be tuned to have enabled metrics via region
> configuration
> (org.apache.ignite.configuration.DataRegionConfiguration#setMetricsEnabled)
> To enable metrics on system region users should use non-public API or use
> deprecated JMX bean
> (org.apache.ignite.mxbean.DataRegionMetricsMXBean#enableMetrics) for it,
> which doesn't solve problem because it needs human attention when node
> restarts.
>
> I want to add a new property to
> *org.apache.ignite.configuration.DataStorageConfiguration* which enables or
> disables collection of metrics on system region (aka sysMemPlc) on node
> start.
>
> API change
> add new public methods:
>
>1.
>
> *org.apache.ignite.configuration.DataStorageConfiguration#isSystemRegionMetricsEnabled()*
>2.
>
> *org.apache.ignite.configuration.DataStorageConfiguration#setSystemRegionMetricsEnabled(boolean)*
>
>
> WDYT?
> --
> Best regards, Eduard.

Re: [VOTE] Create separate Jira project and Confluence space for Ignite 3

[Andrey Gura]

+1

On Wed, Oct 6, 2021 at 8:16 PM Pavel Tupitsyn  wrote:
>
> -1
>
> We have already started using the existing JIRA and Confluence.
> There are lots of tickets, IEPs, and commits interlinked with each other.
> Introducing a new JIRA and Confluence will cause a huge mess.
>
> On Wed, Oct 6, 2021 at 7:57 PM Denis Magda  wrote:
>
> > Val,
> >
> > As we discussed earlier, I don't fully support this idea but if what you
> > propose is the only reasonable and efficient way to carry on with Ignite 3
> > and 2 development efforts, then I trust your word. But even if this vote
> > passes I still would encourage you to experiment with the alternative - a
> > mandatory "Architecture" field in JIRA that requires to pick between
> > "Ignite 2" and "Ignite 3". Talk to the ASF INFRA team, there still might be
> > the way to keep a single JIRA project and simplify the development efforts.
> >
> > -
> > Denis
> >
> >
> > On Wed, Oct 6, 2021 at 3:03 AM Dmitriy Pavlov  wrote:
> >
> > > -0 from my side.
> > >
> > > That's fine if we have a separate project.
> > > But since the mission and the statement of community is the same, I tend
> > to
> > > see 2&3 as the same project. The only difference is that 3 was started
> > > from scratch (from the code point of view).
> > >
> > > ср, 6 окт. 2021 г. в 00:23, Maxim Muzafarov :
> > >
> > > > - 0 from me.
> > > >
> > > > This is OK if the projects are different, but not OK if they are
> > > > sharing the same version history.
> > > >
> > > > On Tue, 5 Oct 2021 at 13:39, Andrey Mashenkov
> > > >  wrote:
> > > > >
> > > > > +1
> > > > >
> > > > > вт, 5 окт. 2021 г., 13:33 Юрий :
> > > > >
> > > > > > +1
> > > > > >
> > > > > > вт, 5 окт. 2021 г. в 02:52, Valentin Kulichenko <
> > > > > > valentin.kuliche...@gmail.com>:
> > > > > >
> > > > > > > Hello Community,
> > > > > > >
> > > > > > > As discussed in [1], I would like to propose the creation of a
> > > > separate
> > > > > > > Jira project and Confluence space for Ignite 3.
> > > > > > >
> > > > > > > Ignite 2 and Ignite 3 are developed in parallel in separate
> > repos,
> > > > so we
> > > > > > > need a clear separation in other tools as well - this will help
> > to
> > > > > > > streamline the development process. Please refer to the
> > discussion
> > > > for
> > > > > > more
> > > > > > > details.
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> > https://lists.apache.org/thread.html/rdcad3fc64b9f3a848c93089baae2bee1124a97869a94f4a04dd80fdf%40%3Cdev.ignite.apache.org%3E
> > > > > > >
> > > > > > > Voting options:
> > > > > > >
> > > > > > >- +1 - Agree with the suggestion
> > > > > > >- 0 - Don't care much about the suggestion
> > > > > > >- -1 - Disagree with the suggestion
> > > > > > >
> > > > > > > This is a majority vote.
> > > > > > >
> > > > > > > Voting ends in 72 hours, at 5pm PDT on October 7:
> > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> > https://www.timeanddate.com/counters/fullscreen.html?mode=a=20211007T17=2021=10=7=17=0=0=224
> > > > > > >
> > > > > > > -Val
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Живи с улыбкой! :D
> > > > > >
> > > >
> > >
> >

Re: Ban Java Streams usage in Ignite 3 code

[Andrey Gura]

Agree that any additional object creation on a hot path could be a
problem. So hot path should not contain stream API and any other
potentially problem code (e.g. iterator instead of for by index).

On Wed, Sep 8, 2021 at 1:45 PM Pavel Tupitsyn  wrote:
>
> Ok, maybe a total ban is overkill, but right now streams are used even on
> some hot paths like getAllAsync [1].
>
> Another issue is that Collectors.toList() and other variants don't accept
> capacity, and we end up with unnecessary reallocations of underlying arrays.
>
> [1]
> https://github.com/apache/ignite-3/blob/1d7d703ff2b18234b15a9a7b03104fbb73388edf/modules/table/src/main/java/org/apache/ignite/internal/table/KVBinaryViewImpl.java#L83
>
> On Wed, Sep 8, 2021 at 1:06 PM Konstantin Orlov  wrote:
>
> > Hi!
> >
> > Agree with Ivan that it’s an overkill. Code readability and
> > maintainability should have
> > the same priority as the performance (with some exceptions of course).
> >
> > BTW the result of your benchmark looks quite strange. The performance
> > penalty on
> > my laptop (Core i7 9750H, 6 cores up to 4.50 GHz) is 25%, not 8 times:
> >
> > Benchmark Mode  Cnt  Score Error   Units
> > JmhIncrementBenchmark.loopSumthrpt   10  32347.819 ± 676.548  ops/ms
> > JmhIncrementBenchmark.streamSum  thrpt   10  24459.196 ± 610.152  ops/ms
> >
> >
> > --
> > Regards,
> > Konstantin Orlov
> >
> >
> > > On 8 Sep 2021, at 12:23, Ivan Bessonov  wrote:
> > >
> > > Hello Igniters,
> > >
> > > I object, banning streams is an overkill. I would argue that most of the
> > > code
> > > is not on hot paths and that allocations in TLAB don't create much
> > pressure
> > > on GC.
> > >
> > > Streams must be used cautiously, developers should know whether they
> > > write hot methods or not. And if methods are not hot, code simplicity
> > must
> > > be
> > > the first priority. I don't want Ignite 3 code to look like Ignite 2
> > code,
> > > where
> > > people would iterate over Lists using explicit access by indexes,
> > because it
> > > saves them a single Iterator allocation. That's absurd.
> > >
> > > ср, 8 сент. 2021 г. в 11:43, Pavel Tupitsyn :
> > >
> > >> Igniters,
> > >>
> > >> Java streams are known to be slower and cause more GC pressure than an
> > >> equivalent loop.
> > >> Below is a simple filter/map/reduce scenario (code [1]):
> > >>
> > >> * Benchmark Mode
> > Cnt
> > >>Score Error   Units
> > >>
> > >> * StreamVsLoopBenchmark.loopSum thrpt
> >   3
> > >> 7987.016 ± 293.013  ops/ms
> > >> * StreamVsLoopBenchmark.loopSum:·gc.alloc.rate  thrpt
> >   3
> > >>   ≈ 10⁻⁴MB/sec
> > >> * StreamVsLoopBenchmark.loopSum:·gc.count   thrpt
> >   3
> > >>  ≈ 0counts
> > >>
> > >> * StreamVsLoopBenchmark.streamSum   thrpt
> >   3
> > >> 1060.244 ±  36.485  ops/ms
> > >> * StreamVsLoopBenchmark.streamSum:·gc.alloc.ratethrpt
> >   3
> > >>  315.819 ±  10.844  MB/sec
> > >> * StreamVsLoopBenchmark.streamSum:·gc.count thrpt
> >   3
> > >>   55.000counts
> > >>
> > >> Loop is several times faster and does not allocate at all.
> > >>
> > >> 1. Performance is one of the most important features of our product.
> > >> 2. Most of our APIs will be on the hot path.
> > >>
> > >> One can argue about performance differences in real-world scenarios,
> > >> but increasing GC pressure just to make the code a little bit nicer is
> > >> unacceptable.
> > >>
> > >> I propose to ban streams usage in the codebase (except for the tests).
> > >>
> > >> Thoughts, objections?
> > >>
> > >> [1] https://gist.github.com/ptupitsyn/5934bbbf8f92ac4937e534af9386da97
> > >>
> > >
> > >
> > > --
> > > Sincerely yours,
> > > Ivan Bessonov
> >
> >

Re: [VOTE] Allow or prohibit usages of the Guava library methods

[Andrey Gura]

-1
Potential compatibility issues which are related to user specific code
(services, jobs).
Potential vulnerabilities which are a problem because we can't manage
the Guava release cycle.
Arguable approach with unnecessary immutability and unmanageable GC pressure.
It's too hard to manage the *reasonable* usage of this library.

On Tue, Sep 7, 2021 at 6:31 PM Alexander Polovtcev
 wrote:
>
> +1
> We already have some utility methods that are copy-pasted from Guava, which
> can be removed, and I believe that *reasonable* usage of this library can
> benefit the code and the product quality.
>
> On Tue, Sep 7, 2021 at 6:27 PM Alexander Polovtcev 
> wrote:
>
> > Dear Igniters,
> >
> > In this thread
> > 
> > we've been discussing the problems and opportunities of using Guava
> >  in Ignite 3. We have agreed that it
> > should be added as a shaded dependency, but we haven't decided whether to
> > allow using Guava methods in the Ignite codebase or not. Therefore I would
> > like to propose a vote:
> >
> > [+1 Allow]: allow using Guava methods, if justified.
> > [-1 Prohibit]: prohibit using all Guava methods.
> >
> > The voting will commence on Monday, September 13th at 9:00 UTC. Also feel
> > free to express your opinion in the original discussion thread.
> >
> > --
> > With regards,
> > Aleksandr Polovtcev
> >
>
>
> --
> With regards,
> Aleksandr Polovtcev

Re: Re[2]: Google Guava in Ignite 3

[Andrey Gura]

Alexander,

Feel free to start voting. Please, refer to this discussion in order
to avoid a new flaming thread in the voting topic.

On Thu, Sep 2, 2021 at 6:04 PM Alexander Polovtcev
 wrote:
>
> Andrey,
>
> > It just doesn't work. If there is precedent of usage of Guava methods
> then somebody will use other methods.
>
> Maybe you are right. I would suggest a vote on whether we should allow Guava 
> methods in the codebase or not. Let's do that in a separate thread? In the 
> meantime we will prohibit using Guava until the voting is complete.
>
> On Thu, Sep 2, 2021 at 3:58 PM Andrey Gura  wrote:
>>
>> > So I would propose to allow at least *some* methods that we consider 
>> > useful, while disallowing everything else.
>>
>> It just doesn't work. If there is precedent of usage of Guava methods
>> then somebody will use other methods.
>>
>> > I think that the benefits of using Guava methods instead of copy-pasting 
>> > them are quite obvious: you don't have to copy-paste code and support it
>>
>> I don't understand why somebody did such a copy-paste, but I believe
>> that nobody supports this code. So, from my point of view, it isn't a
>> problem.
>>
>> On Thu, Sep 2, 2021 at 2:52 PM Alexander Polovtcev
>>  wrote:
>> >
>> > Andrey,
>> > I think that the benefits of using Guava methods instead of copy-pasting 
>> > them are quite obvious: you don't have to copy-paste code and support it. 
>> > I also find this situation quite strange: we have a dependency and 
>> > copy-paste code from it instead of using it directly.
>> >
>> >
>> > On Thu, Sep 2, 2021 at 1:58 PM Andrey Mashenkov 
>> >  wrote:
>> >>
>> >> Alex,
>> >>
>> >> As I understand we agree to shade Guava transitive dependency and
>> >> you said a 'maven-shade-plugin' can drop unused Guava methods to reduce 
>> >> the
>> >> footprint of Ignite jar.
>> >>
>> >> At this point, there is no difference between copy-pasting Guava method to
>> >> Ignite and use Guava one.
>> >> The resulted jar will have one copy of such method, but in the second 
>> >> case,
>> >> we have to care about compatibility when the transitive Guava dependency
>> >> will be updated to a new version.
>> >>
>> >> So, I see no benefits from using Guava directly.
>> >>
>> >>
>> >> On Thu, Sep 2, 2021 at 11:56 AM Alexander Polovtcev 
>> >> 
>> >> wrote:
>> >>
>> >> > Hi, Andrey!
>> >> > I mostly agree with your proposal, but, since we already have some
>> >> > copy-paste in our code, can we at least use Guava to remove it? So I 
>> >> > would
>> >> > propose to allow at least *some* methods that we consider useful, while
>> >> > disallowing everything else. I understand that it may be difficult to
>> >> > formalize, so maybe we can create some kind of a whitelist of Guava
>> >> > methods? What do you think?
>> >> >
>> >> > On Mon, Aug 30, 2021 at 2:54 PM Andrey Gura  wrote:
>> >> >
>> >> > > Follow up
>> >> > >
>> >> > > On Mon, Aug 23, 2021 at 1:22 PM Andrey Gura  wrote:
>> >> > > >
>> >> > > > Igniters,
>> >> > > >
>> >> > > > What I actually didn't understand from this thread: Is Guava allowed
>> >> > > > in production code of Ignite 3 modules (not dependencies like
>> >> > > > Calcite)?
>> >> > > >
>> >> > > > While we agree with using shading I don't see any arguments about
>> >> > > > using Guava library in our code base except for the fact that we 
>> >> > > > have
>> >> > > > some copy-paste of Guava code in the project.
>> >> > > >
>> >> > > > Guava is rich enough library and it has both advantages and
>> >> > > > disadvantages. Ignite code base always strived to be not overloaded
>> >> > > > and minimalistic. For example we usually use immutability very rare,
>> >> > > > we try to avoid complex and overloaded APIs on hot path, we try 
>> >> > > > reduce
>> >> > > > GC pressure, etc. In turn, Guava is some sense forces using of
>> >> > > > immutable collections, has a lot of uti

Re: Re[2]: Google Guava in Ignite 3

[Andrey Gura]

> So I would propose to allow at least *some* methods that we consider useful, 
> while disallowing everything else.

It just doesn't work. If there is precedent of usage of Guava methods
then somebody will use other methods.

> I think that the benefits of using Guava methods instead of copy-pasting them 
> are quite obvious: you don't have to copy-paste code and support it

I don't understand why somebody did such a copy-paste, but I believe
that nobody supports this code. So, from my point of view, it isn't a
problem.

On Thu, Sep 2, 2021 at 2:52 PM Alexander Polovtcev
 wrote:
>
> Andrey,
> I think that the benefits of using Guava methods instead of copy-pasting them 
> are quite obvious: you don't have to copy-paste code and support it. I also 
> find this situation quite strange: we have a dependency and copy-paste code 
> from it instead of using it directly.
>
>
> On Thu, Sep 2, 2021 at 1:58 PM Andrey Mashenkov  
> wrote:
>>
>> Alex,
>>
>> As I understand we agree to shade Guava transitive dependency and
>> you said a 'maven-shade-plugin' can drop unused Guava methods to reduce the
>> footprint of Ignite jar.
>>
>> At this point, there is no difference between copy-pasting Guava method to
>> Ignite and use Guava one.
>> The resulted jar will have one copy of such method, but in the second case,
>> we have to care about compatibility when the transitive Guava dependency
>> will be updated to a new version.
>>
>> So, I see no benefits from using Guava directly.
>>
>>
>> On Thu, Sep 2, 2021 at 11:56 AM Alexander Polovtcev 
>> wrote:
>>
>> > Hi, Andrey!
>> > I mostly agree with your proposal, but, since we already have some
>> > copy-paste in our code, can we at least use Guava to remove it? So I would
>> > propose to allow at least *some* methods that we consider useful, while
>> > disallowing everything else. I understand that it may be difficult to
>> > formalize, so maybe we can create some kind of a whitelist of Guava
>> > methods? What do you think?
>> >
>> > On Mon, Aug 30, 2021 at 2:54 PM Andrey Gura  wrote:
>> >
>> > > Follow up
>> > >
>> > > On Mon, Aug 23, 2021 at 1:22 PM Andrey Gura  wrote:
>> > > >
>> > > > Igniters,
>> > > >
>> > > > What I actually didn't understand from this thread: Is Guava allowed
>> > > > in production code of Ignite 3 modules (not dependencies like
>> > > > Calcite)?
>> > > >
>> > > > While we agree with using shading I don't see any arguments about
>> > > > using Guava library in our code base except for the fact that we have
>> > > > some copy-paste of Guava code in the project.
>> > > >
>> > > > Guava is rich enough library and it has both advantages and
>> > > > disadvantages. Ignite code base always strived to be not overloaded
>> > > > and minimalistic. For example we usually use immutability very rare,
>> > > > we try to avoid complex and overloaded APIs on hot path, we try reduce
>> > > > GC pressure, etc. In turn, Guava is some sense forces using of
>> > > > immutable collections, has a lot of utility methods which produces
>> > > > temp object (yes, I don't trust to escap analysys), etc. Rich set of
>> > > > collections and utility functions will also lead to different
>> > > > programming styles during Ignite development. I can't predict it, but
>> > > > it is usual enough when some developers force immutablity while others
>> > > > remove static type declaration and replace it by var :)
>> > > >
>> > > > This is a common pratice to reduce language possibilies subset and
>> > > > limiting using of some libraries in order to provide more or less
>> > > > unified approach to coding. So I propose to disallow using Guava in
>> > > > our code base.
>> > > >
>> > > > WDYT?
>> > > >
>> > > >
>> > > > On Fri, Aug 20, 2021 at 7:37 PM Alexander Polovtcev
>> > > >  wrote:
>> > > > >
>> > > > > Guys,
>> > > > >
>> > > > > Thanks again for your responses. I've created a ticket about using
>> > the
>> > > > > Shade plugin in order to understand if it is possible to shade and
>> > > minimize
>> > > > > Guava, I will start working on it shortly:
>> > > &

Re: Re[2]: Google Guava in Ignite 3

[Andrey Gura]

Follow up

On Mon, Aug 23, 2021 at 1:22 PM Andrey Gura  wrote:
>
> Igniters,
>
> What I actually didn't understand from this thread: Is Guava allowed
> in production code of Ignite 3 modules (not dependencies like
> Calcite)?
>
> While we agree with using shading I don't see any arguments about
> using Guava library in our code base except for the fact that we have
> some copy-paste of Guava code in the project.
>
> Guava is rich enough library and it has both advantages and
> disadvantages. Ignite code base always strived to be not overloaded
> and minimalistic. For example we usually use immutability very rare,
> we try to avoid complex and overloaded APIs on hot path, we try reduce
> GC pressure, etc. In turn, Guava is some sense forces using of
> immutable collections, has a lot of utility methods which produces
> temp object (yes, I don't trust to escap analysys), etc. Rich set of
> collections and utility functions will also lead to different
> programming styles during Ignite development. I can't predict it, but
> it is usual enough when some developers force immutablity while others
> remove static type declaration and replace it by var :)
>
> This is a common pratice to reduce language possibilies subset and
> limiting using of some libraries in order to provide more or less
> unified approach to coding. So I propose to disallow using Guava in
> our code base.
>
> WDYT?
>
>
> On Fri, Aug 20, 2021 at 7:37 PM Alexander Polovtcev
>  wrote:
> >
> > Guys,
> >
> > Thanks again for your responses. I've created a ticket about using the
> > Shade plugin in order to understand if it is possible to shade and minimize
> > Guava, I will start working on it shortly:
> > https://issues.apache.org/jira/browse/IGNITE-15354
> >
> > On Tue, Aug 10, 2021 at 1:51 PM Courtney Robinson 
> > 
> > wrote:
> >
> > > I think since Calcite brings it in already then your arguments make sense.
> > > Would it be pinned to the same version as Calcite? Risk NoSuchMethodError
> > > at runtime if not.
> > > +1
> > >
> > > On Mon, Aug 9, 2021 at 9:56 AM Alexander Polovtcev <
> > > alexpolovt...@gmail.com>
> > > wrote:
> > >
> > > > Zhenya, Courtney, Andrey,
> > > >
> > > > What do you think about my arguments, was I able to convince you? I 
> > > > would
> > > > like to reach some consensus here. At the moment, my original points
> > > still
> > > > stand, I'm also ok with shading Guava if needed, though I think it is 
> > > > not
> > > > necessary at this point.
> > > >
> > > > On Fri, Aug 6, 2021 at 12:45 PM Alexander Polovtcev <
> > > > alexpolovt...@gmail.com>
> > > > wrote:
> > > >
> > > > > Zhenya,
> > > > >
> > > > > > But there is no restrictions from running ignite server nodes from
> > > some
> > > > > other code with it`s own guava version seems we obtain fast path to 
> > > > > jar
> > > > > hell here?
> > > > >
> > > > > I'm not sure if I fully understand your question, but it looks like we
> > > > are
> > > > > in this situation already, because we have some dependencies that use
> > > > > Guava. That's why I propose to add Guava explicitly to at least have a
> > > > > deterministic runtime configuration (see this link
> > > > > <
> > > >
> > > https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Management
> > > > >
> > > > > for an explanation).
> > > > >
> > > > > On Fri, Aug 6, 2021 at 12:25 PM Zhenya Stanilovsky
> > > > >  wrote:
> > > > >
> > > > >>
> > > > >> Alexander, first of all looks like Ivan Daschinsky approach about 
> > > > >> thin
> > > > >> client use only and shadow plugin are cover all Andrey Mashenkov
> > > listing
> > > > >> problems.
> > > > >> But there is no restrictions from running ignite server nodes from
> > > some
> > > > >> other code with it`s own guava version seems we obtain fast path to
> > > jar
> > > > >> hell here?
> > > > >>
> > > > >>
> > > > >> >Zhenya,
> > > > >> >
> > > > >> >My intentions are the following:
> > > &

Re: Re[2]: Google Guava in Ignite 3

[Andrey Gura]

Igniters,

What I actually didn't understand from this thread: Is Guava allowed
in production code of Ignite 3 modules (not dependencies like
Calcite)?

While we agree with using shading I don't see any arguments about
using Guava library in our code base except for the fact that we have
some copy-paste of Guava code in the project.

Guava is rich enough library and it has both advantages and
disadvantages. Ignite code base always strived to be not overloaded
and minimalistic. For example we usually use immutability very rare,
we try to avoid complex and overloaded APIs on hot path, we try reduce
GC pressure, etc. In turn, Guava is some sense forces using of
immutable collections, has a lot of utility methods which produces
temp object (yes, I don't trust to escap analysys), etc. Rich set of
collections and utility functions will also lead to different
programming styles during Ignite development. I can't predict it, but
it is usual enough when some developers force immutablity while others
remove static type declaration and replace it by var :)

This is a common pratice to reduce language possibilies subset and
limiting using of some libraries in order to provide more or less
unified approach to coding. So I propose to disallow using Guava in
our code base.

WDYT?


On Fri, Aug 20, 2021 at 7:37 PM Alexander Polovtcev
 wrote:
>
> Guys,
>
> Thanks again for your responses. I've created a ticket about using the
> Shade plugin in order to understand if it is possible to shade and minimize
> Guava, I will start working on it shortly:
> https://issues.apache.org/jira/browse/IGNITE-15354
>
> On Tue, Aug 10, 2021 at 1:51 PM Courtney Robinson 
> wrote:
>
> > I think since Calcite brings it in already then your arguments make sense.
> > Would it be pinned to the same version as Calcite? Risk NoSuchMethodError
> > at runtime if not.
> > +1
> >
> > On Mon, Aug 9, 2021 at 9:56 AM Alexander Polovtcev <
> > alexpolovt...@gmail.com>
> > wrote:
> >
> > > Zhenya, Courtney, Andrey,
> > >
> > > What do you think about my arguments, was I able to convince you? I would
> > > like to reach some consensus here. At the moment, my original points
> > still
> > > stand, I'm also ok with shading Guava if needed, though I think it is not
> > > necessary at this point.
> > >
> > > On Fri, Aug 6, 2021 at 12:45 PM Alexander Polovtcev <
> > > alexpolovt...@gmail.com>
> > > wrote:
> > >
> > > > Zhenya,
> > > >
> > > > > But there is no restrictions from running ignite server nodes from
> > some
> > > > other code with it`s own guava version seems we obtain fast path to jar
> > > > hell here?
> > > >
> > > > I'm not sure if I fully understand your question, but it looks like we
> > > are
> > > > in this situation already, because we have some dependencies that use
> > > > Guava. That's why I propose to add Guava explicitly to at least have a
> > > > deterministic runtime configuration (see this link
> > > > <
> > >
> > https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Management
> > > >
> > > > for an explanation).
> > > >
> > > > On Fri, Aug 6, 2021 at 12:25 PM Zhenya Stanilovsky
> > > >  wrote:
> > > >
> > > >>
> > > >> Alexander, first of all looks like Ivan Daschinsky approach about thin
> > > >> client use only and shadow plugin are cover all Andrey Mashenkov
> > listing
> > > >> problems.
> > > >> But there is no restrictions from running ignite server nodes from
> > some
> > > >> other code with it`s own guava version seems we obtain fast path to
> > jar
> > > >> hell here?
> > > >>
> > > >>
> > > >> >Zhenya,
> > > >> >
> > > >> >My intentions are the following:
> > > >> >
> > > >> >1. Remove some copy-pasted code (like the "bytecode" module or some
> > > >> utility
> > > >> >methods). Please see my original message for the links to the code.
> > > >> >2. Explicitly pin the Guava version to avoid conflicts in the
> > runtime.
> > > >> >
> > > >> >About allowing to use Guava in the codebase, my thoughts are the
> > > >> following:
> > > >> >
> > > >> >1. We *already* use some code from Guava either directly (like in the
> > > >> >"calcite" module) or by copy-pasting it into a utility class.
> > > >> >2. I understand that some Guava methods are obsolete as of Java 11,
> > but
> > > >> >some of them still don't have any standard library counterparts, in
> > > which
> > > >> >case I think using Guava is justified (which is supported by point
> > 1).
> > > >> >
> > > >> >Can you please explain why you would disapprove of my proposal?
> > > >> >
> > > >> >On Thu, Aug 5, 2021 at 7:56 PM Zhenya Stanilovsky
> > > >> >< arzamas...@mail.ru.invalid > wrote:
> > > >> >
> > > >> >>
> > > >> >> alexpolovtcev please clarify what do you mean under : «possibility
> > of
> > > >> >> using Guava in Ignite 3», using how necessary dependency of calcite
> > > or
> > > >> >> using like «using in our code» ? If using in code, i -1 here.
> > > >> >> thanks.
> > > >> >>
> > > >> >>
> > > >> >> >Hello, dear 

Re: [DISCUSSION] Code style for Ignite 3

[Andrey Gura]

Looks good to me. But Idea configuration for style check is not
enough. It helps developers but does not automate style checking.

Checkstyle project provides ready to use config based on Google Code
Style [1]. I hope it matches well with Idea config and we'll avoid any
confusing incidents.

Let's start with this convention and adopt it to our needs if needed.

[1] 
https://github.com/checkstyle/checkstyle/blob/master/src/main/resources/google_checks.xml

On Fri, Aug 20, 2021 at 12:02 PM Alexei Scherbakov
 wrote:
>
> +1
>
> пт, 20 авг. 2021 г. в 10:54, Alexander Polovtcev :
>
> > Hi, Val. This is an extremely welcome change, thank you!
> >
> > On Fri, Aug 20, 2021 at 12:17 AM Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> > > Igniters,
> > >
> > > I would like to discuss a potential change to the coding guidelines for
> > > Ignite 3. Currently, we're using the existing guidelines inherited from
> > > Ignite 2, which are described here:
> > > https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines
> > >
> > > Current guidelines, however, exist for many years and have several
> > issues.
> > > They are cumbersome, carry a lot of legacy stuff, and can't be automated.
> > > Every now and then, they seem to cause questions and confusion.
> > >
> > > While it's hard to make drastic changes in Ignite 2, we still have a
> > great
> > > opportunity to update the guidelines in Ignite 3. I would identify two
> > > major goals here:
> > >
> > >1. Simplification. Having too many rules and restrictions tend to
> > >complicate development rather than providing any value. We should come
> > > up
> > >with a minimum set of rules and then make amends one by one if needed.
> > >2. The ability for automation. I hold a strong belief that code style
> > >checking has to become a part of the build. Therefore, we need to make
> > > sure
> > >that any rule that ends up in the guideline can be automatically
> > > verified.
> > >
> > > I propose the following process to define the new guideline:
> > >
> > >1. Use Google code style as the starting point:
> > >https://google.github.io/styleguide/javaguide.html
> > >2. Replace the 100 column limit with 140. The latter is the value we
> > >already use in Ignite 2, and it seems to be more reasonable, in my
> > > opinion.
> > >3. Use 4 spaces block indentation and 8 spaces for continuation (as
> > >opposed to 2 and 4). Nothing wrong with 2 spaces, in my view, but 4
> > > spaces
> > >should provide a smoother transition, as we're really used to this
> > > style.
> > >4. For any other changes, initiate separate discussions going forward.
> > >
> > > Several reasons why I specifically propose Google style:
> > >
> > >1. This is essentially the standard for many projects. I don't think
> > >there is a need for us to reinvent the wheel.
> > >2. It's minimalistic and developer-friendly. No overcomplication.
> > >3. (probably the most important) It comes with all the required tools
> > >and configurations for automation (e.g., here is the config for IDEA:
> > >
> > https://github.com/google/error-prone/blob/master/.idea/GoogleStyle.xml
> > > )
> > >
> > > Please let me know what you think. If there are no objections, I will
> > start
> > > the process.
> > >
> > > -Val
> > >
> >
> >
> > --
> > With regards,
> > Aleksandr Polovtcev
> >
>
>
> --
>
> Best regards,
> Alexei Scherbakov

Re: [Discussion] Race on heartbeat update in checkpoint thread

[Andrey Gura]

Alexey, Ilya,

I took a look at the problem and corresponding fixes. It seems that
you are both right. But Alexey's fix looks like some kind of hack to
me.

We have two problems:

1. Heartbeat update from thread that will complete a future.

I agree with Ilya and Andrey. Only a critical worker itself can update
heartbeat. So, removal of `res.listen(fut ->
heartbeatUpdater.updateHeartbeat());` from
CheckpointContextImpl#executor() method is a good idea. It will avoid
a race and makes Alexey's fix redundant.

2. blockingSection in checkpointer thread really can never detect the
lack of pending tasks progress.

Alexey is right. It is a problem. But what I actually see:
checkpointWritePageThreads and checkpointCollectInfoThreads which
execute pending tasks, are actually critical themselves and should be
also monitored by WorkersRegistry. In such a case the checkpointer
thread can safely use blocking sections at places designated by Ilya
and forementioned threads should also use blocking sections on IO
operations.

So proposal:

- Remove Alexey's fix
- Remove asynchronous heartbeat update
- Add blocking sections (fix of Ilya)
- Monitor checkpointWritePageThreads and checkpointCollectInfoThreads
if they exist (it depends on configuration)

WDYT?

On Thu, Jul 29, 2021 at 5:48 AM Ilya Kazakov  wrote:
>
> Guys. I have discussed just these changes. Look at my PR, please. What do
> you think?
>
> https://issues.apache.org/jira/browse/IGNITE-15192
> https://github.com/apache/ignite/pull/9280/files
>
> чт, 22 июл. 2021 г. в 22:30, Andrey Mashenkov :
>
> > Hi guys,
> >
> > I think updateHeartBeat() method was misused in the future listener and
> > this must be fixed.
> >
> > Actually, GridWorker.heartbeatTs Javadoc says that field is updated by the
> > worker itself.
> > It is consistent with WorkProgressDispatcher.updateHearbeat() javadoc,
> > which said "Notifying dispatcher that work is in progress and thread didn't
> > freeze."
> > GridWorker.heartbeatTs field is marked volatile just to provide consistent
> > visibility guarantee to a watcher.
> >
> > So, CheckpointPagesWriter violates this contract, when runs in another
> > thread(s).
> > But it works fine just because the main (Checkpointer) thread waits for
> > all the Writers to finish their work before it can fall into the blocking
> > section, therefore there is no race possible.
> >
> > As for the broken case, there are 2 possible solutions,
> > 1. Main thread must wait for all children's tasks to finish before going
> > into the blocking section.
> > 2. Or make updateHeartbeat consistent with the blockingSectionBegin/End.
> > Seems, there is no need to mark the method as synchronized, but use call
> > AtomicLongFieldUpdater.getAndUpdate(this, v -> v == Long.MAX_VALUE ? v :
> > U.currentTimeMillis()).
> >
> > I'd suggest
> > * Remove the "thread" mentioning from WorkProgressDispatcher interface to
> > avoid confusion with the current usage and make WorkProgressDispatcher more
> > general.
> > * Avoid unsafe heartbeatUpdater leak to the outside via wrapping
> > CheckpointContextImpl.heartbeatUpdater instance to perform consistent
> > heartbeat update from p.2 above.
> >
> > Does it make sense?
> >
> > On Wed, Jul 21, 2021 at 11:12 AM Alex Plehanov 
> > wrote:
> >
> >> Ilya,
> >>
> >> Race only affects the future listener (which only updates heartbeat), but
> >> not checkpoint listeners, so no other bugs possible caused by this race
> >> except wrong heartbeat update.
> >>
> >> As I've written before, I think it's not a good idea to wait for other
> >> threads by the critical thread in the blocking section. If these threads
> >> are not monitored we can never detect lack of progress and never trigger
> >> the failure handler. Perhaps a good solution will be to register async
> >> threads as critical threads (additionally to waiting for these threads in
> >> the blocing section) and update their own heartbeat. But the current fix
> >> is
> >> required too, to avoid such problems for other critical threads in the
> >> future.
> >>
> >> ср, 21 июл. 2021 г. в 06:29, Ilya Kazakov :
> >>
> >> > 1.
> >> > I mean calling listeners in CheckpointWorkflow.markCheckpointBegin():
> >> >
> >> > This
> >> > --
> >> > for (CheckpointListener lsnr : dbLsnrs)
> >> > lsnr.beforeCheckpointBegin(ctx0);
> >> >
> >> > ctx0.awaitPendingTasksFinished();
> >> > --
> >> >
> >> > And this:
> >> >
> >> > --
> >> > // Listeners must be invoked before we write checkpoint record to WAL.
> >> > for (CheckpointListener lsnr : dbLsnrs)
> >> > lsnr.onMarkCheckpointBegin(ctx0);
> >> >
> >> > ctx0.awaitPendingTasksFinished();
> >> > --
> >> >
> >> > Inside lsnr.beforeCheckpointBegin(ctx0) and
> >> > lsnr.onMarkCheckpointBegin(ctx0) we call
> >> CheckpointContextImpl.executor()
> >> > which submit heartbeat update tasks in 

Re: IEP-61 Transaction API desing for Ignite 3

[Andrey Gura]

Alexey,

could you please describe Transaction interface?

Also it would be great to have a couple examples of using the proposed API.

On Tue, Jul 13, 2021 at 4:43 PM Alexei Scherbakov
 wrote:
>
> Folks,
>
> I've prepared a PR implementing my vision of public transactions API.
>
> API is very simple and similar to Ignite 2, but has some differences.
>
> More details can be found here [1]
>
> Share your thoughts.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-15086
>
> --
>
> Best regards,
> Alexei Scherbakov

Re: [DISCUSSION] Javadoc style requirements and checks in Ignite-3.

[Andrey Gura]

Hi,

Unfortunately, such things as the "many private APIs" rule can't be
formalized for any validation tool. So we have to choose between rules
like "everything should be documented" and "private API documentation
is not mandatory".

The community prefers the second approach. I don't want to argue about
it (although I prefer to document the private API ). My goal is
getting a consistent and customizable way to validate javadoc where it
is mandatory (public API). It also could be applied to private API,
but it should be another rule set which doesn't require javadoc but
validates existing javadoc.

On Wed, Jun 16, 2021 at 11:32 PM Nikita Ivanov  wrote:
>
> Hi,
> In my strong opinion Javadoc is a code. Any errors in Javadoc are
> equal to the bug in the code and must be treated as such. Proper and
> extensive Javadoc for all public and many private APIs is absolutely
> essential for this project, its growth and maturity.
>
> I'm surprised this community still needs to debate fundamental
> engineering issues like that...
> --
> Nikita Ivanov
>
> On Wed, Jun 16, 2021 at 4:47 AM Andrey Gura  wrote:
> >
> > Hi,
> >
> > I think that scope should be limited by public API  for beginning.
> > Also I don't sure that we should support specific tags like @apiNote,
> > @implSpec, @implNote.
> >
> > Let's move using the following plan:
> >
> > 1. Create an empty (effectively) checkstyle config for javadoc
> > checking and commit it to the repository. After this step it will be
> > possible to configure TC in order to use this configuration.
> > 2. Configure TC.
> > 3. Commit a non-empty checkstyle config, but all modules should be
> > excluded from checking on this step.
> > 4. For each module create a JIRA issue. Module maintainers fix
> > corresponding tickets and remove module exclusion from checking.
> > 5. Add information about javadoc validation targets to DEVNOTES.md
> > file (could be done on step 3)
> >
> > Any objections?
> >
> > On Tue, Apr 20, 2021 at 11:40 AM Andrey Mashenkov
> >  wrote:
> > >
> > > I've fixed the PR.
> > >
> > > Now,
> > > 1. Javadoc style is only checked if it is present for the element. All
> > > declared javadoc tags MUST have a description.
> > > So, one should either write correct javadoc or don't write at all.
> > > 2. Additional checks performed for non-internal and non-test classes. All
> > > public and protected elements must have non-emtpy javadoc.
> > >
> > > So, checkstyle detects 500+ missed descriptions (missed javadocs, tags
> > > descriptions, tags themselves) in public scope
> > > and 180+ bad-styles (missed brased, spaces, empty-lines) javadocs in whole
> > > codebase.
> > >
> > > I'd suggest to force non-empty javadocs for all non-test classes including
> > > internal (but their members) as well.
> > >
> > >
> > >
> > > On Mon, Apr 19, 2021 at 6:37 PM Andrey Mashenkov 
> > > 
> > > wrote:
> > >
> > > > Alexey,
> > > >
> > > > These are bad examples and unfortunately, most of the Ignite code 
> > > > doesn't
> > > > look self-descriptive.
> > > > (Do you feel the difference between @SerializableTransient and
> > > > @TransientSerializable?)
> > > >
> > > > To understand the semantic of e.g. 'metricsHistSize' you have to analyze
> > > > its usages.
> > > > Getter and setter for this property look better, but it still not clear
> > > > what happens with metric values above the limit?
> > > >
> > > > I have another example, one of the core components 
> > > > GridDhtPartitionDemander
> > > > has totally useless javadoc.
> > > > It is obviously has nothing with thread pool + "local cache" phrase 
> > > > looks
> > > > very confusing.
> > > >
> > > > /**
> > > >  * Thread pool for requesting partitions from other nodes and 
> > > > populating local cache.
> > > >  */
> > > > public class GridDhtPartitionDemander
> > > >
> > > > To understand the purpose of this internal component I have to analyze 
> > > > its
> > > > code and usages.
> > > > However, if one will face unexpected behavior, he/she may be confused if
> > > > it is a lack of javadoc or wrong behavior.
> > > >
> > > > There is no way to restrict or avoid such issues with any checks.
> > &

Re: [DISCUSSION] Add cache statistics switch to control script

[Andrey Gura]

Hi, Ilya

Could you please provide more details about how it should work (in
issue description)? E.g. what if I pass the cache group name instead
of the name of a particular cache? What if I will not pass any cache
names?

Also we don't use the word "statistics" anymore (while it is still
called statistics in configuration). What about the word "metrics"?

On Wed, Jun 16, 2021 at 2:31 PM Shishkov Ilya  wrote:
>
> Dear Ignite Community!
> I propose to discuss the addition of a new feature for control script:
> cache statistics switch (enabling / disabling). Currently, cache statistics
> can be toggled only via IgniteVisorCmd or JMX.
> I've created the corresponding issue [1] and suggest adding an extra option
> to a '--cache' command of the control script, eg.:
>
> --cache statistics enable|disable [cacheName1,...,cacheNameN]
>
> Please, tell me if you have any suggestions or remarks against this feature.
>
> 1. https://issues.apache.org/jira/browse/IGNITE-14913

Re: [DISCUSSION] Code style. Variable abbrevations

[Andrey Gura]

Hi,

I understand that this rule seems too strict or may be weird. But
removing the rule could lead to review comments like "use future
instead of fut". So my proposal is to change rule from "required" to
"recommended".

On Wed, Jun 16, 2021 at 2:49 PM Valentin Kulichenko
 wrote:
>
> Konstantin, thanks for chiming in.
>
> That's exactly my concern. Overformalization is generally not a good thing.
> Someone used "mess" to abbreviate "message"? That is surely not a good
> coding style, but that's what code reviews are for. I believe that our
> committers are more than capable of identifying such issues.
>
> At the same time, with the current rules, we are *forced* to use
> abbreviations like "locAddrGrpMgr", whether we like it or not. All it does
> is makes it harder to contribute to Ignite, without providing any clear
> value.
>
> -Val
>
> On Thu, Jun 10, 2021 at 9:46 AM Konstantin Orlov 
> wrote:
>
> > +1 for making this optional
> >
> > Common abbreviation rules is good for sure, but sometimes I getting sick
> > of all those locAddrGrpMgr.
> >
> >
> > --
> > Regards,
> > Konstantin Orlov
> >
> >
> >
> >
> > > On 7 Jun 2021, at 14:31, Nikolay Izhikov  wrote:
> > >
> > > Hello, Anton, Alexei.
> > >
> > > Thanks for the feedback.
> > >
> > > Personally, I’m pretty happy current abbreviation rules too.
> > > Let see what we can do to make our codebase even more consistent.
> > >
> > >> 7 июня 2021 г., в 13:23, Alexei Scherbakov <
> > alexey.scherbak...@gmail.com> написал(а):
> > >>
> > >> -1
> > >> Common abbrevs add quality to the code.
> > >>
> > >> пн, 7 июн. 2021 г. в 12:38, Anton Vinogradov :
> > >>
> > >>> -1 here.
> > >>> We can fix the code and set up the rule.
> > >>>
> > >>> This will help to prevent having a weird abbreviation like "mess" (from
> > >>> "message") or "ign" (from "Ignite").
> > >>> Also, the abbreviations list (hardcoded at IDEA plugin) allows to have
> > same
> > >>> names across the whole code, this should simplify the reading.
> > >>>
> > >>> On Sat, Jun 5, 2021 at 10:49 PM Valentin Kulichenko <
> > >>> valentin.kuliche...@gmail.com> wrote:
> > >>>
> >  I also support removing this requirement. It’s not the first time
> > someone
> >  brings this up, and so far we haven’t been able to fix it. Not worth
> > it
> > >>> in
> >  my view.
> > 
> >  -Val
> > 
> >  On Sat, Jun 5, 2021 at 11:54 AM Nikolay Izhikov 
> >  wrote:
> > 
> > > Hello, guys.
> > >
> > > Thanks for the feedback.
> > >
> > > Dmitry,
> > >
> > >> Manual rule enforcement saves a couple of symbols in code
> > >
> > > I’m talking about automatic check.
> > > We can implement it easily.
> > > So, if we decide to keep this rule all we need is to fix current
> > > violations (several thousand!).
> > > After it, checkstyle will automatically enforce this rule.
> > > As you may know, currently, any PR checked according to our
> > checkstyle
> > > rules.
> > > Please, take a look at little green check sign after PR name.
> > >
> > >
> > >> 5 июня 2021 г., в 00:57, Dmitry Pavlov 
> >  написал(а):
> > >>
> > >> +1 for removal. Cnt and count both seem to be fine.
> > >>
> > >> Manual rule enforcement saves a couple of symbols in code, but
> > >>> requires
> > > some time from both contributor and reviewer.
> > >>
> > >> Sincerely,
> > >> Dmitriy Pavlov
> > >>
> > >> On 2021/06/04 19:18:37, Pavel Tupitsyn 
> > wrote:
> > >>> In my opinion, we should remove this rule.
> > >>> Looks like a waste of time. freq or frequency, cnt or count, it is
> >  fine
> > >>> either way.
> > >>>
> > >>> On Fri, Jun 4, 2021 at 7:39 PM Nikolay Izhikov <
> > nizhi...@apache.org
> > 
> > > wrote:
> > >>>
> >  Hello, Igniters.
> > 
> >  Right now, we have the rule to use some predefined list of
> >  abbrevation
> > > for
> >  variable names [1].
> >  Some of the reviewers ask to follow this rule strictly.
> > 
> > > It is required to use abbreviated form for code consistency.
> > 
> >  I tried to implement this rule in form of checkstyle check [2] and
> > >>> it
> >  seems like many of use doesn’t follow this requirement.
> >  My check found 4124 violation in core module.
> > 
> >  Should we make this rule optional in documentation or should we
> >  remove
> > > it
> >  completely?
> >  Or should someone proceed and fix all the violations?
> > 
> >  WDYT?
> > 
> > 
> >  Example of output:
> >  ```
> >  [ERROR]
> > 
> > >
> > 
> > >>>
> > /Users/sbt-izhikov-nv/work/ignite/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/IgnitePdsWithTtlTest.java:94:
> >  Abbrevation should be used for CACHE_NAME_LOCAL_ATOMIC! 

Re: [DISCUSSION] Javadoc style requirements and checks in Ignite-3.

[Andrey Gura]

Hi,

I think that scope should be limited by public API  for beginning.
Also I don't sure that we should support specific tags like @apiNote,
@implSpec, @implNote.

Let's move using the following plan:

1. Create an empty (effectively) checkstyle config for javadoc
checking and commit it to the repository. After this step it will be
possible to configure TC in order to use this configuration.
2. Configure TC.
3. Commit a non-empty checkstyle config, but all modules should be
excluded from checking on this step.
4. For each module create a JIRA issue. Module maintainers fix
corresponding tickets and remove module exclusion from checking.
5. Add information about javadoc validation targets to DEVNOTES.md
file (could be done on step 3)

Any objections?

On Tue, Apr 20, 2021 at 11:40 AM Andrey Mashenkov
 wrote:
>
> I've fixed the PR.
>
> Now,
> 1. Javadoc style is only checked if it is present for the element. All
> declared javadoc tags MUST have a description.
> So, one should either write correct javadoc or don't write at all.
> 2. Additional checks performed for non-internal and non-test classes. All
> public and protected elements must have non-emtpy javadoc.
>
> So, checkstyle detects 500+ missed descriptions (missed javadocs, tags
> descriptions, tags themselves) in public scope
> and 180+ bad-styles (missed brased, spaces, empty-lines) javadocs in whole
> codebase.
>
> I'd suggest to force non-empty javadocs for all non-test classes including
> internal (but their members) as well.
>
>
>
> On Mon, Apr 19, 2021 at 6:37 PM Andrey Mashenkov 
> wrote:
>
> > Alexey,
> >
> > These are bad examples and unfortunately, most of the Ignite code doesn't
> > look self-descriptive.
> > (Do you feel the difference between @SerializableTransient and
> > @TransientSerializable?)
> >
> > To understand the semantic of e.g. 'metricsHistSize' you have to analyze
> > its usages.
> > Getter and setter for this property look better, but it still not clear
> > what happens with metric values above the limit?
> >
> > I have another example, one of the core components GridDhtPartitionDemander
> > has totally useless javadoc.
> > It is obviously has nothing with thread pool + "local cache" phrase looks
> > very confusing.
> >
> > /**
> >  * Thread pool for requesting partitions from other nodes and populating 
> > local cache.
> >  */
> > public class GridDhtPartitionDemander
> >
> > To understand the purpose of this internal component I have to analyze its
> > code and usages.
> > However, if one will face unexpected behavior, he/she may be confused if
> > it is a lack of javadoc or wrong behavior.
> >
> > There is no way to restrict or avoid such issues with any checks.
> > Agree, meaningful javadocs as self-descriptive code is more about culture
> > and discipline.
> >
> > The absence of javadoc on internal API, unreasonably raises the entry
> > threshold for new developers and makes the development of intra-component
> > interaction harder.
> > I admit a high-level description for public classes may be enough to
> > resolve this without pushing developers to write empty/useless docs for
> > every method/field.
> >
> >
> > On Mon, Apr 19, 2021 at 5:21 PM Alexey Kukushkin <
> > kukushkinale...@gmail.com> wrote:
> >
> >> I personally do not understand why we need mandatory javadoc even in
> >> public
> >> modules. I think javadoc is needed only when the code is not
> >> self-descriptive. What value does a javadoc like this bring
> >> <
> >> https://github.com/apache/ignite/blob/2.10.0/modules/core/src/main/java/org/apache/ignite/configuration/IgniteConfiguration.java
> >> >
> >> to a developer:
> >>
> >> /** Default metrics history size (value is {@code 1}). */
> >> public static final int DFLT_METRICS_HISTORY_SIZE = 1;
> >>
> >> /** Metrics history time. */
> >> private int metricsHistSize = DFLT_METRICS_HISTORY_SIZE;
> >>
> >> BTW, I picked a random example and it already has a copy-paste mistake in
> >> the javadoc, which is there for years. I think that indicates it has no
> >> real value and developers just do it to comply with the rule.
> >>
> >> Some say mandatory javadoc is for the discipline but I think Apache Ignite
> >> developers are mature enough to understand when additional documentation
> >> is
> >> really required.
> >>
> >>
> >>
> >> пн, 19 апр. 2021 г. в 16:37, Ivan Pavlukhin :
> >>
> >> > Hi Andrey and Igniters,
> >> >
> >> > Sorry if I misread something but I have totally different opinion in
> >> > one aspect. In my mind it is much better in practice to follow strict
> >> > rules for public API javadocs but not for internals. I would use
> >> > static checks for API packages and disable them for internal ones and
> >> > refine code readability during code review. Main motivation here is
> >> > that ubiquitous javadocs did not work well in ignite-2 and I believe
> >> > it would not in ignite-3.
> >> >
> >> > 2021-04-19 13:30 GMT+03:00, Andrey Mashenkov <
> >> andrey.mashen...@gmail.com>:
> >> > > Hi 

Re: Apache Ignite 2.11

[Andrey Gura]

Maksim,

why do you want to include just one phase of functionality into
release instead of full implementation? History of Ignie development
shows (at least from my point of view) that it is bad practice. As
result users will have unfinished and badly tested functionality. We
have (or had in the past) some examples of such things: MVCC, IEP-35,
etc.

On Mon, May 17, 2021 at 5:11 PM Алексей Гидаспов  wrote:
>
> Hi, Dmitriy!
>
> I suggest this timeline
>
> Scope Freeze: May 31, 2021
> Code Freeze: 14 June, 2021
> Voting Date: 21 June, 2021
> Release Date: 28 June, 2021
>
> пт, 14 мая 2021 г. в 11:32, Maksim Timonin :
>
> > Hi, I'd like to include Phase 1 of index queries feature [1] to the next AI
> > release. Currently there are 2 tickets in progress ([2], [3]), and they
> > will be ready for review next week. What is a time of code freeze if it's
> > going to release AI 2.11 at the end of June?
> >
> > [1]
> >
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-71%3A+Public+API+for+secondary+index+search
> > [2] https://issues.apache.org/jira/browse/IGNITE-14699
> > [3] https://issues.apache.org/jira/browse/IGNITE-14703
> >
> > On Thu, May 13, 2021 at 5:27 PM Dmitry Pavlov  wrote:
> >
> > > Hi Alexey,
> > >
> > > :pmc to sign distribution:
> > > Formally, only PMC member can be a release manager, so part of activities
> > > should be picked up by committer and PMC member. If noone else want to
> > help
> > > you here, I would.
> > >
> > > :timeline:
> > > And, could you estimate all phases of the release.
> > >
> > > :scope:
> > > I guess we can just pick up master current state and release it. But if
> > > someone has some ideas if we should wait for particular feature to be
> > > completed before branch divergence/release candidate build, please let
> > know.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > On 2021/05/13 09:45:02, Алексей Гидаспов  wrote:
> > > > Dear Ignite Community!
> > > >
> > > > I suggest starting Apache Ignite 2.11 release activities. Also suggest
> > > > myself to be a release manager for this release. I plan release at the
> > > end
> > > > of june 2021.
> > > >
> > > > ___
> > > >
> > > > Best Regards,
> > > > Alexey Gidaspov
> > > >
> > >
> >

Re: Exceeding the DataStorageConfiguration#getMaxWalArchiveSize due to historical rebalance

[Andrey Gura]

Stan

> If archive size is less than min or more than max then the system 
> functionality can degrade (e.g. historical rebalance may not work as 
> expected).

Why does the condition "archive size is less than min" lead to system
degradation? Actually, the described case is a normal situation for
brand new clusters.

I'm okay with the proposed minWalArchiveSize property. Looks like
relatively understandable property.

On Sun, May 9, 2021 at 7:12 PM Stanislav Lukyanov
 wrote:
>
> Discuss this with Kirill verbally.
>
> Kirill showed me that having the min threshold doesn't quite work.
> It doesn't work because we no longer know how much WAL we should remove if we 
> reach getMaxWalArchiveSize.
>
> For example, say we have minWalArchiveTimespan=2 hours and 
> maxWalArchiveSize=2GB.
> Say, under normal load on stable topology 2 hours of WAL use 1 GB of space.
> Now, say we're doing historical rebalance and reserve the WAL archive.
> The WAL archive starts growing and soon it occupies 2 GB.
> Now what?
> We're supposed to give up WAL reservations and start agressively removing WAL 
> archive.
> But it is not clear when can we stop removing WAL archive - since last 2 
> hours of WAL are larger than our maxWalArchiveSize
> there is no meaningful point the system can use as a "minimum" WAL size.
>
> I understand the description above is a bit messy but I believe that whoever 
> is interested in this will understand it
> after drawing this on paper.
>
>
> I'm giving up on my latest suggestion about time-based minimum. Let's keep it 
> simple.
>
> I suggest the minWalArchiveSize and maxWalArchvieSize properties as the 
> solution,
> with the behavior as initially described by Kirill.
>
> Stan
>
>
> > On 7 May 2021, at 15:09, ткаленко кирилл  wrote:
> >
> > Stas hello!
> >
> > I didn't quite get your last idea.
> > What will we do if we reach getMaxWalArchiveSize? Shall we not delete the 
> > segment until minWalArchiveTimespan?
> >
> > 06.05.2021, 20:00, "Stanislav Lukyanov" :
> >> An interesting suggestion I heard today.
> >>
> >> The minWalArchiveSize property might actually be minWalArchiveTimespan - 
> >> i.e. be a number of seconds instead of a number of bytes!
> >>
> >> I think this makes perfect sense from the user point of view.
> >> "I want to have WAL archive for at least N hours but I have a limit of M 
> >> gigabytes to store it".
> >>
> >> Do we have checkpoint timestamp stored anywhere? (cp start markers?)
> >> Perhaps we can actually implement this?
> >>
> >> Thanks,
> >> Stan
> >>
> >>>  On 6 May 2021, at 14:13, Stanislav Lukyanov  
> >>> wrote:
> >>>
> >>>  +1 to cancel WAL reservation on reaching getMaxWalArchiveSize
> >>>  +1 to add a public property to replace 
> >>> IGNITE_THRESHOLD_WAL_ARCHIVE_SIZE_PERCENTAGE
> >>>
> >>>  I don't like the name getWalArchiveSize - I think it's a bit confusing 
> >>> (is it the current size? the minimal size? the target size?)
> >>>  I suggest to name the property geMintWalArchiveSize. I think that this 
> >>> is exactly what it is - the minimal size of the archive that we want to 
> >>> have.
> >>>  The archive size at all times should be between min and max.
> >>>  If archive size is less than min or more than max then the system 
> >>> functionality can degrade (e.g. historical rebalance may not work as 
> >>> expected).
> >>>  I think these rules are intuitively understood from the "min" and "max" 
> >>> names.
> >>>
> >>>  Ilya's suggestion about throttling is great although I'd do this in a 
> >>> different ticket.
> >>>
> >>>  Thanks,
> >>>  Stan
> >>>
>   On 5 May 2021, at 19:25, Maxim Muzafarov  wrote:
> 
>   Hello, Kirill
> 
>   +1 for this change, however, there are too many configuration settings
>   that exist for the user to configure Ignite cluster. It is better to
>   keep the options that we already have and fix the behaviour of the
>   rebalance process as you suggested.
> 
>   On Tue, 4 May 2021 at 19:01, ткаленко кирилл  
>  wrote:
> >  Hi Ilya!
> >
> >  Then we can greatly reduce the user load on the cluster until the 
> > rebalance is over. Which can be critical for the user.
> >
> >  04.05.2021, 18:43, "Ilya Kasnacheev" :
> >>  Hello!
> >>
> >>  Maybe we can have a mechanic here similar (or equal) to checkpoint 
> >> based
> >>  write throttling?
> >>
> >>  So we will be throttling for both checkpoint page buffer and WAL 
> >> limit.
> >>
> >>  Regards,
> >>  --
> >>  Ilya Kasnacheev
> >>
> >>  вт, 4 мая 2021 г. в 11:29, ткаленко кирилл :
> >>
> >>>  Hello everybody!
> >>>
> >>>  At the moment, if there are partitions for the rebalance for which 
> >>> the
> >>>  historical rebalance will be used, then we reserve segments in the 
> >>> WAL
> >>>  archive (we do not allow cleaning the WAL archive) until the 
> >>> rebalance for
> >>>  all cache groups is over.
> >>>
> 

Re: Metrics naming formalization.

[Andrey Gura]

Hi, Mikhail!

I've looked at the problem and the fix. The root cause of the problem
is MetricsRegistry design which allows to register metrics in two
ways:

- using a family of methods like MetricRegistre.register(..,
XxxSupplier, ...) and MetricRegistry.xxxMetric. These methods firstly
create metrics and take into account registry's name. These methods
provide the right way for registering metrics in MetricRegistry.
- using the method MetricRegistry.register(Metric metric). This method
takes as a parameter an already created metric which was created out
of the metric registry context. This method is the wrong (roughly
speaking) way for registering metrics in MetricRegistry.

Using the right approach for metrics registering is a guaranteed way
to avoid problems like described in [1]. Moreover, this approach does
not require any metrics naming formalization because it hides internal
naming details from an user.

So, IMHO, the issue [2] ideally should be re-fixed in a right way.

Also, ideally, we should limit the visibility scope of  metric classes
constructors and do not allow instantiate these classes directly.

[1] https://issues.apache.org/jira/browse/IGNITE-14376
[2] https://issues.apache.org/jira/browse/IGNITE-14428

On Fri, Apr 9, 2021 at 1:25 PM Mikhail Petrov  wrote:
>
> Hello, Igniters.
>
> I faced the following problem with the metrics - JmxMetricExporter fails
> to export discovery metrics (ticket - [1]).
>
> The main reason: JMX exporter assumes that each metric name must start
> with the name of the registry it belongs to, but discovery metrics do
> not obey this naming convection.
>
> I propose to formalize the names of the metrics included in the metrics
> registry so that they always start with the name of the register. And
> add corresponding assertions (ticket - [2]).
> It helps to make the metric naming consistent and fixes the mentioned
> above issue.
>
> The following metric names are proposed to be changed:
>
> JoinedNodes -> io.discovery.JoinedNodes
> LeftNodes -> io.discovery.LeftNodes
> FailedNodes -> io.discovery.FailedNodes
> PendingMessagesRegistered -> io.discovery.PendingMessagesRegistered
> CommunicationErrors -> io.discovery.CommunicationErrors
> CurrentTopologyVersion -> io.discovery.CurrentTopologyVersion
>
> Any objections?
>
>
> [1] - https://issues.apache.org/jira/browse/IGNITE-14376
> [2] - https://issues.apache.org/jira/browse/IGNITE-14428
>

Re: Contributor grants request

[Andrey Gura]

Granted Contributor role. Feel free to assign issues.

On Tue, Apr 6, 2021 at 6:25 PM Smolnikov  wrote:
>
> My username is "Smolnikov"
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: Contributor grants request

[Andrey Gura]

Hi, Rodion!

Could you please provide your Apache JIRA login name?

On Mon, Apr 5, 2021 at 5:37 PM Rodion Smolnikov  wrote:
>
> Hello!
> Can I get contributor access?
> Task https://issues.apache.org/jira/browse/IGNITE-14474
>
> --
>
> Rodion Smolnikov
>
> +7(915)256-54-81
> Software Developer, Russia
>
> *gridgain.com *

Re: [ANNOUNCE] Welcome Ivan Bessonov as a new committer

[Andrey Gura]

> It's funny that @WithSystemProperty is mentioned

Simple things can bring a lot of value :)

On Tue, Jan 12, 2021 at 5:34 PM Ivan Bessonov  wrote:
>
> Thank you all!
>
> It's funny that @WithSystemProperty is mentioned, but I'm glad that
> development became easier with it =)
>
> вт, 12 янв. 2021 г. в 15:12, ткаленко кирилл :
>
> > Ivan, congratulations!
> >
> >
> > 12.01.2021, 14:36, "Вячеслав Коптилин" :
> > > Hello,
> > >
> > > My congratulations, Ivan! May the Force be with you!
> > >
> > > Thanks,
> > > S.
> > >
> > > вт, 12 янв. 2021 г. в 14:27, Ivan Pavlukhin :
> > >
> > >>  Ivan, congratulations!
> > >>
> > >>  2021-01-12 13:36 GMT+03:00, Kseniya Romanova <
> > romanova.ks@gmail.com>:
> > >>  > Ivan, my congratulations!
> > >>  >
> > >>  > вт, 12 янв. 2021 г. в 13:32, Andrey Gura :
> > >>  >
> > >>  >> Igniters,
> > >>  >>
> > >>  >> The Apache Ignite Project Management Committee (PMC) has invited
> > Ivan
> > >>  >> Bessonov to become a new committer and are happy to announce that he
> > >>  >> has accepted.
> > >>  >>
> > >>  >> Ivan has a lot of contributions to the Apache Ignite project. He
> > >>  >> implemented Distributed Meta Storage feature which is one of key
> > >>  >> features in the project. Also he contributed non trivial
> > >>  >> improvements to such important parts of the project as
> > Communication,
> > >>  >> Discovery, Page Memory and Native Persistence. Another simple, but
> > >>  >> very useful contribution is the widely used @WithSystemProperty
> > >>  >> annotation for Apache Ignite test framework
> > >>  >>
> > >>  >> Being a committer enables easier contribution to the project since
> > there
> > >>  >> is
> > >>  >> no need to go via the patch submission process. This should enable
> > >>  better
> > >>  >> productivity.
> > >>  >>
> > >>  >> Please join me in welcoming Ivan, and congratulating him on the new
> > role
> > >>  >> in
> > >>  >> the Apache Ignite Community.
> > >>  >>
> > >>  >> Best Regards,
> > >>  >> Andrey Gura
> > >>  >>
> > >>  >
> > >>
> > >>  --
> > >>
> > >>  Best regards,
> > >>  Ivan Pavlukhin
> >
>
>
> --
> Sincerely yours,
> Ivan Bessonov

[ANNOUNCE] Welcome Ivan Bessonov as a new committer

[Andrey Gura]

Igniters,

The Apache Ignite Project Management Committee (PMC) has invited Ivan
Bessonov to become a new committer and are happy to announce that he
has accepted.

Ivan has a lot of contributions to the Apache Ignite project. He
implemented Distributed Meta Storage feature which is one of key
features in the project. Also he contributed non trivial
improvements to such important parts of the project as Communication,
Discovery, Page Memory and Native Persistence. Another simple, but
very useful contribution is the widely used @WithSystemProperty
annotation for Apache Ignite test framework

Being a committer enables easier contribution to the project since there is
no need to go via the patch submission process. This should enable better
productivity.

Please join me in welcoming Ivan, and congratulating him on the new role in
the Apache Ignite Community.

Best Regards,
Andrey Gura

Re: [VOTE] Release Apache Ignite 3.0.0-alpha1 RC3

[Andrey Gura]

+1 (binding)

On Thu, Jan 7, 2021 at 10:42 PM Vladimir Pligin  wrote:
>
> +1 (non-binding)
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/