Re: Micronaut/Ignite

[Michael Pollind]

Not sure, I would probably bring that up in the PR so the micronaut devs
are aware of this. Not sure how the CI will work and what can be arranged
in this case.

On Thu, Aug 27, 2020 at 3:50 PM Denis Magda  wrote:

> Michael,
>
> The docs look good enough for the release. Once the integration is
> released, I'll update and extend my Micronaut tutorial elucidating on how
> to use the integration in practice for various use cases.
>
> Btw, are the ignite-core and ignite-cache Maven artifacts will be released
> for a specific version of Ignite? Ideally, the version of Ignite needs to
> be set in a pom.xml and the Micronaut integration should pick them up.
>
> Also, what are we doing next - reviewing APIs of the ignite-cache module or
> going through a review process of the ignite-core and merging it first to
> the Micronaut repository?
>
> -
> Denis
>
>
> On Wed, Aug 26, 2020 at 3:35 PM Michael Pollind 
> wrote:
>
> > Also you need to sign the CLA since you add a commit to my branch.
> >
> > branch: https://github.com/pollend/micronaut-ignite/tree/feature/rework
> > linked PR:
> https://github.com/micronaut-projects/micronaut-ignite/pull/33
> >
> > --
> > Michael Pollind
> >
> > On Wed, Aug 26, 2020 at 3:29 PM Denis Magda  wrote:
> >
> > > Could you share the full path to the branch? Don't want to pick a wrong
> > one
> > > accidentally.
> > >
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Wed, Aug 26, 2020 at 2:47 PM Michael Pollind 
> > > wrote:
> > >
> > > > so that clears up the extra flag for micronaut-cache. just need to
> > > > intercept the default IgniteConfiguration bean and update the
> > > > configuration. if you pull my latest changes from the rework branch
> you
> > > can
> > > > look at the docs for that micronaut module. `./gradlew docs`.
> > Finalizing
> > > > the docs should be the last part of this.  The configuration from my
> > > > perspective seems pretty much there. Is there anything else that is
> > > > missing?
> > > >
> > > > On Wed, Aug 26, 2020 at 1:31 PM Denis Magda 
> wrote:
> > > >
> > > > >  Nice catch!
> > > > >
> > > > > -
> > > > > Denis
> > > > >
> > > > >
> > > > > On Wed, Aug 26, 2020 at 12:16 PM Michael Pollind <
> mpoll...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > One thing that just occurred to me for the
> > > DefaultIgniteConfiguration.
> > > > > you
> > > > > > can actually just intercept the bean when it's created. This
> should
> > > let
> > > > > us
> > > > > > use the enable flag from ignite-core for ignite-cache without the
> > > > > > additional flag.
> > > > > >
> > > > > > @Singleton
> > > > > > public class IgniteConfigurationFactoryInterceptor implements
> > > > > > BeanCreatedEventListener {
> > > > > > @Override
> > > > > > public IgniteConfiguration
> > > > > > onCreated(BeanCreatedEvent event) {
> > > > > > IgniteConfiguration configuration = event.getBean();
> > > > > > configuration.setIgniteInstanceName("instance-a");
> > > > > > return configuration;
> > > > > > }
> > > > > > }
> > > > > >
> > > > > >
> > > > > > On Wed, Aug 26, 2020 at 11:31 AM Denis Magda 
> > > > wrote:
> > > > > >
> > > > > > > Michael,
> > > > > > >
> > > > > > > Looks terrific! I went ahead and grouped the configuration
> > samples
> > > > by a
> > > > > > > client type:
> > > > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Micronaut+Integration
> > > > > > >
> > > > > > > As you see, the "Ignite Thin Client" section is blank for now.
> > > Would
> > > > > you
> > > > > > > mind filling it with configuration options following the format
> > > taken
> > > > > for
> > > > > > > the thick client?
> > > > > > >
> > > > > > > Ping me here once you're ready to review the API and
> > implementation
> > > > of
> > > > > > the
> > > > > > > ignite-caching module.
> > > > > > >
> > > > > > > -
> > > > > > > Denis
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Aug 25, 2020 at 9:16 PM Michael Pollind <
> > > mpoll...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Here is a sample project I threw together and I've also
> updated
> > > the
> > > > > > wiki
> > > > > > > > just a bit. working on updating the documentation on the
> > > micronaut
> > > > > end.
> > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Micronaut+Integration
> > > > > > > > https://github.com/pollend/micronaut-ignite-sample
> > > > > > > >
> > > > > > > > On Mon, Aug 24, 2020 at 6:57 PM Michael Pollind <
> > > > mpoll...@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > yea, I'm not sure about ignite-cache config. splitting it
> > into
> > > > two
> > > > > > > > modules
> > > > > > > > > seems kind of overly complicated, but I also don't really
> > like
> > > > the
> > > > > > > random
> > > > > > > > > boolean flag. here is the tweaks from the config your
> > provider:
> > > > > > > > > 

Re: IEP-52: Binary Delivery & Upgradability Enhancements

[Denis Magda]

Petr, thanks,

There is also a collection of modules located in our extensions repository:
https://github.com/apache/ignite-extensions

@Saikat Maitra  is migrating all our existing
integrations to that repository and, once this is done, the extensions will
be released to Maven separately. Is it reasonable to expand the scope of
the IEP-52 and contemplate on how to install those extensions?

-
Denis


On Thu, Aug 27, 2020 at 3:40 PM Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Hi Petr,
>
> The proposal makes sense to me - thanks for starting the discussion.
> Current installation and upgrade procedures involve a lot of manual steps
> and quite error-prone, we need to automate and simplify them as much as
> possible.
>
> I believe we eventually should end up with a unified command-line tool
> (ignitectl?) that will incorporate all the operations (enable/disable
> modules, start/stop, configuration updates, activation, baseline, etc.).
> This IEP is a step in this direction.
>
> Looking forward to testing a prototype :)
>
> -Val
>
> On Thu, Aug 27, 2020 at 2:17 AM Petr Ivanov  wrote:
>
> > Hi, Igniters!
> >
> >
> > For Apache Ignite 3.0 I would like to propose vision of enhanced delivery
> > and upgrade processes [1].
> > The key idea is to make main binary as slim as possible (delivering every
> > optional component by demand only) while providing way to run each new
> > version seamlessly without much of the efforts migrating data and
> > configuration between upgrades.
> >
> > I plan to create prototype based on current release of Apache Ignite
> > (2.8.1 or 2.9.0 if it will be released soon) later in September.
> >
> >
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158873958
> >
> >
>

IEPs for Ignite 3.0

[Valentin Kulichenko]

Guys,

Since we started creating IEPs related specifically for 3.0, I thought it
would be easier if we separate them from others.

I went ahead and created a subfolder under "Active Proposals" [1] and moved
the newly created IEP-52 there.

[1]
https://cwiki.apache.org/confluence/display/IGNITE/Proposals+for+Ignite+3.0

Please create any other proposals for 3.0 in this folder.

-Val

Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]

[Denis Magda]

Looks beautiful and easy to use, thanks, Artem! Could you please add the
following copyright to the footer of the pages?

*© 2020 The Apache Software Foundation.*
*Apache, Apache Ignite, the Apache feather and the Apache Ignite logo are
either registered trademarks or trademarks of The Apache Software
Foundation. *
*Privacy Policy*

-
Denis


On Thu, Aug 27, 2020 at 5:20 AM Artem Budnikov 
wrote:

> Hi everyone,
>
> We published the draft of Ignite 2.9 documentation on the Apache Ignite
> web-site. The docs are available via the following link:
>
> https://ignite.apache.org/docs/2.9.0/installation/installing-using-docker
>
> Alex,
>
> Is there an estimate for the release date?
>
> -Artem
>
> On 26.08.2020 17:47, Alex Plehanov wrote:
> > Denis,
> >
> > Currently, we are running mostly IgnitePutTxImplicitBenchmark without
> > persistence. For other benchmarks drop is lower and it's harder to find
> > problematic commit.
> >
> > ср, 26 авг. 2020 г. в 17:34, Denis Magda :
> >
> >> Alex,
> >>
> >> Thanks for sending an update. The drop is quite big. What are the types
> of
> >> benchmarks you are observing the degradation for (atomic puts,
> >> transactions, sql, etc.)?
> >>
> >> Let us know if any help by particular committers is required.
> >>
> >> -
> >> Denis
> >>
> >>
> >> On Wed, Aug 26, 2020 at 12:26 AM Alex Plehanov  >
> >> wrote:
> >>
> >>> Hello, guys!
> >>>
> >>> We finally have some benchmark results. Looks like there is more than
> one
> >>> commit with a performance drop. Detected drops for those commits only
> >>> slightly higher than measurement error, so it was hard to find them and
> >> we
> >>> are not completely sure we found them all and found them right.
> >>>
> >>> Drops detected:
> >>> 2-3% drop on commit 99b0e0143e0 (IGNITE-13060 Tracing: initial
> >>> implementation)
> >>> 2-3% drop on commit 65c30ec6947 (IGNITE-12568 MessageFactory is
> >> refactored
> >>> in order to detect registration of message with the same direct type)
> >>>
> >>> The total drop we have on our environment - 7-8% and perhaps there is
> >>> something else here (benchmarks still in progress, I will write if we
> >> find
> >>> more suspected commits).
> >>>
> >>> Ivan Artiukhov, can you please recheck mentioned above commits on your
> >>> environment?
> >>>
> >>>
> >>> чт, 20 авг. 2020 г. в 11:43, Ilya Kasnacheev <
> ilya.kasnach...@gmail.com
> >>> :
> >>>
>  Hello!
> 
>  Readme.io uses blue book :)
> 
>  https://apacheignite.readme.io/docs/performance-tips
> 
>  I was thinking of something along a blue circle with `i' in it, for
>  information items.
> 
>  Regards,
>  --
>  Ilya Kasnacheev
> 
> 
>  ср, 19 авг. 2020 г. в 18:29, Artem Budnikov <
> >> a.budnikov.ign...@gmail.com
>  :
> 
> >> Search does not seem to work.
> > It uses mockups right now, but it should be ready when the docs are
> > released.
> >
> >> I can see that note blocks are just annotated with "Note." Can we
> >>> have
> > some
> >> image there?
> > Do you have a preference as to which image you would like to see
> >> there?
> > -Artem
> >
> > On 19.08.2020 17:37, Ilya Kasnacheev wrote:
> >> Hello!
> >>
> >> Search does not seem to work. Are we going to have a proper search
> > results
> >> page? It is often the case that there's none.
> >>
> >> I can see that note blocks are just annotated with "Note." Can we
> >>> have
> > some
> >> image there? Example is
> >> http://64.227.57.229/docs/2.9.0/persistence/persistence-tuning
> >>
> >> Regards,
>

Re: Micronaut/Ignite

[Denis Magda]

Michael,

The docs look good enough for the release. Once the integration is
released, I'll update and extend my Micronaut tutorial elucidating on how
to use the integration in practice for various use cases.

Btw, are the ignite-core and ignite-cache Maven artifacts will be released
for a specific version of Ignite? Ideally, the version of Ignite needs to
be set in a pom.xml and the Micronaut integration should pick them up.

Also, what are we doing next - reviewing APIs of the ignite-cache module or
going through a review process of the ignite-core and merging it first to
the Micronaut repository?

-
Denis


On Wed, Aug 26, 2020 at 3:35 PM Michael Pollind  wrote:

> Also you need to sign the CLA since you add a commit to my branch.
>
> branch: https://github.com/pollend/micronaut-ignite/tree/feature/rework
> linked PR: https://github.com/micronaut-projects/micronaut-ignite/pull/33
>
> --
> Michael Pollind
>
> On Wed, Aug 26, 2020 at 3:29 PM Denis Magda  wrote:
>
> > Could you share the full path to the branch? Don't want to pick a wrong
> one
> > accidentally.
> >
> >
> > -
> > Denis
> >
> >
> > On Wed, Aug 26, 2020 at 2:47 PM Michael Pollind 
> > wrote:
> >
> > > so that clears up the extra flag for micronaut-cache. just need to
> > > intercept the default IgniteConfiguration bean and update the
> > > configuration. if you pull my latest changes from the rework branch you
> > can
> > > look at the docs for that micronaut module. `./gradlew docs`.
> Finalizing
> > > the docs should be the last part of this.  The configuration from my
> > > perspective seems pretty much there. Is there anything else that is
> > > missing?
> > >
> > > On Wed, Aug 26, 2020 at 1:31 PM Denis Magda  wrote:
> > >
> > > >  Nice catch!
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Wed, Aug 26, 2020 at 12:16 PM Michael Pollind  >
> > > > wrote:
> > > >
> > > > > One thing that just occurred to me for the
> > DefaultIgniteConfiguration.
> > > > you
> > > > > can actually just intercept the bean when it's created. This should
> > let
> > > > us
> > > > > use the enable flag from ignite-core for ignite-cache without the
> > > > > additional flag.
> > > > >
> > > > > @Singleton
> > > > > public class IgniteConfigurationFactoryInterceptor implements
> > > > > BeanCreatedEventListener {
> > > > > @Override
> > > > > public IgniteConfiguration
> > > > > onCreated(BeanCreatedEvent event) {
> > > > > IgniteConfiguration configuration = event.getBean();
> > > > > configuration.setIgniteInstanceName("instance-a");
> > > > > return configuration;
> > > > > }
> > > > > }
> > > > >
> > > > >
> > > > > On Wed, Aug 26, 2020 at 11:31 AM Denis Magda 
> > > wrote:
> > > > >
> > > > > > Michael,
> > > > > >
> > > > > > Looks terrific! I went ahead and grouped the configuration
> samples
> > > by a
> > > > > > client type:
> > > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Micronaut+Integration
> > > > > >
> > > > > > As you see, the "Ignite Thin Client" section is blank for now.
> > Would
> > > > you
> > > > > > mind filling it with configuration options following the format
> > taken
> > > > for
> > > > > > the thick client?
> > > > > >
> > > > > > Ping me here once you're ready to review the API and
> implementation
> > > of
> > > > > the
> > > > > > ignite-caching module.
> > > > > >
> > > > > > -
> > > > > > Denis
> > > > > >
> > > > > >
> > > > > > On Tue, Aug 25, 2020 at 9:16 PM Michael Pollind <
> > mpoll...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Here is a sample project I threw together and I've also updated
> > the
> > > > > wiki
> > > > > > > just a bit. working on updating the documentation on the
> > micronaut
> > > > end.
> > > > > > >
> > > > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Micronaut+Integration
> > > > > > > https://github.com/pollend/micronaut-ignite-sample
> > > > > > >
> > > > > > > On Mon, Aug 24, 2020 at 6:57 PM Michael Pollind <
> > > mpoll...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > yea, I'm not sure about ignite-cache config. splitting it
> into
> > > two
> > > > > > > modules
> > > > > > > > seems kind of overly complicated, but I also don't really
> like
> > > the
> > > > > > random
> > > > > > > > boolean flag. here is the tweaks from the config your
> provider:
> > > > > > > > https://github.com/pollend/micronaut-ignite/pull/2
> > > > > > > >
> > > > > > > > kind of strange that there are not getters for the
> > > > > > > > TcpDiscoveryKubernetesIpFinder but there is enough to verify
> > that
> > > > the
> > > > > > > > correct ipfinder was set from the unit test i've written.
> > > > > > > >
> > > > > > > > static ip finder
> > > > > > > >
> > > > > > > > "ignite.enabled": true,
> > > > > > > > "ignite.communication-spi.local-port"   :
> > > > > "localhost:1800",
> > > > > > > > 

Re: IEP-52: Binary Delivery & Upgradability Enhancements

[Valentin Kulichenko]

Hi Petr,

The proposal makes sense to me - thanks for starting the discussion.
Current installation and upgrade procedures involve a lot of manual steps
and quite error-prone, we need to automate and simplify them as much as
possible.

I believe we eventually should end up with a unified command-line tool
(ignitectl?) that will incorporate all the operations (enable/disable
modules, start/stop, configuration updates, activation, baseline, etc.).
This IEP is a step in this direction.

Looking forward to testing a prototype :)

-Val

On Thu, Aug 27, 2020 at 2:17 AM Petr Ivanov  wrote:

> Hi, Igniters!
>
>
> For Apache Ignite 3.0 I would like to propose vision of enhanced delivery
> and upgrade processes [1].
> The key idea is to make main binary as slim as possible (delivering every
> optional component by demand only) while providing way to run each new
> version seamlessly without much of the efforts migrating data and
> configuration between upgrades.
>
> I plan to create prototype based on current release of Apache Ignite
> (2.8.1 or 2.9.0 if it will be released soon) later in September.
>
>
>
> [1]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158873958
>
>

Re: [DISCUSSION] Maintenance Mode feature

[Sergey Chugunov]

Hello Nikolay,

Created one, available by link [1]

Initially there was an intention to develop it under IEP-47 [2] and there
is even a separate section for Maintenance Mode there.
But it looks like this feature is useful in more cases and deserves its own
IEP.

[1]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-53%3A+Maintenance+Mode
[2]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-47:+Native+persistence+defragmentation

On Thu, Aug 27, 2020 at 11:01 AM Nikolay Izhikov 
wrote:

> Hello, Sergey!
>
> Thanks for the proposal.
> Let’s have IEP for this feature.
>
> > 27 авг. 2020 г., в 10:25, Sergey Chugunov 
> написал(а):
> >
> > Hello Igniters,
> >
> > I want to start a discussion about new supporting feature that could be
> > very useful in many scenarios where persistent storage is involved:
> > Maintenance Mode.
> >
> > *Summary*
> > Maintenance Mode (MM for short) is a special state of Ignite node when
> node
> > doesn't serve user requests nor joins the cluster but waits for user
> > commands or performs automatic actions for maintenance purposes.
> >
> > *Motivation*
> > There are situations when node cannot participate in regular operations
> but
> > at the same time should not be shut down.
> >
> > One example is a ticket [1] where I developed the first draft of
> > Maintenance Mode.
> > Here we get into a situation when node has potentially corrupted PDS thus
> > cannot proceed with restore routine and join the cluster as usual.
> > At the same time node should not fail nor be stopped for manual cleanup.
> > Manual cleanup is not always an option (e.g. restricted access to file
> > system); in managed environments failed node will be restarted
> > automatically so user won't have time for performing necessary
> operations.
> > Thus node needs to function in a special mode allowing user to connect to
> > it and perform necessary actions.
> >
> > Another example is described in IEP-47 [2] where defragmentation is being
> > developed. Node defragmenting its PDS should not join the cluster until
> the
> > process is finished so it needs to enter Maintenance Mode as well.
> >
> > *Suggested design*
> > I suggest MM to work as follows:
> > 1. Node enters MM if special markers are found on disk. These markers
> > called Maintenance Records could be created automatically (e.g. when
> > storage component detects corrupted storage) or by user request (when
> user
> > requests defragmentation of some caches). So entering MM requires node
> > restart.
> > 2. Started in MM node doesn't join the cluster but finishes startup
> routine
> > so it is able to receive commands and provide metrics to the user.
> > 3. When all necessary maintenance operations are finished, Maintenance
> > Records for these operations are deleted from disk and node restarted
> again
> > to enter normal service.
> >
> > *Example*
> > To put it into a context let's consider an example of how I see the MM
> > workflow in case of PDS corruption.
> >
> >   1. Node has failed in the middle of checkpoint when WAL is disabled for
> >   a particular cache -> data files of the cache are potentially
> corrupted.
> >   2. On next startup node detects this situation, creates Maintenance
> >   Record on disk and shuts down.
> >   3. On next startup node sees Maintenance Record, enters Maintenance
> Mode
> >   and waits for user to do specific actions: clean potentially corrupted
> PDS.
> >   4. When user has done necessary actions he/she removes Maintenance
> >   Record using Maintenance Mode API exposed via control.{sh|bat} script
> or
> >   JMX.
> >   5. On next startup node goes to normal operations as maintenance reason
> >   is fixed.
> >
> >
> > I prepared a PR [3] for ticket [1] with draft implementation. It is not
> > ready to be merged to master branch but is already fully functional and
> can
> > be reviewed.
> >
> > Hope you'll share your feedback on the feature and/or any thoughts on
> > implementation.
> >
> > Thank you!
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-13366
> > [2]
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-47:+Native+persistence+defragmentation
> > [3] https://github.com/apache/ignite/pull/8189
>
>

[jira] [Created] (IGNITE-13389) Add possibility to obtain full stack trace on thin client side.

[Stanilovsky Evgeny (Jira)]

Stanilovsky Evgeny created IGNITE-13389:
---

 Summary: Add possibility to obtain full stack trace on thin client 
side.
 Key: IGNITE-13389
 URL: https://issues.apache.org/jira/browse/IGNITE-13389
 Project: Ignite
  Issue Type: Improvement
  Components: thin client
Affects Versions: 2.8.1
Reporter: Stanilovsky Evgeny
Assignee: Stanilovsky Evgeny


Some server side errors have deeply nested _suppressed_ or _caused by_ errors 
which contains informative messages for further problem recognition. Possible 
such mechanism need to be disabled on production environment. Example of non 
informative error on client side:
{noformat}
org.apache.ignite.internal.client.thin.ClientServerError: Ignite failed to 
process request [1]: Failed to update keys (retry update if possible).: [1] 
(server status code [1])
{noformat}
but full stack holds the root case:

{noformat}
Caused by: class 
org.apache.ignite.internal.processors.cache.CachePartialUpdateCheckedException: 
Failed to update keys (retry update if possible).: [1]
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.onPrimaryError(GridNearAtomicAbstractUpdateFuture.java:397)
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2567)
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2544)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1316)
... 13 more
Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to 
update keys.
... 23 more
Suppressed: class org.apache.ignite.IgniteCheckedException: 
Runtime failure on search row: SearchRow
... 25 more
Caused by: class org.apache.ignite.IgniteCheckedException: 
org.apache.ignite.internal.binary.BinaryObjectImpl cannot be cast to 
org.apache.ignite.client.IgniteBinaryTest$ThinBinaryValue <-- here !!!
at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.update(GridCacheMapEntry.java:6379)
... 30 more
{noformat}

looks like it would be useful to have additional setting in 
ThinClientConfiguration configured both as direct setting and through JMX 
(ClientProcessorMXBean).




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]

[Artem Budnikov]

Hi everyone,

We published the draft of Ignite 2.9 documentation on the Apache Ignite 
web-site. The docs are available via the following link:


https://ignite.apache.org/docs/2.9.0/installation/installing-using-docker

Alex,

Is there an estimate for the release date?

-Artem

On 26.08.2020 17:47, Alex Plehanov wrote:

Denis,

Currently, we are running mostly IgnitePutTxImplicitBenchmark without
persistence. For other benchmarks drop is lower and it's harder to find
problematic commit.

ср, 26 авг. 2020 г. в 17:34, Denis Magda :


Alex,

Thanks for sending an update. The drop is quite big. What are the types of
benchmarks you are observing the degradation for (atomic puts,
transactions, sql, etc.)?

Let us know if any help by particular committers is required.

-
Denis


On Wed, Aug 26, 2020 at 12:26 AM Alex Plehanov 
wrote:


Hello, guys!

We finally have some benchmark results. Looks like there is more than one
commit with a performance drop. Detected drops for those commits only
slightly higher than measurement error, so it was hard to find them and

we

are not completely sure we found them all and found them right.

Drops detected:
2-3% drop on commit 99b0e0143e0 (IGNITE-13060 Tracing: initial
implementation)
2-3% drop on commit 65c30ec6947 (IGNITE-12568 MessageFactory is

refactored

in order to detect registration of message with the same direct type)

The total drop we have on our environment - 7-8% and perhaps there is
something else here (benchmarks still in progress, I will write if we

find

more suspected commits).

Ivan Artiukhov, can you please recheck mentioned above commits on your
environment?


чт, 20 авг. 2020 г. в 11:43, Ilya Kasnacheev 
Hello!

Readme.io uses blue book :)

https://apacheignite.readme.io/docs/performance-tips

I was thinking of something along a blue circle with `i' in it, for
information items.

Regards,
--
Ilya Kasnacheev


ср, 19 авг. 2020 г. в 18:29, Artem Budnikov <

a.budnikov.ign...@gmail.com

:


Search does not seem to work.

It uses mockups right now, but it should be ready when the docs are
released.


I can see that note blocks are just annotated with "Note." Can we

have

some

image there?

Do you have a preference as to which image you would like to see

there?

-Artem

On 19.08.2020 17:37, Ilya Kasnacheev wrote:

Hello!

Search does not seem to work. Are we going to have a proper search

results

page? It is often the case that there's none.

I can see that note blocks are just annotated with "Note." Can we

have

some

image there? Example is
http://64.227.57.229/docs/2.9.0/persistence/persistence-tuning

Regards,

Re: apache-ignite_2.8.0-1_all.deb package has older version of openjdk as dependency

[Artem Budnikov]

I've created a JIRA issue: 
https://issues.apache.org/jira/browse/IGNITE-13388


-Artem

On 27.08.2020 12:05, Petr Ivanov wrote:

Hi, Artem,


Can you file and issue with description what exactly do not work and 
why, please?



On 27 Aug 2020, at 11:55, Artem Budnikov > wrote:


Looks like this issue wasn't fixed in 2.8.1. The Ignite deb package 
v. 2.8.1 can't be installed on debian 10:




$ sudo apt-get install apache-ignite
...
The following packages have unmet dependencies:
 apache-ignite : Depends: openjdk-8-jdk but it is not installable or
    oracle-java8-installer but it is not installable
E: Unable to correct problems, you have held broken packages.



Is anyone working on this? I was going to update the Ignite 2.9 
installation instruction for Debian, but if the issue is staying with 
us, I guess I shouldn't bother.


-Artem

On 27.05.2020 11:55, Ilya Kasnacheev wrote:

Hello!

You can build debian package from apache-ignite source deliverable. 
There you can fix the dependency.


Regards,
--
Ilya Kasnacheev


ср, 27 мая 2020 г. в 11:51, rakshita04 
>:


Can we have the newer debian package for apache ignite with
newer version of
openjdk as dependencies?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

[jira] [Created] (IGNITE-13388) apache-ignite deb package depends on the non-existent package and can't be installed on Debian 10

[Artem Budnikov (Jira)]

Artem Budnikov created IGNITE-13388:
---

 Summary: apache-ignite deb package depends on the non-existent 
package and can't be installed on Debian 10
 Key: IGNITE-13388
 URL: https://issues.apache.org/jira/browse/IGNITE-13388
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.8.1
Reporter: Artem Budnikov


The apache-ignite deb package v. 2.8.1 depends on 'openjdk-8-jdk', which is 
only available in Debian stretch and isn't available in later Debian 
distributions. An attempt to install the package gives this error:
{code:java}
$ sudo apt-get install apache-ignite=2.8.1-1
...
The following packages have unmet dependencies:
 apache-ignite : Depends: openjdk-8-jdk but it is not installable or
  oracle-java8-installer but it is not installable
E: Unable to correct problems, you have held broken packages{code}
 Package information:
{code:java}
$ apt-cache show apache-ignite=2.8.1-1
Package: apache-ignite
Version: 2.8.1-1
Architecture: all
Maintainer: Petr Ivanov 
Installed-Size: 572071
Depends: openjdk-8-jdk | oracle-java8-installer, systemd, passwd
Section: misc
Priority: optional
Homepage: https://ignite.apache.org
Description: Apache Ignite In-Memory Computing, Database and Caching Platform
 Ignite™ is a memory-centric distributed database, caching, and processing
 platform for transactional, analytical, and streaming workloads, delivering
 in-memory speeds at petabyte scale
Description-md5: 6a59db03fa1e142387abef6ef6bb0d83
Filename: pool/main/a/apache-ignite_2.8.1-1_all.deb
SHA1: 67d197a5e582f6ea7c66da26a755f937f8e16fc9
SHA256: fc9a274ecb82716905d4120a715e9c74441dfed67831874eb3c35c4953bfc90d
Size: 399746094

{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

IEP-52: Binary Delivery & Upgradability Enhancements

[Petr Ivanov]

Hi, Igniters!


For Apache Ignite 3.0 I would like to propose vision of enhanced delivery and 
upgrade processes [1].
The key idea is to make main binary as slim as possible (delivering every 
optional component by demand only) while providing way to run each new version 
seamlessly without much of the efforts migrating data and configuration between 
upgrades.

I plan to create prototype based on current release of Apache Ignite (2.8.1 or 
2.9.0 if it will be released soon) later in September.



[1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158873958

Re: apache-ignite_2.8.0-1_all.deb package has older version of openjdk as dependency

[Petr Ivanov]

Hi, Artem,


Can you file and issue with description what exactly do not work and why, 
please?


> On 27 Aug 2020, at 11:55, Artem Budnikov  wrote:
> 
> Looks like this issue wasn't fixed in 2.8.1. The Ignite deb package v. 2.8.1 
> can't be installed on debian 10:
> 
> 
>> $ sudo apt-get install apache-ignite
>> ...
>> The following packages have unmet dependencies:
>>  apache-ignite : Depends: openjdk-8-jdk but it is not installable or
>> oracle-java8-installer but it is not installable
>> E: Unable to correct problems, you have held broken packages.
> 
> 
> Is anyone working on this? I was going to update the Ignite 2.9 installation 
> instruction for Debian, but if the issue is staying with us, I guess I 
> shouldn't bother.
> 
> -Artem
> 
> On 27.05.2020 11:55, Ilya Kasnacheev wrote:
>> Hello!
>> 
>> You can build debian package from apache-ignite source deliverable. There 
>> you can fix the dependency.
>> 
>> Regards,
>> -- 
>> Ilya Kasnacheev
>> 
>> 
>> ср, 27 мая 2020 г. в 11:51, rakshita04 > >:
>> Can we have the newer debian package for apache ignite with newer version of
>> openjdk as dependencies?
>> 
>> 
>> 
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 
>> 

Re: apache-ignite_2.8.0-1_all.deb package has older version of openjdk as dependency

[Artem Budnikov]

Looks like this issue wasn't fixed in 2.8.1. The Ignite deb package v. 
2.8.1 can't be installed on debian 10:



$ sudo apt-get install apache-ignite
...
The following packages have unmet dependencies:
 apache-ignite : Depends: openjdk-8-jdk but it is not installable or
    oracle-java8-installer but it is not installable
E: Unable to correct problems, you have held broken packages.


Is anyone working on this? I was going to update the Ignite 2.9 
installation instruction for Debian, but if the issue is staying with 
us, I guess I shouldn't bother.


-Artem

On 27.05.2020 11:55, Ilya Kasnacheev wrote:

Hello!

You can build debian package from apache-ignite source deliverable. 
There you can fix the dependency.


Regards,
--
Ilya Kasnacheev


ср, 27 мая 2020 г. в 11:51, rakshita04 >:


Can we have the newer debian package for apache ignite with newer
version of
openjdk as dependencies?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: [DISCUSSION] Maintenance Mode feature

[Nikolay Izhikov]

Hello, Sergey!

Thanks for the proposal.
Let’s have IEP for this feature.

> 27 авг. 2020 г., в 10:25, Sergey Chugunov  
> написал(а):
> 
> Hello Igniters,
> 
> I want to start a discussion about new supporting feature that could be
> very useful in many scenarios where persistent storage is involved:
> Maintenance Mode.
> 
> *Summary*
> Maintenance Mode (MM for short) is a special state of Ignite node when node
> doesn't serve user requests nor joins the cluster but waits for user
> commands or performs automatic actions for maintenance purposes.
> 
> *Motivation*
> There are situations when node cannot participate in regular operations but
> at the same time should not be shut down.
> 
> One example is a ticket [1] where I developed the first draft of
> Maintenance Mode.
> Here we get into a situation when node has potentially corrupted PDS thus
> cannot proceed with restore routine and join the cluster as usual.
> At the same time node should not fail nor be stopped for manual cleanup.
> Manual cleanup is not always an option (e.g. restricted access to file
> system); in managed environments failed node will be restarted
> automatically so user won't have time for performing necessary operations.
> Thus node needs to function in a special mode allowing user to connect to
> it and perform necessary actions.
> 
> Another example is described in IEP-47 [2] where defragmentation is being
> developed. Node defragmenting its PDS should not join the cluster until the
> process is finished so it needs to enter Maintenance Mode as well.
> 
> *Suggested design*
> I suggest MM to work as follows:
> 1. Node enters MM if special markers are found on disk. These markers
> called Maintenance Records could be created automatically (e.g. when
> storage component detects corrupted storage) or by user request (when user
> requests defragmentation of some caches). So entering MM requires node
> restart.
> 2. Started in MM node doesn't join the cluster but finishes startup routine
> so it is able to receive commands and provide metrics to the user.
> 3. When all necessary maintenance operations are finished, Maintenance
> Records for these operations are deleted from disk and node restarted again
> to enter normal service.
> 
> *Example*
> To put it into a context let's consider an example of how I see the MM
> workflow in case of PDS corruption.
> 
>   1. Node has failed in the middle of checkpoint when WAL is disabled for
>   a particular cache -> data files of the cache are potentially corrupted.
>   2. On next startup node detects this situation, creates Maintenance
>   Record on disk and shuts down.
>   3. On next startup node sees Maintenance Record, enters Maintenance Mode
>   and waits for user to do specific actions: clean potentially corrupted PDS.
>   4. When user has done necessary actions he/she removes Maintenance
>   Record using Maintenance Mode API exposed via control.{sh|bat} script or
>   JMX.
>   5. On next startup node goes to normal operations as maintenance reason
>   is fixed.
> 
> 
> I prepared a PR [3] for ticket [1] with draft implementation. It is not
> ready to be merged to master branch but is already fully functional and can
> be reviewed.
> 
> Hope you'll share your feedback on the feature and/or any thoughts on
> implementation.
> 
> Thank you!
> 
> [1] https://issues.apache.org/jira/browse/IGNITE-13366
> [2]
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-47:+Native+persistence+defragmentation
> [3] https://github.com/apache/ignite/pull/8189

[DISCUSSION] Maintenance Mode feature

[Sergey Chugunov]

Hello Igniters,

I want to start a discussion about new supporting feature that could be
very useful in many scenarios where persistent storage is involved:
Maintenance Mode.

*Summary*
Maintenance Mode (MM for short) is a special state of Ignite node when node
doesn't serve user requests nor joins the cluster but waits for user
commands or performs automatic actions for maintenance purposes.

*Motivation*
There are situations when node cannot participate in regular operations but
at the same time should not be shut down.

One example is a ticket [1] where I developed the first draft of
Maintenance Mode.
Here we get into a situation when node has potentially corrupted PDS thus
cannot proceed with restore routine and join the cluster as usual.
At the same time node should not fail nor be stopped for manual cleanup.
Manual cleanup is not always an option (e.g. restricted access to file
system); in managed environments failed node will be restarted
automatically so user won't have time for performing necessary operations.
Thus node needs to function in a special mode allowing user to connect to
it and perform necessary actions.

Another example is described in IEP-47 [2] where defragmentation is being
developed. Node defragmenting its PDS should not join the cluster until the
process is finished so it needs to enter Maintenance Mode as well.

*Suggested design*
I suggest MM to work as follows:
1. Node enters MM if special markers are found on disk. These markers
called Maintenance Records could be created automatically (e.g. when
storage component detects corrupted storage) or by user request (when user
requests defragmentation of some caches). So entering MM requires node
restart.
2. Started in MM node doesn't join the cluster but finishes startup routine
so it is able to receive commands and provide metrics to the user.
3. When all necessary maintenance operations are finished, Maintenance
Records for these operations are deleted from disk and node restarted again
to enter normal service.

*Example*
To put it into a context let's consider an example of how I see the MM
workflow in case of PDS corruption.

   1. Node has failed in the middle of checkpoint when WAL is disabled for
   a particular cache -> data files of the cache are potentially corrupted.
   2. On next startup node detects this situation, creates Maintenance
   Record on disk and shuts down.
   3. On next startup node sees Maintenance Record, enters Maintenance Mode
   and waits for user to do specific actions: clean potentially corrupted PDS.
   4. When user has done necessary actions he/she removes Maintenance
   Record using Maintenance Mode API exposed via control.{sh|bat} script or
   JMX.
   5. On next startup node goes to normal operations as maintenance reason
   is fixed.


I prepared a PR [3] for ticket [1] with draft implementation. It is not
ready to be merged to master branch but is already fully functional and can
be reviewed.

Hope you'll share your feedback on the feature and/or any thoughts on
implementation.

Thank you!

[1] https://issues.apache.org/jira/browse/IGNITE-13366
[2]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-47:+Native+persistence+defragmentation
[3] https://github.com/apache/ignite/pull/8189