Re: Proposal of new event QUERY_EXECUTION_EVENT

[Dmitrii Ryabov]

Any objections to create a separate event, which will be fired before
executing a query?

ср, 2 сент. 2020 г. в 22:33, Dmitrii Ryabov :
>
> I agree with Max, we need to add a separate event for starting query
> execution, and EVT_CACHE_QUERY_EXECUTED shouldn't be deprecated,
> because it is another case - it is fired when cache query was
> successfully finished.
>
> > Would the event notification be synchronous? In which thread?
> As Max said, synchronicity depends on implementation. As I see, we
> don't use a separate thread for any record calls.
>
> > What happens in case the event listener fails?
> Exceptions are logged by `U.error(...)` call. Errors are thrown out.
>
> > Should we discuss this within this topic?
> I suggest to separate adding a new event and configuring existing events.
>
> пн, 20 июл. 2020 г. в 14:37, Max Timonin :
> >
> > Looks like EVT_CACHE_QUERY_EXECUTED just works for different use cases:
> > 1. it relates to a specific cache (Event for SQL queries looks different as
> > it could contain multiple caches or none of them);
> > 2. Also the EVT_CACHE_QUERY_EXECUTED event fires multiple times for
> > distributed queries, see GridMapQueryExecutor class (For SQL query it's
> > required to fire once independently on how many nodes are affected).
> >
> > So there are different patterns for events. I think
> > EVT_CACHE_QUERY_EXECUTED should not be deprecated or changed.
> >
> > > What happens in case the event listener fails?
> > > Would the event notification be synchronous?
> > It depends on how other events are implemented. As I see for the
> > EVT_CACHE_QUERY_EXECUTED event - it's synchronous, and listener errors
> > aren't handled.
> >
> > I think these questions are related to GridEventStorageManager as it just
> > provides an API for recording events. Internal implementations (sync
> > / async, error handling) is not related to an event, is it?
> >
> > > I have some doubts about provide text of a query even with
> > hidden arguments, probably it should be configured due to it could lead
> > to security leak
> > Currently event EVT_CACHE_QUERY_EXECUTED provides a sql clause without
> > limitations. If we're going to provide some restrictions it will require
> > additional investigation. I see at least 2 configurations here:
> > 1. Ignite can be configured to hide clase, params only or nothing for all
> > listeners;
> > 2. Only authorized listeners can subscribe to the event.
> >
> > Should we discuss this within this topic?
> >
> > On Mon, Jul 20, 2020 at 1:55 PM Юрий  wrote:
> >
> > > In my opinion existing events EVT_CACHE_QUERY_EXECUTION_STARTED should be
> > > deprecated and added two new for start and for end of queries which should
> > > cover all SQL query types.
> > > I have some doubts about provide text of a query even with hidden
> > > arguments, probably it should be configured due to it could lead to
> > > security leak
> > >
> > > пн, 20 июл. 2020 г. в 12:49, Stanislav Lukyanov :
> > >
> > > > Maksim,
> > > >
> > > > Can we change the EVT_CACHE_QUERY_EXECUTED to fire earlier? Or should
> > > > there be an EVT_CACHE_QUERY_EXECUTION_STARTED for the query start, while
> > > > the old event would continue to work for query finish?
> > > > I think the new event needs to either reuse the old one, or be its 
> > > > mirror
> > > > for the query start. It also means that we probably should resolve the
> > > > issues you've listed.
> > > >
> > > > Would the event notification be synchronous? In which thread?
> > > Asynchronous
> > > > is generally preferred - would it work?
> > > >
> > > > What happens in case the event listener fails?
> > > >
> > > > Thanks,
> > > > Stan
> > > >
> > > > > On 16 Jul 2020, at 18:49, Denis Magda  wrote:
> > > > >
> > > > > Taras, Yury, Ivan,
> > > > >
> > > > > Could you please join this thread and share your thoughts? Do we
> > > already
> > > > > have any plans on tracking of the DDL and DML queries?
> > > > >
> > > > > -
> > > > > Denis
> > > > >
> > > > >
> > > > > On Wed, Jul 15, 2020 at 12:09 AM Max Timonin 
> > > > > wrote:
> > > > >
> > > > >> Hi Denis, thanks for the answer!
> > > > >>
> > > > >> We already checked EVT_CACHE_QUERY_EXECUTED and found that it works
> > > > only in
> > > > >> cases:
> > > > >> 1. Scan queries and Select queries (common pattern is access to cache
> > > > >> data);
> > > > >> 2. This event triggers only if query execution succeeds, in case of
> > > > failure
> > > > >> while execution this event won't fire.
> > > > >>
> > > > >> Our additional requirements are to protocol queries:
> > > > >> 1. that aren't cache related (for example, alter user);
> > > > >> 2. that relate to multiple caches (while EVT_CACHE_QUERY_EXECUTED 
> > > > >> have
> > > > >> field cacheName related to specific cache);
> > > > >> 3. we need to protocol also DDL and DML queries.
> > > > >>
> > > > >> Regards,
> > > > >> Maksim
> > > > >>
> > > > >> On Tue, Jul 14, 2020 at 10:20 PM Denis Magda 
> > > wrote:
> > > > >>
> > > > >>> Hi 

Re: IEP-54: Schema-first approach for 3.0

[Valentin Kulichenko]

Ivan,

I see your point. I agree that with the automatic updates we step into the
schema-last territory.

Actually, if we support automatic evolution, we can as well support
creating a cache without schema and inferring it from the first insert. In
other words, we can have both "schema-first" and "schema-last" modes.

Alexey, what do you think?

-Val

On Mon, Sep 7, 2020 at 5:59 AM Alexey Goncharuk 
wrote:

> Ivan,
>
> Thank you, I got your concern now. As it is mostly regarding the
> terminology, I am absolutely fine with changing the name to whatever fits
> the approach best. Dynamic or evolving schema sounds great. I will make
> corresponding changes to the IEP once we settle on the name.
>
> пн, 7 сент. 2020 г. в 11:33, Ivan Pavlukhin :
>
> > Hi Val,
> >
> > Thank you for your answer!
> >
> > My understanding is a little bit different. Yes, schema evolution
> > definitely should be possible. But I see a main difference in "how
> > schema is updated". I treat a common SQL approach schema-first. Schema
> > and data manipulation operations are clearly separated and it enables
> > interesting capabilities, e.g. preventing untended schema changes by
> > mistaken data operations, restricting user permissions to change
> > schema.
> >
> > > Schema-first means that schema exists in advance and all the stored
> data
> > is compliant with it - that's exactly what is proposed.
> >
> > A schema-last approach mentioned in [1] also assumes that schema
> > exists, but it is inferred from data. Is not it more similar to the
> > proposing approach?
> >
> > And I would like to say, that my main concern so far is mostly about
> > terminology. And I suppose if it confuses me then others might be
> > confused as well. My feeling is closer to "dynamic or liquid or may be
> > evolving schema".
> >
> > [1]
> > https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf
> >
> > 2020-09-07 0:47 GMT+03:00, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com>:
> > > Hi Ivan,
> > >
> > > I don't see an issue with that. Schema-first means that schema exists
> in
> > > advance and all the stored data is compliant with it - that's exactly
> > what
> > > is proposed. There are no restrictions prohibiting changes to the
> schema.
> > >
> > > -Val
> > >
> > > On Sat, Sep 5, 2020 at 9:52 PM Ivan Pavlukhin 
> > wrote:
> > >
> > >> Alexey,
> > >>
> > >> I am a little bit confused with terminology. My understanding conforms
> > >> to a survey [1] (see part X Semi Structured Data). Can we really treat
> > >> a "dynamic schema" approach as a kind of "schema-first"?
> > >>
> > >> [1]
> > >>
> https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf
> > >>
> > >> 2020-09-02 1:53 GMT+03:00, Denis Magda :
> > >> >>
> > >> >> However, could you please elaborate on the relation between Ignite
> > and
> > >> >> ORM?
> > >> >> Is there a use case for Hibernate running on top of Ignite (I
> haven't
> > >> >> seen
> > >> >> one so far)? If so, what is missing exactly on the Ignite side to
> > >> support
> > >> >> this? In my understanding, all you need is SQL API which we already
> > >> have.
> > >> >> Am I missing something?
> > >> >
> > >> >
> > >> > Good point, yes, if all the ORM integrations use Ignite SQL APIs
> > >> > internally, then they can easily translate an Entity object into an
> > >> > INSERT/UPDATE statement that lists all the object's fields. Luckily,
> > >> > our
> > >> > Spring Data integration is already based on the Ignite SQL APIs and
> > >> > needs
> > >> > to be improved once the schema-first approach is supported. That
> would
> > >> > solve a ton of usability issues.
> > >> >
> > >> > I would revise the Hibernate integration as well during the Ignite
> 3.0
> > >> dev
> > >> > phase. Can't say if it's used a lot but Spring Data is getting
> > traction
> > >> for
> > >> > sure.
> > >> >
> > >> > @Michael Pollind, I'll loop you in as long as you've started working
> > on
> > >> the
> > >> > Ignite support for Micornaut Data
> > >> > 
> > and
> > >> > came across some challenges. Just watch this discussion. That's what
> > is
> > >> > coming in Ignite 3.0.
> > >> >
> > >> >
> > >> > -
> > >> > Denis
> > >> >
> > >> >
> > >> > On Mon, Aug 31, 2020 at 5:11 PM Valentin Kulichenko <
> > >> > valentin.kuliche...@gmail.com> wrote:
> > >> >
> > >> >> Hi Denis,
> > >> >>
> > >> >> Generally speaking, I believe that the schema-first approach
> natively
> > >> >> addresses the issue if duplicate fields in key and value objects,
> > >> because
> > >> >> schema will be created for a cache, not for an object, as it
> happens
> > >> now.
> > >> >> Basically, the schema will define whether there is a primary key or
> > >> >> not,
> > >> >> and which fields are included in case there is one. Any API that we
> > >> would
> > >> >> have must be compliant with this, so it becomes fairly easy to work
> > >> >> with
> > >> >> data as with a set of 

Re: [DISCUSSION] Output IgniteSystemProperties via ignite.sh

[Nikolay Izhikov]

Ilya.

> I'm just saying that we can make it possible as a general principle.

+1 to remove as many internal flags as possible.


> 7 сент. 2020 г., в 20:20, Ilya Kasnacheev  
> написал(а):
> 
> Hello!
> 
> I'm not arguing that it should not be discussed. I'm just saying that we
> can make it possible as a general principle.
> 
> Regards,
> -- 
> Ilya Kasnacheev
> 
> 
> пн, 7 сент. 2020 г. в 19:01, Nikolay Izhikov :
> 
>>> We can make a compromise here: we list flags explicitly *but then, we
>>> 
>>> decide that we don't keep backward compatibility anymore*, since the user
>>> of a new version can check whether their flag is still supported by using
>>> control.sh.
>> 
>> It seems removal of any IgniteSystemProperty flag should be discussed
>> separately.
>> 
>> 
>>> 7 сент. 2020 г., в 18:46, Ilya Kasnacheev 
>> написал(а):
>>> 
>>> Hello!
>>> 
>>> We do replace some flags with cfg properties, such as inline size, for
>>> example.
>>> A lot of other flags just duplicate cfg properties.
>>> 
>>> We can make a compromise here: we list flags explicitly *but then, we
>>> decide that we don't keep backward compatibility anymore*, since the user
>>> of a new version can check whether their flag is still supported by using
>>> control.sh.
>>> 
>>> WDYT?
>>> 
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>> 
>>> 
>>> пн, 7 сент. 2020 г. в 18:34, Nikolay Izhikov :
>>> 
 Ilya.
 
> to remove any expectation of forward compatibility.
 
 AFAIK we must keep these flags before Ignite3, due to the backward
 compatibility.
 
 
> Flags should be a temporary measure
 
 This is not true, for now.
 I feel your pain :)
 Personally, I hate these flags, also.
 
 But they exist and the whole point of this change is to make the life of
 Ignite users a bit easier.
 
> 7 сент. 2020 г., в 17:32, Philipp Masharov 
 написал(а):
> 
> Let me notice that in Ignite 3.0 Wishlist [1] we have a bullet point
> "Remove as many IGNITE_ parameters as possible from
 IgniteSystemProperties
> <
 
>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteSystemProperties.html
> "
> 
> 
> [1]
> 
 
>> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0+Wishlist
> "
> 
> On Mon, Sep 7, 2020 at 5:24 PM Ilya Kasnacheev <
 ilya.kasnach...@gmail.com>
> wrote:
> 
>> Hello!
>> 
>> Okay, we can do a simple list of these flags, but I would argue that
>> we
>> should:
>> - Avoid adding extra infrastructure such as flags' human readable
>> names,
>> embedded docs, etc. Flags should be a temporary measure. They are now.
>> Let's not make it look like they're there to stay.
>> - Explicitly mention in the -X output that these flags may be
>> removed/converted to configuration properties in the future, to remove
 any
>> expectation of forward compatibility.
>> 
>> They bother me for quite some time.
>> 
>> Regards,
>> --
>> Ilya Kasnacheev
>> 
>> 
>> пн, 7 сент. 2020 г. в 17:17, Nikolay Izhikov :
>> 
 what’s the logic?
>>> 
>>> I assume that this is a question to the author of these flags.
>>> If you have a specific flag you are interested in, please, write it.
>>> 
>>> My point is simple - we already have these flags.
>>> 
>>> We should explain to the user what we have and what can be configured
>> with
>>> these flags.
>>> 
>>> There is no flag added or removed in this change.
>>> Just makes it more clear to the users.
>>> 
>>> 
 7 сент. 2020 г., в 17:12, Stephen Darlington <
>>> stephen.darling...@gridgain.com> написал(а):
 
 But to Ilya’s point, what’s the logic? Why are some things set using
>>> IGNITE_ properties, others on the command line and others in
>>> IgniteConfiguration? It’s confusing for the user and makes
>> maintenance
>>> harder.
 
 I’m not necessarily arguing against this change, though. Perfect
>> being
>>> the enemy of good and all that.
 
> On 7 Sep 2020, at 13:02, Nikolay Izhikov 
 wrote:
> 
> Hello, Ilya.
> 
>> I think this is a bad idea since it legitimizes wide use of
>> IGNITE_
>>> properties, which shows weakness of our configuration API, etc.
> 
> We already have IGNITE options in the product as a part of public
 API.
>>> See `org.apache.ignite.IgniteSystemProperties`.
> 
>> 7 сент. 2020 г., в 14:40, Ilya Kasnacheev <
 ilya.kasnach...@gmail.com
>>> 
>>> написал(а):
>> 
>> Hello!
>> 
>> I think this is a bad idea since it legitimizes wide use of
>> IGNITE_
>> properties, which shows weakness of our configuration API, etc.
>> 
>> My take:
>> 
>> All of IGNITE_ properties which are 

Re: IGNITE-13407 Upgrade ignite.version to 2.10.0-SNAPSHOT in Ignite Extensions

[Saikat Maitra]

Hi Ivan,

My understanding is in teamcity the build for ignite-extensions is able to
pull dependencies from local staging repository
http://172.25.4.107/artifactory/gridgain-local

Travis is not able to pull the dependencies for the same.

I observed that we publish .NET packages in mynuget in nightly build
https://ci.ignite.apache.org/buildConfiguration/Releases_NightlyRelease_ApacheIgniteNightlyReleaseAssembleNugetPackages?branch==branches#all-projects

in the below path

https://www.myget.org/F/apache-ignite-nightly/api/v2/package

We will have to do similar step in the Assemble Binaries project
https://ci.ignite.apache.org/buildConfiguration/Releases_NightlyRelease_ApacheIgniteNightlyReleaseAssembleBinaries?branch=#all-projects

We used to publish nightly packages earlier in the apache
snapshot repository but we are no longer publishing the packages.

https://repository.apache.org/content/repositories/snapshots/org/apache/ignite/ignite-core/

I have created following Jira story for the same
https://issues.apache.org/jira/browse/IGNITE-13412

Regards,
Saikat


On Mon, Sep 7, 2020 at 3:43 AM Ivan Pavlukhin  wrote:

> Hi Saikat,
>
> Thank you for working on it!
>
> I noticed that Travis build for the PR failed to resolve
> 2.10.0-SNAPSHOT dependency as well.
>
> Could you please hint me how Ignite SNAPSHOT dependencies (ignite-core
> and so on) are intended to be available to ignite-extensions builds?
>
> 2020-09-06 22:01 GMT+03:00, Nikolay Izhikov :
> > +1
> >
> >> 6 сент. 2020 г., в 21:43, Saikat Maitra 
> >> написал(а):
> >>
> >> Hi,
> >>
> >> I observed that the Ignite Extensions build were failing as it was
> >> depending on 2.9.0-SNAPSHOT and it was not available in artifactory.
> >>
> >>
> https://ci.ignite.apache.org/buildConfiguration/Tests_IgniteExtensions_Build/5576078?buildTab=log=3=1777
> >>
> >> I have updated the dependencies and created a PR.
> >>
> >> Jira https://issues.apache.org/jira/browse/IGNITE-13407
> >>
> >> PR https://github.com/apache/ignite-extensions/pull/22
> >>
> >> Please review and share feedback.
> >>
> >> Regards,
> >> Saikat
> >
> >
>
>
> --
>
> Best regards,
> Ivan Pavlukhin
>

[jira] [Created] (IGNITE-13412) Publish nightly snapshot in apache repository

[Saikat Maitra (Jira)]

Saikat Maitra created IGNITE-13412:
--

 Summary: Publish nightly snapshot in apache repository 
 Key: IGNITE-13412
 URL: https://issues.apache.org/jira/browse/IGNITE-13412
 Project: Ignite
  Issue Type: Improvement
  Components: streaming
Reporter: Saikat Maitra


 
We used to publish nightly packages earlier in the apache snapshot repository 
but we are no longer publishing the packages.
 
[https://repository.apache.org/content/repositories/snapshots/org/apache/ignite/ignite-core/]
 
The last published package was 
[2.2.2-SNAPSHOT/|https://repository.apache.org/content/repositories/snapshots/org/apache/ignite/ignite-core/2.2.2-SNAPSHOT/]
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSSION] Output IgniteSystemProperties via ignite.sh

[Ilya Kasnacheev]

Hello!

I'm not arguing that it should not be discussed. I'm just saying that we
can make it possible as a general principle.

Regards,
-- 
Ilya Kasnacheev


пн, 7 сент. 2020 г. в 19:01, Nikolay Izhikov :

> > We can make a compromise here: we list flags explicitly *but then, we
> >
> > decide that we don't keep backward compatibility anymore*, since the user
> > of a new version can check whether their flag is still supported by using
> > control.sh.
>
> It seems removal of any IgniteSystemProperty flag should be discussed
> separately.
>
>
> > 7 сент. 2020 г., в 18:46, Ilya Kasnacheev 
> написал(а):
> >
> > Hello!
> >
> > We do replace some flags with cfg properties, such as inline size, for
> > example.
> > A lot of other flags just duplicate cfg properties.
> >
> > We can make a compromise here: we list flags explicitly *but then, we
> > decide that we don't keep backward compatibility anymore*, since the user
> > of a new version can check whether their flag is still supported by using
> > control.sh.
> >
> > WDYT?
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > пн, 7 сент. 2020 г. в 18:34, Nikolay Izhikov :
> >
> >> Ilya.
> >>
> >>> to remove any expectation of forward compatibility.
> >>
> >> AFAIK we must keep these flags before Ignite3, due to the backward
> >> compatibility.
> >>
> >>
> >>> Flags should be a temporary measure
> >>
> >> This is not true, for now.
> >> I feel your pain :)
> >> Personally, I hate these flags, also.
> >>
> >> But they exist and the whole point of this change is to make the life of
> >> Ignite users a bit easier.
> >>
> >>> 7 сент. 2020 г., в 17:32, Philipp Masharov 
> >> написал(а):
> >>>
> >>> Let me notice that in Ignite 3.0 Wishlist [1] we have a bullet point
> >>> "Remove as many IGNITE_ parameters as possible from
> >> IgniteSystemProperties
> >>> <
> >>
> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteSystemProperties.html
> >>> "
> >>>
> >>>
> >>> [1]
> >>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0+Wishlist
> >>> "
> >>>
> >>> On Mon, Sep 7, 2020 at 5:24 PM Ilya Kasnacheev <
> >> ilya.kasnach...@gmail.com>
> >>> wrote:
> >>>
>  Hello!
> 
>  Okay, we can do a simple list of these flags, but I would argue that
> we
>  should:
>  - Avoid adding extra infrastructure such as flags' human readable
> names,
>  embedded docs, etc. Flags should be a temporary measure. They are now.
>  Let's not make it look like they're there to stay.
>  - Explicitly mention in the -X output that these flags may be
>  removed/converted to configuration properties in the future, to remove
> >> any
>  expectation of forward compatibility.
> 
>  They bother me for quite some time.
> 
>  Regards,
>  --
>  Ilya Kasnacheev
> 
> 
>  пн, 7 сент. 2020 г. в 17:17, Nikolay Izhikov :
> 
> >> what’s the logic?
> >
> > I assume that this is a question to the author of these flags.
> > If you have a specific flag you are interested in, please, write it.
> >
> > My point is simple - we already have these flags.
> >
> > We should explain to the user what we have and what can be configured
>  with
> > these flags.
> >
> > There is no flag added or removed in this change.
> > Just makes it more clear to the users.
> >
> >
> >> 7 сент. 2020 г., в 17:12, Stephen Darlington <
> > stephen.darling...@gridgain.com> написал(а):
> >>
> >> But to Ilya’s point, what’s the logic? Why are some things set using
> > IGNITE_ properties, others on the command line and others in
> > IgniteConfiguration? It’s confusing for the user and makes
> maintenance
> > harder.
> >>
> >> I’m not necessarily arguing against this change, though. Perfect
> being
> > the enemy of good and all that.
> >>
> >>> On 7 Sep 2020, at 13:02, Nikolay Izhikov 
> >> wrote:
> >>>
> >>> Hello, Ilya.
> >>>
>  I think this is a bad idea since it legitimizes wide use of
> IGNITE_
> > properties, which shows weakness of our configuration API, etc.
> >>>
> >>> We already have IGNITE options in the product as a part of public
> >> API.
> > See `org.apache.ignite.IgniteSystemProperties`.
> >>>
>  7 сент. 2020 г., в 14:40, Ilya Kasnacheev <
> >> ilya.kasnach...@gmail.com
> >
> > написал(а):
> 
>  Hello!
> 
>  I think this is a bad idea since it legitimizes wide use of
> IGNITE_
>  properties, which shows weakness of our configuration API, etc.
> 
>  My take:
> 
>  All of IGNITE_ properties which are useful (and will go to -X)
> >> should
>  instead be turned into configuration/metastore settings.
>  All of IGNITE_ properties which are dangerous and/or useless
> should
>  be
>  removed.
> 
>  Regards,
>  --
>  Ilya 

Re: [DISCUSSION] Output IgniteSystemProperties via ignite.sh

[Nikolay Izhikov]

> We can make a compromise here: we list flags explicitly *but then, we
> 
> decide that we don't keep backward compatibility anymore*, since the user
> of a new version can check whether their flag is still supported by using
> control.sh.

It seems removal of any IgniteSystemProperty flag should be discussed 
separately.


> 7 сент. 2020 г., в 18:46, Ilya Kasnacheev  
> написал(а):
> 
> Hello!
> 
> We do replace some flags with cfg properties, such as inline size, for
> example.
> A lot of other flags just duplicate cfg properties.
> 
> We can make a compromise here: we list flags explicitly *but then, we
> decide that we don't keep backward compatibility anymore*, since the user
> of a new version can check whether their flag is still supported by using
> control.sh.
> 
> WDYT?
> 
> Regards,
> -- 
> Ilya Kasnacheev
> 
> 
> пн, 7 сент. 2020 г. в 18:34, Nikolay Izhikov :
> 
>> Ilya.
>> 
>>> to remove any expectation of forward compatibility.
>> 
>> AFAIK we must keep these flags before Ignite3, due to the backward
>> compatibility.
>> 
>> 
>>> Flags should be a temporary measure
>> 
>> This is not true, for now.
>> I feel your pain :)
>> Personally, I hate these flags, also.
>> 
>> But they exist and the whole point of this change is to make the life of
>> Ignite users a bit easier.
>> 
>>> 7 сент. 2020 г., в 17:32, Philipp Masharov 
>> написал(а):
>>> 
>>> Let me notice that in Ignite 3.0 Wishlist [1] we have a bullet point
>>> "Remove as many IGNITE_ parameters as possible from
>> IgniteSystemProperties
>>> <
>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteSystemProperties.html
>>> "
>>> 
>>> 
>>> [1]
>>> 
>> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0+Wishlist
>>> "
>>> 
>>> On Mon, Sep 7, 2020 at 5:24 PM Ilya Kasnacheev <
>> ilya.kasnach...@gmail.com>
>>> wrote:
>>> 
 Hello!
 
 Okay, we can do a simple list of these flags, but I would argue that we
 should:
 - Avoid adding extra infrastructure such as flags' human readable names,
 embedded docs, etc. Flags should be a temporary measure. They are now.
 Let's not make it look like they're there to stay.
 - Explicitly mention in the -X output that these flags may be
 removed/converted to configuration properties in the future, to remove
>> any
 expectation of forward compatibility.
 
 They bother me for quite some time.
 
 Regards,
 --
 Ilya Kasnacheev
 
 
 пн, 7 сент. 2020 г. в 17:17, Nikolay Izhikov :
 
>> what’s the logic?
> 
> I assume that this is a question to the author of these flags.
> If you have a specific flag you are interested in, please, write it.
> 
> My point is simple - we already have these flags.
> 
> We should explain to the user what we have and what can be configured
 with
> these flags.
> 
> There is no flag added or removed in this change.
> Just makes it more clear to the users.
> 
> 
>> 7 сент. 2020 г., в 17:12, Stephen Darlington <
> stephen.darling...@gridgain.com> написал(а):
>> 
>> But to Ilya’s point, what’s the logic? Why are some things set using
> IGNITE_ properties, others on the command line and others in
> IgniteConfiguration? It’s confusing for the user and makes maintenance
> harder.
>> 
>> I’m not necessarily arguing against this change, though. Perfect being
> the enemy of good and all that.
>> 
>>> On 7 Sep 2020, at 13:02, Nikolay Izhikov 
>> wrote:
>>> 
>>> Hello, Ilya.
>>> 
 I think this is a bad idea since it legitimizes wide use of IGNITE_
> properties, which shows weakness of our configuration API, etc.
>>> 
>>> We already have IGNITE options in the product as a part of public
>> API.
> See `org.apache.ignite.IgniteSystemProperties`.
>>> 
 7 сент. 2020 г., в 14:40, Ilya Kasnacheev <
>> ilya.kasnach...@gmail.com
> 
> написал(а):
 
 Hello!
 
 I think this is a bad idea since it legitimizes wide use of IGNITE_
 properties, which shows weakness of our configuration API, etc.
 
 My take:
 
 All of IGNITE_ properties which are useful (and will go to -X)
>> should
 instead be turned into configuration/metastore settings.
 All of IGNITE_ properties which are dangerous and/or useless should
 be
 removed.
 
 Regards,
 --
 Ilya Kasnacheev
 
 
 пт, 21 авг. 2020 г. в 16:50, Nikolay Izhikov :
 
> Hello, Igniters.
> 
> For now, we have dozens of the `IgniteSystemProperties` [1]  that
 can
> tweak Ignite behaviour in the very wide limits.
> But, the issue, for the administrator is the following
> 
> - Documentation about existing properties can be outdated.
> - The only place of the truth is the source 

Re: [DISCUSSION] Add autocompletion for commands in control.sh

[Ilya Kasnacheev]

Hello!

Not everyone is using bash, which leads me to question whether there's any
common approach where we can hint a shell what our executable can do so
that it can discover and auto-complete our control.sh

Regards,
-- 
Ilya Kasnacheev


пн, 7 сент. 2020 г. в 17:47, ткаленко кирилл :

> Hello, folks!
>
> I spent time to analyze the possibility of adding auto completion for the
> "control.sh" with the [1].
>
> To do this, at the beginning, we need to adapt the "control.sh" code to
> [1], then we can automatically create a "bash completion script" via [2],
> and then install it, for example, with the "source" command and the
> "control.sh" script itself via "install".
>
> This is only possible for nix systems.
>
> It is theoretically possible to add the "control.sh" extension via plugins
> and auto-generate "bash completion script".
>
> Thus, I propose a plan:
> 1)Adapt "control.sh" to [1];
> 2)Automatic creation of "bash completion script" for the release build;
> 3)Adding extensibility "control.sh" and automatic re-creation of "bash
> completion script". (optional)
>
> What do you think, comments?
>
> [1] - https://picocli.info/
> [2] -
> https://picocli.info/autocomplete.html#_completion_script_generation_details
>

Re: [DISCUSSION] Output IgniteSystemProperties via ignite.sh

[Ilya Kasnacheev]

Hello!

We do replace some flags with cfg properties, such as inline size, for
example.
A lot of other flags just duplicate cfg properties.

We can make a compromise here: we list flags explicitly *but then, we
decide that we don't keep backward compatibility anymore*, since the user
of a new version can check whether their flag is still supported by using
control.sh.

WDYT?

Regards,
-- 
Ilya Kasnacheev


пн, 7 сент. 2020 г. в 18:34, Nikolay Izhikov :

> Ilya.
>
> > to remove any expectation of forward compatibility.
>
> AFAIK we must keep these flags before Ignite3, due to the backward
> compatibility.
>
>
> > Flags should be a temporary measure
>
> This is not true, for now.
> I feel your pain :)
> Personally, I hate these flags, also.
>
> But they exist and the whole point of this change is to make the life of
> Ignite users a bit easier.
>
> > 7 сент. 2020 г., в 17:32, Philipp Masharov 
> написал(а):
> >
> > Let me notice that in Ignite 3.0 Wishlist [1] we have a bullet point
> > "Remove as many IGNITE_ parameters as possible from
> IgniteSystemProperties
> > <
> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteSystemProperties.html
> >"
> >
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0+Wishlist
> > "
> >
> > On Mon, Sep 7, 2020 at 5:24 PM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com>
> > wrote:
> >
> >> Hello!
> >>
> >> Okay, we can do a simple list of these flags, but I would argue that we
> >> should:
> >> - Avoid adding extra infrastructure such as flags' human readable names,
> >> embedded docs, etc. Flags should be a temporary measure. They are now.
> >> Let's not make it look like they're there to stay.
> >> - Explicitly mention in the -X output that these flags may be
> >> removed/converted to configuration properties in the future, to remove
> any
> >> expectation of forward compatibility.
> >>
> >> They bother me for quite some time.
> >>
> >> Regards,
> >> --
> >> Ilya Kasnacheev
> >>
> >>
> >> пн, 7 сент. 2020 г. в 17:17, Nikolay Izhikov :
> >>
>  what’s the logic?
> >>>
> >>> I assume that this is a question to the author of these flags.
> >>> If you have a specific flag you are interested in, please, write it.
> >>>
> >>> My point is simple - we already have these flags.
> >>>
> >>> We should explain to the user what we have and what can be configured
> >> with
> >>> these flags.
> >>>
> >>> There is no flag added or removed in this change.
> >>> Just makes it more clear to the users.
> >>>
> >>>
>  7 сент. 2020 г., в 17:12, Stephen Darlington <
> >>> stephen.darling...@gridgain.com> написал(а):
> 
>  But to Ilya’s point, what’s the logic? Why are some things set using
> >>> IGNITE_ properties, others on the command line and others in
> >>> IgniteConfiguration? It’s confusing for the user and makes maintenance
> >>> harder.
> 
>  I’m not necessarily arguing against this change, though. Perfect being
> >>> the enemy of good and all that.
> 
> > On 7 Sep 2020, at 13:02, Nikolay Izhikov 
> wrote:
> >
> > Hello, Ilya.
> >
> >> I think this is a bad idea since it legitimizes wide use of IGNITE_
> >>> properties, which shows weakness of our configuration API, etc.
> >
> > We already have IGNITE options in the product as a part of public
> API.
> >>> See `org.apache.ignite.IgniteSystemProperties`.
> >
> >> 7 сент. 2020 г., в 14:40, Ilya Kasnacheev <
> ilya.kasnach...@gmail.com
> >>>
> >>> написал(а):
> >>
> >> Hello!
> >>
> >> I think this is a bad idea since it legitimizes wide use of IGNITE_
> >> properties, which shows weakness of our configuration API, etc.
> >>
> >> My take:
> >>
> >> All of IGNITE_ properties which are useful (and will go to -X)
> should
> >> instead be turned into configuration/metastore settings.
> >> All of IGNITE_ properties which are dangerous and/or useless should
> >> be
> >> removed.
> >>
> >> Regards,
> >> --
> >> Ilya Kasnacheev
> >>
> >>
> >> пт, 21 авг. 2020 г. в 16:50, Nikolay Izhikov :
> >>
> >>> Hello, Igniters.
> >>>
> >>> For now, we have dozens of the `IgniteSystemProperties` [1]  that
> >> can
> >>> tweak Ignite behaviour in the very wide limits.
> >>> But, the issue, for the administrator is the following
> >>>
> >>> - Documentation about existing properties can be outdated.
> >>> - The only place of the truth is the source code.
> >>> - It’s hard to understand what flag is supported in what version.
> >>>
> >>> I propose to implement output of all available properties with it’s
> >>> descriptions in the `ignite.sh -X` command.
> >>>
> >>> Example of the JVM output:
> >>>
> >>> ```
> >>> [16:25:49]~/src/ignite:[master]$ java -X
> >>>
> >>> -Xbatch   disable background compilation
> >>> -Xbootclasspath/a:
> >>>   append to end of bootstrap 

Re: [DISCUSSION] Output IgniteSystemProperties via ignite.sh

[Nikolay Izhikov]

Ilya.

> to remove any expectation of forward compatibility.

AFAIK we must keep these flags before Ignite3, due to the backward 
compatibility.


> Flags should be a temporary measure

This is not true, for now.
I feel your pain :)
Personally, I hate these flags, also.

But they exist and the whole point of this change is to make the life of Ignite 
users a bit easier.

> 7 сент. 2020 г., в 17:32, Philipp Masharov  написал(а):
> 
> Let me notice that in Ignite 3.0 Wishlist [1] we have a bullet point
> "Remove as many IGNITE_ parameters as possible from IgniteSystemProperties
> "
> 
> 
> [1]
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0+Wishlist
> "
> 
> On Mon, Sep 7, 2020 at 5:24 PM Ilya Kasnacheev 
> wrote:
> 
>> Hello!
>> 
>> Okay, we can do a simple list of these flags, but I would argue that we
>> should:
>> - Avoid adding extra infrastructure such as flags' human readable names,
>> embedded docs, etc. Flags should be a temporary measure. They are now.
>> Let's not make it look like they're there to stay.
>> - Explicitly mention in the -X output that these flags may be
>> removed/converted to configuration properties in the future, to remove any
>> expectation of forward compatibility.
>> 
>> They bother me for quite some time.
>> 
>> Regards,
>> --
>> Ilya Kasnacheev
>> 
>> 
>> пн, 7 сент. 2020 г. в 17:17, Nikolay Izhikov :
>> 
 what’s the logic?
>>> 
>>> I assume that this is a question to the author of these flags.
>>> If you have a specific flag you are interested in, please, write it.
>>> 
>>> My point is simple - we already have these flags.
>>> 
>>> We should explain to the user what we have and what can be configured
>> with
>>> these flags.
>>> 
>>> There is no flag added or removed in this change.
>>> Just makes it more clear to the users.
>>> 
>>> 
 7 сент. 2020 г., в 17:12, Stephen Darlington <
>>> stephen.darling...@gridgain.com> написал(а):
 
 But to Ilya’s point, what’s the logic? Why are some things set using
>>> IGNITE_ properties, others on the command line and others in
>>> IgniteConfiguration? It’s confusing for the user and makes maintenance
>>> harder.
 
 I’m not necessarily arguing against this change, though. Perfect being
>>> the enemy of good and all that.
 
> On 7 Sep 2020, at 13:02, Nikolay Izhikov  wrote:
> 
> Hello, Ilya.
> 
>> I think this is a bad idea since it legitimizes wide use of IGNITE_
>>> properties, which shows weakness of our configuration API, etc.
> 
> We already have IGNITE options in the product as a part of public API.
>>> See `org.apache.ignite.IgniteSystemProperties`.
> 
>> 7 сент. 2020 г., в 14:40, Ilya Kasnacheev >> 
>>> написал(а):
>> 
>> Hello!
>> 
>> I think this is a bad idea since it legitimizes wide use of IGNITE_
>> properties, which shows weakness of our configuration API, etc.
>> 
>> My take:
>> 
>> All of IGNITE_ properties which are useful (and will go to -X) should
>> instead be turned into configuration/metastore settings.
>> All of IGNITE_ properties which are dangerous and/or useless should
>> be
>> removed.
>> 
>> Regards,
>> --
>> Ilya Kasnacheev
>> 
>> 
>> пт, 21 авг. 2020 г. в 16:50, Nikolay Izhikov :
>> 
>>> Hello, Igniters.
>>> 
>>> For now, we have dozens of the `IgniteSystemProperties` [1]  that
>> can
>>> tweak Ignite behaviour in the very wide limits.
>>> But, the issue, for the administrator is the following
>>> 
>>> - Documentation about existing properties can be outdated.
>>> - The only place of the truth is the source code.
>>> - It’s hard to understand what flag is supported in what version.
>>> 
>>> I propose to implement output of all available properties with it’s
>>> descriptions in the `ignite.sh -X` command.
>>> 
>>> Example of the JVM output:
>>> 
>>> ```
>>> [16:25:49]~/src/ignite:[master]$ java -X
>>> 
>>> -Xbatch   disable background compilation
>>> -Xbootclasspath/a:
>>>   append to end of bootstrap class path
>>> -Xcheck:jni   perform additional checks for JNI functions
>>> -Xcompforces compilation of methods on first invocation
>>> -Xdebug   provided for backward compatibility
>>> -Xdiagshow additional diagnostic messages
>>> -Xfuture  enable strictest checks, anticipating future
>>> default
>>> -Xint interpreted mode execution only
>>> -Xinternalversion
>>>   displays more detailed JVM version information
>> than
>>> the
>>> 
>>> [16:28:45]~/src/ignite:[master]$ java -XX:+UnlockDiagnosticVMOptions
>>> -XX:+PrintFlagsFinal -version
>>> 
>>> [Global flags]
>>> ccstrlist AOTLibrary   

[jira] [Created] (IGNITE-13411) Optimize tracing when NoopTracingSpi is used

[Aleksey Plekhanov (Jira)]

Aleksey Plekhanov created IGNITE-13411:
--

 Summary: Optimize tracing when NoopTracingSpi is used
 Key: IGNITE-13411
 URL: https://issues.apache.org/jira/browse/IGNITE-13411
 Project: Ignite
  Issue Type: Improvement
Affects Versions: 2.9
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov


Current tracing implementation has some redundant actions for no-op tracing SPI 
which have a negative impact on performance: 
# {{GridNioTracerFilter}} added to communication SPI filters chain even if 
tracing is disabled.
# {{MTC.support}}/{{MTC.supportContinual}} methods in case of no-op span or 
null span return {{TracingSurrounding}} which equivalently does nothing 
({{span.set(oldSpan)}} is redundant, since {{span.set(startSpan)}} was skipped 
for no-op or null span, and {{endRequired}} is always {{false}}. So, instead of 
creating new {{TracingSurrounding}} we can return {{null}} (correctly processed 
by try-with resource block) with the same effect and get rid of some code on 
the hot path.  
# Sometimes we already have {{Span}} on hands and call to {{MTC.span()}} is 
redundant.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[DISCUSSION] Add autocompletion for commands in control.sh

[ткаленко кирилл]

Hello, folks!

I spent time to analyze the possibility of adding auto completion for the 
"control.sh" with the [1].

To do this, at the beginning, we need to adapt the "control.sh" code to [1], 
then we can automatically create a "bash completion script" via [2], and then 
install it, for example, with the "source" command and the "control.sh" script 
itself via "install".

This is only possible for nix systems.

It is theoretically possible to add the "control.sh" extension via plugins and 
auto-generate "bash completion script".

Thus, I propose a plan:
1)Adapt "control.sh" to [1];
2)Automatic creation of "bash completion script" for the release build;
3)Adding extensibility "control.sh" and automatic re-creation of "bash 
completion script". (optional)

What do you think, comments?

[1] - https://picocli.info/
[2] - 
https://picocli.info/autocomplete.html#_completion_script_generation_details

Re: [DISCUSSION] Output IgniteSystemProperties via ignite.sh

[Philipp Masharov]

Let me notice that in Ignite 3.0 Wishlist [1] we have a bullet point
"Remove as many IGNITE_ parameters as possible from IgniteSystemProperties
"


[1]
https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0+Wishlist
"

On Mon, Sep 7, 2020 at 5:24 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> Okay, we can do a simple list of these flags, but I would argue that we
> should:
> - Avoid adding extra infrastructure such as flags' human readable names,
> embedded docs, etc. Flags should be a temporary measure. They are now.
> Let's not make it look like they're there to stay.
> - Explicitly mention in the -X output that these flags may be
> removed/converted to configuration properties in the future, to remove any
> expectation of forward compatibility.
>
> They bother me for quite some time.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пн, 7 сент. 2020 г. в 17:17, Nikolay Izhikov :
>
> > > what’s the logic?
> >
> > I assume that this is a question to the author of these flags.
> > If you have a specific flag you are interested in, please, write it.
> >
> > My point is simple - we already have these flags.
> >
> > We should explain to the user what we have and what can be configured
> with
> > these flags.
> >
> > There is no flag added or removed in this change.
> > Just makes it more clear to the users.
> >
> >
> > > 7 сент. 2020 г., в 17:12, Stephen Darlington <
> > stephen.darling...@gridgain.com> написал(а):
> > >
> > > But to Ilya’s point, what’s the logic? Why are some things set using
> > IGNITE_ properties, others on the command line and others in
> > IgniteConfiguration? It’s confusing for the user and makes maintenance
> > harder.
> > >
> > > I’m not necessarily arguing against this change, though. Perfect being
> > the enemy of good and all that.
> > >
> > >> On 7 Sep 2020, at 13:02, Nikolay Izhikov  wrote:
> > >>
> > >> Hello, Ilya.
> > >>
> > >>> I think this is a bad idea since it legitimizes wide use of IGNITE_
> > properties, which shows weakness of our configuration API, etc.
> > >>
> > >> We already have IGNITE options in the product as a part of public API.
> > See `org.apache.ignite.IgniteSystemProperties`.
> > >>
> > >>> 7 сент. 2020 г., в 14:40, Ilya Kasnacheev  >
> > написал(а):
> > >>>
> > >>> Hello!
> > >>>
> > >>> I think this is a bad idea since it legitimizes wide use of IGNITE_
> > >>> properties, which shows weakness of our configuration API, etc.
> > >>>
> > >>> My take:
> > >>>
> > >>> All of IGNITE_ properties which are useful (and will go to -X) should
> > >>> instead be turned into configuration/metastore settings.
> > >>> All of IGNITE_ properties which are dangerous and/or useless should
> be
> > >>> removed.
> > >>>
> > >>> Regards,
> > >>> --
> > >>> Ilya Kasnacheev
> > >>>
> > >>>
> > >>> пт, 21 авг. 2020 г. в 16:50, Nikolay Izhikov :
> > >>>
> >  Hello, Igniters.
> > 
> >  For now, we have dozens of the `IgniteSystemProperties` [1]  that
> can
> >  tweak Ignite behaviour in the very wide limits.
> >  But, the issue, for the administrator is the following
> > 
> >  - Documentation about existing properties can be outdated.
> >  - The only place of the truth is the source code.
> >  - It’s hard to understand what flag is supported in what version.
> > 
> >  I propose to implement output of all available properties with it’s
> >  descriptions in the `ignite.sh -X` command.
> > 
> >  Example of the JVM output:
> > 
> >  ```
> >  [16:25:49]~/src/ignite:[master]$ java -X
> > 
> >   -Xbatch   disable background compilation
> >   -Xbootclasspath/a:
> > append to end of bootstrap class path
> >   -Xcheck:jni   perform additional checks for JNI functions
> >   -Xcompforces compilation of methods on first invocation
> >   -Xdebug   provided for backward compatibility
> >   -Xdiagshow additional diagnostic messages
> >   -Xfuture  enable strictest checks, anticipating future
> > default
> >   -Xint interpreted mode execution only
> >   -Xinternalversion
> > displays more detailed JVM version information
> than
> >  the
> > 
> >  [16:28:45]~/src/ignite:[master]$ java -XX:+UnlockDiagnosticVMOptions
> >  -XX:+PrintFlagsFinal -version
> > 
> >  [Global flags]
> >  ccstrlist AOTLibrary   =
> >    {product} {default}
> >    bool AbortVMOnCompilationFailure  = false
> > {diagnostic} {default}
> >   ccstr AbortVMOnException   =
> > {diagnostic} {default}
> >   ccstr AbortVMOnExceptionMessage=
> > {diagnostic} {default}
> >    bool AbortVMOnSafepointTimeout= 

Re: [DISCUSSION] Output IgniteSystemProperties via ignite.sh

[Ilya Kasnacheev]

Hello!

Okay, we can do a simple list of these flags, but I would argue that we
should:
- Avoid adding extra infrastructure such as flags' human readable names,
embedded docs, etc. Flags should be a temporary measure. They are now.
Let's not make it look like they're there to stay.
- Explicitly mention in the -X output that these flags may be
removed/converted to configuration properties in the future, to remove any
expectation of forward compatibility.

They bother me for quite some time.

Regards,
-- 
Ilya Kasnacheev


пн, 7 сент. 2020 г. в 17:17, Nikolay Izhikov :

> > what’s the logic?
>
> I assume that this is a question to the author of these flags.
> If you have a specific flag you are interested in, please, write it.
>
> My point is simple - we already have these flags.
>
> We should explain to the user what we have and what can be configured with
> these flags.
>
> There is no flag added or removed in this change.
> Just makes it more clear to the users.
>
>
> > 7 сент. 2020 г., в 17:12, Stephen Darlington <
> stephen.darling...@gridgain.com> написал(а):
> >
> > But to Ilya’s point, what’s the logic? Why are some things set using
> IGNITE_ properties, others on the command line and others in
> IgniteConfiguration? It’s confusing for the user and makes maintenance
> harder.
> >
> > I’m not necessarily arguing against this change, though. Perfect being
> the enemy of good and all that.
> >
> >> On 7 Sep 2020, at 13:02, Nikolay Izhikov  wrote:
> >>
> >> Hello, Ilya.
> >>
> >>> I think this is a bad idea since it legitimizes wide use of IGNITE_
> properties, which shows weakness of our configuration API, etc.
> >>
> >> We already have IGNITE options in the product as a part of public API.
> See `org.apache.ignite.IgniteSystemProperties`.
> >>
> >>> 7 сент. 2020 г., в 14:40, Ilya Kasnacheev 
> написал(а):
> >>>
> >>> Hello!
> >>>
> >>> I think this is a bad idea since it legitimizes wide use of IGNITE_
> >>> properties, which shows weakness of our configuration API, etc.
> >>>
> >>> My take:
> >>>
> >>> All of IGNITE_ properties which are useful (and will go to -X) should
> >>> instead be turned into configuration/metastore settings.
> >>> All of IGNITE_ properties which are dangerous and/or useless should be
> >>> removed.
> >>>
> >>> Regards,
> >>> --
> >>> Ilya Kasnacheev
> >>>
> >>>
> >>> пт, 21 авг. 2020 г. в 16:50, Nikolay Izhikov :
> >>>
>  Hello, Igniters.
> 
>  For now, we have dozens of the `IgniteSystemProperties` [1]  that can
>  tweak Ignite behaviour in the very wide limits.
>  But, the issue, for the administrator is the following
> 
>  - Documentation about existing properties can be outdated.
>  - The only place of the truth is the source code.
>  - It’s hard to understand what flag is supported in what version.
> 
>  I propose to implement output of all available properties with it’s
>  descriptions in the `ignite.sh -X` command.
> 
>  Example of the JVM output:
> 
>  ```
>  [16:25:49]~/src/ignite:[master]$ java -X
> 
>   -Xbatch   disable background compilation
>   -Xbootclasspath/a:
> append to end of bootstrap class path
>   -Xcheck:jni   perform additional checks for JNI functions
>   -Xcompforces compilation of methods on first invocation
>   -Xdebug   provided for backward compatibility
>   -Xdiagshow additional diagnostic messages
>   -Xfuture  enable strictest checks, anticipating future
> default
>   -Xint interpreted mode execution only
>   -Xinternalversion
> displays more detailed JVM version information than
>  the
> 
>  [16:28:45]~/src/ignite:[master]$ java -XX:+UnlockDiagnosticVMOptions
>  -XX:+PrintFlagsFinal -version
> 
>  [Global flags]
>  ccstrlist AOTLibrary   =
>    {product} {default}
>    bool AbortVMOnCompilationFailure  = false
> {diagnostic} {default}
>   ccstr AbortVMOnException   =
> {diagnostic} {default}
>   ccstr AbortVMOnExceptionMessage=
> {diagnostic} {default}
>    bool AbortVMOnSafepointTimeout= false
> {diagnostic} {default}
>    bool AbortVMOnVMOperationTimeout  = false
> {diagnostic} {default}
>    intx AbortVMOnVMOperationTimeoutDelay = 1000
>    {diagnostic} {default}
> int ActiveProcessorCount = -1
>   {product} {default}
> 
>  ```
> 
>  Example of the Ignite output:
> 
>  
> > ignite.sh -X
>  IGNITE_CONFIG_URL
>    -   System property to hold optional configuration URL.
>  IGNITE_SSH_HOST
>-
>    System property to hold SSH host 

Re: [DISCUSSION] Output IgniteSystemProperties via ignite.sh

[Nikolay Izhikov]

> what’s the logic?

I assume that this is a question to the author of these flags.
If you have a specific flag you are interested in, please, write it.

My point is simple - we already have these flags.

We should explain to the user what we have and what can be configured with 
these flags.

There is no flag added or removed in this change.
Just makes it more clear to the users.


> 7 сент. 2020 г., в 17:12, Stephen Darlington 
>  написал(а):
> 
> But to Ilya’s point, what’s the logic? Why are some things set using IGNITE_ 
> properties, others on the command line and others in IgniteConfiguration? 
> It’s confusing for the user and makes maintenance harder.
> 
> I’m not necessarily arguing against this change, though. Perfect being the 
> enemy of good and all that.
> 
>> On 7 Sep 2020, at 13:02, Nikolay Izhikov  wrote:
>> 
>> Hello, Ilya.
>> 
>>> I think this is a bad idea since it legitimizes wide use of IGNITE_ 
>>> properties, which shows weakness of our configuration API, etc.
>> 
>> We already have IGNITE options in the product as a part of public API. See 
>> `org.apache.ignite.IgniteSystemProperties`.
>> 
>>> 7 сент. 2020 г., в 14:40, Ilya Kasnacheev  
>>> написал(а):
>>> 
>>> Hello!
>>> 
>>> I think this is a bad idea since it legitimizes wide use of IGNITE_
>>> properties, which shows weakness of our configuration API, etc.
>>> 
>>> My take:
>>> 
>>> All of IGNITE_ properties which are useful (and will go to -X) should
>>> instead be turned into configuration/metastore settings.
>>> All of IGNITE_ properties which are dangerous and/or useless should be
>>> removed.
>>> 
>>> Regards,
>>> -- 
>>> Ilya Kasnacheev
>>> 
>>> 
>>> пт, 21 авг. 2020 г. в 16:50, Nikolay Izhikov :
>>> 
 Hello, Igniters.
 
 For now, we have dozens of the `IgniteSystemProperties` [1]  that can
 tweak Ignite behaviour in the very wide limits.
 But, the issue, for the administrator is the following
 
 - Documentation about existing properties can be outdated.
 - The only place of the truth is the source code.
 - It’s hard to understand what flag is supported in what version.
 
 I propose to implement output of all available properties with it’s
 descriptions in the `ignite.sh -X` command.
 
 Example of the JVM output:
 
 ```
 [16:25:49]~/src/ignite:[master]$ java -X
 
  -Xbatch   disable background compilation
  -Xbootclasspath/a:
append to end of bootstrap class path
  -Xcheck:jni   perform additional checks for JNI functions
  -Xcompforces compilation of methods on first invocation
  -Xdebug   provided for backward compatibility
  -Xdiagshow additional diagnostic messages
  -Xfuture  enable strictest checks, anticipating future default
  -Xint interpreted mode execution only
  -Xinternalversion
displays more detailed JVM version information than
 the
 
 [16:28:45]~/src/ignite:[master]$ java -XX:+UnlockDiagnosticVMOptions
 -XX:+PrintFlagsFinal -version
 
 [Global flags]
 ccstrlist AOTLibrary   =
   {product} {default}
   bool AbortVMOnCompilationFailure  = false
{diagnostic} {default}
  ccstr AbortVMOnException   =
{diagnostic} {default}
  ccstr AbortVMOnExceptionMessage=
{diagnostic} {default}
   bool AbortVMOnSafepointTimeout= false
{diagnostic} {default}
   bool AbortVMOnVMOperationTimeout  = false
{diagnostic} {default}
   intx AbortVMOnVMOperationTimeoutDelay = 1000
   {diagnostic} {default}
int ActiveProcessorCount = -1
  {product} {default}
 
 ```
 
 Example of the Ignite output:
 
 
> ignite.sh -X
 IGNITE_CONFIG_URL
   -   System property to hold optional configuration URL.
 IGNITE_SSH_HOST -
   System property to hold SSH host for visor-started nodes.
 IGNITE_MIN_BUFFERED_COMMUNICATION_MSG_CNT   -   [DEPRECATED]
 System property to disable buffered communication if node sends less
 messages count than specified by this property. Default value is {@code
 512}.
 
 …
 
 ```
 
 WDYT?
 
 [1]
 https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteSystemProperties.java#L56
>> 
> 
> 

Re: [DISCUSSION] Output IgniteSystemProperties via ignite.sh

[Stephen Darlington]

But to Ilya’s point, what’s the logic? Why are some things set using IGNITE_ 
properties, others on the command line and others in IgniteConfiguration? It’s 
confusing for the user and makes maintenance harder.

I’m not necessarily arguing against this change, though. Perfect being the 
enemy of good and all that.

> On 7 Sep 2020, at 13:02, Nikolay Izhikov  wrote:
> 
> Hello, Ilya.
> 
>> I think this is a bad idea since it legitimizes wide use of IGNITE_ 
>> properties, which shows weakness of our configuration API, etc.
> 
> We already have IGNITE options in the product as a part of public API. See 
> `org.apache.ignite.IgniteSystemProperties`.
> 
>> 7 сент. 2020 г., в 14:40, Ilya Kasnacheev  
>> написал(а):
>> 
>> Hello!
>> 
>> I think this is a bad idea since it legitimizes wide use of IGNITE_
>> properties, which shows weakness of our configuration API, etc.
>> 
>> My take:
>> 
>> All of IGNITE_ properties which are useful (and will go to -X) should
>> instead be turned into configuration/metastore settings.
>> All of IGNITE_ properties which are dangerous and/or useless should be
>> removed.
>> 
>> Regards,
>> -- 
>> Ilya Kasnacheev
>> 
>> 
>> пт, 21 авг. 2020 г. в 16:50, Nikolay Izhikov :
>> 
>>> Hello, Igniters.
>>> 
>>> For now, we have dozens of the `IgniteSystemProperties` [1]  that can
>>> tweak Ignite behaviour in the very wide limits.
>>> But, the issue, for the administrator is the following
>>> 
>>> - Documentation about existing properties can be outdated.
>>> - The only place of the truth is the source code.
>>> - It’s hard to understand what flag is supported in what version.
>>> 
>>> I propose to implement output of all available properties with it’s
>>> descriptions in the `ignite.sh -X` command.
>>> 
>>> Example of the JVM output:
>>> 
>>> ```
>>> [16:25:49]~/src/ignite:[master]$ java -X
>>> 
>>>   -Xbatch   disable background compilation
>>>   -Xbootclasspath/a:
>>> append to end of bootstrap class path
>>>   -Xcheck:jni   perform additional checks for JNI functions
>>>   -Xcompforces compilation of methods on first invocation
>>>   -Xdebug   provided for backward compatibility
>>>   -Xdiagshow additional diagnostic messages
>>>   -Xfuture  enable strictest checks, anticipating future default
>>>   -Xint interpreted mode execution only
>>>   -Xinternalversion
>>> displays more detailed JVM version information than
>>> the
>>> 
>>> [16:28:45]~/src/ignite:[master]$ java -XX:+UnlockDiagnosticVMOptions
>>> -XX:+PrintFlagsFinal -version
>>> 
>>> [Global flags]
>>> ccstrlist AOTLibrary   =
>>>{product} {default}
>>>bool AbortVMOnCompilationFailure  = false
>>> {diagnostic} {default}
>>>   ccstr AbortVMOnException   =
>>> {diagnostic} {default}
>>>   ccstr AbortVMOnExceptionMessage=
>>> {diagnostic} {default}
>>>bool AbortVMOnSafepointTimeout= false
>>> {diagnostic} {default}
>>>bool AbortVMOnVMOperationTimeout  = false
>>> {diagnostic} {default}
>>>intx AbortVMOnVMOperationTimeoutDelay = 1000
>>>{diagnostic} {default}
>>> int ActiveProcessorCount = -1
>>>   {product} {default}
>>> 
>>> ```
>>> 
>>> Example of the Ignite output:
>>> 
>>> 
 ignite.sh -X
>>> IGNITE_CONFIG_URL
>>>-   System property to hold optional configuration URL.
>>> IGNITE_SSH_HOST -
>>>System property to hold SSH host for visor-started nodes.
>>> IGNITE_MIN_BUFFERED_COMMUNICATION_MSG_CNT   -   [DEPRECATED]
>>> System property to disable buffered communication if node sends less
>>> messages count than specified by this property. Default value is {@code
>>> 512}.
>>> 
>>> …
>>> 
>>> ```
>>> 
>>> WDYT?
>>> 
>>> [1]
>>> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteSystemProperties.java#L56
> 

Re: IEP-54: Schema-first approach for 3.0

[Alexey Goncharuk]

Ivan,

Thank you, I got your concern now. As it is mostly regarding the
terminology, I am absolutely fine with changing the name to whatever fits
the approach best. Dynamic or evolving schema sounds great. I will make
corresponding changes to the IEP once we settle on the name.

пн, 7 сент. 2020 г. в 11:33, Ivan Pavlukhin :

> Hi Val,
>
> Thank you for your answer!
>
> My understanding is a little bit different. Yes, schema evolution
> definitely should be possible. But I see a main difference in "how
> schema is updated". I treat a common SQL approach schema-first. Schema
> and data manipulation operations are clearly separated and it enables
> interesting capabilities, e.g. preventing untended schema changes by
> mistaken data operations, restricting user permissions to change
> schema.
>
> > Schema-first means that schema exists in advance and all the stored data
> is compliant with it - that's exactly what is proposed.
>
> A schema-last approach mentioned in [1] also assumes that schema
> exists, but it is inferred from data. Is not it more similar to the
> proposing approach?
>
> And I would like to say, that my main concern so far is mostly about
> terminology. And I suppose if it confuses me then others might be
> confused as well. My feeling is closer to "dynamic or liquid or may be
> evolving schema".
>
> [1]
> https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf
>
> 2020-09-07 0:47 GMT+03:00, Valentin Kulichenko <
> valentin.kuliche...@gmail.com>:
> > Hi Ivan,
> >
> > I don't see an issue with that. Schema-first means that schema exists in
> > advance and all the stored data is compliant with it - that's exactly
> what
> > is proposed. There are no restrictions prohibiting changes to the schema.
> >
> > -Val
> >
> > On Sat, Sep 5, 2020 at 9:52 PM Ivan Pavlukhin 
> wrote:
> >
> >> Alexey,
> >>
> >> I am a little bit confused with terminology. My understanding conforms
> >> to a survey [1] (see part X Semi Structured Data). Can we really treat
> >> a "dynamic schema" approach as a kind of "schema-first"?
> >>
> >> [1]
> >> https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf
> >>
> >> 2020-09-02 1:53 GMT+03:00, Denis Magda :
> >> >>
> >> >> However, could you please elaborate on the relation between Ignite
> and
> >> >> ORM?
> >> >> Is there a use case for Hibernate running on top of Ignite (I haven't
> >> >> seen
> >> >> one so far)? If so, what is missing exactly on the Ignite side to
> >> support
> >> >> this? In my understanding, all you need is SQL API which we already
> >> have.
> >> >> Am I missing something?
> >> >
> >> >
> >> > Good point, yes, if all the ORM integrations use Ignite SQL APIs
> >> > internally, then they can easily translate an Entity object into an
> >> > INSERT/UPDATE statement that lists all the object's fields. Luckily,
> >> > our
> >> > Spring Data integration is already based on the Ignite SQL APIs and
> >> > needs
> >> > to be improved once the schema-first approach is supported. That would
> >> > solve a ton of usability issues.
> >> >
> >> > I would revise the Hibernate integration as well during the Ignite 3.0
> >> dev
> >> > phase. Can't say if it's used a lot but Spring Data is getting
> traction
> >> for
> >> > sure.
> >> >
> >> > @Michael Pollind, I'll loop you in as long as you've started working
> on
> >> the
> >> > Ignite support for Micornaut Data
> >> > 
> and
> >> > came across some challenges. Just watch this discussion. That's what
> is
> >> > coming in Ignite 3.0.
> >> >
> >> >
> >> > -
> >> > Denis
> >> >
> >> >
> >> > On Mon, Aug 31, 2020 at 5:11 PM Valentin Kulichenko <
> >> > valentin.kuliche...@gmail.com> wrote:
> >> >
> >> >> Hi Denis,
> >> >>
> >> >> Generally speaking, I believe that the schema-first approach natively
> >> >> addresses the issue if duplicate fields in key and value objects,
> >> because
> >> >> schema will be created for a cache, not for an object, as it happens
> >> now.
> >> >> Basically, the schema will define whether there is a primary key or
> >> >> not,
> >> >> and which fields are included in case there is one. Any API that we
> >> would
> >> >> have must be compliant with this, so it becomes fairly easy to work
> >> >> with
> >> >> data as with a set of records, rather than key-value pairs.
> >> >>
> >> >> However, could you please elaborate on the relation between Ignite
> and
> >> >> ORM?
> >> >> Is there a use case for Hibernate running on top of Ignite (I haven't
> >> >> seen
> >> >> one so far)? If so, what is missing exactly on the Ignite side to
> >> support
> >> >> this? In my understanding, all you need is SQL API which we already
> >> have.
> >> >> Am I missing something?
> >> >>
> >> >> -Val
> >> >>
> >> >> On Mon, Aug 31, 2020 at 2:08 PM Denis Magda 
> wrote:
> >> >>
> >> >> > Val,
> >> >> >
> >> >> > I would propose adding another point to the motivations list which
> >> >> > is
> >> >> > related to 

[jira] [Created] (IGNITE-13410) .NET: Run Services tests with different service processors

[Pavel Tupitsyn (Jira)]

Pavel Tupitsyn created IGNITE-13410:
---

 Summary: .NET: Run Services tests with different service processors
 Key: IGNITE-13410
 URL: https://issues.apache.org/jira/browse/IGNITE-13410
 Project: Ignite
  Issue Type: Improvement
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 2.10


We have two service processors - legacy and a new event-driven one. Ignite 
system property IGNITE_EVENT_DRIVEN_SERVICE_PROCESSOR_ENABLED specifies the one 
to use.

Make sure thick and thin service tests execute in both modes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSSION] Output IgniteSystemProperties via ignite.sh

[Nikolay Izhikov]

Hello, Ilya.

> I think this is a bad idea since it legitimizes wide use of IGNITE_ 
> properties, which shows weakness of our configuration API, etc.

We already have IGNITE options in the product as a part of public API. See 
`org.apache.ignite.IgniteSystemProperties`.

> 7 сент. 2020 г., в 14:40, Ilya Kasnacheev  
> написал(а):
> 
> Hello!
> 
> I think this is a bad idea since it legitimizes wide use of IGNITE_
> properties, which shows weakness of our configuration API, etc.
> 
> My take:
> 
> All of IGNITE_ properties which are useful (and will go to -X) should
> instead be turned into configuration/metastore settings.
> All of IGNITE_ properties which are dangerous and/or useless should be
> removed.
> 
> Regards,
> -- 
> Ilya Kasnacheev
> 
> 
> пт, 21 авг. 2020 г. в 16:50, Nikolay Izhikov :
> 
>> Hello, Igniters.
>> 
>> For now, we have dozens of the `IgniteSystemProperties` [1]  that can
>> tweak Ignite behaviour in the very wide limits.
>> But, the issue, for the administrator is the following
>> 
>> - Documentation about existing properties can be outdated.
>> - The only place of the truth is the source code.
>> - It’s hard to understand what flag is supported in what version.
>> 
>> I propose to implement output of all available properties with it’s
>> descriptions in the `ignite.sh -X` command.
>> 
>> Example of the JVM output:
>> 
>> ```
>> [16:25:49]~/src/ignite:[master]$ java -X
>> 
>>-Xbatch   disable background compilation
>>-Xbootclasspath/a:
>>  append to end of bootstrap class path
>>-Xcheck:jni   perform additional checks for JNI functions
>>-Xcompforces compilation of methods on first invocation
>>-Xdebug   provided for backward compatibility
>>-Xdiagshow additional diagnostic messages
>>-Xfuture  enable strictest checks, anticipating future default
>>-Xint interpreted mode execution only
>>-Xinternalversion
>>  displays more detailed JVM version information than
>> the
>> 
>> [16:28:45]~/src/ignite:[master]$ java -XX:+UnlockDiagnosticVMOptions
>> -XX:+PrintFlagsFinal -version
>> 
>> [Global flags]
>> ccstrlist AOTLibrary   =
>> {product} {default}
>> bool AbortVMOnCompilationFailure  = false
>>  {diagnostic} {default}
>>ccstr AbortVMOnException   =
>>  {diagnostic} {default}
>>ccstr AbortVMOnExceptionMessage=
>>  {diagnostic} {default}
>> bool AbortVMOnSafepointTimeout= false
>>  {diagnostic} {default}
>> bool AbortVMOnVMOperationTimeout  = false
>>  {diagnostic} {default}
>> intx AbortVMOnVMOperationTimeoutDelay = 1000
>> {diagnostic} {default}
>>  int ActiveProcessorCount = -1
>>{product} {default}
>> 
>> ```
>> 
>> Example of the Ignite output:
>> 
>> 
>>> ignite.sh -X
>> IGNITE_CONFIG_URL
>> -   System property to hold optional configuration URL.
>> IGNITE_SSH_HOST -
>> System property to hold SSH host for visor-started nodes.
>> IGNITE_MIN_BUFFERED_COMMUNICATION_MSG_CNT   -   [DEPRECATED]
>> System property to disable buffered communication if node sends less
>> messages count than specified by this property. Default value is {@code
>> 512}.
>> 
>> …
>> 
>> ```
>> 
>> WDYT?
>> 
>> [1]
>> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteSystemProperties.java#L56

Re: [DISCUSSION] Output IgniteSystemProperties via ignite.sh

[Ilya Kasnacheev]

Hello!

I think this is a bad idea since it legitimizes wide use of IGNITE_
properties, which shows weakness of our configuration API, etc.

My take:

All of IGNITE_ properties which are useful (and will go to -X) should
instead be turned into configuration/metastore settings.
All of IGNITE_ properties which are dangerous and/or useless should be
removed.

Regards,
-- 
Ilya Kasnacheev


пт, 21 авг. 2020 г. в 16:50, Nikolay Izhikov :

> Hello, Igniters.
>
> For now, we have dozens of the `IgniteSystemProperties` [1]  that can
> tweak Ignite behaviour in the very wide limits.
> But, the issue, for the administrator is the following
>
> - Documentation about existing properties can be outdated.
> - The only place of the truth is the source code.
> - It’s hard to understand what flag is supported in what version.
>
> I propose to implement output of all available properties with it’s
> descriptions in the `ignite.sh -X` command.
>
> Example of the JVM output:
>
> ```
> [16:25:49]~/src/ignite:[master]$ java -X
>
> -Xbatch   disable background compilation
> -Xbootclasspath/a:
>   append to end of bootstrap class path
> -Xcheck:jni   perform additional checks for JNI functions
> -Xcompforces compilation of methods on first invocation
> -Xdebug   provided for backward compatibility
> -Xdiagshow additional diagnostic messages
> -Xfuture  enable strictest checks, anticipating future default
> -Xint interpreted mode execution only
> -Xinternalversion
>   displays more detailed JVM version information than
> the
>
> [16:28:45]~/src/ignite:[master]$ java -XX:+UnlockDiagnosticVMOptions
> -XX:+PrintFlagsFinal -version
>
> [Global flags]
> ccstrlist AOTLibrary   =
>  {product} {default}
>  bool AbortVMOnCompilationFailure  = false
>   {diagnostic} {default}
> ccstr AbortVMOnException   =
>   {diagnostic} {default}
> ccstr AbortVMOnExceptionMessage=
>   {diagnostic} {default}
>  bool AbortVMOnSafepointTimeout= false
>   {diagnostic} {default}
>  bool AbortVMOnVMOperationTimeout  = false
>   {diagnostic} {default}
>  intx AbortVMOnVMOperationTimeoutDelay = 1000
>  {diagnostic} {default}
>   int ActiveProcessorCount = -1
> {product} {default}
>
> ```
>
> Example of the Ignite output:
>
> 
> > ignite.sh -X
> IGNITE_CONFIG_URL
>  -   System property to hold optional configuration URL.
> IGNITE_SSH_HOST -
>  System property to hold SSH host for visor-started nodes.
> IGNITE_MIN_BUFFERED_COMMUNICATION_MSG_CNT   -   [DEPRECATED]
> System property to disable buffered communication if node sends less
> messages count than specified by this property. Default value is {@code
> 512}.
>
> …
>
> ```
>
> WDYT?
>
> [1]
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteSystemProperties.java#L56

Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]

[Maxim Muzafarov]

Alexey,

I propose to include [1] issue to the 2.9 release. Since this issue is
related to the new master key change functionality which haven't been
released yet I think it will be safe to cherry-pick commit to the
release branch.

[1] https://issues.apache.org/jira/browse/IGNITE-13390

On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov  wrote:
>
> Hello, Igniters.
>
> Alexey, please, include one more Python thin client fix [1] into the 2.9 
> release
> It fixes kinda major issue - "Python client returns fields in wrong order 
> since the 2 row when fields_count>10"
>
> [1] https://issues.apache.org/jira/browse/IGNITE-12809
> [2] 
> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>
> > 31 авг. 2020 г., в 19:23, Alexey Goncharuk  
> > написал(а):
> >
> > Alexey, thanks, got it. I am not sure we can optimize anything out of the
> > message factory with suppliers (at least I have no ideas right now), so
> > most likely the only move here is to switch back to the switch approach
> > somehow preserving the metrics part. Probably, inlining the Ignite messages
> > to the IgniteMessageFactoryImpl should do the trick. Let me explore the
> > code a bit.
> >
> > P.S. I am surprised by the impact this part makes for the performance.
> > Message creation is indeed on the hot path, but a single virtual call
> > should not make that much of a difference given the amount of other work
> > happening during the message processing.
> >
> > пн, 31 авг. 2020 г. в 18:33, Alex Plehanov :
> >
> >> Alexey, sorry, I wrongly interpreted our benchmark results. Actually, we
> >> were looking for a drop using bi-sect in the range between e6a7f93 (first
> >> commit in the 2.9 branch after 2.8 branch cut) and 6592dfa5 (last commit in
> >> the 2.9 branch). And we found these two problematic commits.
> >>
> >> Perhaps only IGNITE-13060 (Tracing) is responsible for a drop between
> >> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with reverted IGNITE-13060
> >> now and performance looks the same)
> >>
> >> Ticket IGNITE-12568 (MessageFactory refactoring) is not related to drop
> >> between 2.8.1 and 2.9, but still has some performance problem, and we can
> >> win back IGNITE-13060 drop by this ticket.
> >>
> >> Do we need more investigation on IGNITE-13060 or we leave it as is?
> >>
> >> What should we do with IGNITE-12568 (MessageFactory refactoring)?
> >>
> >> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk  >>> :
> >>
> >>> Alexey,
> >>>
> >>> While investigating, I found that IGNITE-12568 has an incorrect fix
> >>> version and is actually present in ignite-2.8.1 branch [1], so it cannot 
> >>> be
> >>> the source of the drop against 2.8.1.
> >>>
> >>> P.S. Looks like we need to enforce a more accurate work with fix versions
> >>> or develop some sort of tooling to verify the fix versions.
> >>>
> >>> --AG
> >>>
> >>> [1]
> >>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
> >>>
> >>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk   :
> >>>
>  пт, 28 авг. 2020 г. в 11:16, Alex Plehanov :
> 
> > Guys,
> >
> > We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568 (reverted
> > it
> > locally) and got the same performance as on 2.8.1
> >
> > IGNITE-13060 (Tracing) - some code was added to hot paths, to trace
> > these
> > hot paths, it's clear why we have performance drop here.
> >
> > IGNITE-12568 (MessageFactory refactoring) - switch/case block was
> > refactored to an array of message suppliers. The message factory is on
> > the
> > hot path, which explains why this commit has an impact on total
> > performance.
> > I've checked JIT assembly output, done some JMH microbenchmarks, and
> > found
> > that old implementation of MessageFactory.create() about 30-35% faster
> > than
> > the new one. The reason - approach with switch/case can effectively
> > inline
> > message creation code, but with an array of suppliers relatively heavy
> > "invokeinterface" cannot be skipped. I've tried to rewrite the code
> > using
> > an abstract class for suppliers instead of an interface (to
> > replace "invokeinterface" with the "invokevirtual"), but it gives back
> > only
> > 10% of method performance and in this case, code looks ugly (lambdas
> > can't
> > be used). Currently, I can't find any more ways to optimize the current
> > approach (except return to the switch/case block). Andrey Gura, as the
> > author of IGNITE-12568, maybe you have some ideas about optimization?
> >
> > Perhaps we should revert IGNITE-12568, but there are some metrics
> > already
> > created, which can't be rewritten using old message factory
> > implementation
> > (IGNITE-12756). Guys, WDYT?
> >
> 
>  Alexey,
> 
>  I see that IGNITE-12756 (metrics improvements) is already released in
>  Ignite 2.8.1 while IGNITE-12568 

Re: [DISCUSSION] control.sh argument to enable experimental commands

[Philipp Masharov]

Nikolay, thanks for mentioning it. I fetched the last changes and got this:

Print metadata command help:
control.(sh|bat) --meta help

  Print list of binary metadata types:
control.(sh|bat) --meta list

  Print detailed info about specified binary type (the type must be
specified by type name or by type identifier):
control.(sh|bat) --meta details [--typeId ] [--typeName
]

  Remove the metadata of the specified type (the type must be specified by
type name or by type identifier) from cluster and saves the removed
metadata to the specified file.
If the file name isn't specified the output file name is: '.bin'
control.(sh|bat) --meta remove [--typeId ] [--typeName
] [--out ]

  Update cluster metadata from specified file (file name is required)
control.(sh|bat) --meta update --in 

Print tracing configuration:
control.(sh|bat) --tracing-configuration

  Print tracing configuration:
control.(sh|bat) --tracing-configuration get_all [--scope
DISCOVERY|EXCHANGE|COMMUNICATION|TX]

  Print specific tracing configuration based on specified --scope and
--label:
control.(sh|bat) --tracing-configuration get (--scope
DISCOVERY|EXCHANGE|COMMUNICATION|TX) [--label]

  Reset all specific tracing configuration the to default. If --scope is
specified, then remove all label specific configuration for the given scope
and reset given scope specific configuration to the default, if --scope is
skipped then reset all tracing configurations to the default. Print tracing
configuration.
control.(sh|bat) --tracing-configuration reset_all [--scope
DISCOVERY|EXCHANGE|COMMUNICATION|TX]

  Reset specific tracing configuration to the default. If both --scope and
--label are specified then remove given configuration, if only --scope is
specified then reset given configuration to the default. Print reseted
configuration.
control.(sh|bat) --tracing-configuration reset (--scope
DISCOVERY|EXCHANGE|COMMUNICATION|TX) [--label]

  Set new tracing configuration. If both --scope and --label are specified
then add or override label specific configuration, if only --scope is
specified, then override scope specific configuration. Print applied
configuration.
control.(sh|bat) --tracing-configuration set (--scope
DISCOVERY|EXCHANGE|COMMUNICATION|TX [--label] [--sampling-rate Decimal
value between 0 and 1, where 0 means never and 1 means always. More or less
reflects the probability of sampling specific trace.] [--included-scopes
Set of scopes with comma as separator  DISCOVERY|EXCHANGE|COMMUNICATION|TX])


On Mon, Sep 7, 2020 at 1:44 PM Nikolay Izhikov  wrote:

> Philipp, at least, we have tracing and binary meta commands in the master.
>
> Please, take a look at current Ignite sources.
>
> > 7 сент. 2020 г., в 13:34, Nikolay Izhikov 
> написал(а):
> >
> > What is wrong with that?
> >
> > We just improve the way user enables experimental commands.
> > JVM uses similar approach, please, take a look at
> '-XX:+UnlockExperimentalVMOptions’.
> >
> >> 7 сент. 2020 г., в 13:22, ткаленко кирилл 
> написал(а):
> >>
> >> Hi, Nikolay!
> >>
> >> It seems that you shouldn't just open experimental commands, that's why
> they are experimental.
> >>
> >>
> >> 07.09.2020, 13:03, "Philipp Masharov" :
> >>> I will try it. Including information about experimental commands into
> >>> documentation sounds like a good idea. Am I need to create a Jira
> ticket?
> >>> Definitely I need to follow some process to enhance the product
> >>> documentation.
> >>>
> >>> On Mon, Sep 7, 2020 at 12:42 PM Nikolay Izhikov 
> wrote:
> >>>
>  Hello, Philipp.
> 
>  Indeed, lack of the information about experimental control.sh command
> is a
>  gap in Ignite documentation.
>  Do you want to contribute corresponding page to close this gap?
> 
>  To see all experimental commands you can build Ignite from sources and
>  execute the following commands
> 
> >> $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
> >> $ ./bin/control.sh
> 
> > 7 сент. 2020 г., в 12:26, Philipp Masharov 
>  написал(а):
> >
> > Hello Nikolay!
> >
> > It's more convenient to use an option instead of an environmental
>  variable.
> > But what is experiment commands? Are these commands not
>  production-ready? I
> > never heard about it and I can't find information about experiment
>  commands
> > in the product documentation.
> >
> > On Mon, Sep 7, 2020 at 12:08 PM Nikolay Izhikov  >
>  wrote:
> >
> >> Hello, Igniters.
> >>
> >> For now, experimental commands in control.sh can be enabled only via
> >> system property
> >>
> >> ```
> >> $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
> >> $ ./bin/control.sh
> >> ```
> >>
> >> I think we should add explicit argument for this.
> >>
> >> ```
> >> ./control.sh —enable-experimental
> >> ```
> >>
> >> WDYT?
> >
>
>

Re: [DISCUSSION] control.sh argument to enable experimental commands

[Nikolay Izhikov]

Philipp, at least, we have tracing and binary meta commands in the master.

Please, take a look at current Ignite sources.

> 7 сент. 2020 г., в 13:34, Nikolay Izhikov  написал(а):
> 
> What is wrong with that?
> 
> We just improve the way user enables experimental commands.
> JVM uses similar approach, please, take a look at 
> '-XX:+UnlockExperimentalVMOptions’.
> 
>> 7 сент. 2020 г., в 13:22, ткаленко кирилл  написал(а):
>> 
>> Hi, Nikolay!
>> 
>> It seems that you shouldn't just open experimental commands, that's why they 
>> are experimental.
>> 
>> 
>> 07.09.2020, 13:03, "Philipp Masharov" :
>>> I will try it. Including information about experimental commands into
>>> documentation sounds like a good idea. Am I need to create a Jira ticket?
>>> Definitely I need to follow some process to enhance the product
>>> documentation.
>>> 
>>> On Mon, Sep 7, 2020 at 12:42 PM Nikolay Izhikov  wrote:
>>> 
 Hello, Philipp.
 
 Indeed, lack of the information about experimental control.sh command is a
 gap in Ignite documentation.
 Do you want to contribute corresponding page to close this gap?
 
 To see all experimental commands you can build Ignite from sources and
 execute the following commands
 
>> $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
>> $ ./bin/control.sh
 
> 7 сент. 2020 г., в 12:26, Philipp Masharov 
 написал(а):
> 
> Hello Nikolay!
> 
> It's more convenient to use an option instead of an environmental
 variable.
> But what is experiment commands? Are these commands not
 production-ready? I
> never heard about it and I can't find information about experiment
 commands
> in the product documentation.
> 
> On Mon, Sep 7, 2020 at 12:08 PM Nikolay Izhikov 
 wrote:
> 
>> Hello, Igniters.
>> 
>> For now, experimental commands in control.sh can be enabled only via
>> system property
>> 
>> ```
>> $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
>> $ ./bin/control.sh
>> ```
>> 
>> I think we should add explicit argument for this.
>> 
>> ```
>> ./control.sh —enable-experimental
>> ```
>> 
>> WDYT?
> 

Re: [DISCUSSION] control.sh argument to enable experimental commands

[Nikolay Izhikov]

What is wrong with that?

We just improve the way user enables experimental commands.
JVM uses similar approach, please, take a look at 
'-XX:+UnlockExperimentalVMOptions’.

> 7 сент. 2020 г., в 13:22, ткаленко кирилл  написал(а):
> 
> Hi, Nikolay!
> 
> It seems that you shouldn't just open experimental commands, that's why they 
> are experimental.
> 
> 
> 07.09.2020, 13:03, "Philipp Masharov" :
>> I will try it. Including information about experimental commands into
>> documentation sounds like a good idea. Am I need to create a Jira ticket?
>> Definitely I need to follow some process to enhance the product
>> documentation.
>> 
>> On Mon, Sep 7, 2020 at 12:42 PM Nikolay Izhikov  wrote:
>> 
>>>  Hello, Philipp.
>>> 
>>>  Indeed, lack of the information about experimental control.sh command is a
>>>  gap in Ignite documentation.
>>>  Do you want to contribute corresponding page to close this gap?
>>> 
>>>  To see all experimental commands you can build Ignite from sources and
>>>  execute the following commands
>>> 
>>>  >> $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
>>>  >> $ ./bin/control.sh
>>> 
>>>  > 7 сент. 2020 г., в 12:26, Philipp Masharov 
>>>  написал(а):
>>>  >
>>>  > Hello Nikolay!
>>>  >
>>>  > It's more convenient to use an option instead of an environmental
>>>  variable.
>>>  > But what is experiment commands? Are these commands not
>>>  production-ready? I
>>>  > never heard about it and I can't find information about experiment
>>>  commands
>>>  > in the product documentation.
>>>  >
>>>  > On Mon, Sep 7, 2020 at 12:08 PM Nikolay Izhikov 
>>>  wrote:
>>>  >
>>>  >> Hello, Igniters.
>>>  >>
>>>  >> For now, experimental commands in control.sh can be enabled only via
>>>  >> system property
>>>  >>
>>>  >> ```
>>>  >> $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
>>>  >> $ ./bin/control.sh
>>>  >> ```
>>>  >>
>>>  >> I think we should add explicit argument for this.
>>>  >>
>>>  >> ```
>>>  >> ./control.sh —enable-experimental
>>>  >> ```
>>>  >>
>>>  >> WDYT?

Re: [DISCUSSION] control.sh argument to enable experimental commands

[ткаленко кирилл]

07.09.2020, 13:26, "ткаленко кирилл" :
> Adding option "—enable-experimental".
>
> 07.09.2020, 13:22, "ткаленко кирилл" :
>> Hi, Nikolay!
>>
>> It seems that you shouldn't just open experimental commands, that's why they 
>> are experimental.
>>
>> 07.09.2020, 13:03, "Philipp Masharov" :
>>>  I will try it. Including information about experimental commands into
>>>  documentation sounds like a good idea. Am I need to create a Jira ticket?
>>>  Definitely I need to follow some process to enhance the product
>>>  documentation.
>>>
>>>  On Mon, Sep 7, 2020 at 12:42 PM Nikolay Izhikov  
>>> wrote:
>>>
   Hello, Philipp.

   Indeed, lack of the information about experimental control.sh command is 
 a
   gap in Ignite documentation.
   Do you want to contribute corresponding page to close this gap?

   To see all experimental commands you can build Ignite from sources and
   execute the following commands

   >> $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
   >> $ ./bin/control.sh

   > 7 сент. 2020 г., в 12:26, Philipp Masharov 
   написал(а):
   >
   > Hello Nikolay!
   >
   > It's more convenient to use an option instead of an environmental
   variable.
   > But what is experiment commands? Are these commands not
   production-ready? I
   > never heard about it and I can't find information about experiment
   commands
   > in the product documentation.
   >
   > On Mon, Sep 7, 2020 at 12:08 PM Nikolay Izhikov 
   wrote:
   >
   >> Hello, Igniters.
   >>
   >> For now, experimental commands in control.sh can be enabled only via
   >> system property
   >>
   >> ```
   >> $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
   >> $ ./bin/control.sh
   >> ```
   >>
   >> I think we should add explicit argument for this.
   >>
   >> ```
   >> ./control.sh —enable-experimental
   >> ```
   >>
   >> WDYT?

Re: [DISCUSSION] control.sh argument to enable experimental commands

[Nikolay Izhikov]

Hello, Kirill.

What do you mean by «just open»?


> 7 сент. 2020 г., в 13:22, ткаленко кирилл  написал(а):
> 
> Hi, Nikolay!
> 
> It seems that you shouldn't just open experimental commands, that's why they 
> are experimental.
> 
> 
> 07.09.2020, 13:03, "Philipp Masharov" :
>> I will try it. Including information about experimental commands into
>> documentation sounds like a good idea. Am I need to create a Jira ticket?
>> Definitely I need to follow some process to enhance the product
>> documentation.
>> 
>> On Mon, Sep 7, 2020 at 12:42 PM Nikolay Izhikov  wrote:
>> 
>>>  Hello, Philipp.
>>> 
>>>  Indeed, lack of the information about experimental control.sh command is a
>>>  gap in Ignite documentation.
>>>  Do you want to contribute corresponding page to close this gap?
>>> 
>>>  To see all experimental commands you can build Ignite from sources and
>>>  execute the following commands
>>> 
>>>  >> $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
>>>  >> $ ./bin/control.sh
>>> 
>>>  > 7 сент. 2020 г., в 12:26, Philipp Masharov 
>>>  написал(а):
>>>  >
>>>  > Hello Nikolay!
>>>  >
>>>  > It's more convenient to use an option instead of an environmental
>>>  variable.
>>>  > But what is experiment commands? Are these commands not
>>>  production-ready? I
>>>  > never heard about it and I can't find information about experiment
>>>  commands
>>>  > in the product documentation.
>>>  >
>>>  > On Mon, Sep 7, 2020 at 12:08 PM Nikolay Izhikov 
>>>  wrote:
>>>  >
>>>  >> Hello, Igniters.
>>>  >>
>>>  >> For now, experimental commands in control.sh can be enabled only via
>>>  >> system property
>>>  >>
>>>  >> ```
>>>  >> $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
>>>  >> $ ./bin/control.sh
>>>  >> ```
>>>  >>
>>>  >> I think we should add explicit argument for this.
>>>  >>
>>>  >> ```
>>>  >> ./control.sh —enable-experimental
>>>  >> ```
>>>  >>
>>>  >> WDYT?

Re: [DISCUSSION] control.sh argument to enable experimental commands

[ткаленко кирилл]

Hi, Nikolay!

It seems that you shouldn't just open experimental commands, that's why they 
are experimental.


07.09.2020, 13:03, "Philipp Masharov" :
> I will try it. Including information about experimental commands into
> documentation sounds like a good idea. Am I need to create a Jira ticket?
> Definitely I need to follow some process to enhance the product
> documentation.
>
> On Mon, Sep 7, 2020 at 12:42 PM Nikolay Izhikov  wrote:
>
>>  Hello, Philipp.
>>
>>  Indeed, lack of the information about experimental control.sh command is a
>>  gap in Ignite documentation.
>>  Do you want to contribute corresponding page to close this gap?
>>
>>  To see all experimental commands you can build Ignite from sources and
>>  execute the following commands
>>
>>  >> $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
>>  >> $ ./bin/control.sh
>>
>>  > 7 сент. 2020 г., в 12:26, Philipp Masharov 
>>  написал(а):
>>  >
>>  > Hello Nikolay!
>>  >
>>  > It's more convenient to use an option instead of an environmental
>>  variable.
>>  > But what is experiment commands? Are these commands not
>>  production-ready? I
>>  > never heard about it and I can't find information about experiment
>>  commands
>>  > in the product documentation.
>>  >
>>  > On Mon, Sep 7, 2020 at 12:08 PM Nikolay Izhikov 
>>  wrote:
>>  >
>>  >> Hello, Igniters.
>>  >>
>>  >> For now, experimental commands in control.sh can be enabled only via
>>  >> system property
>>  >>
>>  >> ```
>>  >> $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
>>  >> $ ./bin/control.sh
>>  >> ```
>>  >>
>>  >> I think we should add explicit argument for this.
>>  >>
>>  >> ```
>>  >> ./control.sh —enable-experimental
>>  >> ```
>>  >>
>>  >> WDYT?

Re: [DISCUSSION] control.sh argument to enable experimental commands

[Philipp Masharov]

I executed control.sh with the option set to true and set false, made a
diff of control.sh outputs and got this:

Print absolute paths of unused archived wal segments on each node:
control.(sh|bat) --wal print
[consistentId1,consistentId2,,consistentIdN]

  Delete unused archived wal segments on each node:
control.(sh|bat) --wal delete
[consistentId1,consistentId2,,consistentIdN] [--yes]

I assume that these commands are in experiment mode.

On Mon, Sep 7, 2020 at 12:42 PM Nikolay Izhikov  wrote:

> Hello, Philipp.
>
> Indeed, lack of the information about experimental control.sh command is a
> gap in Ignite documentation.
> Do you want to contribute corresponding page to close this gap?
>
> To see all experimental commands you can build Ignite from sources and
> execute the following commands
>
> >> $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
> >> $ ./bin/control.sh
>
> > 7 сент. 2020 г., в 12:26, Philipp Masharov 
> написал(а):
> >
> > Hello Nikolay!
> >
> > It's more convenient to use an option instead of an environmental
> variable.
> > But what is experiment commands? Are these commands not
> production-ready? I
> > never heard about it and I can't find information about experiment
> commands
> > in the product documentation.
> >
> > On Mon, Sep 7, 2020 at 12:08 PM Nikolay Izhikov 
> wrote:
> >
> >> Hello, Igniters.
> >>
> >> For now, experimental commands in control.sh can be enabled only via
> >> system property
> >>
> >> ```
> >> $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
> >> $ ./bin/control.sh
> >> ```
> >>
> >> I think we should add explicit argument for this.
> >>
> >> ```
> >> ./control.sh —enable-experimental
> >> ```
> >>
> >> WDYT?
>
>

Re: [DISCUSSION] control.sh argument to enable experimental commands

[Nikolay Izhikov]

Philipp, thanks for your interest.

You may found useful the following links:

https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document


Note, right now, we change the way on how we document.
Please, take a look into "Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]» 
thread.

Don’t hesitate to ask in case you have any questions.

> 7 сент. 2020 г., в 12:26, Philipp Masharov  написал(а):
> 
> Hello Nikolay!
> 
> It's more convenient to use an option instead of an environmental variable.
> But what is experiment commands? Are these commands not production-ready? I
> never heard about it and I can't find information about experiment commands
> in the product documentation.
> 
> On Mon, Sep 7, 2020 at 12:08 PM Nikolay Izhikov  wrote:
> 
>> Hello, Igniters.
>> 
>> For now, experimental commands in control.sh can be enabled only via
>> system property
>> 
>> ```
>> $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
>> $ ./bin/control.sh
>> ```
>> 
>> I think we should add explicit argument for this.
>> 
>> ```
>> ./control.sh —enable-experimental
>> ```
>> 
>> WDYT?

Re: [DISCUSSION] control.sh argument to enable experimental commands

[Philipp Masharov]

I will try it. Including information about experimental commands into
documentation sounds like a good idea. Am I need to create a Jira ticket?
Definitely I need to follow some process to enhance the product
documentation.

On Mon, Sep 7, 2020 at 12:42 PM Nikolay Izhikov  wrote:

> Hello, Philipp.
>
> Indeed, lack of the information about experimental control.sh command is a
> gap in Ignite documentation.
> Do you want to contribute corresponding page to close this gap?
>
> To see all experimental commands you can build Ignite from sources and
> execute the following commands
>
> >> $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
> >> $ ./bin/control.sh
>
> > 7 сент. 2020 г., в 12:26, Philipp Masharov 
> написал(а):
> >
> > Hello Nikolay!
> >
> > It's more convenient to use an option instead of an environmental
> variable.
> > But what is experiment commands? Are these commands not
> production-ready? I
> > never heard about it and I can't find information about experiment
> commands
> > in the product documentation.
> >
> > On Mon, Sep 7, 2020 at 12:08 PM Nikolay Izhikov 
> wrote:
> >
> >> Hello, Igniters.
> >>
> >> For now, experimental commands in control.sh can be enabled only via
> >> system property
> >>
> >> ```
> >> $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
> >> $ ./bin/control.sh
> >> ```
> >>
> >> I think we should add explicit argument for this.
> >>
> >> ```
> >> ./control.sh —enable-experimental
> >> ```
> >>
> >> WDYT?
>
>

Re: [DISCUSSION] control.sh argument to enable experimental commands

[Philipp Masharov]

Hello Nikolay!

It's more convenient to use an option instead of an environmental variable.
But what is experiment commands? Are these commands not production-ready? I
never heard about it and I can't find information about experiment commands
in the product documentation.

On Mon, Sep 7, 2020 at 12:08 PM Nikolay Izhikov  wrote:

> Hello, Igniters.
>
> For now, experimental commands in control.sh can be enabled only via
> system property
>
> ```
> $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
> $ ./bin/control.sh
> ```
>
> I think we should add explicit argument for this.
>
> ```
> ./control.sh —enable-experimental
> ```
>
> WDYT?

[DISCUSSION] control.sh argument to enable experimental commands

[Nikolay Izhikov]

Hello, Igniters.

For now, experimental commands in control.sh can be enabled only via system 
property

```
$ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
$ ./bin/control.sh
```

I think we should add explicit argument for this.

```
./control.sh —enable-experimental
```

WDYT?

[jira] [Created] (IGNITE-13409) System view for metastorage items

[Nikolay Izhikov (Jira)]

Nikolay Izhikov created IGNITE-13409:


 Summary: System view for metastorage items
 Key: IGNITE-13409
 URL: https://issues.apache.org/jira/browse/IGNITE-13409
 Project: Ignite
  Issue Type: Improvement
Reporter: Nikolay Izhikov


Currently, list of binary metadata available via experimental {{control.sh}} 
command.
We need to provide a corresponding System view for this information.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13408) System view for binary metadata

[Nikolay Izhikov (Jira)]

Nikolay Izhikov created IGNITE-13408:


 Summary: System view for binary metadata
 Key: IGNITE-13408
 URL: https://issues.apache.org/jira/browse/IGNITE-13408
 Project: Ignite
  Issue Type: Improvement
Reporter: Nikolay Izhikov


Currently, list of binary metadata available via experimental {{control.sh}} 
command.
We need to provide a corresponding System view for this information.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: IGNITE-13407 Upgrade ignite.version to 2.10.0-SNAPSHOT in Ignite Extensions

[Ivan Pavlukhin]

Hi Saikat,

Thank you for working on it!

I noticed that Travis build for the PR failed to resolve
2.10.0-SNAPSHOT dependency as well.

Could you please hint me how Ignite SNAPSHOT dependencies (ignite-core
and so on) are intended to be available to ignite-extensions builds?

2020-09-06 22:01 GMT+03:00, Nikolay Izhikov :
> +1
>
>> 6 сент. 2020 г., в 21:43, Saikat Maitra 
>> написал(а):
>>
>> Hi,
>>
>> I observed that the Ignite Extensions build were failing as it was
>> depending on 2.9.0-SNAPSHOT and it was not available in artifactory.
>>
>> https://ci.ignite.apache.org/buildConfiguration/Tests_IgniteExtensions_Build/5576078?buildTab=log=3=1777
>>
>> I have updated the dependencies and created a PR.
>>
>> Jira https://issues.apache.org/jira/browse/IGNITE-13407
>>
>> PR https://github.com/apache/ignite-extensions/pull/22
>>
>> Please review and share feedback.
>>
>> Regards,
>> Saikat
>
>


-- 

Best regards,
Ivan Pavlukhin

Re: IEP-54: Schema-first approach for 3.0

[Ivan Pavlukhin]

Hi Val,

Thank you for your answer!

My understanding is a little bit different. Yes, schema evolution
definitely should be possible. But I see a main difference in "how
schema is updated". I treat a common SQL approach schema-first. Schema
and data manipulation operations are clearly separated and it enables
interesting capabilities, e.g. preventing untended schema changes by
mistaken data operations, restricting user permissions to change
schema.

> Schema-first means that schema exists in advance and all the stored data is 
> compliant with it - that's exactly what is proposed.

A schema-last approach mentioned in [1] also assumes that schema
exists, but it is inferred from data. Is not it more similar to the
proposing approach?

And I would like to say, that my main concern so far is mostly about
terminology. And I suppose if it confuses me then others might be
confused as well. My feeling is closer to "dynamic or liquid or may be
evolving schema".

[1] https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf

2020-09-07 0:47 GMT+03:00, Valentin Kulichenko :
> Hi Ivan,
>
> I don't see an issue with that. Schema-first means that schema exists in
> advance and all the stored data is compliant with it - that's exactly what
> is proposed. There are no restrictions prohibiting changes to the schema.
>
> -Val
>
> On Sat, Sep 5, 2020 at 9:52 PM Ivan Pavlukhin  wrote:
>
>> Alexey,
>>
>> I am a little bit confused with terminology. My understanding conforms
>> to a survey [1] (see part X Semi Structured Data). Can we really treat
>> a "dynamic schema" approach as a kind of "schema-first"?
>>
>> [1]
>> https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf
>>
>> 2020-09-02 1:53 GMT+03:00, Denis Magda :
>> >>
>> >> However, could you please elaborate on the relation between Ignite and
>> >> ORM?
>> >> Is there a use case for Hibernate running on top of Ignite (I haven't
>> >> seen
>> >> one so far)? If so, what is missing exactly on the Ignite side to
>> support
>> >> this? In my understanding, all you need is SQL API which we already
>> have.
>> >> Am I missing something?
>> >
>> >
>> > Good point, yes, if all the ORM integrations use Ignite SQL APIs
>> > internally, then they can easily translate an Entity object into an
>> > INSERT/UPDATE statement that lists all the object's fields. Luckily,
>> > our
>> > Spring Data integration is already based on the Ignite SQL APIs and
>> > needs
>> > to be improved once the schema-first approach is supported. That would
>> > solve a ton of usability issues.
>> >
>> > I would revise the Hibernate integration as well during the Ignite 3.0
>> dev
>> > phase. Can't say if it's used a lot but Spring Data is getting traction
>> for
>> > sure.
>> >
>> > @Michael Pollind, I'll loop you in as long as you've started working on
>> the
>> > Ignite support for Micornaut Data
>> >  and
>> > came across some challenges. Just watch this discussion. That's what is
>> > coming in Ignite 3.0.
>> >
>> >
>> > -
>> > Denis
>> >
>> >
>> > On Mon, Aug 31, 2020 at 5:11 PM Valentin Kulichenko <
>> > valentin.kuliche...@gmail.com> wrote:
>> >
>> >> Hi Denis,
>> >>
>> >> Generally speaking, I believe that the schema-first approach natively
>> >> addresses the issue if duplicate fields in key and value objects,
>> because
>> >> schema will be created for a cache, not for an object, as it happens
>> now.
>> >> Basically, the schema will define whether there is a primary key or
>> >> not,
>> >> and which fields are included in case there is one. Any API that we
>> would
>> >> have must be compliant with this, so it becomes fairly easy to work
>> >> with
>> >> data as with a set of records, rather than key-value pairs.
>> >>
>> >> However, could you please elaborate on the relation between Ignite and
>> >> ORM?
>> >> Is there a use case for Hibernate running on top of Ignite (I haven't
>> >> seen
>> >> one so far)? If so, what is missing exactly on the Ignite side to
>> support
>> >> this? In my understanding, all you need is SQL API which we already
>> have.
>> >> Am I missing something?
>> >>
>> >> -Val
>> >>
>> >> On Mon, Aug 31, 2020 at 2:08 PM Denis Magda  wrote:
>> >>
>> >> > Val,
>> >> >
>> >> > I would propose adding another point to the motivations list which
>> >> > is
>> >> > related to the ORM frameworks such as Spring Data, Hibernate,
>> Micronaut
>> >> and
>> >> > many others.
>> >> >
>> >> > Presently, the storage engine requires to distinguish key objects
>> >> > from
>> >> the
>> >> > value ones that complicate the usage of Ignite with those ORM
>> >> > frameworks
>> >> > (especially if a key object comprises several fields). More on this
>> can
>> >> be
>> >> > found here:
>> >> >
>> >> >
>> >>
>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Key-and-Value-fields-with-same-name-and-SQL-DML-td47557.html
>> >> >
>> >> > It will be nice if the new schema-first approach