Re: [VOTE][EXTENSION] Release Apache Ignite spring-data-all-ext extensions 1.0.0 RC5
+1 Checked on Ubuntu 20.04. Created separate Spring Applications for each version of Spring Data Integrations using the Maven RC repository and following the documentation. Tested common use cases. Checked archive signs. Built from the sources. On 01.07.2021 00:31, Sergei Ryzhov wrote: +1 Building modules from source and running tests successfully on Java 8 ср, 30 июн. 2021 г. в 16:02, Alex Plehanov : +1 Check sha512, sign of archives and versions of released artifacts. Check licenses with "mvn validate -DskipTests -P check-licenses" Build modules from the source with Java 8 Run examples: from modules/spring-data-2.0-ext/examples with artifacts: Ignite v2.10.0/ignite-spring-data-2.0-ext v1.0.0/spring-data v2.0 from modules/spring-data-2.2-ext/examples with artifacts: Ignite v2.10.0/ignite-spring-data-2.2-ext v1.0.0/spring-data v2.2 ср, 30 июн. 2021 г. в 15:13, Nikita Amelchev : Dear Ignite Community, I have uploaded a release candidate of the following extension modules: spring-data-commons spring-data-ext spring-data-2.0-ext spring-data-2.2-ext The release candidate of the extensions: https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-spring-data-all-ext/1.0.0-rc5/ The instruction for building from the source package is placed in the DEVNOTES file. The following staging can be used for testing: https://repository.apache.org/content/repositories/orgapacheignite-1524 Tags were created: ignite-spring-data-commons-1.0.0-rc5 ignite-spring-data-ext-1.0.0-rc5 ignite-spring-data-2.2-ext-1.0.0-rc5 ignite-spring-data-2.0-ext-1.0.0-rc5 ignite-spring-data-all-ext-1.0.0-rc5 https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=4a3ba7b542c46501d425dfa378ac8670e253f49a RELEASE NOTES: https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=RELEASE_NOTES.txt;h=22f8d665c194ba2dfa6167354d97ad53d5ef206f;hb=c8de80ee14d1fb76d6cbb0b18513bb70b499c3cb DOCUMENTATION: Documentation of listed extensions was updated in the following issues: https://issues.apache.org/jira/issues/?filter=-1=key%20in%20(IGNITE-14662%2CIGNITE-14398%2CIGNITE-14493) The vote is formal, see voting guidelines https://www.apache.org/foundation/voting.html +1 - to accept all the Apache Ignite spring-data-all-ext extensions 1.0.0-rc5 listed in the thread 0 - don't care either way -1 - DO NOT accept either of the Apache Ignite spring-data-all-ext extensions 1.0.0-rc5 (explain why) The vote will hold for 3 days and will end on July 3 2021 13:00 UTC https://www.timeanddate.com/countdown/generic?iso=20210703T13=0=cursive -- Best wishes, Amelchev Nikita
Re: [VOTE][EXTENSION] Release Apache Ignite spring-data-all-ext extensions 1.0.0 RC5
+1 Building modules from source and running tests successfully on Java 8 ср, 30 июн. 2021 г. в 16:02, Alex Plehanov : > +1 > > Check sha512, sign of archives and versions of released artifacts. > Check licenses with "mvn validate -DskipTests -P check-licenses" > Build modules from the source with Java 8 > Run examples: > from modules/spring-data-2.0-ext/examples with artifacts: Ignite > v2.10.0/ignite-spring-data-2.0-ext v1.0.0/spring-data v2.0 > from modules/spring-data-2.2-ext/examples with artifacts: Ignite > v2.10.0/ignite-spring-data-2.2-ext v1.0.0/spring-data v2.2 > > > ср, 30 июн. 2021 г. в 15:13, Nikita Amelchev : > > > Dear Ignite Community, > > > > I have uploaded a release candidate of the following extension modules: > > > > spring-data-commons > > spring-data-ext > > spring-data-2.0-ext > > spring-data-2.2-ext > > > > The release candidate of the extensions: > > > > > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-spring-data-all-ext/1.0.0-rc5/ > > > > The instruction for building from the source package is placed in the > > DEVNOTES file. > > > > The following staging can be used for testing: > > https://repository.apache.org/content/repositories/orgapacheignite-1524 > > > > Tags were created: > > > > ignite-spring-data-commons-1.0.0-rc5 > > ignite-spring-data-ext-1.0.0-rc5 > > ignite-spring-data-2.2-ext-1.0.0-rc5 > > ignite-spring-data-2.0-ext-1.0.0-rc5 > > ignite-spring-data-all-ext-1.0.0-rc5 > > > > > https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=4a3ba7b542c46501d425dfa378ac8670e253f49a > > > > RELEASE NOTES: > > > > > https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=RELEASE_NOTES.txt;h=22f8d665c194ba2dfa6167354d97ad53d5ef206f;hb=c8de80ee14d1fb76d6cbb0b18513bb70b499c3cb > > > > DOCUMENTATION: > > Documentation of listed extensions was updated in the following issues: > > > > > https://issues.apache.org/jira/issues/?filter=-1=key%20in%20(IGNITE-14662%2CIGNITE-14398%2CIGNITE-14493) > > > > The vote is formal, see voting guidelines > > https://www.apache.org/foundation/voting.html > > > > +1 - to accept all the Apache Ignite spring-data-all-ext extensions > > 1.0.0-rc5 listed in the thread > > 0 - don't care either way > > -1 - DO NOT accept either of the Apache Ignite spring-data-all-ext > > extensions 1.0.0-rc5 (explain why) > > > > The vote will hold for 3 days and will end on July 3 2021 13:00 UTC > > > > > https://www.timeanddate.com/countdown/generic?iso=20210703T13=0=cursive > > > > -- > > Best wishes, > > Amelchev Nikita > > > -- Best regards, Sergei Ryzhov
Re: [VOTE][EXTENSION] Release Apache Ignite performance-statistics-ext extension 1.0.0 RC5
+1 Checked a report with an ignite-performance-statistics-example. -- Best regards, Sergei Ryzhov
Re: [Announcement] Apache Ignite 2.11 Code Freeze started
Hello, Alexey! Is it possible to add system views for BaselineNode attributes [1] and corresponding documentation [2] to 2.11? 1. https://issues.apache.org/jira/browse/IGNITE-15007 2. https://issues.apache.org/jira/browse/IGNITE-15028 ср, 30 июн. 2021 г. в 11:07, Nikita Amelchev : > Thanks! I have cherry-picked the commit to the 2.11 branch. > > ср, 30 июн. 2021 г. в 11:00, Alexey Gidaspov : > > > > Hi, Nikita! > > > > I think it's important fix and should be included in 2.11 release. I've > tagged ticket by fixVersion 2.11. Can you cherry-pick it to ignite-2.11 > branch? And please fill release notes or delete flag. > > > > On 2021/06/30 07:55:04, Nikita Amelchev wrote: > > > Hello, Alexey. > > > > > > I suggest adding to the 2.11 scope the resolved issue [1]. It fixes > > > incorrect values of cache, cache groups, data region metrics after > > > cluster reactivation. > > > WDYT? > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-14990 > > > > > > вт, 29 июн. 2021 г. в 15:09, Alexey Gidaspov : > > > > > > > > Ok, we can add this fix to release scope. > > > > > > > > On 2021/06/29 08:36:00, Сурков Александр > Викторович wrote: > > > > > Hi Alexey. > > > > > > > > > > I think need add ticket: JmxMetricExporter fails to export > discovery metrics - https://issues.apache.org/jira/browse/IGNITE-14376 > > > > > > > > > > On 2021/06/15 14:43:55, Alexey Gidaspov wrote: > > > > > > Apache Ignite 2.11 Code Freeze started now> > > > > > > > > > > > > > > > > > > -- > > > Best wishes, > > > Amelchev Nikita > > > > > > > -- > Best wishes, > Amelchev Nikita >
Re: Ignite 3.0 IgniteTables API Improvement Suggestion
Pavel,
Thanks for your response, makes sense.
Regarding the values() method: I would instead add the required methods to
the Tuple itself. E.g., it can implement Iterable, and additionally have
the value(int index) method, plus anything else that we might need.
I don't like returning a collection, because in Java it's a much wider API.
We will end up returning an implementation that throws
UnsupportedOperationException for most of the methods. I know this approach
is not uncommon in the Java world, but this doesn't make it good.
In .NET and other languages, we can use other abstractions, of course.
Every platform has its own specifics and practices, so APIs don't have to
be identical.
-Val
On Wed, Jun 30, 2021 at 7:44 AM Pavel Tupitsyn wrote:
> Hi Andrey,
>
> > This will force us to bother about interfaces/contracts and compatibility
> > aspects in the future
>
> Schemas and versions are part of thin client wire protocol.
> This protocol is a public API - we'll have to care about compatibility
> anyway.
>
> Schema evolution is an important feature that users should understand.
> I see no reason to hide it from the API.
>
>
> > We already have a ticket [1]...
> > Will 'idx -> column' mapping only be enough for your purposes?
>
> The ticket sounds reasonable, but there are no API details.
> At the very least we need public access to column names, types, and
> indexes.
> I propose something like this:
>
> interface Tuple {
> TupleSchema getSchema();
> }
>
> class TupleSchema {
> int getVersion();
> List getColumns();
> }
>
> class TupleSchemaColumn {
> int index;
> String name;
> String typeName;
> bool isKey;
> }
>
> On Wed, Jun 30, 2021 at 11:05 AM Andrey Mashenkov <
> andrey.mashen...@gmail.com> wrote:
>
> > Hi Pavel,
> >
> > 2. Schema and its version are internal things and shouldn't be exposed,
> > otherwise, eventually, it will lead to the user will manage schemas
> > manually on his side for some purposes.
> > This will force us to bother about interfaces/contracts and compatibility
> > aspects in the future with uncertain benefits for a user.
> >
> > As SchemaDescriptor is an internal API and the user expects a public API.
> > I don't think we want to convert the descriptor back into public API
> terms
> > and it may be impossible for some data.
> >
> > We already have a ticket [1] to support accessing a column by an index
> and
> > exposing some metadata.
> > Will 'idx -> column' mapping only be enough for your purposes?
> >
> > > int Tuple.columnIndex(String)
> >
> >
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-14342
> >
> > On Wed, Jun 30, 2021 at 9:34 AM Pavel Tupitsyn
> > wrote:
> >
> > > Hi Val,
> > >
> > > Thanks for the comments, please see below:
> > >
> > >
> > > > Users will identify tables using names, they will never use IDs
> > >
> > > Ok, let's keep it this way.
> > >
> > >
> > > > Sounds like the Tuple should implement Iterable.
> > >
> > > I don't think Iterable is enough.
> > > We should have a way to get values by column index: Tuple.value(int
> > index),
> > > where index is according to column order in schema.
> > >
> > > Combined with Tuple.schema(), it will provide an efficient way to
> consume
> > > data -
> > > users will be able to read columns in any order, skip some of them,
> etc.
> > >
> > > This is a common pattern in APIs like ODBC or System.Data [1],
> > > which we'll need to implement on top of our thin clients later.
> > >
> > >
> > > > However, whether a user might
> > > > need to get a table schema for a particular version, I'm not sure. Do
> > you
> > > > have a use case in mind for this? If not, I would keep this internal
> > >
> > > Ok, we can keep the Table.schema(ver) method internal, as long as
> > > Table.schemas() is public and includes schema versions.
> > >
> > >
> > > > We already have async counterparts for all the methods where this is
> > > applicable
> > >
> > > IgniteTables.createTable(), dropTable(), tables(), table() do not have
> > > async counterparts,
> > > while internally they perform blocking wait on some futures.
> > >
> > >
> > > [1]
> > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.data.idatareader?view=net-5.0
> > >
> > > On Wed, Jun 30, 2021 at 4:51 AM Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com> wrote:
> > >
> > > > Hi Pavel,
> > > >
> > > > Please see my comments below.
> > > >
> > > > -Val
> > > >
> > > > On Tue, Jun 29, 2021 at 2:23 PM Pavel Tupitsyn >
> > > > wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > While working on "IEP-76 Thin Client Protocol for Ignite 3.0" [1]
> (to
> > > be
> > > > > discussed separately), the following suggestions for the Table API
> > came
> > > > up:
> > > > >
> > > > > 1. Expose table IDs: sending table name with every operation (e.g.
> > GET)
> > > > is
> > > > > inefficient, string serialization is expensive by itself and names
> > can
> > > be
> > > > > long.
> > > > > - Table.id()
> > > > > -
Setting IGNITE_TO_STRING_INCLUDE_SENSITIVE=false prevents VM Arguments output
Hi Igniters, This feature [1, 2] prevents logging of the VM arguments when IGNITE_TO_STRING_INCLUDE_SENSITIVE option is set to false. Till now, method IgniteKernal#ackVmArguments remains mostly the same [3]. Is this behaviour actual now? Often, we should be able to get from logs the actual VM options used at startup even if output of sensitive data is restricted. 1. https://issues.apache.org/jira/browse/IGNITE-4991 2. https://github.com/apache/ignite/pull/2428/commits/4f90b6fd77bd23fa818620f0757b792ba388ef93 3. https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java#L3002
[ANNOUNCE] Apache Ignite 3.0.0-alpha2 is released!
Igniters, I'm happy to announce that Ignite 3 project reached a significant milestone, as we release the 2nd alpha version of the product. On top of the functionality that was previously released, Alpha 2 adds the following major features: - Replication infrastructure based on Raft. - New in-memory atomic storage with the basic insert-read functionality. - New schema management engine and API. There is an ability to create a cluster, and use the new Table to API to write and read the data. Basic code examples are available here: https://github.com/apache/ignite-3/tree/3.0.0-alpha2/examples The best way to try the release out is to go through the Getting Started Guide: https://ignite.apache.org/docs/3.0.0-alpha If there are any questions, issues, or thoughts, please do not hesitate to reply to this email. Ignite Community is welcoming any feedback and will consider it in future development. -Val
Collision SPI Not Adhering to Specification
Hi All, I have been playing around with Collision SPI and specifically used FifoQueueCollisionSPI and noticed that it is not really adhering to the specified task of restricting the number of concurrent tasks that can be run. Specifically, if there is only one slot available and N tasks concurrently land onto the node, all nodes will take the current active count, compare it with maximum jobs count and proceed. Essentially, we have no concurrency safety there. I propose refactoring FifoQueueCollisionSPI to use semaphores and/or atomic variables to ensure that all jobs get a realistic view of the active count. Please share your thoughts, Regards, Atri -- Regards, Atri Apache Concerted
[MTCGA]: new failures in builds [6061491] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. *Test with high flaky rate in master CacheExchangeMergeTest.testJoinExchangeCoordinatorChange_NoMerge_2 https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=8410866350920832275=%3Cdefault%3E=testDetails No changes in the build - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 21:22:26 30-06-2021
Re: Defrag?
Hey Ilya It's the data tables that keep growing not the WAL. We will try to rebuild the cache and see if that fixes the issue On Mon, Jun 28, 2021 at 8:46 AM Ilya Kasnacheev wrote: > Hello! > > Is it WAL (wal/) that is growing or checkpoint space (db/)? If latter, any > specific caches that are growing unbound? > > If letter, you can try creating a new cache, moving the relevant data to > this new cache, switch to using it, and then drop the old cache - should > reclaim the space. > > Regards, > -- > Ilya Kasnacheev > > > пн, 28 июн. 2021 г. в 17:34, Ryan Trollip : > >> Is this why the native disk storage just keeps growing and does not >> reduce after we delete from ignite using SQL? >> We are up to 80GB on disk now on some instances. We implemented a custom >> archiving feature to move older data out of ignite cache to a PostgresSQL >> database but when we delete that data from ignite instance, the disk data >> size ignite is using stays the same, and then keeps growing, and >> growing >> >> On Thu, Jun 24, 2021 at 7:10 PM Denis Magda wrote: >> >>> Ignite fellows, >>> >>> I remember some of us worked on the persistence defragmentation >>> features. Has it been merged? >>> >>> @Valentin Kulichenko probably you know >>> the latest state. >>> >>> - >>> Denis >>> >>> On Thu, Jun 24, 2021 at 11:59 AM Ilya Kasnacheev < >>> ilya.kasnach...@gmail.com> wrote: >>> Hello! You can probably drop the entire cache and then re-populate it via loadCache(), etc. Regards, -- Ilya Kasnacheev ср, 23 июн. 2021 г. в 21:47, Ryan Trollip : > Thanks, Ilya, we may have to consider moving back to non-native > storage and caching more selectively as the performance degrades when > there > is a lot of write/delete activity or tables with large amounts of rows. > This is with SQL with indexes and the use of query plans etc. > > Is there any easy way to rebuild the entire native database after > hours? e.g. with a batch run on the weeknds? > > On Wed, Jun 23, 2021 at 7:39 AM Ilya Kasnacheev < > ilya.kasnach...@gmail.com> wrote: > >> Hello! >> >> I don't think there's anything ready to use, but "killing >> performance" from fragmentation is also not something reported too often. >> >> Regards, >> -- >> Ilya Kasnacheev >> >> >> ср, 16 июн. 2021 г. в 04:39, Ryan Trollip : >> >>> We see continual very large growth to data with ignite native. We >>> have a very chatty use case that's creating and deleting stuff often. >>> The >>> data on disk just keeps growing at an explosive rate. So much so we >>> ported >>> this to a DB to see the difference and the DB is much smaller. I was >>> searching to see if someone has the same issue. This is also killing >>> performance. >>> >>> Founds this: >>> >>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-47%3A+Native+persistence+defragmentation >>> >>> Apparently, there is no auto-rebalancing of pages? or cleanup of >>> pages? >>> >>> Has anyone implemented a workaround to rebuild the cache and indexes >>> say on a weekly basis to get it to behave reasonably? >>> >>> Thanks >>> >>
Re: Ignite 3.0 IgniteTables API Improvement Suggestion
Hi Andrey,
> This will force us to bother about interfaces/contracts and compatibility
> aspects in the future
Schemas and versions are part of thin client wire protocol.
This protocol is a public API - we'll have to care about compatibility
anyway.
Schema evolution is an important feature that users should understand.
I see no reason to hide it from the API.
> We already have a ticket [1]...
> Will 'idx -> column' mapping only be enough for your purposes?
The ticket sounds reasonable, but there are no API details.
At the very least we need public access to column names, types, and indexes.
I propose something like this:
interface Tuple {
TupleSchema getSchema();
}
class TupleSchema {
int getVersion();
List getColumns();
}
class TupleSchemaColumn {
int index;
String name;
String typeName;
bool isKey;
}
On Wed, Jun 30, 2021 at 11:05 AM Andrey Mashenkov <
andrey.mashen...@gmail.com> wrote:
> Hi Pavel,
>
> 2. Schema and its version are internal things and shouldn't be exposed,
> otherwise, eventually, it will lead to the user will manage schemas
> manually on his side for some purposes.
> This will force us to bother about interfaces/contracts and compatibility
> aspects in the future with uncertain benefits for a user.
>
> As SchemaDescriptor is an internal API and the user expects a public API.
> I don't think we want to convert the descriptor back into public API terms
> and it may be impossible for some data.
>
> We already have a ticket [1] to support accessing a column by an index and
> exposing some metadata.
> Will 'idx -> column' mapping only be enough for your purposes?
>
> > int Tuple.columnIndex(String)
>
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-14342
>
> On Wed, Jun 30, 2021 at 9:34 AM Pavel Tupitsyn
> wrote:
>
> > Hi Val,
> >
> > Thanks for the comments, please see below:
> >
> >
> > > Users will identify tables using names, they will never use IDs
> >
> > Ok, let's keep it this way.
> >
> >
> > > Sounds like the Tuple should implement Iterable.
> >
> > I don't think Iterable is enough.
> > We should have a way to get values by column index: Tuple.value(int
> index),
> > where index is according to column order in schema.
> >
> > Combined with Tuple.schema(), it will provide an efficient way to consume
> > data -
> > users will be able to read columns in any order, skip some of them, etc.
> >
> > This is a common pattern in APIs like ODBC or System.Data [1],
> > which we'll need to implement on top of our thin clients later.
> >
> >
> > > However, whether a user might
> > > need to get a table schema for a particular version, I'm not sure. Do
> you
> > > have a use case in mind for this? If not, I would keep this internal
> >
> > Ok, we can keep the Table.schema(ver) method internal, as long as
> > Table.schemas() is public and includes schema versions.
> >
> >
> > > We already have async counterparts for all the methods where this is
> > applicable
> >
> > IgniteTables.createTable(), dropTable(), tables(), table() do not have
> > async counterparts,
> > while internally they perform blocking wait on some futures.
> >
> >
> > [1]
> >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.data.idatareader?view=net-5.0
> >
> > On Wed, Jun 30, 2021 at 4:51 AM Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> > > Hi Pavel,
> > >
> > > Please see my comments below.
> > >
> > > -Val
> > >
> > > On Tue, Jun 29, 2021 at 2:23 PM Pavel Tupitsyn
> > > wrote:
> > >
> > > > Igniters,
> > > >
> > > > While working on "IEP-76 Thin Client Protocol for Ignite 3.0" [1] (to
> > be
> > > > discussed separately), the following suggestions for the Table API
> came
> > > up:
> > > >
> > > > 1. Expose table IDs: sending table name with every operation (e.g.
> GET)
> > > is
> > > > inefficient, string serialization is expensive by itself and names
> can
> > be
> > > > long.
> > > > - Table.id()
> > > > - IgniteTables.table(UUID)
> > > > - IgniteTables.dropTable(UUID)
> > > >
> > >
> > > I don't think this should be a part of the public API. Users will
> > identify
> > > tables using names, they will never use IDs. As an internal
> optimization
> > > though - sure, we can have that. Sounds similar to the cache ID in 2.x.
> > >
> > >
> > > >
> > > > 2. Expose tuple schemas: to reduce payload size when sending tuples
> to
> > > the
> > > > client, we'll write only the schema version and column values, then
> the
> > > > client can retrieve and cache schemas (ordered set of columns per
> > > version).
> > > > - Tuple.schema()
> > > > - Table.schemas()
> > > > - Table.schema(ver)
> > > >
> > >
> > > Exposing the schema of a tuple makes sense. However, whether a user
> might
> > > need to get a table schema for a particular version, I'm not sure. Do
> you
> > > have a use case in mind for this? If not, I would keep this internal as
> > > well.
> > >
> > >
> > > >
> > > > 3. Expose tuple values as a collection: to serialize
Re: [VOTE][EXTENSION] Release Apache Ignite spring-data-all-ext extensions 1.0.0 RC5
+1 Check sha512, sign of archives and versions of released artifacts. Check licenses with "mvn validate -DskipTests -P check-licenses" Build modules from the source with Java 8 Run examples: from modules/spring-data-2.0-ext/examples with artifacts: Ignite v2.10.0/ignite-spring-data-2.0-ext v1.0.0/spring-data v2.0 from modules/spring-data-2.2-ext/examples with artifacts: Ignite v2.10.0/ignite-spring-data-2.2-ext v1.0.0/spring-data v2.2 ср, 30 июн. 2021 г. в 15:13, Nikita Amelchev : > Dear Ignite Community, > > I have uploaded a release candidate of the following extension modules: > > spring-data-commons > spring-data-ext > spring-data-2.0-ext > spring-data-2.2-ext > > The release candidate of the extensions: > > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-spring-data-all-ext/1.0.0-rc5/ > > The instruction for building from the source package is placed in the > DEVNOTES file. > > The following staging can be used for testing: > https://repository.apache.org/content/repositories/orgapacheignite-1524 > > Tags were created: > > ignite-spring-data-commons-1.0.0-rc5 > ignite-spring-data-ext-1.0.0-rc5 > ignite-spring-data-2.2-ext-1.0.0-rc5 > ignite-spring-data-2.0-ext-1.0.0-rc5 > ignite-spring-data-all-ext-1.0.0-rc5 > > https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=4a3ba7b542c46501d425dfa378ac8670e253f49a > > RELEASE NOTES: > > https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=RELEASE_NOTES.txt;h=22f8d665c194ba2dfa6167354d97ad53d5ef206f;hb=c8de80ee14d1fb76d6cbb0b18513bb70b499c3cb > > DOCUMENTATION: > Documentation of listed extensions was updated in the following issues: > > https://issues.apache.org/jira/issues/?filter=-1=key%20in%20(IGNITE-14662%2CIGNITE-14398%2CIGNITE-14493) > > The vote is formal, see voting guidelines > https://www.apache.org/foundation/voting.html > > +1 - to accept all the Apache Ignite spring-data-all-ext extensions > 1.0.0-rc5 listed in the thread > 0 - don't care either way > -1 - DO NOT accept either of the Apache Ignite spring-data-all-ext > extensions 1.0.0-rc5 (explain why) > > The vote will hold for 3 days and will end on July 3 2021 13:00 UTC > > https://www.timeanddate.com/countdown/generic?iso=20210703T13=0=cursive > > -- > Best wishes, > Amelchev Nikita >
[VOTE][EXTENSION] Release Apache Ignite spring-data-all-ext extensions 1.0.0 RC5
Dear Ignite Community, I have uploaded a release candidate of the following extension modules: spring-data-commons spring-data-ext spring-data-2.0-ext spring-data-2.2-ext The release candidate of the extensions: https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-spring-data-all-ext/1.0.0-rc5/ The instruction for building from the source package is placed in the DEVNOTES file. The following staging can be used for testing: https://repository.apache.org/content/repositories/orgapacheignite-1524 Tags were created: ignite-spring-data-commons-1.0.0-rc5 ignite-spring-data-ext-1.0.0-rc5 ignite-spring-data-2.2-ext-1.0.0-rc5 ignite-spring-data-2.0-ext-1.0.0-rc5 ignite-spring-data-all-ext-1.0.0-rc5 https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=4a3ba7b542c46501d425dfa378ac8670e253f49a RELEASE NOTES: https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=RELEASE_NOTES.txt;h=22f8d665c194ba2dfa6167354d97ad53d5ef206f;hb=c8de80ee14d1fb76d6cbb0b18513bb70b499c3cb DOCUMENTATION: Documentation of listed extensions was updated in the following issues: https://issues.apache.org/jira/issues/?filter=-1=key%20in%20(IGNITE-14662%2CIGNITE-14398%2CIGNITE-14493) The vote is formal, see voting guidelines https://www.apache.org/foundation/voting.html +1 - to accept all the Apache Ignite spring-data-all-ext extensions 1.0.0-rc5 listed in the thread 0 - don't care either way -1 - DO NOT accept either of the Apache Ignite spring-data-all-ext extensions 1.0.0-rc5 (explain why) The vote will hold for 3 days and will end on July 3 2021 13:00 UTC https://www.timeanddate.com/countdown/generic?iso=20210703T13=0=cursive -- Best wishes, Amelchev Nikita
[VOTE][EXTENSION] Release Apache Ignite performance-statistics-ext extension 1.0.0 RC5
Dear Ignite Community, I have uploaded a release candidate of the performance-statistics-ext extension module. The release candidate of the performance-statistics-ext extension: https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-performance-statistics-ext/1.0.0-rc5/ The instruction for building from the source package is placed in the DEVNOTES file. The following staging can be used for testing: https://repository.apache.org/content/repositories/orgapacheignite-1523 Tag was created: ignite-performance-statistics-ext-1.0.0-rc5 https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=refs/tags/ignite-performance-statistics-ext-1.0.0-rc5 RELEASE NOTES: https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=RELEASE_NOTES.txt;h=22f8d665c194ba2dfa6167354d97ad53d5ef206f;hb=c8de80ee14d1fb76d6cbb0b18513bb70b499c3cb DOCUMENTATION: Documentation of extension was updated in the issue: https://issues.apache.org/jira/browse/IGNITE-14417 The vote is formal, see voting guidelines https://www.apache.org/foundation/voting.html +1 - to accept the Apache Ignite performance-statistics-ext extension 1.0.0-rc5 0 - don't care either way -1 - DO NOT accept the Apache Ignite performance-statistics-ext extension 1.0.0-rc5 (explain why) The vote will hold for 3 days and will end on July 3 2021 13:00 UTC https://www.timeanddate.com/countdown/generic?iso=20210703T13=0=cursive -- Best wishes, Amelchev Nikita
Re: [VOTE][EXTENSION] Release Apache Ignite performance-statistics-ext extension 1.0.0 RC4
Folks, Permission to create and configure the extensions release project can't be granted to me. I have created the issue [1]. Petr, could you take a look, please? I think we can start RC5 now and use TC with the next an extension release. [1] https://issues.apache.org/jira/browse/IGNITE-15032 вт, 29 июн. 2021 г. в 13:21, Nikita Amelchev : > > +1 for TC automatization, > > I can try to add such a process but I don't have a proper role. Can > anyone grant me permission to create and configure the extensions > release project on the TC? > > вт, 29 июн. 2021 г. в 12:56, Anton Vinogradov : > > > > Folks, seems we MUST automate such a check, as well as the whole > > extension release process before the next release attempt? > > We may (must?) use AI release automation as a booster. > > Please let me know if any help required. > > > > On Tue, Jun 29, 2021 at 11:59 AM Ilya Kasnacheev > > wrote: > > > > > Hello! > > > > > > -1 (binding) > > > > > > I did not check the rest of the package, but both of zip deliverables > > > suffer from having a lot of files at the root of zip file. > > > > > >- ignite-performance-statistics-ext-1.0.0-bin.zip > > >< > > > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-performance-statistics-ext/1.0.0-rc4/ignite-performance-statistics-ext-1.0.0-bin.zip > > > > > > > should > > >have ignite-performance-statistics-ext-1.0.0-bin/ as a root directory > > >and nothing else on the 1st level > > >- ignite-performance-statistics-ext-1.0.0-src.zip > > >< > > > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-performance-statistics-ext/1.0.0-rc4/ignite-performance-statistics-ext-1.0.0-src.zip > > > > > > > should > > >have ignite-performance-statistics-ext-1.0.0-src/ as a root directory > > > and > > >nothing else on the 1st level. > > > > > > > > > While we are waiting for rebuild, I hope other users will try the actual > > > functionality. > > > > > > Regards, > > > -- > > > Ilya Kasnacheev > > > > > > > > > вт, 29 июн. 2021 г. в 01:51, Saikat Maitra : > > > > > > > +1 > > > > > > > > On Fri, Jun 25, 2021 at 6:34 AM Sergei Ryzhov > > > > wrote: > > > > > > > > > +1 > > > > > > > > > > -- > > > > > Best regards, > > > > > Sergei Ryzhov > > > > > > > > > > > > > > > > -- > Best wishes, > Amelchev Nikita -- Best wishes, Amelchev Nikita
Re: [Announcement] Apache Ignite 2.11 Code Freeze started
Thanks! I have cherry-picked the commit to the 2.11 branch. ср, 30 июн. 2021 г. в 11:00, Alexey Gidaspov : > > Hi, Nikita! > > I think it's important fix and should be included in 2.11 release. I've > tagged ticket by fixVersion 2.11. Can you cherry-pick it to ignite-2.11 > branch? And please fill release notes or delete flag. > > On 2021/06/30 07:55:04, Nikita Amelchev wrote: > > Hello, Alexey. > > > > I suggest adding to the 2.11 scope the resolved issue [1]. It fixes > > incorrect values of cache, cache groups, data region metrics after > > cluster reactivation. > > WDYT? > > > > [1] https://issues.apache.org/jira/browse/IGNITE-14990 > > > > вт, 29 июн. 2021 г. в 15:09, Alexey Gidaspov : > > > > > > Ok, we can add this fix to release scope. > > > > > > On 2021/06/29 08:36:00, Сурков Александр > > > Викторович wrote: > > > > Hi Alexey. > > > > > > > > I think need add ticket: JmxMetricExporter fails to export discovery > > > > metrics - https://issues.apache.org/jira/browse/IGNITE-14376 > > > > > > > > On 2021/06/15 14:43:55, Alexey Gidaspov wrote: > > > > > Apache Ignite 2.11 Code Freeze started now> > > > > > > > > > > > > > -- > > Best wishes, > > Amelchev Nikita > > -- Best wishes, Amelchev Nikita
Re: Ignite 3.0 IgniteTables API Improvement Suggestion
Hi Pavel, 2. Schema and its version are internal things and shouldn't be exposed, otherwise, eventually, it will lead to the user will manage schemas manually on his side for some purposes. This will force us to bother about interfaces/contracts and compatibility aspects in the future with uncertain benefits for a user. As SchemaDescriptor is an internal API and the user expects a public API. I don't think we want to convert the descriptor back into public API terms and it may be impossible for some data. We already have a ticket [1] to support accessing a column by an index and exposing some metadata. Will 'idx -> column' mapping only be enough for your purposes? > int Tuple.columnIndex(String) [1] https://issues.apache.org/jira/browse/IGNITE-14342 On Wed, Jun 30, 2021 at 9:34 AM Pavel Tupitsyn wrote: > Hi Val, > > Thanks for the comments, please see below: > > > > Users will identify tables using names, they will never use IDs > > Ok, let's keep it this way. > > > > Sounds like the Tuple should implement Iterable. > > I don't think Iterable is enough. > We should have a way to get values by column index: Tuple.value(int index), > where index is according to column order in schema. > > Combined with Tuple.schema(), it will provide an efficient way to consume > data - > users will be able to read columns in any order, skip some of them, etc. > > This is a common pattern in APIs like ODBC or System.Data [1], > which we'll need to implement on top of our thin clients later. > > > > However, whether a user might > > need to get a table schema for a particular version, I'm not sure. Do you > > have a use case in mind for this? If not, I would keep this internal > > Ok, we can keep the Table.schema(ver) method internal, as long as > Table.schemas() is public and includes schema versions. > > > > We already have async counterparts for all the methods where this is > applicable > > IgniteTables.createTable(), dropTable(), tables(), table() do not have > async counterparts, > while internally they perform blocking wait on some futures. > > > [1] > > https://docs.microsoft.com/en-us/dotnet/api/system.data.idatareader?view=net-5.0 > > On Wed, Jun 30, 2021 at 4:51 AM Valentin Kulichenko < > valentin.kuliche...@gmail.com> wrote: > > > Hi Pavel, > > > > Please see my comments below. > > > > -Val > > > > On Tue, Jun 29, 2021 at 2:23 PM Pavel Tupitsyn > > wrote: > > > > > Igniters, > > > > > > While working on "IEP-76 Thin Client Protocol for Ignite 3.0" [1] (to > be > > > discussed separately), the following suggestions for the Table API came > > up: > > > > > > 1. Expose table IDs: sending table name with every operation (e.g. GET) > > is > > > inefficient, string serialization is expensive by itself and names can > be > > > long. > > > - Table.id() > > > - IgniteTables.table(UUID) > > > - IgniteTables.dropTable(UUID) > > > > > > > I don't think this should be a part of the public API. Users will > identify > > tables using names, they will never use IDs. As an internal optimization > > though - sure, we can have that. Sounds similar to the cache ID in 2.x. > > > > > > > > > > 2. Expose tuple schemas: to reduce payload size when sending tuples to > > the > > > client, we'll write only the schema version and column values, then the > > > client can retrieve and cache schemas (ordered set of columns per > > version). > > > - Tuple.schema() > > > - Table.schemas() > > > - Table.schema(ver) > > > > > > > Exposing the schema of a tuple makes sense. However, whether a user might > > need to get a table schema for a particular version, I'm not sure. Do you > > have a use case in mind for this? If not, I would keep this internal as > > well. > > > > > > > > > > 3. Expose tuple values as a collection: to serialize tuples efficiently > > > (see p2) we need an API to get all values at once. Right now the only > API > > > is to get values by column name, which involves a HashMap lookup on > every > > > call. > > > - Tuple.values() > > > > > > > Sounds like the Tuple should implement Iterable. > > > > > > > > > > 4. Simplify createTable API: use POJO-based configuration. > > > Creating a Consumer when some properties are optional seems to be > > > non-trivial. > > > > > > > Yes, it's currently convoluted for sure. To my knowledge, there are plans > > to improve this, but I will let other folks chime in with more details. > > > > > > > > > > 5. Add async methods to IgniteTables API (all methods are async inside > > > anyway) > > > > > > > We already have async counterparts for all the methods where this is > > applicable. E.g., for TableView#get, there is TableView#getAsync. Is > there > > anything else that you propose to add? > > > > > > > > > > > > > Thoughts, objections? > > > > > > > > > [1] > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-76+Thin+Client+Protocol+for+Ignite+3.0 > > > > > > [2] > > > > > > > > >
Re: [Announcement] Apache Ignite 2.11 Code Freeze started
Hi, Nikita! I think it's important fix and should be included in 2.11 release. I've tagged ticket by fixVersion 2.11. Can you cherry-pick it to ignite-2.11 branch? And please fill release notes or delete flag. On 2021/06/30 07:55:04, Nikita Amelchev wrote: > Hello, Alexey. > > I suggest adding to the 2.11 scope the resolved issue [1]. It fixes > incorrect values of cache, cache groups, data region metrics after > cluster reactivation. > WDYT? > > [1] https://issues.apache.org/jira/browse/IGNITE-14990 > > вт, 29 июн. 2021 г. в 15:09, Alexey Gidaspov : > > > > Ok, we can add this fix to release scope. > > > > On 2021/06/29 08:36:00, Сурков Александр > > Викторович wrote: > > > Hi Alexey. > > > > > > I think need add ticket: JmxMetricExporter fails to export discovery > > > metrics - https://issues.apache.org/jira/browse/IGNITE-14376 > > > > > > On 2021/06/15 14:43:55, Alexey Gidaspov wrote: > > > > Apache Ignite 2.11 Code Freeze started now> > > > > > > > > -- > Best wishes, > Amelchev Nikita >
Re: [Announcement] Apache Ignite 2.11 Code Freeze started
Hello, Alexey. I suggest adding to the 2.11 scope the resolved issue [1]. It fixes incorrect values of cache, cache groups, data region metrics after cluster reactivation. WDYT? [1] https://issues.apache.org/jira/browse/IGNITE-14990 вт, 29 июн. 2021 г. в 15:09, Alexey Gidaspov : > > Ok, we can add this fix to release scope. > > On 2021/06/29 08:36:00, Сурков Александр > Викторович wrote: > > Hi Alexey. > > > > I think need add ticket: JmxMetricExporter fails to export discovery > > metrics - https://issues.apache.org/jira/browse/IGNITE-14376 > > > > On 2021/06/15 14:43:55, Alexey Gidaspov wrote: > > > Apache Ignite 2.11 Code Freeze started now> > > > -- Best wishes, Amelchev Nikita
Re: Apache Ignite 2.11
Hi, Nikita! Your help with documentation is really needed. Can you describe how we can start with it? On 2021/06/09 10:06:09, Nikita Safonov wrote: > Hello Igniters! > > I guess that AI 2.11 will include documentation work. > I'll be glad to help with this. > And I'll be happy to know the scope of features that we need to document. > > Regards, > Nikita > > пт, 4 июн. 2021 г. в 03:50, 18624049226 <18624049...@163.com>: > > > hello, > > > > can someone can help to merge the following patch into master? > > > > https://issues.apache.org/jira/browse/IGNITE-12852 > > > > This patch has been completed for a long time, and I think it is a > > valuable one. > > > > 在 2021/6/3 下午11:12, Dmitry Pavlov 写道: > > > Sure, thanks for your reply > > > > > > Folks, could you please advice how to setup JIRA account integration for > > non-committers? > > > > > > For the page > > > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.11 > > > I can see all issues, but Alexey's login seems to be not sufficient. Can > > wiki admins set up smth? Or Alexey should do some setup on his own? > > > > > > On 2021/06/03 14:55:25, Maksim Timonin wrote: > > >> I mean I'm OK to release it with 2.12. So, I am glad to hear there are > > >> chances for 2.11, but I guess we can postpone it. > > >> > > >> > > >> On Thu, Jun 3, 2021 at 5:39 PM Dmitry Pavlov > > wrote: > > >> > > >>> ok, Maksim, keep us posted. > > >>> > > >>> We're in the middle of paperwork rigth now, so there is a chance )) > > >>> > > >>> On 2021/06/03 14:30:55, Maksim Timonin > > wrote: > > > Regarding non-SQL index API it really depends on readiness. I guess > > if > > >>> it > > is not in the master, we can skip and release it next time. > > > > Hi, Dmitry! Yes, currently it is on review. I'm OK to release it > > within > > >>> the > > next release. > > > > On Thu, Jun 3, 2021 at 5:26 PM Dmitry Pavlov > > wrote: > > > > > Hi Alexey, > > > > > > Releasing master as-is makes sence for me. > > > > > > Regarding non-SQL index API it really depends on readiness. I guess > > if > > >>> it > > > is not in the master, we can skip and release it next time. > > > > > > We've a bit overdue initial scope freeze date. It's not a big deal, > > but > > > still makes sense to start discussion, create and share wiki page and > > > create branch. > > > > > > Since committer privileges required to create branch, could you > > please > > > notify me in advance. > > > > > > Sincerely, > > > Dmitriy Pavlov > > > > > > On 2021/06/03 14:08:34, �� < > > > olive.c...@gmail.com> wrote: > > >> :scope: > > >> > > >> Here is a list [1] of 195 tickets, i picked for Apache Ignite 2.11 > > > release > > >> Let's start discussion > > >> > > >> [1] > > >>> > > https://issues.apache.org/jira/browse/IGNITE-14781?jql=project%20%3D%20IGNITE%20%20AND%20status%20%20%3D%20Resolved%20and%20fixVersion%20%3D%202.11%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC > > >> On 2021/05/13 14:27:33, Dmitry Pavlov wrote: > > >>> Hi Alexey, > > >>> > > >>> :pmc to sign distribution: > > >>> Formally, only PMC member can be a release manager, so part of > > > activities should be picked up by committer and PMC member. If noone > > >>> else > > > want to help you here, I would. > > >>> :timeline: > > >>> And, could you estimate all phases of the release. > > >>> > > >>> :scope: > > >>> I guess we can just pick up master current state and release it. > > >>> But > > > if someone has some ideas if we should wait for particular feature to > > >>> be > > > completed before branch divergence/release candidate build, please > > let > > >>> know. > > >>> Sincerely, > > >>> Dmitriy Pavlov > > >>> > > >>> On 2021/05/13 09:45:02, Алексей Гидаспов > > > wrote: > > Dear Ignite Community! > > > > I suggest starting Apache Ignite 2.11 release activities. Also > > > suggest > > myself to be a release manager for this release. I plan release > > >>> at > > > the end > > of june 2021. > > > > ___ > > > > Best Regards, > > Alexey Gidaspov > > > > >
Re: Ignite 3.0 IgniteTables API Improvement Suggestion
Hi Val, Thanks for the comments, please see below: > Users will identify tables using names, they will never use IDs Ok, let's keep it this way. > Sounds like the Tuple should implement Iterable. I don't think Iterable is enough. We should have a way to get values by column index: Tuple.value(int index), where index is according to column order in schema. Combined with Tuple.schema(), it will provide an efficient way to consume data - users will be able to read columns in any order, skip some of them, etc. This is a common pattern in APIs like ODBC or System.Data [1], which we'll need to implement on top of our thin clients later. > However, whether a user might > need to get a table schema for a particular version, I'm not sure. Do you > have a use case in mind for this? If not, I would keep this internal Ok, we can keep the Table.schema(ver) method internal, as long as Table.schemas() is public and includes schema versions. > We already have async counterparts for all the methods where this is applicable IgniteTables.createTable(), dropTable(), tables(), table() do not have async counterparts, while internally they perform blocking wait on some futures. [1] https://docs.microsoft.com/en-us/dotnet/api/system.data.idatareader?view=net-5.0 On Wed, Jun 30, 2021 at 4:51 AM Valentin Kulichenko < valentin.kuliche...@gmail.com> wrote: > Hi Pavel, > > Please see my comments below. > > -Val > > On Tue, Jun 29, 2021 at 2:23 PM Pavel Tupitsyn > wrote: > > > Igniters, > > > > While working on "IEP-76 Thin Client Protocol for Ignite 3.0" [1] (to be > > discussed separately), the following suggestions for the Table API came > up: > > > > 1. Expose table IDs: sending table name with every operation (e.g. GET) > is > > inefficient, string serialization is expensive by itself and names can be > > long. > > - Table.id() > > - IgniteTables.table(UUID) > > - IgniteTables.dropTable(UUID) > > > > I don't think this should be a part of the public API. Users will identify > tables using names, they will never use IDs. As an internal optimization > though - sure, we can have that. Sounds similar to the cache ID in 2.x. > > > > > > 2. Expose tuple schemas: to reduce payload size when sending tuples to > the > > client, we'll write only the schema version and column values, then the > > client can retrieve and cache schemas (ordered set of columns per > version). > > - Tuple.schema() > > - Table.schemas() > > - Table.schema(ver) > > > > Exposing the schema of a tuple makes sense. However, whether a user might > need to get a table schema for a particular version, I'm not sure. Do you > have a use case in mind for this? If not, I would keep this internal as > well. > > > > > > 3. Expose tuple values as a collection: to serialize tuples efficiently > > (see p2) we need an API to get all values at once. Right now the only API > > is to get values by column name, which involves a HashMap lookup on every > > call. > > - Tuple.values() > > > > Sounds like the Tuple should implement Iterable. > > > > > > 4. Simplify createTable API: use POJO-based configuration. > > Creating a Consumer when some properties are optional seems to be > > non-trivial. > > > > Yes, it's currently convoluted for sure. To my knowledge, there are plans > to improve this, but I will let other folks chime in with more details. > > > > > > 5. Add async methods to IgniteTables API (all methods are async inside > > anyway) > > > > We already have async counterparts for all the methods where this is > applicable. E.g., for TableView#get, there is TableView#getAsync. Is there > anything else that you propose to add? > > > > > > > > Thoughts, objections? > > > > > > [1] > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-76+Thin+Client+Protocol+for+Ignite+3.0 > > > > [2] > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach > > >
