Re: I want to contribute to Apache Ignite

[Igor Sapego]

Hi,

Added as a contributor to the JIRA project.

Best Regards,
Igor


On Fri, Sep 1, 2023 at 12:55 PM Андрей Хитрин 
wrote:

> Hello everyone!
>
> My name is Andrei Khitrin. I want to contribute to Ignite. I've already
> filled several issues, and now am ready to send small patches. My ASF JIRA
> username is akhitrin.
>
> --
> Andrei Khitrin
>

Re: [ANNOUNCE] Apache Ignite 3.0.0-beta1 is released

[Igor Sapego]

Congrats, guys!

Best Regards,
Igor


On Thu, Nov 17, 2022 at 4:39 PM Вячеслав Коптилин 
wrote:

> Dear Igniters,
>
> I'm happy to announce that the 1st beta version of Ignite 3 is out!
>
> On top of the functionality that was previously released, Beta 5 adds the
> following major features:
> - RPM and DEB packages: simplified installation and node management with
> system services.
> - Client's Partition Awareness: Clients are now aware of data distribution
> over the cluster nodes which helps avoid additional network transmissions
> and lowers operations latency.
> - C++ client:  Basic C++ client, able to perform operations on data.
> - Autogenerated values: now a function can be specified as a default value
> generator during a table creation. Currently only gen_random_uuid is
> supported.
> - SQL Transactions.
> - Transactional Protocol: improved locking model, multi-version based
> lock-free read-only transactions.
> - Storage: A number of improvements to memory-only and on-disk engines
> based on Page Memory.
> - Indexes: Basic functionality, hash and sorted indexes.
> - Client logging: A LoggerFactory may be provided during client creation to
> specify a custom logger for logs generated by the client.
> - Metrics framework.
>
> Code examples have been added for the new features, you can check them out
> https://github.com/apache/ignite-3/tree/ignite-3.0.0-beta1/examples
>
> If there are any questions, issues, or thoughts, please do not hesitate to
> reply to this email. Ignite Community is welcoming any feedback and will
> consider it in future development.
>
> Thanks,
> S.
>

Re: [ANNOUNCE] Apache IGNITE python thin client (pyignite) 0.6.0 have been released

[Igor Sapego]

Great work

Best Regards,
Igor


On Wed, Nov 16, 2022 at 1:50 PM Ivan Daschinsky 
wrote:

> The Apache Ignite Community is pleased to announce the release of
> Apache IGNITE python thin client (pyignite) 0.6.0.
>
> This new release is mostly the maintenance one. However, there are some
> new important features and fixes:
>
> 1. Fixed non-intuitive automatically setting of flag use_ssl when the
> authentication is enabled
> 2. Added timeout support for cache operations in the async client.
> 3. Fixed incorrect result setting of already completed futures in async
> connection implementation
>
> For the full list of changes, you can look at the RELEASE_NOTES:
> https://ignite.apache.org/releases/pyignite/0.6.0/release_notes.html
>
> You can install this version using pip
> >> pip install pyignite=0.6.0
>
> Alternatively, you can download sources and binary packages (wheels) from
> here:
> https://dist.apache.org/repos/dist/release/ignite/pyignite/0.6.0/
>
> Please let us know if you have any problems
> https://ignite.apache.org/community/resources.html#ask
>
> Regards,
> Ivan Daschinsky on behalf of the Apache Ignite community.
>

Re: [VOTE] Release Apache Ignite 3.0.0-beta1 RC2

[Igor Sapego]

+1 (binding)

Best Regards,
Igor


On Tue, Nov 15, 2022 at 11:10 PM Vladislav Pyatkov 
wrote:

> +1
>
> On Tue, Nov 15, 2022 at 3:35 PM Denis C  wrote:
> >
> > +1
> >
> > вт, 15 нояб. 2022 г. в 13:33, Alexander Lapin :
> >
> > > +1
> > >
> > > вт, 15 нояб. 2022 г. в 08:48, Pavel Tupitsyn :
> > >
> > > > +1 (binding)
> > > >
> > > > On Mon, Nov 14, 2022 at 9:05 PM Вячеслав Коптилин <
> > > > slava.kopti...@gmail.com>
> > > > wrote:
> > > >
> > > > > Dear Community,
> > > > >
> > > > > Ignite 3 is moving forward and I think we're in a good spot to
> release
> > > > the
> > > > > first beta version. In the last few months the following major
> features
> > > > > have been added:
> > > > > - RPM and DEB packages: simplified installation and node management
> > > with
> > > > > system services.
> > > > > - Client's Partition Awareness: Clients are now aware of data
> > > > distribution
> > > > > over the cluster nodes which helps avoid additional network
> > > transmissions
> > > > > and lowers operations latency.
> > > > > - C++ client:  Basic C++ client, able to perform operations on
> data.
> > > > > - Autogenerated values: now a function can be specified as a
> default
> > > > value
> > > > > generator during a table creation. Currently only gen_random_uuid
> is
> > > > > supported.
> > > > > - SQL Transactions.
> > > > > - Transactional Protocol: improved locking model, multi-version
> based
> > > > > lock-free read-only transactions.
> > > > > - Storage: A number of improvements to memory-only and on-disk
> engines
> > > > > based on Page Memory.
> > > > > - Indexes: Basic functionality, hash and sorted indexes.
> > > > > - Client logging: A LoggerFactory may be provided during client
> > > creation
> > > > to
> > > > > specify a custom logger for logs generated by the client.
> > > > > - Metrics framework: Collection and export of cluster metrics.
> > > > >
> > > > > I propose to release 3.0.0-beta1 with the features listed above.
> > > > >
> > > > > Release Candidate:
> > > > > https://dist.apache.org/repos/dist/dev/ignite/3.0.0-beta1-rc2/
> > > > > Maven Staging:
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapacheignite-1556/
> > > > > Tag: https://github.com/apache/ignite-3/tree/3.0.0-beta1-rc2
> > > > >
> > > > > +1 - accept Apache Ignite 3.0.0-beta1 RC2
> > > > >  0 - don't care either way
> > > > > -1 - DO NOT accept Apache Ignite 3.0.0-beta1 RC2 (explain why)
> > > > >
> > > > > Voting guidelines: https://www.apache.org/foundation/voting.html
> > > > > How to verify the release:
> > > https://www.apache.org/info/verification.html
> > > > >
> > > > > The vote will be closed on Wednesday, 16 November 2022, 18:00:00
> (UTC
> > > > time)
> > > > >
> > > > >
> > > >
> > >
> https://www.timeanddate.com/countdown/generic?iso=20221116T18=1440=Apache+Ignite+3.0.0-beta1+RC2=cursive=1
> > > > >
> > > > > Thanks,
> > > > > S.
> > > > >
> > > >
> > >
>
>
>
> --
> Vladislav Pyatkov
>

Re: [VOTE] Release pyignite 0.6.0.rc1

[Igor Sapego]

+1

Best Regards,
Igor


On Mon, Nov 14, 2022 at 5:23 PM Ivan Daschinsky 
wrote:

> >>
>
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.6.0.rc0/examples.html
>
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.6.0.rc1/examples.html
>
> пн, 14 нояб. 2022 г. в 16:21, Ivan Daschinsky :
>
> > Dear Igniters!
> >
> > Release candidate binaries for subj are uploaded and ready for vote
> > You can find them here:
> > https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.6.0.rc1
> >
> > If you follow the link above, you will find source packages (*.zip)
> > and binary packages (wheels) for windows (amd64), mac os x (amd64) and
> > linux (x86_64)
> > for pythons 37, 38, 39, 310 and 311. Also, there are sha512 and gpg
> > signatures.
> > Code signing keys can be found here --
> > https://downloads.apache.org/ignite/KEYS
> > Here you can find instructions how to verify packages
> > https://www.apache.org/info/verification.html
> >
> > You can install binary package for specific version of python using pip
> > For example do this on linux for python 3.8
> > >> pip
> > install
> pyignite-0.6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
> >
> > You can build and install package from source using this command:
> > >> pip install pyignite-0.6.0.zip
> > You can build wheel on your platform using this command:
> > >> pip wheel --no-deps pyignite-0.6.0.zip
> >
> > For building C module, you should have python headers and C compiler
> > installed.
> > (i.e. for ubuntu sudo apt install build-essential python3-dev)
> > In Mac OS X xcode-tools and python from homebrew are the best option.
> >
> > In order to check whether C module works, use following:
> > >> from pyignite import _cutils
> > >> print(_cutils.hashcode('test'))
> > >> 3556498
> >
> > You can find documentation here:
> >
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.6.0.rc1/
> >
> > You can find examples here (to check them, you should start ignite
> > locally):
> >
> >
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.6.0.rc0/examples.html
> > Also, examples can be found in source archive in examples subfolder.
> > docker-compose.yml is supplied in order to start ignite quickly. (Use
> > `docker-compose up -d` to start 3 nodes cluster and `docker-compose
> > down` to shut down it)
> >
> > Release notes:
> >
> >
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=blob_plain;f=RELEASE_NOTES.txt;hb=995bda81780402116e89d76523949da88136f260
> >
> > Git release tag was created:
> >
> >
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=tag;h=ea3f180a0300abf25c992ed8c241defdc2b4bd4b
> >
> > The vote is formal, see voting guidelines
> > https://www.apache.org/foundation/voting.html
> >
> > +1 - to accept pyignite-0.6.0.rc0
> > 0 - don't care either way
> > -1 - DO NOT accept pyignite-0.6.0.rc0
> >
> > The vote finishes at 11/15/2022 15:00 UTC
> >
>

Re: [VOTE] Release pyignite 0.6.0.rc0

[Igor Sapego]

+1

Best Regards,
Igor


On Fri, Nov 11, 2022 at 5:41 PM Ivan Daschinsky 
wrote:

> Dear Igniters!
>
> Release candidate binaries for subj are uploaded and ready for vote
> You can find them here:
> https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.6.0.rc0
>
> If you follow the link above, you will find source packages (*.zip)
> and binary packages (wheels) for windows (amd64), mac os x (amd64) and
> linux (x86_64)
> for pythons 37, 38, 39, 310 and 311. Also, there are sha512 and gpg
> signatures.
> Code signing keys can be found here --
> https://downloads.apache.org/ignite/KEYS
> Here you can find instructions how to verify packages
> https://www.apache.org/info/verification.html
>
> You can install binary package for specific version of python using pip
> For example do this on linux for python 3.8
> >> pip
> install
> pyignite-0.6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
>
> You can build and install package from source using this command:
> >> pip install pyignite-0.6.0.zip
> You can build wheel on your platform using this command:
> >> pip wheel --no-deps pyignite-0.6.0.zip
>
> For building C module, you should have python headers and C compiler
> installed.
> (i.e. for ubuntu sudo apt install build-essential python3-dev)
> In Mac OS X xcode-tools and python from homebrew are the best option.
>
> In order to check whether C module works, use following:
> >> from pyignite import _cutils
> >> print(_cutils.hashcode('test'))
> >> 3556498
>
> You can find documentation here:
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.6.0.rc0/
>
> You can find examples here (to check them, you should start ignite
> locally):
>
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.6.0.rc0/examples.html
> Also, examples can be found in source archive in examples subfolder.
> docker-compose.yml is supplied in order to start ignite quickly. (Use
> `docker-compose up -d` to start 3 nodes cluster and `docker-compose
> down` to shut down it)
>
> Release notes:
>
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=blob_plain;f=RELEASE_NOTES.txt;hb=a0600047e7b29fc23350f77d4b087cfb55032d72
>
> Git release tag was created:
>
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=commit;h=a0600047e7b29fc23350f77d4b087cfb55032d72
>
> The vote is formal, see voting guidelines
> https://www.apache.org/foundation/voting.html
>
> +1 - to accept pyignite-0.6.0.rc0
> 0 - don't care either way
> -1 - DO NOT accept pyignite-0.6.0.rc0
>
> The vote finishes at 11/15/2022 15:00 UTC
>

Re: [ANNOUNCE] Release pyignite-0.6.0

[Igor Sapego]

+1

Github Actions look great to me.

Best Regards,
Igor


On Mon, Nov 7, 2022 at 5:38 PM Ivan Daschinsky  wrote:

> Hi, Igniters!
>
> I suppose it is time to release pyignite 0.6.0, since we have released our
> previous release more than a year ago.
>
> Firstly, python 3.6 reached its EOL, the brand new python, 3.11, was just
> released. So it is a good idea to build new wheels targeting the new
> versions.
>
> Also, there are few new features, namely support of timeouts in asyncio
> client for cache operations. This patch also fixes one tiny but annoying
> bug in asyncio client [1]
>
> Full list of tickets for release is here [2]
>
> Actually, I have tested a new approach for testing and building artifacts
> -- Github Actions, nowadays it is a preferred approach in the ASF.
>
> I think that code freeze will be at 15:00 UTC 11/11. And on monday, 11/14 I
> will publish a release candidate for voting.
>
> WDYT?
>
> [1] -- https://issues.apache.org/jira/browse/IGNITE-18006
> [2] --
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%20python-0.6.0
>

Re: Apache Ignite 3.0.0 beta 1 RELEASE [Time, Scope, Manager]

[Igor Sapego]

+1 from me

Best Regards,
Igor


On Wed, Nov 2, 2022 at 3:48 AM Stanislav Lukyanov 
wrote:

> Igniters,
>
> The initial code freeze date for 3.0.0 beta 1 was missed, so we need to
> pick a new timeline.
>
> There are currently 5 tickets in progress or in review that are in the
> scope, with significant progress in each of them.
>
> Let's set the following dates:
>
> Scope Freeze: October 12, 2022
> Code Freeze: November 4, 2022
> Voting Date: November 7, 2022
> Release Date: November 11, 2022
>
> WDYT?
>
> Thanks,
> Stan
>
> > On 13 Oct 2022, at 16:10, Andrey Gura  wrote:
> >
> > Igniters,
> >
> > I removed the "3.0.0-alpha6" version and created the "3.0.0-beta1"
> > version. All issues were rescheduled to the "3.0.0-beta1" version.
> >
> > Despite the fact that formally was on 12th october there is still
> > possibility to add issues to the release scope. Deadline is 14th
> > october. Tomorrow I will announce the scope freeze officially. It
> > means that any issue could be added to the release only after
> > discussion with the community and a release manager.
> >
> > On Mon, Oct 10, 2022 at 7:41 AM Aleksandr Pakhomov 
> wrote:
> >>
> >> +1
> >>
> >>> On 7 Oct 2022, at 23:05, Andrey Gura  wrote:
> >>>
> >>> Hi, Igniters!
> >>>
> >>> It's time for a new release of Apache Ignite 3 beta 1. The expected
> >>> feature list consists of:
> >>>
> >>> - RPM and DEB packages: simplified installation and node management
> >>> with system services.
> >>> - Client's Partition Awareness: Clients are now aware of data
> >>> distribution over the cluster nodes which helps avoid additional
> >>> network transmissions and lowers operations latency.
> >>> - C++ client:  Basic C++ client, able to perform operations on data.
> >>> - Autogenerated values: now a function can be specified as a default
> >>> value generator during a table creation. Currently only
> >>> gen_random_uuid is supported.
> >>> - SQL Transactions.
> >>> - Transactional Protocol: improved locking model, multi-version based
> >>> lock-free read-only transactions.
> >>> - Storage: A number of improvements to memory-only and on-disk engines
> >>> based on Page Memory.
> >>> - Indexes: Basic functionality, hash and sorted indexes.
> >>> - Client logging: A LoggerFactory may be provided during client
> >>> creation to specify a custom logger for logs generated by the client.
> >>> - Metrics framework: Collection and export of cluster metrics.
> >>>
> >>> I want to propose myself to be the release manager of the Apache
> >>> Ignite 3 beta 1.
> >>>
> >>> Also I propose the following milestones for the release:
> >>>
> >>> Scope Freeze: October 12, 2022
> >>> Code Freeze: October 20, 2022
> >>> Voting Date: October 31, 2022
> >>> Release Date: November 5, 2022
> >>>
> >>> WDYT?
> >>
>
>

Re: [ANNOUNCE] SCOPE FREEZE for Apache Ignite 3.0.0 beta 1 RELEASE

[Igor Sapego]

Guys,

I've merged https://issues.apache.org/jira/browse/IGNITE-17590 to main and
I really think it should be included in this release, because without it
current
C++ client implementation is pretty much useless. And it does not affect any
other parts of the product except for the C++ part anyway so there won't be
additional risks for the core of the product.

Best Regards,
Igor


On Tue, Nov 1, 2022 at 2:08 PM Вячеслав Коптилин 
wrote:

> Hello Mikhail,
>
> I agree that these tickets should be included in the release. Let's prepare
> corresponding pull-request(s).
>
> Thanks,
> Slava.
>
>
> пт, 28 окт. 2022 г. в 18:07, Mikhail Pochatkin :
>
> > Hello, I would like to add following tickets to ignite-3.0.0-beta1
> > [IGNITE-17966] Fix problem with stuck Gradle processes in .NET tests -
> ASF
> > JIRA (apache.org) 
> > [IGNITE-17965] Enable remote build cache for Gradle - ASF JIRA (
> apache.org
> > )
> > 
> > [IGNITE-17980] ./gradlew clean build -x test fails - ASF JIRA (
> apache.org)
> > 
> > [IGNITE-18009] Fix gradle build - ASF JIRA (apache.org)
> > 
> >
> > These tickets need to unblock problems with Gradle build and required for
> > packaging scope of beta1. Thanks!
> >
> > On Mon, Oct 24, 2022 at 10:27 PM Mikhail Pochatkin
>  > >
> > wrote:
> >
> > > Hello, Igniters.
> > >
> > > I want to point out that the current beta seems to be blocked by
> > > [IGNITE-17966] .
> The
> > > main problem is that we cannot enable Gradle build on CI at this
> moment,
> > > but we need it because all beta distributions are implemented via
> Gradle
> > > build. So, I am trying to fix it in a short time.
> > >
> > > On Sat, Oct 22, 2022 at 5:35 PM Stanislav Lukyanov <
> > stanlukya...@gmail.com>
> > > wrote:
> > >
> > >> There are 11 unresolved tickets in the scope now, 4 In Progress and 7
> > >> Patch Available.
> > >>
> > >> I think we should try to set the code freeze according to the ticket
> > >> estimates instead of just setting it to the end of next week. I'll
> work
> > >> with each ticket owner to determine the critical path.
> > >>
> > >> I also saw an Open ticket that was added to the scope outside of the
> > >> process. I descoped it already, but we need to be careful of new
> tickets
> > >> being added to the scope.
> > >>
> > >> Thanks,
> > >> Stan
> > >>
> > >> > On 20 Oct 2022, at 15:59, Вячеслав Коптилин <
> slava.kopti...@gmail.com
> > >
> > >> wrote:
> > >> >
> > >> > Hello Alexandr,
> > >> >
> > >> > Ok, I added these tickets to the scope.
> > >> >
> > >> > There are 12 tickets that are not resolved and included into the
> > scope.
> > >> So,
> > >> > we have to move the code freeze to the end of next week.
> > >> >
> > >> > Thanks,
> > >> > Slava.
> > >> >
> > >> > чт, 20 окт. 2022 г. в 08:36, Aleksandr Pakhomov :
> > >> >
> > >> >> Hi, Igniters.
> > >> >>
> > >> >> I would like to ask you to add a couple of tickets that are
> required
> > >> for
> > >> >> packaging:
> > >> >>
> > >> >> https://issues.apache.org/jira/browse/IGNITE-17781 <
> > >> >> https://issues.apache.org/jira/browse/IGNITE-17781>
> > >> >> https://issues.apache.org/jira/browse/IGNITE-17773 <
> > >> >> https://issues.apache.org/jira/browse/IGNITE-17773>
> > >> >>
> > >> >> These tickets are the last tickets in the packaging scope for
> beta1.
> > >> >>
> > >> >> --
> > >> >> Best regards,
> > >> >> Aleksandr
> > >> >>
> > >> >>> On 19 Oct 2022, at 21:48, Вячеслав Коптилин <
> > slava.kopti...@gmail.com
> > >> >
> > >> >> wrote:
> > >> >>>
> > >> >>> Hi Yuriy,
> > >> >>>
> > >> >>> Let's add them. I hope that these three tasks are the last ones.
> > >> >>>
> > >> >>> Thanks,
> > >> >>> S.
> > >> >>>
> > >> >>> ср, 19 окт. 2022 г. в 16:41, Юрий :
> > >> >>>
> > >>  Slava, thank you.
> > >> 
> > >>  During cherry picking one of aforementioned ticket was observed
> > >> >> dependency
> > >>  on
> > >>  https://issues.apache.org/jira/browse/IGNITE-17907
> > >>  https://issues.apache.org/jira/browse/IGNITE-17671
> > >>  https://issues.apache.org/jira/browse/IGNITE-17816
> > >> 
> > >>  So, I propose adding them into the release scope.
> > >> 
> > >>  ср, 19 окт. 2022 г. в 15:53, Вячеслав Коптилин <
> > >> >> slava.kopti...@gmail.com>:
> > >> 
> > >> > Hi Yuriy,
> > >> >
> > >> > I agree, let's add them to the scope.
> > >> >
> > >> > Thanks,
> > >> > S.
> > >> >
> > >> >
> > >> > ср, 19 окт. 2022 г. в 15:20, Юрий  >:
> > >> >
> > >> >> Dear Release managers and Igniters,
> > >> >>
> > >> >> I would like to add the following tickets to Ignite 3.0.0
> beta1:
> > >> >>
> > >> >> https://issues.apache.org/jira/browse/IGNITE-17820 -
> improvement
> > >> SQL
> > >>  and
> > >> 

Re: [DISCUSSION] IEP-95 Client Partition Awareness (Ignite 3)

[Igor Sapego]

Pavel,

Great, this feature showed great results in Ignite 2, so It's a good idea
to implement
it in Ignite 3 as well.

The IEP itself looks good to me, except it's not clear how a server would
know when
assignment has changed.

Regardless of the Tracking Assignment Changes section, I like the first
approach
the best. Yeah, idle clients do not get updates, but why would idle clients
need
assignment update anyway if they are idle?

Best Regards,
Igor


On Thu, Sep 15, 2022 at 11:21 AM Pavel Tupitsyn 
wrote:

> Igniters,
>
> Please review the IEP for client partition awareness in Ignite 3 and let me
> know what you think.
>
> [1]
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-95+Client+Partition+Awareness
> [2] https://github.com/apache/ignite-3/pull/1080
>

Re: IEP-90 Ignite 3 Client Lifecycle

[Igor Sapego]

Pavel, let's postpone developing a sessions part until all the corner cases
are clarified. I'll let you know when new the proposal is ready for
discussion
again.

Best Regards,
Igor


On Mon, May 23, 2022 at 12:25 PM Andrey Gura  wrote:

> Thanks for the answers, Igor.
>
> On Thu, May 19, 2022 at 10:55 PM Igor Sapego  wrote:
> >
> > Andrey,
> >
> > 1. If a server generates a UUID that already exists it can check and just
> > re-generate it straight away
> > as a check is just a simple map lookup.
> >
> > 2. Well, yes. This proposal does not consider monitoring, but with
> current
> > approach it will be definitely
> > easier to implement it properly.
> >
> > Pavel,
> >
> > Wow, that's really cool that you've already started working on it.
> >
> > Regarding your proposal I like it - it will really make a procedure
> easier
> > and faster for a client. I'll change
> > the IEP accordingly.
> >
> > Best Regards,
> > Igor
> >
> >
> > On Thu, May 19, 2022 at 11:40 PM Pavel Tupitsyn 
> > wrote:
> >
> > > Igor,
> > >
> > > I've started working on the initial sessions implementation [1],
> > > and I'd like to clarify the procedure of logical connection restore.
> > >
> > > > client in its turn tries to establish a new *node connection*
> > > > and restore *logical connection *using ConnectionRestoreReq
> > >
> > > This implies either an additional request, or something that replaces
> > > normal handshake.
> > > However, we need to handle two cases here:
> > > - Logical connection is restored
> > > - Logical connection is not restored (timed out, server restarted,
> etc), in
> > > which case we still establish the connection and use it.
> > >
> > > To account for the second case, we should always start with a regular
> > > handshake.
> > > I propose to add a nullable UUID field to the handshake request to
> cover
> > > both cases:
> > > - connectionId is null or not found on server: proceed with normal
> > > handshake, return newly generated connectionId.
> > > - connectionId is not null and found on server: restore logical
> connection,
> > > return the same connectionId
> > >
> > > Client checks if returned connectionId equals to the original and
> > > understands whether logical connection was restored or not.
> > >
> > > Thoughts?
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-16928
> > >
> > > On Thu, May 19, 2022 at 10:27 PM Pavel Tupitsyn 
> > > wrote:
> > >
> > > > Andrey,
> > > >
> > > > > different connections on different client instances theoretically
> > > > > could generate an already existing connection ID
> > > >
> > > > Do you mean an UUID collision?
> > > >
> > > > On Thu, May 19, 2022 at 1:09 AM Andrey Gura 
> wrote:
> > > >
> > > >> Igor,
> > > >>
> > > >> Thanks for the proposal.
> > > >>
> > > >> I understand that such a situation is almost impossible but "if
> > > >> anything can go wrong, it will". Does the protocol take into account
> > > >> that different connections on different client instances
> theoretically
> > > >> could generate an already existing connection ID?
> > > >>
> > > >> Also, do I understand correctly that a server has enough information
> > > >> about client connections so it will be possible to observe a
> > > >> connections list on the server? It would be useful for cluster
> > > >> monitoring purposes.
> > > >>
> > > >> On Tue, May 17, 2022 at 3:11 PM Igor Sapego 
> wrote:
> > > >> >
> > > >> > 1. I've tried to clarify IDs part;
> > > >> > 2. Maybe you are right, though in this case we'd need to use
> > > >> authentication
> > > >> > in ConnectionRestoreReq. Which sounds reasonable to me.
> > > >> >
> > > >> > Best Regards,
> > > >> > Igor
> > > >> >
> > > >> >
> > > >> > On Tue, May 17, 2022 at 10:47 AM Pavel Tupitsyn <
> ptupit...@apache.org
> > > >
> > > >> > wrote:
> > > >> >
> > > >> > > Igor,
> > > >> > >
> > &g

Re: IgniteSet, thin client, and WeakReferenceCloseableIterator

[Igor Sapego]

I like the closable iterator approach more as well. The difference in API
is not as critical in my opinion, as we have a lot of differences in thick
and
thin APIs already and users would normally seek thin client examples and
not just re-use thick client code with thin client.

Best Regards,
Igor


On Fri, Jun 24, 2022 at 7:35 PM Alexander Polovtcev 
wrote:

> I would personally prefer the closeable iterator approach, since it is what
> most Java libraries use and most users should be familiar with it. Also
> most IDEs will warn you, if you don't close an AutoCloseable.
>
> On Fri, Jun 24, 2022 at 6:24 PM Pavel Tupitsyn 
> wrote:
>
> > Igniters,
> >
> > I'm working on IgniteSet in Java thin client [1], and I'm not sure how to
> > deal with iterator release.
> >
> > On the "thick" API side we use WeakReferenceCloseableIterator [2] to
> > release iterator resources automatically when the iterator instance is
> > claimed by GC.
> >
> > a) Reuse this on the client side somehow.
> > Pros: consistent behavior, fool-proof
> > Cons: complexity, nondeterministic resource release
> >
> > b) Make client-side iterator AutoCloseable
> > Pros: simple, deterministic
> > Cons: error-prone for users, not consistent with thick API
> >
> > Thoughts?
> >
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-16897
> > [2]
> >
> >
> https://github.com/apache/ignite/blob/71ff768ac98b3411fee876d42cad9689f1b16c2a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheWeakQueryIteratorsHolder.java#L350
> >
>
>
> --
> With regards,
> Aleksandr Polovtcev
>

Re: [VOTE] Release Apache Ignite 3.0.0-alpha5 RC1

[Igor Sapego]

+1

Best Regards,
Igor


On Fri, Jun 10, 2022 at 1:37 AM Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> +1
>
> On Thu, Jun 9, 2022 at 9:17 AM Alexander Polovtcev <
> alexpolovt...@gmail.com>
> wrote:
>
> > Looks good, so many great features! +1
> >
> > On Thu, Jun 9, 2022 at 6:42 PM Andrey Gura  wrote:
> >
> > > Dear Community,
> > >
> > > In the last few month the following major features have been added:
> > >
> > >   - Pluggable storages: ability to choose a specific storage for a
> > > table (RocksDB based storage, Page memory persistent and in-memory
> > > storage) with some known limitations.
> > >   - Compute API (A simple remote job execution): The first phase of
> > > Compute API design and implementation. Of course, with known
> > > limitations.
> > >   - Data colocation: The colocation key concept replaces the affinity
> > > key concept. DDL introduces COLOCATE BY clause. Colocated job
> > > execution.
> > >   - Open API for the Ignite REST endpoints: A Specification to
> > > generate a client for any language + auto-generated docs for REST API.
> > >   - Ignite REPL: The Ignite CLI as a REPL with autocompletion and
> > improved
> > > UX.
> > >   - Cluster lifecycle: It introduces cluster initialization logic and
> > > allows to specify cluster management and meta storage groups. Improved
> > > node join protocol.
> > >   - Local and distributed recovery: Now it is possible to restart a
> > > cluster/node without data loss.
> > >   - Data rebalance improvements (in progress and could be excluded
> > > from the release), including dynamically changing the number of
> > > partition replicas.
> > >   - Robust client connection with seamless reconnection support and
> > > retry policies.
> > >   - Java API for SQL: A simplified API for executing SQL
> > > queries on a cluster.
> > >
> > > This is an important and interesting milestone and we should share the
> > > current state
> > > with the broader community. I propose to release 3.0.0 Alpha5 with the
> > > features listed above.
> > >
> > > Please vote.
> > >
> > > ---
> > >
> > > Release candidate:
> > > https://dist.apache.org/repos/dist/dev/ignite/3.0.0-alpha5-rc1/
> > > Maven staging:
> > >
> https://repository.apache.org/content/repositories/orgapacheignite-1551/
> > > Git tag: https://github.com/apache/ignite-3/tree/3.0.0-alpha5-rc1
> > >
> > > +1 - accept Apache Ignite 3.0.0-alpha5 RC1
> > >  0 - don't care either way
> > > -1 - DO NOT accept Apache Ignite 3.0.0-alpha5 RC1 (explain why)
> > >
> > > Voting guidelines: https://www.apache.org/foundation/voting.html
> > > How to verify the release:
> https://www.apache.org/info/verification.html
> > >
> > > The vote will be open for 96 hours and will close on June 13th at
> 6:45pm
> > > https://www.timeanddate.com/countdown/to?iso=20220613T1845=352
> > >
> >
> >
> > --
> > With regards,
> > Aleksandr Polovtcev
> >
>

Re: IEP-90 Ignite 3 Client Lifecycle

[Igor Sapego]

Andrey,

1. If a server generates a UUID that already exists it can check and just
re-generate it straight away
as a check is just a simple map lookup.

2. Well, yes. This proposal does not consider monitoring, but with current
approach it will be definitely
easier to implement it properly.

Pavel,

Wow, that's really cool that you've already started working on it.

Regarding your proposal I like it - it will really make a procedure easier
and faster for a client. I'll change
the IEP accordingly.

Best Regards,
Igor


On Thu, May 19, 2022 at 11:40 PM Pavel Tupitsyn 
wrote:

> Igor,
>
> I've started working on the initial sessions implementation [1],
> and I'd like to clarify the procedure of logical connection restore.
>
> > client in its turn tries to establish a new *node connection*
> > and restore *logical connection *using ConnectionRestoreReq
>
> This implies either an additional request, or something that replaces
> normal handshake.
> However, we need to handle two cases here:
> - Logical connection is restored
> - Logical connection is not restored (timed out, server restarted, etc), in
> which case we still establish the connection and use it.
>
> To account for the second case, we should always start with a regular
> handshake.
> I propose to add a nullable UUID field to the handshake request to cover
> both cases:
> - connectionId is null or not found on server: proceed with normal
> handshake, return newly generated connectionId.
> - connectionId is not null and found on server: restore logical connection,
> return the same connectionId
>
> Client checks if returned connectionId equals to the original and
> understands whether logical connection was restored or not.
>
> Thoughts?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-16928
>
> On Thu, May 19, 2022 at 10:27 PM Pavel Tupitsyn 
> wrote:
>
> > Andrey,
> >
> > > different connections on different client instances theoretically
> > > could generate an already existing connection ID
> >
> > Do you mean an UUID collision?
> >
> > On Thu, May 19, 2022 at 1:09 AM Andrey Gura  wrote:
> >
> >> Igor,
> >>
> >> Thanks for the proposal.
> >>
> >> I understand that such a situation is almost impossible but "if
> >> anything can go wrong, it will". Does the protocol take into account
> >> that different connections on different client instances theoretically
> >> could generate an already existing connection ID?
> >>
> >> Also, do I understand correctly that a server has enough information
> >> about client connections so it will be possible to observe a
> >> connections list on the server? It would be useful for cluster
> >> monitoring purposes.
> >>
> >> On Tue, May 17, 2022 at 3:11 PM Igor Sapego  wrote:
> >> >
> >> > 1. I've tried to clarify IDs part;
> >> > 2. Maybe you are right, though in this case we'd need to use
> >> authentication
> >> > in ConnectionRestoreReq. Which sounds reasonable to me.
> >> >
> >> > Best Regards,
> >> > Igor
> >> >
> >> >
> >> > On Tue, May 17, 2022 at 10:47 AM Pavel Tupitsyn  >
> >> > wrote:
> >> >
> >> > > Igor,
> >> > >
> >> > > The proposal looks good to me. Very detailed!
> >> > >
> >> > > A couple comments:
> >> > >
> >> > > 1. There is a bit of a term mixup with "Connection ID", "Node ID",
> >> "Token"
> >> > > - can you please review those?
> >> > >
> >> > > 2. > The Connection ID should be generated using a proper secure
> >> algorithm
> >> > > (additional research is required here) to make sure an intruder can
> >> not
> >> > > generate an existing Connection ID
> >> > > Not sure about the reasoning here. I think randomUUID() should be
> >> enough:
> >> > > - In the case of an unsecured cluster it does not matter, because
> >> anyone
> >> > > can do anything.
> >> > > - In the case of a secured cluster it does not matter, because
> >> > > authentication/authorization keeps intruders out.
> >> > >
> >> > >
> >> > > On Mon, May 16, 2022 at 11:07 PM Igor Sapego 
> >> wrote:
> >> > >
> >> > > > Hi, Igniters
> >> > > >
> >> > > > I've prepared an IEP for Ignite 3 Client Lifecycle [1]. The main
> >> idea is
> >> > > to
> >> > > > define client lifecycle as well as core algorithms and mechanisms
> >> used by
> >> > > > clients. This proposal can be used as a reference for
> >> implementation of a
> >> > > > new client for Ignite when dealing with such problems as:
> >> > > >
> >> > > >- Resolving of user-provided addresses;
> >> > > >- Initial connection to a cluster;
> >> > > >- Maintaining cluster connection;
> >> > > >- Connection recovery;
> >> > > >- Connection break handling.
> >> > > >
> >> > > > So take a look and let me know what you think guys.
> >> > > >
> >> > > > [1] -
> >> > > >
> >> > >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-90+Client+Lifecycle
> >> > > >
> >> > > > Best Regards,
> >> > > > Igor
> >> > > >
> >> > >
> >>
> >
>

Re: IEP-90 Ignite 3 Client Lifecycle

[Igor Sapego]

1. I've tried to clarify IDs part;
2. Maybe you are right, though in this case we'd need to use authentication
in ConnectionRestoreReq. Which sounds reasonable to me.

Best Regards,
Igor


On Tue, May 17, 2022 at 10:47 AM Pavel Tupitsyn 
wrote:

> Igor,
>
> The proposal looks good to me. Very detailed!
>
> A couple comments:
>
> 1. There is a bit of a term mixup with "Connection ID", "Node ID", "Token"
> - can you please review those?
>
> 2. > The Connection ID should be generated using a proper secure algorithm
> (additional research is required here) to make sure an intruder can not
> generate an existing Connection ID
> Not sure about the reasoning here. I think randomUUID() should be enough:
> - In the case of an unsecured cluster it does not matter, because anyone
> can do anything.
> - In the case of a secured cluster it does not matter, because
> authentication/authorization keeps intruders out.
>
>
> On Mon, May 16, 2022 at 11:07 PM Igor Sapego  wrote:
>
> > Hi, Igniters
> >
> > I've prepared an IEP for Ignite 3 Client Lifecycle [1]. The main idea is
> to
> > define client lifecycle as well as core algorithms and mechanisms used by
> > clients. This proposal can be used as a reference for implementation of a
> > new client for Ignite when dealing with such problems as:
> >
> >- Resolving of user-provided addresses;
> >- Initial connection to a cluster;
> >- Maintaining cluster connection;
> >- Connection recovery;
> >- Connection break handling.
> >
> > So take a look and let me know what you think guys.
> >
> > [1] -
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-90+Client+Lifecycle
> >
> > Best Regards,
> > Igor
> >
>

IEP-90 Ignite 3 Client Lifecycle

[Igor Sapego]

Hi, Igniters

I've prepared an IEP for Ignite 3 Client Lifecycle [1]. The main idea is to
define client lifecycle as well as core algorithms and mechanisms used by
clients. This proposal can be used as a reference for implementation of a
new client for Ignite when dealing with such problems as:

   - Resolving of user-provided addresses;
   - Initial connection to a cluster;
   - Maintaining cluster connection;
   - Connection recovery;
   - Connection break handling.

So take a look and let me know what you think guys.

[1] -
https://cwiki.apache.org/confluence/display/IGNITE/IEP-90+Client+Lifecycle

Best Regards,
Igor

Re: IEP-83 Thin Client Keepalive (heartbeat)

[Igor Sapego]

Feature seems useful for me as it makes connection management more robust
and
predictable.

I agree with Pavel, that we should print warning when heartbeat period is
larger than
idle timeout, but I see a problem here as idle timeout is configured on
server and is not
known to clients, while heartbeats configured on clients and their period
is not known
to the server. Maybe clients should pass this information on to the
handshake.

Regarding Python and PHP clients - can not we use some kind of timers to
implement
this feature?

Best Regards,
Igor


On Mon, Feb 7, 2022 at 5:23 PM Pavel Tupitsyn  wrote:

> Maksim, agree. Let's not be too clever and only log a warning.
>
> On Mon, Feb 7, 2022 at 5:23 PM Pavel Tupitsyn 
> wrote:
>
> > Ivan, idleTimeout already exists, I don't think we should change the way
> > it works (or did I misunderstand you?)
> >
> > Of course, enabling heartbeats means that otherwise idle clients will no
> > longer be disconnected by the server.
> > I think we should cross-link those properties in the documentation and
> > explain this behavior.
> >
> > On Mon, Feb 7, 2022 at 4:39 PM Ivan Daschinsky 
> > wrote:
> >
> >> >>3. Already implemented: when ClientConnectorConfiguration#idleTimeout
> is
> >> not zero, server disconnects idle clients
> >> >>
> >> But I suppose it would be great to have:
> >> 1. If client supports keep alive, use idleTimeout
> >> 2. If not, do not use it.
> >>
> >> But I am not sure if it is correct or not.
> >>
> >> пн, 7 февр. 2022 г. в 16:01, Maksim Timonin :
> >>
> >> > I believe explicit is better than implicit :) Also in case of dynamic
> >> > calculation of timeout, it can change dynamically, for example
> >> restarting a
> >> > cluster with different configuration should reconfigure clients too.
> >> Looks
> >> > complicated.
> >> >
> >> > My vote for WARN + javadocs with mention of this issue.
> >> >
> >> > On Mon, Feb 7, 2022 at 3:51 PM Pavel Tupitsyn 
> >> > wrote:
> >> >
> >> > > > WDYT, should we add a WARN message for clients that configure
> >> > > > keepAliveTimeout greater than idleTimeout on the server side?
> >> > >
> >> > > I think we should either log a WARN, or retrieve idleTimeout from
> >> server
> >> > > and configure heartbeatTimeout accordingly (e.g. divide by 2).
> >> > > Thoughts?
> >> > >
> >> > > On Mon, Feb 7, 2022 at 3:14 PM Maksim Timonin <
> >> timoninma...@apache.org>
> >> > > wrote:
> >> > >
> >> > > > Hi Pavel,
> >> > > >
> >> > > > Thanks for the links. Yes, I forgot that the flag of changed
> >> topology
> >> > is
> >> > > > lazy. Also I missed that the keepAlive setting is configured on
> the
> >> > > client
> >> > > > side (alternatively to idleTimeout that is on the server side).
> >> > > >
> >> > > > Now I understand, this feature can be helpful then. Every client
> can
> >> > > > configure itself in case it's possible to be idle sometimes, and
> >> choose
> >> > > > an appropriate timeout by itself too. And by default the feature
> >> should
> >> > > be
> >> > > > disabled.
> >> > > >
> >> > > > WDYT, should we add a WARN message for clients that configure
> >> > > > keepAliveTimeout greater than idleTimeout on the server side?
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Mon, Feb 7, 2022 at 1:05 PM Pavel Tupitsyn <
> ptupit...@apache.org
> >> >
> >> > > > wrote:
> >> > > >
> >> > > > > Ivan,
> >> > > > >
> >> > > > > I suggest the following:
> >> > > > >
> >> > > > > 1. Server sends KEEP_ALIVE feature flag, which means it accepts
> >> > > > > OP_KEEP_ALIVE empty message
> >> > > > > 2. Client sends OP_KEEP_ALIVE when the connection is idle for a
> >> > > > > certain period of time
> >> > > > > 3. Already implemented: when
> >> ClientConnectorConfiguration#idleTimeout
> >> > > is
> >> > > > > not zero, server disconnects idle clients
> >> > > > >
> >> > > > > This way we don't need server->client keepalives, as you
> correctly
> >> > > noted.
> >> > > > >
> >> > > > > On Mon, Feb 7, 2022 at 12:43 PM Ivan Daschinsky <
> >> ivanda...@gmail.com
> >> > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Pavel, I suppose that ideally:
> >> > > > > > 1. Client send in handshake flag, that it supports KEEP_ALIVE
> >> > feature
> >> > > > and
> >> > > > > > server takes it into account.
> >> > > > > > 2. Each request of client can be considered as keep-alive
> ping.
> >> > > > > > 3. Client send failure should be processed using retry policy.
> >> > > > > > 4. Server should not send keep-alive packets, it is redundant,
> >> but
> >> > > > server
> >> > > > > > should track requests from client and if there is no requests
> >> from
> >> > > > client
> >> > > > > > with KEEP_ALIVE feature,
> >> > > > > > automatically close connection and free resources.
> >> > > > > >
> >> > > > > > Similar approach is used in zookeeper clients.
> >> > > > > >
> >> > > > > > пн, 7 февр. 2022 г. в 12:24, Pavel Tupitsyn <
> >> ptupit...@apache.org
> >> > >:
> >> > > > > >
> >> > > > > > > Ivan,
> >> > > > > > >
> >> > > > > > > Ideally, the check 

Re: [VOTE] Release Apache Ignite 3.0.0-alpha4 RC1

[Igor Sapego]

+1 (binding)

Best Regards,
Igor


On Tue, Jan 25, 2022 at 10:44 PM Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> +1 (binding)
>
> On Tue, Jan 25, 2022 at 11:43 AM Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > Dear Community,
> >
> > Ignite 3 is moving forward and I think we're in a good spot to release
> > another alpha version. In the last few month the following major features
> > have been added:
> >
> >- Transactional API
> >- Record and KeyValue views with POJO mapping support
> >- DDL
> >
> > This is a significant milestone, so I think it will be valuable to share
> > the current state with the broader community. I propose to release 3.0.0
> > Alpha4 with the features listed above.
> >
> > Please vote.
> >
> > ---
> >
> > Release candidate:
> > https://dist.apache.org/repos/dist/dev/ignite/3.0.0-alpha4-rc1/
> > Maven staging:
> > https://repository.apache.org/content/repositories/orgapacheignite-1540/
> > Git tag: https://github.com/apache/ignite-3/tree/3.0.0-alpha4-rc1
> >
> > +1 - accept Apache Ignite 3.0.0-alpha4 RC1
> >  0 - don't care either way
> > -1 - DO NOT accept Apache Ignite 3.0.0-alpha4 RC1 (explain why)
> >
> > Voting guidelines: https://www.apache.org/foundation/voting.html
> > How to verify the release: https://www.apache.org/info/verification.html
> >
> > The vote will be open for 72 hours and will close on January 28th at 8pm
> > UTC: https://www.timeanddate.com/countdown/to?iso=20220128T12=224
> >
> > -Val
> >
>

Re: IEP-82 Thin Client Retry Policy

[Igor Sapego]

Pavel,

What is ClientOperationType? Will it list basically all operations or only
types like Idempotent, NonIdempotent?

Best Regards,
Igor


On Wed, Nov 24, 2021 at 5:21 PM Pavel Tupitsyn  wrote:

> Igniters,
>
> I've prepared a proposal about thin client retry behavior.
> Please review and let me know your thoughts.
>
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-82+Thin+Client+Retry+Policy
>

Re: [VOTE] Release Apache Ignite 3.0.0-alpha3 RC1

[Igor Sapego]

Sorry,

I meant we need to publish the package as part of RC, so it can be reviewed.

Best Regards,
Igor


On Thu, Oct 14, 2021 at 11:34 AM Igor Sapego  wrote:

> Val,
>
> I think we need to upload the nuget package we want to upload so the
> community
> would know what we are going to upload and can check that everything is
> right.
>
> WDYT?
>
> Best Regards,
> Igor
>
>
> On Wed, Oct 13, 2021 at 8:03 PM Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
>> Pavel,
>>
>> You've mentioned in the ticket that "Note that NuGet, unfortunately, has
>> no
>> concept of "staging" (unlike Maven). A package with the given version can
>> be published only once, and it can't be undone. We can only publish the
>> packages after the successful vote."
>>
>> With that, will you be okay if we proceed with the release, and upload the
>> NuGet package after the vote is accepted?
>>
>> We can then have a separate discussion on the overall packaging approach.
>>
>> -Val
>>
>> On Wed, Oct 13, 2021 at 9:57 AM Valentin Kulichenko <
>> valentin.kuliche...@gmail.com> wrote:
>>
>> > Hi Pavel,
>> >
>> > That's actually by design. The current packaging model assumes that we
>> use
>> > Maven/NuGet to deliver binaries - both for servers side and client
>> side. In
>> > case you have any objections to the overall approach, we surely can
>> have a
>> > discussion, but I propose we do this separately after the release.
>> >
>> > However, it's great that you pointed this out as we indeed don't have
>> the
>> > .NET package deployed. I've created a blocker ticket for this:
>> > https://issues.apache.org/jira/browse/IGNITE-15741
>> >
>> > Java client is deployed to Maven staging and examples are
>> > fully-functional, so we are good on that part.
>> >
>> > -Val
>> >
>> > On Wed, Oct 13, 2021 at 9:11 AM Pavel Tupitsyn 
>> > wrote:
>> >
>> >> To clarify, Java thin client has the following features:
>> >> - Table API
>> >> - Key-value API
>> >> - JDBC driver
>> >>
>> >> .NET thin client:
>> >> - Table API
>> >>
>> >> I think all of this should be included in the binary release.
>> >>
>> >> On Wed, Oct 13, 2021 at 10:10 AM Pavel Tupitsyn 
>> >> wrote:
>> >>
>> >> > -1 (binding)
>> >> >
>> >> > Java thin client and .NET thin client are missing from the binary
>> >> package.
>> >> >
>> >> >
>> >> > On Wed, Oct 13, 2021 at 3:03 AM Saikat Maitra <
>> saikat.mai...@gmail.com>
>> >> > wrote:
>> >> >
>> >> >> +1 (binding)
>> >> >>
>> >> >> Regards
>> >> >> Saikat
>> >> >>
>> >> >> On Tue, Oct 12, 2021 at 5:03 PM Denis Magda 
>> wrote:
>> >> >>
>> >> >> > +1 (binding)
>> >> >> >
>> >> >> > -
>> >> >> > Denis
>> >> >> >
>> >> >> >
>> >> >> > On Tue, Oct 12, 2021 at 6:01 PM Valentin Kulichenko <
>> >> >> > valentin.kuliche...@gmail.com> wrote:
>> >> >> >
>> >> >> > > Dear Community,
>> >> >> > >
>> >> >> > > Ignite 3 development is moving forward. In the last several
>> months
>> >> >> we've
>> >> >> > > added the following features:
>> >> >> > >
>> >> >> > >- New SQL engine based on Apache Calcite + JDBC driver.
>> >> >> > >- Persistence implementation based on RocksDB.
>> >> >> > >- New binary client protocol with an implementation in Java.
>> >> >> > >- Data rebalancing.
>> >> >> > >
>> >> >> > > This is a significant milestone, so I think it will be valuable
>> to
>> >> >> share
>> >> >> > > the current state with the broader community.
>> >> >> > >
>> >> >> > > I propose to release 3.0.0 Alpha3 with the features listed
>> above.
>> >> >> > >
>> >> >> > > Please vote.
>> >> >> > >
>> >> >> > > ---
>> >> >> > >
>> >> >> > > Release candidate:
>> >> >> > > https://dist.apache.org/repos/dist/dev/ignite/3.0.0-alpha3-rc1/
>> >> >> > > Maven staging:
>> >> >> > >
>> >> >>
>> >>
>> https://repository.apache.org/content/repositories/orgapacheignite-1532
>> >> >> > > Git tag:
>> https://github.com/apache/ignite-3/tree/3.0.0-alpha3-rc1
>> >> >> > >
>> >> >> > > +1 - accept Apache Ignite 3.0.0-alpha3 RC1
>> >> >> > >  0 - don't care either way
>> >> >> > > -1 - DO NOT accept Apache Ignite 3.0.0-alpha3 RC1 (explain why)
>> >> >> > >
>> >> >> > > Voting guidelines:
>> https://www.apache.org/foundation/voting.html
>> >> >> > > How to verify the release:
>> >> >> https://www.apache.org/info/verification.html
>> >> >> > >
>> >> >> > > The vote will be open for 72 hours and will close on October
>> 15th
>> >> at
>> >> >> 10pm
>> >> >> > > UTC:
>> >> >> > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> https://www.timeanddate.com/counters/fullscreen.html?mode=a=20211015T15=2021=10=15=15=0=0=224
>> >> >> > >
>> >> >> > > -Val
>> >> >> > >
>> >> >> >
>> >> >>
>> >> >
>> >>
>> >
>>
>

Re: [VOTE] Release Apache Ignite 3.0.0-alpha3 RC1

[Igor Sapego]

Val,

I think we need to upload the nuget package we want to upload so the
community
would know what we are going to upload and can check that everything is
right.

WDYT?

Best Regards,
Igor


On Wed, Oct 13, 2021 at 8:03 PM Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Pavel,
>
> You've mentioned in the ticket that "Note that NuGet, unfortunately, has no
> concept of "staging" (unlike Maven). A package with the given version can
> be published only once, and it can't be undone. We can only publish the
> packages after the successful vote."
>
> With that, will you be okay if we proceed with the release, and upload the
> NuGet package after the vote is accepted?
>
> We can then have a separate discussion on the overall packaging approach.
>
> -Val
>
> On Wed, Oct 13, 2021 at 9:57 AM Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > Hi Pavel,
> >
> > That's actually by design. The current packaging model assumes that we
> use
> > Maven/NuGet to deliver binaries - both for servers side and client side.
> In
> > case you have any objections to the overall approach, we surely can have
> a
> > discussion, but I propose we do this separately after the release.
> >
> > However, it's great that you pointed this out as we indeed don't have the
> > .NET package deployed. I've created a blocker ticket for this:
> > https://issues.apache.org/jira/browse/IGNITE-15741
> >
> > Java client is deployed to Maven staging and examples are
> > fully-functional, so we are good on that part.
> >
> > -Val
> >
> > On Wed, Oct 13, 2021 at 9:11 AM Pavel Tupitsyn 
> > wrote:
> >
> >> To clarify, Java thin client has the following features:
> >> - Table API
> >> - Key-value API
> >> - JDBC driver
> >>
> >> .NET thin client:
> >> - Table API
> >>
> >> I think all of this should be included in the binary release.
> >>
> >> On Wed, Oct 13, 2021 at 10:10 AM Pavel Tupitsyn 
> >> wrote:
> >>
> >> > -1 (binding)
> >> >
> >> > Java thin client and .NET thin client are missing from the binary
> >> package.
> >> >
> >> >
> >> > On Wed, Oct 13, 2021 at 3:03 AM Saikat Maitra <
> saikat.mai...@gmail.com>
> >> > wrote:
> >> >
> >> >> +1 (binding)
> >> >>
> >> >> Regards
> >> >> Saikat
> >> >>
> >> >> On Tue, Oct 12, 2021 at 5:03 PM Denis Magda 
> wrote:
> >> >>
> >> >> > +1 (binding)
> >> >> >
> >> >> > -
> >> >> > Denis
> >> >> >
> >> >> >
> >> >> > On Tue, Oct 12, 2021 at 6:01 PM Valentin Kulichenko <
> >> >> > valentin.kuliche...@gmail.com> wrote:
> >> >> >
> >> >> > > Dear Community,
> >> >> > >
> >> >> > > Ignite 3 development is moving forward. In the last several
> months
> >> >> we've
> >> >> > > added the following features:
> >> >> > >
> >> >> > >- New SQL engine based on Apache Calcite + JDBC driver.
> >> >> > >- Persistence implementation based on RocksDB.
> >> >> > >- New binary client protocol with an implementation in Java.
> >> >> > >- Data rebalancing.
> >> >> > >
> >> >> > > This is a significant milestone, so I think it will be valuable
> to
> >> >> share
> >> >> > > the current state with the broader community.
> >> >> > >
> >> >> > > I propose to release 3.0.0 Alpha3 with the features listed above.
> >> >> > >
> >> >> > > Please vote.
> >> >> > >
> >> >> > > ---
> >> >> > >
> >> >> > > Release candidate:
> >> >> > > https://dist.apache.org/repos/dist/dev/ignite/3.0.0-alpha3-rc1/
> >> >> > > Maven staging:
> >> >> > >
> >> >>
> >> https://repository.apache.org/content/repositories/orgapacheignite-1532
> >> >> > > Git tag:
> https://github.com/apache/ignite-3/tree/3.0.0-alpha3-rc1
> >> >> > >
> >> >> > > +1 - accept Apache Ignite 3.0.0-alpha3 RC1
> >> >> > >  0 - don't care either way
> >> >> > > -1 - DO NOT accept Apache Ignite 3.0.0-alpha3 RC1 (explain why)
> >> >> > >
> >> >> > > Voting guidelines: https://www.apache.org/foundation/voting.html
> >> >> > > How to verify the release:
> >> >> https://www.apache.org/info/verification.html
> >> >> > >
> >> >> > > The vote will be open for 72 hours and will close on October 15th
> >> at
> >> >> 10pm
> >> >> > > UTC:
> >> >> > >
> >> >> > >
> >> >> >
> >> >>
> >>
> https://www.timeanddate.com/counters/fullscreen.html?mode=a=20211015T15=2021=10=15=15=0=0=224
> >> >> > >
> >> >> > > -Val
> >> >> > >
> >> >> >
> >> >>
> >> >
> >>
> >
>

Re: Updating Javascript npm Package

[Igor Sapego]

Kevin,

Basically, to change this we need people who would actively drive
development of the client and be active community members.

Best Regards,
Igor


On Mon, Sep 27, 2021 at 6:02 PM Ivan Daschinsky  wrote:

> Hi! I can share my experience how to drive this activity. Personally, I've
> driven four recent releases of python thin client (pyignite). First of all,
> you should start a discussion, this thread is a good place to start.
> Secondly, you should prepare release branch (with help of commiters),
> create realease notes, etc. Secondly, PMC member should prepare rc
> artifact, sign with signature, theb publish it in apache svn for voting.
> After successful voting, release artifact can be published to npmjs.
>
> Let's start this activity!
>
> пн, 27 сент. 2021 г., 18:52 Kevin Corbett :
>
>> Hello Igniters!
>>
>> I’ve been helping with the Ignite social media channels for the last
>> month now. Lately I’ve been trying to increase awareness of our Javascript
>> capabilities, namely the thin client. Someone on our Twitter page had
>> mentioned the npm package hasn’t been updated in 3 years (!)
>>
>> I see our Github for the thin client was last updated in January this
>> year. What can we do to get this updated?
>> https://www.npmjs.com/package/apache-ignite-client <
>> https://www.npmjs.com/package/apache-ignite-client>
>>
>> Also, I saw there was an attempt to add Apache Ignite support to TypeORM
>> but there were many issues with the client.
>> https://github.com/typeorm/typeorm/pull/7012
>
>

Re: [DISCUSS] Custom service proxy context

[Igor Sapego]

Hi guys,

Why can not a user implement such context on application level?
I believe Ignite provides all necessary tools for that. User can just
implement such a context as user type and pass it to services they
need. Are the arguments why would Ignite need a separate feature
for such a use case?

Best Regards,
Igor


On Fri, Oct 8, 2021 at 2:17 PM Eduard Rakhmankulov 
wrote:

> I am not aware .NET capabilities, but as I can see service must be
> implemented in *java* and even if can't serialize other that Map on .NET
> side, on java side we can wrap this map with provided TypedContext (context
> should be convertible from map in this case).
> That leads to a situation when Java can use TypedContext but other clients
> can't. I believe that the majority of services users are using Java and it
> should be taken in accordance.
>
> P.S. I think it is possible to send plain objects from .NET context to
> cluster.
>
> Best regards, Ed
>
> On Fri, 8 Oct 2021 at 14:40, Pavel Pereslegin  wrote:
>
> > Hi, Eduard!
> >
> > Thanks for your feedback.
> >
> > The idea sounds very good, but don't forget about the platform services.
> > For example, we may call Java service from .Net and vice-versa. I'm
> > not sure if the context can be implemented as a custom class (instead
> > of Map/Dictionary) in this case.
> >
> > пт, 8 окт. 2021 г. в 14:21, Eduard Rakhmankulov :
> > >
> > > Hi, Pavel
> > >
> > > Is it possible to provide type-safe API for ServiceProxyContext ?
> > > I think constructions like int arg1 = ctx.attribute("arg1"); are error
> > > prone.
> > >
> > > Can we make something like this :
> > >
> > > //Signature with two generic params which allow the compiler to check
> > > if the service will be called with the wrong type context.
> > >
> > > public , CtxType> T
> > > serviceProxyTyped(ClusterGroup prj, String name, Class
> > > srvcCls, CtxType optCtx, boolean sticky, long timeout)
> > >
> > > //new interface which services with scoped context should implement
> > >
> > > public interface ContextedWith {
> > > T getCtx();
> > > }
> > >
> > > // implementation can delegate to Map-like context or be POJO.
> > > interface MyServiceContext {
> > > int getArg1();
> > > String getUserId();
> > > }
> > >
> > > class MyService implements ContextedWith {
> > > void doThings() {
> > > MyServiceContext ctx = getCtx();
> > >
> > > System.out.println("ctx.getArg1() = " + ctx.getArg1());
> > > }
> > >
> > > @Override public MyServiceContext getCtx() {
> > > return ServiceProxyContext.current();
> > > }
> > > }
> > >
> > > WDYT?
> > >
> > > Best regards, Ed.
> > >
> > > On Fri, 8 Oct 2021 at 13:26, Pavel Pereslegin 
> wrote:
> > >
> > > > Hello Igniters!
> > > >
> > > > I want to implement a feature to support a custom "caller" context in
> > > > ignite services (see example in ticket description [1]).
> > > >
> > > > Sometimes, when using Ignite services, it becomes necessary to pass
> > > > custom parameters from the "request source" to the service. This is
> > > > most commonly used to track the origin of a service call (user id,
> > > > request id, session id eg see this user question [2]).
> > > > At the moment, the only way to pass such parameters to a service is
> by
> > > > adding argument(s) to all called methods of the service, which makes
> > > > the code messy and also complicates development and maintenance.
> > > >
> > > > I propose letting the user set a custom context for the service proxy
> > > > and implicitly pass that context to the methods being called. This
> > > > function should not affect the execution of service methods in any
> way
> > > > unless the user has specified a context.
> > > >
> > > > An example of using the proposed API [1].
> > > > PoC (except thin clients) [3].
> > > >
> > > > WDYT?
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-15572
> > > > [2]
> > > >
> >
> https://stackoverflow.com/questions/57459071/apache-ignite-service-grid-service-call-context
> > > > [3] https://github.com/apache/ignite/pull/9440
> > > >
> >
>

Re: API Proposal: Declare IgniteClient::close that throws no exceptions (IGNITE-15688)

[Igor Sapego]

Sounds good, no objections from my side.

Best Regards,
Igor


On Wed, Oct 6, 2021 at 11:46 AM Stanislav Lukyanov 
wrote:

> Hi Igniters,
>
> I found the following usability issue with java thin client API.
>
> Whenever you do `try (IgniteClient client = Ignition.startClient(cfg))`,
> you're forced to declare `catch (Exception e)`.
>
> This is because IgniteClient interface currently doesn't override close()
> from AutoClosable. Because of that, it inherits `close() throws Exception`.
>
> In fact, this shouldn't be required. `TcpIgniteClient implements
> IgniteClient` currently throws Exception but it doesn't need to - its code
> doesn't throw any checked exceptions.
>
> Proposal:
> • Add `@Overrides public void close()` with no `throws` to
> IgniteClient.
> • Remove `throws Exception` from `TcpIgniteClient::close`
> Note: this change is fully backwards-compatible. It is legal to narrow the
> scope of `throws` in a new version of a method.
>
> I plan to do this in https://issues.apache.org/jira/browse/IGNITE-15688.
>
> Any objections?
>
> Thanks,
> Stan

Re: Tuple equality in Ignite 3.x

[Igor Sapego]

Sounds very reasonable to me.

+1

Though the default comparator should be implemented very carefully
as we had issues with comparison of binary objects in 2.x

Best Regards,
Igor


On Thu, Sep 9, 2021 at 4:04 PM Pavel Tupitsyn  wrote:

> Igniters,
>
> Tuple in Ignite 3.x is a replacement for BinaryObject in Ignite 2.x.
> Let's discuss equality and sorting.
>
> - We have multiple Tuple implementations, and our API allows custom,
> user-defined Tuples as well (which can be useful for performance when
> bridging Ignite with another system or importing the data from somewhere).
> - We don't have equals()/hashCode() overrides, so it is not possible to
> compare tuples or put them into a Map.
>
> Proposal:
> - Add public TupleComparator implements Comparator, based on the
> tuple contents (column names and values)
> - Implement common TupleComparator#hashCode(Tuple t) method that combines
> hash codes of column names and values
> - Implement equals(), hashCode(), and Comparable on all built-in tuples,
> delegate the logic to TupleComparator
> - Make Tuple extend Compable
>
> This way we cover all sorting/comparing/mapping scenarios for built-in
> tuples and provide reusable code for third-party implementations.
>
> Thoughts?
>

Re: Replace Map with List and Iterable in KeyValueView Ignite 3 APIs

[Igor Sapego]

I actually agree with Pavel, at least at putAll() part. We require a Map
from user
when we do not really need a Map in this method. What we really need here is
an iterable collection of pairs. Can not see why user can not pass for
example an
array here.

Now, when we talk about getAll() method it's more complicated, as in many
cases
I think a user would expect the returned collection to provide Map methods.
In C++
you can deal with it by providing a method that takes a write iterator.
Using this
iterator we can put objects in any collection, provided by the user. Maybe
we can
achieve something similar using generics in Java, but I believe if we are
using this
approach then we still should provide simple method returning Map, because
to me
it looks like this is still going to be the most popular scenario.

Best Regards,
Igor


On Fri, Sep 10, 2021 at 9:50 AM Pavel Tupitsyn  wrote:

> Val, Alexei - thanks for your replies.
> Let's keep the Map approach then.
>
> Regarding tuple equality - there is another thread [1], please have a look.
>
>
> https://lists.apache.org/thread.html/r9ed68cd515bffab6df821bbeccb8e44d0e5536000e5e7dd05bd87017%40%3Cdev.ignite.apache.org%3E
>
> On Thu, Sep 9, 2021 at 11:21 PM Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > How do we handle the "equality" part in 2.x? The same problem exists
> there
> > as well, but we still somehow return a Map.
> >
> > Generally, I like Pavel's ideas, but there are a couple of issues with
> > them. Firstly, Java developers are really used to maps in this context.
> > Introducing something else might be confusing - it's a significant risk.
> > Secondly, there is no standard Pair class, so we'll have to introduce a
> > custom one. That said, I would not change this API in Java. In other
> > languages, however, we can consider this.
> >
> > -Val
> >
> > On Thu, Sep 9, 2021 at 8:01 AM Alexei Scherbakov <
> > alexey.scherbak...@gmail.com> wrote:
> >
> > > Pavel,
> > >
> > > I think the current API looks more natural compared to your proposal.
> > >
> > > -1  from my side, see comments below.
> > >
> > > чт, 9 сент. 2021 г. в 15:38, Pavel Tupitsyn :
> > >
> > > > Igniters,
> > > >
> > > > I propose to replace Map with List in getAll and invokeAll, and
> > > > Iterable in putAll APIs of Ignite 3.x KeyValueView.
> > > >
> > > > 1. Performance
> > > > putAll simply iterates over the map, we can easily accept Iterable
> > > instead.
> > > > Iterable can be implemented over anything, it can lazily read data
> > from a
> > > > file or some other place, instead of allocating a huge collection and
> > > > performing unnecessary hashing.
> > > >
> > > > getAll returns a Map, but we don't know if the user code needs a map
> or
> > > > just wants to iterate over the results, in which case Map is just
> > > overhead.
> > > >
> > >
> > > "allocating a huge collection" -
> > > This method is not intended for loading a huge number of entries.
> > > The allowed map size for the putAll should be limited to some
> reasonable
> > > value.
> > >
> > > Streaming loading should be addressed in a separate API similar to
> > > DataStream from Ignite 2.
> > >
> > >
> > > >
> > > > 2. Equality
> > > > getAll returns Map, but in many cases, the map will be useless
> > > > because K does not have proper equals()/hashCode() implementation, so
> > > > map.get(key) does not work.
> > > >
> > >
> > > We shouldn't rely on user equals/hashCode while working with key-value
> > API.
> > > Internally it uses binary representation of a user object for
> comparison
> > > purposes.
> > > The returned map implementation should work in the same way.
> > >
> > >
> > > >
> > > > Notes:
> > > > - It is not clear which Pair class to use yet - IgniteBiTuple or
> > > something
> > > > else.
> > > > - Ignite 3 won't deadlock due to putAll entry order, so we don't have
> > to
> > > > worry about sorting.
> > > >
> > > > Thoughts, objections?
> > > >
> > >
> > >
> > > --
> > >
> > > Best regards,
> > > Alexei Scherbakov
> > >
> >
>

Re: Release of pyignite 0.5.2 proposal

[Igor Sapego]

+1

Best Regards,
Igor


On Thu, Sep 9, 2021 at 8:33 PM Maxim Muzafarov  wrote:

> +1
>
> On Thu, 9 Sept 2021 at 14:08, Nikolay Izhikov  wrote:
> >
> > +1 to release ASAP.
> >
> > > 9 сент. 2021 г., в 13:43, Ivan Daschinsky 
> написал(а):
> > >
> > > TC build of release branch --
> > >
> https://tc.sbt-ignite-dev.ru/buildConfiguration/IgniteThinClients_Tests_ThinClientPython/6130289
> > >
> > > чт, 9 сент. 2021 г. в 13:40, Ivan Daschinsky :
> > >
> > >> Hi, folks.
> > >>
> > >> Unfortunately, a quite severe bug found in pyignite since 0.4.0 was
> found
> > >> [1]. Fortunately, it is already fixed. Also there are also a few bugs
> > >> already fixed.
> > >>
> > >> I propose to prepare the next minor release. Branch was already cut
> and
> > >> commits were cherry-picked [3].
> > >>
> > >> If there are no objections, I will prepare a release and will start
> voting
> > >> thread in a day or two.
> > >>
> > >> [1] -- https://issues.apache.org/jira/browse/IGNITE-15479
> > >> [2] --
> > >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%20python-0.5.2
> > >> [3] --
> > >>
> https://github.com/apache/ignite-python-thin-client/commits/pyignite-0.5.2
> > >>
> > >
> > >
> > > --
> > > Sincerely yours, Ivan Daschinskiy
> >
>

Re: Sync vs async APIs in Ignite 3

[Igor Sapego]

Ivan,

We are not going to adopt C++20 in the near future, as new standards
are not adopted as fast in C++ world and if we'd do that most probably
almost none of our users could use our client.

But yeah, if we'd use coroutines, I'd probably suggest that we only
provide the async API.

Best Regards,
Igor



On Thu, Sep 9, 2021 at 2:29 PM Ivan Daschinsky  wrote:

> Igor, and what about C++20 and coroutines [1]
>
> [1] -- https://en.cppreference.com/w/cpp/language/coroutines
>
> чт, 9 сент. 2021 г. в 14:12, Igor Sapego :
>
> > Well, fortunately we do not provide a C client if you don't consider ODBC
> > as one so we should not think about it. For C++ I believe we should use
> > standard std::future+std::promise for async, but still can provide sync
> > methods,
> > built on top of async methods. There is no continuation problem in C++
> API
> > as
> > std::future do not provide continuation methods :)
> >
> > Now, for the sync/async problem you've mentioned. Can not we solve it by
> > using different thread pools for completion and continuation of futures
> or
> > does
> > CompletableFuture interface not allowing for that?
> >
> > Best Regards,
> > Igor
> >
> >
> > On Thu, Sep 9, 2021 at 1:15 PM Ivan Daschinsky 
> > wrote:
> >
> > > And what about C/C++?  There are so many options for them...
> > >
> > > чт, 9 сент. 2021 г. в 13:00, Pavel Tupitsyn :
> > >
> > > > Talked to Ivan in private, and came up with the following
> per-language
> > > > summary:
> > > >
> > > > * Java: sync and async
> > > > *. NET: only async
> > > > * Python: sync and async
> > > > * JavaScript: only async (sync is not possible)
> > > >
> > > > Thin clients in other languages are to be discussed.
> > > >
> > > > On Thu, Sep 9, 2021 at 11:49 AM Ivan Daschinsky  >
> > > > wrote:
> > > >
> > > > > >> You can mix them easily.
> > > > > This is far from easily (you have already mentioned continuation
> > > > problem),
> > > > > but for i.e. in python it is absolutely not.
> > > > > For kotlin it is a little bit easier, but also not fluent and a
> > little
> > > > bit
> > > > > ugly.
> > > > >
> > > > > чт, 9 сент. 2021 г. в 11:37, Pavel Tupitsyn  >:
> > > > >
> > > > > > Ivan,
> > > > > >
> > > > > > > Pavel, is it really true, that in .NET sync versions of
> libraries
> > > and
> > > > > > tools
> > > > > > > are completely eliminated?
> > > > > >
> > > > > > Far from being eliminated, but there is a movement in this
> > direction.
> > > > > > For example, HttpClient [1] only provides async variants for most
> > of
> > > > the
> > > > > > methods.
> > > > > >
> > > > > > Exposing sync wrappers for async methods is not recommended [2].
> > > > > > There is a good explanation of potential threading issues there,
> > and
> > > it
> > > > > can
> > > > > > be applied to Java too.
> > > > > >
> > > > > >
> > > > > > > you cannot mix both functions easily
> > > > > >
> > > > > > You can mix them easily.
> > > > > > C#: table.GetAsync(k).Result or
> > > > > table.GetAsync(k).GetAwaiter().GetResult()
> > > > > > (different exception handling)
> > > > > > Java: table.getAsync(k).get()
> > > > > >
> > > > > > The same thing we do in sync wrapper for async method is now in
> the
> > > > user
> > > > > > code.
> > > > > > Which is better for two reasons:
> > > > > > - users won't accidentally use sync API when async API should be
> > used
> > > > > > - when they really have to use sync API, potentially dangerous
> > > behavior
> > > > > is
> > > > > > not hidden
> > > > > >
> > > > > >
> > > > > > [1]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclient?view=net-5.0#methods
> > > > > > [2]
> >

Re: Sync vs async APIs in Ignite 3

[Igor Sapego]

Well, fortunately we do not provide a C client if you don't consider ODBC
as one so we should not think about it. For C++ I believe we should use
standard std::future+std::promise for async, but still can provide sync
methods,
built on top of async methods. There is no continuation problem in C++ API
as
std::future do not provide continuation methods :)

Now, for the sync/async problem you've mentioned. Can not we solve it by
using different thread pools for completion and continuation of futures or
does
CompletableFuture interface not allowing for that?

Best Regards,
Igor


On Thu, Sep 9, 2021 at 1:15 PM Ivan Daschinsky  wrote:

> And what about C/C++?  There are so many options for them...
>
> чт, 9 сент. 2021 г. в 13:00, Pavel Tupitsyn :
>
> > Talked to Ivan in private, and came up with the following per-language
> > summary:
> >
> > * Java: sync and async
> > *. NET: only async
> > * Python: sync and async
> > * JavaScript: only async (sync is not possible)
> >
> > Thin clients in other languages are to be discussed.
> >
> > On Thu, Sep 9, 2021 at 11:49 AM Ivan Daschinsky 
> > wrote:
> >
> > > >> You can mix them easily.
> > > This is far from easily (you have already mentioned continuation
> > problem),
> > > but for i.e. in python it is absolutely not.
> > > For kotlin it is a little bit easier, but also not fluent and a little
> > bit
> > > ugly.
> > >
> > > чт, 9 сент. 2021 г. в 11:37, Pavel Tupitsyn :
> > >
> > > > Ivan,
> > > >
> > > > > Pavel, is it really true, that in .NET sync versions of libraries
> and
> > > > tools
> > > > > are completely eliminated?
> > > >
> > > > Far from being eliminated, but there is a movement in this direction.
> > > > For example, HttpClient [1] only provides async variants for most of
> > the
> > > > methods.
> > > >
> > > > Exposing sync wrappers for async methods is not recommended [2].
> > > > There is a good explanation of potential threading issues there, and
> it
> > > can
> > > > be applied to Java too.
> > > >
> > > >
> > > > > you cannot mix both functions easily
> > > >
> > > > You can mix them easily.
> > > > C#: table.GetAsync(k).Result or
> > > table.GetAsync(k).GetAwaiter().GetResult()
> > > > (different exception handling)
> > > > Java: table.getAsync(k).get()
> > > >
> > > > The same thing we do in sync wrapper for async method is now in the
> > user
> > > > code.
> > > > Which is better for two reasons:
> > > > - users won't accidentally use sync API when async API should be used
> > > > - when they really have to use sync API, potentially dangerous
> behavior
> > > is
> > > > not hidden
> > > >
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclient?view=net-5.0#methods
> > > > [2]
> > > >
> > > >
> > >
> >
> https://devblogs.microsoft.com/pfxteam/should-i-expose-synchronous-wrappers-for-asynchronous-methods/
> > > >
> > > > On Thu, Sep 9, 2021 at 11:16 AM Ivan Daschinsky  >
> > > > wrote:
> > > >
> > > > > >> 2. In languages with proper async support (async-await, etc.),
> we
> > > can
> > > > > skip sync API altogether.
> > > > > It sounds pretty strange, as for me. Usually you cannot mix both
> > > > functions
> > > > > easily, it is called blue-red functions problem [1]
> > > > > In python you definitely cannot do sync over async, for example
> > > > > (principally can, but nobody do that because of mediocre
> performance
> > of
> > > > > that solution). And many developers
> > > > > still prefer writing code withou asyncio at all.
> > > > >
> > > > > Pavel, is it really true, that in .NET sync versions of libraries
> and
> > > > tools
> > > > > are completely eliminated?
> > > > >
> > > > > [1] --
> > > > >
> > >
> https://elizarov.medium.com/how-do-you-color-your-functions-a6bb423d936d
> > > > >
> > > > > ср, 8 сент. 2021 г. в 22:33, Pavel Tupitsyn  >:
> > > > >
> > > > > > To put it another way:
> > > > > > - true sync operation completes by itself
> > > > > > - sync-over-async operation requires another thread to complete
> > > > > >
> > > > > > On Wed, Sep 8, 2021 at 10:15 PM Pavel Tupitsyn <
> > ptupit...@apache.org
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Val,
> > > > > > >
> > > > > > > That's exactly what I have in mind.
> > > > > > > Yes, we block the user thread, but then we should unblock it by
> > > > > > completing
> > > > > > > the future.
> > > > > > > We can't complete the future from a Netty thread [1], so we'll
> > need
> > > > > some
> > > > > > > other thread from some executor.
> > > > > > > If there are no threads available (because they are blocked by
> > the
> > > > sync
> > > > > > > API above), the future won't complete => deadlock.
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/r528659381d983a177d779f56ef3d7da6fe17eb3504383f5f87727514%40%3Cdev.ignite.apache.org%3E
> > > > > > >
> > > > > > > On Wed, Sep 8, 2021 at 9:40 PM Valentin Kulichenko <
> > > > > > > 

Re: [Announcement] Apache Ignite 2.11 Code Freeze started

[Igor Sapego]

Cherry-picked to release branch.

Best Regards,
Igor


On Fri, Jul 30, 2021 at 3:11 PM Alexey Gidaspov 
wrote:

> Ok, I think we should add [1] to 2.11 release. Can you cherry-pick it to
> release branch?
> [1] https://issues.apache.org/jira/browse/IGNITE-14815
>
> On 2021/07/30 02:25:25, Igor Sapego  wrote:
> > I'm fine with any of these two solutions.
> >
> > Best Regards,
> > Igor
> >
> >
> > On Thu, Jul 29, 2021 at 6:36 PM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com>
> > wrote:
> >
> > > Hello!
> > >
> > > I think it does make sense to include changes which will let us avoid
> > > migration issues in the future.
> > >
> > > Alternatively, maybe we can revert the incomplete change from 2.11 in
> order
> > > to reintroduce it in 2.12 in full form if Igor agrees and RE thinks
> this is
> > > the best course of action.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > чт, 29 июл. 2021 г. в 18:07, Igor Sapego :
> > >
> > > > Alexey,
> > > >
> > > > It contains changes we could not introduce if we release ignite in
> its
> > > > current state as they are going to be breaking changes.
> > > >
> > > > Best Regards,
> > > > Igor
> > > >
> > > >
> > > > On Thu, Jul 29, 2021 at 4:13 PM Alexey Gidaspov <
> olive.c...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi, Igor!
> > > > >
> > > > > Now we are in stabilization phase and accepting only blockers. I
> may be
> > > > > wrong, but this ticket doesn't seem to be of that kind.
> > > > >
> > > > > On 2021/07/28 21:00:15, Igor Sapego  wrote:
> > > > > > Igniters,
> > > > > >
> > > > > > I suggest adding [1] to the scope of release, because it contains
> > > > > > changes to code introduced by [2], which is already included in
> > > > release.
> > > > > >
> > > > > > [1] - https://issues.apache.org/jira/browse/IGNITE-14815
> > > > > > [2] - https://issues.apache.org/jira/browse/IGNITE-14658
> > > > > >
> > > > > > Best Regards,
> > > > > > Igor
> > > > > >
> > > > > >
> > > > > > On Mon, Jul 26, 2021 at 11:30 PM Maxim Muzafarov <
> mmu...@apache.org>
> > > > > wrote:
> > > > > >
> > > > > > > Folks,
> > > > > > >
> > > > > > > The issues mentioned above were successfully resolved and
> > > > > > > cherry-picked to the ignite-2.11 branch. Sorry for the delay. I
> > > think
> > > > > > > we can proceed with the release.
> > > > > > >
> > > > > > > On Thu, 22 Jul 2021 at 15:44, Maxim Muzafarov <
> mmu...@apache.org>
> > > > > wrote:
> > > > > > > >
> > > > > > > > Folks,
> > > > > > > >
> > > > > > > > Yes, both the fixes [1] [2] will not require performance
> tests
> > > > rerun.
> > > > > > > >
> > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-15146
> > > > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-15170
> > > > > > > >
> > > > > > > > On Thu, 22 Jul 2021 at 15:30, Dmitry Pavlov <
> dpav...@apache.org>
> > > > > wrote:
> > > > > > > > >
> > > > > > > > > I personally trust opinion of Nikolay and Maxim, we can
> > > consider
> > > > > both
> > > > > > > as blockers.
> > > > > > > > >
> > > > > > > > > Just an idea to consider:
> > > > > > > > > For fixed ticket/PR (
> > > > > > > https://issues.apache.org/jira/browse/IGNITE-15170)  most
> likely
> > > we
> > > > > don't
> > > > > > > need to re-run performance tests.
> > > > > > > > >
> > > > > > > > > If second issue has no impact on performance, we can take
> perf
> > > > > results
> > > > > > > from rc.-1 and run only functional t

Re: [Announcement] Apache Ignite 2.11 Code Freeze started

[Igor Sapego]

I'm fine with any of these two solutions.

Best Regards,
Igor


On Thu, Jul 29, 2021 at 6:36 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> I think it does make sense to include changes which will let us avoid
> migration issues in the future.
>
> Alternatively, maybe we can revert the incomplete change from 2.11 in order
> to reintroduce it in 2.12 in full form if Igor agrees and RE thinks this is
> the best course of action.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> чт, 29 июл. 2021 г. в 18:07, Igor Sapego :
>
> > Alexey,
> >
> > It contains changes we could not introduce if we release ignite in its
> > current state as they are going to be breaking changes.
> >
> > Best Regards,
> > Igor
> >
> >
> > On Thu, Jul 29, 2021 at 4:13 PM Alexey Gidaspov 
> > wrote:
> >
> > > Hi, Igor!
> > >
> > > Now we are in stabilization phase and accepting only blockers. I may be
> > > wrong, but this ticket doesn't seem to be of that kind.
> > >
> > > On 2021/07/28 21:00:15, Igor Sapego  wrote:
> > > > Igniters,
> > > >
> > > > I suggest adding [1] to the scope of release, because it contains
> > > > changes to code introduced by [2], which is already included in
> > release.
> > > >
> > > > [1] - https://issues.apache.org/jira/browse/IGNITE-14815
> > > > [2] - https://issues.apache.org/jira/browse/IGNITE-14658
> > > >
> > > > Best Regards,
> > > > Igor
> > > >
> > > >
> > > > On Mon, Jul 26, 2021 at 11:30 PM Maxim Muzafarov 
> > > wrote:
> > > >
> > > > > Folks,
> > > > >
> > > > > The issues mentioned above were successfully resolved and
> > > > > cherry-picked to the ignite-2.11 branch. Sorry for the delay. I
> think
> > > > > we can proceed with the release.
> > > > >
> > > > > On Thu, 22 Jul 2021 at 15:44, Maxim Muzafarov 
> > > wrote:
> > > > > >
> > > > > > Folks,
> > > > > >
> > > > > > Yes, both the fixes [1] [2] will not require performance tests
> > rerun.
> > > > > >
> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-15146
> > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-15170
> > > > > >
> > > > > > On Thu, 22 Jul 2021 at 15:30, Dmitry Pavlov 
> > > wrote:
> > > > > > >
> > > > > > > I personally trust opinion of Nikolay and Maxim, we can
> consider
> > > both
> > > > > as blockers.
> > > > > > >
> > > > > > > Just an idea to consider:
> > > > > > > For fixed ticket/PR (
> > > > > https://issues.apache.org/jira/browse/IGNITE-15170)  most likely
> we
> > > don't
> > > > > need to re-run performance tests.
> > > > > > >
> > > > > > > If second issue has no impact on performance, we can take perf
> > > results
> > > > > from rc.-1 and run only functional tests for rc.0.
> > > > > > >
> > > > > > > On 2021/07/22 04:44:01, "Николай Ижиков" 
> > > wrote:
> > > > > > > > +1 to fix both prior to release
> > > > > > > >
> > > > > > > > Отправлено с iPhone
> > > > > > > >
> > > > > > > > > 22 июля 2021 г., в 02:36, Maxim Muzafarov <
> mmu...@apache.org
> > >
> > > > > написал(а):
> > > > > > > > >
> > > > > > > > > Folks,
> > > > > > > > >
> > > > > > > > > We've faced [1][2] issues related to the new functionality
> > > added to
> > > > > > > > > the 2.11 release (it will be safe to merge to the release
> > > branch).
> > > > > > > > > From my point of view, both of them are critical and must
> be
> > > > > included
> > > > > > > > > in the 2.11 release.
> > > > > > > > > The [2] is ready for merge. The [1] will be completed by
> the
> > > end
> > > > > of this week.
> > > > > > > > >
> > > > > > > > > Please share your thoughts.
> > > > > > > > >
> 

Re: [Announcement] Apache Ignite 2.11 Code Freeze started

[Igor Sapego]

Alexey,

It contains changes we could not introduce if we release ignite in its
current state as they are going to be breaking changes.

Best Regards,
Igor


On Thu, Jul 29, 2021 at 4:13 PM Alexey Gidaspov 
wrote:

> Hi, Igor!
>
> Now we are in stabilization phase and accepting only blockers. I may be
> wrong, but this ticket doesn't seem to be of that kind.
>
> On 2021/07/28 21:00:15, Igor Sapego  wrote:
> > Igniters,
> >
> > I suggest adding [1] to the scope of release, because it contains
> > changes to code introduced by [2], which is already included in release.
> >
> > [1] - https://issues.apache.org/jira/browse/IGNITE-14815
> > [2] - https://issues.apache.org/jira/browse/IGNITE-14658
> >
> > Best Regards,
> > Igor
> >
> >
> > On Mon, Jul 26, 2021 at 11:30 PM Maxim Muzafarov 
> wrote:
> >
> > > Folks,
> > >
> > > The issues mentioned above were successfully resolved and
> > > cherry-picked to the ignite-2.11 branch. Sorry for the delay. I think
> > > we can proceed with the release.
> > >
> > > On Thu, 22 Jul 2021 at 15:44, Maxim Muzafarov 
> wrote:
> > > >
> > > > Folks,
> > > >
> > > > Yes, both the fixes [1] [2] will not require performance tests rerun.
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-15146
> > > > [2] https://issues.apache.org/jira/browse/IGNITE-15170
> > > >
> > > > On Thu, 22 Jul 2021 at 15:30, Dmitry Pavlov 
> wrote:
> > > > >
> > > > > I personally trust opinion of Nikolay and Maxim, we can consider
> both
> > > as blockers.
> > > > >
> > > > > Just an idea to consider:
> > > > > For fixed ticket/PR (
> > > https://issues.apache.org/jira/browse/IGNITE-15170)  most likely we
> don't
> > > need to re-run performance tests.
> > > > >
> > > > > If second issue has no impact on performance, we can take perf
> results
> > > from rc.-1 and run only functional tests for rc.0.
> > > > >
> > > > > On 2021/07/22 04:44:01, "Николай Ижиков" 
> wrote:
> > > > > > +1 to fix both prior to release
> > > > > >
> > > > > > Отправлено с iPhone
> > > > > >
> > > > > > > 22 июля 2021 г., в 02:36, Maxim Muzafarov 
> > > написал(а):
> > > > > > >
> > > > > > > Folks,
> > > > > > >
> > > > > > > We've faced [1][2] issues related to the new functionality
> added to
> > > > > > > the 2.11 release (it will be safe to merge to the release
> branch).
> > > > > > > From my point of view, both of them are critical and must be
> > > included
> > > > > > > in the 2.11 release.
> > > > > > > The [2] is ready for merge. The [1] will be completed by the
> end
> > > of this week.
> > > > > > >
> > > > > > > Please share your thoughts.
> > > > > > >
> > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-15146
> > > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-15170
> > > > > > >
> > > > > > >> On Tue, 20 Jul 2021 at 15:08, Maxim Muzafarov <
> mmu...@apache.org>
> > > wrote:
> > > > > > >>
> > > > > > >> Folks,
> > > > > > >>
> > > > > > >> Since the release branch was created some time ago, should we
> > > bump up
> > > > > > >> the master branch version to the next one?
> > > > > > >>
> > > > > > >>> On Thu, 1 Jul 2021 at 17:15, Alexey Gidaspov <
> > > olive.c...@gmail.com> wrote:
> > > > > > >>>
> > > > > > >>> Hi, Pavel.
> > > > > > >>>
> > > > > > >>> I think, it looks like blocker. Please cherry-pick it to 2.11
> > > release branch
> > > > > > >>>
> > > > > > >>> On 2021/07/01 09:29:57, Pavel Pereslegin 
> > > wrote:
> > > > > > >>>> Hello, Alexey!
> > > > > > >>>>
> > > > > > >>>> Is it possible to include a hotfix that corrects the command
> > > syntax
> > > > > > 

Re: [Announcement] Apache Ignite 2.11 Code Freeze started

[Igor Sapego]

Igniters,

I suggest adding [1] to the scope of release, because it contains
changes to code introduced by [2], which is already included in release.

[1] - https://issues.apache.org/jira/browse/IGNITE-14815
[2] - https://issues.apache.org/jira/browse/IGNITE-14658

Best Regards,
Igor


On Mon, Jul 26, 2021 at 11:30 PM Maxim Muzafarov  wrote:

> Folks,
>
> The issues mentioned above were successfully resolved and
> cherry-picked to the ignite-2.11 branch. Sorry for the delay. I think
> we can proceed with the release.
>
> On Thu, 22 Jul 2021 at 15:44, Maxim Muzafarov  wrote:
> >
> > Folks,
> >
> > Yes, both the fixes [1] [2] will not require performance tests rerun.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-15146
> > [2] https://issues.apache.org/jira/browse/IGNITE-15170
> >
> > On Thu, 22 Jul 2021 at 15:30, Dmitry Pavlov  wrote:
> > >
> > > I personally trust opinion of Nikolay and Maxim, we can consider both
> as blockers.
> > >
> > > Just an idea to consider:
> > > For fixed ticket/PR (
> https://issues.apache.org/jira/browse/IGNITE-15170)  most likely we don't
> need to re-run performance tests.
> > >
> > > If second issue has no impact on performance, we can take perf results
> from rc.-1 and run only functional tests for rc.0.
> > >
> > > On 2021/07/22 04:44:01, "Николай Ижиков"  wrote:
> > > > +1 to fix both prior to release
> > > >
> > > > Отправлено с iPhone
> > > >
> > > > > 22 июля 2021 г., в 02:36, Maxim Muzafarov 
> написал(а):
> > > > >
> > > > > Folks,
> > > > >
> > > > > We've faced [1][2] issues related to the new functionality added to
> > > > > the 2.11 release (it will be safe to merge to the release branch).
> > > > > From my point of view, both of them are critical and must be
> included
> > > > > in the 2.11 release.
> > > > > The [2] is ready for merge. The [1] will be completed by the end
> of this week.
> > > > >
> > > > > Please share your thoughts.
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-15146
> > > > > [2] https://issues.apache.org/jira/browse/IGNITE-15170
> > > > >
> > > > >> On Tue, 20 Jul 2021 at 15:08, Maxim Muzafarov 
> wrote:
> > > > >>
> > > > >> Folks,
> > > > >>
> > > > >> Since the release branch was created some time ago, should we
> bump up
> > > > >> the master branch version to the next one?
> > > > >>
> > > > >>> On Thu, 1 Jul 2021 at 17:15, Alexey Gidaspov <
> olive.c...@gmail.com> wrote:
> > > > >>>
> > > > >>> Hi, Pavel.
> > > > >>>
> > > > >>> I think, it looks like blocker. Please cherry-pick it to 2.11
> release branch
> > > > >>>
> > > > >>> On 2021/07/01 09:29:57, Pavel Pereslegin 
> wrote:
> > > >  Hello, Alexey!
> > > > 
> > > >  Is it possible to include a hotfix that corrects the command
> syntax
> > > >  output in the control script? [1]
> > > > 
> > > >  This bug can significantly complicate the use of the snapshot
> restore
> > > >  function (one of the important features of 2.11). In addition,
> this
> > > >  may raise a number of questions to the product support, which
> we can
> > > >  prevent by adding this patch in 2.11.
> > > > 
> > > >  This patch does not affect any functions other than the "help"
> output
> > > >  of the control script.
> > > > 
> > > >  [1] https://issues.apache.org/jira/browse/IGNITE-14989
> > > > 
> > > >  чт, 1 июл. 2021 г. в 11:56, Alexey Gidaspov <
> olive.c...@gmail.com>:
> > > > >
> > > > > Hi, Iilya!
> > > > >
> > > > > As I can see, this feature highly improves debugging during
> incidents. So I think we can call it blocker and cherry-pick to ignite-2.11
> branch
> > > > >
> > > > > On 2021/06/30 20:26:43, Shishkov Ilya 
> wrote:
> > > > >> Hello, Alexey!
> > > > >> Is it possible to add system views for BaselineNode
> attributes [1] and
> > > > >> corresponding documentation [2] to 2.11?
> > > > >>
> > > > >> 1. https://issues.apache.org/jira/browse/IGNITE-15007
> > > > >> 2. https://issues.apache.org/jira/browse/IGNITE-15028
> > > > >>
> > > > >> ср, 30 июн. 2021 г. в 11:07, Nikita Amelchev <
> namelc...@apache.org>:
> > > > >>
> > > > >>> Thanks! I have cherry-picked the commit to the 2.11 branch.
> > > > >>>
> > > > >>> ср, 30 июн. 2021 г. в 11:00, Alexey Gidaspov <
> olive.c...@gmail.com>:
> > > > 
> > > >  Hi, Nikita!
> > > > 
> > > >  I think it's important fix and should be included in 2.11
> release. I've
> > > > >>> tagged ticket by fixVersion 2.11. Can you cherry-pick it to
> ignite-2.11
> > > > >>> branch? And please fill release notes or delete flag.
> > > > 
> > > >  On 2021/06/30 07:55:04, Nikita Amelchev <
> namelc...@apache.org> wrote:
> > > > > Hello, Alexey.
> > > > >
> > > > > I suggest adding to the 2.11 scope the resolved issue [1].
> It fixes
> > > > > incorrect values of cache, cache groups, data region
> metrics after
> > > 

Re: [VOTE] Release pyignite 0.5.1-rc0

[Igor Sapego]

+1 from me

Best Regards,
Igor


On Fri, Jul 23, 2021 at 3:32 PM Ivan Daschinsky 
wrote:

> +1 From me
> 1. Checked binary packages, c module and examples on windows 10 amd64 for
> pythons 3.6, 3.7, 3.8, 3.9
> 2. Checked binary packages, c module and examples on ubuntu 20.04 amd64 for
> pythons 3.6, 3.7, 3.8, 3.9
> 3. Checked source installation and building binary packages on ubuntu 20.04
> amd 64 for pythons 3.6, 3.7, 3.8, 3.9
> 4. Checked documentation on
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.1.rc0
> 5. Checked sha512 checksums and gpg signatures (signed by Igor Sapego (CODE
> SIGNING KEY)  5C10 A072 2D94 7727 923C  98B5 AF35 DBD9
> 58FE 8DC5)
> key is inside https://downloads.apache.org/ignite/KEYS)
>
> пт, 23 июл. 2021 г. в 13:52, Ivan Daschinsky :
>
> > The voting finishes at 07/27/2021 12:00 UTC
> >
> > пт, 23 июл. 2021 г. в 13:49, Ivan Daschinsky :
> >
> >> Dear Igniters!
> >>
> >> Release candidate binaries for subj are uploaded and ready for vote
> >> You can find them here:
> >> https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.5.1-rc0
> >>
> >> If you follow the link above, you will find source packages (*.tar.gz
> and
> >> *.zip)
> >> and binary packages (wheels) for windows (amd64) and linux (x86_64)
> >> for pythons 36, 37, 38, 39. Also, there are sha512 and gpg signatures.
> >> Code signing keys can be found here --
> >> https://downloads.apache.org/ignite/KEYS
> >> Here you can find instructions how to verify packages
> >> https://www.apache.org/info/verification.html
> >>
> >> You can install binary package for specific version of python using pip
> >> For example do this on linux for python 3.8
> >> >> pip install pyignite-0.5.1-cp38-cp38-manylinux1_x86_64.whl
> >>
> >> You can build and install package from source using this command:
> >> >> pip install pyignite-0.5.1.tar.gz
> >> You can build wheel on your platform using this command:
> >> >> pip wheel --no-deps pyignite-0.5.1.tar.gz
> >>
> >> For building C module, you should have python headers and C compiler
> >> installed.
> >> (i.e. for ubuntu sudo apt install build-essential python3-dev)
> >> In Mac OS X xcode-tools and python from homebrew are the best option.
> >>
> >> In order to check whether C module works, use following:
> >> >> from pyignite import _cutils
> >> >> print(_cutils.hashcode('test'))
> >> >> 3556498
> >>
> >> You can find documentation here:
> >>
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.1.rc0
> >>
> >> You can find examples here (to check them, you should start ignite
> >> locally):
> >>
> >>
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.1.rc0/examples.html
> >> Also, examples can be found in source archive in examples subfolder.
> >> docker-compose.yml is supplied in order to start ignite quickly. (Use
> >> `docker-compose up -d` to start 3 nodes cluster and `docker-compose
> >> down` to shut down it)
> >>
> >> Release notes:
> >>
> >>
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=blob;f=RELEASE_NOTES.txt;h=c6cbd419684cd4a97485707471bac84957b42891;hb=b48dd5dec37064b458031358c394789d15a756fc
> >>
> >> Git release tag was created:
> >>
> >>
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=tag;h=refs/tags/0.5.1.rc0
> >>
> >> The vote is formal, see voting guidelines
> >> https://www.apache.org/foundation/voting.html
> >>
> >> +1 - to accept pyignite-0.5.1-rc0
> >> 0 - don't care either way
> >> -1 - DO NOT accept pyignite-0.5.1-rc0
> >>
> >>
> >
> > --
> > Sincerely yours, Ivan Daschinskiy
> >
>

Re: IEP-76 Thin Client Protocol for Ignite 3.0

[Igor Sapego]

Ivan, what are extra serde steps you are talking about?

Best Regards,
Igor


On Thu, Jul 1, 2021 at 5:52 PM Ivan Daschinsky  wrote:

> > I agree. But this was decided before in IEP-54, and is out of scope for
> current IEP.
> Would you like to start a separate thread to discuss this? Or I can do this
> a bit later.
>
> Great idea, let's discuss it. I suppose this will simplify many aspects of
> realization and improve performance a lot
>
> чт, 1 июл. 2021 г., 17:50 Ivan Daschinsky :
>
> > > Here is the description of TUPLE_GET_ALL:
> > - UUID: table ID
> > - int: schema ID
> > - arr of arr: array of rows with values for all columns in given schema
> >
> > I suppose that we should describe this more verbose and explicit. I
> > nevertheless suggest to also consider writing values this way:
> > - arr of fields names (if name is missed, corresponding field is nil)
> > - arr of rows (row as array, length equal to fields array)
> >
> > It is quite simple and if we use str8 (it is more than enough for any
> > utf-8 reasonable field name), overhead will be negligible, but
> realization
> > of a client will be way simpler
> >
> > чт, 1 июл. 2021 г., 16:57 Pavel Tupitsyn :
> >
> >> > No it isn't, I have carefully read code and IEP, in your code you
> write
> >> > schema id in each tuple.
> >>
> >> There is no code for batch operations yet.
> >>
> >> Here is the description of TUPLE_GET_ALL:
> >> - UUID: table ID
> >> - int: schema ID
> >> - arr of arr: array of rows with values for all columns in given schema
> >> (nil when value is missing for a column)
> >>
> >> As you can see, schema ID is written once for all rows.
> >> A row is just a set of values according to the schema.
> >>
> >>
> >> > Also, my biggest concern -- extra serde step. I suppose we should pass
> >> > bytearray to internal api, and use msgpack throughout all wire
> >> protocols,
> >> > as tarantool does.
> >>
> >> I agree. But this was decided before in IEP-54, and is out of scope for
> >> current IEP.
> >> Would you like to start a separate thread to discuss this? Or I can do
> >> this
> >> a bit later.
> >>
> >> On Thu, Jul 1, 2021 at 4:41 PM Ivan Daschinsky 
> >> wrote:
> >>
> >> > > This is described in all operations that include multiple tuples.
> >> > No it isn't, I have carefully read code and IEP, in your code you
> write
> >> > schema id in each tuple.
> >> >
> >> > Also, my biggest concern -- extra serde step. I suppose we should pass
> >> > bytearray to internal api, and use msgpack throughout all wire
> >> protocols,
> >> > as tarantool does.
> >> >
> >> > чт, 1 июл. 2021 г., 16:15 Pavel Tupitsyn :
> >> >
> >> > > Ivan,
> >> > >
> >> > > >  that there is not neccesary to write schema versions in each row
> >> > > > in collectionof tuples
> >> > >
> >> > > This is described in all operations that include multiple tuples.
> >> > >
> >> > >
> >> > > > it is not clear from your code (probably
> >> > > > mistake?) how differ key tuples and value tuples from each other
> >> > >
> >> > > Key tuples include only key columns. Key columns come first in the
> >> > schema.
> >> > > Value tuples include all columns, key and value. Added "Key tuples"
> >> > > section.
> >> > >
> >> > >
> >> > > > As for me, these excercises with schema's doesn't worth a lot
> >> > >
> >> > > I'll add a benchmark and we'll see.
> >> > >
> >> > >
> >> > > On Thu, Jul 1, 2021 at 3:17 PM Ivan Daschinsky  >
> >> > > wrote:
> >> > >
> >> > > > I suppose, that there is not neccesary to write schema versions in
> >> each
> >> > > row
> >> > > > in collectionof tuples. Also it is not clear from your code
> >> (probably
> >> > > > mistake?) how differ key tuples and value tuples from each other.
> In
> >> > > > readTuple you always read full schema and check for full length.
> As
> >> for
> >> > > me,
> >> > > > these excercises with schema's doesn't worth a lot. I.e. postgres
> >> just
> >> > > > writes field names and then simpy rows with data. Saving few bytes
> >> > > doesn't
> >> > > > make much deal. Btw, msgpack has special types for short strings
> >> (i.e.
> >> > > > str8). It is much easier use it and write field name as is.
> >> > > >
> >> > > > чт, 1 июл. 2021 г., 14:56 Pavel Tupitsyn :
> >> > > >
> >> > > > > Ivan, tuple serialization section added to the IEP, let me know
> >> if it
> >> > > is
> >> > > > > clear enough.
> >> > > > >
> >> > > > > Thanks!
> >> > > > >
> >> > > > > On Thu, Jul 1, 2021 at 2:06 PM Ivan Daschinsky <
> >> ivanda...@gmail.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > I can't find any description of tuple serialization in IEP,
> >> only in
> >> > > > code
> >> > > > > >
> >> > > > > > чт, 1 июл. 2021 г., 13:59 Pavel Tupitsyn <
> ptupit...@apache.org
> >> >:
> >> > > > > >
> >> > > > > > > Ivan,
> >> > > > > > >
> >> > > > > > > 0. The IEP is not in progress, it is ready for review and
> >> > > discussion.
> >> > > > > > > 1. Tuple serialization is described in the IEP and
> >> demonstrated
> >> > in
> >> > > > the
> >> > > > > 

Re: [VOTE] Release Apache Ignite 3.0.0-alpha2 RC1

[Igor Sapego]

+1

Best Regards,
Igor


On Sat, Jun 26, 2021 at 1:41 AM Nikita Ivanov  wrote:

> +1
>
> --
> Nikita Ivanov
>
>
>
> On Fri, Jun 25, 2021 at 3:31 PM Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > Dear Community,
> >
> > In the last several months, the development of Ignite 3 has been moving
> > forward significantly. On top of what we already had in the first Alpha,
> we
> > have the following features ready:
> >
> > - Replication infrastructure based on Raft
> > - In-memory atomic storage with the basic insert-read functionality
> > - New schema management engine and API
> >
> > In my view, this constitutes a significant milestone, and I propose to
> > release it as the Alpha 2. This way it will be available for download,
> and
> > anyone will be able to play with the project, run examples, get a feel of
> > how Ignite will work in the future, and provide feedback.
> >
> > Please vote.
> >
> > ---
> >
> > The release candidate is uploaded here:
> > https://dist.apache.org/repos/dist/dev/ignite/3.0.0-alpha2-rc1/
> > Maven staging:
> > https://repository.apache.org/content/repositories/orgapacheignite-1522/
> > Git tag: https://github.com/apache/ignite-3/tree/3.0.0-alpha2-rc1
> >
> > Jira tickets:
> >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%2012315922%20AND%20fixVersion%20%3D%2012349574%20ORDER%20BY%20priority%20DESC%2C%20key%20ASC
> >
> > DEVNOTES:
> > https://github.com/apache/ignite-3/blob/3.0.0-alpha2-rc1/DEVNOTES.md
> >
> > The vote is formal, see voting guidelines:
> > https://www.apache.org/foundation/voting.html
> >
> > +1 - accept Apache Ignite 3.0.0-alpha2 RC1
> > 0 - don't care either way
> > -1 - DO NOT accept Apache Ignite 3.0.0-alpha2 RC1 (explain why)
> >
> > See notes on how to verify release here:
> > https://www.apache.org/info/verification.html
> >
> > This vote will be open for 72 hours and will close on June 28th at 11pm
> > UTC:
> >
> >
> https://www.timeanddate.com/countdown/to?iso=20210628T16=224=Apache+Ignite+3.0.0-alpha2+RC1=cursive=1
> >
> > -Val
> >
>

Re: [VOTE] Release pyignite 0.5.0-rc1

[Igor Sapego]

Well,

This behaviour maybe is not obvious, but it seems to be safe and
should not cause user issues. On the other hand, if we change this
behavior now, it may lead to implicit disable of SSL for users that
updated their client which seems more dangerous to me.

So I'd keep the current behaviour.

Best Regards,
Igor


On Fri, Jun 18, 2021 at 1:17 PM Ivan Daschinsky  wrote:

> I suppose that we should not cancel this release, despite the fact that
> this is not obvious behaviour. This is not a regression, this behaviour is
> documented and this behaviour lasts for few years. Lets remove it, if the
> majority are against it, but in the next release.
>
> пт, 18 июн. 2021 г. в 13:08, Ivan Daschinsky :
>
> > >>  we explicitly set use_ssl=True.
> > Sorry, typo -- implicitly
> >
> > пт, 18 июн. 2021 г. в 12:59, Ivan Daschinsky :
> >
> >> AHA! I see, this is not a bug -- this is a feature. If you pass username
> >> and password, we explicitly set use_ssl=True. So if your cluster is
> >> configured without ssl but with authentication,
> >> you should explicitly pass use_ssl=False.
> >>
> >> This behaviour is from old version and I suppose it is correct. Who
> wants
> >> authentication that sent without encryption?
> >>
> >> пт, 18 июн. 2021 г. в 12:54, Ivan Daschinsky :
> >>
> >>> Just rechecked test on release branch, add extra check with cluster
> >>> activation and putting some data -- everything works ok. Authentication
> >>> enabled, persistence enabled,
> >>> with and without ssl. Could you please provide you ignite config and
> >>> your code.
> >>>
> >>> пт, 18 июн. 2021 г. в 12:46, Ivan Daschinsky :
> >>>
> >>>> There is a test for it.
> >>>>
> >>>> пт, 18 июн. 2021 г. в 12:30, Stephen Darlington <
> >>>> stephen.darling...@gridgain.com>:
> >>>>
> >>>>> Oh… can someone else check this: it appears that authenticated
> >>>>> connections fail.
> >>>>>
> >>>>> With Ignite 2.10 the connection times-out:
> >>>>>
> >>>>>
> [10:28:58,015][WARNING][grid-timeout-worker-#22][ClientListenerNioListener]
> >>>>> Unable to perform handshake within timeout [timeout=1,
> remoteAddr=/
> >>>>> 127.0.0.1:54044]
> >>>>>
> >>>>> Didn’t try this with 0.4.0 so not sure if it’s a regression, but it’s
> >>>>> not great.
> >>>>>
> >>>>> > On 18 Jun 2021, at 09:36, Stephen Darlington <
> >>>>> stephen.darling...@gridgain.com> wrote:
> >>>>> >
> >>>>> > +1
> >>>>> >
> >>>>> > Checked on macOS, played with the new expiry APIs and a bunch of
> >>>>> thefundamentals.
> >>>>> >
> >>>>> >> On 17 Jun 2021, at 12:46, Pavel Tupitsyn 
> >>>>> wrote:
> >>>>> >>
> >>>>> >> +1
> >>>>> >>
> >>>>> >> Checked pip install from tar.gz on Python 3.8 on Ubuntu 20.04, ran
> >>>>> some of
> >>>>> >> the examples.
> >>>>> >>
> >>>>> >> On Thu, Jun 17, 2021 at 2:32 PM Igor Sapego 
> >>>>> wrote:
> >>>>> >>
> >>>>> >>> +1 from me
> >>>>> >>>
> >>>>> >>> Best Regards,
> >>>>> >>> Igor
> >>>>> >>>
> >>>>> >>>
> >>>>> >>> On Thu, Jun 17, 2021 at 12:10 PM Ivan Daschinsky <
> >>>>> ivanda...@gmail.com>
> >>>>> >>> wrote:
> >>>>> >>>
> >>>>> >>>> +1 From me
> >>>>> >>>> Checked on Ubuntu 20.04 and windows 10
> >>>>> >>>> 1. Installation from wheels for pythons 3.6 3.7 3.8 3.9
> >>>>> >>>> 2. Native module work
> >>>>> >>>> 3. Examples
> >>>>> >>>>
> >>>>> >>>> Checked on Ubuntu 20.04 building from source package and correct
> >>>>> work of
> >>>>> >>>> result package.
> >>>>> >>>>
> >>>>> >>>> Checked all sha256 ch

Re: [VOTE] Release pyignite 0.5.0-rc1

[Igor Sapego]

+1 from me

Best Regards,
Igor


On Thu, Jun 17, 2021 at 12:10 PM Ivan Daschinsky 
wrote:

> +1 From me
> Checked on Ubuntu 20.04 and windows 10
> 1. Installation from wheels for pythons 3.6 3.7 3.8 3.9
> 2. Native module work
> 3. Examples
>
> Checked on Ubuntu 20.04 building from source package and correct work of
> result package.
>
> Checked all sha256 checksums and gpg signatures.
>
> Let's extend voting period till June 18, 15:00 UTC
>
>
>
> ср, 16 июн. 2021 г. в 17:34, Ivan Daschinsky :
>
> > The vote will end at June, 17 15:00 UTC.
> >
> > ср, 16 июн. 2021 г. в 17:33, Ivan Daschinsky :
> > >
> > > Dear Igniters!
> > >
> > > Release candidate binaries for subj are uploaded and ready for vote
> > > You can find them here:
> > > https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.5.0-rc1
> > >
> > > If you follow the link above, you will find source package (*.tar.gz
> and
> > *.zip)
> > > and binary packages (wheels) for windows (amd64) and linux (x86_64)
> > > for pythons 36, 37, 38, 39. Also, there are sha512 and gpg signatures.
> > > Code signing keys can be found here --
> > https://downloads.apache.org/ignite/KEYS
> > > Here you can find instructions how to verify packages
> > > https://www.apache.org/info/verification.html
> > >
> > > You can install binary package for specific version of python using pip
> > > For example do this on linux for python 3.8
> > > >> pip install pyignite-0.5.0-cp38-cp38-manylinux1_x86_64.whl
> > >
> > > You can build and install package from source using this command:
> > > >> pip install pyignite-0.5.0.tar.gz
> > > You can build wheel on your platform using this command:
> > > >> pip wheel --no-deps pyignite-0.5.0.tar.gz
> > >
> > > For building C module, you should have python headers and C compiler
> > installed.
> > > (i.e. for ubuntu sudo apt install build-essential python3-dev)
> > > In Mac OS X xcode-tools and python from homebrew are the best option.
> > >
> > > In order to check whether C module works, use following:
> > > >> from pyignite import _cutils
> > > >> print(_cutils.hashcode('test'))
> > > >> 3556498
> > >
> > > You can find documentation here:
> > >
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc1
> > >
> > > You can find examples here (to check them, you should start ignite
> > locally):
> > >
> >
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc1/examples.html
> > > Also, examples can be found in source archive in examples subfolder.
> > > docker-compose.yml is supplied in order to start ignite quickly. (Use
> > > `docker-compose up -d` to start 3 nodes cluster and `docker-compose
> > > down` to shut down it)
> > >
> > > Release notes:
> > >
> >
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=blob;f=RELEASE_NOTES.txt;h=9d2ae81af2de22ce9e8c9d3b7ece14dd9e75ca0e;hb=61c83cb0ab6752f019518b4a2cb0724bd027755f
> > >
> > > Git release tag was created:
> > >
> >
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=tag;h=refs/tags/0.5.0.rc1
> > >
> > > The vote is formal, see voting guidelines
> > > https://www.apache.org/foundation/voting.html
> > >
> > > +1 - to accept pyignite-0.5.0-rc1
> > > 0 - don't care either way
> > > -1 - DO NOT accept pyignite-0.5.0-rc1
> >
>
>
> --
> Sincerely yours, Ivan Daschinskiy
>

Re: [VOTE] Release pyignite 0.5.0-rc0

[Igor Sapego]

I propose to cancel this release and fix the issue which was highlighted in
the
"Seconds and milliseconds confusion in python thin client" thread.

WDYT?

Best Regards,
Igor


On Wed, Jun 16, 2021 at 12:10 AM Igor Sapego  wrote:

> +1 from me
>
> Uploaded to test.pipy.org: https://test.pypi.org/project/pyignite/0.5.0/
>
> Everything looks good.
>
> Best Regards,
> Igor
>
>
> On Tue, Jun 15, 2021 at 10:09 PM Ivan Daschinsky 
> wrote:
>
>> Also checked hash sums and signature. Packages are verified and
>> signature is OK, signed by Igor Sapego with key
>> 5C10A0722D947727923C98B5AF35DBD958FE8DC5
>> Keys can be found here -- https://downloads.apache.org/ignite/KEYS
>> Here you can find instructions how to verify packages
>> https://www.apache.org/info/verification.html
>>
>> вт, 15 июн. 2021 г. в 22:01, Ivan Daschinsky :
>> >
>> > +1 From me
>> > Checked building from source on Ubuntu 20.04 amd64 for pythons 3.6.12
>> > 3.7.9 3.8.6 3.9.1.
>> > Checked installing binary packages on Ubuntu 20.04 amd64 and Windows
>> > 10 amd 64 for pythons 3.6.12 3.7.9 3.8.6 3.9.1.
>> > Run examples for both platforms
>> > Check that native module works
>> > Checked documentation for tag 0.5.0.rc0 on readthedocs.io
>> >
>> > вт, 15 июн. 2021 г. в 21:58, Ivan Daschinsky :
>> > >
>> > > Dear Igniters!
>> > >
>> > > Release candidate binaries for subj are uploaded and ready for vote
>> > > You can find them here:
>> > > https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.5.0-rc0
>> > >
>> > > If you follow the link above, you will find source package (*.tar.gz
>> and *.zip)
>> > > and binary packages (wheels) for windows (amd64) and linux (x86_64)
>> > > for pythons 36, 37, 38, 39. Also, there are sha512 and gpg signatures.
>> > >
>> > > You can install binary package for specific version of python using
>> pip
>> > > For example do this on linux for python 3.8
>> > > >> pip install pyignite-0.4.0-cp38-cp38-manylinux1_x86_64.whl
>> > >
>> > > You can build and install package from source using this command:
>> > > >> pip install pyignite-0.4.0.tar.gz
>> > > You can build wheel on your platform using this command:
>> > > >> pip wheel --no-deps pyignite-0.4.0.tar.gz
>> > >
>> > > For building C module, you should have python headers and C compiler
>> installed.
>> > > (i.e. for ubuntu sudo apt install build-essential python3-dev)
>> > > In Mac OS X xcode-tools and python from homebrew are the best option.
>> > >
>> > > In order to check whether C module works, use following:
>> > > >> from pyignite import _cutils
>> > > >> print(_cutils.hashcode('test'))
>> > > >> 3556498
>> > >
>> > > You can find documentation here:
>> > >
>> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc0
>> > >
>> > > You can find examples here (to check them, you should start ignite
>> locally):
>> > >
>> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc0/examples.html
>> > > Also, examples can be found in source archive in examples subfolder.
>> > > docker-compose.yml is supplied in order to start ignite quickly. (Use
>> > > `docker-compose up -d` to start 3 nodes cluster and `docker-compose
>> > > down` to shut down it)
>> > >
>> > > Release notes:
>> > >
>> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=blob;f=RELEASE_NOTES.txt;h=9d2ae81af2de22ce9e8c9d3b7ece14dd9e75ca0e;hb=61c83cb0ab6752f019518b4a2cb0724bd027755f
>> > >
>> > > Git release tag was created:
>> > >
>> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=tag;h=refs/tags/0.5.0.rc0
>> > >
>> > > The vote is formal, see voting guidelines
>> > > https://www.apache.org/foundation/voting.html
>> > >
>> > > +1 - to accept pyignite-0.5.0-rc0
>> > > 0 - don't care either way
>> > > -1 - DO NOT accept pyignite-0.5.0-rc0
>> > >
>> > > The vote will end at June, 17 15:00 UTC.
>> >
>> >
>> >
>> > --
>> > Sincerely yours, Ivan Daschinskiy
>>
>>
>>
>> --
>> Sincerely yours, Ivan Daschinskiy
>>
>

Re: Seconds and milliseconds confusion in python thin client

[Igor Sapego]

Why is it not released?

I can see client.sql(timeout) in 0.4.0 for example, which is int number of
ms.

Best Regards,
Igor


On Tue, Jun 15, 2021 at 11:52 PM Ivan Daschinsky 
wrote:

> BTW, common approach is to treat both ints and floats as seconds. Floats
> are used to set timeout with millisecods precision. I.e. asyncio.sleep(1.0)
> and asyncio.sleep(1) pauses coroutine for 1 sec. Lets create ticket for it,
> stop voting for 0.5.0.rc0 and schedule next vote.
>
> вт, 15 июн. 2021 г., 23:49 Ivan Daschinsky :
>
> > Igor, I suppose that you are probably right. But there is no need to
> > notice or deprecate something. This functionality is not released yet
> >
> > вт, 15 июн. 2021 г., 23:41 Igor Sapego :
> >
> >> Hi Igniters,
> >>
> >> I've noticed a weird behaviour of python thin client. In those places
> >> where
> >> we have
> >> timeouts or any other parameters that take time in some places we treat
> it
> >> like integer
> >> number of milliseconds, in others it can take both floats (as a number
> of
> >> seconds)
> >> and ints (number of milliseconds). This approach looks very confusing to
> >> me
> >> as
> >> it leads to things where tx_start(1) and tx_start(1.0) are not actually
> >> the
> >> same thing.
> >>
> >> AFAIK in python the most common way to pass time to such functions is to
> >> use floats
> >> as a number of seconds. This is the approach I propose to use in our API
> >> as
> >> well. Let's
> >> deprecate usage of ints in those functions with the appropriate warning
> >> before getting rid
> >> of it.
> >>
> >> What do you think?
> >>
> >> Best Regards,
> >> Igor
> >>
> >
>

Re: [VOTE] Release pyignite 0.5.0-rc0

[Igor Sapego]

+1 from me

Uploaded to test.pipy.org: https://test.pypi.org/project/pyignite/0.5.0/

Everything looks good.

Best Regards,
Igor


On Tue, Jun 15, 2021 at 10:09 PM Ivan Daschinsky 
wrote:

> Also checked hash sums and signature. Packages are verified and
> signature is OK, signed by Igor Sapego with key
> 5C10A0722D947727923C98B5AF35DBD958FE8DC5
> Keys can be found here -- https://downloads.apache.org/ignite/KEYS
> Here you can find instructions how to verify packages
> https://www.apache.org/info/verification.html
>
> вт, 15 июн. 2021 г. в 22:01, Ivan Daschinsky :
> >
> > +1 From me
> > Checked building from source on Ubuntu 20.04 amd64 for pythons 3.6.12
> > 3.7.9 3.8.6 3.9.1.
> > Checked installing binary packages on Ubuntu 20.04 amd64 and Windows
> > 10 amd 64 for pythons 3.6.12 3.7.9 3.8.6 3.9.1.
> > Run examples for both platforms
> > Check that native module works
> > Checked documentation for tag 0.5.0.rc0 on readthedocs.io
> >
> > вт, 15 июн. 2021 г. в 21:58, Ivan Daschinsky :
> > >
> > > Dear Igniters!
> > >
> > > Release candidate binaries for subj are uploaded and ready for vote
> > > You can find them here:
> > > https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.5.0-rc0
> > >
> > > If you follow the link above, you will find source package (*.tar.gz
> and *.zip)
> > > and binary packages (wheels) for windows (amd64) and linux (x86_64)
> > > for pythons 36, 37, 38, 39. Also, there are sha512 and gpg signatures.
> > >
> > > You can install binary package for specific version of python using pip
> > > For example do this on linux for python 3.8
> > > >> pip install pyignite-0.4.0-cp38-cp38-manylinux1_x86_64.whl
> > >
> > > You can build and install package from source using this command:
> > > >> pip install pyignite-0.4.0.tar.gz
> > > You can build wheel on your platform using this command:
> > > >> pip wheel --no-deps pyignite-0.4.0.tar.gz
> > >
> > > For building C module, you should have python headers and C compiler
> installed.
> > > (i.e. for ubuntu sudo apt install build-essential python3-dev)
> > > In Mac OS X xcode-tools and python from homebrew are the best option.
> > >
> > > In order to check whether C module works, use following:
> > > >> from pyignite import _cutils
> > > >> print(_cutils.hashcode('test'))
> > > >> 3556498
> > >
> > > You can find documentation here:
> > >
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc0
> > >
> > > You can find examples here (to check them, you should start ignite
> locally):
> > >
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc0/examples.html
> > > Also, examples can be found in source archive in examples subfolder.
> > > docker-compose.yml is supplied in order to start ignite quickly. (Use
> > > `docker-compose up -d` to start 3 nodes cluster and `docker-compose
> > > down` to shut down it)
> > >
> > > Release notes:
> > >
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=blob;f=RELEASE_NOTES.txt;h=9d2ae81af2de22ce9e8c9d3b7ece14dd9e75ca0e;hb=61c83cb0ab6752f019518b4a2cb0724bd027755f
> > >
> > > Git release tag was created:
> > >
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=tag;h=refs/tags/0.5.0.rc0
> > >
> > > The vote is formal, see voting guidelines
> > > https://www.apache.org/foundation/voting.html
> > >
> > > +1 - to accept pyignite-0.5.0-rc0
> > > 0 - don't care either way
> > > -1 - DO NOT accept pyignite-0.5.0-rc0
> > >
> > > The vote will end at June, 17 15:00 UTC.
> >
> >
> >
> > --
> > Sincerely yours, Ivan Daschinskiy
>
>
>
> --
> Sincerely yours, Ivan Daschinskiy
>

Seconds and milliseconds confusion in python thin client

[Igor Sapego]

Hi Igniters,

I've noticed a weird behaviour of python thin client. In those places where
we have
timeouts or any other parameters that take time in some places we treat it
like integer
number of milliseconds, in others it can take both floats (as a number of
seconds)
and ints (number of milliseconds). This approach looks very confusing to me
as
it leads to things where tx_start(1) and tx_start(1.0) are not actually the
same thing.

AFAIK in python the most common way to pass time to such functions is to
use floats
as a number of seconds. This is the approach I propose to use in our API as
well. Let's
deprecate usage of ints in those functions with the appropriate warning
before getting rid
of it.

What do you think?

Best Regards,
Igor

Re: IEP-68: Thin Client Data Streamer

[Igor Sapego]

r 5, 2021 at 4:12 PM Ivan Daschinsky <
> >> ivanda...@gmail.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > >>> - Corresponding types should exist on server nodes
> >> > > > > > Ok, but I suppose this is of questional usability, same for
> >> > existing
> >> > > > > > implementation for ScanQuery and ContinousQuery.
> >> > > > > > For queries it's probably ok to add some custom filters and
> put
> >> > them
> >> > > in
> >> > > > > > servers' classpathes, but I can hardly imagine
> >> > > > > > somebody wants to create custom Receiver this way.
> >> > > > > >
> >> > > > > > пт, 5 мар. 2021 г. в 15:54, Pavel Tupitsyn <
> >> ptupit...@apache.org>:
> >> > > > > >
> >> > > > > > > > How do you suggests to serialize this object?
> >> > > > > > >
> >> > > > > > > Like a normal binary object. This scenario already exists
> for
> >> > > > ScanQuery
> >> > > > > > and
> >> > > > > > > ContinuousQuery filters.
> >> > > > > > > - It works for Java and .NET; potentially for other
> platforms
> >> > > > > > > - Corresponding types should exist on server nodes
> >> > > > > > >
> >> > > > > > > E.g. if a Java class `MyFilter { String containsText }` is
> >> loaded
> >> > > on
> >> > > > > > server
> >> > > > > > > nodes,
> >> > > > > > > a Python thin client can easily write a corresponding
> >> > BinaryObject
> >> > > > with
> >> > > > > > one
> >> > > > > > > field to achieve server-side filtering.
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > > I also suppose, that closing should be done wia
> >> > OP_RESOURCE_CLOSE
> >> > > > > > >
> >> > > > > > > Thanks, forgot to mention this - updated the IEP.
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > On Fri, Mar 5, 2021 at 3:42 PM Ivan Daschinsky <
> >> > > ivanda...@gmail.com>
> >> > > > > > > wrote:
> >> > > > > > >
> >> > > > > > > > I also suppose, that closing should be done wia
> >> > > OP_RESOURCE_CLOSE.
> >> > > > > > It'is
> >> > > > > > > > more consistent and resembles with existing cursor api.
> >> > > > > > > >
> >> > > > > > > > пт, 5 мар. 2021 г. в 15:37, Ivan Daschinsky <
> >> > ivanda...@gmail.com
> >> > > >:
> >> > > > > > > >
> >> > > > > > > > > >> Both proposed requests have a Flush flag - please see
> >> > > Details
> >> > > > > > > sections
> >> > > > > > > > > in the IEP.
> >> > > > > > > > > Ok, sorry, I missed this.
> >> > > > > > > > > >> StreamReceiver is public API [1]
> >> > > > > > > > > I know it. But this "object" should contains custom
> logic
> >> for
> >> > > > > putting
> >> > > > > > > > data
> >> > > > > > > > > in cache. How do you suggests to serialize this object?
> >> > > > > > > > > Implement custom classloader for java? What about other
> >> > > > platforms?
> >> > > > > > > > >
> >> > > > > > > > > I suppose that we should not add this field, because it
> is
> >> > > > > confusing.
> >> > > > > > > > > Everything will work for local unit tests and only for
> >> java.
> >> > > > > > > > > But in real environment this will not work at all.
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > 

Re: Thin Clients: enable partition awareness by default

[Igor Sapego]

+1 from me. There were no major issues with this feature and it gives
good performance boost for many cases.

Best Regards,
Igor


On Wed, May 12, 2021 at 5:18 PM Ivan Daschinsky  wrote:

> Huge +1 from me. PA should be enabled by default.
>
> ср, 12 мая 2021 г. в 13:33, Pavel Tupitsyn :
> >
> > Igniters,
> >
> > Partition Awareness (PA) is implemented in 5 out of 6 thin clients [1].
> >
> > However, this feature is disabled by default in most clients for
> > compatibility reasons:
> > initially we only used one connection to the cluster, but with PA enabled
> > we establish
> > connections to every server node, which may be not desirable in some use
> > cases.
> >
> > I expect those scenarios to be rare, and in most cases it makes sense to
> > enable PA by default.
> > Thoughts, objections?
> >
> >
> > [1]
> https://cwiki.apache.org/confluence/display/IGNITE/Thin+clients+features
>
>
>
> --
> Sincerely yours, Ivan Daschinskiy
>

Re: NodeJS thin client: full API

[Igor Sapego]

Thanks for issuing a ticket. I'll take a look at it.

Best Regards,
Igor


On Fri, Apr 23, 2021 at 1:40 PM teligenz.dheeraj 
wrote:

> Team,
>
> I have used nodejs thin client to connect ignite. With single query at time
> on socket works fine. But when hit multiple request simultaneously, getting
> frequent error message "Invalid response id: XXX".
>
> when we try to get 5-10 query per sec we getting below error random times.
>
> debug - Error: Invalid response id: 4122254909997320969at
> /webapp/node_modules/apache-ignite-client/lib/internal/ClientSocket.js:344:28
>
> at Map.forEach ()at ClientSocket._disconnect
> (/webapp/node_modules/apache-ignite-client/lib/internal/ClientSocket.js:343:24)
>
> at Socket.
> (/webapp/node_modules/apache-ignite-client/lib/internal/ClientSocket.js:170:22)
>
> at runMicrotasks ()at processTicksAndRejections
>
> I have mark complete issue detail in on https://issues.apache.org/ with
> issue id IGNITE-14550
>
> https://issues.apache.org/jira/browse/IGNITE-14550
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

Re: [DISCUSSION] Documentation of thin clients (python, php, nodejs)

[Igor Sapego]

Hi,

Ivan, I like your suggestion. To me it looks better than the current
approach.

Best Regards,
Igor


On Mon, Apr 26, 2021 at 11:47 AM Nikita Safonov 
wrote:

> Hi Ivan,
>
> Thanks for sharing the information.
> I'll look through the docs and share my thoughts and suggestions soon.
>
> Regards,
> Nikita
>
>
> пт, 23 апр. 2021 г. в 14:06, Ivan Daschinsky :
>
> > Hi, Nikita! Thanks for support.
> >
> > As developers of thin client, we also would try to update
> > documentation as soon as possible.
> > For example, recently I've merged to master expiry policy support for
> > python thin client and documentation was updated simultaneously [2][3]
> >
> > I am planning to update documentation frequently and I would
> > appreciate any feedback about our current documentation on
> > readthedocs.io
> >
> >
> > [1] -- https://issues.apache.org/jira/browse/IGNITE-14595
> > [2] --
> >
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/examples.html#expirypolicy
> > [3] --
> >
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/async_examples.html#expirypolicy
> >
> >
> >
> > пт, 23 апр. 2021 г. в 13:29, Никита Сафонов :
> > >
> > > Hi Ivan,
> > >
> > > Thank you for the suggestion.
> > >
> > > I agree that Ignite main docs on thin clients probably need a clean-up
> to
> > > be kept up-to-date.
> > > And migrating the key information to readthedocs.io is really an
> option.
> > >
> > > Let's see what other Igniters think about this idea.
> > >
> > > I'd be glad if Igor Gusev could share his opinion on this.
> > >
> > > Regards,
> > > Nikita
> > >
> > >
> > >
> > > чт, 22 апр. 2021 г. в 22:15, Ivan Daschinsky :
> > >
> > > > Igniters, there are some questions regarding the documentation state
> > > > of thin clients.
> > > >
> > > > Recently, we have released pyignite 0.4.0. Traditionally,
> > > > documentation for python thin client is autogenerated from source and
> > > > contains in the same git repository, as the client itself.
> > > > Documentation is autogenerated and hosted in familiar for python
> > > > developers manner -- on readthedocs.io (Namely,
> > > >
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/
> > )
> > > >
> > > > I suppose the same statement is true for other separately developed
> > > > clients (nodejs, php).
> > > >
> > > > So I'd like to discuss a current documentation approach for thin
> > clients.
> > > > 1. I strongly believe that the main documentation site for at least
> > > > python thin client should be readthedocs.io.
> > > > 2. Documentation should be maintained in the same repository as the
> > > > thin client itself.
> > > > 3. As the main documentation's version is tightly coupled with ignite
> > > > release cycle, it is by default outdated and doesn't resemble the
> > > > latest version of thin client.
> > > >
> > > > I suggest just remove all documentation from the main docs except
> > > > simple installation instruction (i.e. pip install pyignite) and link
> > > > to readthedocs.io
> > > >
> > > > Regards, Ivan Daschinsky
> > > >
> >
> >
> >
> > --
> > Sincerely yours, Ivan Daschinskiy
> >
>

Re: [VOTE] Release pyignite 0.4.0-rc1

[Igor Sapego]

+1 from me.

Best Regards,
Igor


On Fri, Apr 16, 2021 at 5:05 PM Ivan Daschinsky 
wrote:

> Ivan Daschinsky 
> чт, 15 апр., 21:37 (19 часов назад)
> кому: dev
> Dear Igniters!
>
> Release candidate binaries are at least uploaded and ready for vote
> You can find them here:
> https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.4.0-rc1
>
> If you follow the link above, you will find source package (*.tar.gz and
> *.zip)
> and binary packages (wheels) for windows (x86, amd64) and linux (x86 and
> x86_64)
> for pythons 36, 37, 38, 39. Also, there are sha512 and gpg signatures.
>
> You can install binary package for specific version of python using pip
> For example do this on linux for python 3.8
> >> pip install pyignite-0.4.0-cp38-cp38-manylinux1_x86_64.whl
>
> You can build and install package from source using this command:
> >> pip install pyignite-0.4.0.tar.gz
> You can build wheel on your platform using this command:
> >> pip wheel --no-deps pyignite-0.4.0.tar.gz
>
> For building C module, you should have python headers and C compiler
> installed.
> (i.e. for ubuntu sudo apt install build-essential python3-dev)
> In Mac OS X xcode-tools and python from homebrew are the best option.
>
> In order to check whether C module works, use following:
> >> from pyignite import _cutils
> >> print(_cutils.hashcode('test'))
> >> 3556498
>
> You can find documentation here:
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.4.0.rc1
>
> You can find examples here (to check them, you should start ignite
> locally):
>
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.4.0.rc1/examples.html
> Also, examples can be found in source archive in examples subfolder.
> docker-compose.yml is supplied in order to start ignite quickly.
>
> Release notes:
>
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=blob;f=RELEASE_NOTES.txt;h=9fee8ead3f70718767f6e11f93d1c7e77c61657b;hb=466b54527e6e42bc585c594d840a959d0b8626ef
>
> Git release tag was created:
>
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=tag;h=refs/tags/0.4.0.rc1
>
> The vote is formal, see voting guidelines
> https://www.apache.org/foundation/voting.html
>
> +1 - to accept pyignite-0.4.0-rc1
> 0 - don't care either way
> -1 - DO NOT accept pyignite-0.4.0-rc1
>
> The vote will end at April, 21 15:00 UTC.
>

[jira] [Created] (IGNITE-14534) Python thin: Add script for building wheels on Windows

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-14534:


 Summary: Python thin: Add script for building wheels on Windows
 Key: IGNITE-14534
 URL: https://issues.apache.org/jira/browse/IGNITE-14534
 Project: Ignite
  Issue Type: Improvement
  Components: thin client
Affects Versions: python-0.3.4
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: python-0.3.4


We already have scripts to build wheels on Linux, now we need script to build 
wheels on Windows.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[ANNOUNCE] Welcome Ivan Daschinsky as a new committer

[Igor Sapego]

The Project Management Committee (PMC) for Apache Ignite has invited
Ivan Daschinsky to become a committer and we are pleased to announce that
he has accepted.

Ivan made a lot of contributions to Apache Ignite.
He helped a lot to improve our Python Thin Client fixing a lot of different
bugs and contributing major feature such as asyncio support and C-extension
which improved performance significantly for many cases. Thanks to Ivan
Python
Thin client has become much more stable and production-ready. He also
introduced the CMake building system for Ignite C++, and has made a number
of
other important improvements. Besides the code contributions, Ivan is also
an
active community member.

Being a committer enables easier contribution to the project since there is
no need to go via the patch submission process. This should enable better
productivity.

Please join me in welcoming Ivan, and congratulating him on the new role in
the Apache Ignite Community.

Best Regards,
Igor

[jira] [Created] (IGNITE-14475) C++/dotnet query-example select result is various with additional node

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-14475:


 Summary: C++/dotnet query-example select result is various with 
additional node
 Key: IGNITE-14475
 URL: https://issues.apache.org/jira/browse/IGNITE-14475
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Affects Versions: 2.10
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.11


Steps:

Start some additional nodes

Execute query-example

Expected output: 

{noformat}
Names of all employees and organizations they belong to: 
Jane Doe is working in ApacheIgnite
John Doe is working in ApacheIgnite
Jane Smith is working in Other
John Smith is working in Other
{noformat}

Actual (in 80% cases):

{noformat}
Names of all employees and organizations they belong to: 
Jane Doe is working in ApacheIgnite
John Doe is working in ApacheIgnite
John Smith is working in Other
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-14465) Add the ability to activate the cluster via the Python thin client

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-14465:


 Summary: Add the ability to activate the cluster via the Python 
thin client
 Key: IGNITE-14465
 URL: https://issues.apache.org/jira/browse/IGNITE-14465
 Project: Ignite
  Issue Type: Bug
  Components: python, thin client
Affects Versions: python-0.3.4
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: python-0.4.0


This feature will be extremely useful when working with clusters that have the 
"persistenceEnabled" option. Since they require activation to get started. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-14432) Python thin: Support "with" statement for connection method

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-14432:


 Summary: Python thin: Support "with" statement for connection 
method
 Key: IGNITE-14432
 URL: https://issues.apache.org/jira/browse/IGNITE-14432
 Project: Ignite
  Issue Type: Bug
  Components: python, thin client
Reporter: Igor Sapego
 Fix For: python-0.4.0


[pep-0343|https://www.python.org/dev/peps/pep-0343/] 

instead of this:

{code:python}client.connect(environ['DB_HOST'], int(environ['DB_PORT']))
try:
return client.sql(q, query_args=q_args, include_field_names=True)
finally:
client.close(){code}

The user will be able to use this:

{code:python}with client.connect(environ['DB_HOST'], int(environ['DB_PORT'])):
return client.sql(q, query_args=q_args, include_field_names=True){code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: IEP-70: Async Continuation Executor

[Igor Sapego]

Pavel, I like the proposal,

+1 from me

Best Regards,
Igor


On Tue, Mar 16, 2021 at 6:49 PM Pavel Tupitsyn  wrote:

> Alexey,
>
> .NET thick API delegates to Java directly.
>
> When you do ICache.PutAsync():
> * Future is created on Java side, .listen() is called
> * TaskCompletionSource is created on .NET side, its Task is returned to the
> user
> * Operation completes, Future listener is called on the Java side
> * Listener invokes JNI callback to .NET, where
> TaskCompletionSource.SetResult is called
>
> Therefore, .NET user code (in ContinueWith or after await) will be executed
> on the Java
> thread that invokes the future listener.
>
> After the proposed fix, future listeners will be invoked on
> ForkJoinPool#commonPool (instead of striped pool).
> So .NET continuations will end up in commonPool as well, which solves the
> problem for .NET automatically, no changes required.
>
> Does that make sense?
>
> On Tue, Mar 16, 2021 at 1:52 PM Alexey Kukushkin <
> kukushkinale...@gmail.com>
> wrote:
>
> > Hi Pavel,
> >
> > Extending Java async API with additional Executor parameters looks OK to
> > me.
> >
> > It is not clear from the IEP how you are going to do that for .NET async
> > API. My understanding is in .NET we do not add any Executors. Instead,
> the
> > Ignite Async API should use (SynchronizationContext.Current ??
> > TaskScheduler.Current) by default and it should have exciting behavior
> (use
> > Ignite striped pool) if ConfigureAwait(false) was specified for the Task
> > result.
> >
> > Is my understanding correct?
> >
> >
> > пн, 15 мар. 2021 г. в 19:24, Pavel Tupitsyn :
> >
> > > Igniters,
> > >
> > > Please review the IEP [1] and let me know your thoughts.
> > >
> > > [1]
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-70%3A+Async+Continuation+Executor
> > >
> >
> >
> > --
> > Best regards,
> > Alexey
> >
>

Re: Thin client implementation removal

[Igor Sapego]

Nikolay,

That's because we now have separate repos for them: [1], [2] and [3].
Actually, active development is moved to those repos some time ago and
those directories in main repo are not actual anymore anyway.

Decision of moving those clients to separate repos was discussed in [4]

[1] - https://github.com/apache/ignite-python-thin-client
[2] - https://github.com/apache/ignite-nodejs-thin-client
[3] - https://github.com/apache/ignite-php-thin-client
[4] -
http://apache-ignite-developers.2346864.n4.nabble.com/Moving-python-php-and-node-js-in-separate-repos-and-release-cycles-td47030.html

Best Regards,
Igor


On Mon, Mar 15, 2021 at 2:20 PM Nikolay Izhikov  wrote:

> Hello, Igor.
>
> I’ve noticed that Python, NodeJS, PHP thin client implementations were
> removed from master by you.
> Can you, please, comment?
> Why do you make those decision?

Re: [VOTE] Release Apache Ignite 2.10.0 RC1

[Igor Sapego]

+1 (binding)

Checked C++ compilation, C++ examples

Best Regards,
Igor


On Fri, Mar 5, 2021 at 12:32 AM Denis Magda  wrote:

> +1 (binding)
>
> Downloaded the binary package and started a 2-node cluster on MacOS with
> ignite.sh.
>
> -
> Denis
>
>
> On Wed, Mar 3, 2021 at 1:03 PM Maxim Muzafarov  wrote:
>
> > Dear Community,
> >
> > The release candidate is ready. The rest of the documentation pages
> > will be completed prior to the release announcement message. Please,
> > see the links below.
> >
> >
> > I have uploaded a release candidate to:
> > https://dist.apache.org/repos/dist/dev/ignite/2.10.0-rc1/
> > https://dist.apache.org/repos/dist/dev/ignite/packages_2.10.0-rc1/
> >
> > The following staging can be used for testing:
> > https://repository.apache.org/content/repositories/orgapacheignite-1506
> >
> >
> https://www.myget.org/feed/apache-ignite-staging/package/nuget/Apache.Ignite
> >
> > Tag name is 2.10.0-rc1:
> >
> >
> https://gitbox.apache.org/repos/asf?p=ignite.git;a=commit;h=refs/tags/2.10.0-rc1
> >
> > RELEASE_NOTES:
> >
> >
> https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob_plain;f=RELEASE_NOTES.txt;hb=ignite-2.10
> >
> > Complete list of resolved issues:
> >
> >
> https://issues.apache.org/jira/issues/?jql=(project%20%3D%20'Ignite'%20AND%20fixVersion%20is%20not%20empty%20AND%20fixVersion%20in%20('2.10'))
> >
> > DEVNOTES:
> >
> >
> https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob_plain;f=DEVNOTES.txt;hb=ignite-2.10
> >
> >
> > Additional checks have been performed (available for users included into
> > the release group on TeamCity).
> >
> > TC [Check RC: Licenses, compile, chksum]
> >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=5901349=ApacheIgniteReleaseJava8_PrepareVote4CheckRcLicensesChecksum
> >
> > TC [3] Build & Upload Nuget Staging Packages
> >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=5901347=ApacheIgniteReleaseJava8_PrepareVote3BuildNuGetPackages
> >
> > TC [2] Compare w/ Previous Release
> >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=5901351=ApacheIgniteReleaseJava8_IgniteRelease72CheckFileConsistency=artifacts_Releases_ApacheIgniteMain=ignite-2.9.1#%2Fresults
> >
> >
> > The vote is formal, see voting guidelines
> > https://www.apache.org/foundation/voting.html
> >
> > +1 - to accept Apache Ignite 2.10.0-rc1
> > 0 - don't care either way
> > -1 - DO NOT accept Apache Ignite Ignite 2.10.0-rc1 (explain why)
> >
> > See notes on how to verify release here
> > https://www.apache.org/info/verification.html
> > and
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification
> >
> > This vote will be open until Mon Mar 8, 15:00 UTC.
> > Please, write me down the thread if you need additional time to check
> > the release.
> >
> >
> https://www.timeanddate.com/countdown/vote?iso=20210308T18=166=VOTE+on+the+Apache+Ignite+Release+2.10.0+RC1=sanserif
> >
>

Re: IEP-68: Thin Client Data Streamer

[Igor Sapego]

Pavel,

I've checked the IEP and I like it. The only thing that seems a bit
confusing to me
is that there are 4 different variants for clients but there are cons and
pros for
different variants. Maybe at least few sentences should be written here to
give developers who are not familiar with DataStreamer some ideas and make
it easier for them to choose.

Best Regards,
Igor


On Thu, Mar 4, 2021 at 3:14 PM Pavel Tupitsyn  wrote:

> Igniters,
>
> Please review the IEP [1] and let me know what you think.
>
> [1]
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-68%3A+Thin+Client+Data+Streamer
>

[jira] [Created] (IGNITE-14265) Python thin: Support passwords for certificates

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-14265:


 Summary: Python thin: Support passwords for certificates 
 Key: IGNITE-14265
 URL: https://issues.apache.org/jira/browse/IGNITE-14265
 Project: Ignite
  Issue Type: Bug
  Components: python, thin client
Affects Versions: python-0.3.4
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: python-0.4.0


Need to add support for certfiles with passwords.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Re[2]: [DISCUSSION] Apache Ignite Release 2.10 (time, scope, manager)

[Igor Sapego]

The following commit should be
cherry-picked: 0675e2a7e800730c9c8230332b82809754ddae5a

Sorry for a delay.

Best Regards,
Igor


On Mon, Mar 1, 2021 at 9:06 PM Igor Sapego  wrote:

> Maxim,
>
> The issue is fixed and is merged to master now.
>
> Best Regards,
> Igor
>
>
> On Sun, Feb 28, 2021 at 8:12 PM Maxim Muzafarov  wrote:
>
>> Hello Igor,
>>
>> >  I believe I could fix the ticket [1] by the end of the next week.
>> Should we still wait a bit for t his fix or postpone it to 2.10.1? WDYT?
>>
>>
>>
>> [1] https://issues.apache.org/jira/browse/IGNITE-14204
>>
>> On Fri, 26 Feb 2021 at 11:20, Max Timonin 
>> wrote:
>> >
>> > Hi, Maxim!
>> >
>> > The bug IGNITE-14206 [1] fixed. There are 2 commits to cherry-pick
>> >
>> > 1.
>> >
>> https://github.com/apache/ignite/commit/851f650ba03e0b6c081cfe23f411fd2bf6be0228
>> > 2.
>> >
>> https://github.com/apache/ignite/commit/93b74922bd04b164301d7bc5a2788b9693d4a8a4
>> >
>> >
>> > https://issues.apache.org/jira/browse/IGNITE-14206
>> >
>> > On Wed, Feb 24, 2021 at 5:47 PM Maxim Muzafarov 
>> wrote:
>> >
>> > > Anton,
>> > >
>> > > Your improvement is very important, for sure, but it's almost 2 months
>> > > have passed since the initiation of the release branch. I think we
>> > > should prepare the changes as fast as possible for now and initiate
>> > > with the next one release. In general, it would be nice to have a
>> > > release plan for the year ahead so each developer will understand on
>> > > which release his changes can get into.
>> > >
>> > > That's why I'm going to prepare an RC-build on Monday 1-st March.
>> > >
>> > > On Wed, 24 Feb 2021 at 11:13, Anton Vinogradov  wrote:
>> > > >
>> > > > Maxim,
>> > > >
>> > > > https://issues.apache.org/jira/browse/IGNITE-13873
>> > > > Is ready for review, is it possible to wait for it?
>> > > >
>> > > > On Wed, Feb 24, 2021 at 12:01 AM Maxim Muzafarov > >
>> > > wrote:
>> > > >
>> > > > > Folks,
>> > > > >
>> > > > >
>> > > > > I'd like to cherry-pick to the release branch the following fixes:
>> > > > >
>> > > > > Fix security context for JDBC bulk-load operations
>> > > > > https://issues.apache.org/jira/browse/IGNITE-13472
>> > > > >
>> > > > > Fixed an issue that caused a deadlock when user cache was created
>> in
>> > > > > parallel with TTL worker was in progress.
>> > > > > https://issues.apache.org/jira/browse/IGNITE-14078
>> > > > >
>> > > > > On Thu, 18 Feb 2021 at 20:26, Maxim Muzafarov 
>> > > wrote:
>> > > > > >
>> > > > > > Maxim, Igor,
>> > > > > >
>> > > > > > Thanks for sharing details.
>> > > > > > Let's wait for these fixes.
>> > > > > >
>> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-14206
>> > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-14204
>> > > > > >
>> > > > > > On Thu, 18 Feb 2021 at 18:35, Igor Sapego 
>> > > wrote:
>> > > > > > >
>> > > > > > > Maxim,
>> > > > > > >
>> > > > > > > I believe I could fix the ticket [1] by the end of the next
>> week.
>> > > > > > >
>> > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-14204
>> > > > > > >
>> > > > > > > Best Regards,
>> > > > > > > Igor
>> > > > > > >
>> > > > > > >
>> > > > > > > On Thu, Feb 18, 2021 at 6:30 PM Max Timonin <
>> > > timonin.ma...@gmail.com>
>> > > > > wrote:
>> > > > > > >
>> > > > > > > > Hi! I've today found an issue [1], there is wrong handling
>> of
>> > > > > inlined POJO.
>> > > > > > > > This bug appeared in 2.9.0 and makes it impossible to work
>> with
>> > > > > > > > multi-column indexes created on old AI versions t

Re: Re[2]: [DISCUSSION] Apache Ignite Release 2.10 (time, scope, manager)

[Igor Sapego]

Maxim,

The issue is fixed and is merged to master now.

Best Regards,
Igor


On Sun, Feb 28, 2021 at 8:12 PM Maxim Muzafarov  wrote:

> Hello Igor,
>
> >  I believe I could fix the ticket [1] by the end of the next week.
> Should we still wait a bit for t his fix or postpone it to 2.10.1? WDYT?
>
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-14204
>
> On Fri, 26 Feb 2021 at 11:20, Max Timonin  wrote:
> >
> > Hi, Maxim!
> >
> > The bug IGNITE-14206 [1] fixed. There are 2 commits to cherry-pick
> >
> > 1.
> >
> https://github.com/apache/ignite/commit/851f650ba03e0b6c081cfe23f411fd2bf6be0228
> > 2.
> >
> https://github.com/apache/ignite/commit/93b74922bd04b164301d7bc5a2788b9693d4a8a4
> >
> >
> > https://issues.apache.org/jira/browse/IGNITE-14206
> >
> > On Wed, Feb 24, 2021 at 5:47 PM Maxim Muzafarov 
> wrote:
> >
> > > Anton,
> > >
> > > Your improvement is very important, for sure, but it's almost 2 months
> > > have passed since the initiation of the release branch. I think we
> > > should prepare the changes as fast as possible for now and initiate
> > > with the next one release. In general, it would be nice to have a
> > > release plan for the year ahead so each developer will understand on
> > > which release his changes can get into.
> > >
> > > That's why I'm going to prepare an RC-build on Monday 1-st March.
> > >
> > > On Wed, 24 Feb 2021 at 11:13, Anton Vinogradov  wrote:
> > > >
> > > > Maxim,
> > > >
> > > > https://issues.apache.org/jira/browse/IGNITE-13873
> > > > Is ready for review, is it possible to wait for it?
> > > >
> > > > On Wed, Feb 24, 2021 at 12:01 AM Maxim Muzafarov 
> > > wrote:
> > > >
> > > > > Folks,
> > > > >
> > > > >
> > > > > I'd like to cherry-pick to the release branch the following fixes:
> > > > >
> > > > > Fix security context for JDBC bulk-load operations
> > > > > https://issues.apache.org/jira/browse/IGNITE-13472
> > > > >
> > > > > Fixed an issue that caused a deadlock when user cache was created
> in
> > > > > parallel with TTL worker was in progress.
> > > > > https://issues.apache.org/jira/browse/IGNITE-14078
> > > > >
> > > > > On Thu, 18 Feb 2021 at 20:26, Maxim Muzafarov 
> > > wrote:
> > > > > >
> > > > > > Maxim, Igor,
> > > > > >
> > > > > > Thanks for sharing details.
> > > > > > Let's wait for these fixes.
> > > > > >
> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-14206
> > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-14204
> > > > > >
> > > > > > On Thu, 18 Feb 2021 at 18:35, Igor Sapego 
> > > wrote:
> > > > > > >
> > > > > > > Maxim,
> > > > > > >
> > > > > > > I believe I could fix the ticket [1] by the end of the next
> week.
> > > > > > >
> > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-14204
> > > > > > >
> > > > > > > Best Regards,
> > > > > > > Igor
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Feb 18, 2021 at 6:30 PM Max Timonin <
> > > timonin.ma...@gmail.com>
> > > > > wrote:
> > > > > > >
> > > > > > > > Hi! I've today found an issue [1], there is wrong handling of
> > > > > inlined POJO.
> > > > > > > > This bug appeared in 2.9.0 and makes it impossible to work
> with
> > > > > > > > multi-column indexes created on old AI versions that contain
> > > inlined
> > > > > POJO
> > > > > > > > keys in the middle.
> > > > > > > >
> > > > > > > > I'm working on the fix and asked Konstantin Orlov to review
> it. I
> > > > > think
> > > > > > > > that it will take only a couple of days to fix this issue.
> From
> > > my
> > > > > side, it
> > > > > > > > looks like a release blocker, but we've been living with that
> > > since
> > > > > 2.9.0.
> > > > > &

[jira] [Created] (IGNITE-14211) Python thin: SQL API is broken

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-14211:


 Summary: Python thin: SQL API is broken
 Key: IGNITE-14211
 URL: https://issues.apache.org/jira/browse/IGNITE-14211
 Project: Ignite
  Issue Type: Bug
Affects Versions: python-0.3.4
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: python-0.4.0


Currently, Client.sql() call uses a 'schema' argument to get cache instance and 
fails when there are no such cache. It leads to weird fails when user tries to 
run SQL over PUBLIC schema, and may fail in other cases when cache name is 
different from schema name.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Re[2]: [DISCUSSION] Apache Ignite Release 2.10 (time, scope, manager)

[Igor Sapego]

Maxim,

I believe I could fix the ticket [1] by the end of the next week.

[1] https://issues.apache.org/jira/browse/IGNITE-14204

Best Regards,
Igor


On Thu, Feb 18, 2021 at 6:30 PM Max Timonin  wrote:

> Hi! I've today found an issue [1], there is wrong handling of inlined POJO.
> This bug appeared in 2.9.0 and makes it impossible to work with
> multi-column indexes created on old AI versions that contain inlined POJO
> keys in the middle.
>
> I'm working on the fix and asked Konstantin Orlov to review it. I think
> that it will take only a couple of days to fix this issue. From my side, it
> looks like a release blocker, but we've been living with that since 2.9.0.
> WDYT if the release can wait for the fix?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-14206
>
> On Thu, Feb 18, 2021 at 5:38 PM Maxim Muzafarov  wrote:
>
> > Folks,
> >
> >
> > Do we have any estimations of how long does it take to fix the issue
> > [1] in C++ thin client? The bug must be fixed for sure, however, I
> > tend to continue with the release (if fixing the bug require a few
> > weeks) and prepare a batch of fixes for the 2.10.1.
> >
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-14204
> >
> > On Thu, 18 Feb 2021 at 13:22, Ilya Kasnacheev  >
> > wrote:
> > >
> > > Hello!
> > >
> > > There was a ticket filed about the new feature, transactions in C++
> thin
> > > client, making this feature unusable if there's more than one
> connection
> > > endpoint:
> > > https://issues.apache.org/jira/browse/IGNITE-14204
> > >
> > > I don't think this is a genuine blocker for 2.10, since there is a
> > > workaround (use just one endpoint), nevertheless it is a critical
> crasher
> > > bug, so I wonder if Igor could take a look at it promptly, include the
> > fix
> > > in 2.10.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > чт, 18 февр. 2021 г. в 09:03, Zhenya Stanilovsky
> >  > > >:
> > >
> > > >
> > > >
> > > > I fill the ticket with drop problem, plz take a look [1]
> > > >
> > > > [1]  https://issues.apache.org/jira/browse/IGNITE-14205
> > > >
> > > > >Ilya,
> > > > >
> > > > >Thanks!
> > > > >I've added this step to the Release Process wiki page also [1].
> > > > >
> > > > >[1]
> > > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P1.2ImplementationandScopeFinalization
> > > > >
> > > > >On Wed, 17 Feb 2021 at 18:05, Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com
> > > > > wrote:
> > > > >>
> > > > >> Hello!
> > > > >>
> > > > >> I have added ignite-2.10 to Nightly build triggering. I hope we
> will
> > > > have a
> > > > >> 2.10 nightly tomorrow. Shame that I didn't do it earlier.
> > > > >>
> > > >
> >
> https://ci.ignite.apache.org/buildConfiguration/Releases_NightlyRelease_RunApacheIgniteNightlyRelease?branch=ignite-2.10=overview=builds
> > > > >>
> > > > >> Regards,
> > > > >> --
> > > > >> Ilya Kasnacheev
> > > > >>
> > > > >>
> > > > >> ср, 17 февр. 2021 г. в 17:37, Maxim Muzafarov < mmu...@apache.org
> > >:
> > > > >>
> > > > >> > Folks,
> > > > >> >
> > > > >> > There is a possible issue with the performance for 2.9.1 vs
> 2.10.
> > I'm
> > > > >> > trying to reproduce on a stress-testing environment.
> > > > >> > [1]
> > > > >> >
> > > >
> >
> https://cwiki.apache.org/confluence/download/attachments/165224767/atomicPutRandomValueFullSync.jpg?api=v2
> > > > >> >
> > > > >> > On Fri, 12 Feb 2021 at 20:46, Maxim Muzafarov <
> mmu...@apache.org
> > >
> > > > wrote:
> > > > >> > >
> > > > >> > > Folks,
> > > > >> > >
> > > > >> > > I'm going to cherry-pick these issues to the release branch,
> any
> > > > >> > objections?
> > > > >> > >
> > > > >> > >
> > > > >> > > Checkpointer thread holds write lock too long
> > > > >> > >  https://issues.apache.org/jira/browse/IGNITE-14140
> > > > >> > >
> > > > >> > > Incorrect initialize checkpoint-runner-cpu thread pool
> > > > >> > >  https://issues.apache.org/jira/browse/IGNITE-14139
> > > > >> > >
> > > > >> > > On Wed, 10 Feb 2021 at 21:59, Maxim Muzafarov <
> > mmu...@apache.org
> > > > > wrote:
> > > > >> > > >
> > > > >> > > > Folks,
> > > > >> > > >
> > > > >> > > > Do we need any other critical issues from the master branch
> > that
> > > > need
> > > > >> > > > to be cherry-picked picked from the master branch? I've
> > marked the
> > > > >> > > > latest select issues with patching version 2.10.
> > > > >> > > >
> > > > >> > > > - benchmarks completed (I'll do another one prior to
> > preparing rc)
> > > > >> > > > - the release notes merged
> > > > >> > > > - cherry-picked issue (IGNITE-14073 Fixed transactions
> > failover)
> > > > >> > > > - most of the documentation pages also merged
> > > > >> > > >
> > > > >> > > > Hopefully, by Friday the 12th everything will be ready for
> the
> > > > >> > > > preparation of a release candidate.
> > > > >> > > >
> > > > >> > > > On Tue, 9 Feb 2021 at 05:09, Никита Сафонов <
> > > > vlasovpavel2...@gmail.com >
> > > > >> > wrote:
> > > > >> > > > >
> > > > >> 

[jira] [Created] (IGNITE-14174) Python thin: returns the time value + the client's time zone, instead of the time value from the database.

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-14174:


 Summary: Python thin: returns the time value + the client's time 
zone, instead of the time value from the database.
 Key: IGNITE-14174
 URL: https://issues.apache.org/jira/browse/IGNITE-14174
 Project: Ignite
  Issue Type: Bug
Reporter: Igor Sapego


The server is located in Europe, the client is in the Moscow time zone:
{code:python}
from pyignite import Client
from datetime import datetime

c = Client(username='', password='', use_ssl=False)
c.connect('35.158.109.154', 10800)
c.sql('create table test(key int primary key, date datetime)')
current_time = datetime.now()
c.sql(f"insert into test (key, date) VALUES (1, '{current_time}')")
for row in c.sql('SELECT date FROM test'):
print(current_time, row[0][0])
c.close()
{code}
output:

{code:python}
2021-01-20 12:28:44.850381 2021-01-20 15:28:44.85
{code}





--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-14162) Python thin gives ambiguous message when using a wrong username/password to connect

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-14162:


 Summary: Python thin gives ambiguous message when using a wrong 
username/password to connect
 Key: IGNITE-14162
 URL: https://issues.apache.org/jira/browse/IGNITE-14162
 Project: Ignite
  Issue Type: Improvement
  Components: python, thin client
Affects Versions: python-0.3.4
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: python-0.4.0


Starting a server w/authentication, then connecting to it via a python thin 
client w/an incorrect username and password.

The message/stacktrace is ambiguous leading clients to believe that they've 
connected, and something is amiss w/the server.

{code:python}client = Client(username='incorrect-user-name', 
password='incorrect-password', use_ssl=False)
client.connect('127.0.0.1', 10800)


#client(nodes)
print("connected to: ".format(client))
print("cache names: ".format(client.get_cache_names)){code}

the stacktrace is :

{code:python}connected to: 
Traceback (most recent call last):
cache names: 
  File "C:/dev/python-thin-client/examples/get_and_put.py", line 30, in 
print(client.get_cache_names())
  File "C:\dev\python-thin-client\pyignite\utils.py", line 249, in ste_wrapper
result = fn(*args, **kwargs)
  File "C:\dev\python-thin-client\pyignite\client.py", line 521, in 
get_cache_names
return cache_get_names(self.random_node)
  File "C:\dev\python-thin-client\pyignite\utils.py", line 273, in wrapper
return func(obj, *args, **kwargs)
  File "C:\dev\python-thin-client\pyignite\client.py", line 222, in random_node
raise ReconnectError('Can not reconnect: out of nodes.') from None
pyignite.exceptions.ReconnectError: Can not reconnect: out of nodes.{code}


*ReconnectError: Can not reconnect: out of nodes exception*: allows the  user 
to make the conclusion that the connection is successful, but something is 
wrong with the server.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-14127) Python Thin: increase default query page size

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-14127:


 Summary: Python Thin: increase default query page size
 Key: IGNITE-14127
 URL: https://issues.apache.org/jira/browse/IGNITE-14127
 Project: Ignite
  Issue Type: Improvement
  Components: python, thin client
Affects Versions: python-0.3.4
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: python-0.4.0


Python has query cursor page size set to 1 by default, which is inefficient. 
Other clients use 1024.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: IEP-66: Thin Client Automatic Binary Configuration

[Igor Sapego]

Guys,

I agree with your concerns about backward compatibility, but
I think this can be a good feature that will help us to add support
for compact footer to thin clients that do not support it currently.

Best Regards,
Igor


On Tue, Feb 2, 2021 at 11:49 AM Pavel Tupitsyn  wrote:

> Alex,
>
> This is a very good point, thank you for the heads up.
>
> The proposal does not include changing any existing behavior,
> we simply add an operation to the protocol, so there is
> no compatibility risk from the server-side changes alone.
>
> However, it looks like only .NET thin client can make use
> of the compactFooter flag, since it is true by default there,
> and changing from true to false is safe.
>
> Additionally, we can't use the nameMapper value even in .NET,
> since it changes the default behavior and potentially breaks
> compatibility on existing clusters. All we can do is log a warning.
>
> I've updated the proposal accordingly.
>
>
> On Tue, Feb 2, 2021 at 9:50 AM Alex Plehanov 
> wrote:
>
> > Hello Pavel,
> >
> > Changing the "compact footer" property for an existing database can lead
> to
> > compatibility issues. Some clients (at least java) have "false" as the
> > default value of the "compact footer" property, but the default value for
> > the server-side is "true". If thin client will receive binary
> configuration
> > from the server it will be unable to get values using user-defined type
> as
> > a key inserted by previous versions of the client (see [1]).
> > Also, AFAIK, some clients still don't have compact footer support.
> > So we should think carefully about how to provide compatibility using
> this
> > feature (for example, enable this feature only for clusters with no data
> > inserted into caches by previous versions of thin clients)
> >
> > [1]: https://issues.apache.org/jira/browse/IGNITE-10960
> >
> > пн, 1 февр. 2021 г. в 19:33, Pavel Tupitsyn :
> >
> > > Igniters,
> > >
> > > Please review the IEP [1] and let me know your thoughts.
> > >
> > > [1]
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-66%3A+Thin+Client+Automatic+Binary+Configuration
> > >
> >
>

[jira] [Created] (IGNITE-14099) CPP: Remove 32-bit configs

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-14099:


 Summary: CPP: Remove 32-bit configs
 Key: IGNITE-14099
 URL: https://issues.apache.org/jira/browse/IGNITE-14099
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Affects Versions: 2.9.1
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.11


32-bit test config files are not needed anymore, as we do not run them anyway. 
Remove them.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-14075) Python client incorrect hash code calculation for classes as composite keys

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-14075:


 Summary: Python client incorrect hash code calculation for classes 
as composite keys
 Key: IGNITE-14075
 URL: https://issues.apache.org/jira/browse/IGNITE-14075
 Project: Ignite
  Issue Type: Bug
  Components: python, thin client
Affects Versions: python-0.3.4
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: python-0.4.0


Python code calculates hash codes for simple binary objects incorrectly, such 
as ones containing just int and string.

Leading to possibility of putting same key twice with same key column values, 
but different hash code, visible as two rows, and also impossibility of getting 
entries populated via SQL.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-14059) Thin Python client doesn't work with nested complex objects

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-14059:


 Summary: Thin Python client doesn't work with nested complex 
objects
 Key: IGNITE-14059
 URL: https://issues.apache.org/jira/browse/IGNITE-14059
 Project: Ignite
  Issue Type: Bug
  Components: python, thin client
Affects Versions: python-0.3.4
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: python-0.4.0
 Attachments: main.py

Reproducer attached.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-14058) Python Thin client: get boolean type return integers

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-14058:


 Summary: Python Thin client: get boolean type return integers
 Key: IGNITE-14058
 URL: https://issues.apache.org/jira/browse/IGNITE-14058
 Project: Ignite
  Issue Type: Bug
  Components: python, thin client
Affects Versions: python-0.3.4
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: python-0.4.0


Steps:
- put boolean value in cluster
{code}
from pyignite import Client
from pyignite.datatypes import *
client = Client()
client.connect('127.0.0.1', 10800)
cache = client.get_or_create_cache("test")
cache.put(1, [True, False], key_hint=IntObject, value_hint=BoolArrayObject)
{code}
- get value
{code}
from pyignite import Client
from pyignite.datatypes import *
client = Client()
client.connect('127.0.0.1', 10800)
cache = client.get_or_create_cache("test")
result = cache.get(1, key_hint=IntObject)
print(*[str(arr_item).lower() for arr_item in result])
{code}

Actual:
- return integers
{code}
0 1
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-14057) Python thin: implement support for big-endianness

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-14057:


 Summary: Python thin: implement support for big-endianness
 Key: IGNITE-14057
 URL: https://issues.apache.org/jira/browse/IGNITE-14057
 Project: Ignite
  Issue Type: Bug
  Components: python, thin client
Affects Versions: python-0.3.4
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: python-0.4.0


Currently, big endian byte order is not supported in Python Thin client. Need 
to implement.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-14056) Python thin: Readme and other docs are outdated

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-14056:


 Summary: Python thin: Readme and other docs are outdated
 Key: IGNITE-14056
 URL: https://issues.apache.org/jira/browse/IGNITE-14056
 Project: Ignite
  Issue Type: Bug
  Components: python, thin client
Reporter: Igor Sapego
Assignee: Igor Sapego


The readme files for thin clients should be updated accordingly as now they're 
separate products be following rules:
* readme is located in root directory
* how to install from zip
* how to upgrade if a previous version installed




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Python thin client development approach.

[Igor Sapego]

Ivan,

Though I generally agree with the approach you've suggested, I can see
a problem here. Since we now have a separate repos for thin clients, for
some features we may need to introduce changes to Ignite and python-thin
repos in a single ticket and we should have an ability to run tests on with
changes on both python client and server nodes. Current TC suites provide
such ability, Travis does not. So I believe, it's too soon to abandon TC
for thin
clients, at least until we could solve the issue I've described.

Best Regards,
Igor


On Fri, Dec 25, 2020 at 1:49 PM Nikolay Izhikov  wrote:

> Hello, Ivan.
>
> I’m +1 for your proposal.
>
> > 25 дек. 2020 г., в 13:14, Ivan Daschinsky 
> написал(а):
> >
> > Hi folks!
> >
> > Since we already have a separate repo for thin-clients [1], [2]
> > I'd like to propose some improvements in development process/
> >
> > 1. We should simplify and automate unit tests run for different versions
> of
> > python
> > 2. We should add travis integration per commit and pr. Tests could be run
> > against latest dockered image of ignite
> > 3. There should be ability to run tests against multiple pythons on TC
> > 4. For thin client development process, travis should be the first
> option.
> > TC suite should be used only to check that compatibility is not broken
> > and when new functionality is developed (rare case).
> >
> > I've prepared fix [3], you can see successful builds for travis. It uses
> > tox [5], the most common tool to run tests in multiple environments.
> > There are few environments set up in tox.ini -- with and without docker,
> > with or without ssl, etc. This helped a lot
> > to setup travis CI build (you can see in commits list of PR) and simplify
> > run tests for developers. Also docker-compose was introduced
> > to help python thin client developers.
> >
> > But  I need some assistance for TC:
> > 1. There is outdated python setuptools on TC agents, so tests cannot be
> run
> > with updated pytest etc.
> > 2. Multiple pythons should be installed on TC agents (via
> > https://github.com/pyenv/pyenv), latest minor versions
> > for 3.6, 3.7 and 3.8
> > 3. After that, TC job should be changed to utilize tox
> >
> > WDYT about this initiative?
> >
> >
> > [1] -- https://issues.apache.org/jira/browse/IGNITE-13767
> > [2] -- https://github.com/apache/ignite-python-thin-client
> > [3] -- https://issues.apache.org/jira/browse/IGNITE-13903
> > [4] --
> https://github.com/apache/ignite-python-thin-client/pull/1/commits
> > [5] -- https://tox.readthedocs.io/en/latest/
> >
> > --
> > Sincerely yours, Ivan Daschinskiy
>
>

[jira] [Created] (IGNITE-13997) CPP Thin: Transactions can cause deadlock when client shared in multiple threads

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13997:


 Summary: CPP Thin: Transactions can cause deadlock when client 
shared in multiple threads
 Key: IGNITE-13997
 URL: https://issues.apache.org/jira/browse/IGNITE-13997
 Project: Ignite
  Issue Type: Bug
  Components: thin client
Affects Versions: 2.9.1
Reporter: Igor Sapego


In our tests we have detected a deadlock when following piece of code is
executed for more than one thread on our application:

{code:cpp}
ClientTransactions transactions = client.ClientTransactions();
ClientTransaction tx = transactions.TxStart(PESSIMISTIC, READ_COMMITTED);

// This call should atomically get the current value for "key" and put
"value" instead, locking the "key" cache entry at the same time
auto oldValue = cache.GetAndPut(key, value);

// Only the thread able of locking "key" should reach this code. Others have
to wait for tx.Commit() to complete
cache.Put (key, newValue);

// After this call, other thread waiting in GetAndPut for "key" to be
released should be able of continuing
tx.Commit ();
{code:cpp}

The thread reaching "cache.Put (key, newValue);" call, gets blocked in
there, concretely in the lockGuard object created at the beginning of
DataChannel::InternalSyncMessage function (data_channel.cpp:108).  After
debugging, we realized that this lockGuard is owned by a different thread,
which is currently waiting on socket while executing GetAndPut function.
According to this, my guess is that data routing for C++ Thin Clients is not
multithread friendly.  

Reported in 
http://apache-ignite-users.70518.x6.nabble.com/Multithread-transactions-in-a-C-Thin-Client-td35145.html



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSSION] .Net BinaryTypes transparency

[Igor Sapego]

Agree with Pavel, this should be disabled by default.

To me it looks pretty dangerous as users do not explicitly control what's
going to be
registered and it could lead to hard-to-debug mistakes when wrong classes
get
registered or with wrong names. Also it can be hard to use with classes
that should
not be registered at all, which is often the case because not everyone use
.NET client
with Java services.

To me it looks familiar to another of our features - peer class loading,
which is disabled
by default for similar reasons.

Also, why register types which are used as method arguments for any
service? Wouldn't
it be more reasonable to only register those which are used with Java
services?

Best Regards,
Igor


On Tue, Jan 12, 2021 at 7:35 PM Pavel Tupitsyn  wrote:

> Nikolay,
>
> I think this behavior should be opt-in - let's add a flag to
> BinaryConfiguration.
> Registering every .NET type as a Java type can lead to typeId collisions
> and break existing user code,
> so we can't enable this by default.
>
>
> Ignite stores typeId -> className mappings separately for Java and .NET
> [1].
> Separate maps were introduced because C# and Java have different naming
> conventions,
> typically you have org.foo.bar.Person in Java and Foo.Bar.Person in .NET,
> so we allow the users to map two different type names to the same typeId
> with a custom name or id mapper.
>
> This proposal says we should always put Foo.Bar.Person from .NET to the
> Java mapping,
> in the hope that there is a class with that name in Java, which is a very
> rare use case.
>
> [1]
> org.apache.ignite.internal.MarshallerContextImpl#registerClassName(byte,
> int, java.lang.String, boolean)
>
> On Tue, Jan 12, 2021 at 4:56 PM Nikolay Izhikov 
> wrote:
>
> > Hello, Igniters.
> >
> > Currently, in case of usage .Net platform client it’s required for the
> > user to explicitly register each custom type.
> >
> > ```
> > _marsh = new Marshaller(new BinaryConfiguration
> > {
> > TypeConfigurations = new List
> > {
> > new BinaryTypeConfiguration(typeof (Address)),
> > new BinaryTypeConfiguration(typeof (TestModel))
> > }
> > });
> > ```
> >
> > Note, we have a special public interface to map java type to .net types -
> > `IBinaryNameMapper`
> >
> > ```
> > public interface IBinaryNameMapper
> > {
> > string GetTypeName(string name);
> > string GetFieldName(string name);
> > }
> > }
> > ```
> >
> > Users found this approach annoying and not convenient.
> > So, recently we made a several improvements for the Service API to
> > implicitly register each custom type from service method arguments. [1],
> > [2], [3]
> > For now, user can just call Java service from .Net without any additional
> > configuration in case FQN NameMapper used.
> >
> > I propose to extends approach with the implicit binary type registration
> > to all APIs and prepare PR for it [4].
> >
> > What do you think?
> >
> > [1]
> >
> https://github.com/apache/ignite/commit/35f551c023a12b3570d65c803d10c89480f7d5e4
> > [2]
> >
> https://github.com/apache/ignite/commit/6d02e32e7f049e4f78f7abd37f4ff91d77f738c2
> > [3]
> >
> https://github.com/apache/ignite/commit/c2204cda29e70294cc93756eabc844b64e07a42e
> > [4] https://github.com/apache/ignite/pull/8635
>

Re: [DISCUSS] IEP-65: .NET Examples Modernization

[Igor Sapego]

Looks like a great improvement, +1 from me

Best Regards,
Igor


On Tue, Jan 12, 2021 at 4:43 PM Pavel Tupitsyn  wrote:

> Igniters,
>
> Our .NET examples are in need of a refreshment.
> I've prepared an IEP [1], please let me know what you think.
>
> Thanks,
> Pavel
>
>
> [1]
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-65%3A+.NET+Examples+Modernization
>

[jira] [Created] (IGNITE-13909) Node.js client glob dependency is missing

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13909:


 Summary: Node.js client glob dependency is missing
 Key: IGNITE-13909
 URL: https://issues.apache.org/jira/browse/IGNITE-13909
 Project: Ignite
  Issue Type: Bug
  Components: thin client
Reporter: Igor Sapego
Assignee: Igor Sapego


This results in following errors:
{noformat}
Suite error: sql query test suite >
  Message:
Failed: Cannot find module 'glob'
  Stack:
error properties: Object({ code: 'MODULE_NOT_FOUND' })
Error: Cannot find module 'glob'
at Function.Module._resolveFilename 
(internal/modules/cjs/loader.js:636:15)
at Function.Module._load (internal/modules/cjs/loader.js:562:25)
at Module.require (internal/modules/cjs/loader.js:692:17)
at require (internal/modules/cjs/helpers.js:25:18)
at Function.getLogFiles 
(D:\Development\git\ignite-nodejs-thin-client\spec\TestingHelper.js:478:22)
at Function.clearLogs 
(D:\Development\git\ignite-nodejs-thin-client\spec\TestingHelper.js:486:39)
at Function._startNode 
(D:\Development\git\ignite-nodejs-thin-client\spec\TestingHelper.js:491:23)
at Function.startTestServer 
(D:\Development\git\ignite-nodejs-thin-client\spec\TestingHelper.js:350:57)
at Function.startTestServers 
(D:\Development\git\ignite-nodejs-thin-client\spec\TestingHelper.js:340:33)
at Function.init 
(D:\Development\git\ignite-nodejs-thin-client\spec\TestingHelper.js:211:29)
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13908) ODBC driver should show real nullability info

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13908:


 Summary: ODBC driver should show real nullability info
 Key: IGNITE-13908
 URL: https://issues.apache.org/jira/browse/IGNITE-13908
 Project: Ignite
  Issue Type: Improvement
  Components: odbc
Affects Versions: 2.9
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.10


Currently, it always return SQL_NULLABLE_UNKNOWN on request of nullability type 
of the column.

Let's provide that information to the user of ODBC driver



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13907) CPP thin: allow user to limit number of max active connections per client

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13907:


 Summary: CPP thin: allow user to limit number of max active 
connections per client
 Key: IGNITE-13907
 URL: https://issues.apache.org/jira/browse/IGNITE-13907
 Project: Ignite
  Issue Type: New Feature
  Components: thin client
Affects Versions: 2.9
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.10


Thin clients now can establish connections with multiple servers 
simultaneously. It is implemented this way to make partition awareness [1] work 
or for fast failover if partition awareness is not used. However, sometimes it 
can create excessive load for cluster in use cases when there are a lot of thin 
clients. We should give user a possibility to limit maximum number of active 
connections established by a client.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Limiting number of active connection for thin client

[Igor Sapego]

Hi Igniters,

As you may know thin clients now can establish connections with
multiple servers simultaneously. It is implemented this way to make
partition awareness [1] work or for fast failover if partition awareness
is not used. However, sometimes it can create excessive load for
cluster in use cases when there are a lot of thin clients. I think, we
should give user a possibility to limit maximum number of active
connections established by a client. What do you think?

[1] -
https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients

Best Regards,
Igor

[jira] [Created] (IGNITE-13900) CPP: Fix test flaky Affinity tests

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13900:


 Summary: CPP: Fix test flaky Affinity tests
 Key: IGNITE-13900
 URL: https://issues.apache.org/jira/browse/IGNITE-13900
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Affects Versions: 2.9
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.10


Some CPP affinity tests are flaky on TC. For example, 
IgniteAffinityMapPartitionsToNodes, IgniteAffinityCall, 
IgniteAffinityCallAsync. Need to fix them.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13863) Python thin client hangs when object has Boolean field

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13863:


 Summary: Python thin client hangs when object has Boolean field
 Key: IGNITE-13863
 URL: https://issues.apache.org/jira/browse/IGNITE-13863
 Project: Ignite
  Issue Type: Bug
  Components: thin client
Affects Versions: 2.9
Reporter: Igor Sapego
Assignee: Igor Sapego


The first run of python client returns empty result set, all consequence runs 
just hang:

{code:java}
from pygridgain import Client, GenericObjectMeta
import collections
from pygridgain.datatypes import  *

if __name__ == '__main__':
client = Client()
client.connect('localhost', 10800)
cache = client.get_cache('Batch')
#client.register_binary_type(BatchConfigurationPK)
#client.register_binary_type(BatchConfiguration)
result = cache.scan()
print([b for b in result])
{code}


{code:java}

public class ObjectTest {

  @QuerySqlField
  private String field1;

  @QuerySqlField
  private String field2;

  @QuerySqlField
  private String field3;

  @QuerySqlField
  private String field4;

  @QuerySqlField
  private Boolean enabled;


  public ObjectTest(String field1, String field2, String field3) {
this.field1 = field1;
this.field2 = field2;
this.field3 = field3;
  }
}
{code}

{code:java}

public static void main(String[] args) throws IgniteException {
Ignite ignite = Ignition.start("examples/config/example-ignite.xml");

IgniteCache cache = ignite.getOrCreateCache("Batch");

for (int i = 0; i < 100; i++) {
cache.put(i, new ObjectTest("" + i, "" + i, "" + i));
}
}
{code}





--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Migrating NodeJS client to TypeScript

[Igor Sapego]

Cool, I like the idea.
You've got a +1 from me

Best Regards,
Igor


On Tue, Dec 1, 2020 at 12:20 PM Данилов Семён  wrote:

> Yes, TS compiler produces JS files and TS-typings. Users that are already
> using JS version will have a seamless migration. Note: we'll have to
> publish compiled JS files along with typings.
>
> -
> Semyon.
>
> 01.12.2020, 09:59, "Ivan Daschinsky" :
> >>  Is there any value in keeping both versions - the plain JavaScript one
> >
> > and the TypeScript specific
> > Hi! No, there is no value. TS sources should be transpiled to JS before
> > execution.
> >
> > вт, 1 дек. 2020 г. в 01:31, Denis Magda :
> >
> >>  Hi Semyon,
> >>
> >>  Is there any value in keeping both versions - the plain JavaScript one
> and
> >>  the TypeScript specific?
> >>
> >>  -
> >>  Denis
> >>
> >>  On Mon, Nov 30, 2020 at 12:16 PM Данилов Семён 
> wrote:
> >>
> >>  > Hello Igniters!
> >>  >
> >>  > I'd like to propose a big change for the nodejs thin client: a full
> >>  > rewrite from JavaScript to TypeScript.
> >>  >
> >>  > Strong typing will not only help us in future while developing new
> >>  > features, but will also indicate existing type inconsistencies (I
> know
> >>  > there is a couple).
> >>  > Also, we will have out of the box types for API documentation, which
> is
> >>  > very handy for users.
> >>  >
> >>  > So what do you think?
> >>  >
> >>  > Kind regards,
> >>  > Semyon.
> >>  >
> >
> > --
> > Sincerely yours, Ivan Daschinskiy
>

[jira] [Created] (IGNITE-13825) Decimal columns in SQL result set have invalid precision and scale

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13825:


 Summary: Decimal columns in SQL result set have invalid precision 
and scale
 Key: IGNITE-13825
 URL: https://issues.apache.org/jira/browse/IGNITE-13825
 Project: Ignite
  Issue Type: Improvement
  Components: sql
Affects Versions: 2.9
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.10


If the SQL result set of contains Decimal column it now returns MAX_SHORT as 
precision and MAX_USHORT as scale, no matter what is the precision and scale of 
the original table column.

SQL:
{code:sql}
create table person(id int, name character(10), age decimal(3,0), primary key 
(id));
{code}

Java (from internal component)
{code:java}
GridQueryFieldMetadata meta = kernal.query().getIndexing().resultMetaData(
"PUBLIC", "select age from person;"
).iterator().next();

assert meta.precision() == 3;
assert meta.scale() == 0;
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Use GridNioServer in Java thin client

[Igor Sapego]

Agree. Great job.

Best Regards,
Igor


On Thu, Nov 26, 2020 at 3:10 PM Ivan Daschinsky  wrote:

> Pavel, good job and great benchmark results!
>
> чт, 26 нояб. 2020 г. в 15:01, Pavel Tupitsyn :
>
> > PR is ready for review [1]
> >
> > I've added a simple put/get benchmark, there is some performance
> > improvement over existing implementation, see results in the PR
> > description.
> >
> > [1] https://github.com/apache/ignite/pull/8483
> >
> > On Fri, Nov 20, 2020 at 10:39 AM Pavel Tupitsyn 
> > wrote:
> >
> > > Since there are no objections, I've updated the IEP accordingly [1]
> > > and started working on it [2]
> > >
> > > [1]
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-60%3A+Java+Thin+Client+Non-Blocking+Async+IO
> > > [2] https://github.com/apache/ignite/pull/8483
> > >
> > > On Mon, Nov 9, 2020 at 4:07 PM Ivan Daschinsky 
> > > wrote:
> > >
> > >> I suppose that the best variant -- ability to switch to netty if this
> > lib
> > >> is in classpath
> > >>
> > >> пн, 9 нояб. 2020 г. в 15:58, Igor Sapego :
> > >>
> > >> > Sounds like a good idea to me.
> > >> >
> > >> > Best Regards,
> > >> > Igor
> > >> >
> > >> >
> > >> > On Mon, Nov 9, 2020 at 3:32 PM Alex Plehanov <
> plehanov.a...@gmail.com
> > >
> > >> > wrote:
> > >> >
> > >> > > +1 for using GridNioServer as java thin client communication
> layer.
> > >> > >
> > >> > > вс, 8 нояб. 2020 г. в 19:12, Pavel Tupitsyn  >:
> > >> > >
> > >> > > > Igniters,
> > >> > > >
> > >> > > > This is a continuation of "Use Netty for Java thin client" [1],
> > >> > > > I'm starting a new thread for better visibility.
> > >> > > >
> > >> > > > The problems with current Java thin client are:
> > >> > > > * Socket writes block user threads
> > >> > > > * Every connection uses a separate listener thread (with
> partition
> > >> > > > awareness there is a thread for every server node within a
> single
> > >> > > > IgniteClient)
> > >> > > >
> > >> > > > GridNioServer can work in client mode and solves both of these
> > >> > problems.
> > >> > > > It is the most practical choice as well at the moment - no extra
> > >> > > > dependencies required.
> > >> > > >
> > >> > > > A potential drawback is increased coupling between thin client
> and
> > >> core
> > >> > > > code,
> > >> > > > which I'm going to mitigate by abstracting GridNioServer behind
> a
> > >> > simpler
> > >> > > > facade,
> > >> > > > so we can replace it with Netty or something else easier if we
> > >> decide
> > >> > to
> > >> > > > split the code.
> > >> > > >
> > >> > > > Thoughts, objections?
> > >> > > >
> > >> > > > [1]
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Use-Netty-for-Java-thin-client-td49732.html
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >>
> > >> --
> > >> Sincerely yours, Ivan Daschinskiy
> > >>
> > >
> >
>
>
> --
> Sincerely yours, Ivan Daschinskiy
>

[jira] [Created] (IGNITE-13801) ODBC: Check ODBC driver with Ab Initio and fix all issues

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13801:


 Summary: ODBC: Check ODBC driver with Ab Initio and fix all issues
 Key: IGNITE-13801
 URL: https://issues.apache.org/jira/browse/IGNITE-13801
 Project: Ignite
  Issue Type: Improvement
  Components: odbc
Affects Versions: 2.9
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.10


I want to check our ODBC driver with Ab Initio and fix issues if there are any.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13793) ODBC: Implement SQLRowCount for select queries

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13793:


 Summary: ODBC: Implement SQLRowCount for select queries
 Key: IGNITE-13793
 URL: https://issues.apache.org/jira/browse/IGNITE-13793
 Project: Ignite
  Issue Type: Improvement
  Components: odbc
Affects Versions: 2.9
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.10


SQLRowCount() should return estimation of row count of the query. Currently, we 
can not provide any estimations of this kind, but we still need to implement 
this function as sometimes some third-party software uses it. It is proposed to 
use some kind of constant value, e.g. page size.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13774) Create TC suites for release Python Node.js and PHP clients

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13774:


 Summary: Create TC suites for release Python Node.js and PHP 
clients
 Key: IGNITE-13774
 URL: https://issues.apache.org/jira/browse/IGNITE-13774
 Project: Ignite
  Issue Type: Improvement
  Components: thin client
Affects Versions: 2.9
 Environment: Create TC suites for release of new clients
Reporter: Igor Sapego
 Fix For: 2.10


We now have separate repos for Python, Node.js and PHP thin clients. Need to 
create a TC suites to release them as a separate packages on Ignite site and in 
pip, npm and composer.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13771) ODBC crashes on Linux if SQLConnect is called

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13771:


 Summary: ODBC crashes on Linux if SQLConnect is called
 Key: IGNITE-13771
 URL: https://issues.apache.org/jira/browse/IGNITE-13771
 Project: Ignite
  Issue Type: Bug
  Components: odbc
Affects Versions: 2.9
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.10


According to user report the following line 

{code:cpp}

et = SQLConnect(dbc, (SQLCHAR*)DSN, SQL_NTS, (SQLCHAR*)"", SQL_NTS,
(SQLCHAR*)"", SQL_NTS);

{code:cpp}

Gives the following result:
{noformat}
symbol lookup error: /usr/local/lib/libignite-odbc.so: undefined
symbol: SQLGetPrivateProfileString {noformat}
 

Details can be found on userlist: 
https://lists.apache.org/thread.html/r023842d3fa08d9e7c6e8dd6686efaa85b46ea2b14dd22883eecfbe17%40



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13767) Remove Python PHP and Node.js thin clients from main repository

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13767:


 Summary: Remove Python PHP and Node.js thin clients from main 
repository
 Key: IGNITE-13767
 URL: https://issues.apache.org/jira/browse/IGNITE-13767
 Project: Ignite
  Issue Type: Improvement
  Components: thin client
Affects Versions: 2.9
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.10


Need to remove the following directories, as we now have separate repos for 
Python, Node.js and PHP thin clients:

modules/platforms/python

modules/platforms/nodejs

modules/platforms/php

 

Also, need to check all the places in code where those directories are 
referenced.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: IEP-54: Schema-first approach for 3.0

[Igor Sapego]

Actually, I think it is not so hard to implement comparison of unsigned
numbers in
SQL even in Java, so it does not seem to be a big issue from my perspective.

Now to the usage of unsigned types from Java - I think, if a user uses
unsigned type
in a schema and is going to interact with it from Java he knows what he is
doing.

Mostly they are for use from platforms where they have native support and
widely
used, like in C++ or .NET, where users currently have to make a manual type
casting
or even just stop using unsigned types when they use Ignite.

Best Regards,
Igor


On Tue, Nov 24, 2020 at 3:06 PM Pavel Tupitsyn  wrote:

> Andrey,
>
> I think it is much simpler:
> - Add protocol support for those types (basically, just add more type ids)
> - Treat uLong as long in Java (bitwise representation is the same)
>
> ANSI SQL does not have unsigned integers, so we can simply say that
> unsigned value relative comparison is not supported in SQL (equality will
> work).
>
>
> On Tue, Nov 24, 2020 at 2:40 PM Andrey Mashenkov <
> andrey.mashen...@gmail.com>
> wrote:
>
> > Thanks, Pavel and Igor.
> >
> > I like your ideas to have i8 or int8 instead of Integer.
> > But the naming doesn't address the issue.
> >
> > I agree internal types should be portable across different systems with
> and
> > without unsigned type support.
> > The only issue here is that unsigned types cover different ranges.
> >
> > Let's assume we want to introduce a uLong.
> > It doesn't look like a big deal to add uLong type support at storage
> level
> > and fit it to a 8 bytes and then use it in e.g. .Net only.
> > But how we could support it in e.g. Java?
> >
> > Let's keep in mind Long range is about (2^-63 .. 2^63) and uLong range is
> > (0 .. 2^64)
> > 1. The first option is to restrict range to (0 .. 2^63). This allows to
> use
> > signed in e.g.
> > Java with no conversion, but doesn't look like a 'real' unsigned uLong
> > support. Things go worse when the user will use uByte, as limitation can
> > make uByte totally unusable.
> >
> > 2. The second one is to map unsigned types to a type of wider type and
> add
> > a constraint for negative values. E.g. uLong to BigInteger.
> > So, we can't use primitive Java type for Long here. However, it is still
> > possible to store uLong in 8 bytes, but have a special comparator for
> > unsigned types to avoid unwanted deserialization.
> >
> > WDYT?
> >
> >
> >
> >
> >
> >
> > On Tue, Nov 24, 2020 at 2:04 PM Pavel Tupitsyn 
> > wrote:
> >
> > > Agree, let's get rid of "long, short, byte" in the protocol definition.
> > >
> > > We can use Rust style, which is concise and unambiguous:
> > > i8, u8, i16, u16, etc
> > >
> > > On Tue, Nov 24, 2020 at 1:58 PM Igor Sapego 
> wrote:
> > >
> > > > Pavel,
> > > >
> > > > I totally support that. Also, if we are aiming for
> > > > stronger platform-independance,
> > > > in our schemas we may want to support bit-notation (int32, uint64)?
> For
> > > > example
> > > > "long" can mean a different type on different platforms and it's easy
> > to
> > > > confuse
> > > > them (happens often when using ODBC for example).
> > > >
> > > > Best Regards,
> > > > Igor
> > > >
> > > >
> > > > On Tue, Nov 24, 2020 at 1:34 PM Pavel Tupitsyn  >
> > > > wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > I think we should support unsigned data types:
> > > > > uByte, uShort, uInt, uLong
> > > > >
> > > > > Java does not have them, but many other languages do,
> > > > > and with the growing number of thin clients this is important.
> > > > >
> > > > > For example, in current Ignite.NET implementation we store unsigned
> > > > values
> > > > > as signed internally,
> > > > > but this is a huge pain when it comes to metadata, binary objects,
> > etc.
> > > > > (it is easy to deserialize int as uint when you have a class, but
> not
> > > > with
> > > > > BinaryObject.GetField)
> > > > >
> > > > > Any objections?
> > > > >
> > > > > On Tue, Nov 24, 2020 at 12:28 PM Andrey Mashenkov <
> > > > > andrey.mashen...@gmail.com> wrote:
> > > > >
> > > > > > Deni

Re: IEP-54: Schema-first approach for 3.0

[Igor Sapego]

Pavel,

I totally support that. Also, if we are aiming for
stronger platform-independance,
in our schemas we may want to support bit-notation (int32, uint64)? For
example
"long" can mean a different type on different platforms and it's easy to
confuse
them (happens often when using ODBC for example).

Best Regards,
Igor


On Tue, Nov 24, 2020 at 1:34 PM Pavel Tupitsyn  wrote:

> Igniters,
>
> I think we should support unsigned data types:
> uByte, uShort, uInt, uLong
>
> Java does not have them, but many other languages do,
> and with the growing number of thin clients this is important.
>
> For example, in current Ignite.NET implementation we store unsigned values
> as signed internally,
> but this is a huge pain when it comes to metadata, binary objects, etc.
> (it is easy to deserialize int as uint when you have a class, but not with
> BinaryObject.GetField)
>
> Any objections?
>
> On Tue, Nov 24, 2020 at 12:28 PM Andrey Mashenkov <
> andrey.mashen...@gmail.com> wrote:
>
> > Denis,
> >
> > Good point. Both serializers use reflection API.
> > However, we will allow users to configure static schema along with
> 'strict'
> > schema mode, we still need to validate user classes on client nodes
> against
> > the latest schema in the grid  and reflection API is the only way to do
> it.
> > One can find a few articles on the internet on how to enable reflection
> in
> > GraalVM.
> >
> > I'll create a task for supporting GraalVM, and maybe someone who is
> > familiar with GraalVM will suggest a solution or a proper workaround. Or
> > I'll do it a bit later.
> > If no workaround is found, we could allow users to write it's own
> > serializer, but I don't think it is a good idea to expose any internal
> > classes to the public.
> >
> > On Tue, Nov 24, 2020 at 2:55 AM Denis Magda  wrote:
> >
> > > Andrey, thanks for the update,
> > >
> > > Does any of the serializers take into consideration the
> > > native-image-generation feature of GraalVM?
> > > https://www.graalvm.org/reference-manual/native-image/
> > >
> > > With the current binary marshaller, we can't even generate a native
> image
> > > for the code using our thin client APIs.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Mon, Nov 23, 2020 at 4:39 AM Andrey Mashenkov <
> > > andrey.mashen...@gmail.com>
> > > wrote:
> > >
> > > > Hi Igniters,
> > > >
> > > > I'd like to continue discussion of IEP-54 (Schema-first approach).
> > > >
> > > > Hope everyone who is interested had a chance to get familiar with the
> > > > proposal [1].
> > > > Please, do not hesitate to ask questions and share your ideas.
> > > >
> > > > I've prepared a prototype of serializer [2] for the data layout
> > described
> > > > in the proposal.
> > > > In prototy, I compared 2 approaches to (de)serialize objects, the
> first
> > > one
> > > > uses java reflection/unsafe API and similar to one we already use in
> > > Ignite
> > > > and the second one generates serializer for particular user class and
> > > uses
> > > > Janino library for compilation.
> > > > Second one shows better results in benchmarks.
> > > > I think we can go with it as default serializer and have
> > reflection-based
> > > > implementation as a fallback if someone will have issues with the
> first
> > > > one.
> > > > WDYT?
> > > >
> > > > There are a number of tasks under the umbrella ticket [3] waiting for
> > the
> > > > assignee.
> > > >
> > > > BTW, I'm going to create more tickets for schema manager modes
> > > > implementation, but would like to clarify some details.
> > > >
> > > > I thought schemaManager on each node should held:
> > > >   1. Local mapping of "schema version" <--> validated local key/value
> > > > classes pair.
> > > >   2. Cluster-wide schema changes history.
> > > > On the client side. Before any key-value API operation we should
> > > validate a
> > > > schema for a given key-value pair.
> > > > If there is no local-mapping exists for a given key-value pair or if
> a
> > > > cluster wide schema has a more recent version then the key-value pair
> > > > should be validated against the latest version and local mapping
> should
> > > be
> > > > updated/actualized.
> > > > If an object doesn't fit to the latest schema then it depends on
> schema
> > > > mode: either fail the operation ('strict' mode) or a new mapping
> should
> > > be
> > > > created and a new schema version should be propagated to the cluster.
> > > >
> > > > On the server side we usually have no key-value classes and we
> operate
> > > with
> > > > tuples.
> > > > As schema change history is available and a tuple has schema version,
> > > then
> > > > it is possible to upgrade any received tuple to the last version
> > without
> > > > desialization.
> > > > Thus we could allow nodes to send key-value pairs of previous
> versions
> > > (if
> > > > they didn't receive a schema update yet) without reverting schema
> > changes
> > > > made by a node with newer classes.
> > > >
> > > > Alex, Val, Ivan did you mean the same?
> 

[jira] [Created] (IGNITE-13737) Move PHP thin client to a separate git repo

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13737:


 Summary: Move PHP thin client to a separate git repo
 Key: IGNITE-13737
 URL: https://issues.apache.org/jira/browse/IGNITE-13737
 Project: Ignite
  Issue Type: Improvement
  Components: thin client
Affects Versions: 2.9
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.10


We now have separate repo for PHP thin client. Need to move PHP thin client 
from ignite/modules/platforms/php to 
[https://gitbox.apache.org/repos/asf/ignite-php-thin-client.git|https://gitbox.apache.org/repos/asf/ignite-python-thin-client.git]
 preserving git history.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13736) Move Node.js thin client to a separate git repo

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13736:


 Summary: Move Node.js thin client to a separate git repo
 Key: IGNITE-13736
 URL: https://issues.apache.org/jira/browse/IGNITE-13736
 Project: Ignite
  Issue Type: Improvement
  Components: thin client
Affects Versions: 2.9
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.10


We now have separate repo for Node.js thin client. Need to move Node.js thin 
client from ignite/modules/platforms/nodejs to 
[https://gitbox.apache.org/repos/asf/ignite-nodejs-thin-client.git|https://gitbox.apache.org/repos/asf/ignite-python-thin-client.git]
 preserving git history.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13735) Move Python thin client to a separate git repo

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13735:


 Summary: Move Python thin client to a separate git repo
 Key: IGNITE-13735
 URL: https://issues.apache.org/jira/browse/IGNITE-13735
 Project: Ignite
  Issue Type: Improvement
  Components: thin client
Affects Versions: 2.9
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.10


We now have separate repo for Python thin client. Need to move python thin 
client from ignite/modules/platforms/python to 
[https://gitbox.apache.org/repos/asf/ignite-python-thin-client.git] preserving 
git history.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Use GridNioServer in Java thin client

[Igor Sapego]

Sounds like a good idea to me.

Best Regards,
Igor


On Mon, Nov 9, 2020 at 3:32 PM Alex Plehanov 
wrote:

> +1 for using GridNioServer as java thin client communication layer.
>
> вс, 8 нояб. 2020 г. в 19:12, Pavel Tupitsyn :
>
> > Igniters,
> >
> > This is a continuation of "Use Netty for Java thin client" [1],
> > I'm starting a new thread for better visibility.
> >
> > The problems with current Java thin client are:
> > * Socket writes block user threads
> > * Every connection uses a separate listener thread (with partition
> > awareness there is a thread for every server node within a single
> > IgniteClient)
> >
> > GridNioServer can work in client mode and solves both of these problems.
> > It is the most practical choice as well at the moment - no extra
> > dependencies required.
> >
> > A potential drawback is increased coupling between thin client and core
> > code,
> > which I'm going to mitigate by abstracting GridNioServer behind a simpler
> > facade,
> > so we can replace it with Netty or something else easier if we decide to
> > split the code.
> >
> > Thoughts, objections?
> >
> > [1]
> >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Use-Netty-for-Java-thin-client-td49732.html
> >
>

[jira] [Created] (IGNITE-13637) ODBC: Add support of SQL_ATTR_ROW_ARRAY_SIZE with value more than one

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13637:


 Summary: ODBC: Add support of SQL_ATTR_ROW_ARRAY_SIZE with value 
more than one
 Key: IGNITE-13637
 URL: https://issues.apache.org/jira/browse/IGNITE-13637
 Project: Ignite
  Issue Type: Improvement
  Components: odbc
Affects Versions: 2.9
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.9.1


Currently, we only support fetching of result set one by one. Implement 
fetching in a batch.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13636) ODBC driver assigns SQL_BINARY type to DATE fields

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13636:


 Summary: ODBC driver assigns SQL_BINARY type to DATE fields
 Key: IGNITE-13636
 URL: https://issues.apache.org/jira/browse/IGNITE-13636
 Project: Ignite
  Issue Type: Bug
  Components: odbc
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.9.1


"The DATE types should not be reported as SQL_BINARY. They should be returned 
as SQL_DATE"
{noformat}
DATE_FIELD DataType SQL_BINARY, DecimalDigits 0, Nullable 2, ColumnSize 10, 
UnsignedNumber 1, is_output_column 
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [EXTERNAL] Re: cpp thin client vector resize

[Igor Sapego]

No problem, feel free to write again if you encounter any issues.

Best Regards,
Igor


On Tue, Sep 29, 2020 at 8:02 PM Brett Elliott  wrote:

> Hi Igor,
>
> Your request to see my Read method made me examine it a little closer. The
> particular vector resize that's the problem is my own fault. Thanks for
> making me have a look at that again.
>
> Thanks,
> Brett
>
> -Original Message-
> From: Igor Sapego [mailto:isap...@apache.org]
> Sent: Tuesday, September 29, 2020 2:22 AM
> To: dev 
> Subject: [EXTERNAL] Re: cpp thin client vector resize
>
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you recognize the sender and know
> the content is safe.
>
>
> Hi,
>
> Can you share your ignite::binary::BinaryType::Read method where reading
> of the std::vector is going on?
>
> Also, are your strings large too or only vectors?
>
> Best Regards,
> Igor
>
>
> On Mon, Sep 28, 2020 at 8:29 PM Brett Elliott 
> wrote:
>
> > Hello,
> >
> > Tl;dr: I'm doing some profiling, and the cpp thin client is spending a
> > lot of time in memset. It's spending almost as much time as in the
> > socket recv call.
> >
> > Longer version:
> >
> > I was profiling a test harness that does a Get from our ingite grid.
> > The test harness is written in c++ using the thin client. I profiled
> > the code using gperftools, and I found that memset (__memset_sse2 in
> > my case) was taking a large portion of the execution time. I'm
> > Get-ting a BinaryType which contains a std::string, an int32_t, and an
> > array of int8_t. For my test case, the array of int8_t values can
> > vary, but I got the best throughput on my machine using about 100MB for
> the size of that array.
> >
> > I profiled the test code doing a single Get, and doing 8 Gets. In the
> > 8 Get case, the number of memset calls increased, but the percentage
> > of overall time spent in memset was reduced. However it was not
> > reduced as much as I'd hoped. I was hoping that the first Get call
> > would have a large memset, and the rest of the Get calls would skip
> > it, but that's maybe not the case.
> >
> > I'm seeing almost as much time spent in memset as is being spent in
> > recv (__libc_recv in this case). That seems like a lot of time spent
> > initializing. I suspect that it's std::vector initialization caused by
> > resize.
> >
> > I believe memset is being invoked by a std::vector::resize operation
> > inside of ignite::binary::BinaryType::Read. I believe the source file
> > is
> > modules/platforms/cpp/binary/src/impl/binary/binary_reader_impl.cpp.
> > In the code I'm looking at it's line 905. There are only two calls to
> > resize in this sourcefile, and it's the one in ReadTopObject0 which I
> > think is the culprit. I didn't compile with debug symbols to confirm
> > the particular resize call, but my profiler's callstack shows that
> resize is to blame for all the memset calls.
> >
> > Is there any way we can avoid std::vector::resize? I suspect that
> > ultimately the problem is that the buffer somewhere gets passed to a
> > socket recv call, and recv call takes a pointer and length. In that
> > case, there's no way that I know of to use a std::vector for the
> > buffer and avoid the unnecessary initialization/memset in the resize
> call.
> >
> > Could another container be used instead of a vector?
> > Could the vector be reused, so on subsequent calls we don't need to
> > resize it again?
> > Could something like uvector (which skips initialization) be used
> instead?
> >
> > Thank,
> > Brett
> >
> >
> >
>

Re: cpp thin client vector resize

[Igor Sapego]

I believe the vector itself should not be a problem. So there is no point
in changing
std::vector to anything else. It is possible though that there is a problem
with code
that deals with the vector.

Also it would help though not necessary if you could provide profiling
results.

Best Regards,
Igor


On Tue, Sep 29, 2020 at 11:22 AM Igor Sapego  wrote:

> Hi,
>
> Can you share your ignite::binary::BinaryType::Read method where reading of
> the std::vector is going on?
>
> Also, are your strings large too or only vectors?
>
> Best Regards,
> Igor
>
>
> On Mon, Sep 28, 2020 at 8:29 PM Brett Elliott 
> wrote:
>
>> Hello,
>>
>> Tl;dr: I'm doing some profiling, and the cpp thin client is spending a
>> lot of time in memset. It's spending almost as much time as in the socket
>> recv call.
>>
>> Longer version:
>>
>> I was profiling a test harness that does a Get from our ingite grid. The
>> test harness is written in c++ using the thin client. I profiled the code
>> using gperftools, and I found that memset (__memset_sse2 in my case) was
>> taking a large portion of the execution time. I'm Get-ting a BinaryType
>> which contains a std::string, an int32_t, and an array of int8_t. For my
>> test case, the array of int8_t values can vary, but I got the best
>> throughput on my machine using about 100MB for the size of that array.
>>
>> I profiled the test code doing a single Get, and doing 8 Gets. In the 8
>> Get case, the number of memset calls increased, but the percentage of
>> overall time spent in memset was reduced. However it was not reduced as
>> much as I'd hoped. I was hoping that the first Get call would have a large
>> memset, and the rest of the Get calls would skip it, but that's maybe not
>> the case.
>>
>> I'm seeing almost as much time spent in memset as is being spent in recv
>> (__libc_recv in this case). That seems like a lot of time spent
>> initializing. I suspect that it's std::vector initialization caused by
>> resize.
>>
>> I believe memset is being invoked by a std::vector::resize operation
>> inside of ignite::binary::BinaryType::Read. I believe the source file is
>> modules/platforms/cpp/binary/src/impl/binary/binary_reader_impl.cpp. In the
>> code I'm looking at it's line 905. There are only two calls to resize in
>> this sourcefile, and it's the one in ReadTopObject0 which I think is the
>> culprit. I didn't compile with debug symbols to confirm the particular
>> resize call, but my profiler's callstack shows that resize is to blame for
>> all the memset calls.
>>
>> Is there any way we can avoid std::vector::resize? I suspect that
>> ultimately the problem is that the buffer somewhere gets passed to a socket
>> recv call, and recv call takes a pointer and length. In that case, there's
>> no way that I know of to use a std::vector for the buffer and avoid the
>> unnecessary initialization/memset in the resize call.
>>
>> Could another container be used instead of a vector?
>> Could the vector be reused, so on subsequent calls we don't need to
>> resize it again?
>> Could something like uvector (which skips initialization) be used instead?
>>
>> Thank,
>> Brett
>>
>>
>>

Re: cpp thin client vector resize

[Igor Sapego]

Hi,

Can you share your ignite::binary::BinaryType::Read method where reading of
the std::vector is going on?

Also, are your strings large too or only vectors?

Best Regards,
Igor


On Mon, Sep 28, 2020 at 8:29 PM Brett Elliott  wrote:

> Hello,
>
> Tl;dr: I'm doing some profiling, and the cpp thin client is spending a lot
> of time in memset. It's spending almost as much time as in the socket recv
> call.
>
> Longer version:
>
> I was profiling a test harness that does a Get from our ingite grid. The
> test harness is written in c++ using the thin client. I profiled the code
> using gperftools, and I found that memset (__memset_sse2 in my case) was
> taking a large portion of the execution time. I'm Get-ting a BinaryType
> which contains a std::string, an int32_t, and an array of int8_t. For my
> test case, the array of int8_t values can vary, but I got the best
> throughput on my machine using about 100MB for the size of that array.
>
> I profiled the test code doing a single Get, and doing 8 Gets. In the 8
> Get case, the number of memset calls increased, but the percentage of
> overall time spent in memset was reduced. However it was not reduced as
> much as I'd hoped. I was hoping that the first Get call would have a large
> memset, and the rest of the Get calls would skip it, but that's maybe not
> the case.
>
> I'm seeing almost as much time spent in memset as is being spent in recv
> (__libc_recv in this case). That seems like a lot of time spent
> initializing. I suspect that it's std::vector initialization caused by
> resize.
>
> I believe memset is being invoked by a std::vector::resize operation
> inside of ignite::binary::BinaryType::Read. I believe the source file is
> modules/platforms/cpp/binary/src/impl/binary/binary_reader_impl.cpp. In the
> code I'm looking at it's line 905. There are only two calls to resize in
> this sourcefile, and it's the one in ReadTopObject0 which I think is the
> culprit. I didn't compile with debug symbols to confirm the particular
> resize call, but my profiler's callstack shows that resize is to blame for
> all the memset calls.
>
> Is there any way we can avoid std::vector::resize? I suspect that
> ultimately the problem is that the buffer somewhere gets passed to a socket
> recv call, and recv call takes a pointer and length. In that case, there's
> no way that I know of to use a std::vector for the buffer and avoid the
> unnecessary initialization/memset in the resize call.
>
> Could another container be used instead of a vector?
> Could the vector be reused, so on subsequent calls we don't need to resize
> it again?
> Could something like uvector (which skips initialization) be used instead?
>
> Thank,
> Brett
>
>
>