Re: Interoperable Ignite.NET Dates

2020-11-03 Thread Alexey Kukushkin 
Pavel,

Thank you for your positive feedback! I will create an IEP.

вт, 3 нояб. 2020 г. в 15:48, Pavel Tupitsyn :

> Alexey,
>
> The proposal looks good to me.
>
> Usually I would try to avoid extra dependencies in the core assembly,
> but NodaTime seems to be worth it.
>
> Would you like to prepare an IEP?
>
> Thanks,
> Pavel
>
>
>
> On Tue, Nov 3, 2020 at 3:15 PM Alexey Kukushkin  >
> wrote:
>
> > We already created tickets and tested the proposed solution with our
> > applications (already in PROD) and an internal fork of Ignite. This
> > discussion is a proposal to donate this change to open source Apache
> Ignite
> > is the community accepts it:
> >
> >- IGNITE-12824 .NET: Write DateTime in interoperable format by default
> >
> >- IGNITE-12825 Serialize Java and .NET dates using same calendars
> >
> >
> >
> > вт, 3 нояб. 2020 г. в 15:08, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com>:
> >
> > > Cool, makes sense to me then. Please feel free to create a ticket and
> > mark
> > > it with the "ignite-3" label.
> > >
> > > -Val
> > >
> > > On Tue, Nov 3, 2020 at 4:03 AM Alexey Kukushkin <
> > kukushkinale...@gmail.com
> > > >
> > > wrote:
> > >
> > > > Val, absolutely - our proposal affects .NET API only.
> > > >
> > > > вт, 3 нояб. 2020 г. в 15:00, Valentin Kulichenko <
> > > > valentin.kuliche...@gmail.com>:
> > > >
> > > > > Got it. So it only affects the .NET side, correct?
> > > > >
> > > > > -Val
> > > > >
> > > > > On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin <
> > > > kukushkinale...@gmail.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Val,
> > > > > >
> > > > > > Our proposal does not overlap with IEP-54
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > > > >,
> > > > > > which proposes changing Ignite date storage format. Our proposal
> is
> > > > > > enhancing Ignite to handle .NET dates in a truly portable way
> > instead
> > > > of
> > > > > > requiring the application developers to repeat this task every
> > time:
> > > > > >
> > > > > >- Write .NET dates as Ignite dates (today .NET dates are
> written
> > > as
> > > > > >generic Ignite objects)
> > > > > >- Convert Local .NET dates to UTC inside Ignite and to it
> using
> > > Java
> > > > > >calendars.
> > > > > >
> > > > > >
> > > > > > вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> > > > > > valentin.kuliche...@gmail.com>:
> > > > > >
> > > > > > > Hi Alexey,
> > > > > > >
> > > > > > > The IEP-54 [1] describes the data layout proposed for Ignite
> 3.0,
> > > it
> > > > > > > includes various date/time types. Can you please take a look
> and
> > > > check
> > > > > if
> > > > > > > this addresses your concerns? We can then discuss further if
> > > needed.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > > > >
> > > > > > > -Val
> > > > > > >
> > > > > > > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > > > > > > kukushkinale...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Pavel,
> > > > > > > >
> > > > > > > > We propose changing public API so this is for Ignite 3.0.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > >
> > > > > > > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn <
> > > ptupit...@apache.org
> > > > >:
> > > > > > > >
> > > > > > > > > Alexey,
> > > > > > > > >
> > > > > > > > > Just to clarify before we start the discussion:
> > > > > > > > > this proposal seems to introduce some breaking changes, so
> we
> > > are
> > > > > > > talking
> > > > > > > > > about Ignite 3.0, correct?
> > > > > > > > >
> > > > > > > > > Pavel
> > > > > > > > >
> > > > > > > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > > > > > > kukushkinale...@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Igniters,
> > > > > > > > > >
> > > > > > > > > > What do you think about changing .NET API to read/write
> > > > portable
> > > > > > > dates
> > > > > > > > by
> > > > > > > > > > default and making that really portable?
> > > > > > > > > >
> > > > > > > > > > *The Problem*
> > > > > > > > > > Presently .NET API writes dates as composite Ignite
> > objects.
> > > > Only
> > > > > > > .NET
> > > > > > > > > > clients can read such dates: any other client (JDBC,
> Java,
> > > etc)
> > > > > > does
> > > > > > > > not
> > > > > > > > > > understand it without custom deserialization.
> > > > > > > > > >
> > > > > > > > > > It is still possible to configure .NET serialization to
> > write
> > > > > dates
> > > > > > > as
> > > > > > > > > > Ignite dates - see DateTime Serialization note
> > > > > > > > > > <
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
>

Re: Interoperable Ignite.NET Dates

2020-11-03 Thread Pavel Tupitsyn 
Alexey,

The proposal looks good to me.

Usually I would try to avoid extra dependencies in the core assembly,
but NodaTime seems to be worth it.

Would you like to prepare an IEP?

Thanks,
Pavel



On Tue, Nov 3, 2020 at 3:15 PM Alexey Kukushkin 
wrote:

> We already created tickets and tested the proposed solution with our
> applications (already in PROD) and an internal fork of Ignite. This
> discussion is a proposal to donate this change to open source Apache Ignite
> is the community accepts it:
>
>- IGNITE-12824 .NET: Write DateTime in interoperable format by default
>
>- IGNITE-12825 Serialize Java and .NET dates using same calendars
>
>
>
> вт, 3 нояб. 2020 г. в 15:08, Valentin Kulichenko <
> valentin.kuliche...@gmail.com>:
>
> > Cool, makes sense to me then. Please feel free to create a ticket and
> mark
> > it with the "ignite-3" label.
> >
> > -Val
> >
> > On Tue, Nov 3, 2020 at 4:03 AM Alexey Kukushkin <
> kukushkinale...@gmail.com
> > >
> > wrote:
> >
> > > Val, absolutely - our proposal affects .NET API only.
> > >
> > > вт, 3 нояб. 2020 г. в 15:00, Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com>:
> > >
> > > > Got it. So it only affects the .NET side, correct?
> > > >
> > > > -Val
> > > >
> > > > On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin <
> > > kukushkinale...@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi Val,
> > > > >
> > > > > Our proposal does not overlap with IEP-54
> > > > > <
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > > >,
> > > > > which proposes changing Ignite date storage format. Our proposal is
> > > > > enhancing Ignite to handle .NET dates in a truly portable way
> instead
> > > of
> > > > > requiring the application developers to repeat this task every
> time:
> > > > >
> > > > >- Write .NET dates as Ignite dates (today .NET dates are written
> > as
> > > > >generic Ignite objects)
> > > > >- Convert Local .NET dates to UTC inside Ignite and to it using
> > Java
> > > > >calendars.
> > > > >
> > > > >
> > > > > вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> > > > > valentin.kuliche...@gmail.com>:
> > > > >
> > > > > > Hi Alexey,
> > > > > >
> > > > > > The IEP-54 [1] describes the data layout proposed for Ignite 3.0,
> > it
> > > > > > includes various date/time types. Can you please take a look and
> > > check
> > > > if
> > > > > > this addresses your concerns? We can then discuss further if
> > needed.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > > >
> > > > > > -Val
> > > > > >
> > > > > > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > > > > > kukushkinale...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Pavel,
> > > > > > >
> > > > > > > We propose changing public API so this is for Ignite 3.0.
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn <
> > ptupit...@apache.org
> > > >:
> > > > > > >
> > > > > > > > Alexey,
> > > > > > > >
> > > > > > > > Just to clarify before we start the discussion:
> > > > > > > > this proposal seems to introduce some breaking changes, so we
> > are
> > > > > > talking
> > > > > > > > about Ignite 3.0, correct?
> > > > > > > >
> > > > > > > > Pavel
> > > > > > > >
> > > > > > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > > > > > kukushkinale...@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Igniters,
> > > > > > > > >
> > > > > > > > > What do you think about changing .NET API to read/write
> > > portable
> > > > > > dates
> > > > > > > by
> > > > > > > > > default and making that really portable?
> > > > > > > > >
> > > > > > > > > *The Problem*
> > > > > > > > > Presently .NET API writes dates as composite Ignite
> objects.
> > > Only
> > > > > > .NET
> > > > > > > > > clients can read such dates: any other client (JDBC, Java,
> > etc)
> > > > > does
> > > > > > > not
> > > > > > > > > understand it without custom deserialization.
> > > > > > > > >
> > > > > > > > > It is still possible to configure .NET serialization to
> write
> > > > dates
> > > > > > as
> > > > > > > > > Ignite dates - see DateTime Serialization note
> > > > > > > > > <
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > > > > > > >.
> > > > > > > > > But then Ignite accepts only UTC dates, requiring the
> > > application
> > > > > > > > > developers to convert local dates to UTC dates and back.
> This
> > > > task
> > > > > is
> > > > > > > not
> > > > > > > > > trivial: DateTime.ToUniversalTime
> > > > > > > > > <
> > > > > > > > >
> > > > > > > >
> > > > >

Re: Interoperable Ignite.NET Dates

2020-11-03 Thread Alexey Kukushkin 
We already created tickets and tested the proposed solution with our
applications (already in PROD) and an internal fork of Ignite. This
discussion is a proposal to donate this change to open source Apache Ignite
is the community accepts it:

   - IGNITE-12824 .NET: Write DateTime in interoperable format by default
   
   - IGNITE-12825 Serialize Java and .NET dates using same calendars
   


вт, 3 нояб. 2020 г. в 15:08, Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Cool, makes sense to me then. Please feel free to create a ticket and mark
> it with the "ignite-3" label.
>
> -Val
>
> On Tue, Nov 3, 2020 at 4:03 AM Alexey Kukushkin  >
> wrote:
>
> > Val, absolutely - our proposal affects .NET API only.
> >
> > вт, 3 нояб. 2020 г. в 15:00, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com>:
> >
> > > Got it. So it only affects the .NET side, correct?
> > >
> > > -Val
> > >
> > > On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin <
> > kukushkinale...@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hi Val,
> > > >
> > > > Our proposal does not overlap with IEP-54
> > > > <
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > >,
> > > > which proposes changing Ignite date storage format. Our proposal is
> > > > enhancing Ignite to handle .NET dates in a truly portable way instead
> > of
> > > > requiring the application developers to repeat this task every time:
> > > >
> > > >- Write .NET dates as Ignite dates (today .NET dates are written
> as
> > > >generic Ignite objects)
> > > >- Convert Local .NET dates to UTC inside Ignite and to it using
> Java
> > > >calendars.
> > > >
> > > >
> > > > вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> > > > valentin.kuliche...@gmail.com>:
> > > >
> > > > > Hi Alexey,
> > > > >
> > > > > The IEP-54 [1] describes the data layout proposed for Ignite 3.0,
> it
> > > > > includes various date/time types. Can you please take a look and
> > check
> > > if
> > > > > this addresses your concerns? We can then discuss further if
> needed.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > >
> > > > > -Val
> > > > >
> > > > > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > > > > kukushkinale...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Pavel,
> > > > > >
> > > > > > We propose changing public API so this is for Ignite 3.0.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn <
> ptupit...@apache.org
> > >:
> > > > > >
> > > > > > > Alexey,
> > > > > > >
> > > > > > > Just to clarify before we start the discussion:
> > > > > > > this proposal seems to introduce some breaking changes, so we
> are
> > > > > talking
> > > > > > > about Ignite 3.0, correct?
> > > > > > >
> > > > > > > Pavel
> > > > > > >
> > > > > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > > > > kukushkinale...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Igniters,
> > > > > > > >
> > > > > > > > What do you think about changing .NET API to read/write
> > portable
> > > > > dates
> > > > > > by
> > > > > > > > default and making that really portable?
> > > > > > > >
> > > > > > > > *The Problem*
> > > > > > > > Presently .NET API writes dates as composite Ignite objects.
> > Only
> > > > > .NET
> > > > > > > > clients can read such dates: any other client (JDBC, Java,
> etc)
> > > > does
> > > > > > not
> > > > > > > > understand it without custom deserialization.
> > > > > > > >
> > > > > > > > It is still possible to configure .NET serialization to write
> > > dates
> > > > > as
> > > > > > > > Ignite dates - see DateTime Serialization note
> > > > > > > > <
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > > > > > >.
> > > > > > > > But then Ignite accepts only UTC dates, requiring the
> > application
> > > > > > > > developers to convert local dates to UTC dates and back. This
> > > task
> > > > is
> > > > > > not
> > > > > > > > trivial: DateTime.ToUniversalTime
> > > > > > > > <
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > > > > > > >
> > > > > > > > uses
> > > > > > > > calendars different from Java (and the .NET calendars are the
> > > > invalid
> > > > > > > ones
> > > > > > > > - especially for pre-1990 dates).
> > > > > > > >
> > > > > > > > The motivation for the current default behavior was probably
> > the
> > > > > desire
> > > > > > > to
> > > > > > > > keep the Time Zone information: Ignite dates do not store
> time
> > > > zones.
> > > > > > > >
> > > > > > > > In our

Re: Interoperable Ignite.NET Dates

2020-11-03 Thread Valentin Kulichenko 
Cool, makes sense to me then. Please feel free to create a ticket and mark
it with the "ignite-3" label.

-Val

On Tue, Nov 3, 2020 at 4:03 AM Alexey Kukushkin 
wrote:

> Val, absolutely - our proposal affects .NET API only.
>
> вт, 3 нояб. 2020 г. в 15:00, Valentin Kulichenko <
> valentin.kuliche...@gmail.com>:
>
> > Got it. So it only affects the .NET side, correct?
> >
> > -Val
> >
> > On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin <
> kukushkinale...@gmail.com
> > >
> > wrote:
> >
> > > Hi Val,
> > >
> > > Our proposal does not overlap with IEP-54
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > >,
> > > which proposes changing Ignite date storage format. Our proposal is
> > > enhancing Ignite to handle .NET dates in a truly portable way instead
> of
> > > requiring the application developers to repeat this task every time:
> > >
> > >- Write .NET dates as Ignite dates (today .NET dates are written as
> > >generic Ignite objects)
> > >- Convert Local .NET dates to UTC inside Ignite and to it using Java
> > >calendars.
> > >
> > >
> > > вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com>:
> > >
> > > > Hi Alexey,
> > > >
> > > > The IEP-54 [1] describes the data layout proposed for Ignite 3.0, it
> > > > includes various date/time types. Can you please take a look and
> check
> > if
> > > > this addresses your concerns? We can then discuss further if needed.
> > > >
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > >
> > > > -Val
> > > >
> > > > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > > > kukushkinale...@gmail.com>
> > > > wrote:
> > > >
> > > > > Pavel,
> > > > >
> > > > > We propose changing public API so this is for Ignite 3.0.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn  >:
> > > > >
> > > > > > Alexey,
> > > > > >
> > > > > > Just to clarify before we start the discussion:
> > > > > > this proposal seems to introduce some breaking changes, so we are
> > > > talking
> > > > > > about Ignite 3.0, correct?
> > > > > >
> > > > > > Pavel
> > > > > >
> > > > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > > > kukushkinale...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > What do you think about changing .NET API to read/write
> portable
> > > > dates
> > > > > by
> > > > > > > default and making that really portable?
> > > > > > >
> > > > > > > *The Problem*
> > > > > > > Presently .NET API writes dates as composite Ignite objects.
> Only
> > > > .NET
> > > > > > > clients can read such dates: any other client (JDBC, Java, etc)
> > > does
> > > > > not
> > > > > > > understand it without custom deserialization.
> > > > > > >
> > > > > > > It is still possible to configure .NET serialization to write
> > dates
> > > > as
> > > > > > > Ignite dates - see DateTime Serialization note
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > > > > >.
> > > > > > > But then Ignite accepts only UTC dates, requiring the
> application
> > > > > > > developers to convert local dates to UTC dates and back. This
> > task
> > > is
> > > > > not
> > > > > > > trivial: DateTime.ToUniversalTime
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > > > > > >
> > > > > > > uses
> > > > > > > calendars different from Java (and the .NET calendars are the
> > > invalid
> > > > > > ones
> > > > > > > - especially for pre-1990 dates).
> > > > > > >
> > > > > > > The motivation for the current default behavior was probably
> the
> > > > desire
> > > > > > to
> > > > > > > keep the Time Zone information: Ignite dates do not store time
> > > zones.
> > > > > > >
> > > > > > > In our experience interoperability is more important than
> storing
> > > > time
> > > > > > zone
> > > > > > > info.
> > > > > > >
> > > > > > > *The Proposal*
> > > > > > >
> > > > > > >1. Always write .NET dates as portable Ignite dates: get rid
> > of
> > > > the
> > > > > > >BinaryReflectiveSerializer.ForceTimestamp flag that
> currently
> > > > > triggers
> > > > > > > this
> > > > > > >behavior.
> > > > > > >   - We could still keep the ForceTimestamp flag if saving
> > .NET
> > > > > dates
> > > > > > as
> > > > > > >   transparent objects seems a useful case. We do not think
> it
> > > is
> > > > > > > useful.
> > > > > > >2. Automatically convert Local dates to UTC and back
> *inside*
> > > > > > >Ignite.NET.
> > > > > > >   - In this case we lose the DateTime.Kind of UTC dates: we
> > > > write a
> > > > > > UTC
> > > > > > >   date but we would read a Local date since Ignite

Re: Interoperable Ignite.NET Dates

2020-11-03 Thread Alexey Kukushkin 
Val, absolutely - our proposal affects .NET API only.

вт, 3 нояб. 2020 г. в 15:00, Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Got it. So it only affects the .NET side, correct?
>
> -Val
>
> On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin  >
> wrote:
>
> > Hi Val,
> >
> > Our proposal does not overlap with IEP-54
> > <
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > >,
> > which proposes changing Ignite date storage format. Our proposal is
> > enhancing Ignite to handle .NET dates in a truly portable way instead of
> > requiring the application developers to repeat this task every time:
> >
> >- Write .NET dates as Ignite dates (today .NET dates are written as
> >generic Ignite objects)
> >- Convert Local .NET dates to UTC inside Ignite and to it using Java
> >calendars.
> >
> >
> > вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com>:
> >
> > > Hi Alexey,
> > >
> > > The IEP-54 [1] describes the data layout proposed for Ignite 3.0, it
> > > includes various date/time types. Can you please take a look and check
> if
> > > this addresses your concerns? We can then discuss further if needed.
> > >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > >
> > > -Val
> > >
> > > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > > kukushkinale...@gmail.com>
> > > wrote:
> > >
> > > > Pavel,
> > > >
> > > > We propose changing public API so this is for Ignite 3.0.
> > > >
> > > > Thanks!
> > > >
> > > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn :
> > > >
> > > > > Alexey,
> > > > >
> > > > > Just to clarify before we start the discussion:
> > > > > this proposal seems to introduce some breaking changes, so we are
> > > talking
> > > > > about Ignite 3.0, correct?
> > > > >
> > > > > Pavel
> > > > >
> > > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > > kukushkinale...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > What do you think about changing .NET API to read/write portable
> > > dates
> > > > by
> > > > > > default and making that really portable?
> > > > > >
> > > > > > *The Problem*
> > > > > > Presently .NET API writes dates as composite Ignite objects. Only
> > > .NET
> > > > > > clients can read such dates: any other client (JDBC, Java, etc)
> > does
> > > > not
> > > > > > understand it without custom deserialization.
> > > > > >
> > > > > > It is still possible to configure .NET serialization to write
> dates
> > > as
> > > > > > Ignite dates - see DateTime Serialization note
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > > > >.
> > > > > > But then Ignite accepts only UTC dates, requiring the application
> > > > > > developers to convert local dates to UTC dates and back. This
> task
> > is
> > > > not
> > > > > > trivial: DateTime.ToUniversalTime
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > > > > >
> > > > > > uses
> > > > > > calendars different from Java (and the .NET calendars are the
> > invalid
> > > > > ones
> > > > > > - especially for pre-1990 dates).
> > > > > >
> > > > > > The motivation for the current default behavior was probably the
> > > desire
> > > > > to
> > > > > > keep the Time Zone information: Ignite dates do not store time
> > zones.
> > > > > >
> > > > > > In our experience interoperability is more important than storing
> > > time
> > > > > zone
> > > > > > info.
> > > > > >
> > > > > > *The Proposal*
> > > > > >
> > > > > >1. Always write .NET dates as portable Ignite dates: get rid
> of
> > > the
> > > > > >BinaryReflectiveSerializer.ForceTimestamp flag that currently
> > > > triggers
> > > > > > this
> > > > > >behavior.
> > > > > >   - We could still keep the ForceTimestamp flag if saving
> .NET
> > > > dates
> > > > > as
> > > > > >   transparent objects seems a useful case. We do not think it
> > is
> > > > > > useful.
> > > > > >2. Automatically convert Local dates to UTC and back *inside*
> > > > > >Ignite.NET.
> > > > > >   - In this case we lose the DateTime.Kind of UTC dates: we
> > > write a
> > > > > UTC
> > > > > >   date but we would read a Local date since Ignite would
> always
> > > > > > convert UTC
> > > > > >   to Local when reading.
> > > > > >   We could add a UtcDate date flag to QuerySqlFieldAttribute
> > > > > >   and BinaryReflectiveSerializer to control the
> deserialization
> > > > > > behavior if
> > > > > >   keeping dates in UTC format use case seems important.
> > > > > >3. Use NodaTime  for UTC<->Local
> > > > conversions.
> > > > > >Noda time uses Java calendars making the conversion truely
> > > portable.
> > > > > >
> >

Re: Interoperable Ignite.NET Dates

2020-11-03 Thread Valentin Kulichenko 
Got it. So it only affects the .NET side, correct?

-Val

On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin 
wrote:

> Hi Val,
>
> Our proposal does not overlap with IEP-54
> <
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> >,
> which proposes changing Ignite date storage format. Our proposal is
> enhancing Ignite to handle .NET dates in a truly portable way instead of
> requiring the application developers to repeat this task every time:
>
>- Write .NET dates as Ignite dates (today .NET dates are written as
>generic Ignite objects)
>- Convert Local .NET dates to UTC inside Ignite and to it using Java
>calendars.
>
>
> вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> valentin.kuliche...@gmail.com>:
>
> > Hi Alexey,
> >
> > The IEP-54 [1] describes the data layout proposed for Ignite 3.0, it
> > includes various date/time types. Can you please take a look and check if
> > this addresses your concerns? We can then discuss further if needed.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> >
> > -Val
> >
> > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > kukushkinale...@gmail.com>
> > wrote:
> >
> > > Pavel,
> > >
> > > We propose changing public API so this is for Ignite 3.0.
> > >
> > > Thanks!
> > >
> > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn :
> > >
> > > > Alexey,
> > > >
> > > > Just to clarify before we start the discussion:
> > > > this proposal seems to introduce some breaking changes, so we are
> > talking
> > > > about Ignite 3.0, correct?
> > > >
> > > > Pavel
> > > >
> > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > kukushkinale...@gmail.com>
> > > > wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > What do you think about changing .NET API to read/write portable
> > dates
> > > by
> > > > > default and making that really portable?
> > > > >
> > > > > *The Problem*
> > > > > Presently .NET API writes dates as composite Ignite objects. Only
> > .NET
> > > > > clients can read such dates: any other client (JDBC, Java, etc)
> does
> > > not
> > > > > understand it without custom deserialization.
> > > > >
> > > > > It is still possible to configure .NET serialization to write dates
> > as
> > > > > Ignite dates - see DateTime Serialization note
> > > > > <
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > > >.
> > > > > But then Ignite accepts only UTC dates, requiring the application
> > > > > developers to convert local dates to UTC dates and back. This task
> is
> > > not
> > > > > trivial: DateTime.ToUniversalTime
> > > > > <
> > > > >
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > > > >
> > > > > uses
> > > > > calendars different from Java (and the .NET calendars are the
> invalid
> > > > ones
> > > > > - especially for pre-1990 dates).
> > > > >
> > > > > The motivation for the current default behavior was probably the
> > desire
> > > > to
> > > > > keep the Time Zone information: Ignite dates do not store time
> zones.
> > > > >
> > > > > In our experience interoperability is more important than storing
> > time
> > > > zone
> > > > > info.
> > > > >
> > > > > *The Proposal*
> > > > >
> > > > >1. Always write .NET dates as portable Ignite dates: get rid of
> > the
> > > > >BinaryReflectiveSerializer.ForceTimestamp flag that currently
> > > triggers
> > > > > this
> > > > >behavior.
> > > > >   - We could still keep the ForceTimestamp flag if saving .NET
> > > dates
> > > > as
> > > > >   transparent objects seems a useful case. We do not think it
> is
> > > > > useful.
> > > > >2. Automatically convert Local dates to UTC and back *inside*
> > > > >Ignite.NET.
> > > > >   - In this case we lose the DateTime.Kind of UTC dates: we
> > write a
> > > > UTC
> > > > >   date but we would read a Local date since Ignite would always
> > > > > convert UTC
> > > > >   to Local when reading.
> > > > >   We could add a UtcDate date flag to QuerySqlFieldAttribute
> > > > >   and BinaryReflectiveSerializer to control the deserialization
> > > > > behavior if
> > > > >   keeping dates in UTC format use case seems important.
> > > > >3. Use NodaTime  for UTC<->Local
> > > conversions.
> > > > >Noda time uses Java calendars making the conversion truely
> > portable.
> > > > >
> > > > > *The Benefits*
> > > > >
> > > > >1. We think portable dates are much more important than storing
> > time
> > > > >zone info. Why do we store time zones for every date on the
> server
> > > > > anyway?
> > > > >Time zone is client-side info.
> > > > >2. Simpler application code: no more manual configuration to
> > trigger
> > > > the
> > > > >portable behavior.
> > > > >3. Non-trivial code to make the truly portable

Re: Interoperable Ignite.NET Dates

2020-11-03 Thread Alexey Kukushkin 
Hi Val,

Our proposal does not overlap with IEP-54
,
which proposes changing Ignite date storage format. Our proposal is
enhancing Ignite to handle .NET dates in a truly portable way instead of
requiring the application developers to repeat this task every time:

   - Write .NET dates as Ignite dates (today .NET dates are written as
   generic Ignite objects)
   - Convert Local .NET dates to UTC inside Ignite and to it using Java
   calendars.


вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Hi Alexey,
>
> The IEP-54 [1] describes the data layout proposed for Ignite 3.0, it
> includes various date/time types. Can you please take a look and check if
> this addresses your concerns? We can then discuss further if needed.
>
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
>
> -Val
>
> On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> kukushkinale...@gmail.com>
> wrote:
>
> > Pavel,
> >
> > We propose changing public API so this is for Ignite 3.0.
> >
> > Thanks!
> >
> > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn :
> >
> > > Alexey,
> > >
> > > Just to clarify before we start the discussion:
> > > this proposal seems to introduce some breaking changes, so we are
> talking
> > > about Ignite 3.0, correct?
> > >
> > > Pavel
> > >
> > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > kukushkinale...@gmail.com>
> > > wrote:
> > >
> > > > Igniters,
> > > >
> > > > What do you think about changing .NET API to read/write portable
> dates
> > by
> > > > default and making that really portable?
> > > >
> > > > *The Problem*
> > > > Presently .NET API writes dates as composite Ignite objects. Only
> .NET
> > > > clients can read such dates: any other client (JDBC, Java, etc) does
> > not
> > > > understand it without custom deserialization.
> > > >
> > > > It is still possible to configure .NET serialization to write dates
> as
> > > > Ignite dates - see DateTime Serialization note
> > > > <
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > >.
> > > > But then Ignite accepts only UTC dates, requiring the application
> > > > developers to convert local dates to UTC dates and back. This task is
> > not
> > > > trivial: DateTime.ToUniversalTime
> > > > <
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > > >
> > > > uses
> > > > calendars different from Java (and the .NET calendars are the invalid
> > > ones
> > > > - especially for pre-1990 dates).
> > > >
> > > > The motivation for the current default behavior was probably the
> desire
> > > to
> > > > keep the Time Zone information: Ignite dates do not store time zones.
> > > >
> > > > In our experience interoperability is more important than storing
> time
> > > zone
> > > > info.
> > > >
> > > > *The Proposal*
> > > >
> > > >1. Always write .NET dates as portable Ignite dates: get rid of
> the
> > > >BinaryReflectiveSerializer.ForceTimestamp flag that currently
> > triggers
> > > > this
> > > >behavior.
> > > >   - We could still keep the ForceTimestamp flag if saving .NET
> > dates
> > > as
> > > >   transparent objects seems a useful case. We do not think it is
> > > > useful.
> > > >2. Automatically convert Local dates to UTC and back *inside*
> > > >Ignite.NET.
> > > >   - In this case we lose the DateTime.Kind of UTC dates: we
> write a
> > > UTC
> > > >   date but we would read a Local date since Ignite would always
> > > > convert UTC
> > > >   to Local when reading.
> > > >   We could add a UtcDate date flag to QuerySqlFieldAttribute
> > > >   and BinaryReflectiveSerializer to control the deserialization
> > > > behavior if
> > > >   keeping dates in UTC format use case seems important.
> > > >3. Use NodaTime  for UTC<->Local
> > conversions.
> > > >Noda time uses Java calendars making the conversion truely
> portable.
> > > >
> > > > *The Benefits*
> > > >
> > > >1. We think portable dates are much more important than storing
> time
> > > >zone info. Why do we store time zones for every date on the server
> > > > anyway?
> > > >Time zone is client-side info.
> > > >2. Simpler application code: no more manual configuration to
> trigger
> > > the
> > > >portable behavior.
> > > >3. Non-trivial code to make the truly portable UTC<->Local
> > conversion
> > > is
> > > >implemented once inside Ignite instead of having every Ignite.NET
> > > >application implementing it.
> > > >
> > > > Thank you!
> > > >
> > > > --
> > > > Best regards,
> > > > Alexey
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Alexey
> >
>


-- 
Best regards,
Alexey

Re: Interoperable Ignite.NET Dates

2020-11-03 Thread Valentin Kulichenko 
Hi Alexey,

The IEP-54 [1] describes the data layout proposed for Ignite 3.0, it
includes various date/time types. Can you please take a look and check if
this addresses your concerns? We can then discuss further if needed.

https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach

-Val

On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin 
wrote:

> Pavel,
>
> We propose changing public API so this is for Ignite 3.0.
>
> Thanks!
>
> вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn :
>
> > Alexey,
> >
> > Just to clarify before we start the discussion:
> > this proposal seems to introduce some breaking changes, so we are talking
> > about Ignite 3.0, correct?
> >
> > Pavel
> >
> > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > kukushkinale...@gmail.com>
> > wrote:
> >
> > > Igniters,
> > >
> > > What do you think about changing .NET API to read/write portable dates
> by
> > > default and making that really portable?
> > >
> > > *The Problem*
> > > Presently .NET API writes dates as composite Ignite objects. Only .NET
> > > clients can read such dates: any other client (JDBC, Java, etc) does
> not
> > > understand it without custom deserialization.
> > >
> > > It is still possible to configure .NET serialization to write dates as
> > > Ignite dates - see DateTime Serialization note
> > > <
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > >.
> > > But then Ignite accepts only UTC dates, requiring the application
> > > developers to convert local dates to UTC dates and back. This task is
> not
> > > trivial: DateTime.ToUniversalTime
> > > <
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > >
> > > uses
> > > calendars different from Java (and the .NET calendars are the invalid
> > ones
> > > - especially for pre-1990 dates).
> > >
> > > The motivation for the current default behavior was probably the desire
> > to
> > > keep the Time Zone information: Ignite dates do not store time zones.
> > >
> > > In our experience interoperability is more important than storing time
> > zone
> > > info.
> > >
> > > *The Proposal*
> > >
> > >1. Always write .NET dates as portable Ignite dates: get rid of the
> > >BinaryReflectiveSerializer.ForceTimestamp flag that currently
> triggers
> > > this
> > >behavior.
> > >   - We could still keep the ForceTimestamp flag if saving .NET
> dates
> > as
> > >   transparent objects seems a useful case. We do not think it is
> > > useful.
> > >2. Automatically convert Local dates to UTC and back *inside*
> > >Ignite.NET.
> > >   - In this case we lose the DateTime.Kind of UTC dates: we write a
> > UTC
> > >   date but we would read a Local date since Ignite would always
> > > convert UTC
> > >   to Local when reading.
> > >   We could add a UtcDate date flag to QuerySqlFieldAttribute
> > >   and BinaryReflectiveSerializer to control the deserialization
> > > behavior if
> > >   keeping dates in UTC format use case seems important.
> > >3. Use NodaTime  for UTC<->Local
> conversions.
> > >Noda time uses Java calendars making the conversion truely portable.
> > >
> > > *The Benefits*
> > >
> > >1. We think portable dates are much more important than storing time
> > >zone info. Why do we store time zones for every date on the server
> > > anyway?
> > >Time zone is client-side info.
> > >2. Simpler application code: no more manual configuration to trigger
> > the
> > >portable behavior.
> > >3. Non-trivial code to make the truly portable UTC<->Local
> conversion
> > is
> > >implemented once inside Ignite instead of having every Ignite.NET
> > >application implementing it.
> > >
> > > Thank you!
> > >
> > > --
> > > Best regards,
> > > Alexey
> > >
> >
>
>
> --
> Best regards,
> Alexey
>

Re: Interoperable Ignite.NET Dates

2020-11-03 Thread Alexey Kukushkin 
Pavel,

We propose changing public API so this is for Ignite 3.0.

Thanks!

вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn :

> Alexey,
>
> Just to clarify before we start the discussion:
> this proposal seems to introduce some breaking changes, so we are talking
> about Ignite 3.0, correct?
>
> Pavel
>
> On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> kukushkinale...@gmail.com>
> wrote:
>
> > Igniters,
> >
> > What do you think about changing .NET API to read/write portable dates by
> > default and making that really portable?
> >
> > *The Problem*
> > Presently .NET API writes dates as composite Ignite objects. Only .NET
> > clients can read such dates: any other client (JDBC, Java, etc) does not
> > understand it without custom deserialization.
> >
> > It is still possible to configure .NET serialization to write dates as
> > Ignite dates - see DateTime Serialization note
> > <
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > >.
> > But then Ignite accepts only UTC dates, requiring the application
> > developers to convert local dates to UTC dates and back. This task is not
> > trivial: DateTime.ToUniversalTime
> > <
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > >
> > uses
> > calendars different from Java (and the .NET calendars are the invalid
> ones
> > - especially for pre-1990 dates).
> >
> > The motivation for the current default behavior was probably the desire
> to
> > keep the Time Zone information: Ignite dates do not store time zones.
> >
> > In our experience interoperability is more important than storing time
> zone
> > info.
> >
> > *The Proposal*
> >
> >1. Always write .NET dates as portable Ignite dates: get rid of the
> >BinaryReflectiveSerializer.ForceTimestamp flag that currently triggers
> > this
> >behavior.
> >   - We could still keep the ForceTimestamp flag if saving .NET dates
> as
> >   transparent objects seems a useful case. We do not think it is
> > useful.
> >2. Automatically convert Local dates to UTC and back *inside*
> >Ignite.NET.
> >   - In this case we lose the DateTime.Kind of UTC dates: we write a
> UTC
> >   date but we would read a Local date since Ignite would always
> > convert UTC
> >   to Local when reading.
> >   We could add a UtcDate date flag to QuerySqlFieldAttribute
> >   and BinaryReflectiveSerializer to control the deserialization
> > behavior if
> >   keeping dates in UTC format use case seems important.
> >3. Use NodaTime  for UTC<->Local conversions.
> >Noda time uses Java calendars making the conversion truely portable.
> >
> > *The Benefits*
> >
> >1. We think portable dates are much more important than storing time
> >zone info. Why do we store time zones for every date on the server
> > anyway?
> >Time zone is client-side info.
> >2. Simpler application code: no more manual configuration to trigger
> the
> >portable behavior.
> >3. Non-trivial code to make the truly portable UTC<->Local conversion
> is
> >implemented once inside Ignite instead of having every Ignite.NET
> >application implementing it.
> >
> > Thank you!
> >
> > --
> > Best regards,
> > Alexey
> >
>


-- 
Best regards,
Alexey

Re: Interoperable Ignite.NET Dates

2020-11-02 Thread Pavel Tupitsyn 
Alexey,

Just to clarify before we start the discussion:
this proposal seems to introduce some breaking changes, so we are talking
about Ignite 3.0, correct?

Pavel

On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin 
wrote:

> Igniters,
>
> What do you think about changing .NET API to read/write portable dates by
> default and making that really portable?
>
> *The Problem*
> Presently .NET API writes dates as composite Ignite objects. Only .NET
> clients can read such dates: any other client (JDBC, Java, etc) does not
> understand it without custom deserialization.
>
> It is still possible to configure .NET serialization to write dates as
> Ignite dates - see DateTime Serialization note
> <
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> >.
> But then Ignite accepts only UTC dates, requiring the application
> developers to convert local dates to UTC dates and back. This task is not
> trivial: DateTime.ToUniversalTime
> <
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> >
> uses
> calendars different from Java (and the .NET calendars are the invalid ones
> - especially for pre-1990 dates).
>
> The motivation for the current default behavior was probably the desire to
> keep the Time Zone information: Ignite dates do not store time zones.
>
> In our experience interoperability is more important than storing time zone
> info.
>
> *The Proposal*
>
>1. Always write .NET dates as portable Ignite dates: get rid of the
>BinaryReflectiveSerializer.ForceTimestamp flag that currently triggers
> this
>behavior.
>   - We could still keep the ForceTimestamp flag if saving .NET dates as
>   transparent objects seems a useful case. We do not think it is
> useful.
>2. Automatically convert Local dates to UTC and back *inside*
>Ignite.NET.
>   - In this case we lose the DateTime.Kind of UTC dates: we write a UTC
>   date but we would read a Local date since Ignite would always
> convert UTC
>   to Local when reading.
>   We could add a UtcDate date flag to QuerySqlFieldAttribute
>   and BinaryReflectiveSerializer to control the deserialization
> behavior if
>   keeping dates in UTC format use case seems important.
>3. Use NodaTime  for UTC<->Local conversions.
>Noda time uses Java calendars making the conversion truely portable.
>
> *The Benefits*
>
>1. We think portable dates are much more important than storing time
>zone info. Why do we store time zones for every date on the server
> anyway?
>Time zone is client-side info.
>2. Simpler application code: no more manual configuration to trigger the
>portable behavior.
>3. Non-trivial code to make the truly portable UTC<->Local conversion is
>implemented once inside Ignite instead of having every Ignite.NET
>application implementing it.
>
> Thank you!
>
> --
> Best regards,
> Alexey
>