Re: Incorrect failure handler exceptions

[Ivan Daschinsky]

There is not a big problem to fix it.

1. Blocked thread is detected by another worker. Currently we take dump
before printing it to log with zero depth. This can be easily fixed.

2. We should propagate BlockedThreadExceptiob to failureCtx with original
stacktrace of blocked thread. StackFrames, obtained in previous step, can
be used to construct stacktrace of BlockedThreadException.


вт, 3 нояб. 2020 г., 19:15 Ilya Kasnacheev :

> Hello!
>
> https://issues.apache.org/jira/browse/IGNITE-13665
>
> The notorious problem with failure handler's thread dump, which nobody has
> bothered to investigate, until now.
> "IgnitionEx$IgniteNamedInstance$2.apply"
>
> If anybody knows how to fix it, please suggest.
>
> Regards,
> --
> Ilya Kasnacheev
>

Re: Custom Affinity Functions proposed for removal?

[Raymond Wilson]

If I have a primary key vector like this , where  &  can
have varying values for , and I want all keys with the same  and 
to reside in the same partition for processing colocation requirements then
I can't use the standard Ignite mapping to do this, I need to use a custom
mapper than uses just the  &  components.


On Wed, Nov 4, 2020 at 1:33 AM Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Raymond,
>
> Thanks for the details. So it sounds like you have a custom affinity
> mapper, not affinity function. This makes things simpler, but I'm still
> failing to understand why standard mechanisms for collocation [1] didn't
> work for you. Could you please clarify?
>
> [1]
> https://ignite.apache.org/docs/latest/data-modeling/affinity-collocation
>
> -Val
>
> On Tue, Nov 3, 2020 at 12:25 AM Raymond Wilson  >
> wrote:
>
> > In terms of Key -> Partition -> Node mapping, we provide customer
> affinity
> > mappers for the Key -> Partition stage an allow Ignite to map partitions
> to
> > nodes.
> >
> > Our keys are structs with multiple fields forming a composite primary
> key,
> > parts of which are spatially identifying and parts contain other
> > characteristics. We want to ensure all related spatial elements resolve
> to
> > the same partition and so will be placed on the same node for query
> > colocation purposes.
> >
> >
> >
> > On Tue, Nov 3, 2020 at 9:20 PM Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> > > Moti, Raymond,
> > >
> > > Could you please describe your use cases in more detail? What are the
> > types
> > > used as cache keys? What is the custom logic that you use for affinity
> > > mapping? What was the exact reason to customize versus using built-in
> > > collocation mechanisms?
> > >
> > > Ultimately, I'm sure that custom affinity functions will go away cause
> > they
> > > introduce multiple potential issues. However, I would definitely like
> to
> > > make sure that your use cases are still supported in Ignite 3.0 via
> some
> > > other means.
> > >
> > > -Val
> > >
> > > On Mon, Nov 2, 2020 at 12:21 PM Alexei Scherbakov <
> > > alexey.scherbak...@gmail.com> wrote:
> > >
> > > > Thanks for the clarification.
> > > >
> > > > There was no intention to remove the customizable key to partition
> > > mapping.
> > > >
> > > > Difficulties arise when mapping partitions to nodes, so it's
> desirable
> > to
> > > > have internally tested implementation with a way to customize it's
> > > behavior
> > > > without additional coding on the user side.
> > > >
> > > > пн, 2 нояб. 2020 г. в 23:01, Raymond Wilson <
> > raymond_wil...@trimble.com
> > > >:
> > > >
> > > > > Just to be clear, the affinity functions we are using convert keys
> to
> > > > > partitions, we do not map partitions to nodes and leave that to
> > Ignite.
> > > > >
> > > > > On Tue, Nov 3, 2020 at 8:48 AM Alexei Scherbakov <
> > > > > alexey.scherbak...@gmail.com> wrote:
> > > > >
> > > > > > Hello.
> > > > > >
> > > > > > Custom affinity functions can cause weird bugs and data loss if
> > > > > implemented
> > > > > > wrongly.
> > > > > > There is an intention  to keep a backup filter based on user
> > > attributes
> > > > > > (with additional validation logic to ensure correctness) for
> > > > controllable
> > > > > > data placement.
> > > > > >
> > > > > > Can you describe more precisely why you had to implement custom
> > > > affinity
> > > > > > functions and not resort to default rendezvous affinity + backup
> > > > filter ?
> > > > > >
> > > > > >
> > > > > > пн, 2 нояб. 2020 г. в 21:45, Raymond Wilson <
> > > > raymond_wil...@trimble.com
> > > > > >:
> > > > > >
> > > > > > > We also use custom affinity functions (vis the C# client).
> > > > > > >
> > > > > > > The wish list mentions use of a particular annotation
> > > > > > > (@CentralizedAffinityFunction):
> > > > > > > Is the wish to remove just this annotation, or the ability to
> > > define
> > > > > > custom
> > > > > > > affinity functions at all?
> > > > > > >
> > > > > > > In our case we use affinity functions to ensure particular
> > > > distribution
> > > > > > of
> > > > > > > spatial data across a processing cluster to ensure even load
> > > > > management.
> > > > > > >
> > > > > > > On Tue, Nov 3, 2020 at 5:31 AM Moti Nisenson <
> > > > moti.nisen...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > I saw at
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0+Wishlist
> > > > > > > > that custom affinity functions are on the potential wishlist
> > for
> > > > > > removal.
> > > > > > > > The way we're using it's very critical that we be able to
> > control
> > > > the
> > > > > > > > placement of data quite precisely - as part of that we
> specify
> > > > > > explicitly
> > > > > > > > the partition we want in the key, and then our affinity
> > function
> > > > uses
> > > > > > > that
> > > > > > > > (else 

[jira] [Created] (IGNITE-13670) Omit nullmap where possible

[Alexey Goncharuk (Jira)]

Alexey Goncharuk created IGNITE-13670:
-

 Summary: Omit nullmap where possible
 Key: IGNITE-13670
 URL: https://issues.apache.org/jira/browse/IGNITE-13670
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexey Goncharuk


The nullmap is currently always written to the tuple layout. However, it can be 
fully omitted for schemas where all columns are non-null - this saves both a 
little space and runtime for offsets folding.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13669) Implement date native types

[Alexey Goncharuk (Jira)]

Alexey Goncharuk created IGNITE-13669:
-

 Summary: Implement date native types
 Key: IGNITE-13669
 URL: https://issues.apache.org/jira/browse/IGNITE-13669
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexey Goncharuk


Besides the types themselves, it may be beneficial to provide date/time field 
extraction methods so that they can be read without object creation. The layout 
is described in the IEP.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13668) Implement Number(n) and Decimal native types

[Alexey Goncharuk (Jira)]

Alexey Goncharuk created IGNITE-13668:
-

 Summary: Implement Number(n) and Decimal native types
 Key: IGNITE-13668
 URL: https://issues.apache.org/jira/browse/IGNITE-13668
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexey Goncharuk


Number(n) is an {{n}}-bytes two-complement integer signed value encoded in the 
varlong style (so that Number(4) can be mapped to integer and Number(8) can be 
mapped to long during (de)serialization). Larger numbers can be represented as 
{{BigInteger}}. The Number(n) is a varlen type, so it will take two additional 
bytes in the varlen table, so types smaller than Number(2) are better 
represented by {{byte}} and {{short}} types as their fixlen encoding takes 
exactly 1 and 2 bytes respectively.

Decimal is a direct mapping to BigDecimal value.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13667) Add schema columns relocation table to map from user order to system order

[Alexey Goncharuk (Jira)]

Alexey Goncharuk created IGNITE-13667:
-

 Summary: Add schema columns relocation table to map from user 
order to system order
 Key: IGNITE-13667
 URL: https://issues.apache.org/jira/browse/IGNITE-13667
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexey Goncharuk


When a schema is defined, the key chunk columns and value chunk columns are 
sorted so that fixlen columns go first and varlen columns go second, so the 
sorted column order differs from the order of the user-defined columns.
We need to add a simple relocation table which is a permutation of indices 
{{[0..n)}}, so that an internal column order for user index {{n}} is 
{{relocationTbl[n]}}.
NB: the tuple assembler will still need to access the internal sorted order for 
proper tuple assembly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Incorrect failure handler exceptions

[Ilya Kasnacheev]

Hello!

https://issues.apache.org/jira/browse/IGNITE-13665

The notorious problem with failure handler's thread dump, which nobody has
bothered to investigate, until now. "IgnitionEx$IgniteNamedInstance$2.apply"

If anybody knows how to fix it, please suggest.

Regards,
-- 
Ilya Kasnacheev

[jira] [Created] (IGNITE-13665) Useless failure trace "IgnitionEx$IgniteNamedInstance$2.apply"

[Ilya Kasnacheev (Jira)]

Ilya Kasnacheev created IGNITE-13665:


 Summary: Useless failure trace 
"IgnitionEx$IgniteNamedInstance$2.apply"
 Key: IGNITE-13665
 URL: https://issues.apache.org/jira/browse/IGNITE-13665
 Project: Ignite
  Issue Type: Bug
  Components: general
Reporter: Ilya Kasnacheev
Assignee: Ilya Kasnacheev
 Fix For: 2.9.1


When failure handler catches an issue in striped pool, maybe in some other 
cases, thread dump is as folows:
{code}
[2020-10-10T10:10:10,100][WARN ][grid-timeout-worker-#22%cluster%][] Possible 
failure suppressed accordingly to a configured handler 
[hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, 
super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet 
[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], 
failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class 
o.a.i.IgniteException: GridWorker [name=sys-stripe-0, 
igniteInstanceName=EPE_CLUSTER_PERF, finished=false, 
heartbeatTs=1601234567890]]]
org.apache.ignite.IgniteException: GridWorker [name=sys-stripe-0, 
igniteInstanceName=instance, finished=false, heartbeatTs=1601234567890]
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1859)
 [ignite-core.jar]
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1854)
 [ignite-core.jar]
at 
org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233)
 [ignite-core.jar]
at 
org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:296) 
[ignite-core.jar]
at 
org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor$TimeoutWorker.body(GridTimeoutProcessor.java:265)
 [ignite-core.jar]
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119) 
[ignite-core.jar]
at java.lang.Thread.run(Thread.java:834) [?:?]
{code}

when the actual stack trace from the relevant thread is hidden somewhere deep 
below. And may even be suppressed. This is a usability nightmare.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: delete is too slow, sometimes even causes OOM

[Denis Magda]

Hi Frank,

Thanks for starting this discussion. Our contributors with SQL experience
should step in.

Generally, I came across this performance issue a week or so ago while
doing training about Ignite. In my case, a simple DELETE was running for a
minute over a table of negligible size.

-
Denis


On Tue, Nov 3, 2020 at 7:53 AM frank li  wrote:

> Current code logic for DELETE is as follows:
> if WHERE clause contains a condition as "key=xxx", it uses fastUpdate
> which remove the related item directly.
>
> else
> do select for update;
> for each row, call closure code "RMV" to remove it.
>
> 1. As "executeSelectForDml" get _KEY and _VAL columns for all condidate
> rows, it often causes OOM when there are a lot of data  to delete. Why do
> we verify "val" during remove operation?
>
> 2. After selection,  why don't we just remove it with cache.remove as
> fastUpdate does?
>
>
>

delete is too slow, sometimes even causes OOM

[frank li]

Current code logic for DELETE is as follows:
if WHERE clause contains a condition as "key=xxx", it uses fastUpdate which 
remove the related item directly.

else
do select for update;
for each row, call closure code "RMV" to remove it.

1. As "executeSelectForDml" get _KEY and _VAL columns for all condidate rows, 
it often causes OOM when there are a lot of data  to delete. Why do we verify 
"val" during remove operation?

2. After selection,  why don't we just remove it with cache.remove as 
fastUpdate does? 

[jira] [Created] (IGNITE-13664) GridCommandHandlerTest.testPersistenceCleanCorruptedCachesCommand fails on Windows platform

[Vyacheslav Koptilin (Jira)]

Vyacheslav Koptilin created IGNITE-13664:


 Summary: 
GridCommandHandlerTest.testPersistenceCleanCorruptedCachesCommand fails on 
Windows platform
 Key: IGNITE-13664
 URL: https://issues.apache.org/jira/browse/IGNITE-13664
 Project: Ignite
  Issue Type: Bug
Reporter: Vyacheslav Koptilin
 Fix For: 2.10


This test fails to start Ignite node due to the following exception:
{noformat}
[2020-11-03 
18:02:02,044][ERROR][test-runner-#1%gridCommandHandlerTest%][IgniteKernal%gridCommandHandlerTest1]
 Exception during start processors, node will be stopped and close 
connections[2020-11-03 
18:02:02,044][ERROR][test-runner-#1%gridCommandHandlerTest%][IgniteKernal%gridCommandHandlerTest1]
 Exception during start processors, node will be stopped and close 
connectionsjava.util.regex.PatternSyntaxException: Unexpected internal error 
near index 1\ at java.util.regex.Pattern.error(Pattern.java:1969) at 
java.util.regex.Pattern.compile(Pattern.java:1708) at 
java.util.regex.Pattern.(Pattern.java:1352) at 
java.util.regex.Pattern.compile(Pattern.java:1028) at 
java.lang.String.split(String.java:2380) at 
java.lang.String.split(String.java:2422) at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.startMemoryRestore(GridCacheDatabaseSharedManager.java:1601)
 at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1282) at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2096)
 at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1748)
 at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1143) at 
org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:641) at 
org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:1231)
 at 
org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:1152)
 at 
org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:1128)
 at 
org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:997)
 at 
org.apache.ignite.util.GridCommandHandlerTest.testPersistenceCleanCorruptedCachesCommand(GridCommandHandlerTest.java:414)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498) at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
 at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
 at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
 at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
 at 
org.apache.ignite.testframework.junits.GridAbstractTest$7.run(GridAbstractTest.java:2375)
 at java.lang.Thread.run(Thread.java:748){noformat}
The root cause of this failure is that {{File.separator}} on Windows platform 
is slash, which is interpreted as an erroneous regular expression by 
{{String.split()}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: 2.9.1 release proposal

[Yaroslav Molochkov]

Nikolay, hi!

Done.

On Tue, Nov 3, 2020 at 10:48 AM Nikolay Izhikov  wrote:

> Hello, Yaroslav.
>
> Looks like we have fixVersion 2.9.1 in Jira.
> Let’s mark all tickets with label «2.9.1-rc1» with fixVersion=2.9.1 and
> use fixVersion instead of label further.
>
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%202.9.1
>
> > 2 нояб. 2020 г., в 19:42, Yaroslav Molochkov 
> написал(а):
> >
> > Guys,
> >
> > should you agree that issues with the 2.9.1-rc tag are good enough
> > for a maintenance release, i'd like to give it a go.
> >
> > On Thu, Oct 29, 2020 at 5:33 PM Alexey Zinoviev 
> > wrote:
> >
> >> Let's discuss the possible planning dates for feature freeze for 2.10,
> for
> >> example? Do you have any plans or ideas?
> >>
> >> чт, 29 окт. 2020 г. в 17:24, Andrey Gura :
> >>
> >>> Hi,
> >>>
> >>> I agree with Alexey. We should release 2.9.1 and start preparing for
> >>> the 2.10 release.
> >>>
> >>> 2.x releases usually take a lot of time. So we can release 2.9.1, and
> >>> even 2.9.2 before 2.10.
> >>>
> >>> On Thu, Oct 29, 2020 at 12:02 PM Alexey Goncharuk
> >>>  wrote:
> 
>  Hello folks,
> 
>  I think we should start both 2.9.1 and 2.10. In practice, maintenance
>  release contains only critical and usability bugfixes (for example, I
> >>> would
>  include this ticket [1] to include in 2.9.1 as it prevents users from
> >>> using
>  tracing) and is released much faster than a minor release.
> 
>  [1] https://issues.apache.org/jira/browse/IGNITE-13640
> 
>  вт, 27 окт. 2020 г. в 21:41, Guru Stron :
> 
> > Hello,
> >
> > Agree with Pavel.
> >
> > On Tue, 27 Oct 2020 at 19:22, Pavel Tupitsyn 
> >>> wrote:
> >
> >> Igniters,
> >>
> >> I think we should plan 2.10 instead of 2.9.1.
> >> ignite-2.9 branch was cut 4 months ago, a bunch of new features are
> > waiting
> >> to be released.
> >>
> >> On Tue, Oct 27, 2020 at 4:23 AM 18624049226 <18624049...@163.com>
> >>> wrote:
> >>
> >>> Hello,
> >>>
> >>> I suggest that the remaining document issue in version 2.9.0 can
> >> be
> >>> solved in version 2.9.1, and it is not a good practice to
> >> postpone
> >>> to
> >>> version 2.10.
> >>>
> >>> 在 2020/10/27 上午2:00, Maxim Muzafarov 写道:
>  Hello,
> 
>  +1
>  Should we start the discussion about the release leader and
> >>> release
> >>> dates?
> 
>  On Tue, 20 Oct 2020 at 10:12, Anton Vinogradov 
> > wrote:
> > +1 to start the 2.9.1 release process once as soon as 2.9
> >>> released.
> >
> > On Mon, Oct 19, 2020 at 8:49 PM Nikolay Izhikov <
> > nizhi...@apache.org>
> >>> wrote:
> >
> >> Hello, Yaroslav.
> >>
> >> I support the idea.
> >> But, we should carefully work with the release scope.
> >>
> >> IGNITE-13477 - fix for a SQL tracing that will be available
> >>> only in
> >>> 2.10
> >> IGNITE-13427 - fix for a new system view that exists in
> >> master
> > only,
> >> we
> >> need to include IGNITE-13409
> >>
> >> Other tickets should be checked, also.
> >> Is this a real fix of a released bug or we fix some issue we
> >>> bring
> >> with
> >> the brand new contribution.
> >>
> >>
> >> I propose to include the following tickets, also:
> >>
> >> CMD tools improvements:
> >>
> >> IGNITE-13488 - Command to print metric value
> >> IGNITE-13426 - Command to print system view content
> >> IGNITE-13422 - Parameter to explicitly enable experimental
> >>> commands
> >>
> >> IGNITE-13380 - Output IgniteSystemProperties via ignite.sh
> >>
> >> New system views:
> >>
> >> IGNITE-13409 Metastorage and DistributedMetastorage viewы.
> >> IGNITE-13408 BinaryMetadata view.
> >>
> >>
> >>> 19 окт. 2020 г., в 18:20, Yaroslav Molochkov <
> > molochko...@gmail.com
> >>>
> >> написал(а):
> >>> Hello, Igniters!
> >>>
> >>> I've compiled a list of tickets that, i think, deserve to be
> >> released
> >>> in
> >> a
> >>> minor Ignite release (meaning in 2.9.1) after 2.9. Here they
> >>> are:
> >>>
> >>> IGNITE-13569
> >>> disable archiving + walCompactionEnabled probably broke
> >>> reading
> > from
> >>> wal
> >> on
> >>> server restart
> >>>
> >>> IGNITE-13418
> >>> Deadlock on multiple cache delete
> >>>
> >>> IGNITE-13563
> >>> Deserializing IBinaryObject containing an IBinaryObject
> >> field
> > fails
> >>>
> >>> IGNITE-13575
> >>> Invalid blocking section in GridNioWorker and
> >>> GridNioClientWorker
> 

[jira] [Created] (IGNITE-13663) Represent in the documenttion affection of several node addresses on failure detection v2.

[Vladimir Steshin (Jira)]

Vladimir Steshin created IGNITE-13663:
-

 Summary: Represent in the documenttion affection of several node 
addresses on failure detection v2.
 Key: IGNITE-13663
 URL: https://issues.apache.org/jira/browse/IGNITE-13663
 Project: Ignite
  Issue Type: Improvement
  Components: documentation
Affects Versions: 2.9
Reporter: Vladimir Steshin
Assignee: Vladimir Steshin
 Fix For: 2.10






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Interoperable Ignite.NET Dates

[Alexey Kukushkin]

Pavel,

Thank you for your positive feedback! I will create an IEP.

вт, 3 нояб. 2020 г. в 15:48, Pavel Tupitsyn :

> Alexey,
>
> The proposal looks good to me.
>
> Usually I would try to avoid extra dependencies in the core assembly,
> but NodaTime seems to be worth it.
>
> Would you like to prepare an IEP?
>
> Thanks,
> Pavel
>
>
>
> On Tue, Nov 3, 2020 at 3:15 PM Alexey Kukushkin  >
> wrote:
>
> > We already created tickets and tested the proposed solution with our
> > applications (already in PROD) and an internal fork of Ignite. This
> > discussion is a proposal to donate this change to open source Apache
> Ignite
> > is the community accepts it:
> >
> >- IGNITE-12824 .NET: Write DateTime in interoperable format by default
> >
> >- IGNITE-12825 Serialize Java and .NET dates using same calendars
> >
> >
> >
> > вт, 3 нояб. 2020 г. в 15:08, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com>:
> >
> > > Cool, makes sense to me then. Please feel free to create a ticket and
> > mark
> > > it with the "ignite-3" label.
> > >
> > > -Val
> > >
> > > On Tue, Nov 3, 2020 at 4:03 AM Alexey Kukushkin <
> > kukushkinale...@gmail.com
> > > >
> > > wrote:
> > >
> > > > Val, absolutely - our proposal affects .NET API only.
> > > >
> > > > вт, 3 нояб. 2020 г. в 15:00, Valentin Kulichenko <
> > > > valentin.kuliche...@gmail.com>:
> > > >
> > > > > Got it. So it only affects the .NET side, correct?
> > > > >
> > > > > -Val
> > > > >
> > > > > On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin <
> > > > kukushkinale...@gmail.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Val,
> > > > > >
> > > > > > Our proposal does not overlap with IEP-54
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > > > >,
> > > > > > which proposes changing Ignite date storage format. Our proposal
> is
> > > > > > enhancing Ignite to handle .NET dates in a truly portable way
> > instead
> > > > of
> > > > > > requiring the application developers to repeat this task every
> > time:
> > > > > >
> > > > > >- Write .NET dates as Ignite dates (today .NET dates are
> written
> > > as
> > > > > >generic Ignite objects)
> > > > > >- Convert Local .NET dates to UTC inside Ignite and to it
> using
> > > Java
> > > > > >calendars.
> > > > > >
> > > > > >
> > > > > > вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> > > > > > valentin.kuliche...@gmail.com>:
> > > > > >
> > > > > > > Hi Alexey,
> > > > > > >
> > > > > > > The IEP-54 [1] describes the data layout proposed for Ignite
> 3.0,
> > > it
> > > > > > > includes various date/time types. Can you please take a look
> and
> > > > check
> > > > > if
> > > > > > > this addresses your concerns? We can then discuss further if
> > > needed.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > > > >
> > > > > > > -Val
> > > > > > >
> > > > > > > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > > > > > > kukushkinale...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Pavel,
> > > > > > > >
> > > > > > > > We propose changing public API so this is for Ignite 3.0.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > >
> > > > > > > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn <
> > > ptupit...@apache.org
> > > > >:
> > > > > > > >
> > > > > > > > > Alexey,
> > > > > > > > >
> > > > > > > > > Just to clarify before we start the discussion:
> > > > > > > > > this proposal seems to introduce some breaking changes, so
> we
> > > are
> > > > > > > talking
> > > > > > > > > about Ignite 3.0, correct?
> > > > > > > > >
> > > > > > > > > Pavel
> > > > > > > > >
> > > > > > > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > > > > > > kukushkinale...@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Igniters,
> > > > > > > > > >
> > > > > > > > > > What do you think about changing .NET API to read/write
> > > > portable
> > > > > > > dates
> > > > > > > > by
> > > > > > > > > > default and making that really portable?
> > > > > > > > > >
> > > > > > > > > > *The Problem*
> > > > > > > > > > Presently .NET API writes dates as composite Ignite
> > objects.
> > > > Only
> > > > > > > .NET
> > > > > > > > > > clients can read such dates: any other client (JDBC,
> Java,
> > > etc)
> > > > > > does
> > > > > > > > not
> > > > > > > > > > understand it without custom deserialization.
> > > > > > > > > >
> > > > > > > > > > It is still possible to configure .NET serialization to
> > write
> > > > > dates
> > > > > > > as
> > > > > > > > > > Ignite dates - see DateTime Serialization note
> > > > > > > > > > <
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> 

[jira] [Created] (IGNITE-13662) Discribe soLinger setting in TCP Discovery and SSL issues.

[Vladimir Steshin (Jira)]

Vladimir Steshin created IGNITE-13662:
-

 Summary: Discribe soLinger setting in TCP Discovery and SSL issues.
 Key: IGNITE-13662
 URL: https://issues.apache.org/jira/browse/IGNITE-13662
 Project: Ignite
  Issue Type: Improvement
Reporter: Vladimir Steshin
Assignee: Vladimir Steshin


Discribe soLinger setting in TCP Discovery and SSL issues.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Interoperable Ignite.NET Dates

[Pavel Tupitsyn]

Alexey,

The proposal looks good to me.

Usually I would try to avoid extra dependencies in the core assembly,
but NodaTime seems to be worth it.

Would you like to prepare an IEP?

Thanks,
Pavel



On Tue, Nov 3, 2020 at 3:15 PM Alexey Kukushkin 
wrote:

> We already created tickets and tested the proposed solution with our
> applications (already in PROD) and an internal fork of Ignite. This
> discussion is a proposal to donate this change to open source Apache Ignite
> is the community accepts it:
>
>- IGNITE-12824 .NET: Write DateTime in interoperable format by default
>
>- IGNITE-12825 Serialize Java and .NET dates using same calendars
>
>
>
> вт, 3 нояб. 2020 г. в 15:08, Valentin Kulichenko <
> valentin.kuliche...@gmail.com>:
>
> > Cool, makes sense to me then. Please feel free to create a ticket and
> mark
> > it with the "ignite-3" label.
> >
> > -Val
> >
> > On Tue, Nov 3, 2020 at 4:03 AM Alexey Kukushkin <
> kukushkinale...@gmail.com
> > >
> > wrote:
> >
> > > Val, absolutely - our proposal affects .NET API only.
> > >
> > > вт, 3 нояб. 2020 г. в 15:00, Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com>:
> > >
> > > > Got it. So it only affects the .NET side, correct?
> > > >
> > > > -Val
> > > >
> > > > On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin <
> > > kukushkinale...@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi Val,
> > > > >
> > > > > Our proposal does not overlap with IEP-54
> > > > > <
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > > >,
> > > > > which proposes changing Ignite date storage format. Our proposal is
> > > > > enhancing Ignite to handle .NET dates in a truly portable way
> instead
> > > of
> > > > > requiring the application developers to repeat this task every
> time:
> > > > >
> > > > >- Write .NET dates as Ignite dates (today .NET dates are written
> > as
> > > > >generic Ignite objects)
> > > > >- Convert Local .NET dates to UTC inside Ignite and to it using
> > Java
> > > > >calendars.
> > > > >
> > > > >
> > > > > вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> > > > > valentin.kuliche...@gmail.com>:
> > > > >
> > > > > > Hi Alexey,
> > > > > >
> > > > > > The IEP-54 [1] describes the data layout proposed for Ignite 3.0,
> > it
> > > > > > includes various date/time types. Can you please take a look and
> > > check
> > > > if
> > > > > > this addresses your concerns? We can then discuss further if
> > needed.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > > >
> > > > > > -Val
> > > > > >
> > > > > > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > > > > > kukushkinale...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Pavel,
> > > > > > >
> > > > > > > We propose changing public API so this is for Ignite 3.0.
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn <
> > ptupit...@apache.org
> > > >:
> > > > > > >
> > > > > > > > Alexey,
> > > > > > > >
> > > > > > > > Just to clarify before we start the discussion:
> > > > > > > > this proposal seems to introduce some breaking changes, so we
> > are
> > > > > > talking
> > > > > > > > about Ignite 3.0, correct?
> > > > > > > >
> > > > > > > > Pavel
> > > > > > > >
> > > > > > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > > > > > kukushkinale...@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Igniters,
> > > > > > > > >
> > > > > > > > > What do you think about changing .NET API to read/write
> > > portable
> > > > > > dates
> > > > > > > by
> > > > > > > > > default and making that really portable?
> > > > > > > > >
> > > > > > > > > *The Problem*
> > > > > > > > > Presently .NET API writes dates as composite Ignite
> objects.
> > > Only
> > > > > > .NET
> > > > > > > > > clients can read such dates: any other client (JDBC, Java,
> > etc)
> > > > > does
> > > > > > > not
> > > > > > > > > understand it without custom deserialization.
> > > > > > > > >
> > > > > > > > > It is still possible to configure .NET serialization to
> write
> > > > dates
> > > > > > as
> > > > > > > > > Ignite dates - see DateTime Serialization note
> > > > > > > > > <
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > > > > > > >.
> > > > > > > > > But then Ignite accepts only UTC dates, requiring the
> > > application
> > > > > > > > > developers to convert local dates to UTC dates and back.
> This
> > > > task
> > > > > is
> > > > > > > not
> > > > > > > > > trivial: DateTime.ToUniversalTime
> > > > > > > > > <
> > > > > > > > >
> > > > > > > >
> > > > > 

Re: Custom Affinity Functions proposed for removal?

[Valentin Kulichenko]

Raymond,

Thanks for the details. So it sounds like you have a custom affinity
mapper, not affinity function. This makes things simpler, but I'm still
failing to understand why standard mechanisms for collocation [1] didn't
work for you. Could you please clarify?

[1] https://ignite.apache.org/docs/latest/data-modeling/affinity-collocation

-Val

On Tue, Nov 3, 2020 at 12:25 AM Raymond Wilson 
wrote:

> In terms of Key -> Partition -> Node mapping, we provide customer affinity
> mappers for the Key -> Partition stage an allow Ignite to map partitions to
> nodes.
>
> Our keys are structs with multiple fields forming a composite primary key,
> parts of which are spatially identifying and parts contain other
> characteristics. We want to ensure all related spatial elements resolve to
> the same partition and so will be placed on the same node for query
> colocation purposes.
>
>
>
> On Tue, Nov 3, 2020 at 9:20 PM Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > Moti, Raymond,
> >
> > Could you please describe your use cases in more detail? What are the
> types
> > used as cache keys? What is the custom logic that you use for affinity
> > mapping? What was the exact reason to customize versus using built-in
> > collocation mechanisms?
> >
> > Ultimately, I'm sure that custom affinity functions will go away cause
> they
> > introduce multiple potential issues. However, I would definitely like to
> > make sure that your use cases are still supported in Ignite 3.0 via some
> > other means.
> >
> > -Val
> >
> > On Mon, Nov 2, 2020 at 12:21 PM Alexei Scherbakov <
> > alexey.scherbak...@gmail.com> wrote:
> >
> > > Thanks for the clarification.
> > >
> > > There was no intention to remove the customizable key to partition
> > mapping.
> > >
> > > Difficulties arise when mapping partitions to nodes, so it's desirable
> to
> > > have internally tested implementation with a way to customize it's
> > behavior
> > > without additional coding on the user side.
> > >
> > > пн, 2 нояб. 2020 г. в 23:01, Raymond Wilson <
> raymond_wil...@trimble.com
> > >:
> > >
> > > > Just to be clear, the affinity functions we are using convert keys to
> > > > partitions, we do not map partitions to nodes and leave that to
> Ignite.
> > > >
> > > > On Tue, Nov 3, 2020 at 8:48 AM Alexei Scherbakov <
> > > > alexey.scherbak...@gmail.com> wrote:
> > > >
> > > > > Hello.
> > > > >
> > > > > Custom affinity functions can cause weird bugs and data loss if
> > > > implemented
> > > > > wrongly.
> > > > > There is an intention  to keep a backup filter based on user
> > attributes
> > > > > (with additional validation logic to ensure correctness) for
> > > controllable
> > > > > data placement.
> > > > >
> > > > > Can you describe more precisely why you had to implement custom
> > > affinity
> > > > > functions and not resort to default rendezvous affinity + backup
> > > filter ?
> > > > >
> > > > >
> > > > > пн, 2 нояб. 2020 г. в 21:45, Raymond Wilson <
> > > raymond_wil...@trimble.com
> > > > >:
> > > > >
> > > > > > We also use custom affinity functions (vis the C# client).
> > > > > >
> > > > > > The wish list mentions use of a particular annotation
> > > > > > (@CentralizedAffinityFunction):
> > > > > > Is the wish to remove just this annotation, or the ability to
> > define
> > > > > custom
> > > > > > affinity functions at all?
> > > > > >
> > > > > > In our case we use affinity functions to ensure particular
> > > distribution
> > > > > of
> > > > > > spatial data across a processing cluster to ensure even load
> > > > management.
> > > > > >
> > > > > > On Tue, Nov 3, 2020 at 5:31 AM Moti Nisenson <
> > > moti.nisen...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > I saw at
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0+Wishlist
> > > > > > > that custom affinity functions are on the potential wishlist
> for
> > > > > removal.
> > > > > > > The way we're using it's very critical that we be able to
> control
> > > the
> > > > > > > placement of data quite precisely - as part of that we specify
> > > > > explicitly
> > > > > > > the partition we want in the key, and then our affinity
> function
> > > uses
> > > > > > that
> > > > > > > (else delegating to default rendezvous). We don't need all the
> > > > > > > abilities there, although I think that often others do.
> > > > > > >
> > > > > > > This seems to me to be a case that the benefit of removing this
> > is
> > > > > > minimal
> > > > > > > and could cause quite a lot of disruption to users.
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > 
> > > > > > Raymond Wilson
> > > > > > Solution Architect, Civil Construction Software Systems (CCSS)
> > > > > > 11 Birmingham Drive | Christchurch, New Zealand
> > > > > > +64-21-2013317 Mobile
> > > > > > raymond_wil...@trimble.com
> > > > > >
> > > > > 

Re: [DISCUSS] Ignite 3.0 development approach

[Valentin Kulichenko]

Ksenia, thanks for scheduling this on such short notice!

As for the original topic, I do support Alexey's idea. We're not going to
rewrite anything from scratch, as most of the components are going to be
moved as-is or with minimal modifications. However, the changes that are
proposed imply serious rework of the core parts of the code, which are not
properly decoupled from each other and from other parts. This makes the
incremental approach borderline impossible. Developing in a new repo,
however, addresses this concern. As a bonus, we can also refactor the code,
introduce better decoupling, get rid of kernel context, and develop unit
tests (finally!).

Basically, this proposal only affects the *process*, not the set of changes
we had discussed before. Ignite 3.0 is our unique chance to make things
right.

-Val

On Tue, Nov 3, 2020 at 3:06 AM Kseniya Romanova 
wrote:

> Pavel, all the interesting points will be anyway published here in English
> (as the principal "if it's not on devlist it doesn't happened" is still
> relevant). This is just a quick call for a group of developers. Later we
> can do a separate presentation of idea and discussion in English as we did
> for the Ignite 3.0 draft of changes.
>
> вт, 3 нояб. 2020 г. в 13:52, Pavel Tupitsyn :
>
> > Kseniya,
> >
> > Thanks for scheduling this call.
> > Do you think we can switch to English if non-Russian speaking community
> > members decide to join?
> >
> > On Tue, Nov 3, 2020 at 1:32 PM Kseniya Romanova <
> romanova.ks@gmail.com
> > >
> > wrote:
> >
> > > Let's do this community discussion open. Here's the link on zoom call
> in
> > > Russian for Friday 6 PM:
> > > https://www.meetup.com/Moscow-Apache-Ignite-Meetup/events/274360378/
> > >
> > > вт, 3 нояб. 2020 г. в 12:49, Nikolay Izhikov :
> > >
> > > > Time works for me.
> > > >
> > > > > 3 нояб. 2020 г., в 12:40, Alexey Goncharuk <
> > alexey.goncha...@gmail.com
> > > >
> > > > написал(а):
> > > > >
> > > > > Nikolay,
> > > > >
> > > > > I am up for the call. I will try to explain my reasoning in greater
> > > > detail
> > > > > and will be glad to hear the concerns. Will this Friday, Nov 6th,
> > work?
> > > > >
> > > > > вт, 3 нояб. 2020 г. в 10:09, Nikolay Izhikov  >:
> > > > >
> > > > >> Igniters, should we have a call for this topic?
> > > > >>
> > > > >>> 2 нояб. 2020 г., в 18:53, Pavel Tupitsyn 
> > > > >> написал(а):
> > > > >>>
> > > >  not intend to rewrite everything from scratch
> > > > >>>
> > > >  Every single test from Ignite 2.x should be moved to Ignite 3
> > > >  regardless of how we choose to proceed.
> > > > >>>
> > > > >>> Alexey, thank you for the explanation, this addresses all of my
> > > > concerns.
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> On Mon, Nov 2, 2020 at 6:43 PM Andrey Mashenkov <
> > > > >> andrey.mashen...@gmail.com>
> > > > >>> wrote:
> > > > >>>
> > > >  Hi, Igniters.
> > > > 
> > > >  * AFAIU, we need a new repo if we want to apply different
> > > restrictions
> > > > >> to
> > > >  pull requests,
> > > >  otherwise I see no difference for myself.
> > > >  E.g. make static analysis (do we have?), compile, styles, and
> > > javadoc
> > > >  checks mandatory.
> > > > 
> > > >  I think that relaxed requirements here will lead to bad product
> > > > quality.
> > > > 
> > > >  * Agree with Pavel, we should 'keep' integrations tests somehow.
> > > >  During active development tests will be broken most of time, so,
> > > >  I'd port them e.g. suite-by-suite once we will have a stable and
> > > > >> featured
> > > >  environment to run them and of course make test's code clear and
> > > avoid
> > > >  bad/non-relevant ones.
> > > > 
> > > >  * I like bottom-up approach.
> > > >  With it we could make a better framework. I mean clear component
> > > > >> lifecycle,
> > > >  component wiring mechanics, general methods to approach core
> > > > components
> > > >  such as exchange/communication
> > > >  to avoid code mess like we have in ExchangeFuture with all these
> > > > custom
> > > >  callbacks for each component, interfaces like
> > > >  PartitionsExchangeAware, IgniteChangeGlobalStateSupport and
> > > >  a pack of
> > > > >> start/stop/onKernalStart/onPluginStart/onActivate/onDisconnected
> > > >  and so on in various unexpected places.
> > > >  Hope, we will be able to port most of the good code to the new
> > > > framework
> > > >  version.
> > > > 
> > > > 
> > > > 
> > > >  On Mon, Nov 2, 2020 at 6:18 PM Alexey Goncharuk <
> > > >  alexey.goncha...@gmail.com>
> > > >  wrote:
> > > > 
> > > > > Nikolay, Pavel,
> > > > >
> > > > > Thanks for the feedback! First of all, I wanted to stress that
> I
> > do
> > > > not
> > > > > intend to rewrite everything from scratch (I never used this
> > > phrase).
> > > >  There
> > > > > are significant parts of 

[jira] [Created] (IGNITE-13661) Example for transactions in thin clients

[Stepan Pilschikov (Jira)]

Stepan Pilschikov created IGNITE-13661:
--

 Summary: Example for transactions in thin clients
 Key: IGNITE-13661
 URL: https://issues.apache.org/jira/browse/IGNITE-13661
 Project: Ignite
  Issue Type: Wish
  Components: examples, thin client
Reporter: Stepan Pilschikov


We have no examples for many main features in thin clients
Not so long time ago was added transactions in Dotnet
It's better to add appropriate examples for these features



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Interoperable Ignite.NET Dates

[Alexey Kukushkin]

We already created tickets and tested the proposed solution with our
applications (already in PROD) and an internal fork of Ignite. This
discussion is a proposal to donate this change to open source Apache Ignite
is the community accepts it:

   - IGNITE-12824 .NET: Write DateTime in interoperable format by default
   
   - IGNITE-12825 Serialize Java and .NET dates using same calendars
   


вт, 3 нояб. 2020 г. в 15:08, Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Cool, makes sense to me then. Please feel free to create a ticket and mark
> it with the "ignite-3" label.
>
> -Val
>
> On Tue, Nov 3, 2020 at 4:03 AM Alexey Kukushkin  >
> wrote:
>
> > Val, absolutely - our proposal affects .NET API only.
> >
> > вт, 3 нояб. 2020 г. в 15:00, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com>:
> >
> > > Got it. So it only affects the .NET side, correct?
> > >
> > > -Val
> > >
> > > On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin <
> > kukushkinale...@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hi Val,
> > > >
> > > > Our proposal does not overlap with IEP-54
> > > > <
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > >,
> > > > which proposes changing Ignite date storage format. Our proposal is
> > > > enhancing Ignite to handle .NET dates in a truly portable way instead
> > of
> > > > requiring the application developers to repeat this task every time:
> > > >
> > > >- Write .NET dates as Ignite dates (today .NET dates are written
> as
> > > >generic Ignite objects)
> > > >- Convert Local .NET dates to UTC inside Ignite and to it using
> Java
> > > >calendars.
> > > >
> > > >
> > > > вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> > > > valentin.kuliche...@gmail.com>:
> > > >
> > > > > Hi Alexey,
> > > > >
> > > > > The IEP-54 [1] describes the data layout proposed for Ignite 3.0,
> it
> > > > > includes various date/time types. Can you please take a look and
> > check
> > > if
> > > > > this addresses your concerns? We can then discuss further if
> needed.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > >
> > > > > -Val
> > > > >
> > > > > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > > > > kukushkinale...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Pavel,
> > > > > >
> > > > > > We propose changing public API so this is for Ignite 3.0.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn <
> ptupit...@apache.org
> > >:
> > > > > >
> > > > > > > Alexey,
> > > > > > >
> > > > > > > Just to clarify before we start the discussion:
> > > > > > > this proposal seems to introduce some breaking changes, so we
> are
> > > > > talking
> > > > > > > about Ignite 3.0, correct?
> > > > > > >
> > > > > > > Pavel
> > > > > > >
> > > > > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > > > > kukushkinale...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Igniters,
> > > > > > > >
> > > > > > > > What do you think about changing .NET API to read/write
> > portable
> > > > > dates
> > > > > > by
> > > > > > > > default and making that really portable?
> > > > > > > >
> > > > > > > > *The Problem*
> > > > > > > > Presently .NET API writes dates as composite Ignite objects.
> > Only
> > > > > .NET
> > > > > > > > clients can read such dates: any other client (JDBC, Java,
> etc)
> > > > does
> > > > > > not
> > > > > > > > understand it without custom deserialization.
> > > > > > > >
> > > > > > > > It is still possible to configure .NET serialization to write
> > > dates
> > > > > as
> > > > > > > > Ignite dates - see DateTime Serialization note
> > > > > > > > <
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > > > > > >.
> > > > > > > > But then Ignite accepts only UTC dates, requiring the
> > application
> > > > > > > > developers to convert local dates to UTC dates and back. This
> > > task
> > > > is
> > > > > > not
> > > > > > > > trivial: DateTime.ToUniversalTime
> > > > > > > > <
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > > > > > > >
> > > > > > > > uses
> > > > > > > > calendars different from Java (and the .NET calendars are the
> > > > invalid
> > > > > > > ones
> > > > > > > > - especially for pre-1990 dates).
> > > > > > > >
> > > > > > > > The motivation for the current default behavior was probably
> > the
> > > > > desire
> > > > > > > to
> > > > > > > > keep the Time Zone information: Ignite dates do not store
> time
> > > > zones.
> > > > > > > >
> > > > > > > > In our 

Re: usage analytics

[Valentin Kulichenko]

Makes sense to me. I would love to know which components/APIs are used more
than others. Obviously, we should make sure everything is anonymous and we
don't collect any private user data, but I believe this is already
guaranteed by Google Analytics.

-Val

On Tue, Nov 3, 2020 at 3:59 AM Alexey Goncharuk 
wrote:

> Folks,
>
> I want to bump up this discussion and slightly change the format suggested
> by Nikita. I dot think it is correct to gather any information related to
> the user environment. However, can we collect just the fact of some of the
> Ignite APIs/subsystems being used with no user information whatsoever?
> Having started thinking about Ignite 3.0 I realized that we lack even some
> very basic knowledge on the impact of changing one or another feature or
> API.
>
> To my knowledge, the Ignite website already uses google analytics which is
> available to the community. The google analytics platform already has
> tooling to track app screen hits in a completely anonymous way, so we can
> use this tool to track Ignite components usage (once per node startup)
> sending solely component name and a unique environment hash - no IP
> addresses, no jdk/os/other information. The information will be available
> in the same toolkit we are already using to analyze the website and
> optimize our docs.
>
> WDYT?
>
> ср, 19 июл. 2017 г. в 01:15, :
>
> > I would try to ping legal again and see if they respond. If not, I think
> > we will need to come up with a simpler approach, that does not require
> > legal approval.
> >
> > ⁣D.​
> >
> > On Jul 18, 2017, 2:23 PM, at 2:23 PM, Nikita Ivanov  >
> > wrote:
> > >Igniters,
> > >Just a quick update. I haven't gotten response from ASF Legal on this
> > >thread and I frankly don't know how to proceed here. What's the process
> > >to
> > >arrive to a decision point here?
> > >
> > >Thanks!
> > >--
> > >Nikita Ivanov
> > >
> > >
> > >On Mon, Jul 10, 2017 at 3:11 PM, Konstantin Boudnik 
> > >wrote:
> > >
> > >> On Sat, Jul 08, 2017 at 11:04AM, Nikita Ivanov wrote:
> > >> > Cos,
> > >> > Based on my experience having it off by default negates the entire
> > >> > purpose... We need statistically meaningful data set to make any
> > >> inferences
> > >> > from it. Moreover, if we are going to ask folks to turn it on it
> > >will
> > >> > significantly skew the resulting data set anyways and show full
> > >picture.
> > >> I
> > >> > think "on" by default is the better option if we are to collect
> > >usage
> > >> stats
> > >> > to begin with.
> > >>
> > >> yes, sure. But having this "on" by default is likely to expose us to
> > >> another
> > >> shit-storm down the road. An interesting dilemma to have indeed. In
> > >my
> > >> experience, whenever I install something like a browser or an
> > >operating
> > >> system, it would ask if I want to make the particular piece of
> > >software
> > >> better
> > >> by sending back some anonymized stats. Basically, I am given a way to
> > >> explicitly opt-out if I wish.
> > >>
> > >> By turning the feature "on" by default is like saying: "we'll be
> > >collecting
> > >> some stats, but if you don't want to you can go here and there and
> > >disable
> > >> the
> > >> collection. Oh, and by the way - you need to go and figure out the
> > >exact
> > >> steps
> > >> to disable it."
> > >>
> > >> > Also, I want to re-iterate it again to avoid misunderstanding:
> > >there is
> > >> no
> > >> > proposal nor will there be a technical way to attribute collected
> > >data
> > >> back
> > >> > to a certain company. That's not what this is all about. We should
> > >only
> > >> be
> > >> > interested in aggregated stats (community size, geo information,
> > >language
> > >> > information, components usage).
> > >>
> > >> Yes, I think it is clear, but never hurts to re-iterate.
> > >>
> > >> Cos
> > >>
> > >> > Thoughts?
> > >> >
> > >> > --
> > >> > Nikita Ivanov
> > >> > Founder & CTO
> > >> > GridGain Systems
> > >> >
> > >> > On Fri, Jul 7, 2017 at 8:17 PM, Konstantin Boudnik 
> > >> wrote:
> > >> >
> > >> > > Actually, that should be OFF by default. It sounds like this
> > >reduce the
> > >> > > amount
> > >> > > of the data collected, but this would address the concerns of
> > >companies
> > >> > > like
> > >> > > Roman's. I know for sure that a few of my clients would sue my
> > >ass out
> > >> of
> > >> > > existence if I gave them the platform collecting their
> > >data-centers
> > >> info.
> > >> > >
> > >> > > Let's have it, set if off by default and document and easy way to
> > >turn
> > >> it
> > >> > > off.
> > >> > > Then start making rounds asking our user base to share _some_ of
> > >the
> > >> stats
> > >> > > with the community, so we can track the growth of the install
> > >base,
> > >> etc.
> > >> > >
> > >> > > Cos
> > >> > >
> > >> > > On Thu, Jul 06, 2017 at 08:20AM, Nikita Ivanov wrote:
> > >> > > > The idea so far is to have a single system property in
> > >configuration
> > >> that
> > >> > > > turns this off 

Re: Interoperable Ignite.NET Dates

[Valentin Kulichenko]

Cool, makes sense to me then. Please feel free to create a ticket and mark
it with the "ignite-3" label.

-Val

On Tue, Nov 3, 2020 at 4:03 AM Alexey Kukushkin 
wrote:

> Val, absolutely - our proposal affects .NET API only.
>
> вт, 3 нояб. 2020 г. в 15:00, Valentin Kulichenko <
> valentin.kuliche...@gmail.com>:
>
> > Got it. So it only affects the .NET side, correct?
> >
> > -Val
> >
> > On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin <
> kukushkinale...@gmail.com
> > >
> > wrote:
> >
> > > Hi Val,
> > >
> > > Our proposal does not overlap with IEP-54
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > >,
> > > which proposes changing Ignite date storage format. Our proposal is
> > > enhancing Ignite to handle .NET dates in a truly portable way instead
> of
> > > requiring the application developers to repeat this task every time:
> > >
> > >- Write .NET dates as Ignite dates (today .NET dates are written as
> > >generic Ignite objects)
> > >- Convert Local .NET dates to UTC inside Ignite and to it using Java
> > >calendars.
> > >
> > >
> > > вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com>:
> > >
> > > > Hi Alexey,
> > > >
> > > > The IEP-54 [1] describes the data layout proposed for Ignite 3.0, it
> > > > includes various date/time types. Can you please take a look and
> check
> > if
> > > > this addresses your concerns? We can then discuss further if needed.
> > > >
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > >
> > > > -Val
> > > >
> > > > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > > > kukushkinale...@gmail.com>
> > > > wrote:
> > > >
> > > > > Pavel,
> > > > >
> > > > > We propose changing public API so this is for Ignite 3.0.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn  >:
> > > > >
> > > > > > Alexey,
> > > > > >
> > > > > > Just to clarify before we start the discussion:
> > > > > > this proposal seems to introduce some breaking changes, so we are
> > > > talking
> > > > > > about Ignite 3.0, correct?
> > > > > >
> > > > > > Pavel
> > > > > >
> > > > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > > > kukushkinale...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > What do you think about changing .NET API to read/write
> portable
> > > > dates
> > > > > by
> > > > > > > default and making that really portable?
> > > > > > >
> > > > > > > *The Problem*
> > > > > > > Presently .NET API writes dates as composite Ignite objects.
> Only
> > > > .NET
> > > > > > > clients can read such dates: any other client (JDBC, Java, etc)
> > > does
> > > > > not
> > > > > > > understand it without custom deserialization.
> > > > > > >
> > > > > > > It is still possible to configure .NET serialization to write
> > dates
> > > > as
> > > > > > > Ignite dates - see DateTime Serialization note
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > > > > >.
> > > > > > > But then Ignite accepts only UTC dates, requiring the
> application
> > > > > > > developers to convert local dates to UTC dates and back. This
> > task
> > > is
> > > > > not
> > > > > > > trivial: DateTime.ToUniversalTime
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > > > > > >
> > > > > > > uses
> > > > > > > calendars different from Java (and the .NET calendars are the
> > > invalid
> > > > > > ones
> > > > > > > - especially for pre-1990 dates).
> > > > > > >
> > > > > > > The motivation for the current default behavior was probably
> the
> > > > desire
> > > > > > to
> > > > > > > keep the Time Zone information: Ignite dates do not store time
> > > zones.
> > > > > > >
> > > > > > > In our experience interoperability is more important than
> storing
> > > > time
> > > > > > zone
> > > > > > > info.
> > > > > > >
> > > > > > > *The Proposal*
> > > > > > >
> > > > > > >1. Always write .NET dates as portable Ignite dates: get rid
> > of
> > > > the
> > > > > > >BinaryReflectiveSerializer.ForceTimestamp flag that
> currently
> > > > > triggers
> > > > > > > this
> > > > > > >behavior.
> > > > > > >   - We could still keep the ForceTimestamp flag if saving
> > .NET
> > > > > dates
> > > > > > as
> > > > > > >   transparent objects seems a useful case. We do not think
> it
> > > is
> > > > > > > useful.
> > > > > > >2. Automatically convert Local dates to UTC and back
> *inside*
> > > > > > >Ignite.NET.
> > > > > > >   - In this case we lose the DateTime.Kind of UTC dates: we
> > > > write a
> > > > > > UTC
> > > > > > >   date but we would read a Local date since Ignite 

Re: Interoperable Ignite.NET Dates

[Alexey Kukushkin]

Val, absolutely - our proposal affects .NET API only.

вт, 3 нояб. 2020 г. в 15:00, Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Got it. So it only affects the .NET side, correct?
>
> -Val
>
> On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin  >
> wrote:
>
> > Hi Val,
> >
> > Our proposal does not overlap with IEP-54
> > <
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > >,
> > which proposes changing Ignite date storage format. Our proposal is
> > enhancing Ignite to handle .NET dates in a truly portable way instead of
> > requiring the application developers to repeat this task every time:
> >
> >- Write .NET dates as Ignite dates (today .NET dates are written as
> >generic Ignite objects)
> >- Convert Local .NET dates to UTC inside Ignite and to it using Java
> >calendars.
> >
> >
> > вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com>:
> >
> > > Hi Alexey,
> > >
> > > The IEP-54 [1] describes the data layout proposed for Ignite 3.0, it
> > > includes various date/time types. Can you please take a look and check
> if
> > > this addresses your concerns? We can then discuss further if needed.
> > >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > >
> > > -Val
> > >
> > > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > > kukushkinale...@gmail.com>
> > > wrote:
> > >
> > > > Pavel,
> > > >
> > > > We propose changing public API so this is for Ignite 3.0.
> > > >
> > > > Thanks!
> > > >
> > > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn :
> > > >
> > > > > Alexey,
> > > > >
> > > > > Just to clarify before we start the discussion:
> > > > > this proposal seems to introduce some breaking changes, so we are
> > > talking
> > > > > about Ignite 3.0, correct?
> > > > >
> > > > > Pavel
> > > > >
> > > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > > kukushkinale...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > What do you think about changing .NET API to read/write portable
> > > dates
> > > > by
> > > > > > default and making that really portable?
> > > > > >
> > > > > > *The Problem*
> > > > > > Presently .NET API writes dates as composite Ignite objects. Only
> > > .NET
> > > > > > clients can read such dates: any other client (JDBC, Java, etc)
> > does
> > > > not
> > > > > > understand it without custom deserialization.
> > > > > >
> > > > > > It is still possible to configure .NET serialization to write
> dates
> > > as
> > > > > > Ignite dates - see DateTime Serialization note
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > > > >.
> > > > > > But then Ignite accepts only UTC dates, requiring the application
> > > > > > developers to convert local dates to UTC dates and back. This
> task
> > is
> > > > not
> > > > > > trivial: DateTime.ToUniversalTime
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > > > > >
> > > > > > uses
> > > > > > calendars different from Java (and the .NET calendars are the
> > invalid
> > > > > ones
> > > > > > - especially for pre-1990 dates).
> > > > > >
> > > > > > The motivation for the current default behavior was probably the
> > > desire
> > > > > to
> > > > > > keep the Time Zone information: Ignite dates do not store time
> > zones.
> > > > > >
> > > > > > In our experience interoperability is more important than storing
> > > time
> > > > > zone
> > > > > > info.
> > > > > >
> > > > > > *The Proposal*
> > > > > >
> > > > > >1. Always write .NET dates as portable Ignite dates: get rid
> of
> > > the
> > > > > >BinaryReflectiveSerializer.ForceTimestamp flag that currently
> > > > triggers
> > > > > > this
> > > > > >behavior.
> > > > > >   - We could still keep the ForceTimestamp flag if saving
> .NET
> > > > dates
> > > > > as
> > > > > >   transparent objects seems a useful case. We do not think it
> > is
> > > > > > useful.
> > > > > >2. Automatically convert Local dates to UTC and back *inside*
> > > > > >Ignite.NET.
> > > > > >   - In this case we lose the DateTime.Kind of UTC dates: we
> > > write a
> > > > > UTC
> > > > > >   date but we would read a Local date since Ignite would
> always
> > > > > > convert UTC
> > > > > >   to Local when reading.
> > > > > >   We could add a UtcDate date flag to QuerySqlFieldAttribute
> > > > > >   and BinaryReflectiveSerializer to control the
> deserialization
> > > > > > behavior if
> > > > > >   keeping dates in UTC format use case seems important.
> > > > > >3. Use NodaTime  for UTC<->Local
> > > > conversions.
> > > > > >Noda time uses Java calendars making the conversion truely
> > > portable.
> > > > > >
> > 

Re: Interoperable Ignite.NET Dates

[Valentin Kulichenko]

Got it. So it only affects the .NET side, correct?

-Val

On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin 
wrote:

> Hi Val,
>
> Our proposal does not overlap with IEP-54
> <
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> >,
> which proposes changing Ignite date storage format. Our proposal is
> enhancing Ignite to handle .NET dates in a truly portable way instead of
> requiring the application developers to repeat this task every time:
>
>- Write .NET dates as Ignite dates (today .NET dates are written as
>generic Ignite objects)
>- Convert Local .NET dates to UTC inside Ignite and to it using Java
>calendars.
>
>
> вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> valentin.kuliche...@gmail.com>:
>
> > Hi Alexey,
> >
> > The IEP-54 [1] describes the data layout proposed for Ignite 3.0, it
> > includes various date/time types. Can you please take a look and check if
> > this addresses your concerns? We can then discuss further if needed.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> >
> > -Val
> >
> > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > kukushkinale...@gmail.com>
> > wrote:
> >
> > > Pavel,
> > >
> > > We propose changing public API so this is for Ignite 3.0.
> > >
> > > Thanks!
> > >
> > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn :
> > >
> > > > Alexey,
> > > >
> > > > Just to clarify before we start the discussion:
> > > > this proposal seems to introduce some breaking changes, so we are
> > talking
> > > > about Ignite 3.0, correct?
> > > >
> > > > Pavel
> > > >
> > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > kukushkinale...@gmail.com>
> > > > wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > What do you think about changing .NET API to read/write portable
> > dates
> > > by
> > > > > default and making that really portable?
> > > > >
> > > > > *The Problem*
> > > > > Presently .NET API writes dates as composite Ignite objects. Only
> > .NET
> > > > > clients can read such dates: any other client (JDBC, Java, etc)
> does
> > > not
> > > > > understand it without custom deserialization.
> > > > >
> > > > > It is still possible to configure .NET serialization to write dates
> > as
> > > > > Ignite dates - see DateTime Serialization note
> > > > > <
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > > >.
> > > > > But then Ignite accepts only UTC dates, requiring the application
> > > > > developers to convert local dates to UTC dates and back. This task
> is
> > > not
> > > > > trivial: DateTime.ToUniversalTime
> > > > > <
> > > > >
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > > > >
> > > > > uses
> > > > > calendars different from Java (and the .NET calendars are the
> invalid
> > > > ones
> > > > > - especially for pre-1990 dates).
> > > > >
> > > > > The motivation for the current default behavior was probably the
> > desire
> > > > to
> > > > > keep the Time Zone information: Ignite dates do not store time
> zones.
> > > > >
> > > > > In our experience interoperability is more important than storing
> > time
> > > > zone
> > > > > info.
> > > > >
> > > > > *The Proposal*
> > > > >
> > > > >1. Always write .NET dates as portable Ignite dates: get rid of
> > the
> > > > >BinaryReflectiveSerializer.ForceTimestamp flag that currently
> > > triggers
> > > > > this
> > > > >behavior.
> > > > >   - We could still keep the ForceTimestamp flag if saving .NET
> > > dates
> > > > as
> > > > >   transparent objects seems a useful case. We do not think it
> is
> > > > > useful.
> > > > >2. Automatically convert Local dates to UTC and back *inside*
> > > > >Ignite.NET.
> > > > >   - In this case we lose the DateTime.Kind of UTC dates: we
> > write a
> > > > UTC
> > > > >   date but we would read a Local date since Ignite would always
> > > > > convert UTC
> > > > >   to Local when reading.
> > > > >   We could add a UtcDate date flag to QuerySqlFieldAttribute
> > > > >   and BinaryReflectiveSerializer to control the deserialization
> > > > > behavior if
> > > > >   keeping dates in UTC format use case seems important.
> > > > >3. Use NodaTime  for UTC<->Local
> > > conversions.
> > > > >Noda time uses Java calendars making the conversion truely
> > portable.
> > > > >
> > > > > *The Benefits*
> > > > >
> > > > >1. We think portable dates are much more important than storing
> > time
> > > > >zone info. Why do we store time zones for every date on the
> server
> > > > > anyway?
> > > > >Time zone is client-side info.
> > > > >2. Simpler application code: no more manual configuration to
> > trigger
> > > > the
> > > > >portable behavior.
> > > > >3. Non-trivial code to make the truly portable 

Re: usage analytics

[Alexey Goncharuk]

Folks,

I want to bump up this discussion and slightly change the format suggested
by Nikita. I dot think it is correct to gather any information related to
the user environment. However, can we collect just the fact of some of the
Ignite APIs/subsystems being used with no user information whatsoever?
Having started thinking about Ignite 3.0 I realized that we lack even some
very basic knowledge on the impact of changing one or another feature or
API.

To my knowledge, the Ignite website already uses google analytics which is
available to the community. The google analytics platform already has
tooling to track app screen hits in a completely anonymous way, so we can
use this tool to track Ignite components usage (once per node startup)
sending solely component name and a unique environment hash - no IP
addresses, no jdk/os/other information. The information will be available
in the same toolkit we are already using to analyze the website and
optimize our docs.

WDYT?

ср, 19 июл. 2017 г. в 01:15, :

> I would try to ping legal again and see if they respond. If not, I think
> we will need to come up with a simpler approach, that does not require
> legal approval.
>
> ⁣D.​
>
> On Jul 18, 2017, 2:23 PM, at 2:23 PM, Nikita Ivanov 
> wrote:
> >Igniters,
> >Just a quick update. I haven't gotten response from ASF Legal on this
> >thread and I frankly don't know how to proceed here. What's the process
> >to
> >arrive to a decision point here?
> >
> >Thanks!
> >--
> >Nikita Ivanov
> >
> >
> >On Mon, Jul 10, 2017 at 3:11 PM, Konstantin Boudnik 
> >wrote:
> >
> >> On Sat, Jul 08, 2017 at 11:04AM, Nikita Ivanov wrote:
> >> > Cos,
> >> > Based on my experience having it off by default negates the entire
> >> > purpose... We need statistically meaningful data set to make any
> >> inferences
> >> > from it. Moreover, if we are going to ask folks to turn it on it
> >will
> >> > significantly skew the resulting data set anyways and show full
> >picture.
> >> I
> >> > think "on" by default is the better option if we are to collect
> >usage
> >> stats
> >> > to begin with.
> >>
> >> yes, sure. But having this "on" by default is likely to expose us to
> >> another
> >> shit-storm down the road. An interesting dilemma to have indeed. In
> >my
> >> experience, whenever I install something like a browser or an
> >operating
> >> system, it would ask if I want to make the particular piece of
> >software
> >> better
> >> by sending back some anonymized stats. Basically, I am given a way to
> >> explicitly opt-out if I wish.
> >>
> >> By turning the feature "on" by default is like saying: "we'll be
> >collecting
> >> some stats, but if you don't want to you can go here and there and
> >disable
> >> the
> >> collection. Oh, and by the way - you need to go and figure out the
> >exact
> >> steps
> >> to disable it."
> >>
> >> > Also, I want to re-iterate it again to avoid misunderstanding:
> >there is
> >> no
> >> > proposal nor will there be a technical way to attribute collected
> >data
> >> back
> >> > to a certain company. That's not what this is all about. We should
> >only
> >> be
> >> > interested in aggregated stats (community size, geo information,
> >language
> >> > information, components usage).
> >>
> >> Yes, I think it is clear, but never hurts to re-iterate.
> >>
> >> Cos
> >>
> >> > Thoughts?
> >> >
> >> > --
> >> > Nikita Ivanov
> >> > Founder & CTO
> >> > GridGain Systems
> >> >
> >> > On Fri, Jul 7, 2017 at 8:17 PM, Konstantin Boudnik 
> >> wrote:
> >> >
> >> > > Actually, that should be OFF by default. It sounds like this
> >reduce the
> >> > > amount
> >> > > of the data collected, but this would address the concerns of
> >companies
> >> > > like
> >> > > Roman's. I know for sure that a few of my clients would sue my
> >ass out
> >> of
> >> > > existence if I gave them the platform collecting their
> >data-centers
> >> info.
> >> > >
> >> > > Let's have it, set if off by default and document and easy way to
> >turn
> >> it
> >> > > off.
> >> > > Then start making rounds asking our user base to share _some_ of
> >the
> >> stats
> >> > > with the community, so we can track the growth of the install
> >base,
> >> etc.
> >> > >
> >> > > Cos
> >> > >
> >> > > On Thu, Jul 06, 2017 at 08:20AM, Nikita Ivanov wrote:
> >> > > > The idea so far is to have a single system property in
> >configuration
> >> that
> >> > > > turns this off completely. I envision that this will be
> >prominently
> >> > > > featured on Ignite website so that everyone who would like to
> >> disable it
> >> > > -
> >> > > > can do it in seconds.
> >> > > >
> >> > > > Thoughts?
> >> > > >
> >> > > > --
> >> > > > Nikita Ivanov
> >> > > > Founder & CTO
> >> > > > GridGain Systems
> >> > > >
> >> > > > On Wed, Jul 5, 2017 at 9:27 PM, Roman Shtykh
> >
> >> wrote:
> >> > > >
> >> > > > > Nikita,
> >> > > > >
> >> > > > > Sending and storing (somewhere the company cannot securely
> >handle)
> >> any
> >> > > > > information (OS version, IP addresses, 

Re: Interoperable Ignite.NET Dates

[Alexey Kukushkin]

Hi Val,

Our proposal does not overlap with IEP-54
,
which proposes changing Ignite date storage format. Our proposal is
enhancing Ignite to handle .NET dates in a truly portable way instead of
requiring the application developers to repeat this task every time:

   - Write .NET dates as Ignite dates (today .NET dates are written as
   generic Ignite objects)
   - Convert Local .NET dates to UTC inside Ignite and to it using Java
   calendars.


вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Hi Alexey,
>
> The IEP-54 [1] describes the data layout proposed for Ignite 3.0, it
> includes various date/time types. Can you please take a look and check if
> this addresses your concerns? We can then discuss further if needed.
>
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
>
> -Val
>
> On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> kukushkinale...@gmail.com>
> wrote:
>
> > Pavel,
> >
> > We propose changing public API so this is for Ignite 3.0.
> >
> > Thanks!
> >
> > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn :
> >
> > > Alexey,
> > >
> > > Just to clarify before we start the discussion:
> > > this proposal seems to introduce some breaking changes, so we are
> talking
> > > about Ignite 3.0, correct?
> > >
> > > Pavel
> > >
> > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > kukushkinale...@gmail.com>
> > > wrote:
> > >
> > > > Igniters,
> > > >
> > > > What do you think about changing .NET API to read/write portable
> dates
> > by
> > > > default and making that really portable?
> > > >
> > > > *The Problem*
> > > > Presently .NET API writes dates as composite Ignite objects. Only
> .NET
> > > > clients can read such dates: any other client (JDBC, Java, etc) does
> > not
> > > > understand it without custom deserialization.
> > > >
> > > > It is still possible to configure .NET serialization to write dates
> as
> > > > Ignite dates - see DateTime Serialization note
> > > > <
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > >.
> > > > But then Ignite accepts only UTC dates, requiring the application
> > > > developers to convert local dates to UTC dates and back. This task is
> > not
> > > > trivial: DateTime.ToUniversalTime
> > > > <
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > > >
> > > > uses
> > > > calendars different from Java (and the .NET calendars are the invalid
> > > ones
> > > > - especially for pre-1990 dates).
> > > >
> > > > The motivation for the current default behavior was probably the
> desire
> > > to
> > > > keep the Time Zone information: Ignite dates do not store time zones.
> > > >
> > > > In our experience interoperability is more important than storing
> time
> > > zone
> > > > info.
> > > >
> > > > *The Proposal*
> > > >
> > > >1. Always write .NET dates as portable Ignite dates: get rid of
> the
> > > >BinaryReflectiveSerializer.ForceTimestamp flag that currently
> > triggers
> > > > this
> > > >behavior.
> > > >   - We could still keep the ForceTimestamp flag if saving .NET
> > dates
> > > as
> > > >   transparent objects seems a useful case. We do not think it is
> > > > useful.
> > > >2. Automatically convert Local dates to UTC and back *inside*
> > > >Ignite.NET.
> > > >   - In this case we lose the DateTime.Kind of UTC dates: we
> write a
> > > UTC
> > > >   date but we would read a Local date since Ignite would always
> > > > convert UTC
> > > >   to Local when reading.
> > > >   We could add a UtcDate date flag to QuerySqlFieldAttribute
> > > >   and BinaryReflectiveSerializer to control the deserialization
> > > > behavior if
> > > >   keeping dates in UTC format use case seems important.
> > > >3. Use NodaTime  for UTC<->Local
> > conversions.
> > > >Noda time uses Java calendars making the conversion truely
> portable.
> > > >
> > > > *The Benefits*
> > > >
> > > >1. We think portable dates are much more important than storing
> time
> > > >zone info. Why do we store time zones for every date on the server
> > > > anyway?
> > > >Time zone is client-side info.
> > > >2. Simpler application code: no more manual configuration to
> trigger
> > > the
> > > >portable behavior.
> > > >3. Non-trivial code to make the truly portable UTC<->Local
> > conversion
> > > is
> > > >implemented once inside Ignite instead of having every Ignite.NET
> > > >application implementing it.
> > > >
> > > > Thank you!
> > > >
> > > > --
> > > > Best regards,
> > > > Alexey
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Alexey
> >
>


-- 
Best regards,
Alexey

[jira] [Created] (IGNITE-13660) Unexpected NODE_LEFT on graceful stop (at ducktests)

[Anton Vinogradov (Jira)]

Anton Vinogradov created IGNITE-13660:
-

 Summary: Unexpected NODE_LEFT on graceful stop (at ducktests)
 Key: IGNITE-13660
 URL: https://issues.apache.org/jira/browse/IGNITE-13660
 Project: Ignite
  Issue Type: Bug
Reporter: Anton Vinogradov
Assignee: Anton Vinogradov


1) Graceful shutdown may cause NODE_LEFT.

2) Self-termination apps have logs till the end.
Apps terminated by ducktape with SIGTERM stop their logging at node stop.

Seems, shutdown hook stops kill the application before the Ignite stops.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Disable socket linger by default in TCP discovery SPI.

[Steshin Vladimir]

    Ilya, hi.


    Of course: /TcpDiscoverySpi.setSoLinger(int)/ property. Always been.


02.11.2020 20:14, Ilya Kasnacheev пишет:

Hello!

Is there any option to re-enable linger on SSL sockets?

Telling people to re-configure does not help if they can't.

Regards,

Re: [DISCUSS] Ignite 3.0 development approach

[Kseniya Romanova]

Pavel, all the interesting points will be anyway published here in English
(as the principal "if it's not on devlist it doesn't happened" is still
relevant). This is just a quick call for a group of developers. Later we
can do a separate presentation of idea and discussion in English as we did
for the Ignite 3.0 draft of changes.

вт, 3 нояб. 2020 г. в 13:52, Pavel Tupitsyn :

> Kseniya,
>
> Thanks for scheduling this call.
> Do you think we can switch to English if non-Russian speaking community
> members decide to join?
>
> On Tue, Nov 3, 2020 at 1:32 PM Kseniya Romanova  >
> wrote:
>
> > Let's do this community discussion open. Here's the link on zoom call in
> > Russian for Friday 6 PM:
> > https://www.meetup.com/Moscow-Apache-Ignite-Meetup/events/274360378/
> >
> > вт, 3 нояб. 2020 г. в 12:49, Nikolay Izhikov :
> >
> > > Time works for me.
> > >
> > > > 3 нояб. 2020 г., в 12:40, Alexey Goncharuk <
> alexey.goncha...@gmail.com
> > >
> > > написал(а):
> > > >
> > > > Nikolay,
> > > >
> > > > I am up for the call. I will try to explain my reasoning in greater
> > > detail
> > > > and will be glad to hear the concerns. Will this Friday, Nov 6th,
> work?
> > > >
> > > > вт, 3 нояб. 2020 г. в 10:09, Nikolay Izhikov :
> > > >
> > > >> Igniters, should we have a call for this topic?
> > > >>
> > > >>> 2 нояб. 2020 г., в 18:53, Pavel Tupitsyn 
> > > >> написал(а):
> > > >>>
> > >  not intend to rewrite everything from scratch
> > > >>>
> > >  Every single test from Ignite 2.x should be moved to Ignite 3
> > >  regardless of how we choose to proceed.
> > > >>>
> > > >>> Alexey, thank you for the explanation, this addresses all of my
> > > concerns.
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Mon, Nov 2, 2020 at 6:43 PM Andrey Mashenkov <
> > > >> andrey.mashen...@gmail.com>
> > > >>> wrote:
> > > >>>
> > >  Hi, Igniters.
> > > 
> > >  * AFAIU, we need a new repo if we want to apply different
> > restrictions
> > > >> to
> > >  pull requests,
> > >  otherwise I see no difference for myself.
> > >  E.g. make static analysis (do we have?), compile, styles, and
> > javadoc
> > >  checks mandatory.
> > > 
> > >  I think that relaxed requirements here will lead to bad product
> > > quality.
> > > 
> > >  * Agree with Pavel, we should 'keep' integrations tests somehow.
> > >  During active development tests will be broken most of time, so,
> > >  I'd port them e.g. suite-by-suite once we will have a stable and
> > > >> featured
> > >  environment to run them and of course make test's code clear and
> > avoid
> > >  bad/non-relevant ones.
> > > 
> > >  * I like bottom-up approach.
> > >  With it we could make a better framework. I mean clear component
> > > >> lifecycle,
> > >  component wiring mechanics, general methods to approach core
> > > components
> > >  such as exchange/communication
> > >  to avoid code mess like we have in ExchangeFuture with all these
> > > custom
> > >  callbacks for each component, interfaces like
> > >  PartitionsExchangeAware, IgniteChangeGlobalStateSupport and
> > >  a pack of
> > > >> start/stop/onKernalStart/onPluginStart/onActivate/onDisconnected
> > >  and so on in various unexpected places.
> > >  Hope, we will be able to port most of the good code to the new
> > > framework
> > >  version.
> > > 
> > > 
> > > 
> > >  On Mon, Nov 2, 2020 at 6:18 PM Alexey Goncharuk <
> > >  alexey.goncha...@gmail.com>
> > >  wrote:
> > > 
> > > > Nikolay, Pavel,
> > > >
> > > > Thanks for the feedback! First of all, I wanted to stress that I
> do
> > > not
> > > > intend to rewrite everything from scratch (I never used this
> > phrase).
> > >  There
> > > > are significant parts of code that would be moved with minimal
> > > > modifications. Second, I never said that we will get rid of the
> old
> > > >> tests
> > > > codebase. Every single test from Ignite 2.x should be moved to
> > > Ignite 3
> > > > regardless of how we choose to proceed.
> > > >
> > > > My point is that for some parts of the code a clean bottom-up
> > > > implementation will be cheaper in many ways. Let me give you a
> few
> > >  concrete
> > > > examples:
> > > >
> > > >  - I think no one can object that we need a cleanly separated
> > >  persistence
> > > >  layer for Ignite. There is a very raw draft IEP for this
> already.
> > On
> > >  the
> > > >  other hand, I think we also can agree that we need a split-brain
> > > > resistant
> > > >  replication protocol for caches. There is also an IEP for this.
> > >  Neither
> > > > of
> > > >  the changes is a good fit for 2.x because they are likely to
> > > >> introduce
> > > >  breaking changes in the persistence layer, configuration and
> > > >> behavior.
> > > >  Additionally, these components are now 

Re: [DISCUSS] Ignite 3.0 development approach

[Pavel Tupitsyn]

Kseniya,

Thanks for scheduling this call.
Do you think we can switch to English if non-Russian speaking community
members decide to join?

On Tue, Nov 3, 2020 at 1:32 PM Kseniya Romanova 
wrote:

> Let's do this community discussion open. Here's the link on zoom call in
> Russian for Friday 6 PM:
> https://www.meetup.com/Moscow-Apache-Ignite-Meetup/events/274360378/
>
> вт, 3 нояб. 2020 г. в 12:49, Nikolay Izhikov :
>
> > Time works for me.
> >
> > > 3 нояб. 2020 г., в 12:40, Alexey Goncharuk  >
> > написал(а):
> > >
> > > Nikolay,
> > >
> > > I am up for the call. I will try to explain my reasoning in greater
> > detail
> > > and will be glad to hear the concerns. Will this Friday, Nov 6th, work?
> > >
> > > вт, 3 нояб. 2020 г. в 10:09, Nikolay Izhikov :
> > >
> > >> Igniters, should we have a call for this topic?
> > >>
> > >>> 2 нояб. 2020 г., в 18:53, Pavel Tupitsyn 
> > >> написал(а):
> > >>>
> >  not intend to rewrite everything from scratch
> > >>>
> >  Every single test from Ignite 2.x should be moved to Ignite 3
> >  regardless of how we choose to proceed.
> > >>>
> > >>> Alexey, thank you for the explanation, this addresses all of my
> > concerns.
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> On Mon, Nov 2, 2020 at 6:43 PM Andrey Mashenkov <
> > >> andrey.mashen...@gmail.com>
> > >>> wrote:
> > >>>
> >  Hi, Igniters.
> > 
> >  * AFAIU, we need a new repo if we want to apply different
> restrictions
> > >> to
> >  pull requests,
> >  otherwise I see no difference for myself.
> >  E.g. make static analysis (do we have?), compile, styles, and
> javadoc
> >  checks mandatory.
> > 
> >  I think that relaxed requirements here will lead to bad product
> > quality.
> > 
> >  * Agree with Pavel, we should 'keep' integrations tests somehow.
> >  During active development tests will be broken most of time, so,
> >  I'd port them e.g. suite-by-suite once we will have a stable and
> > >> featured
> >  environment to run them and of course make test's code clear and
> avoid
> >  bad/non-relevant ones.
> > 
> >  * I like bottom-up approach.
> >  With it we could make a better framework. I mean clear component
> > >> lifecycle,
> >  component wiring mechanics, general methods to approach core
> > components
> >  such as exchange/communication
> >  to avoid code mess like we have in ExchangeFuture with all these
> > custom
> >  callbacks for each component, interfaces like
> >  PartitionsExchangeAware, IgniteChangeGlobalStateSupport and
> >  a pack of
> > >> start/stop/onKernalStart/onPluginStart/onActivate/onDisconnected
> >  and so on in various unexpected places.
> >  Hope, we will be able to port most of the good code to the new
> > framework
> >  version.
> > 
> > 
> > 
> >  On Mon, Nov 2, 2020 at 6:18 PM Alexey Goncharuk <
> >  alexey.goncha...@gmail.com>
> >  wrote:
> > 
> > > Nikolay, Pavel,
> > >
> > > Thanks for the feedback! First of all, I wanted to stress that I do
> > not
> > > intend to rewrite everything from scratch (I never used this
> phrase).
> >  There
> > > are significant parts of code that would be moved with minimal
> > > modifications. Second, I never said that we will get rid of the old
> > >> tests
> > > codebase. Every single test from Ignite 2.x should be moved to
> > Ignite 3
> > > regardless of how we choose to proceed.
> > >
> > > My point is that for some parts of the code a clean bottom-up
> > > implementation will be cheaper in many ways. Let me give you a few
> >  concrete
> > > examples:
> > >
> > >  - I think no one can object that we need a cleanly separated
> >  persistence
> > >  layer for Ignite. There is a very raw draft IEP for this already.
> On
> >  the
> > >  other hand, I think we also can agree that we need a split-brain
> > > resistant
> > >  replication protocol for caches. There is also an IEP for this.
> >  Neither
> > > of
> > >  the changes is a good fit for 2.x because they are likely to
> > >> introduce
> > >  breaking changes in the persistence layer, configuration and
> > >> behavior.
> > >  Additionally, these components are now tightly coupled, so there
> is
> > >> no
> > > way
> > >  these two changes can be implemented in parallel and then merged
> > > together
> > >  easily. So what we will end up with is having to implement these
> >  changes
> > >  sequentially, fixing all existing tests twice, and essentially
> >  throwing
> > >  away half of the work done because the other part of the change is
> > >  re-implemented
> > >  - Similar example goes with getting rid of IgniteInternalFuture
> and
> > >  replacing it with CompletableFuture, and any other change that
> > >> touches
> > > the
> > >  asynchronous part of the code.
> > >
> > 

Re: [DISCUSS] Ignite 3.0 development approach

[Kseniya Romanova]

Let's do this community discussion open. Here's the link on zoom call in
Russian for Friday 6 PM:
https://www.meetup.com/Moscow-Apache-Ignite-Meetup/events/274360378/

вт, 3 нояб. 2020 г. в 12:49, Nikolay Izhikov :

> Time works for me.
>
> > 3 нояб. 2020 г., в 12:40, Alexey Goncharuk 
> написал(а):
> >
> > Nikolay,
> >
> > I am up for the call. I will try to explain my reasoning in greater
> detail
> > and will be glad to hear the concerns. Will this Friday, Nov 6th, work?
> >
> > вт, 3 нояб. 2020 г. в 10:09, Nikolay Izhikov :
> >
> >> Igniters, should we have a call for this topic?
> >>
> >>> 2 нояб. 2020 г., в 18:53, Pavel Tupitsyn 
> >> написал(а):
> >>>
>  not intend to rewrite everything from scratch
> >>>
>  Every single test from Ignite 2.x should be moved to Ignite 3
>  regardless of how we choose to proceed.
> >>>
> >>> Alexey, thank you for the explanation, this addresses all of my
> concerns.
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Mon, Nov 2, 2020 at 6:43 PM Andrey Mashenkov <
> >> andrey.mashen...@gmail.com>
> >>> wrote:
> >>>
>  Hi, Igniters.
> 
>  * AFAIU, we need a new repo if we want to apply different restrictions
> >> to
>  pull requests,
>  otherwise I see no difference for myself.
>  E.g. make static analysis (do we have?), compile, styles, and javadoc
>  checks mandatory.
> 
>  I think that relaxed requirements here will lead to bad product
> quality.
> 
>  * Agree with Pavel, we should 'keep' integrations tests somehow.
>  During active development tests will be broken most of time, so,
>  I'd port them e.g. suite-by-suite once we will have a stable and
> >> featured
>  environment to run them and of course make test's code clear and avoid
>  bad/non-relevant ones.
> 
>  * I like bottom-up approach.
>  With it we could make a better framework. I mean clear component
> >> lifecycle,
>  component wiring mechanics, general methods to approach core
> components
>  such as exchange/communication
>  to avoid code mess like we have in ExchangeFuture with all these
> custom
>  callbacks for each component, interfaces like
>  PartitionsExchangeAware, IgniteChangeGlobalStateSupport and
>  a pack of
> >> start/stop/onKernalStart/onPluginStart/onActivate/onDisconnected
>  and so on in various unexpected places.
>  Hope, we will be able to port most of the good code to the new
> framework
>  version.
> 
> 
> 
>  On Mon, Nov 2, 2020 at 6:18 PM Alexey Goncharuk <
>  alexey.goncha...@gmail.com>
>  wrote:
> 
> > Nikolay, Pavel,
> >
> > Thanks for the feedback! First of all, I wanted to stress that I do
> not
> > intend to rewrite everything from scratch (I never used this phrase).
>  There
> > are significant parts of code that would be moved with minimal
> > modifications. Second, I never said that we will get rid of the old
> >> tests
> > codebase. Every single test from Ignite 2.x should be moved to
> Ignite 3
> > regardless of how we choose to proceed.
> >
> > My point is that for some parts of the code a clean bottom-up
> > implementation will be cheaper in many ways. Let me give you a few
>  concrete
> > examples:
> >
> >  - I think no one can object that we need a cleanly separated
>  persistence
> >  layer for Ignite. There is a very raw draft IEP for this already. On
>  the
> >  other hand, I think we also can agree that we need a split-brain
> > resistant
> >  replication protocol for caches. There is also an IEP for this.
>  Neither
> > of
> >  the changes is a good fit for 2.x because they are likely to
> >> introduce
> >  breaking changes in the persistence layer, configuration and
> >> behavior.
> >  Additionally, these components are now tightly coupled, so there is
> >> no
> > way
> >  these two changes can be implemented in parallel and then merged
> > together
> >  easily. So what we will end up with is having to implement these
>  changes
> >  sequentially, fixing all existing tests twice, and essentially
>  throwing
> >  away half of the work done because the other part of the change is
> >  re-implemented
> >  - Similar example goes with getting rid of IgniteInternalFuture and
> >  replacing it with CompletableFuture, and any other change that
> >> touches
> > the
> >  asynchronous part of the code.
> >
> > Third, I do not see how this choice affects the UX of Ignite. The end
>  user
> > experience must be fixed in the IEP regardless of the development
> >> process
> > and the fact that we have gaps in this area in Ignite 2.x just
> confirms
> > that.
> >
> > Pavel, agree that a repo/branch is a technicality, I guess if
>  reformulate,
> > my point is that we might agree to have a single development master
>  branch
> > with 

Re: [DISCUSS] Ignite 3.0 development approach

[Nikolay Izhikov]

Time works for me.

> 3 нояб. 2020 г., в 12:40, Alexey Goncharuk  
> написал(а):
> 
> Nikolay,
> 
> I am up for the call. I will try to explain my reasoning in greater detail
> and will be glad to hear the concerns. Will this Friday, Nov 6th, work?
> 
> вт, 3 нояб. 2020 г. в 10:09, Nikolay Izhikov :
> 
>> Igniters, should we have a call for this topic?
>> 
>>> 2 нояб. 2020 г., в 18:53, Pavel Tupitsyn 
>> написал(а):
>>> 
 not intend to rewrite everything from scratch
>>> 
 Every single test from Ignite 2.x should be moved to Ignite 3
 regardless of how we choose to proceed.
>>> 
>>> Alexey, thank you for the explanation, this addresses all of my concerns.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On Mon, Nov 2, 2020 at 6:43 PM Andrey Mashenkov <
>> andrey.mashen...@gmail.com>
>>> wrote:
>>> 
 Hi, Igniters.
 
 * AFAIU, we need a new repo if we want to apply different restrictions
>> to
 pull requests,
 otherwise I see no difference for myself.
 E.g. make static analysis (do we have?), compile, styles, and javadoc
 checks mandatory.
 
 I think that relaxed requirements here will lead to bad product quality.
 
 * Agree with Pavel, we should 'keep' integrations tests somehow.
 During active development tests will be broken most of time, so,
 I'd port them e.g. suite-by-suite once we will have a stable and
>> featured
 environment to run them and of course make test's code clear and avoid
 bad/non-relevant ones.
 
 * I like bottom-up approach.
 With it we could make a better framework. I mean clear component
>> lifecycle,
 component wiring mechanics, general methods to approach core components
 such as exchange/communication
 to avoid code mess like we have in ExchangeFuture with all these custom
 callbacks for each component, interfaces like
 PartitionsExchangeAware, IgniteChangeGlobalStateSupport and
 a pack of
>> start/stop/onKernalStart/onPluginStart/onActivate/onDisconnected
 and so on in various unexpected places.
 Hope, we will be able to port most of the good code to the new framework
 version.
 
 
 
 On Mon, Nov 2, 2020 at 6:18 PM Alexey Goncharuk <
 alexey.goncha...@gmail.com>
 wrote:
 
> Nikolay, Pavel,
> 
> Thanks for the feedback! First of all, I wanted to stress that I do not
> intend to rewrite everything from scratch (I never used this phrase).
 There
> are significant parts of code that would be moved with minimal
> modifications. Second, I never said that we will get rid of the old
>> tests
> codebase. Every single test from Ignite 2.x should be moved to Ignite 3
> regardless of how we choose to proceed.
> 
> My point is that for some parts of the code a clean bottom-up
> implementation will be cheaper in many ways. Let me give you a few
 concrete
> examples:
> 
>  - I think no one can object that we need a cleanly separated
 persistence
>  layer for Ignite. There is a very raw draft IEP for this already. On
 the
>  other hand, I think we also can agree that we need a split-brain
> resistant
>  replication protocol for caches. There is also an IEP for this.
 Neither
> of
>  the changes is a good fit for 2.x because they are likely to
>> introduce
>  breaking changes in the persistence layer, configuration and
>> behavior.
>  Additionally, these components are now tightly coupled, so there is
>> no
> way
>  these two changes can be implemented in parallel and then merged
> together
>  easily. So what we will end up with is having to implement these
 changes
>  sequentially, fixing all existing tests twice, and essentially
 throwing
>  away half of the work done because the other part of the change is
>  re-implemented
>  - Similar example goes with getting rid of IgniteInternalFuture and
>  replacing it with CompletableFuture, and any other change that
>> touches
> the
>  asynchronous part of the code.
> 
> Third, I do not see how this choice affects the UX of Ignite. The end
 user
> experience must be fixed in the IEP regardless of the development
>> process
> and the fact that we have gaps in this area in Ignite 2.x just confirms
> that.
> 
> Pavel, agree that a repo/branch is a technicality, I guess if
 reformulate,
> my point is that we might agree to have a single development master
 branch
> with 'disassembled' end-to-end functionality for some period of time to
> speed up development, and re-assemble the core features after having
> submodules tested independently.
> 
> Nikolay,
>> We have many features that have to evolve.
>> Snapshots, rebalance, tooling, tracing, zookeeper support, etc.
> This is not very specific. In the end, resources are limited and we
>> will
> not be able to drive both tracks simultaneously, 

Re: [DISCUSS] Ignite 3.0 development approach

[Alexey Goncharuk]

Nikolay,

I am up for the call. I will try to explain my reasoning in greater detail
and will be glad to hear the concerns. Will this Friday, Nov 6th, work?

вт, 3 нояб. 2020 г. в 10:09, Nikolay Izhikov :

> Igniters, should we have a call for this topic?
>
> > 2 нояб. 2020 г., в 18:53, Pavel Tupitsyn 
> написал(а):
> >
> >> not intend to rewrite everything from scratch
> >
> >> Every single test from Ignite 2.x should be moved to Ignite 3
> >> regardless of how we choose to proceed.
> >
> > Alexey, thank you for the explanation, this addresses all of my concerns.
> >
> >
> >
> >
> >
> > On Mon, Nov 2, 2020 at 6:43 PM Andrey Mashenkov <
> andrey.mashen...@gmail.com>
> > wrote:
> >
> >> Hi, Igniters.
> >>
> >> * AFAIU, we need a new repo if we want to apply different restrictions
> to
> >> pull requests,
> >> otherwise I see no difference for myself.
> >> E.g. make static analysis (do we have?), compile, styles, and javadoc
> >> checks mandatory.
> >>
> >> I think that relaxed requirements here will lead to bad product quality.
> >>
> >> * Agree with Pavel, we should 'keep' integrations tests somehow.
> >> During active development tests will be broken most of time, so,
> >> I'd port them e.g. suite-by-suite once we will have a stable and
> featured
> >> environment to run them and of course make test's code clear and avoid
> >> bad/non-relevant ones.
> >>
> >> * I like bottom-up approach.
> >> With it we could make a better framework. I mean clear component
> lifecycle,
> >> component wiring mechanics, general methods to approach core components
> >> such as exchange/communication
> >> to avoid code mess like we have in ExchangeFuture with all these custom
> >> callbacks for each component, interfaces like
> >> PartitionsExchangeAware, IgniteChangeGlobalStateSupport and
> >> a pack of
> start/stop/onKernalStart/onPluginStart/onActivate/onDisconnected
> >> and so on in various unexpected places.
> >> Hope, we will be able to port most of the good code to the new framework
> >> version.
> >>
> >>
> >>
> >> On Mon, Nov 2, 2020 at 6:18 PM Alexey Goncharuk <
> >> alexey.goncha...@gmail.com>
> >> wrote:
> >>
> >>> Nikolay, Pavel,
> >>>
> >>> Thanks for the feedback! First of all, I wanted to stress that I do not
> >>> intend to rewrite everything from scratch (I never used this phrase).
> >> There
> >>> are significant parts of code that would be moved with minimal
> >>> modifications. Second, I never said that we will get rid of the old
> tests
> >>> codebase. Every single test from Ignite 2.x should be moved to Ignite 3
> >>> regardless of how we choose to proceed.
> >>>
> >>> My point is that for some parts of the code a clean bottom-up
> >>> implementation will be cheaper in many ways. Let me give you a few
> >> concrete
> >>> examples:
> >>>
> >>>   - I think no one can object that we need a cleanly separated
> >> persistence
> >>>   layer for Ignite. There is a very raw draft IEP for this already. On
> >> the
> >>>   other hand, I think we also can agree that we need a split-brain
> >>> resistant
> >>>   replication protocol for caches. There is also an IEP for this.
> >> Neither
> >>> of
> >>>   the changes is a good fit for 2.x because they are likely to
> introduce
> >>>   breaking changes in the persistence layer, configuration and
> behavior.
> >>>   Additionally, these components are now tightly coupled, so there is
> no
> >>> way
> >>>   these two changes can be implemented in parallel and then merged
> >>> together
> >>>   easily. So what we will end up with is having to implement these
> >> changes
> >>>   sequentially, fixing all existing tests twice, and essentially
> >> throwing
> >>>   away half of the work done because the other part of the change is
> >>>   re-implemented
> >>>   - Similar example goes with getting rid of IgniteInternalFuture and
> >>>   replacing it with CompletableFuture, and any other change that
> touches
> >>> the
> >>>   asynchronous part of the code.
> >>>
> >>> Third, I do not see how this choice affects the UX of Ignite. The end
> >> user
> >>> experience must be fixed in the IEP regardless of the development
> process
> >>> and the fact that we have gaps in this area in Ignite 2.x just confirms
> >>> that.
> >>>
> >>> Pavel, agree that a repo/branch is a technicality, I guess if
> >> reformulate,
> >>> my point is that we might agree to have a single development master
> >> branch
> >>> with 'disassembled' end-to-end functionality for some period of time to
> >>> speed up development, and re-assemble the core features after having
> >>> submodules tested independently.
> >>>
> >>> Nikolay,
>  We have many features that have to evolve.
>  Snapshots, rebalance, tooling, tracing, zookeeper support, etc.
> >>> This is not very specific. In the end, resources are limited and we
> will
> >>> not be able to drive both tracks simultaneously, especially after a
> >> couple
> >>> of features having been implemented for Ignite 3.0. If there are indeed
> >>> some 

[jira] [Created] (IGNITE-13659) TDE - Phase 3. Documentation

[Pavel Pereslegin (Jira)]

Pavel Pereslegin created IGNITE-13659:
-

 Summary: TDE - Phase 3. Documentation
 Key: IGNITE-13659
 URL: https://issues.apache.org/jira/browse/IGNITE-13659
 Project: Ignite
  Issue Type: Task
  Components: documentation
Reporter: Pavel Pereslegin
Assignee: Pavel Pereslegin
 Fix For: 2.10


Write documentation on the cache encryption key rotation function.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13658) Volatile cache writes to persistent storage.

[Stanilovsky Evgeny (Jira)]

Stanilovsky Evgeny created IGNITE-13658:
---

 Summary: Volatile cache writes to persistent storage.
 Key: IGNITE-13658
 URL: https://issues.apache.org/jira/browse/IGNITE-13658
 Project: Ignite
  Issue Type: Bug
  Components: data structures
Affects Versions: 2.9
Reporter: Stanilovsky Evgeny
Assignee: Stanilovsky Evgeny


Various cluster synchronization primitives (latch, lock, semaphore e.t.c) are 
using volatile cache (`default-volatile-ds-group`). State of this primitives 
does not make sense on storage (restore, recovery), primitive's state is useful 
only for process in action.
But it was stored in system data region.

{noformat}
if (cacheType != CacheType.USER && cfg.getDataRegionName() == null)
 
cfg.setDataRegionName(cacheProcessor.context().database().systemDateRegionName());
{noformat}
Which persisted if cluster have any persistent region:

{noformat}
addDataRegion(
memCfg,
createSystemDataRegion(
memCfg.getSystemRegionInitialSize(),
memCfg.getSystemRegionMaxSize(),
CU.isPersistenceEnabled(memCfg)
),
CU.isPersistenceEnabled(memCfg)
);
{noformat}
Should to have dedicate system region for volatile cache, and it should be 
in-memory.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Custom Affinity Functions proposed for removal?

[Raymond Wilson]

In terms of Key -> Partition -> Node mapping, we provide customer affinity
mappers for the Key -> Partition stage an allow Ignite to map partitions to
nodes.

Our keys are structs with multiple fields forming a composite primary key,
parts of which are spatially identifying and parts contain other
characteristics. We want to ensure all related spatial elements resolve to
the same partition and so will be placed on the same node for query
colocation purposes.



On Tue, Nov 3, 2020 at 9:20 PM Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Moti, Raymond,
>
> Could you please describe your use cases in more detail? What are the types
> used as cache keys? What is the custom logic that you use for affinity
> mapping? What was the exact reason to customize versus using built-in
> collocation mechanisms?
>
> Ultimately, I'm sure that custom affinity functions will go away cause they
> introduce multiple potential issues. However, I would definitely like to
> make sure that your use cases are still supported in Ignite 3.0 via some
> other means.
>
> -Val
>
> On Mon, Nov 2, 2020 at 12:21 PM Alexei Scherbakov <
> alexey.scherbak...@gmail.com> wrote:
>
> > Thanks for the clarification.
> >
> > There was no intention to remove the customizable key to partition
> mapping.
> >
> > Difficulties arise when mapping partitions to nodes, so it's desirable to
> > have internally tested implementation with a way to customize it's
> behavior
> > without additional coding on the user side.
> >
> > пн, 2 нояб. 2020 г. в 23:01, Raymond Wilson  >:
> >
> > > Just to be clear, the affinity functions we are using convert keys to
> > > partitions, we do not map partitions to nodes and leave that to Ignite.
> > >
> > > On Tue, Nov 3, 2020 at 8:48 AM Alexei Scherbakov <
> > > alexey.scherbak...@gmail.com> wrote:
> > >
> > > > Hello.
> > > >
> > > > Custom affinity functions can cause weird bugs and data loss if
> > > implemented
> > > > wrongly.
> > > > There is an intention  to keep a backup filter based on user
> attributes
> > > > (with additional validation logic to ensure correctness) for
> > controllable
> > > > data placement.
> > > >
> > > > Can you describe more precisely why you had to implement custom
> > affinity
> > > > functions and not resort to default rendezvous affinity + backup
> > filter ?
> > > >
> > > >
> > > > пн, 2 нояб. 2020 г. в 21:45, Raymond Wilson <
> > raymond_wil...@trimble.com
> > > >:
> > > >
> > > > > We also use custom affinity functions (vis the C# client).
> > > > >
> > > > > The wish list mentions use of a particular annotation
> > > > > (@CentralizedAffinityFunction):
> > > > > Is the wish to remove just this annotation, or the ability to
> define
> > > > custom
> > > > > affinity functions at all?
> > > > >
> > > > > In our case we use affinity functions to ensure particular
> > distribution
> > > > of
> > > > > spatial data across a processing cluster to ensure even load
> > > management.
> > > > >
> > > > > On Tue, Nov 3, 2020 at 5:31 AM Moti Nisenson <
> > moti.nisen...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > I saw at
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0+Wishlist
> > > > > > that custom affinity functions are on the potential wishlist for
> > > > removal.
> > > > > > The way we're using it's very critical that we be able to control
> > the
> > > > > > placement of data quite precisely - as part of that we specify
> > > > explicitly
> > > > > > the partition we want in the key, and then our affinity function
> > uses
> > > > > that
> > > > > > (else delegating to default rendezvous). We don't need all the
> > > > > > abilities there, although I think that often others do.
> > > > > >
> > > > > > This seems to me to be a case that the benefit of removing this
> is
> > > > > minimal
> > > > > > and could cause quite a lot of disruption to users.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > 
> > > > > Raymond Wilson
> > > > > Solution Architect, Civil Construction Software Systems (CCSS)
> > > > > 11 Birmingham Drive | Christchurch, New Zealand
> > > > > +64-21-2013317 Mobile
> > > > > raymond_wil...@trimble.com
> > > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Best regards,
> > > > Alexei Scherbakov
> > > >
> > >
> > >
> > > --
> > > 
> > > Raymond Wilson
> > > Solution Architect, Civil Construction Software Systems (CCSS)
> > > 11 Birmingham Drive | Christchurch, New Zealand
> > > +64-21-2013317 Mobile
> > > raymond_wil...@trimble.com
> > >
> > > <
> > >
> >
> https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch
> > > >
> > >
> >
> >
> > --
> >
> > Best regards,
> > Alexei Scherbakov
> >
>


-- 

Re: Custom Affinity Functions proposed for removal?

[Valentin Kulichenko]

Moti, Raymond,

Could you please describe your use cases in more detail? What are the types
used as cache keys? What is the custom logic that you use for affinity
mapping? What was the exact reason to customize versus using built-in
collocation mechanisms?

Ultimately, I'm sure that custom affinity functions will go away cause they
introduce multiple potential issues. However, I would definitely like to
make sure that your use cases are still supported in Ignite 3.0 via some
other means.

-Val

On Mon, Nov 2, 2020 at 12:21 PM Alexei Scherbakov <
alexey.scherbak...@gmail.com> wrote:

> Thanks for the clarification.
>
> There was no intention to remove the customizable key to partition mapping.
>
> Difficulties arise when mapping partitions to nodes, so it's desirable to
> have internally tested implementation with a way to customize it's behavior
> without additional coding on the user side.
>
> пн, 2 нояб. 2020 г. в 23:01, Raymond Wilson :
>
> > Just to be clear, the affinity functions we are using convert keys to
> > partitions, we do not map partitions to nodes and leave that to Ignite.
> >
> > On Tue, Nov 3, 2020 at 8:48 AM Alexei Scherbakov <
> > alexey.scherbak...@gmail.com> wrote:
> >
> > > Hello.
> > >
> > > Custom affinity functions can cause weird bugs and data loss if
> > implemented
> > > wrongly.
> > > There is an intention  to keep a backup filter based on user attributes
> > > (with additional validation logic to ensure correctness) for
> controllable
> > > data placement.
> > >
> > > Can you describe more precisely why you had to implement custom
> affinity
> > > functions and not resort to default rendezvous affinity + backup
> filter ?
> > >
> > >
> > > пн, 2 нояб. 2020 г. в 21:45, Raymond Wilson <
> raymond_wil...@trimble.com
> > >:
> > >
> > > > We also use custom affinity functions (vis the C# client).
> > > >
> > > > The wish list mentions use of a particular annotation
> > > > (@CentralizedAffinityFunction):
> > > > Is the wish to remove just this annotation, or the ability to define
> > > custom
> > > > affinity functions at all?
> > > >
> > > > In our case we use affinity functions to ensure particular
> distribution
> > > of
> > > > spatial data across a processing cluster to ensure even load
> > management.
> > > >
> > > > On Tue, Nov 3, 2020 at 5:31 AM Moti Nisenson <
> moti.nisen...@gmail.com>
> > > > wrote:
> > > >
> > > > > I saw at
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0+Wishlist
> > > > > that custom affinity functions are on the potential wishlist for
> > > removal.
> > > > > The way we're using it's very critical that we be able to control
> the
> > > > > placement of data quite precisely - as part of that we specify
> > > explicitly
> > > > > the partition we want in the key, and then our affinity function
> uses
> > > > that
> > > > > (else delegating to default rendezvous). We don't need all the
> > > > > abilities there, although I think that often others do.
> > > > >
> > > > > This seems to me to be a case that the benefit of removing this is
> > > > minimal
> > > > > and could cause quite a lot of disruption to users.
> > > > >
> > > > > Thanks!
> > > > >
> > > >
> > > >
> > > > --
> > > > 
> > > > Raymond Wilson
> > > > Solution Architect, Civil Construction Software Systems (CCSS)
> > > > 11 Birmingham Drive | Christchurch, New Zealand
> > > > +64-21-2013317 Mobile
> > > > raymond_wil...@trimble.com
> > > >
> > > > <
> > > >
> > >
> >
> https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > Best regards,
> > > Alexei Scherbakov
> > >
> >
> >
> > --
> > 
> > Raymond Wilson
> > Solution Architect, Civil Construction Software Systems (CCSS)
> > 11 Birmingham Drive | Christchurch, New Zealand
> > +64-21-2013317 Mobile
> > raymond_wil...@trimble.com
> >
> > <
> >
> https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch
> > >
> >
>
>
> --
>
> Best regards,
> Alexei Scherbakov
>

Re: Interoperable Ignite.NET Dates

[Valentin Kulichenko]

Hi Alexey,

The IEP-54 [1] describes the data layout proposed for Ignite 3.0, it
includes various date/time types. Can you please take a look and check if
this addresses your concerns? We can then discuss further if needed.

https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach

-Val

On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin 
wrote:

> Pavel,
>
> We propose changing public API so this is for Ignite 3.0.
>
> Thanks!
>
> вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn :
>
> > Alexey,
> >
> > Just to clarify before we start the discussion:
> > this proposal seems to introduce some breaking changes, so we are talking
> > about Ignite 3.0, correct?
> >
> > Pavel
> >
> > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > kukushkinale...@gmail.com>
> > wrote:
> >
> > > Igniters,
> > >
> > > What do you think about changing .NET API to read/write portable dates
> by
> > > default and making that really portable?
> > >
> > > *The Problem*
> > > Presently .NET API writes dates as composite Ignite objects. Only .NET
> > > clients can read such dates: any other client (JDBC, Java, etc) does
> not
> > > understand it without custom deserialization.
> > >
> > > It is still possible to configure .NET serialization to write dates as
> > > Ignite dates - see DateTime Serialization note
> > > <
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > >.
> > > But then Ignite accepts only UTC dates, requiring the application
> > > developers to convert local dates to UTC dates and back. This task is
> not
> > > trivial: DateTime.ToUniversalTime
> > > <
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > >
> > > uses
> > > calendars different from Java (and the .NET calendars are the invalid
> > ones
> > > - especially for pre-1990 dates).
> > >
> > > The motivation for the current default behavior was probably the desire
> > to
> > > keep the Time Zone information: Ignite dates do not store time zones.
> > >
> > > In our experience interoperability is more important than storing time
> > zone
> > > info.
> > >
> > > *The Proposal*
> > >
> > >1. Always write .NET dates as portable Ignite dates: get rid of the
> > >BinaryReflectiveSerializer.ForceTimestamp flag that currently
> triggers
> > > this
> > >behavior.
> > >   - We could still keep the ForceTimestamp flag if saving .NET
> dates
> > as
> > >   transparent objects seems a useful case. We do not think it is
> > > useful.
> > >2. Automatically convert Local dates to UTC and back *inside*
> > >Ignite.NET.
> > >   - In this case we lose the DateTime.Kind of UTC dates: we write a
> > UTC
> > >   date but we would read a Local date since Ignite would always
> > > convert UTC
> > >   to Local when reading.
> > >   We could add a UtcDate date flag to QuerySqlFieldAttribute
> > >   and BinaryReflectiveSerializer to control the deserialization
> > > behavior if
> > >   keeping dates in UTC format use case seems important.
> > >3. Use NodaTime  for UTC<->Local
> conversions.
> > >Noda time uses Java calendars making the conversion truely portable.
> > >
> > > *The Benefits*
> > >
> > >1. We think portable dates are much more important than storing time
> > >zone info. Why do we store time zones for every date on the server
> > > anyway?
> > >Time zone is client-side info.
> > >2. Simpler application code: no more manual configuration to trigger
> > the
> > >portable behavior.
> > >3. Non-trivial code to make the truly portable UTC<->Local
> conversion
> > is
> > >implemented once inside Ignite instead of having every Ignite.NET
> > >application implementing it.
> > >
> > > Thank you!
> > >
> > > --
> > > Best regards,
> > > Alexey
> > >
> >
>
>
> --
> Best regards,
> Alexey
>

Re: Interoperable Ignite.NET Dates

[Alexey Kukushkin]

Pavel,

We propose changing public API so this is for Ignite 3.0.

Thanks!

вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn :

> Alexey,
>
> Just to clarify before we start the discussion:
> this proposal seems to introduce some breaking changes, so we are talking
> about Ignite 3.0, correct?
>
> Pavel
>
> On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> kukushkinale...@gmail.com>
> wrote:
>
> > Igniters,
> >
> > What do you think about changing .NET API to read/write portable dates by
> > default and making that really portable?
> >
> > *The Problem*
> > Presently .NET API writes dates as composite Ignite objects. Only .NET
> > clients can read such dates: any other client (JDBC, Java, etc) does not
> > understand it without custom deserialization.
> >
> > It is still possible to configure .NET serialization to write dates as
> > Ignite dates - see DateTime Serialization note
> > <
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > >.
> > But then Ignite accepts only UTC dates, requiring the application
> > developers to convert local dates to UTC dates and back. This task is not
> > trivial: DateTime.ToUniversalTime
> > <
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > >
> > uses
> > calendars different from Java (and the .NET calendars are the invalid
> ones
> > - especially for pre-1990 dates).
> >
> > The motivation for the current default behavior was probably the desire
> to
> > keep the Time Zone information: Ignite dates do not store time zones.
> >
> > In our experience interoperability is more important than storing time
> zone
> > info.
> >
> > *The Proposal*
> >
> >1. Always write .NET dates as portable Ignite dates: get rid of the
> >BinaryReflectiveSerializer.ForceTimestamp flag that currently triggers
> > this
> >behavior.
> >   - We could still keep the ForceTimestamp flag if saving .NET dates
> as
> >   transparent objects seems a useful case. We do not think it is
> > useful.
> >2. Automatically convert Local dates to UTC and back *inside*
> >Ignite.NET.
> >   - In this case we lose the DateTime.Kind of UTC dates: we write a
> UTC
> >   date but we would read a Local date since Ignite would always
> > convert UTC
> >   to Local when reading.
> >   We could add a UtcDate date flag to QuerySqlFieldAttribute
> >   and BinaryReflectiveSerializer to control the deserialization
> > behavior if
> >   keeping dates in UTC format use case seems important.
> >3. Use NodaTime  for UTC<->Local conversions.
> >Noda time uses Java calendars making the conversion truely portable.
> >
> > *The Benefits*
> >
> >1. We think portable dates are much more important than storing time
> >zone info. Why do we store time zones for every date on the server
> > anyway?
> >Time zone is client-side info.
> >2. Simpler application code: no more manual configuration to trigger
> the
> >portable behavior.
> >3. Non-trivial code to make the truly portable UTC<->Local conversion
> is
> >implemented once inside Ignite instead of having every Ignite.NET
> >application implementing it.
> >
> > Thank you!
> >
> > --
> > Best regards,
> > Alexey
> >
>


-- 
Best regards,
Alexey

[jira] [Created] (IGNITE-13657) Flaky test TxOptimisticDeadlockDetectionCrossCacheTest

[Aleksey Plekhanov (Jira)]

Aleksey Plekhanov created IGNITE-13657:
--

 Summary: Flaky test TxOptimisticDeadlockDetectionCrossCacheTest
 Key: IGNITE-13657
 URL: https://issues.apache.org/jira/browse/IGNITE-13657
 Project: Ignite
  Issue Type: Bug
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov


Test TxOptimisticDeadlockDetectionCrossCacheTest is flaky with about 13% 
failure rate. Reason: EVT_CACHE_OBJECT_LOCKED is not listened by default. This 
event should be added as interested explicitly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)