Re: IgniteTxStateImpl's non-threadsafe fields may cause crashes and/or data loss

2023-06-21 Thread Alexei Scherbakov
> >>> inaccuracy then performance reducing. > >>> > >>> > >>> > >>> > >>> On Fri, May 19, 2023 at 7:32 PM Ivan Daschinsky > >>> wrote: > >>> > >>> > >> Tx processing is supposed to be thread bou

Re: IgniteTxStateImpl's non-threadsafe fields may cause crashes and/or data loss

2023-05-19 Thread Alexei Scherbakov
usions if necessary. > > [1] https://issues.apache.org/jira/browse/IGNITE-19445 > -- Best regards, Alexei Scherbakov

Re: Naming style for configuration properties defining durations in Ignite 3

2022-12-26 Thread Alexei Scherbakov
zeros] > > 2. Use short suffixes (ns/ms/sec/min/hr/days): joinTimeoutMs. There is > > a problem with microseconds if we ever need them (how do we abbreviate > > it according to this style?) > > 3. Use longer suffixes (nanos/micros/millis/secs/mins/hours/days): > > joinTimeoutMillis. A bit longer than the previous one. > > 4. ... or something else? > > > > What do you think? > > > -- Best regards, Alexei Scherbakov

Re: [DISCUSSION] Change default behaviour of atomic operations inside transactions

2022-10-28 Thread Alexei Scherbakov
;> > >> Hi all, > >> > >> > >> > >> Can someone please clarify what specific changes will be > implemented > >> in > >> > 2.15 and 2.16? What will be in release notes in 2.15 and 2.16? > >> > >> > >> &g

Re: [DISCUSSION] Change default behaviour of atomic operations inside transactions

2022-10-17 Thread Alexei Scherbakov
By placing the @Deprecated annotation on the property. пн, 17 окт. 2022 г. в 19:07, Anton Vinogradov : > How can we deprecate this? > > On Mon, Oct 17, 2022 at 5:30 PM Alexei Scherbakov < > alexey.scherbak...@gmail.com> wrote: > > > We can do breaking changes by follo

Re: [DISCUSSION] Change default behaviour of atomic operations inside transactions

2022-10-17 Thread Alexei Scherbakov
this ticket as it includes the changes of the default API and we should > not > > break backward compatibility. > > > > Do we need these changes? Should the ticket be closed with "won't fix"? > > > > Have a nice day, > > Julia > > > -- Best regards, Alexei Scherbakov

Re: [DISCUSSION] IEP-91 Transaction protocol

2022-10-12 Thread Alexei Scherbakov
n interface, or that they need Transaction to > be passed as a parameter. Or, if there is still some other mechanism, > I missed it while reading. > > пн, 26 сент. 2022 г. в 16:32, Alexei Scherbakov < > alexey.scherbak...@gmail.com>: > > > > Hello, igniters.

Re: Apache Ignite 3.0.0 beta 1 RELEASE [Time, Scope, Manager]

2022-10-08 Thread Alexei Scherbakov
trics framework: Collection and export of cluster metrics. > > I want to propose myself to be the release manager of the Apache > Ignite 3 beta 1. > > Also I propose the following milestones for the release: > > Scope Freeze: October 12, 2022 > Code Freeze: October 20, 2022 > Voting

[DISCUSSION] IEP-91 Transaction protocol

2022-09-26 Thread Alexei Scherbakov
Hello, igniters. After a long and hard work I've finally pushed the IEP <https://cwiki.apache.org/confluence/display/IGNITE/IEP-91%3A+Transaction+protocol> to the necessary detail level. Any useful feedback would be greatly appreciated. -- Best regards, Alexei Scherbakov

Re: [DISCUSSION] Thin client: Colocated Data Transactional Get

2022-08-11 Thread Alexei Scherbakov
t on the required affinity channel; > > > WDYT? > > > [1] > https://ignite.apache.org/docs/latest/data-modeling/affinity-collocation#affinity-colocation > [2] > https://ignite.apache.org/docs/latest/thin-clients/getting-started-with-thin-clients#partition-awareness > [3] https://issues.apache.org/jira/browse/IGNITE-17316 > [4] > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/client/thin/TcpClientCache.java#L994 > -- Best regards, Alexei Scherbakov

Re: [ANNOUNCE] Welcome Alexander Lapin as a new committer

2022-03-11 Thread Alexei Scherbakov
content about Tracing and Data Rebalance. Now, he is deeply involved > in > >>>> developing Apache Ignite 3.0. > >>>> > >>>> Being a committer enables easier contribution to the project since > there > >>>> is > >>>> no need to go via the patch submission process. This should enable > >> better > >>>> productivity. > >>>> > >>>> Please join me in welcoming Alexander, and congratulating him on the > new > >>>> role in the Apache Ignite Community. > >>>> > >>>> Cheers, > >>>> Kseniya Romanova > >>> > >> > >> -- > >> > >> Best regards, > >> Ivan Pavlukhin > >> > -- Best regards, Alexei Scherbakov

Re: [DISCUSSION] @Nullable/@NotNull annotation usage in Ignite 3

2021-12-16 Thread Alexei Scherbakov
in case > we > > will not be able to agree on any of the above three. No annotations - no > > need for the discussion. > > > > What do you think? Are there any other options out there? I would like to > > collect as many options as possible and organize a vote some time later. > > > > -- > > With regards, > > Aleksandr Polovtcev > > > -- Best regards, Alexei Scherbakov

Re: IEP-61 Transaction API desing for Ignite 3

2021-11-30 Thread Alexei Scherbakov
the primary API. > > > > -Val > > > > On Mon, Nov 29, 2021 at 5:03 AM Pavel Tupitsyn > > wrote: > > > >> Alexei, > >> > >> Are we going to offer an overload without tx parameter? > >> > >> getAsync(K key);

Re: IEP-61 Transaction API desing for Ignite 3

2021-11-29 Thread Alexei Scherbakov
ecordView txView = view.withTransaction(); > > > > > > txView.getAsync(makeKey(1)).thenCompose(r -> > > > txView.upsertAsync(makeValue(1, r.doubleValue("balance") + DELTA), > > > tx)).thenCompose(txView.transaction().commitAsync())

Re: IEP-61 Transaction API desing for Ignite 3

2021-11-28 Thread Alexei Scherbakov
E-15085 [2] org.apache.ignite.tx.IgniteTransactions#runInTransaction(java.util.function.Consumer) [3] org.apache.ignite.tx.IgniteTransactions#runInTransaction(java.util.function.Function) [4] org.apache.ignite.internal.table.TxAbstractTest#testMixedPutGet ср, 14 июл. 2021 г. в 14:12, Alexei S

Re: [VOTE] Allow or prohibit usages of the Guava library methods

2021-09-09 Thread Alexei Scherbakov
fore I > > would > > > >like to propose a vote: > > > > > > > >[+1 Allow]: allow using Guava methods, if justified. > > > >[-1 Prohibit]: prohibit using all Guava methods. > > > > > > > >The voting will commence on Monday, September 13th at 9:00 UTC. Also > > feel > > > >free to express your opinion in the original discussion thread. > > > > > > > >-- > > > >With regards, > > > >Aleksandr Polovtcev > > > > > > > > > > > > > > > > > > > > -- > > Best regards, > > Andrey V. Mashenkov > > > -- Best regards, Alexei Scherbakov

Re: Replace Map with List and Iterable in KeyValueView Ignite 3 APIs

2021-09-09 Thread Alexei Scherbakov
> Notes: > - It is not clear which Pair class to use yet - IgniteBiTuple or something > else. > - Ignite 3 won't deadlock due to putAll entry order, so we don't have to > worry about sorting. > > Thoughts, objections? > -- Best regards, Alexei Scherbakov

Re: [DISCUSSION] Code style for Ignite 3

2021-08-20 Thread Alexei Scherbakov
comes with all the required tools > >and configurations for automation (e.g., here is the config for IDEA: > > > https://github.com/google/error-prone/blob/master/.idea/GoogleStyle.xml > > ) > > > > Please let me know what you think. If there are no objections, I will > start > > the process. > > > > -Val > > > > > -- > With regards, > Aleksandr Polovtcev > -- Best regards, Alexei Scherbakov

Re: Static hierarchy in jmx tree

2021-08-19 Thread Alexei Scherbakov
w JMX is turned off by default, so there is no problem > > > - if you want to use jmx you have to turn it on manually and it would > work. After my patch consistent id in case of persistent or node id in > other cases would be used. > > > - if you want to specify instance name yo

Re: Google Guava in Ignite 3

2021-08-05 Thread Alexei Scherbakov
uava.html?vendor_id=1224 > > > shows no major vulnerability issues for the last three years. > > Taking these points into account, I propose to allow using Guava both in > production and test code and to add it as an explicit dependency. > > What do you think? > > -- > With regards, > Aleksandr Polovtcev > -- Best regards, Alexei Scherbakov

Re: Re[2]: [ANNOUNCE] Welcome Zhenya Stanilovsky as a new committer

2021-08-03 Thread Alexei Scherbakov
modules and important contributions to > the > > > >>>>>> Apache Ignite codebase. He is actively involved in integrating > the > > > >>>>>> Calcite framework with Apache Ignite. Moreover, he has been a > great > > > >>>>>> help in the preparation of several Apache Ignite major releases > by > > > >>>>>> carrying out stress-load tests. Besides the code contributions, > > > >>> Zhenya > > > >>>>>> is also an active community member and help users on dev and > users > > > >>>>>> lists. > > > >>>>>> > > > >>>>>> Being a committer enables easier contribution to the project > since > > > >>>> there > > > >>>>> is > > > >>>>>> no need to go via the patch submission process. This should > enable > > > >>>> better > > > >>>>>> productivity. > > > >>>>>> > > > >>>>>> Please join me in welcoming Ivan, and congratulating him on the > new > > > >>>> role > > > >>>>> in > > > >>>>>> the Apache Ignite Community. > > > >>>>>> > > > >>>>>> Best Regards, > > > >>>>>> Maxim Muzafarov > > > >>>>>> > > > >>>>> > > > >>>>> -- > > > >>>>> Best regards, > > > >>>>> Andrey V. Mashenkov > > > >>>> > > > >>>> Конец пересылаемого сообщения > > > >>>> > > > >>> > > > > > > > > > > > > > > > -- Best regards, Alexei Scherbakov

Re: IEP-61 Transaction API desing for Ignite 3

2021-07-14 Thread Alexei Scherbakov
n) { > > > > return Transactions > > > > .beginTransaction() > > > > .thenCompose(tx -> { > > > > CompletableFuture result = action.apply(tx) > > > > .handle((v, ex) -> null); >

Re: IEP-61 Transaction API desing for Ignite 3

2021-07-14 Thread Alexei Scherbakov
Adding table.kvView().withTx(tx) seems fine to me. ср, 14 июл. 2021 г. в 12:15, Alexei Scherbakov : > Ivan, > > And what if I have already committed transaction? Is it safe rollback > already committed transaction? Rollback will silently return and do > nothing? - ye

Re: IEP-61 Transaction API desing for Ignite 3

2021-07-14 Thread Alexei Scherbakov
-> tx.closeAsync()); > > }); > > } > > > > Async api is not very readable, but this method can help user write code, > > this is rewritten first example: > > > > Transactions.inTxAsync(tx -> { > > CacheApi txCache = cache.withTx(tx); >

Re: IEP-61 Transaction API desing for Ignite 3

2021-07-14 Thread Alexei Scherbakov
t; > Transactions.inTxAsync(tx -> { > CacheApi txCache = cache.withTx(tx); > return txCache.getAsync("key") > .thenCompose(val -> { > if (val == "test") { > return txCache.putAsync("key", "

Re: IEP-61 Transaction API desing for Ignite 3

2021-07-14 Thread Alexei Scherbakov
/ignite/internal/table/TxTest.java вт, 13 июл. 2021 г. в 19:41, Andrey Gura : > Alexey, > > could you please describe Transaction interface? > > Also it would be great to have a couple examples of using the proposed API. > > On Tue, Jul 13, 2021 at 4:43 PM Alexe

Re: IEP-61 Transaction API desing for Ignite 3

2021-07-13 Thread Alexei Scherbakov
and similar to Ignite 2, but has some differences. > > > > More details can be found here [1] > > > > Share your thoughts. > > > > [1] https://issues.apache.org/jira/browse/IGNITE-15086 > > > > -- > > > > Best regards, > > Alexei Scherbakov > > > -- Best regards, Alexei Scherbakov

IEP-61 Transaction API desing for Ignite 3

2021-07-13 Thread Alexei Scherbakov
Folks, I've prepared a PR implementing my vision of public transactions API. API is very simple and similar to Ignite 2, but has some differences. More details can be found here [1] Share your thoughts. [1] https://issues.apache.org/jira/browse/IGNITE-15086 -- Best regards, Alexei

Re: Ignite 3.0 Tuple API: how to check if value is null?

2021-07-12 Thread Alexei Scherbakov
tuple.isNull(colName) to test for emptiness also seems useful. пн, 12 июл. 2021 г. в 18:20, Alexei Scherbakov : > +1 to make some improvements here. > > Using Optional doesn't make sense to me because it always involves boxing > (and we already have tuple.value(colName)). > >

Re: Ignite 3.0 Tuple API: how to check if value is null?

2021-07-12 Thread Alexei Scherbakov
PI yet, > > > > > > > > or do we? > > > > > > > > > > > > > > > > On Tue, Jul 6, 2021 at 12:44 AM Valentin Kulichenko < > > > > > > > > valentin.kuliche...@gmail.com> wrote: > > > > > > > > > > > > > > > > > Pavel, > > > > > > > > > > > > > > > > > > That's a good point, but I don't think there is a > > significantly > > > > > > better > > > > > > > > way > > > > > > > > > of doing this in Java. > > > > > > > > > > > > > > > > > > There should be a way to check if a field is nullable or > not > > > > > though. > > > > > > > > Schema > > > > > > > > > already provides this information, doesn't it? > > > > > > > > > > > > > > > > > > -Val > > > > > > > > > > > > > > > > > > On Mon, Jul 5, 2021 at 11:03 AM Pavel Tupitsyn < > > > > > ptupit...@apache.org > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > > > > > Looks like Tuple API has no efficient way to tell if a > > value > > > > for > > > > > a > > > > > > > > > nullable > > > > > > > > > > column of primitive type is null. > > > > > > > > > > > > > > > > > > > > - Tuple#intValue() will return 0 when the actual value is > > > null > > > > => > > > > > > not > > > > > > > > > clear > > > > > > > > > > if 0 is 0 or null. > > > > > > > > > > - Tuple#value() works, but is more expensive due to > boxing > > > and > > > > > type > > > > > > > > > lookup. > > > > > > > > > > > > > > > > > > > > Any ideas on how to improve this? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Best regards, Alexei Scherbakov

Re: Problem with dropping the index

2021-07-08 Thread Alexei Scherbakov
does it) in parallel. > > > > If we consider your proposal, then the creation and deletion of the index > > will go in parallel and they will look at the same roots. > > > > You can

Re: MOVING Partitions and Rebalancing

2021-07-07 Thread Alexei Scherbakov
eries (maybe assign/use a > > globally unique ID per entry, and use that to remove duplicates during > > the merge phase?) > > > > Please share thoughts and inputs. > > > > Atri > > > -- Best regards, Alexei Scherbakov

Re: Problem with dropping the index

2021-07-07 Thread Alexei Scherbakov
me an example ? >From my understanding, the suggested solution should work ok for any number of create/drop sequences. > > 06.07.2021, 14:00, "Alexei Scherbakov" : > > Can you clarify what it means to rename root index trees ? > > > > The simple solution which

Re: Problem with dropping the index

2021-07-06 Thread Alexei Scherbakov
ask. > Thus, if we find the renamed root pages in task 1, we can clear the index > trees to the end, otherwise the task can be completed. > Also, if we find that rename pages are present, and the step 2 has not > been completed, then we can start rebuilding the indexes. > > WDYT? > -- Best regards, Alexei Scherbakov

Re: IEP-61 Technical discussion

2021-07-01 Thread Alexei Scherbakov
/transactions сб, 20 мар. 2021 г. в 21:00, Alexei Scherbakov : > Folks, > > I want to share some information about progress in implementing the raft > protocol in ignite 3, which is a prerequisite for metastorage. > > The implementation will consist of client and server m

Re: Text Queries Support

2021-06-21 Thread Alexei Scherbakov
score. Since the scoring function is homogeneous, this > > means that across nodes, we can compare scores and merge sort. > > > > Please let me know if I can take this up. > > > > Atri > > > > -- > > Regards, > > > > Atri > > Apache Concerted > > > -- Best regards, Alexei Scherbakov

Re: [DISCUSSION] Code style. Variable abbrevations

2021-06-07 Thread Alexei Scherbakov
> > > >>> Abbrevation should be used for MAX_KEY_COUNT! Please, use cnt, > > instead > > > of > > > >>> COUNT [IgniteAbbrevationsRule] > > > >>> [ERROR] > > > >>> > > > > > > /Users/sbt-izhikov-nv/work/ignite/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/IgnitePdsTransactionsHangTest.java:78: > > > >>> Abbrevation should be used for CHECKPOINT_FREQUENCY! Please, use > > freq, > > > >>> instead of FREQUENCY [IgniteAbbrevationsRule] > > > >>> [ERROR] > > > >>> > > > > > > /Users/sbt-izhikov-nv/work/ignite/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/SlowHistoricalRebalanceSmallHistoryTest.java:57: > > > >>> Abbrevation should be used for SUPPLY_MESSAGE_LATCH! Please, use > msg, > > > >>> instead of MESSAGE [IgniteAbbrevationsRule] > > > >>> [ERROR] > > > >>> > > > > > > /Users/sbt-izhikov-nv/work/ignite/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/IgniteLogicalRecoveryTest.java:74: > > > >>> Abbrevation should be used for SHARED_GROUP_NAME! Please, use grp, > > > instead > > > >>> of GROUP [IgniteAbbrevationsRule] > > > >>> [ERROR] > > > >>> > > > > > > /Users/sbt-izhikov-nv/work/ignite/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/IgniteLogicalRecoveryTest.java:200: > > > >>> Abbrevation should be used for cacheLoader! Please, use ldr, > instead > > of > > > >>> Loader [IgniteAbbrevationsRule] > > > >>> ``` > > > >>> > > > >>> [1] > > > >>> > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules#AbbreviationRules-VariableAbbreviation > > > >>> [2] https://github.com/apache/ignite/pull/9153 > > > >>> > > > >>> > > > >> > > > > > > > > > -- Best regards, Alexei Scherbakov

Re: Node and cluster life-cycle in ignite-3

2021-06-03 Thread Alexei Scherbakov
I've made a small mistake above, the correct sentence is void stop(); // Stop a component. Invoked in "depends on" relation order. In the example above, RaftGroup is stopped before the Network. чт, 3 июн. 2021 г. в 17:36, Alexei Scherbakov : > Sergey, I'm ok with the runl

Re: Node and cluster life-cycle in ignite-3

2021-06-03 Thread Alexei Scherbakov
; >1. As I see right now it is hard to standardize initialization events > >components notify each other with. > > > >2. It is not clear if run-levels should be organized into one rigid > >hierarchy (when the first run-level should always precede the second > > and so > >on) or they should be more independent. > > > > > > What do you think? > > > > [1] > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-73%3A+Node+startup > > > -- Best regards, Alexei Scherbakov

Re: Static hierarchy in jmx tree

2021-04-28 Thread Alexei Scherbakov
sig-email?utm_medium=email_source=link_campaign=sig-email_content=webmail> <#m_-5956278403710209994_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> пт, 16 апр. 2021 г. в 14:16, Igor Akkuratov : > Is there anybody alive? > -- Best regards, Alexei Scherbakov <https://www.avast.com/sig-em

Re: [DISCUSSION] Error handling in Ignite 3

2021-04-21 Thread Alexei Scherbakov
n a proper message to the user, so I would say that > some error conversion/wrapping will be required no matter what. > > --AG > > пт, 16 апр. 2021 г. в 16:31, Alexei Scherbakov < > alexey.scherbak...@gmail.com > >: > > > чт, 15 апр. 2021 г. в 18:21, Andrey Mas

Re: [DISCUSSION] Error handling in Ignite 3

2021-04-21 Thread Alexei Scherbakov
at about String representation? > > I think we should use a common prefix for error codes in error messages. > Such codes will be more searchable and as a bonus, vendor-specific. > String representation might look like IGN-001042 or IGN-TBL042. > > > > > On Wed,

Re: [DISCUSSION] Error handling in Ignite 3

2021-04-21 Thread Alexei Scherbakov
I've create the ticket for implementing this [1] [1] https://issues.apache.org/jira/browse/IGNITE-14611 пт, 16 апр. 2021 г. в 16:30, Alexei Scherbakov : > > > чт, 15 апр. 2021 г. в 18:21, Andrey Mashenkov >: > >> Hi Alexey, >> I like the idea. >> >

Re: [DISCUSSION] Error handling in Ignite 3

2021-04-16 Thread Alexei Scherbakov
cases? > > The first approach may looks confusing, while the second one produces too > many "UNK-" codes. > What code should get the user in that case? > > > The method should always report root cause, in your example it will be B-, no matter which module

Re: [DISCUSSION] Error handling in Ignite 3

2021-04-15 Thread Alexei Scherbakov
use > > pattern like this: > > try{ > > ... > > } > > catch (Exception e) { > > if (X.hasCause(e, TimeoutException.class)) > > throw e; > > > > if (X.hasCause(e, ConnectException.class, EOFException.class)) > > continue; > >

Re: [DISCUSSION] Error handling in Ignite 3

2021-04-14 Thread Alexei Scherbakov
de + message should provide enough information to the user. >- For async methods returning a Future we may have a universal rule on >how to handle exceptions. For example, we may specify that any async > method >can throw only invalid argument exceptions. A

[DISCUSSION] Error handling in Ignite 3

2021-04-13 Thread Alexei Scherbakov
/cd/B10501_01/server.920/a96525/preface.htm -- Best regards, Alexei Scherbakov

Re: [ANNOUNCE] Welcome Ivan Daschinsky as a new committer

2021-04-13 Thread Alexei Scherbakov
t; Being a committer enables easier contribution to the project since > there > >>> is > >>> no need to go via the patch submission process. This should enable > >>> better > >>> productivity. > >>> > >>> Please join me in welcoming Ivan, and congratulating him on the new > role > >>> in > >>> the Apache Ignite Community. > >>> > >>> Best Regards, > >>> Igor > >> > >> > >> > >> > > > > > > > -- > > Best regards, > Ivan Pavlukhin > -- Best regards, Alexei Scherbakov

Re: Model of permissions for Ignite 3

2021-04-12 Thread Alexei Scherbakov
> > Interface 'GridSecurityProcessor' has a default implementation of > the > > > > > 'authorize' method. > > > > > 'SecurityTestSuite' is green. > > > > > > > > > > > > > > > This representation of permission, IMHO, has the following > > advantages: > > > > > - A developer can easily add new permission without needing to > touch > > > the > > > > > core module. > > > > > - There is no need to implement complicated logic to authorize an > > > > > operation inside a security plugin. > > > > >But a developer has the opportunity to add custom logic. > > > > > - Wildcards for permission's name from a box, for example, 'new > > > > > CachePermission("x.y.z.*", "get,put")'. > > > > > - There is no need to implement 'SecurityPermissionSet' and a set > of > > > > > methods from 'SecurityContex' ('xxxAllowed(String, > > > SecurityPermission))'. > > > > > - We can define a security policy in a file as java does. It could > > > > > simplify work for administrators. > > > > > > > > > > WDYT? > > > > > > > > > > > > > > > > > > -- > > > Best regards, > > > Andrey Kuznetsov. > > > > > > -- Best regards, Alexei Scherbakov

Re: Terms clarification and modules splitting logic

2021-04-08 Thread Alexei Scherbakov
violation directly > at the usage site because there will be an import statement with an > 'internal' package. You can check this as simple as an obvious grep > command, which will not work with the annotation. > > --AG > > ср, 31 мар. 2021 г. в 21:04, Alexei Scherbakov < >

Re: [DISSCUSSION] Common logger interface.

2021-04-08 Thread Alexei Scherbakov
in an IgniteThread field to avoid lookups into thread-local-map. > > > > For user threads, only ThreadLocals can be used. > > Is it really need to use thread-locals for user threads? > > Will it be always obvious which node exception was thrown on? Any kind of > > e

Re: [DISSCUSSION] Common logger interface.

2021-04-08 Thread Alexei Scherbakov
bled(); > > boolean isDebugEnabled(); > > or > > boolean isLoggable(Level) - the same way which System.Logger suggests > > > > Thanks, > > S. > > > > пт, 26 мар. 2021 г. в 17:41, Alexei Scherbakov < > > alexey.scherbak...@gmail.com >

Re: Terms clarification and modules splitting logic

2021-03-31 Thread Alexei Scherbakov
gt; alexey.goncha...@gmail.com > > >: > > > > > Alexei, > > > > > > I had the same opinion regarding the internal package, but we still > need > > to > > > somehow distinguish between public and internal classes in the > > ignite-util

Re: Terms clarification and modules splitting logic

2021-03-31 Thread Alexei Scherbakov
in the util, we should follow > the same structure in other modules. > > Thoughts? > > вт, 30 мар. 2021 г. в 18:37, Alexei Scherbakov < > alexey.scherbak...@gmail.com > >: > > > +1 to package and module naming. > > +1 to service definition as "com

Re: Terms clarification and modules splitting logic

2021-03-30 Thread Alexei Scherbakov
> > > > > >- affinity // TO be created. > > >- api [public] > > >- baseline // TO be created. > > >- bytecode > > >- cli > > >- cli-common > > >- configuration > > >- configuration-annotation-processor > > >- core // Module with classes like IgniteUuid. Should we raname it > to > > >lang/utils/commons? > > >- metastorage-client // To be created. > > >- metastorage-common // To be created. > > >- metastorage-server // TO be created. > > >- network > > >- raft // raft-server? > > >- raft-client > > >- rest > > >- runner > > >- schema > > >- table // Seems that there might be a conflict between the meaning > of > > >table module that we already have and table module with > > > create/dropTable() > > >- vault > > > > > > Also it's not quite clear to me how we should split lang and util > classes > > > some of which belong to the public api, and some to the private. > > > > > > Please share your thoughts about topics mentioned above. > > > > > > Best regards, > > > Alexander > > > > > > > > -- > Best regards, > Andrey V. Mashenkov > -- Best regards, Alexei Scherbakov

Re: [DISSCUSSION] Common logger interface.

2021-03-26 Thread Alexei Scherbakov
fault and > > extend it with shortcut methods. > > > > I've created a ticket to unify logger usage in Ignite-3.0 project to fix > > already existed code. > > > > Any thoughts or objections? > > > > -- > > Best regards, > > Andrey V. Mashenkov > > > > > -- > Best regards, > Andrey V. Mashenkov > -- Best regards, Alexei Scherbakov

Re: IEP-70: Async Continuation Executor

2021-03-26 Thread Alexei Scherbakov
I know Java is late to the async party and not everyone is used to this > > mindset, > > but the situation changes, more and more code bases go async all the way, > > use CompletionStage everywhere, etc. > > > > > > > If we go with the public pool - no additional options

Re: [DISCUSSION] IgniteFuture class future in Ignite-3.0.

2021-03-25 Thread Alexei Scherbakov
ust a legacy.' > > > > Any thoughts? > > > > [1] > > > > > https://spotify.github.io/completable-futures/apidocs/com/spotify/futures/ConcurrencyReducer.html > > [2] > > > > > https://lettuce.io/lettuce-4/release/api/com/lambdaworks/redis/RedisFuture.html > > [3] > > > > > https://javadoc.io/static/org.jspare.vertx/vertx-jspare/1.1.0-M03/org/jspare/vertx/concurrent/VertxCompletableFuture.html > > -- > > Best regards, > > Andrey V. Mashenkov > > > -- Best regards, Alexei Scherbakov

Re: IEP-70: Async Continuation Executor

2021-03-25 Thread Alexei Scherbakov
major issues). > > On Thu, Mar 25, 2021 at 3:06 PM Alexei Scherbakov < > alexey.scherbak...@gmail.com> wrote: > > > Pavel, > > > > While I understand the issue and overall agree with you, I'm against the > > execution of listeners in separate thread poo

Re: [DISCUSSION] Patch completely breaks MVCC.

2021-03-25 Thread Alexei Scherbakov
test. I just removed it from a > > > parameters list; > > > 2. There are tests that run only for MVCC. I marked them with the > @Ignore > > > annotation. > > > > > > But would it better just completely remove all such tests that are > broken > > > by the patch? > > > > > > [1] > > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/RESULT-VOTE-Removing-MVCC-public-API-td50705.html#a50706 > > > > > > > > -- > Best regards, > Andrey V. Mashenkov > -- Best regards, Alexei Scherbakov

Re: IEP-70: Async Continuation Executor

2021-03-25 Thread Alexei Scherbakov
, the code that is provided by IEP can changed as > > >>>>>>> follows: > > >>>>>>>>>> > > >>>>>>>>>> IgniteFuture fut = cache.putAsync(1, 1); > > >>>>>>>>>> fut.listenAync(f -> { > > >>>>>>>>>>// Executes on Striped pool and deadlocks. > > >>>>>>>>>>cache.replace(1, 2); > > >>>>>>>>>> }, ForkJoinPool.commonPool()); > > >>>>>>>>>> > > >>>>>>>>>> Of course, it does not mean that this fact should not be > > >>>>> properly > > >>>>>>>>>> documented. > > >>>>>>>>>> Perhaps, I am missing something. > > >>>>>>>>>> > > >>>>>>>>>> Thanks, > > >>>>>>>>>> S. > > >>>>>>>>>> > > >>>>>>>>>> ср, 17 мар. 2021 г. в 16:01, Pavel Tupitsyn < > > >>>>> ptupit...@apache.org > > >>>>>>> : > > >>>>>>>>>> > > >>>>>>>>>>> Slava, > > >>>>>>>>>>> > > >>>>>>>>>>> Your suggestion is to keep things as is and discard the IEP, > > >>>>>> right? > > >>>>>>>>>>> > > >>>>>>>>>>>> this can lead to significant overhead > > >>>>>>>>>>> Yes, there is some overhead, but the cost of accidentally > > >>>>>> starving > > >>>>>>>> the > > >>>>>>>>>>> striped pool is worse, > > >>>>>>>>>>> not to mention the deadlocks. > > >>>>>>>>>>> > > >>>>>>>>>>> I believe that we should favor correctness over performance > > >>>>> in > > >>>>>> any > > >>>>>>>>> case. > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>>> On Wed, Mar 17, 2021 at 3:34 PM Вячеслав Коптилин < > > >>>>>>>>>>> slava.kopti...@gmail.com> > > >>>>>>>>>>> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>>> Well, the specified method already exists :) > > >>>>>>>>>>>> > > >>>>>>>>>>>>/** > > >>>>>>>>>>>> * Registers listener closure to be asynchronously > > >>>>> notified > > >>>>>>>>>> whenever > > >>>>>>>>>>>> future completes. > > >>>>>>>>>>>> * Closure will be processed in specified executor. > > >>>>>>>>>>>> * > > >>>>>>>>>>>> * @param lsnr Listener closure to register. Cannot be > > >>>>>> {@code > > >>>>>>>>>> null}. > > >>>>>>>>>>>> * @param exec Executor to run listener. Cannot be > > >>>>> {@code > > >>>>>>>> null}. > > >>>>>>>>>>>> */ > > >>>>>>>>>>>>public void listenAsync(IgniteInClosure > >>>>>>>> IgniteFuture> > > >>>>>>>>>>> lsnr, > > >>>>>>>>>>>> Executor exec); > > >>>>>>>>>>>> > > >>>>>>>>>>>> Thanks, > > >>>>>>>>>>>> S. > > >>>>>>>>>>>> > > >>>>>>>>>>>> ср, 17 мар. 2021 г. в 15:25, Вячеслав Коптилин < > > >>>>>>>>>> slava.kopti...@gmail.com > > >>>>>>>>>>>> : > > >>>>>>>>>>>> > > >>>>>>>>>>>>> Hello Pavel, > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> I took a look at your IEP and pool request. I have the > > >>>>>>> following > > >>>>>>>>>>>> concerns. > > >>>>>>>>>>>>> First of all, this change breaks the contract of > > >>>>>>>>>>>> IgniteFuture#listen(lsnr) > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>/** > > >>>>>>>>>>>>> * Registers listener closure to be asynchronously > > >>>>>> notified > > >>>>>>>>>>> whenever > > >>>>>>>>>>>>> future completes. > > >>>>>>>>>>>>> * Closure will be processed in thread that > > >>>>> completes > > >>>>>> this > > >>>>>>>>> future > > >>>>>>>>>>> or > > >>>>>>>>>>>>> (if future already > > >>>>>>>>>>>>> * completed) immediately in current thread. > > >>>>>>>>>>>>> * > > >>>>>>>>>>>>> * @param lsnr Listener closure to register. Cannot > > >>>>> be > > >>>>>>> {@code > > >>>>>>>>>>> null}. > > >>>>>>>>>>>>> */ > > >>>>>>>>>>>>>public void listen(IgniteInClosure > >>>>>> IgniteFuture> > > >>>>>>>>>> lsnr); > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>In your pull request, the listener is always called > > >>>>> from > > >>>>>> a > > >>>>>>>>>>> specified > > >>>>>>>>>>>>> thread pool (which is fork-join by default) > > >>>>>>>>>>>>>even though the future is already completed at the > > >>>>> moment > > >>>>>>> the > > >>>>>>>>>>> listen > > >>>>>>>>>>>>> method is called. > > >>>>>>>>>>>>>In my opinion, this can lead to significant > > >>>>> overhead - > > >>>>>>>>> submission > > >>>>>>>>>>>>> requires acquiring a lock and notifying a pool thread. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>It seems to me, that we should not change the > > >>>>> current > > >>>>>>>> behavior. > > >>>>>>>>>>>>> However, thread pool executor can be added as an > > >>>>> optional > > >>>>>>>> parameter > > >>>>>>>>>> of > > >>>>>>>>>>>>> listen() method as follows: > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>public void listen(IgniteInClosure > >>>>>>>> IgniteFuture> > > >>>>>>>>>>> lsnr, > > >>>>>>>>>>>>> Executor exec); > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Thanks, > > >>>>>>>>>>>>> S. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> пн, 15 мар. 2021 г. в 19:24, Pavel Tupitsyn < > > >>>>>>>> ptupit...@apache.org > > >>>>>>>>>> : > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> Igniters, > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> Please review the IEP [1] and let me know your > > >>>>> thoughts. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> [1] > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>> > > >>>>> > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-70%3A+Async+Continuation+Executor > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>> > > >>>>> > > >>>>> > > >>>>> -- > > >>>>> <http://www.trimble.com/> > > >>>>> Raymond Wilson > > >>>>> Solution Architect, Civil Construction Software Systems (CCSS) > > >>>>> 11 Birmingham Drive | Christchurch, New Zealand > > >>>>> raymond_wil...@trimble.com > > >>>>> > > >>>>> < > > >>>>> > > > https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch > > >>>>>> > > >>>>> > > >>>> > > > -- Best regards, Alexei Scherbakov

Re: IEP-61 Technical discussion

2021-03-20 Thread Alexei Scherbakov
gt; > > > >>> >> > >> > > > > > > >>> >> > >> PacificA suggests internal heartbeats mechanism for > > > failure > > > > > > >>> >> detection of > > > > > > >>> >> > >> replicated group, but it says nothing about initial > > > > discovery > > > > > > of > > > > > > >>> >> nodes. > > > > > > >>> >> > >> > > > > > > >>> >> > >> WDYT? > > > > > > >>> >> > >> > > > > > > >>> >> > >> [1] -- > https://www.consul.io/docs/architecture/gossip > > > > > > >>> >> > >> [2] -- https://www.serf.io/ > > > > > > >>> >> > >> > > > > > > >>> >> > >> чт, 19 нояб. 2020 г. в 12:46, Alexey Goncharuk < > > > > > > >>> >> > >> alexey.goncha...@gmail.com>: > > > > > > >>> >> > >> > > > > > > >>> >> > >>> Following up the Ignite 3.0 scope/development > approach > > > > > > threads, > > > > > > >>> >> this is > > > > > > >>> >> > >>> a separate thread to discuss technical aspects of > the > > > IEP. > > > > > > >>> >> > >>> > > > > > > >>> >> > >>> Let's reiterate one more time on the questions > raised > > by > > > > > Ivan > > > > > > >>> and > > > > > > >>> >> also > > > > > > >>> >> > >>> see if there are any other thoughts on the IEP: > > > > > > >>> >> > >>> > > > > > > >>> >> > >>>- *Whether to deploy metastorage on a separate > > subset > > > > of > > > > > > the > > > > > > >>> >> nodes > > > > > > >>> >> > >>>or allow Ignite to choose these nodes > > > automatically.* I > > > > > > >>> think it > > > > > > >>> >> is > > > > > > >>> >> > >>>feasible to maintain both modes: by default, > Ignite > > > > will > > > > > > >>> choose > > > > > > >>> >> > >>>metastorage nodes automatically which essentially > > > will > > > > > > >>> provide > > > > > > >>> >> the > > > > > > >>> >> > same > > > > > > >>> >> > >>>seamless user experience as TCP discovery SPI - > no > > > > > separate > > > > > > >>> >> roles, > > > > > > >>> >> > >>>simplistic deployment. For deployments where > people > > > > want > > > > > to > > > > > > >>> have > > > > > > >>> >> > more > > > > > > >>> >> > >>>fine-grained control over the nodes' assignments, > > we > > > > will > > > > > > >>> >> provide a > > > > > > >>> >> > runtime > > > > > > >>> >> > >>>configuration which will allow pinning > metastorage > > > > group > > > > > to > > > > > > >>> >> certain > > > > > > >>> >> > nodes, > > > > > > >>> >> > >>>thus eliminating the latency concerns. > > > > > > >>> >> > >>>- *Whether there are any TLA+ specs for the > > PacificA > > > > > > >>> protocol.* > > > > > > >>> >> Not > > > > > > >>> >> > >>>to my knowledge, but it is known to be used in > > > > production > > > > > > by > > > > > > >>> >> > Microsoft and > > > > > > >>> >> > >>>other projects, e.g. [1] > > > > > > >>> >> > >>> > > > > > > >>> >> > >>> I would like to collect general feedback on the IEP, > > as > > > > well > > > > > > as > > > > > > >>> >> > feedback > > > > > > >>> >> > >>> on specific parts of it, such as: > > > > > > >>> >> > >>> > > > > > > >>> >> > >>>- Metastorage API > > > > > > >>> >> > >>>- Any existing library that can be used to avoid > > > > > > >>> re-implementing > > > > > > >>> >> the > > > > > > >>> >> > >>>protocol ourselves? Perhaps, porting the existing > > > > > > >>> implementation > > > > > > >>> >> to > > > > > > >>> >> > Java > > > > > > >>> >> > >>>(the way TiKV did with etcd-raft [2] [3]? This > is a > > > > very > > > > > > >>> neat way > > > > > > >>> >> > btw in my > > > > > > >>> >> > >>>opinion because I like the finite automata-like > > > > approach > > > > > of > > > > > > >>> the > > > > > > >>> >> > replication > > > > > > >>> >> > >>>module, and, additionally, we could sync bug > fixes > > > and > > > > > > >>> >> improvements > > > > > > >>> >> > from > > > > > > >>> >> > >>>the upstream project) > > > > > > >>> >> > >>> > > > > > > >>> >> > >>> > > > > > > >>> >> > >>> Thanks, > > > > > > >>> >> > >>> --AG > > > > > > >>> >> > >>> > > > > > > >>> >> > >>> [1] > > > > > > >>> >> > >>> > > > > > > >>> >> > > > > > > > > > https://cwiki.apache.org/confluence/display/INCUBATOR/PegasusProposal > > > > > > >>> >> > >>> [2] > https://github.com/etcd-io/etcd/tree/master/raft > > > > > > >>> >> > >>> [3] https://github.com/tikv/raft-rs > > > > > > >>> >> > >>> > > > > > > >>> >> > >> > > > > > > >>> >> > >> > > > > > > >>> >> > >> -- > > > > > > >>> >> > >> Sincerely yours, Ivan Daschinskiy > > > > > > >>> >> > >> > > > > > > >>> >> > >> > > > > > > >>> >> > >> -- > > > > > > >>> >> > >> Sincerely yours, Ivan Daschinskiy > > > > > > >>> >> > >> > > > > > > >>> >> > > > > > > > > >>> >> > > > > > > > > >>> >> > > -- > > > > > > >>> >> > > Sincerely yours, Ivan Daschinskiy > > > > > > >>> >> > > > > > > > > >>> >> > > > > > > > >>> >> > > > > > > > >>> >> > -- > > > > > > >>> >> > Sincerely yours, Ivan Daschinskiy > > > > > > >>> >> > > > > > > > >>> >> > > > > > > >>> > > > > > > > >>> > > > > > > > >>> > -- > > > > > > >>> > Sincerely yours, Ivan Daschinskiy > > > > > > >>> > > > > > > > >>> > > > > > > >>> > > > > > > >>> -- > > > > > > >>> Sincerely yours, Ivan Daschinskiy > > > > > > >>> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > -- > > > > > Sincerely yours, Ivan Daschinskiy > > > > > > > > > > > > > > > > > > -- > > > Sincerely yours, Ivan Daschinskiy > > > > > > -- Best regards, Alexei Scherbakov

Re: [DISCUSSION] IEP-69 The evolutionary release process

2021-03-09 Thread Alexei Scherbakov
grand.major.bug-fix[-rc_number] > > > > > > > > > > > > > > 1. Prepare the next 3.0 release based on 2.x with some breaking > > > > > > > compatibility changes. The same things happen from time to time > > > with > > > > > > > other Apache projects like Hadoop, Spark. > > > > > > > > > > > > > > 2. Discuss with the whole Community and assign the right > release > > > > > > > version to the activities related to the development of the new > > > Ignite > > > > > > > architecture (currently all the changes you can find in the > > > ignite-3 > > > > > > > branch). > > > > > > > I see no 3.0 discussions on the dev-list and I see no-activity > with > > > > > > > the 3.0 version currently. So, it's better to remove the > `lock` > > > from > > > > > > > the 3.0 version and allow the removal of obsolete features. > > > > > > > > > > > > > > 3. Guarantee the PDS compatibility between the `grand` > versions of > > > the > > > > > > > Apache Ignite for the next year. > > > > > > > > > > > > > > 4. Guarantee the bug-fix release for the last 2.x Apache Ignite > > > > > > > version for the next year. > > > > > > > > > > > > > > 5. Perform some improvements which break the backward > > > compatibility, > > > > > > > for instance: removing @deprecated API (except metrics), > removing > > > > > > > obsolete modules, changing the cluster defaults. You can find > > > > > > > additional details on the IEP-69 page [1]. > > > > > > > > > > > > > > > > > > > > > Please, share your thoughts. > > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-69%3A+The+evolutionary+release+process > > > > > > > > > > > > > > > > > -- Best regards, Alexei Scherbakov

Re: [DISCUSSION] Java 11 for Ignite 3.0 development

2020-12-08 Thread Alexei Scherbakov
with running Ignite > > under Java > > >>>> > > 9+ (modularization which requires dozens of command-line > > options in > > >>>> > > order to forcibly export corresponding packages) and GraalVM. > > Such a > > >>>> > > situation could be described as bad user experience and we > > should fix > > >>>> > > it. Var handles [2] could be used for solving described > > problems. > > >>>> > > > > >>>> > > * Java 9+ introduces Flight Recorder API [3] which could be > > used in > > >>>> > > the Ignite project for lightweight profiling of internal > > processes. > > >>>> > > > > >>>> > > Please, share your opinions, objections and ideas about this > > topic. I > > >>>> > > hope we will not have serious disagreements and the consensus > > will be > > >>>> > > reached quickly. > > >>>> > > > > >>>> > > > > >>>> > > 1. > > >>>> > > > > >>>> > > > >>>> > > > http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Ignite-3-0-development-approach-tp49922p50295.html > > >>>> > > 2. > > >>>> > > > > >>>> > > > >>>> > > > https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/VarHandle.html > > >>>> > > 3. > > >>>> > > > > >>>> > > > >>>> > > > https://docs.oracle.com/en/java/javase/11/docs/api/jdk.jfr/jdk/jfr/FlightRecorder.html > > >>>> > > > > >>>> > > > > > > -- > Sincerely yours, > Ivan Bessonov > -- Best regards, Alexei Scherbakov

Re: Removing MVCC public API

2020-12-08 Thread Alexei Scherbakov
t;>>> versions. > >>>>>>>> Also, the MVCC feature is in an experimental state, so it can be > >>>> modified > >>>>>>>> in any way, I think. > >>>>>>>> > >>>>>>>> +1 - to accept removing MVVC feature from public API > >>>>>>>> 0 - don't care either way > >>>>>>>> -1 - do not accept removing API (explain why) > >>>>>>>> > >>>>>>>> The vote will hold for 7 days and will end on Wednesday, December > >>>> 16th at > >>>>>>>> 19:00 UTC: > >>>>>>>> > >>>>>>>> > >>>> > https://www.timeanddate.com/countdown/generic?iso=20201216T19=1440=cursive > >>>>>>>> > >>>>>>>> [1] > >>>>>>>> > >>>>>>>> > >>>> > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-td45669.html > >>>>>>>> [2] > >>>>>>>> > >>>>>>>> > >>>> > http://apache-ignite-developers.2346864.n4.nabble.com/Disable-MVCC-test-suites-td50416.html > >>>>>>>> [3] > >>>>>>>> > >>>>>>>> > >>>> > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45727.html > >>>>>>>> [4] > >>>>>>>> > >>>>>>>> > >>>> > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45716.html > >>>>>>>> [5] > >>>>>>>> > >>>>>>>> > >>>> > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45714.html > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> Slava. > >>>>>>>> > >>> > >>> -- > >>> Best regards, > >>> Andrey V. Mashenkov > > > > > > > > -- > > Best wishes, > > Amelchev Nikita > > -- Best regards, Alexei Scherbakov

Re: [DISCUSS] Ignite 3.0 development approach

2020-11-10 Thread Alexei Scherbakov
t; nizhi...@apache.org > > > > > > > > > > > > > > > > > >>>>>> wrote: > > > > > > > > >&g

Re: Custom Affinity Functions proposed for removal?

2020-11-02 Thread Alexei Scherbakov
side. пн, 2 нояб. 2020 г. в 23:01, Raymond Wilson : > Just to be clear, the affinity functions we are using convert keys to > partitions, we do not map partitions to nodes and leave that to Ignite. > > On Tue, Nov 3, 2020 at 8:48 AM Alexei Scherbakov < > alexey.scherbak..

Re: Custom Affinity Functions proposed for removal?

2020-11-02 Thread Alexei Scherbakov
p://www.trimble.com/> > Raymond Wilson > Solution Architect, Civil Construction Software Systems (CCSS) > 11 Birmingham Drive | Christchurch, New Zealand > +64-21-2013317 Mobile > raymond_wil...@trimble.com > > < > https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch > > > -- Best regards, Alexei Scherbakov

Re: IEP-52: Binary Delivery & Upgradability Enhancements

2020-09-16 Thread Alexei Scherbakov
ng > > >>>> integrations to that repository and, once this is done, the > extensions > > >>> will > > >>>> be released to Maven separately. Is it reasonable to expand the > scope > > of > > >>>> the IEP-52 and contemplate on how to install those extensions? > > >>>> > > >>>> - > > >>>> Denis > > >>>> > > >>>> > > >>>> On Thu, Aug 27, 2020 at 3:40 PM Valentin Kulichenko < > > >>>> valentin.kuliche...@gmail.com> wrote: > > >>>> > > >>>>> Hi Petr, > > >>>>> > > >>>>> The proposal makes sense to me - thanks for starting the > discussion. > > >>>>> Current installation and upgrade procedures involve a lot of manual > > >>> steps > > >>>>> and quite error-prone, we need to automate and simplify them as > much > > as > > >>>>> possible. > > >>>>> > > >>>>> I believe we eventually should end up with a unified command-line > > tool > > >>>>> (ignitectl?) that will incorporate all the operations > (enable/disable > > >>>>> modules, start/stop, configuration updates, activation, baseline, > > etc.). > > >>>>> This IEP is a step in this direction. > > >>>>> > > >>>>> Looking forward to testing a prototype :) > > >>>>> > > >>>>> -Val > > >>>>> > > >>>>> On Thu, Aug 27, 2020 at 2:17 AM Petr Ivanov > > >>> wrote: > > >>>>> > > >>>>>> Hi, Igniters! > > >>>>>> > > >>>>>> > > >>>>>> For Apache Ignite 3.0 I would like to propose vision of enhanced > > >>> delivery > > >>>>>> and upgrade processes [1]. > > >>>>>> The key idea is to make main binary as slim as possible > (delivering > > >>> every > > >>>>>> optional component by demand only) while providing way to run each > > new > > >>>>>> version seamlessly without much of the efforts migrating data and > > >>>>>> configuration between upgrades. > > >>>>>> > > >>>>>> I plan to create prototype based on current release of Apache > Ignite > > >>>>>> (2.8.1 or 2.9.0 if it will be released soon) later in September. > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> [1] > > >>>>>> > > >>>>> > > >>> > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158873958 > > >>>>>> > > >>>>>> > > >>>>> > > >>> > > >>> > > > > > > > > -- Best regards, Alexei Scherbakov

Re: Getting rid of NONE cache rebalance mode

2020-07-22 Thread Alexei Scherbakov
As a reminder - we already have a ticket for a deprecation of rebalanceDelay as well [1] [1] https://issues.apache.org/jira/browse/IGNITE-12662 ср, 22 июл. 2020 г. в 09:39, Alexei Scherbakov : > Ivan, > My opinion the ASYNC rebalancing is a best approach for off-loading 3-d > pa

Re: [MTCGA]: new failures in builds [5479110] needs to be handled

2020-07-22 Thread Alexei Scherbakov
Regards, > Apache Ignite TeamCity Bot > https://github.com/apache/ignite-teamcity-bot > Notification generated at 02:14:19 22-07-2020 > -- Best regards, Alexei Scherbakov

Re: Getting rid of NONE cache rebalance mode

2020-07-22 Thread Alexei Scherbakov
onfuses and creates issues for > >> > >> users > >> > >> [2]. > >> > >> > >> > >> What about deprecating it in one of the next releases and even > >> ignoring > >> > >> this constant in further releases, interpreting it as ASYNC, before > >> > >> Ignite > >> > >> 3.0? I find it hard to believe that any Ignite user actually has > >> > >> RebalanceMode.NONE set in their configuration due to its absolutely > >> > >> unpredictable behavior. > >> > >> > >> > >> Thanks for your thoughts, > >> > >> --AG > >> > >> > >> > >> [1] > >> > >> > >> > >> > >> > > >> > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0+Wishlist > >> > >> [2] > >> > >> > >> > >> > >> > > >> > http://apache-ignite-developers.2346864.n4.nabble.com/About-Rebalance-Mode-SYNC-amp-NONE-td47279.html > >> > >> > >> > > > >> > > >> > > >> > -- > >> > > >> > Best regards, > >> > Ivan Pavlukhin > >> > > >> > > > > > -- > > Best regards, > Ivan Pavlukhin > -- Best regards, Alexei Scherbakov

Re: [MTCGA]: new failures in builds [5479103] needs to be handled

2020-07-21 Thread Alexei Scherbakov
Regards, > Apache Ignite TeamCity Bot > https://github.com/apache/ignite-teamcity-bot > Notification generated at 06:29:19 22-07-2020 > -- Best regards, Alexei Scherbakov

Re: Choosing historical rebalance heuristics

2020-07-17 Thread Alexei Scherbakov
r implementation is need to estimate a size of data which > > >> will be > > >> > transferred owe network. In other word if need to rebalance a part > of > > >> WAL > > >> > that contains N updates, for recover a partition on another node, > > which > > >> > have to contain M rows at all, need chooses a historical rebalance > on > > >> the > > >> > case where N < M (WAL history should be presented as well). > > >> > > > >> > This approach is easy implemented, because a coordinator node has > the > > >> size > > >> > of partitions and counters' interval. But in this case cluster still > > can > > >> > find not many updates in too long WAL history. I assume a > possibility > > to > > >> > work around it, if rebalance historical iterator will not handle > > >> > checkpoints where not contains updates of particular cache. > > Checkpoints > > >> can > > >> > skip if counters for the cache (maybe even a specific partitions) > was > > >> not > > >> > changed between it and next one. > > >> > > > >> > Ticket for improvement rebalance historical iterator[2] > > >> > > > >> > I want to hear a view of community on the thought above. > > >> > Maybe anyone has another opinion? > > >> > > > >> > [1]: https://issues.apache.org/jira/browse/IGNITE-13253 > > >> > [2]: https://issues.apache.org/jira/browse/IGNITE-13254 > > >> > > > >> > -- > > >> > Vladislav Pyatkov > > >> > > > >> > > > > > > > > > -- > > > Vladislav Pyatkov > > > > > > > > > -- > > Vladislav Pyatkov > > > -- Best regards, Alexei Scherbakov

Re: Extended logging for rebalance performance analysis

2020-06-23 Thread Alexei Scherbakov
eId=2, [supplier=94a3fcbc-18d5-4c64-b0ab-4313aba1, > partitions=5, entries=100, duration=12ms, bytesRcvd=5M], > > [supplier=94a3fcbc-18d5-4c64-b0ab-4313aba2, partitions=5, > entries=100, duration=12ms, bytesRcvd=5M]] > > > > I can add "rebalanceId" to ea

Re: [MTCGA]: new failures in builds [5395772] needs to be handled

2020-06-22 Thread Alexei Scherbakov
ntribute > - Should you have any questions please contact > dev@ignite.apache.org > > Best Regards, > Apache Ignite TeamCity Bot > https://github.com/apache/ignite-teamcity-bot > Notification generated at 05:20:15 22-06-2020 > -- Best regards, Alexei Scherbakov

Re: [DISCUSS] Add flag methods to ClusterState enum

2020-06-10 Thread Alexei Scherbakov
But it looks like we do not need methods *readOnly *and *inactive*. What is the point in adding them ? ср, 10 июн. 2020 г. в 21:05, Alexei Scherbakov : > Sergey Antonov, > > The proposal looks good to me. > Use of org.apache.ignite.cluster.ClusterState#active adds a > boilerpl

Re: [DISCUSS] Add flag methods to ClusterState enum

2020-06-10 Thread Alexei Scherbakov
; > I'm going to do that on the ticket [1]. > > > > Any objections? > > > > [1] https://issues.apache.org/jira/browse/IGNITE-13144 > > -- > > BR, Sergey Antonov > > > -- Best regards, Alexei Scherbakov

Re: Various shutdown guaranties

2020-06-09 Thread Alexei Scherbakov
MEDIATE|GRACEFUL > > > > Looks better that DEFAULT and WAIT_FOR_BACKUP. > > > > I general I consider job cancellation need to added in these policies' > > enumeration. > > But we can do it in the future. > > > > > > > > -- > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > > > -- Best regards, Alexei Scherbakov

Re: Various shutdown guaranties

2020-06-09 Thread Alexei Scherbakov
overriding by calling > > ignite.cluster().setShutdownPolicy(USE_STATIC_CONFIGURATION). After that, > > behavior will be resolved based in IgniteConfiguration#shutdownPolicy > again. > > If we agree on this mechanism, I propose to use IMMEDIATE name instead of > &

Re: Various shutdown guaranties

2020-06-09 Thread Alexei Scherbakov
ALL > > }/ > > > > Method close without parameter Ignite.close() will get shutdown behavior > > configured for cluster wide. It will be implemented through distributed > > meta > > storage and additional utilities for configuration. > > Also, will be added a method to configure shutdown on start, this is look > > as > > IgniteConfiguration.setShutdown(Shutdown). > > If shutting down did not configure all be worked as before according to > > IMMEDIATE behavior. > > All other close method will be marked as deprecated. > > > > I will be waiting for your opinions. > > > > > > > > -- > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > > > -- Best regards, Alexei Scherbakov

Re: Various shutdown guaranties

2020-06-08 Thread Alexei Scherbakov
Graceful policy should only be applicable to caches having a number of backups > 0. пн, 8 июн. 2020 г. в 14:54, Alexei Scherbakov : > V.Pyatkov > > > While I agree we need a way to prevent unintentional data loss on > shutdown, I do not like the proposed shutdown flags enum. &

Re: Various shutdown guaranties

2020-06-08 Thread Alexei Scherbakov
rked as deprecated. > > I will be waiting for your opinions. > > > > -- > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > -- Best regards, Alexei Scherbakov

Re: Extended logging for rebalance performance analysis

2020-05-26 Thread Alexei Scherbakov
, bytesRcvd=5M]] > > I can add "rebalanceId" to each message that you gave at above. > > A detailed message will help us understand how correctly the suppliers > were selected. > > 06.05.2020, 22:08, "Alexei Scherbakov" : > > Hello. > > >

Re: [DISCUSS] Best way to re-encrypt existing data (TDE cache key rotation).

2020-05-25 Thread Alexei Scherbakov
ption and online re-encryption using rebalancing as a first step. Next step can be online in-place re-encryption. It's important to measure business impact from it on online grid. > > > > 25 мая 2020 г., в 11:46, Alexei Scherbakov > написал(а): > > >

Re: [DISCUSS] Best way to re-encrypt existing data (TDE cache key rotation).

2020-05-25 Thread Alexei Scherbakov
ommand that will reencrypt all data without > starting node > 3. Start node. > > Pavel, can you, please, write it one more time - what disadvantages in > offline procedure? > > > 25 мая 2020 г., в 11:20, Alexei Scherbakov > написал(а): > > > > And definitely thi

Re: [DISCUSS] Best way to re-encrypt existing data (TDE cache key rotation).

2020-05-25 Thread Alexei Scherbakov
And definitely this approach is much simplier to implement because all corner cases are handled by rebalancing code. пн, 25 мая 2020 г. в 11:16, Alexei Scherbakov : > I mean: serving supply requests. > > пн, 25 мая 2020 г. в 11:15, Alexei Scherbakov < > alexey.scherb

Re: [DISCUSS] Best way to re-encrypt existing data (TDE cache key rotation).

2020-05-25 Thread Alexei Scherbakov
I mean: serving supply requests. пн, 25 мая 2020 г. в 11:15, Alexei Scherbakov : > Nikolay, > > Can you explain why such restriction is necessary ? > Most likely having a currently re-encrypting node serving only demand > requests will have least preformance impact on a grid. >

Re: [DISCUSS] Best way to re-encrypt existing data (TDE cache key rotation).

2020-05-25 Thread Alexei Scherbakov
s feature without nodes restart. > In the ideal scenario all nodes will stay alive and respond to the user > requests. > > > 24 мая 2020 г., в 15:24, Alexei Scherbakov > написал(а): > > > > Pavel Pereslegin, > > > > I see another opportunity. > > We c

Re: [DISCUSS] Best way to re-encrypt existing data (TDE cache key rotation).

2020-05-24 Thread Alexei Scherbakov
should work faster than "in place" re-encryption. > > > > > cons: > > > > > - re-encryption in active cluster (and on unstable topology) can > be > > > > > difficult to implement. > > > > > > > > > > (See more detail

Re: [MTCGA]: new failures in builds [5329193] needs to be handled

2020-05-24 Thread Alexei Scherbakov
tribute > - Should you have any questions please contact > dev@ignite.apache.org > > Best Regards, > Apache Ignite TeamCity Bot > https://github.com/apache/ignite-teamcity-bot > Notification generated at 06:46:34 23-05-2020 > -- Best regards, Alexei Scherbakov

Re: [DISCUSS] Data loss handling improvements

2020-05-07 Thread Alexei Scherbakov
we should make a decision for every lost partition before calling the > reset. > > On Wed, May 6, 2020 at 8:02 PM Alexei Scherbakov < > alexey.scherbak...@gmail.com> wrote: > > > ср, 6 мая 2020 г. в 12:54, Anton Vinogradov : > > > > > Alexei, &

Re: Extended logging for rebalance performance analysis

2020-05-06 Thread Alexei Scherbakov
ses: pr - > > > primary, bu - backup, su - supplier node, h - historical, nodeId > mapping > > > (nodeId=id,consistentId) [0=rebalancing.RebalanceStatisticsTest0] > > > [1=rebalancing.RebalanceStatisticsTest2] > > > [2=rebalancing.RebalanceStatisticsTest1] > > > > > > Interrupted rebalance of group cache. > > > Rebalance information per cache group (interrupted rebalance): > > > [id=644280849, name=default2, startTime=2020-04-13 14:55:24,969, > > > finishTime=2020-04-13 14:55:24,969, d=0 ms, restarted=0] > > > > > > Total full and historical rebalance for all cache groups. > > > Rebalance total information (including successful and not rebalances): > > > [startTime=2020-04-13 10:55:18,726, finishTime=2020-04-13 10:55:18,780, > > > d=54 ms] Supplier statistics: [nodeId=0, p=60, e=250, b=25000, d=54 ms] > > > [nodeId=1, p=60, e=250, b=24945, d=54 ms] Aliases: p - partitions, e - > > > entries, b - bytes, d - duration, h - historical, nodeId mapping > > > (nodeId=id,consistentId) [0=rebalancing.RebalanceStatisticsTest1] > > > [1=rebalancing.RebalanceStatisticsTest0] > > > Rebalance total information (including successful and not rebalances): > > > [startTime=2020-04-13 15:01:43,822, finishTime=2020-04-13 15:01:44,116, > > > d=294 ms] Supplier statistics: [nodeId=0, hp=20, he=500, hb=50445, > d=294 > > > ms] Aliases: p - partitions, e - entries, b - bytes, d - duration, h - > > > historical, nodeId mapping (nodeId=id,consistentId) > > > [0=rebalancing.RebalanceStatisticsTest0] > > > > > > [1] - https://issues.apache.org/jira/browse/IGNITE-12080 > -- Best regards, Alexei Scherbakov

Re: [DISCUSS] Data loss handling improvements

2020-05-06 Thread Alexei Scherbakov
ed, data is correct. > Will it be possible to perform operations on non-lost partitions when the > cluster has at least one lost partition? > Yes it will be. > > On Wed, May 6, 2020 at 11:45 AM Alexei Scherbakov < > alexey.scherbak...@gmail.com> wrote: > > >

[DISCUSS] Data loss handling improvements

2020-05-06 Thread Alexei Scherbakov
Folks, I've almost finished a patch bringing some improvements to the data loss handling code, and I wish to discuss proposed changes with the community before submitting. *The issue* During the grid's lifetime, it's possible to get into a situation when some data nodes have failed or

Re: About Rebalance Mode (SYNC & NONE)

2020-05-03 Thread Alexei Scherbakov
the > value of this parameter design? Is this a legacy of history that is > about to be abandoned? > > -- Best regards, Alexei Scherbakov

Re: [DISCUSSION] Major changes in Ignite in 2020

2020-04-11 Thread Alexei Scherbakov
> > and maybe Rust support. > > > > I'm NOT sure of 'AFAIK' ? > > > > We may need to implement your Restful API to provide support for Golang > and > > Rust, provided it's complete? > > > > Thanks, > > Steve > > > > > > > > > > > > -- > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > > > -- Best regards, Alexei Scherbakov

Re: Security Subject of thin client on remote nodes

2020-03-24 Thread Alexei Scherbakov
gt; > wrote: > > > > > Hi Alexei, Denis, > > > > > > One of the main usecases of thin client authentication is to be able to > > > audit the changes done using the thin client user. > > > To enable that : > > > We really need to resolve

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

2020-03-24 Thread Alexei Scherbakov
t;>> As fix for user cases - yes. My idea is to emphasize overall ability > to > >> lose various objects, not only data. Probably might be reconsidered in > >> future. > >>>> > >>>> > >>>> 17.03.2020 13:49, Nikolay Izhikov пишет: > >>>>> Hello, Vladimir. > >>>>> > >>>>> If there is at lease one persistent data region then system data > >> region also becomes persistent. > >>>>> Your example applies only to pure in-memory clusters. > >>>>> > >>>>> And should be covered with the —force parameter we added. > >>>>> > >>>>> What do you think? > >>>>> > >>>>>> 17 марта 2020 г., в 13:45, Vladimir Steshin > >> написал(а): > >>>>>> > >>>>>> Hi, all. > >>>>>> > >>>>>> Fixes for control.sh and the REST have been merged. Could anyone > take > >> a look to the previous email with an issue? Isn't this conductvery > wierd? > >>>>>> > >> > > -- Best regards, Alexei Scherbakov

Re: Security Subject of thin client on remote nodes

2020-03-18 Thread Alexei Scherbakov
Ignite users don't use thin clients. > > Summary: > I suggest adding a new method to GridSecurityProcessor because > it has a clear contract and enforces compatibility check natural way. > > вс, 15 мар. 2020 г. в 17:13, Alexei Scherbakov < > alexey.scherbak...@gmail.

  1   2   3   4   >