Re: [DISCUSSION] Reject join of nodes with different character encodings

2021-12-13 Thread Ivan Pavlukhin
Do encodings in question somehow influence on actual stored data (bytes)? If so, using an implicit platform encoding sounds quite dangerous. Moving data between servers (or perhaps even rebalancing) can lead to bad consequences. Anyways, IMHO an implicit encoding is not good, but sensible default

Re: New Ignite Website: Released and Check the Dev Instructions

2021-12-13 Thread Erlan Aytpaev
Hi Igniters, I've added the "Extensions" section on the "Downloads" page. Please review my pull request https://github.com/apache/ignite-website/pull/91. This is my first commit. And I'm going to continue contributing updates on the website. Best Regards, Erlan. On Tue, Dec 14, 2021 at 9:55 AM

Re: Apache Ignite 2.12 RELEASE [Time, Scope, Manager]

2021-12-13 Thread Nikita Amelchev
+1 to move these auth issues to the next release. I will prepare RC at the nearest time. пн, 13 дек. 2021 г. в 16:13, Pavel Tupitsyn : > > Agree with Nikolay, let's proceed with the release. > > On Mon, Dec 13, 2021 at 3:31 PM Nikolay Izhikov wrote: > > > Nikita. > > > > I don’t think

Re: New Ignite Website: Released and Check the Dev Instructions

2021-12-13 Thread Mauricio Stekl
Hi, I noticed the "Extensions" section was not merged correctly into the Downloads page. I have created a ticket https://issues.apache.org/jira/browse/IGNITE-16112, and we'll be solving this as soon as possible. My apologies for the inconvenience Best, Mauricio On Mon, Dec 13, 2021 at 4:46 PM

Re: [DISCUSSION] Reject join of nodes with different character encodings

2021-12-13 Thread Ivan Daschinsky
Unpaited surrogates are emoji symbols. One should be completely insane to use emojis in login. пн, 13 дек. 2021 г., 21:30 Mikhail Petrov : > Ivan, string with unpaired surrogates symbols are serialized and > deserialized by java UTF-8 decoder successfully but the result does not > match the

New Ignite Website: Released and Check the Dev Instructions

2021-12-13 Thread Denis Magda
Igniters, Please welcome the new Ignite website: https://ignite.apache.org Hope you like it! The website uses gulpjs to simplify the dev/maintenance cycles, thus, I would encourage all of you who release Ignite (and update the downloads page) or edit any other pages to check the updated

Re: [DISCUSSION] Reject join of nodes with different character encodings

2021-12-13 Thread Mikhail Petrov
Ivan, string with unpaired surrogates symbols are serialized and deserialized by java UTF-8 decoder successfully but the result does not match the initial string. It may result in that if the user's login contains these symbols, it will be distorted after deserialization and the user will not

Re: [DISCUSSION] Reject join of nodes with different character encodings

2021-12-13 Thread Ivan Pavlukhin
> I guess Nikolay is talking about the problem with UTF-8 in case string > contains unpaired surrogate symbols Folks, give me a clue why it is a problem? Naively it seems to be a good restriction rather than problem. What problems can it cause in practice? 2021-12-13 16:32 GMT+03:00, Ilya

Re: Ignite Website UI and UX improvements

2021-12-13 Thread Mauricio Stekl
Hi, I completed merging the code and the new website is live now. Also updated the wiki instructions to build and test the site locally: https://cwiki.apache.org/confluence/display/IGNITE/Website+Development Please let me know if you have any questions or comments. Best, Mauricio On Mon, Dec

Re: [DISCUSSION] Reject join of nodes with different character encodings

2021-12-13 Thread Ilya Kasnacheev
Hello! We already have a warning about this, see IgniteKernal.checkFileEncoding() Regards, -- Ilya Kasnacheev пн, 13 дек. 2021 г. в 16:26, Ivan Daschinsky : > >> But now multiple components > >> independently serialize strings for their needs and use default encoding > >> for this. > >> For

Re: [DISCUSSION] Reject join of nodes with different character encodings

2021-12-13 Thread Ivan Daschinsky
>> But now multiple components >> independently serialize strings for their needs and use default encoding >> for this. >> For example DirectByteBufferStreamImplV2#writeString, >> MetaStorage#writeRaw and so on We should fix all of them. >> BinaryUtils#utf8BytesToStr Lets use this everywhere.

Re: Re[2]: NUMA aware allocator, PR review request

2021-12-13 Thread Ivan Daschinsky
Thanks all for review, merged into master. пн, 6 дек. 2021 г. в 23:26, Ivan Daschinsky : > You are not wrong, it is built from source, every night. And every TC run. > I don't understand why numa allocator cannot be treated the same. Moreover, > it is built using maven, with maven plugin and

Re: Apache Ignite 2.12 RELEASE [Time, Scope, Manager]

2021-12-13 Thread Pavel Tupitsyn
Agree with Nikolay, let's proceed with the release. On Mon, Dec 13, 2021 at 3:31 PM Nikolay Izhikov wrote: > Nikita. > > I don’t think IGNITE-16068 is a release blocker. > > 1. It only happens when authentication enabled. > 2. There is at lease 2 workarounds: > a. Use the same

Re: 0-day CVE in log4j

2021-12-13 Thread Maxim Muzafarov
Folks, Should we describe all the WA available for the issue [1]? There is already a lot of information about CVE, and nevertheless, it will not be superfluous. [1] https://issues.apache.org/jira/browse/IGNITE-16101 On Mon, 13 Dec 2021 at 15:37, Ivan Daschinsky wrote: > > Unfortunately, we

Re: 0-day CVE in log4j

2021-12-13 Thread Ivan Daschinsky
Unfortunately, we need patch our Log4j2 adapter in order to work with log4j-2.15 So there is no choice other than to release 2.11.1 пн, 13 дек. 2021 г. в 15:21, Anton Vinogradov : > Folks, > > My 200 rubles here, > > I want to include it to the 2.12 scope. > Why not 2.11.1 as well? > We should

Re: 0-day CVE in log4j

2021-12-13 Thread Maxim Muzafarov
+1 for the 2.11.1 On Mon, 13 Dec 2021 at 15:21, Anton Vinogradov wrote: > > Folks, > > My 200 rubles here, > > I want to include it to the 2.12 scope. > Why not 2.11.1 as well? > We should provide a fixed version for current customers asap. > 2.12 require migration, while 2.11.1 can be applied

Re: Ignite Website UI and UX improvements

2021-12-13 Thread Mauricio Stekl
Hi all, I'm happy to let you know the new website is ready, and I will start merging the code and do the launch today. Please avoid committing new changes to the current website until this process is ready. I will share an update once it's live. Thanks! Mauricio On Tue, Nov 9, 2021 at 9:00 PM

Re: Apache Ignite 2.12 RELEASE [Time, Scope, Manager]

2021-12-13 Thread Nikolay Izhikov
Nikita. I don’t think IGNITE-16068 is a release blocker. 1. It only happens when authentication enabled. 2. There is at lease 2 workarounds: a. Use the same file.encoding for all Ignite nodes. b. Use only latin characters in user login. I propose to postpone this ticket and move

Re: Apache Ignite 2.12 RELEASE [Time, Scope, Manager]

2021-12-13 Thread Nikita Amelchev
Hello. I have cherry-picked the blocked [1] to the 2.12. It updates the log4j version to 2.15. The release is blocked by one blocker issue with auth. [1] https://issues.apache.org/jira/browse/IGNITE-16101 [2] https://issues.apache.org/jira/browse/IGNITE-16068 чт, 9 дек. 2021 г. в 12:44, Nikita

Re: 0-day CVE in log4j

2021-12-13 Thread Anton Vinogradov
Folks, My 200 rubles here, > I want to include it to the 2.12 scope. Why not 2.11.1 as well? We should provide a fixed version for current customers asap. 2.12 require migration, while 2.11.1 can be applied as-is. On Mon, Dec 13, 2021 at 12:18 PM Stephen Darlington <

Re: [DISCUSSION] Reject join of nodes with different character encodings

2021-12-13 Thread Nikolay Izhikov
> Does Java String support all unicode characters and particularly does it > support more characters than UTF-8 It’s not about Java, it’s about UTF-8 standard. Please, take a look at [1] > In November 2003, UTF-8 was restricted by RFC 3629 to match the constraints > of the UTF-16 character

Re: [DISCUSSION] Reject join of nodes with different character encodings

2021-12-13 Thread Mikhail Petrov
Ivan Daschinsky, better variant is to enforce all strings to be encoded in UTF-8 I agree that it is possible way to go. But now multiple components independently serialize strings for their needs and use default encoding for this. For example  DirectByteBufferStreamImplV2#writeString,

Re: [DISCUSSION] Reject join of nodes with different character encodings

2021-12-13 Thread Ivan Pavlukhin
> UTF-8 can’t encode all UNICODE characters. Nikolay, could you please elaborate? My understanding is that encoding we speak about matters for conversion from byte arrays to strings. Does Java String support all unicode characters and particularly does it support more characters than UTF-8 (I am

Re: [DISCUSSION] Reserve partitions for CacheQueries

2021-12-13 Thread Ivan Pavlukhin
Thanks for details! 2021-12-13 11:14 GMT+03:00, Maksim Timonin : > Hi, Ivan! > >> Does IndexQuery has separate codebase? Does it share some code with > ScanQuery > > Yes, IndexQuery mostly shares processing with ScanQuery: > requests/responses, a remote filter. Reducer (merge of query results) on

Re: [DISCUSSION] Reject join of nodes with different character encodings

2021-12-13 Thread Ivan Daschinsky
UTF-8 is already a default encoding in our BinaryObject format. So I am for unification. пн, 13 дек. 2021 г. в 12:50, Nikolay Izhikov : > Hello, Ivan. > > UTF-8 can’t encode all UNICODE characters. > > > 13 дек. 2021 г., в 12:49, Ivan Daschinsky > написал(а): > > > > Khm, maybe a better

Re: [DISCUSSION] Reject join of nodes with different character encodings

2021-12-13 Thread Nikolay Izhikov
Hello, Ivan. UTF-8 can’t encode all UNICODE characters. > 13 дек. 2021 г., в 12:49, Ivan Daschinsky написал(а): > > Khm, maybe a better variant is to enforce all strings to be encoded in > UTF-8? > AFAIK multi OS cluster is a quite common case. > > > пн, 13 дек. 2021 г. в 11:36, Mikhail

Re: [DISCUSSION] Reject join of nodes with different character encodings

2021-12-13 Thread Ivan Daschinsky
Khm, maybe a better variant is to enforce all strings to be encoded in UTF-8? AFAIK multi OS cluster is a quite common case. пн, 13 дек. 2021 г. в 11:36, Mikhail Petrov : > Igniters, > > Recently we faced the problem that if the cluster consists of nodes > running in the JVM with different

Re: 0-day CVE in log4j

2021-12-13 Thread Stephen Darlington
Another workaround appears to be using the -Dlog4j2.formatMsgNoLookups=true option. Also, “Java versions greater than 6u211, 7u201, 8u191, and 11.0.1 are less affected by this attack vector, at least in theory, because the JNDI can't load remote code using LDAP.”

[DISCUSSION] Reject join of nodes with different character encodings

2021-12-13 Thread Mikhail Petrov
Igniters, Recently we faced the problem that if the cluster consists of nodes running in the JVM with different encodings, many issues arise. The root cause of the mentioned issues is components that use `String#getBytes()` and `new String()`, which relies on the system default encoding.

Re: [DISCUSSION] Reserve partitions for CacheQueries

2021-12-13 Thread Maksim Timonin
Hi, Ivan! > Does IndexQuery has separate codebase? Does it share some code with ScanQuery Yes, IndexQuery mostly shares processing with ScanQuery: requests/responses, a remote filter. Reducer (merge of query results) on an initiator node has its own implementation for IndexQuery (MergeSort), but

Re: [DISCUSSION] Reserve partitions for CacheQueries

2021-12-13 Thread Ivan Pavlukhin
> Then, if there are no objections, the short-term plan is: Sounds ok for me. > AFAIR Scan and SQL queries implementations are totally different. Could you > tell me how Index queries fit there? I suppose my questions was misleading. Actually I would like to know how code is organized today.