Re: List of running Continuous queries or CacheEntryListener per cache or node
Hello, Dmitry. I have some experience with ContinuousQuery codebase so ca take care of this ticket. Can you provide it ID? В Вт, 26/12/2017 в 18:10 +0300, Dmitry Karachentsev пишет: > Hi Nikolay, > > I think it may be useful too. Will try to describe possible API in a > ticket. > > Thanks! > -Dmitry > > 21.12.2017 13:18, Nikolay Izhikov пишет: > > Hello, Dmitry. > > > > I think it a great idea. > > > > Do we a feature to list all running ComputeTasks? > > > > I, personally, think we have to implement possibility to track all > > user-provided tasks - CacheListener, ContinuousQuery, ComputeTasks, > > etc. > > > > В Чт, 21/12/2017 в 10:13 +0300, Dmitry Karachentsev пишет: > > > Crossposting to devlist. > > > > > > Hi Igniters! > > > > > > It's might be a nice feature to have - get list of registered > > > continuous > > > queries with ability to deregister them. > > > > > > What do you think? > > > > > > Thanks! > > > -Dmitry > > > > > > 20.12.2017 16:59, fefe пишет: > > > > For sanity checks or tests. I want to be sure that I haven't > > > > forgot > > > > to > > > > deregister any listener. > > > > > > > > Its also very important metric to see how many continuous > > > > queries/listeners > > > > are currently running. > > > > > > > > > > > > > > > > -- > > > > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > >
[GitHub] ignite pull request #3307: IGNITE-7323: Update Storm dependencies to 1.1.1.
GitHub user shroman opened a pull request: https://github.com/apache/ignite/pull/3307 IGNITE-7323: Update Storm dependencies to 1.1.1. You can merge this pull request into a Git repository by running: $ git pull https://github.com/shroman/ignite IGNITE-7323 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3307.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3307 commit 47e04536650cd4fd6eeaf5cd4c718f1ba89fac87 Author: shromanDate: 2017-12-28T03:52:34Z IGNITE-7323: Update Storm dependencies to 1.1.1. ---
[jira] [Created] (IGNITE-7323) Up
Roman Shtykh created IGNITE-7323: Summary: Up Key: IGNITE-7323 URL: https://issues.apache.org/jira/browse/IGNITE-7323 Project: Ignite Issue Type: Bug Reporter: Roman Shtykh -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Thin Client documentation: OP_QUERY_SQL_CURSOR_GET_PAGE
Pavel, I have 2 records in my cache, and when I run an example for OP_QUERY_SQL=2002 with cursor page size 1, I get the following result (expected) len: 83 resReqId: 1 status code: 0 cursorId: 1 rowCount: 1 moreResults: true Then, I run the OP_QUERY_SQL_CURSOR_GET_PAGE = 2003, using the following code: DataOutputStream out = new DataOutputStream(socket.getOutputStream()); // Message length writeIntLittleEndian(18, out); // Op code = QUERY_SQL_CURSOR_GET_PAGE writeShortLittleEndian(2003, out); // Request id (can be anything) long reqId = 1; writeLongLittleEndian(reqId, out); // Cursor Id writeLongLittleEndian(1, out); // Send request out.flush(); // Read result DataInputStream in = new DataInputStream(socket.getInputStream()); // Response length final int len = readIntLittleEndian(in); System.out.println("len: " + len); // Request id long resReqId = readLongLittleEndian(in); System.out.println("resReqId: " + resReqId); // Success int statusCode = readIntLittleEndian(in); System.out.println("status code: " + statusCode); int rowCount = readIntLittleEndian(in); System.out.println("rowCount: " + rowCount); // Read entries (as user objects) for (int i = 0; i < rowCount; i++){ // ... } boolean moreResults = readBooleanLittleEndian(in); System.out.println("moreResults: " + moreResults); but the result is not what is expected: len: 1 resReqId: 44096429228032 status code: 721512192 rowCount: 1672392448 moreResults: true Also, there is no error on server side. Anything wrong in the OP_QUERY_SQL_CURSOR_GET_PAGE request that I am sending? -Prachi
Re: Request for contributors permissions
Hello Jason, Granted you the required permissions. Thanks for the contribution! Igor, could you validate the changes? — Denis > On Dec 26, 2017, at 5:53 PM, Jason Manwrote: > > Hello, > > Please grant me access to assign myself Ignite issues. I want to fix one > of Ignite's issue IGNITE-6853, which I have a PR ready for review. > > Thanks, > Jason
Re: Handling slashes in cache names
On Wed, Dec 27, 2017 at 8:05 AM, Pavel Tupitsynwrote: > Yep, base64 is just an example. > We need some kind of urlencode, but tailored for file names, so that > names remain readable. > > To avoid uppercase/lowercase collisions on Windows, we can restrict allowed > characters to lowercase English letters and numbers, - and _, and escape > everything > else in some way. > I think that we should allow users to specify any case they like, but internally we should always convert to upper or lower case, whichever one we choose.
Example for OP_PUT_BINARY_TYPE
Pavel, I am trying to come up with an example for OP_PUT_BINARY_TYPE = 3003. However, few things are not clear to me from the wiki doc - 1. In BinaryField, What is Field id? 2. If there is no enum, what should I put for 'Enum values'? 3. What is Schema? -P
Re: Handling slashes in cache names
Igniters Use cache name for file and directory names on a file system is bad idea. In that case we should keep in mind many limitiations vary FS. Why do not use mapping cache name to an identifier tolerated to FS lacks? On Wed, Dec 27, 2017 at 7:05 PM, Pavel Tupitsynwrote: > Yep, base64 is just an example. > We need some kind of urlencode, but tailored for file names, so that > names remain readable. > > To avoid uppercase/lowercase collisions on Windows, we can restrict allowed > characters > to lowercase English letters and numbers, - and _, and escape everything > else in some way. > > On Wed, Dec 27, 2017 at 5:36 PM, Dmitriy Setrakyan > wrote: > > > On Wed, Dec 27, 2017 at 6:25 AM, Vladimir Ozerov > > wrote: > > > > > Having different policies for persistent and non-persistent caches > sounds > > > like a bad idea for me, because there could be troubles should user try > > to > > > switch to persistent mode. It would require code changes. > > > > > > Can we just escape all non-latin symbols (e.g. using base64), while > > leaving > > > the rest as is? With this approach in most cases cache name will remain > > the > > > same, and only multibyte characters would be affected. > > > > > > > Agree, if we can keep cache names in human readable form. Would be nice > to > > see some examples. > > > -- Sergey Kozlov GridGain Systems www.gridgain.com
Re: IGNITE-7135 needs review
Ticket was assigned to me until the patch available state has been reached. Till now no one committer take responsibility to review and merge the PR. 2017-12-26 21:08 GMT+07:00 Dmitriy Setrakyan: > Here is the link to the ticket: > https://issues.apache.org/jira/browse/IGNITE-7135 > > For some odd reason, the ticket is in unassigned state. Alexander, > shouldn't it be assigned to you? > > D. > > On Mon, Dec 25, 2017 at 11:58 PM, Alexandr Kuramshin > > wrote: > > > Hello community! > > > > I've implemented IGNITE-7135 doing two improvements: > > > > 1) control remote node startup (successful or not) through > > IgniteCluster.startNodes(); > > > > 2) keep the first Java principle working "Compile once, run everywhere" - > > from now running remotely on Windows also supported. > > > > Committers, please review. > > > > -- > > Thanks, > > Alexandr Kuramshin > > > -- Thanks, Alexandr Kuramshin
[jira] [Created] (IGNITE-7322) Distributed MLP training
Artem Malykh created IGNITE-7322: Summary: Distributed MLP training Key: IGNITE-7322 URL: https://issues.apache.org/jira/browse/IGNITE-7322 Project: Ignite Issue Type: Sub-task Affects Versions: 2.4 Reporter: Artem Malykh Assignee: Artem Malykh -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] ignite pull request #3306: ignite-6736 Unsafe.monitorEnter/Exit usages are r...
GitHub user agura opened a pull request: https://github.com/apache/ignite/pull/3306 ignite-6736 Unsafe.monitorEnter/Exit usages are removed You can merge this pull request into a Git repository by running: $ git pull https://github.com/agura/incubator-ignite ignite-2.4-java8 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3306.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3306 commit ff93f145a3589537064065134d124bf384b03558 Author: Andrey GuraDate: 2017-12-27T16:14:22Z ignite-6736 Unsafe.monitorEnter/Exit usages are removed ---
Re: Handling slashes in cache names
Yep, base64 is just an example. We need some kind of urlencode, but tailored for file names, so that names remain readable. To avoid uppercase/lowercase collisions on Windows, we can restrict allowed characters to lowercase English letters and numbers, - and _, and escape everything else in some way. On Wed, Dec 27, 2017 at 5:36 PM, Dmitriy Setrakyanwrote: > On Wed, Dec 27, 2017 at 6:25 AM, Vladimir Ozerov > wrote: > > > Having different policies for persistent and non-persistent caches sounds > > like a bad idea for me, because there could be troubles should user try > to > > switch to persistent mode. It would require code changes. > > > > Can we just escape all non-latin symbols (e.g. using base64), while > leaving > > the rest as is? With this approach in most cases cache name will remain > the > > same, and only multibyte characters would be affected. > > > > Agree, if we can keep cache names in human readable form. Would be nice to > see some examples. >
[GitHub] ignite pull request #3305: ignite-6745 URLClassLoader.getURLs() usages are r...
GitHub user agura opened a pull request: https://github.com/apache/ignite/pull/3305 ignite-6745 URLClassLoader.getURLs() usages are removed You can merge this pull request into a Git repository by running: $ git pull https://github.com/agura/incubator-ignite ignite-6745 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3305.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3305 commit a3098d93a9fee3bae27ca122a627706d5b39508c Author: Andrey GuraDate: 2017-12-27T15:50:12Z ignite-6745 URLClassLoader.getURLs() usages are removed ---
[GitHub] ignite pull request #3304: IGNITE-7259
GitHub user alexpaschenko opened a pull request: https://github.com/apache/ignite/pull/3304 IGNITE-7259 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-7259 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3304.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3304 commit f6e982540e65ab17d439dba990794f35616a30dd Author: sboikovDate: 2017-08-30T09:45:40Z ignite-3478 commit 275a85db5cd6923b36126166ae99b15e876192be Author: sboikov Date: 2017-08-31T07:44:07Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit b7b9089f0102b8cab9942a9c887d93e9f26cc7d2 Author: sboikov Date: 2017-08-31T09:00:36Z disco cache cleanup commit 855c2d45794c300d41e386b4e6fa40736cc3e40d Author: sboikov Date: 2017-08-31T09:09:58Z Merge branch 'ignite-3478-1' into ignite-3478 commit 08be7310a93d3ce455215b97cf8ab1a2c3f0ab31 Author: sboikov Date: 2017-08-31T09:52:23Z ignite-3478 commit fce2e31f0fd2f4f6a9944422e40408a0c65cfe90 Author: sboikov Date: 2017-09-04T08:13:50Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit d3c049952384750c5543a9f88b383c033ef74096 Author: sboikov Date: 2017-09-04T08:52:11Z ignite-3478 commit e71ce1937a18dd32448e92b1038dc48d4cb6f8ab Author: sboikov Date: 2017-09-04T10:16:03Z ignite-3478 commit 5fac5b0965e97f8951e16e10ca9229a2e78ddb0c Author: sboikov Date: 2017-09-05T10:16:44Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java commit 2e0c9c08e046e8d6af1b5358d9053eae999b1fb4 Author: sboikov Date: 2017-09-05T11:30:55Z ignite-3478 commit e1e07ffdf2d711ba3e72f316f5a3970eff27372e Author: sboikov Date: 2017-09-05T11:31:14Z ignite-3478 commit cbada3934a386668da0b11d4de7d0f58a4d04dfe Author: sboikov Date: 2017-09-05T11:49:49Z ignite-3484 commit 5a82c68dcd1927bb6fded8b7def38c91ff6e145b Author: sboikov Date: 2017-09-05T11:59:49Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit bc9134c94b7a738dc1664e96ca6eabb059f1c268 Author: sboikov Date: 2017-09-05T12:03:39Z Merge branch 'ignite-3478' into ignite-3484 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/tree/AbstractDataInnerIO.java commit b4bfcde78825c6517232e49d389bdb5de19f05a9 Author: sboikov Date: 2017-09-05T12:27:51Z ignite-3484 commit 43834aaab9e2c3cd5fdd55289fdc4a9ff8ab6599 Author: sboikov Date: 2017-09-05T13:13:00Z ignite-3478 commit d1b828095713fcadfa260cf94fef01b42a1b12fd Author: sboikov Date: 2017-09-05T13:13:33Z Merge branch 'ignite-3478' into ignite-3484 commit 6be6779b6336c36cd71eef0a25199a6a877ce6b5 Author: sboikov Date: 2017-09-05T13:47:11Z ignite-3484 commit e3bba83256c1eb53c4b40fbd9ddba47fcf9d58d5 Author: sboikov Date: 2017-09-06T07:10:26Z ignite-3484 commit dd0afb28466094b801506da8afa3601bfaebd853 Author: sboikov Date: 2017-09-06T07:30:04Z ignite-3484 commit 27b87b413348b03986a463551db24b7726321732 Author: sboikov Date: 2017-09-06T08:19:18Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java commit dcaf8801accd6ee089849a82b2ccd558aec81895 Author: sboikov Date: 2017-09-06T08:19:30Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java commit c966451d0bf7059575de92bcfae43d72096ebce4 Author: sboikov Date: 2017-09-06T08:27:04Z Merge branch 'ignite-3478' into ignite-3484 commit 91b9911731a387a3199ddbbc22704bc14af09995 Author: sboikov Date: 2017-09-06T09:22:22Z ignite-3484 commit e40b4d9dcd6fe6c1cd2640bdd7116ca5a08ed781 Author: sboikov Date: 2017-09-07T09:12:32Z ignite-3484 commit 41a1c571e6ba1765941e2f1679dc4ac1582275c4 Author: sboikov Date: 2017-09-08T07:55:24Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3484 commit 1626130801dc330bcaf693f46906f6564cce6802 Author: sboikov Date:
[GitHub] ignite pull request #3303: IGNITE-7205: Dataset API
GitHub user zaleslaw opened a pull request: https://github.com/apache/ignite/pull/3303 IGNITE-7205: Dataset API 1. Changed Labeled Dataset 2. Added Dataset and DatasetRow classes 3. Moved a few methods to Normalizer and LabeledDatasetLoader 4. Changed examples and tests 5. Kept LabeledDataset API (minor changes in signatures) 6. Added prototype class for FeatureMetadata (could be extended in future) You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-7205 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3303.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3303 commit 7d22bcd33fcea3a7588c46368b5289d5b306db7f Author: Zinoviev AlexeyDate: 2017-12-07T11:01:43Z Merge from master commit 84cbd8b380772ce773790b18066c86b663b1af27 Author: Zinoviev Alexey Date: 2017-12-13T10:36:56Z Merge branch 'master' of https://github.com/apache/ignite commit b58db6a44be3c625174c1d89c577240771d1bfb4 Author: Zinoviev Alexey Date: 2017-12-26T16:37:24Z Merged from master commit c050cf6e9ad052ca2d5d15b035da9536eab5ba80 Author: Zinoviev Alexey Date: 2017-12-26T19:06:24Z IGNITE-7205: Added draft commit 6506cf1d0e3bdf50125004494d075d81b863a0a2 Author: Zinoviev Alexey Date: 2017-12-27T08:06:30Z IGNITE-7205: Fixed classes commit 427743e0ad8ab1cbbc2a45d3e2b032a6b66c5632 Author: Zinoviev Alexey Date: 2017-12-27T14:19:10Z IGNITE-7205: Refactored Labeled Dataset commit 1f4792fe2a4cfddbee5390b4e79d74f1cf74585e Author: Zinoviev Alexey Date: 2017-12-27T14:57:49Z IGNITE-7205: Fixed serialization issue ---
[GitHub] ignite pull request #3302: IGNITE-7312: Made use of java.util.Base64 for bas...
GitHub user andrey-kuznetsov opened a pull request: https://github.com/apache/ignite/pull/3302 IGNITE-7312: Made use of java.util.Base64 for base64 encoding. You can merge this pull request into a Git repository by running: $ git pull https://github.com/andrey-kuznetsov/ignite ignite-7312 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3302.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3302 commit 870c6e6b6b76e9b3a15bb8ea006361b18b537edf Author: Andrey KuznetsovDate: 2017-12-27T14:54:01Z IGNITE-7312: Made use of java.util.Base64 for base64 encoding. ---
Re: Handling slashes in cache names
On Wed, Dec 27, 2017 at 6:25 AM, Vladimir Ozerovwrote: > Having different policies for persistent and non-persistent caches sounds > like a bad idea for me, because there could be troubles should user try to > switch to persistent mode. It would require code changes. > > Can we just escape all non-latin symbols (e.g. using base64), while leaving > the rest as is? With this approach in most cases cache name will remain the > same, and only multibyte characters would be affected. > Agree, if we can keep cache names in human readable form. Would be nice to see some examples.
Re: Handling slashes in cache names
Having different policies for persistent and non-persistent caches sounds like a bad idea for me, because there could be troubles should user try to switch to persistent mode. It would require code changes. Can we just escape all non-latin symbols (e.g. using base64), while leaving the rest as is? With this approach in most cases cache name will remain the same, and only multibyte characters would be affected. On Wed, Dec 27, 2017 at 5:15 PM, Dmitriy Setrakyanwrote: > On Wed, Dec 27, 2017 at 3:42 AM, Pavel Tupitsyn > wrote: > > > Agree with Stan and Vladimir. > > We should not impose any restrictions on cache names, some users may have > > issues with that. > > > > Using cache names as file names is internal implementation detail. > > We can use cache id or some kind of encoding (base64, etc) to avoid file > > system issues. > > > > > Pavel, I disagree. I want to look at the file system and be able to clearly > tell which folder belongs to which cache. If you use encryption or some > other encoding, this would be impossible. > > I doubt that introducing cache name validation for *persistent* caches > would affect any existing users. It sounds like for non-persistent caches > the validation is not needed, right? > > D. >
Re: Handling slashes in cache names
On Wed, Dec 27, 2017 at 3:42 AM, Pavel Tupitsynwrote: > Agree with Stan and Vladimir. > We should not impose any restrictions on cache names, some users may have > issues with that. > > Using cache names as file names is internal implementation detail. > We can use cache id or some kind of encoding (base64, etc) to avoid file > system issues. > > Pavel, I disagree. I want to look at the file system and be able to clearly tell which folder belongs to which cache. If you use encryption or some other encoding, this would be impossible. I doubt that introducing cache name validation for *persistent* caches would affect any existing users. It sounds like for non-persistent caches the validation is not needed, right? D.
[GitHub] ignite pull request #3301: Ignite 7217
GitHub user slukyano opened a pull request: https://github.com/apache/ignite/pull/3301 Ignite 7217 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-7217 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3301.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3301 commit 127be326b76916d6afeb70d4c35a001d1e18a7a6 Author: Stanislav LukyanovDate: 2017-12-20T15:38:25Z IGNITE-7217: added logging for custom thread pools commit a21731651994e7267e328a3dcd8be2830ec1c77c Author: Stanislav Lukyanov Date: 2017-12-21T10:44:34Z IGNITE-7217: added JMX beans for custom executors commit 35512905d933215f1b4698aea84930a04dbc3d6a Author: Stanislav Lukyanov Date: 2017-12-21T11:24:46Z IGNITE-7217: added MBeans test to the basic test suite commit c1bb23d3c7a9d801ebdc677db5b16ae0bdc9 Author: Stanislav Lukyanov Date: 2017-12-22T10:02:52Z IGNITE-7217: fixed Java 7 compatibility commit 99f044b1efb05e8fc6a428fdbeed60a59b7770a1 Author: Stanislav Lukyanov Date: 2017-12-22T11:03:16Z IGNITE-7217: refactored mbeans registration/unregistration commit a7c95714eaa3bb9ffcc5bb8eb4801e6eae3fe366 Author: Stanislav Lukyanov Date: 2017-12-22T11:35:05Z IGNITE-7217: further refactoring commit 5f9a14a899ced28ba8297aac0938fd906b7af4b7 Author: Stanislav Lukyanov Date: 2017-12-22T12:55:01Z IGNITE-7217: added MBeans for all thread pools commit d9be82c27171cfc80340cf794ee936631ab948d3 Author: xmitya Date: 2017-12-26T12:00:41Z Merge branch 'master' into IGNITE-7217 commit 39b34928680a36a30f035ff94b6ba0dfe2bd2e6e Author: xmitya Date: 2017-12-26T12:19:34Z Formatting adjustments. ---
[GitHub] ignite pull request #3300: IGNITE-7321 mvcc enabled
Github user gvvinblade closed the pull request at: https://github.com/apache/ignite/pull/3300 ---
[GitHub] ignite pull request #3268: IGNITE-7217: Add abilities to monitor custom thre...
Github user slukyano closed the pull request at: https://github.com/apache/ignite/pull/3268 ---
[GitHub] ignite pull request #3299: IGNITE-7321
Github user gvvinblade closed the pull request at: https://github.com/apache/ignite/pull/3299 ---
[GitHub] ignite pull request #3300: IGNITE-7321 mvcc enabled
GitHub user gvvinblade opened a pull request: https://github.com/apache/ignite/pull/3300 IGNITE-7321 mvcc enabled You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-7321-mvcc-enabled Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3300.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3300 commit 6be6779b6336c36cd71eef0a25199a6a877ce6b5 Author: sboikovDate: 2017-09-05T13:47:11Z ignite-3484 commit e3bba83256c1eb53c4b40fbd9ddba47fcf9d58d5 Author: sboikov Date: 2017-09-06T07:10:26Z ignite-3484 commit dd0afb28466094b801506da8afa3601bfaebd853 Author: sboikov Date: 2017-09-06T07:30:04Z ignite-3484 commit 27b87b413348b03986a463551db24b7726321732 Author: sboikov Date: 2017-09-06T08:19:18Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java commit dcaf8801accd6ee089849a82b2ccd558aec81895 Author: sboikov Date: 2017-09-06T08:19:30Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java commit c966451d0bf7059575de92bcfae43d72096ebce4 Author: sboikov Date: 2017-09-06T08:27:04Z Merge branch 'ignite-3478' into ignite-3484 commit 91b9911731a387a3199ddbbc22704bc14af09995 Author: sboikov Date: 2017-09-06T09:22:22Z ignite-3484 commit e40b4d9dcd6fe6c1cd2640bdd7116ca5a08ed781 Author: sboikov Date: 2017-09-07T09:12:32Z ignite-3484 commit 41a1c571e6ba1765941e2f1679dc4ac1582275c4 Author: sboikov Date: 2017-09-08T07:55:24Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3484 commit 1626130801dc330bcaf693f46906f6564cce6802 Author: sboikov Date: 2017-09-08T08:04:57Z ignite-3484 commit 91bbb7cd24f38a38e2e65fc3ebf5637572b11b25 Author: sboikov Date: 2017-09-08T12:34:52Z Merge remote-tracking branch 'remotes/origin/master' into ignite-6149 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/near/GridNearTxPrepareFutureAdapter.java commit 085a3219088bc1610d1928c437f075a6fb9a4f9b Author: sboikov Date: 2017-09-08T13:02:16Z ignite-6149 commit c6f894817ef063984cee1ea886313eecc8da3be0 Author: sboikov Date: 2017-09-11T13:51:11Z ignite-6149 commit 774af444cfc03ab292124ae027e5e85b61496011 Author: sboikov Date: 2017-09-12T07:45:36Z Merge remote-tracking branch 'remotes/origin/master' into ignite-6149 commit 6d1702646fc0abbba690b1b4f7b7fde718d9f4d2 Author: sboikov Date: 2017-09-12T08:48:16Z ignite-6149 commit 0079a0052215f2c1b66b300c34bcf30e4a1da1b0 Author: sboikov Date: 2017-09-12T08:48:16Z ignite-6149 commit 072322f9002df3c2c9033108897d2b7a3de35ca5 Author: sboikov Date: 2017-09-12T14:19:18Z Merge remote-tracking branch 'origin/ignite-6149' into ignite-6149 # Conflicts: # modules/core/src/test/java/org/apache/ignite/internal/processors/cache/mvcc/CacheMvccTransactionsTest.java commit b5cac1c0d818967553c72d4699d1c66c57c1fb10 Author: sboikov Date: 2017-09-13T07:36:42Z Merge remote-tracking branch 'remotes/origin/master' into ignite-6149 commit 16e52541ef799f1e135576636a69d89591a1fee2 Author: sboikov Date: 2017-09-13T08:06:37Z ignite-6149 commit 9d90972df23a0266246baf6b41774184542233e9 Author: sboikov Date: 2017-09-13T08:49:39Z ignite-6149 commit 7a99432655307acf7c5e190e20952322f68e0e4f Author: sboikov Date: 2017-09-13T09:44:29Z ignite-6149 commit c961bd91186217807e51453fad906cd5118a8b9a Author: sboikov Date: 2017-09-13T11:24:21Z ignite-6149 commit f3f58214047d4fa1d9cb5f7ce50ac7278ee17569 Author: sboikov Date: 2017-09-13T14:15:55Z ignite-6149 commit 1a7e10ac56e57ddbe1d9ccf99acab90062dc6957 Author: sboikov Date: 2017-09-13T15:01:42Z Merge remote-tracking branch 'origin/ignite-6149' into ignite-6149 commit 393f802465c8b68d5a5de59ed9107fe25ad23777 Author: sboikov Date: 2017-09-15T08:06:58Z Merge remote-tracking branch 'remotes/origin/master' into ignite-6149 commit 5e886653ca4cf6f538121a0be854a8904e1b4417 Author: sboikov Date: 2017-09-15T09:05:44Z ignite-6149 commit 0feca31637feae8ed37dd5d2f53112e3df0c8c00
Re: Handling slashes in cache names
Also, considering case-insensitivity issue, we need to choose some encoding that only uses upper or lower case letters in encoding result. By the way, such encoding will resolve cache name clashes due to case-insensitivity issue. Best Regards, Igor On Wed, Dec 27, 2017 at 4:18 PM, Igor Sapegowrote: > I personally like a Pavel's suggestion - base64 encoding seems like > a good solution, while string hashes will arise a collision issue. > > Best Regards, > Igor > > On Wed, Dec 27, 2017 at 3:29 PM, Petr Ivanov wrote: > >> Special characters banning seems to be exclusive way and cannot be >> controlled in future if new symbols arise. >> Maybe better solution will be choosing the array of permitted symbols for >> caches names (i.e. [a-zA-Z0-9_-])? >> >> >> Also +1 for using abstract hash string for directories names. >> >> >> > On 27 Dec 2017, at 15:14, Stanislav Lukyanov >> wrote: >> > >> > We can – by mapping a cache name to some (safe) string to be used as a >> directory name, say via Base64 as Pavel has suggested. >> > >> > However, I think that banning certain characters might be reasonable. >> > Some characters might be considered reserved (e.g. slashes, colon, >> asterisk, etc) to be used later, in case some future feature requires cache >> names to have an actual meaning. >> > Some characters might be banned just as a precaution (e.g. control >> characters or whitespaces) because they might cause problems with logging >> or elsewhere (you might have a bad time processing a cache name with \0 in >> it :) ). >> > >> > The question is whether or not these considerations worth adding code >> and/or changing existing behavior. >> > >> > BTW Java folks had similar discussion on Java module names resulting in >> http://mail.openjdk.java.net/pipermail/jpms-spec-experts/201 >> 6-December/000515.html. >> > >> > Thanks, >> > Stan >> > >> > From: Vladimir Ozerov >> > Sent: 27 декабря 2017 г. 14:37 >> > To: dev@ignite.apache.org >> > Subject: Re: Handling slashes in cache names >> > >> > Cache name appears to me purely logical entity. Can we simply store >> cache >> > ID in file system paths without adding any restrictions to cache names? >> > >> > On Wed, Dec 27, 2017 at 2:26 PM, Stanislav Lukyanov < >> stanlukya...@gmail.com> >> > wrote: >> > >> >> Well, that’s my question too :) >> >> Do we have any compatibility guidelines or other documents on what can >> or >> >> cannot be in a minor/major release? >> >> >> >> Also, it might be helpful to add an environment variable (like >> >> IGNITE_DISABLE_CACHE_NAME_RESTRICTIONS) to restore the old behavior, >> just >> >> in case. >> >> >> >> Thanks, >> >> Stan >> >> >> >> From: Dmitriy Setrakyan >> >> Sent: 26 декабря 2017 г. 17:02 >> >> To: dev@ignite.apache.org >> >> Subject: Re: Handling slashes in cache names >> >> >> >> Looks good to me. Is this going to be an exception on startup? If yes, >> is >> >> it safe to release it, or should we wait till 3.0? >> >> >> >> On Tue, Dec 26, 2017 at 2:08 AM, Stanislav Lukyanov < >> >> stanlukya...@gmail.com> >> >> wrote: >> >> >> >>> Thanks for the feedback. >> >>> >> >>> It seems that another thing to handle is case-insensitive FS – >> “mycache” >> >>> and “MyCache” is the same on Windows, so it might be reasonable to >> >> disallow >> >>> having two caches with names that are equal ignoring case. >> >>> And one more thing is control characters – forbidding at least range >> of >> >>> ASCII 0x00-0x20 seems reasonable. >> >>> >> >>> To summarize, a possible set of restrictions would be >> >>> - Whitespace characters (via Character.isWhitespaceCharacter) >> >>> - Control characters (via Character.isISOCharacter) >> >>> - Slashes >> >>> - Characters reserved in Windows (<>:"/\|?*) >> >>> - Length (say, up to 255) >> >>> - Distinct names of caches when ignoring case >> >>> It seems reasonable to enforce that even regardless of persistence >> >>> directories naming (AFAIU that’s what Dmitry meant by forbidding >> things >> >>> altogether), so that’s what I’m going to do. >> >>> Any concerns? >> >>> Specifically, would it be OK from backward compatibility point of >> view to >> >>> forbid all these characters now for all caches? >> >>> >> >>> Thanks, >> >>> Stan >> >>> >> >>> >> >>> From: Alexey Kuznetsov >> >>> Sent: 26 декабря 2017 г. 7:51 >> >>> To: dev@ignite.apache.org >> >>> Subject: Re: Handling slashes in cache names >> >>> >> >>> It also make sense to limit cache name length to reasonable length. >> >>> Because some File systems could have limitations on path length. >> >>> See: https://en.wikipedia.org/wiki/Filename#Length_restrictions >> >>> >> >>> On Tue, Dec 26, 2017 at 1:41 AM, Dmitriy Setrakyan < >> >> dsetrak...@apache.org> >> >>> wrote: >> >>> >> My preference would be to prohibit forward and backward slashes in >> >> cache >> names altogether, as they may create a false feeling of some >> directory >> structure, which does not
[GitHub] ignite pull request #3299: IGNITE-7321
GitHub user gvvinblade opened a pull request: https://github.com/apache/ignite/pull/3299 IGNITE-7321 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-7321 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3299.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3299 commit f6e982540e65ab17d439dba990794f35616a30dd Author: sboikovDate: 2017-08-30T09:45:40Z ignite-3478 commit 275a85db5cd6923b36126166ae99b15e876192be Author: sboikov Date: 2017-08-31T07:44:07Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit b7b9089f0102b8cab9942a9c887d93e9f26cc7d2 Author: sboikov Date: 2017-08-31T09:00:36Z disco cache cleanup commit 855c2d45794c300d41e386b4e6fa40736cc3e40d Author: sboikov Date: 2017-08-31T09:09:58Z Merge branch 'ignite-3478-1' into ignite-3478 commit 08be7310a93d3ce455215b97cf8ab1a2c3f0ab31 Author: sboikov Date: 2017-08-31T09:52:23Z ignite-3478 commit fce2e31f0fd2f4f6a9944422e40408a0c65cfe90 Author: sboikov Date: 2017-09-04T08:13:50Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit d3c049952384750c5543a9f88b383c033ef74096 Author: sboikov Date: 2017-09-04T08:52:11Z ignite-3478 commit e71ce1937a18dd32448e92b1038dc48d4cb6f8ab Author: sboikov Date: 2017-09-04T10:16:03Z ignite-3478 commit 5fac5b0965e97f8951e16e10ca9229a2e78ddb0c Author: sboikov Date: 2017-09-05T10:16:44Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java commit 2e0c9c08e046e8d6af1b5358d9053eae999b1fb4 Author: sboikov Date: 2017-09-05T11:30:55Z ignite-3478 commit e1e07ffdf2d711ba3e72f316f5a3970eff27372e Author: sboikov Date: 2017-09-05T11:31:14Z ignite-3478 commit cbada3934a386668da0b11d4de7d0f58a4d04dfe Author: sboikov Date: 2017-09-05T11:49:49Z ignite-3484 commit 5a82c68dcd1927bb6fded8b7def38c91ff6e145b Author: sboikov Date: 2017-09-05T11:59:49Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit bc9134c94b7a738dc1664e96ca6eabb059f1c268 Author: sboikov Date: 2017-09-05T12:03:39Z Merge branch 'ignite-3478' into ignite-3484 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/tree/AbstractDataInnerIO.java commit b4bfcde78825c6517232e49d389bdb5de19f05a9 Author: sboikov Date: 2017-09-05T12:27:51Z ignite-3484 commit 43834aaab9e2c3cd5fdd55289fdc4a9ff8ab6599 Author: sboikov Date: 2017-09-05T13:13:00Z ignite-3478 commit d1b828095713fcadfa260cf94fef01b42a1b12fd Author: sboikov Date: 2017-09-05T13:13:33Z Merge branch 'ignite-3478' into ignite-3484 commit 6be6779b6336c36cd71eef0a25199a6a877ce6b5 Author: sboikov Date: 2017-09-05T13:47:11Z ignite-3484 commit e3bba83256c1eb53c4b40fbd9ddba47fcf9d58d5 Author: sboikov Date: 2017-09-06T07:10:26Z ignite-3484 commit dd0afb28466094b801506da8afa3601bfaebd853 Author: sboikov Date: 2017-09-06T07:30:04Z ignite-3484 commit 27b87b413348b03986a463551db24b7726321732 Author: sboikov Date: 2017-09-06T08:19:18Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java commit dcaf8801accd6ee089849a82b2ccd558aec81895 Author: sboikov Date: 2017-09-06T08:19:30Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java commit c966451d0bf7059575de92bcfae43d72096ebce4 Author: sboikov Date: 2017-09-06T08:27:04Z Merge branch 'ignite-3478' into ignite-3484 commit 91b9911731a387a3199ddbbc22704bc14af09995 Author: sboikov Date: 2017-09-06T09:22:22Z ignite-3484 commit e40b4d9dcd6fe6c1cd2640bdd7116ca5a08ed781 Author: sboikov Date: 2017-09-07T09:12:32Z ignite-3484 commit 41a1c571e6ba1765941e2f1679dc4ac1582275c4 Author: sboikov Date: 2017-09-08T07:55:24Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3484 commit 1626130801dc330bcaf693f46906f6564cce6802 Author: sboikov Date:
Re: Handling slashes in cache names
I personally like a Pavel's suggestion - base64 encoding seems like a good solution, while string hashes will arise a collision issue. Best Regards, Igor On Wed, Dec 27, 2017 at 3:29 PM, Petr Ivanovwrote: > Special characters banning seems to be exclusive way and cannot be > controlled in future if new symbols arise. > Maybe better solution will be choosing the array of permitted symbols for > caches names (i.e. [a-zA-Z0-9_-])? > > > Also +1 for using abstract hash string for directories names. > > > > On 27 Dec 2017, at 15:14, Stanislav Lukyanov > wrote: > > > > We can – by mapping a cache name to some (safe) string to be used as a > directory name, say via Base64 as Pavel has suggested. > > > > However, I think that banning certain characters might be reasonable. > > Some characters might be considered reserved (e.g. slashes, colon, > asterisk, etc) to be used later, in case some future feature requires cache > names to have an actual meaning. > > Some characters might be banned just as a precaution (e.g. control > characters or whitespaces) because they might cause problems with logging > or elsewhere (you might have a bad time processing a cache name with \0 in > it :) ). > > > > The question is whether or not these considerations worth adding code > and/or changing existing behavior. > > > > BTW Java folks had similar discussion on Java module names resulting in > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/ > 2016-December/000515.html. > > > > Thanks, > > Stan > > > > From: Vladimir Ozerov > > Sent: 27 декабря 2017 г. 14:37 > > To: dev@ignite.apache.org > > Subject: Re: Handling slashes in cache names > > > > Cache name appears to me purely logical entity. Can we simply store cache > > ID in file system paths without adding any restrictions to cache names? > > > > On Wed, Dec 27, 2017 at 2:26 PM, Stanislav Lukyanov < > stanlukya...@gmail.com> > > wrote: > > > >> Well, that’s my question too :) > >> Do we have any compatibility guidelines or other documents on what can > or > >> cannot be in a minor/major release? > >> > >> Also, it might be helpful to add an environment variable (like > >> IGNITE_DISABLE_CACHE_NAME_RESTRICTIONS) to restore the old behavior, > just > >> in case. > >> > >> Thanks, > >> Stan > >> > >> From: Dmitriy Setrakyan > >> Sent: 26 декабря 2017 г. 17:02 > >> To: dev@ignite.apache.org > >> Subject: Re: Handling slashes in cache names > >> > >> Looks good to me. Is this going to be an exception on startup? If yes, > is > >> it safe to release it, or should we wait till 3.0? > >> > >> On Tue, Dec 26, 2017 at 2:08 AM, Stanislav Lukyanov < > >> stanlukya...@gmail.com> > >> wrote: > >> > >>> Thanks for the feedback. > >>> > >>> It seems that another thing to handle is case-insensitive FS – > “mycache” > >>> and “MyCache” is the same on Windows, so it might be reasonable to > >> disallow > >>> having two caches with names that are equal ignoring case. > >>> And one more thing is control characters – forbidding at least range of > >>> ASCII 0x00-0x20 seems reasonable. > >>> > >>> To summarize, a possible set of restrictions would be > >>> - Whitespace characters (via Character.isWhitespaceCharacter) > >>> - Control characters (via Character.isISOCharacter) > >>> - Slashes > >>> - Characters reserved in Windows (<>:"/\|?*) > >>> - Length (say, up to 255) > >>> - Distinct names of caches when ignoring case > >>> It seems reasonable to enforce that even regardless of persistence > >>> directories naming (AFAIU that’s what Dmitry meant by forbidding things > >>> altogether), so that’s what I’m going to do. > >>> Any concerns? > >>> Specifically, would it be OK from backward compatibility point of view > to > >>> forbid all these characters now for all caches? > >>> > >>> Thanks, > >>> Stan > >>> > >>> > >>> From: Alexey Kuznetsov > >>> Sent: 26 декабря 2017 г. 7:51 > >>> To: dev@ignite.apache.org > >>> Subject: Re: Handling slashes in cache names > >>> > >>> It also make sense to limit cache name length to reasonable length. > >>> Because some File systems could have limitations on path length. > >>> See: https://en.wikipedia.org/wiki/Filename#Length_restrictions > >>> > >>> On Tue, Dec 26, 2017 at 1:41 AM, Dmitriy Setrakyan < > >> dsetrak...@apache.org> > >>> wrote: > >>> > My preference would be to prohibit forward and backward slashes in > >> cache > names altogether, as they may create a false feeling of some directory > structure, which does not exist. We should also prohibit spaces as > >> well. > > D. > > On Mon, Dec 25, 2017 at 7:09 AM, Stanislav Lukyanov < > stanlukya...@gmail.com> > wrote: > > > Hi all, > > > > I’m looking into https://issues.apache.org/jira/browse/IGNITE-7264, > >>> and > I > > need some guidance on what’s the best way to approach it. > > > > The problem is that cache names are not restricted, but if > >> persistence > >>>
Re: Handling slashes in cache names
Special characters banning seems to be exclusive way and cannot be controlled in future if new symbols arise. Maybe better solution will be choosing the array of permitted symbols for caches names (i.e. [a-zA-Z0-9_-])? Also +1 for using abstract hash string for directories names. > On 27 Dec 2017, at 15:14, Stanislav Lukyanovwrote: > > We can – by mapping a cache name to some (safe) string to be used as a > directory name, say via Base64 as Pavel has suggested. > > However, I think that banning certain characters might be reasonable. > Some characters might be considered reserved (e.g. slashes, colon, asterisk, > etc) to be used later, in case some future feature requires cache names to > have an actual meaning. > Some characters might be banned just as a precaution (e.g. control characters > or whitespaces) because they might cause problems with logging or elsewhere > (you might have a bad time processing a cache name with \0 in it :) ). > > The question is whether or not these considerations worth adding code and/or > changing existing behavior. > > BTW Java folks had similar discussion on Java module names resulting in > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-December/000515.html. > > Thanks, > Stan > > From: Vladimir Ozerov > Sent: 27 декабря 2017 г. 14:37 > To: dev@ignite.apache.org > Subject: Re: Handling slashes in cache names > > Cache name appears to me purely logical entity. Can we simply store cache > ID in file system paths without adding any restrictions to cache names? > > On Wed, Dec 27, 2017 at 2:26 PM, Stanislav Lukyanov > wrote: > >> Well, that’s my question too :) >> Do we have any compatibility guidelines or other documents on what can or >> cannot be in a minor/major release? >> >> Also, it might be helpful to add an environment variable (like >> IGNITE_DISABLE_CACHE_NAME_RESTRICTIONS) to restore the old behavior, just >> in case. >> >> Thanks, >> Stan >> >> From: Dmitriy Setrakyan >> Sent: 26 декабря 2017 г. 17:02 >> To: dev@ignite.apache.org >> Subject: Re: Handling slashes in cache names >> >> Looks good to me. Is this going to be an exception on startup? If yes, is >> it safe to release it, or should we wait till 3.0? >> >> On Tue, Dec 26, 2017 at 2:08 AM, Stanislav Lukyanov < >> stanlukya...@gmail.com> >> wrote: >> >>> Thanks for the feedback. >>> >>> It seems that another thing to handle is case-insensitive FS – “mycache” >>> and “MyCache” is the same on Windows, so it might be reasonable to >> disallow >>> having two caches with names that are equal ignoring case. >>> And one more thing is control characters – forbidding at least range of >>> ASCII 0x00-0x20 seems reasonable. >>> >>> To summarize, a possible set of restrictions would be >>> - Whitespace characters (via Character.isWhitespaceCharacter) >>> - Control characters (via Character.isISOCharacter) >>> - Slashes >>> - Characters reserved in Windows (<>:"/\|?*) >>> - Length (say, up to 255) >>> - Distinct names of caches when ignoring case >>> It seems reasonable to enforce that even regardless of persistence >>> directories naming (AFAIU that’s what Dmitry meant by forbidding things >>> altogether), so that’s what I’m going to do. >>> Any concerns? >>> Specifically, would it be OK from backward compatibility point of view to >>> forbid all these characters now for all caches? >>> >>> Thanks, >>> Stan >>> >>> >>> From: Alexey Kuznetsov >>> Sent: 26 декабря 2017 г. 7:51 >>> To: dev@ignite.apache.org >>> Subject: Re: Handling slashes in cache names >>> >>> It also make sense to limit cache name length to reasonable length. >>> Because some File systems could have limitations on path length. >>> See: https://en.wikipedia.org/wiki/Filename#Length_restrictions >>> >>> On Tue, Dec 26, 2017 at 1:41 AM, Dmitriy Setrakyan < >> dsetrak...@apache.org> >>> wrote: >>> My preference would be to prohibit forward and backward slashes in >> cache names altogether, as they may create a false feeling of some directory structure, which does not exist. We should also prohibit spaces as >> well. D. On Mon, Dec 25, 2017 at 7:09 AM, Stanislav Lukyanov < stanlukya...@gmail.com> wrote: > Hi all, > > I’m looking into https://issues.apache.org/jira/browse/IGNITE-7264, >>> and I > need some guidance on what’s the best way to approach it. > > The problem is that cache names are not restricted, but if >> persistence >>> is > enabled the cache needs to have a corresponding directory on the file > system (“cache-…”) which can’t be created if the cache name contains > certain characters (or a reserved system name). > > A straightforward approach would be to check if a cache name is >> allowed on > the local system (e.g. via `Paths.get(name)`) and fail to create >> cache >>> if > it isn’t, but I’m a bit concerned with the
RE: Handling slashes in cache names
We can – by mapping a cache name to some (safe) string to be used as a directory name, say via Base64 as Pavel has suggested. However, I think that banning certain characters might be reasonable. Some characters might be considered reserved (e.g. slashes, colon, asterisk, etc) to be used later, in case some future feature requires cache names to have an actual meaning. Some characters might be banned just as a precaution (e.g. control characters or whitespaces) because they might cause problems with logging or elsewhere (you might have a bad time processing a cache name with \0 in it :) ). The question is whether or not these considerations worth adding code and/or changing existing behavior. BTW Java folks had similar discussion on Java module names resulting in http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-December/000515.html. Thanks, Stan From: Vladimir Ozerov Sent: 27 декабря 2017 г. 14:37 To: dev@ignite.apache.org Subject: Re: Handling slashes in cache names Cache name appears to me purely logical entity. Can we simply store cache ID in file system paths without adding any restrictions to cache names? On Wed, Dec 27, 2017 at 2:26 PM, Stanislav Lukyanovwrote: > Well, that’s my question too :) > Do we have any compatibility guidelines or other documents on what can or > cannot be in a minor/major release? > > Also, it might be helpful to add an environment variable (like > IGNITE_DISABLE_CACHE_NAME_RESTRICTIONS) to restore the old behavior, just > in case. > > Thanks, > Stan > > From: Dmitriy Setrakyan > Sent: 26 декабря 2017 г. 17:02 > To: dev@ignite.apache.org > Subject: Re: Handling slashes in cache names > > Looks good to me. Is this going to be an exception on startup? If yes, is > it safe to release it, or should we wait till 3.0? > > On Tue, Dec 26, 2017 at 2:08 AM, Stanislav Lukyanov < > stanlukya...@gmail.com> > wrote: > > > Thanks for the feedback. > > > > It seems that another thing to handle is case-insensitive FS – “mycache” > > and “MyCache” is the same on Windows, so it might be reasonable to > disallow > > having two caches with names that are equal ignoring case. > > And one more thing is control characters – forbidding at least range of > > ASCII 0x00-0x20 seems reasonable. > > > > To summarize, a possible set of restrictions would be > > - Whitespace characters (via Character.isWhitespaceCharacter) > > - Control characters (via Character.isISOCharacter) > > - Slashes > > - Characters reserved in Windows (<>:"/\|?*) > > - Length (say, up to 255) > > - Distinct names of caches when ignoring case > > It seems reasonable to enforce that even regardless of persistence > > directories naming (AFAIU that’s what Dmitry meant by forbidding things > > altogether), so that’s what I’m going to do. > > Any concerns? > > Specifically, would it be OK from backward compatibility point of view to > > forbid all these characters now for all caches? > > > > Thanks, > > Stan > > > > > > From: Alexey Kuznetsov > > Sent: 26 декабря 2017 г. 7:51 > > To: dev@ignite.apache.org > > Subject: Re: Handling slashes in cache names > > > > It also make sense to limit cache name length to reasonable length. > > Because some File systems could have limitations on path length. > > See: https://en.wikipedia.org/wiki/Filename#Length_restrictions > > > > On Tue, Dec 26, 2017 at 1:41 AM, Dmitriy Setrakyan < > dsetrak...@apache.org> > > wrote: > > > > > My preference would be to prohibit forward and backward slashes in > cache > > > names altogether, as they may create a false feeling of some directory > > > structure, which does not exist. We should also prohibit spaces as > well. > > > > > > D. > > > > > > On Mon, Dec 25, 2017 at 7:09 AM, Stanislav Lukyanov < > > > stanlukya...@gmail.com> > > > wrote: > > > > > > > Hi all, > > > > > > > > I’m looking into https://issues.apache.org/jira/browse/IGNITE-7264, > > and > > > I > > > > need some guidance on what’s the best way to approach it. > > > > > > > > The problem is that cache names are not restricted, but if > persistence > > is > > > > enabled the cache needs to have a corresponding directory on the file > > > > system (“cache-…”) which can’t be created if the cache name contains > > > > certain characters (or a reserved system name). > > > > > > > > A straightforward approach would be to check if a cache name is > allowed > > > on > > > > the local system (e.g. via `Paths.get(name)`) and fail to create > cache > > if > > > > it isn’t, but I’m a bit concerned with the consistency of the > behavior > > > (the > > > > same cache name be allowed on one system and not on another). > > > > I think a better way would be to replace special characters (say, all > > > > non-alphanumeric characters) with underscores in file names (not > > changing > > > > the cache configuration). Would this be OK? Are there any risks I’m > not > > > > considering? > > > > > > > > WDYT? > > > > > > > > Thanks, > > > > Stan > > > > > > > >
[GitHub] ignite pull request #3298: IGNITE-6022: Implemented native batch execution s...
GitHub user dolphin1414 opened a pull request: https://github.com/apache/ignite/pull/3298 IGNITE-6022: Implemented native batch execution support for DML statements. Implemented native batch execution support for DML statements. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-6022 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3298.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3298 commit eacd01fc01c9dea0b81e7aec8d5bbd89852401ae Author: rkondakovDate: 2017-12-27T11:53:30Z IGNITE-6022: Implemented native batch execution support for DML statements. ---
[GitHub] ignite pull request #3297: IGNITE-6783: Create common mechanism for group tr...
GitHub user artemmalykh opened a pull request: https://github.com/apache/ignite/pull/3297 IGNITE-6783: Create common mechanism for group training. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-6783 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3297.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3297 commit c4dbf2113d54976ed201171298174adc9a825eda Author: Artem MalykhDate: 2017-10-27T17:03:31Z IGNITE-6783: First steps. commit ebb5b1f9c1a3d001b089b29c019acb80d02d8602 Author: Artem Malykh Date: 2017-10-27T17:10:59Z IGNITE-6783: First steps[1]. commit 4af8bf1551d19830d35f3a36f7ad8a684c6260f9 Author: Artem Malykh Date: 2017-10-29T10:59:44Z IGNITE-6783: First steps[2]. commit 6c4c77c262f9818142819f8b40a4855a62aabd4f Author: Artem Malykh Date: 2017-10-29T12:42:43Z IGNITE-6783: First steps[3]. commit dd8df49c4f04cf04e4cebdccdb0df72022e67886 Author: Artem Malykh Date: 2017-10-30T11:08:02Z IGNITE-6783: First steps[4]. commit a335496869096ff0e23e9f6ba529afcbd26116a3 Author: Artem Malykh Date: 2017-10-30T13:58:14Z IGNITE-6783: Cherrypicked some functions. commit 7ed9ff9b2674042509b4d9620d406f13db9af357 Author: Artem Malykh Date: 2017-10-30T15:51:11Z IGNITE-6783: First steps[5]. commit 6ef704d1d6846c14e2d7f1636c21269cd0843b9a Author: Artem Malykh Date: 2017-10-31T15:40:42Z IGNITE-6783: Workerschain is almost complete. commit 8f6d713f2beeb7156c6fdfc977fceae18d9e8965 Author: Artem Malykh Date: 2017-11-01T13:05:36Z IGNITE-6783: Monad-like workers chain. commit e87f9b046f7fa45e0cdca2499f268baf527ca1c8 Author: Artem Malykh Date: 2017-11-01T15:49:39Z IGNITE-6783: Enriched GroupTrainingCacheKey with data. commit c9b818803e6b1f7d3ae4579afd9392bf9e34f039 Author: Artem Malykh Date: 2017-11-02T12:53:26Z IGNITE-6783: Some fixes and test class. commit c048bcc0088386bcdfe82ed7ae0c690be7c6c406 Author: Artem Malykh Date: 2017-11-08T18:15:22Z IGNITE-6783: Intermidiate commit [uncompilable]. commit 58ea6b196fda151ccddda635ba18bc4ebd64aecd Author: Artem Malykh Date: 2017-11-13T19:51:34Z IGNITE-6783: WIP commit f15e634eee17f322ee46e83f8d9d8a99e42c3e9e Author: Artem Malykh Date: 2017-11-13T21:08:14Z Merge branch 'master-apache' into ignite-6783 # Conflicts: # modules/ml/src/main/java/org/apache/ignite/ml/math/distributed/CacheUtils.java # modules/ml/src/main/java/org/apache/ignite/ml/math/functions/Functions.java # modules/ml/src/main/java/org/apache/ignite/ml/math/functions/IgniteBinaryOperator.java commit 915845f818467c5d7b9aaedfac5cf93b40ad8ae5 Author: Artem Malykh Date: 2017-11-13T21:08:30Z IGNITE-6783: WIP commit 77389db431642d8b84d92b122285f16036bc4669 Author: Artem Malykh Date: 2017-11-15T12:56:00Z IGNITE-6783: Compilable version. commit 813da2cce68e94741b27bfb6dd952cdc24985234 Author: Artem Malykh Date: 2017-11-15T17:19:54Z IGNITE-6783: Initial tests for Destributed Trainer Workers Chain. commit 1d4f90312a4849ee53df282d0d5483d73e5a45db Author: Artem Malykh Date: 2017-11-16T18:02:24Z IGNITE-6783: Started writing GroupTrainerTest. commit f56a89fc911c74f1034e579a89f7f419681be0f2 Author: Artem Malykh Date: 2017-11-17T13:46:55Z IGNITE-6783: GroupTrainerTest is compilable. commit c37b274799c5fa2bcb1c657a3209aae9267f0a0a Author: Artem Malykh Date: 2017-11-19T14:47:15Z IGNITE-6783: WIP. commit 8ed15f0b12f1f8fc0cd1aab948689ea6af4d76ba Author: Artem Malykh Date: 2017-11-20T10:48:57Z IGNITE-6783: WIP. commit a38a47ac6966044e0db713037ffda53193306c70 Author: Artem Malykh Date: 2017-11-20T11:15:57Z IGNITE-6783: WIP. commit caefdb685638dd9fc3dcd50a1a13fb7ee526a371 Author: Artem Malykh Date: 2017-11-23T14:29:02Z IGNITE-6882: WIP. commit abc7ae0cdd272f17170c5e4f4ee88df13278957a Author: Artem Malykh Date: 2017-11-23T15:36:12Z IGNITE-6783: WIP. commit a3ff2ad60358dcec55e901e87086bb3586489862 Author: Artem Malykh Date: 2017-11-23T15:36:48Z IGNITE-7007: Added missing comments. commit 814d0415faeff8736c0a40ee6681ba63518dcf55 Author: Artem Malykh Date: 2017-11-23T15:47:26Z IGNITE-6783: WIP. commit eb8da5824cdf66f0e55989929832d8669e8b5384 Author: Artem Malykh Date: 2017-11-23T23:17:09Z IGNITE-6783: WIP. commit 7fbe887e1e559426396321b162dcd7f4a352bdcf Author: Artem
[jira] [Created] (IGNITE-7321) DML operation hangs in case exception is thrown from DHT enlist future
Vladimir Ozerov created IGNITE-7321: --- Summary: DML operation hangs in case exception is thrown from DHT enlist future Key: IGNITE-7321 URL: https://issues.apache.org/jira/browse/IGNITE-7321 Project: Ignite Issue Type: Bug Components: sql Reporter: Vladimir Ozerov Assignee: Igor Seliverstov Fix For: 2.5 Root cause: when exception is thrown on primary node (e.g. double-insert or MVCC version conflict), primary future is completed, but exception doesn't seem to be propagated to near node. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Handling slashes in cache names
Agree with Stan and Vladimir. We should not impose any restrictions on cache names, some users may have issues with that. Using cache names as file names is internal implementation detail. We can use cache id or some kind of encoding (base64, etc) to avoid file system issues. Thanks, Pavel On Wed, Dec 27, 2017 at 2:38 PM, Stanislav Lukyanovwrote: > That’s interesting, thanks. > So, do you think the locale-specific file separators should be banned as > well? > Handling all kinds of cases like this might be complicated. > > I’d rather use something else if the cache name is not a valid file name, > a hash of the cache name. > This way all corner cases can be handled at once. > The algorithm would be > 1) Check that cache name doesn’t contain banned characters > 2) Try to create a Path for “cache-” > 3) If failed, create a Path for “cache-” > > Stan > > From: Igor Sapego > Sent: 26 декабря 2017 г. 17:59 > To: dev@ignite.apache.org > Subject: Re: Handling slashes in cache names > > There are also some international features that you might want to > address. For example, instead of backslash some other characters > may be used on Windows - ¥ on the Japanese version, ₩ on the > Korean version. > See [1] for more info. > > Here is the citation: > Security Considerations for Character Sets in File Names > Windows code page and OEM character sets used on > Japanese-language systems contain the Yen symbol (¥) instead of > a backslash (\). Thus, the Yen character is a prohibited character for > NTFS and FAT file systems. When mapping Unicode to > a Japanese-language code page, conversion functions map both > backslash (U+005C) and the normal Unicode Yen symbol (U+00A5) > to this same character. For security reasons, your applications should > not typically allow the character U+00A5 in a Unicode string that > might be converted for use as a FAT file name. > > [1] - https://msdn.microsoft.com/en-us/library/dd374047(v=vs.85).aspx > > > Best Regards, > Igor > > On Tue, Dec 26, 2017 at 5:01 PM, Dmitriy Setrakyan > wrote: > > > Looks good to me. Is this going to be an exception on startup? If yes, is > > it safe to release it, or should we wait till 3.0? > > > > On Tue, Dec 26, 2017 at 2:08 AM, Stanislav Lukyanov < > > stanlukya...@gmail.com> > > wrote: > > > > > Thanks for the feedback. > > > > > > It seems that another thing to handle is case-insensitive FS – > “mycache” > > > and “MyCache” is the same on Windows, so it might be reasonable to > > disallow > > > having two caches with names that are equal ignoring case. > > > And one more thing is control characters – forbidding at least range of > > > ASCII 0x00-0x20 seems reasonable. > > > > > > To summarize, a possible set of restrictions would be > > > - Whitespace characters (via Character.isWhitespaceCharacter) > > > - Control characters (via Character.isISOCharacter) > > > - Slashes > > > - Characters reserved in Windows (<>:"/\|?*) > > > - Length (say, up to 255) > > > - Distinct names of caches when ignoring case > > > It seems reasonable to enforce that even regardless of persistence > > > directories naming (AFAIU that’s what Dmitry meant by forbidding things > > > altogether), so that’s what I’m going to do. > > > Any concerns? > > > Specifically, would it be OK from backward compatibility point of view > to > > > forbid all these characters now for all caches? > > > > > > Thanks, > > > Stan > > > > > > > > > From: Alexey Kuznetsov > > > Sent: 26 декабря 2017 г. 7:51 > > > To: dev@ignite.apache.org > > > Subject: Re: Handling slashes in cache names > > > > > > It also make sense to limit cache name length to reasonable length. > > > Because some File systems could have limitations on path length. > > > See: https://en.wikipedia.org/wiki/Filename#Length_restrictions > > > > > > On Tue, Dec 26, 2017 at 1:41 AM, Dmitriy Setrakyan < > > dsetrak...@apache.org> > > > wrote: > > > > > > > My preference would be to prohibit forward and backward slashes in > > cache > > > > names altogether, as they may create a false feeling of some > directory > > > > structure, which does not exist. We should also prohibit spaces as > > well. > > > > > > > > D. > > > > > > > > On Mon, Dec 25, 2017 at 7:09 AM, Stanislav Lukyanov < > > > > stanlukya...@gmail.com> > > > > wrote: > > > > > > > > > Hi all, > > > > > > > > > > I’m looking into https://issues.apache.org/jira/browse/IGNITE-7264 > , > > > and > > > > I > > > > > need some guidance on what’s the best way to approach it. > > > > > > > > > > The problem is that cache names are not restricted, but if > > persistence > > > is > > > > > enabled the cache needs to have a corresponding directory on the > file > > > > > system (“cache-…”) which can’t be created if the cache name > contains > > > > > certain characters (or a reserved system name). > > > > > > > > > > A straightforward approach would be to check if a cache name is > > allowed > > > > on > > > > > the local system
RE: Handling slashes in cache names
That’s interesting, thanks. So, do you think the locale-specific file separators should be banned as well? Handling all kinds of cases like this might be complicated. I’d rather use something else if the cache name is not a valid file name, a hash of the cache name. This way all corner cases can be handled at once. The algorithm would be 1) Check that cache name doesn’t contain banned characters 2) Try to create a Path for “cache-” 3) If failed, create a Path for “cache-” Stan From: Igor Sapego Sent: 26 декабря 2017 г. 17:59 To: dev@ignite.apache.org Subject: Re: Handling slashes in cache names There are also some international features that you might want to address. For example, instead of backslash some other characters may be used on Windows - ¥ on the Japanese version, ₩ on the Korean version. See [1] for more info. Here is the citation: Security Considerations for Character Sets in File Names Windows code page and OEM character sets used on Japanese-language systems contain the Yen symbol (¥) instead of a backslash (\). Thus, the Yen character is a prohibited character for NTFS and FAT file systems. When mapping Unicode to a Japanese-language code page, conversion functions map both backslash (U+005C) and the normal Unicode Yen symbol (U+00A5) to this same character. For security reasons, your applications should not typically allow the character U+00A5 in a Unicode string that might be converted for use as a FAT file name. [1] - https://msdn.microsoft.com/en-us/library/dd374047(v=vs.85).aspx Best Regards, Igor On Tue, Dec 26, 2017 at 5:01 PM, Dmitriy Setrakyanwrote: > Looks good to me. Is this going to be an exception on startup? If yes, is > it safe to release it, or should we wait till 3.0? > > On Tue, Dec 26, 2017 at 2:08 AM, Stanislav Lukyanov < > stanlukya...@gmail.com> > wrote: > > > Thanks for the feedback. > > > > It seems that another thing to handle is case-insensitive FS – “mycache” > > and “MyCache” is the same on Windows, so it might be reasonable to > disallow > > having two caches with names that are equal ignoring case. > > And one more thing is control characters – forbidding at least range of > > ASCII 0x00-0x20 seems reasonable. > > > > To summarize, a possible set of restrictions would be > > - Whitespace characters (via Character.isWhitespaceCharacter) > > - Control characters (via Character.isISOCharacter) > > - Slashes > > - Characters reserved in Windows (<>:"/\|?*) > > - Length (say, up to 255) > > - Distinct names of caches when ignoring case > > It seems reasonable to enforce that even regardless of persistence > > directories naming (AFAIU that’s what Dmitry meant by forbidding things > > altogether), so that’s what I’m going to do. > > Any concerns? > > Specifically, would it be OK from backward compatibility point of view to > > forbid all these characters now for all caches? > > > > Thanks, > > Stan > > > > > > From: Alexey Kuznetsov > > Sent: 26 декабря 2017 г. 7:51 > > To: dev@ignite.apache.org > > Subject: Re: Handling slashes in cache names > > > > It also make sense to limit cache name length to reasonable length. > > Because some File systems could have limitations on path length. > > See: https://en.wikipedia.org/wiki/Filename#Length_restrictions > > > > On Tue, Dec 26, 2017 at 1:41 AM, Dmitriy Setrakyan < > dsetrak...@apache.org> > > wrote: > > > > > My preference would be to prohibit forward and backward slashes in > cache > > > names altogether, as they may create a false feeling of some directory > > > structure, which does not exist. We should also prohibit spaces as > well. > > > > > > D. > > > > > > On Mon, Dec 25, 2017 at 7:09 AM, Stanislav Lukyanov < > > > stanlukya...@gmail.com> > > > wrote: > > > > > > > Hi all, > > > > > > > > I’m looking into https://issues.apache.org/jira/browse/IGNITE-7264, > > and > > > I > > > > need some guidance on what’s the best way to approach it. > > > > > > > > The problem is that cache names are not restricted, but if > persistence > > is > > > > enabled the cache needs to have a corresponding directory on the file > > > > system (“cache-…”) which can’t be created if the cache name contains > > > > certain characters (or a reserved system name). > > > > > > > > A straightforward approach would be to check if a cache name is > allowed > > > on > > > > the local system (e.g. via `Paths.get(name)`) and fail to create > cache > > if > > > > it isn’t, but I’m a bit concerned with the consistency of the > behavior > > > (the > > > > same cache name be allowed on one system and not on another). > > > > I think a better way would be to replace special characters (say, all > > > > non-alphanumeric characters) with underscores in file names (not > > changing > > > > the cache configuration). Would this be OK? Are there any risks I’m > not > > > > considering? > > > > > > > > WDYT? > > > > > > > > Thanks, > > > > Stan > > > > > > > > > > > > > > > -- > > Alexey Kuznetsov > > > > >
Re: Handling slashes in cache names
Cache name appears to me purely logical entity. Can we simply store cache ID in file system paths without adding any restrictions to cache names? On Wed, Dec 27, 2017 at 2:26 PM, Stanislav Lukyanovwrote: > Well, that’s my question too :) > Do we have any compatibility guidelines or other documents on what can or > cannot be in a minor/major release? > > Also, it might be helpful to add an environment variable (like > IGNITE_DISABLE_CACHE_NAME_RESTRICTIONS) to restore the old behavior, just > in case. > > Thanks, > Stan > > From: Dmitriy Setrakyan > Sent: 26 декабря 2017 г. 17:02 > To: dev@ignite.apache.org > Subject: Re: Handling slashes in cache names > > Looks good to me. Is this going to be an exception on startup? If yes, is > it safe to release it, or should we wait till 3.0? > > On Tue, Dec 26, 2017 at 2:08 AM, Stanislav Lukyanov < > stanlukya...@gmail.com> > wrote: > > > Thanks for the feedback. > > > > It seems that another thing to handle is case-insensitive FS – “mycache” > > and “MyCache” is the same on Windows, so it might be reasonable to > disallow > > having two caches with names that are equal ignoring case. > > And one more thing is control characters – forbidding at least range of > > ASCII 0x00-0x20 seems reasonable. > > > > To summarize, a possible set of restrictions would be > > - Whitespace characters (via Character.isWhitespaceCharacter) > > - Control characters (via Character.isISOCharacter) > > - Slashes > > - Characters reserved in Windows (<>:"/\|?*) > > - Length (say, up to 255) > > - Distinct names of caches when ignoring case > > It seems reasonable to enforce that even regardless of persistence > > directories naming (AFAIU that’s what Dmitry meant by forbidding things > > altogether), so that’s what I’m going to do. > > Any concerns? > > Specifically, would it be OK from backward compatibility point of view to > > forbid all these characters now for all caches? > > > > Thanks, > > Stan > > > > > > From: Alexey Kuznetsov > > Sent: 26 декабря 2017 г. 7:51 > > To: dev@ignite.apache.org > > Subject: Re: Handling slashes in cache names > > > > It also make sense to limit cache name length to reasonable length. > > Because some File systems could have limitations on path length. > > See: https://en.wikipedia.org/wiki/Filename#Length_restrictions > > > > On Tue, Dec 26, 2017 at 1:41 AM, Dmitriy Setrakyan < > dsetrak...@apache.org> > > wrote: > > > > > My preference would be to prohibit forward and backward slashes in > cache > > > names altogether, as they may create a false feeling of some directory > > > structure, which does not exist. We should also prohibit spaces as > well. > > > > > > D. > > > > > > On Mon, Dec 25, 2017 at 7:09 AM, Stanislav Lukyanov < > > > stanlukya...@gmail.com> > > > wrote: > > > > > > > Hi all, > > > > > > > > I’m looking into https://issues.apache.org/jira/browse/IGNITE-7264, > > and > > > I > > > > need some guidance on what’s the best way to approach it. > > > > > > > > The problem is that cache names are not restricted, but if > persistence > > is > > > > enabled the cache needs to have a corresponding directory on the file > > > > system (“cache-…”) which can’t be created if the cache name contains > > > > certain characters (or a reserved system name). > > > > > > > > A straightforward approach would be to check if a cache name is > allowed > > > on > > > > the local system (e.g. via `Paths.get(name)`) and fail to create > cache > > if > > > > it isn’t, but I’m a bit concerned with the consistency of the > behavior > > > (the > > > > same cache name be allowed on one system and not on another). > > > > I think a better way would be to replace special characters (say, all > > > > non-alphanumeric characters) with underscores in file names (not > > changing > > > > the cache configuration). Would this be OK? Are there any risks I’m > not > > > > considering? > > > > > > > > WDYT? > > > > > > > > Thanks, > > > > Stan > > > > > > > > > > > > > > > -- > > Alexey Kuznetsov > > > > > >
[GitHub] ignite pull request #3296: IGNITE-7304
GitHub user bbarin opened a pull request: https://github.com/apache/ignite/pull/3296 IGNITE-7304 I didn't figured out how to write integration tests without providing the AWS credentials. You can merge this pull request into a Git repository by running: $ git pull https://github.com/bbarin/ignite IGNITE-7304 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3296.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3296 commit f25c6ba62053c01bd06fc4afe166bdc4551901b3 Author: Bruno BarinDate: 2017-12-27T11:12:49Z Added AWS ELB-based ip finder. ---
[GitHub] ignite pull request #3295: IGNITE-7293
GitHub user devozerov opened a pull request: https://github.com/apache/ignite/pull/3295 IGNITE-7293 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-7293 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3295.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3295 commit 877218e0e185cdaf955293c92b16f8c2e1f391ef Author: devozerovDate: 2017-12-27T11:07:56Z Fixed incorrect schema ID calculation when not fields are set. ---
How to handle CacheLocalStore on clients node?
Hi, Igniters! I’m working on an implementation of 3rd party persistence solution. The solution is annotated by CacheLocalStore annotation. As far as I know, CacheLocalStore annotation means that server node persists its own primary and backup partitions (please correct me if I’m wrong). How about client nodes? It’s possible to define CacheStoreFactory in CacheConfiguration which creates CacheStoreStore even on clients node. But as far as I can see ‘read-through’ and ‘write-through’ methods aren’t called on clients nodes in case of using CacheLocalStore. It may be a problem because of CacheLocalStore which may require the creation of some temporary resources which are not needed on clients nodes. How must this situation be handled? I see some ways: 1) Ignite should validate this situation at cache creation on client node 2) CacheStoreFactory should throw some exception in case of creation CacheLocalStore on client nodes 3) CacheStoreFactory should create some stub of CacheLocalStore to do nothing Any thoughts? -- Best Regards, Vyacheslav D.
Re: Losing data during restarting cluster with persistence enabled
Hi, looks like there is no much profit when PDS throttling is enabled and tuned according to an article [1]. I’ve benchmarked the solutions with ‘put’ operation for 3 hours via Ignite Yardstick. I see quite similar results with the write-heavy pattern. Most time PDS works ~10% faster. Only one thing looks strange: PDS has degradation over time in comparison with RocksDB. [1] https://apacheignite.readme.io/docs/durable-memory-tuning On Wed, Dec 6, 2017 at 9:24 PM, Valentin Kulichenkowrote: > Vyacheslav, > > In this case community should definitely take a look and investigate. > Please share your results when you have a chance. > > -Val > > On Wed, Dec 6, 2017 at 1:45 AM, Vyacheslav Daradur > wrote: > >> Evgeniy, as far as I understand PDS and rebalancing are based on >> page-memory approach instead of entry-based 3rd Party Persistence, so >> I'm not sure how to extend rebalancing behavior properly. >> >> Dmitry, the performance is the only reason of why I try to solve >> rebalancing issue. >> I've benchmarked RocksDB as 3rd party persistence and PDS via Ignite >> Yardstick with "fsync" enabled in both cases. >> The result shows that PDS is twice slower on "put" operation on the >> single node, but I had had no time to do benchmarks on all sides. >> I'll try to do that next week and will share results if the community >> is interested. Maybe there will be no reason for using RocksDB. >> >> >> >> On Fri, Nov 24, 2017 at 4:58 PM, Dmitry Pavlov >> wrote: >> > Please see the discussion on the user list. It seems that the same >> happened >> > there: >> > >> > http://apache-ignite-users.70518.x6.nabble.com/Reassign- >> partitions-td7461.html#a7468 >> > >> > it contains examples when the data can diverge. >> > >> > пт, 24 нояб. 2017 г. в 16:42, Dmitry Pavlov : >> > >> >> If we compare native and 3rd party persistence (cache store): >> >> - Updating and reading data from DBMS is slower in most scenarios. >> >> - Non-clustered DBMS is a single point of failure, it is hard to scale. >> >> - Ignite SQL does not extend to External (3rd party persitsence) Cache >> >> Store (and queries ignore DBMS changes). >> >> >> >> >> >> Which is why I am wondering if Native persistence is applicable in this >> >> case decribed by Vyacheslav. >> >> >> >> пт, 24 нояб. 2017 г. в 12:23, Evgeniy Ignatiev < >> >> yevgeniy.ignat...@gmail.com>: >> >> >> >>> Sorry linked the wrong page, the latter url is not the example. >> >>> >> >>> >> >>> On 11/24/2017 1:12 PM, Evgeniy Ignatiev wrote: >> >>> > By the way I remembered that there is an annotation CacheLocalStore >> >>> > for marking exactly the CacheStore that is not distributed - >> >>> > >> >>> http://apache-ignite-developers.2346864.n4.nabble. >> com/CacheLocalStore-td734.html >> >>> > - here is short explanation and this - >> >>> > >> >>> https://github.com/gridgain/gridgain-advanced-examples/ >> blob/master/src/main/java/org/gridgain/examples/localstore/ >> LocalRecoverableStoreExample.java >> >>> > - is example implementation. >> >>> > >> >>> > >> >>> > On 11/23/2017 4:42 PM, Dmitry Pavlov wrote: >> >>> >> Hi Evgeniy, >> >>> >> >> >>> >> Technically it is, of course, possible, but still >> >>> >> - it is not simple at all >> >>> >> - IgniteCacheOffheapManager & IgniteWriteAheadLogManager are >> internal >> >>> >> APIs, >> >>> >> and community can change any APIs here in any time. >> >>> >> >> >>> >> Vyacheslav, >> >>> >> >> >>> >> Why Ignite Native Persistence is not suitable for this case? >> >>> >> >> >>> >> Sincerely, >> >>> >> Dmitriy Pavlov >> >>> >> >> >>> >> чт, 23 нояб. 2017 г. в 11:01, Evgeniy Ignatiev >> >>> >> > >>> >>> : >> >>> >>> As far as I remember, last webinar I heard on Ignite Native >> >>> Persistence >> >>> >>> - it actually exposes some interfaces like >> IgniteWriteAheadLogManager, >> >>> >>> PageStore, PageStoreManager, etc., with the file-based >> implementation >> >>> >>> provided by Ignite being only one possible approach, and users can >> >>> >>> create their own Native Persistence variations. At least that what >> has >> >>> >>> been said by Denis Magda at that time. >> >>> >>> >> >>> >>> May be creating own implementation of Ignite Native Persistence >> rather >> >>> >>> than CacheStore based persistence is an option here? >> >>> >>> >> >>> >>> On 11/23/2017 2:23 AM, Valentin Kulichenko wrote: >> >>> Vyacheslav, >> >>> >> >>> There is no way to do this and I'm not sure why you want to do >> this. >> >>> >>> Ignite >> >>> persistence was developed to solve exactly the problems you're >> >>> >>> describing. >> >>> Just use it :) >> >>> >> >>> -Val >> >>> >> >>> On Wed, Nov 22, 2017 at 12:36 AM, Vyacheslav Daradur < >> >>> >>> daradu...@gmail.com> >> >>> wrote: >> >>> >> >>> > Valentin, Evgeniy thanks for your help! >> >>> > >> >>> > Valentin,
[GitHub] ignite pull request #3294: IGNITE-7302
GitHub user alexpaschenko opened a pull request: https://github.com/apache/ignite/pull/3294 IGNITE-7302 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-7302 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3294.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3294 commit f6e982540e65ab17d439dba990794f35616a30dd Author: sboikovDate: 2017-08-30T09:45:40Z ignite-3478 commit 275a85db5cd6923b36126166ae99b15e876192be Author: sboikov Date: 2017-08-31T07:44:07Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit b7b9089f0102b8cab9942a9c887d93e9f26cc7d2 Author: sboikov Date: 2017-08-31T09:00:36Z disco cache cleanup commit 855c2d45794c300d41e386b4e6fa40736cc3e40d Author: sboikov Date: 2017-08-31T09:09:58Z Merge branch 'ignite-3478-1' into ignite-3478 commit 08be7310a93d3ce455215b97cf8ab1a2c3f0ab31 Author: sboikov Date: 2017-08-31T09:52:23Z ignite-3478 commit fce2e31f0fd2f4f6a9944422e40408a0c65cfe90 Author: sboikov Date: 2017-09-04T08:13:50Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit d3c049952384750c5543a9f88b383c033ef74096 Author: sboikov Date: 2017-09-04T08:52:11Z ignite-3478 commit e71ce1937a18dd32448e92b1038dc48d4cb6f8ab Author: sboikov Date: 2017-09-04T10:16:03Z ignite-3478 commit 5fac5b0965e97f8951e16e10ca9229a2e78ddb0c Author: sboikov Date: 2017-09-05T10:16:44Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java commit 2e0c9c08e046e8d6af1b5358d9053eae999b1fb4 Author: sboikov Date: 2017-09-05T11:30:55Z ignite-3478 commit e1e07ffdf2d711ba3e72f316f5a3970eff27372e Author: sboikov Date: 2017-09-05T11:31:14Z ignite-3478 commit cbada3934a386668da0b11d4de7d0f58a4d04dfe Author: sboikov Date: 2017-09-05T11:49:49Z ignite-3484 commit 5a82c68dcd1927bb6fded8b7def38c91ff6e145b Author: sboikov Date: 2017-09-05T11:59:49Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit bc9134c94b7a738dc1664e96ca6eabb059f1c268 Author: sboikov Date: 2017-09-05T12:03:39Z Merge branch 'ignite-3478' into ignite-3484 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/tree/AbstractDataInnerIO.java commit b4bfcde78825c6517232e49d389bdb5de19f05a9 Author: sboikov Date: 2017-09-05T12:27:51Z ignite-3484 commit 43834aaab9e2c3cd5fdd55289fdc4a9ff8ab6599 Author: sboikov Date: 2017-09-05T13:13:00Z ignite-3478 commit d1b828095713fcadfa260cf94fef01b42a1b12fd Author: sboikov Date: 2017-09-05T13:13:33Z Merge branch 'ignite-3478' into ignite-3484 commit 6be6779b6336c36cd71eef0a25199a6a877ce6b5 Author: sboikov Date: 2017-09-05T13:47:11Z ignite-3484 commit e3bba83256c1eb53c4b40fbd9ddba47fcf9d58d5 Author: sboikov Date: 2017-09-06T07:10:26Z ignite-3484 commit dd0afb28466094b801506da8afa3601bfaebd853 Author: sboikov Date: 2017-09-06T07:30:04Z ignite-3484 commit 27b87b413348b03986a463551db24b7726321732 Author: sboikov Date: 2017-09-06T08:19:18Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java commit dcaf8801accd6ee089849a82b2ccd558aec81895 Author: sboikov Date: 2017-09-06T08:19:30Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java commit c966451d0bf7059575de92bcfae43d72096ebce4 Author: sboikov Date: 2017-09-06T08:27:04Z Merge branch 'ignite-3478' into ignite-3484 commit 91b9911731a387a3199ddbbc22704bc14af09995 Author: sboikov Date: 2017-09-06T09:22:22Z ignite-3484 commit e40b4d9dcd6fe6c1cd2640bdd7116ca5a08ed781 Author: sboikov Date: 2017-09-07T09:12:32Z ignite-3484 commit 41a1c571e6ba1765941e2f1679dc4ac1582275c4 Author: sboikov Date: 2017-09-08T07:55:24Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3484 commit 1626130801dc330bcaf693f46906f6564cce6802 Author: sboikov Date:
[GitHub] ignite pull request #3293: IGNITE-7280
GitHub user alexpaschenko opened a pull request: https://github.com/apache/ignite/pull/3293 IGNITE-7280 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-7280 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3293.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3293 commit f6e982540e65ab17d439dba990794f35616a30dd Author: sboikovDate: 2017-08-30T09:45:40Z ignite-3478 commit 275a85db5cd6923b36126166ae99b15e876192be Author: sboikov Date: 2017-08-31T07:44:07Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit b7b9089f0102b8cab9942a9c887d93e9f26cc7d2 Author: sboikov Date: 2017-08-31T09:00:36Z disco cache cleanup commit 855c2d45794c300d41e386b4e6fa40736cc3e40d Author: sboikov Date: 2017-08-31T09:09:58Z Merge branch 'ignite-3478-1' into ignite-3478 commit 08be7310a93d3ce455215b97cf8ab1a2c3f0ab31 Author: sboikov Date: 2017-08-31T09:52:23Z ignite-3478 commit fce2e31f0fd2f4f6a9944422e40408a0c65cfe90 Author: sboikov Date: 2017-09-04T08:13:50Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit d3c049952384750c5543a9f88b383c033ef74096 Author: sboikov Date: 2017-09-04T08:52:11Z ignite-3478 commit e71ce1937a18dd32448e92b1038dc48d4cb6f8ab Author: sboikov Date: 2017-09-04T10:16:03Z ignite-3478 commit 5fac5b0965e97f8951e16e10ca9229a2e78ddb0c Author: sboikov Date: 2017-09-05T10:16:44Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java commit 2e0c9c08e046e8d6af1b5358d9053eae999b1fb4 Author: sboikov Date: 2017-09-05T11:30:55Z ignite-3478 commit e1e07ffdf2d711ba3e72f316f5a3970eff27372e Author: sboikov Date: 2017-09-05T11:31:14Z ignite-3478 commit cbada3934a386668da0b11d4de7d0f58a4d04dfe Author: sboikov Date: 2017-09-05T11:49:49Z ignite-3484 commit 5a82c68dcd1927bb6fded8b7def38c91ff6e145b Author: sboikov Date: 2017-09-05T11:59:49Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit bc9134c94b7a738dc1664e96ca6eabb059f1c268 Author: sboikov Date: 2017-09-05T12:03:39Z Merge branch 'ignite-3478' into ignite-3484 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/tree/AbstractDataInnerIO.java commit b4bfcde78825c6517232e49d389bdb5de19f05a9 Author: sboikov Date: 2017-09-05T12:27:51Z ignite-3484 commit 43834aaab9e2c3cd5fdd55289fdc4a9ff8ab6599 Author: sboikov Date: 2017-09-05T13:13:00Z ignite-3478 commit d1b828095713fcadfa260cf94fef01b42a1b12fd Author: sboikov Date: 2017-09-05T13:13:33Z Merge branch 'ignite-3478' into ignite-3484 commit 6be6779b6336c36cd71eef0a25199a6a877ce6b5 Author: sboikov Date: 2017-09-05T13:47:11Z ignite-3484 commit e3bba83256c1eb53c4b40fbd9ddba47fcf9d58d5 Author: sboikov Date: 2017-09-06T07:10:26Z ignite-3484 commit dd0afb28466094b801506da8afa3601bfaebd853 Author: sboikov Date: 2017-09-06T07:30:04Z ignite-3484 commit 27b87b413348b03986a463551db24b7726321732 Author: sboikov Date: 2017-09-06T08:19:18Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java commit dcaf8801accd6ee089849a82b2ccd558aec81895 Author: sboikov Date: 2017-09-06T08:19:30Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java commit c966451d0bf7059575de92bcfae43d72096ebce4 Author: sboikov Date: 2017-09-06T08:27:04Z Merge branch 'ignite-3478' into ignite-3484 commit 91b9911731a387a3199ddbbc22704bc14af09995 Author: sboikov Date: 2017-09-06T09:22:22Z ignite-3484 commit e40b4d9dcd6fe6c1cd2640bdd7116ca5a08ed781 Author: sboikov Date: 2017-09-07T09:12:32Z ignite-3484 commit 41a1c571e6ba1765941e2f1679dc4ac1582275c4 Author: sboikov Date: 2017-09-08T07:55:24Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3484 commit 1626130801dc330bcaf693f46906f6564cce6802 Author: sboikov Date:
[jira] [Created] (IGNITE-7320) Multilevel header in admin panel looks wierd in Safari and Edge
Alexander Kalinin created IGNITE-7320: - Summary: Multilevel header in admin panel looks wierd in Safari and Edge Key: IGNITE-7320 URL: https://issues.apache.org/jira/browse/IGNITE-7320 Project: Ignite Issue Type: Bug Components: wizards Reporter: Alexander Kalinin Assignee: Alexander Kalinin Priority: Minor -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] ignite pull request #3287: IGNITE-7287 Thin client: idle connection timeout
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3287 ---
[GitHub] ignite pull request #3292: IGNITE-6923 Optimize metrics calculation
GitHub user alex-plekhanov opened a pull request: https://github.com/apache/ignite/pull/3292 IGNITE-6923 Optimize metrics calculation You can merge this pull request into a Git repository by running: $ git pull https://github.com/alex-plekhanov/ignite IGNITE-6923 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3292.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3292 commit 6a887ada0b76d62c34b871f760f79838215e6cbc Author: Aleksey PlekhanovDate: 2017-12-19T14:44:06Z IGNITE-6923 Optimized nonHeapMemoryUsed commit d1742f7c598e8029db58f48b0b0ab3cf463a55d9 Author: Aleksey Plekhanov Date: 2017-12-20T16:13:34Z IGNITE-6923 getTotalPartitionsCount, getRebalancingPartitionsCount optimization commit 1bff257d4da4bdf33da1824a193ba6d202b8f9ac Author: Aleksey Plekhanov Date: 2017-12-22T08:56:36Z IGNITE-6923 Cluster metrics optimization (proxy class implemented) commit 4980e29e7145a8ea9f53e51ba7d0ae67f6709d1b Author: Aleksey Plekhanov Date: 2017-12-25T08:40:24Z IGNITE-6923 Partitions metrics optimization commit 6c53ac5a66cc951f8dfa12dcf2d9bc6953e3c6ed Author: Aleksey Plekhanov Date: 2017-12-26T12:57:01Z IGNITE-6923 Cache metrics optimization commit e77096ec6ec8c6b5128b83a5be41c61d73e9a169 Author: Aleksey Plekhanov Date: 2017-12-26T12:58:30Z IGNITE-6923 Cache metrics optimization commit 9bb904f445c558f5bf044d6cbafd2a9aeb9d4eb7 Author: Aleksey Plekhanov Date: 2017-12-27T07:48:56Z IGNITE-6923 Disable cache metrics update flag ---