Hi, I've prepared demo PR for approach 1, just checking UUID and trying to use node00-UUID. https://github.com/apache/ignite/pull/6266/ Actually for this approach it is really simple, no API change is required.
I guess the case is not very often because automatic folders assignment works well in most cases. WDYT? ср, 13 мар. 2019 г. в 09:42, Павлухин Иван <vololo...@gmail.com>: > Alex, > > Why do we need > > - check if consistent ID is set to a string 'nodeXX-UUID'. In this case > > the consistent ID is set to UUID, and the storage folder is chosen > > according to the proper rules. This change has a minimal chance to affect > > current users because it's unlikely that somebody is using auto-generated > > folder naming scheme as consistent ID. > ? > It looks hacky as well. The thing I do not like here is that a > consistentId specified in configuration is not a consistentId used by > a node sometimes. > > Can we go just with > > - either check if consistent ID is an instance of UUID and then take the > > appropriate folder. This approach is straightforward, but may affect > > current users > ? > > And as a last chance a user will have a possibility to rename a directory. > > > вт, 12 мар. 2019 г. в 19:33, Alexey Goncharuk <alexey.goncha...@gmail.com > >: > > > > Igniters, > > > > I came across the same issue during development and found no sane > > workaround for this issue. I believe the solution should be as simple as > > possible because we are already adding a warning to let users know that > it > > is good to specify a consistent ID in production deployments. > > > > As for the current solution, I do not like adding a new configuration > > property like 'storageFolder' because it's another way to shoot yourself > in > > the leg (e.g. different nodes have different consistent IDs but > configured > > to have the same storage folder). > > Why can't we: > > - either check if consistent ID is an instance of UUID and then take the > > appropriate folder. This approach is straightforward, but may affect > > current users > > - check if consistent ID is set to a string 'nodeXX-UUID'. In this case > > the consistent ID is set to UUID, and the storage folder is chosen > > according to the proper rules. This change has a minimal chance to affect > > current users because it's unlikely that somebody is using auto-generated > > folder naming scheme as consistent ID. > > > > Thoughts? > > > > вт, 12 мар. 2019 г. в 16:51, Dmitriy Pavlov <dpav...@apache.org>: > > > > > Hi Igniters, > > > > > > A full description can be found at wiki page > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-SubfoldersGeneration > > > > > > I like the idea of introducing a property like nodeStorageName to > specify > > > exactly name of a directory to use. For example, it could have higher > > > priority then IgniteConfiguration.getConsistentId(). It will impact the > > > mentioned algorithm, but it seems to be easier to understand by users. > > > > > > If nobody else minds, I will try to implement this idea. > > > > > > Sincerely, > > > Dmitriy Pavlov > > > > > > > > > сб, 2 мар. 2019 г. в 08:33, Павлухин Иван <vololo...@gmail.com>: > > > > > > > Dmitriy, > > > > > > > > You wrote > > > > > we need an order to scan and lock random-UUID based folders. > > > > > > > > It would be great if you provide a discussion about that order to > > > > complete the picture. Currently I cannot understand why the order is > > > > important. > > > > > > > > Also, couple of raw thoughts: > > > > 1. Can we extend a directory lookup procedure when consistent id is > > > > specified to check nodeXX-consistentId directories as well? > > > > 2. Can we introduce a property like nodeStorageName to specify exact > > > > name of a directory to use? It looks like a straightforward and > > > > universal workaround. Or is node index better in some sense? Why? > > > > > > > > Please, share your thoughts. > > > > > > > > чт, 28 февр. 2019 г. в 17:43, Dmitriy Pavlov <dpav...@apache.org>: > > > > > > > > > > Hi Ivan, > > > > > > > > > > Yes, you catch me, I'm a little bit cheating with lazy consensus on > > > code > > > > > modification without providing a PR because I was expecting that > nobody > > > > > comes to discussion. I will prepare PR shortly. And since we anyway > > > have > > > > a > > > > > discussion, I will not apply anything by lazy approval. > > > > > > > > > > - storageNodeIndex without consistent ID will not work. > > > > > cfg.getDataStorageConfiguration().setNodeIdx() will be required > only > > > for > > > > > case we have consistent ID. > > > > > > > > > > Hi Stanislav, > > > > > > > > > > We can't use only consistent ID because > > > > > > > > > > 1) we need an order to scan and lock random-UUID based folders. > Node > > > > index > > > > > provides the order of scan. I can find the corresponding > discussion, > > > but > > > > I > > > > > guess it is not needed. > > > > > 2) we need to separate backward compatible folders from new > random-UUID > > > > > based folders. Using UUID as folder will not allow us to scan only > new > > > > name > > > > > format folders. > > > > > > > > > > I guess specifying node index is a quite rare case and good JavaDoc > > > will > > > > > always help. > > > > > > > > > > DataStorageConfiguration().setNodeIdx() JavaDoc may include > following > > > > > notes: > > > > > Node index used for persistent store folders in case several nodes > > > reuse > > > > > one persistent store root folder. > > > > > > > > > > Sincerely, > > > > > Dmitriy Pavlov > > > > > > > > > > чт, 28 февр. 2019 г. в 08:03, Павлухин Иван <vololo...@gmail.com>: > > > > > > > > > > > Dmitiy, > > > > > > > > > > > > Could please clarify one thing: > > > > > > 1. Will it be enough to use only storageNodeIndex in order to > reuse > > > > > > the same persistence folders when consistentId is auto-generated? > > > E.g. > > > > > > I have a configuration with storageNodeIndex=1 and without > explicitly > > > > > > specified consistentId, will the node after restart use the same > > > > > > persistence folder as before restart? > > > > > > > > > > > > Also a side note: > > > > > > > Please share your vision. I'm going to apply this change by > lazy > > > > > > consensus > > > > > > in 3 days. > > > > > > What do you mean by "apply"? I have not seen any PR yet. > > > > > > > > > > > > ср, 27 февр. 2019 г. в 21:12, Dmitriy Pavlov <dpav...@apache.org > >: > > > > > > > > > > > > > > Hi Igniters, > > > > > > > > > > > > > > I would like to fix the issue > > > > > > > https://issues.apache.org/jira/browse/IGNITE-11432 about > > > specifying > > > > some > > > > > > > previous randomly generated UUID as a new consistent ID. Folder > > > > > > generation > > > > > > > algorithm here ( > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood > > > > > > ) > > > > > > > allows two options > > > > > > > -node00+random UUID > > > > > > > - consistendId > > > > > > > > > > > > > > I would like to add to Ignite configuration new property > nodeIndex > > > in > > > > > > > addition to consistent Id. New Property will be named as > > > > > > storageNodeIndex, > > > > > > > int, zero-based. > > > > > > > This will add the third option of subfolders processing: > > > > > > > node{storageNodeIndex}+consistentID > > > > > > > > > > > > > > Please share your vision. I'm going to apply this change by > lazy > > > > > > consensus > > > > > > > in 3 days. > > > > > > > > > > > > > > Sincerely, > > > > > > > Dmitriy Pavlov > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Best regards, > > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > > > > > > > -- > > > > Best regards, > > > > Ivan Pavlukhin > > > > > > > > > > > -- > Best regards, > Ivan Pavlukhin >
