Hi Ivan, yes, after the restart we need to re-use both folder name and UUID as consistent ID.
About returning exact directory name: I'm just guessing here, but the persistent store is not the only folder, there are several more folders, e.g. binary-meta, etc. About several nodes reusing same root folder, yes, the proposed solution will not works because 00 is hardcoded here. But I expect it will be too rare case. Sincerely, Dmitriy Pavlov пт, 15 мар. 2019 г. в 11:29, Павлухин Иван <vololo...@gmail.com>: > Hi Dmitriy, > > It looks like that case with only one node in working directory is > covered. But what is problem is solved? As I understand before fix in > such case node after restart uses proper folder and proper > consistentId. After fix we can specify automatically generated UUID in > configuration and everything will go smooth. Is it the main problem > case? If yes then such simplicity is very attractive. > > But honestly I do not fully understand why do we need to return a node > to node00-UUID directory after that directory was deleted? > > чт, 14 мар. 2019 г. в 17:21, Dmitriy Pavlov <dpav...@apache.org>: > > > > Hi, I've prepared demo PR for approach 1, just checking UUID and trying > to > > use node00-UUID. https://github.com/apache/ignite/pull/6266/ Actually > for > > this approach it is really simple, no API change is required. > > > > I guess the case is not very often because automatic folders assignment > > works well in most cases. > > > > WDYT? > > > > ср, 13 мар. 2019 г. в 09:42, Павлухин Иван <vololo...@gmail.com>: > > > > > Alex, > > > > > > Why do we need > > > > - check if consistent ID is set to a string 'nodeXX-UUID'. In this > case > > > > the consistent ID is set to UUID, and the storage folder is chosen > > > > according to the proper rules. This change has a minimal chance to > affect > > > > current users because it's unlikely that somebody is using > auto-generated > > > > folder naming scheme as consistent ID. > > > ? > > > It looks hacky as well. The thing I do not like here is that a > > > consistentId specified in configuration is not a consistentId used by > > > a node sometimes. > > > > > > Can we go just with > > > > - either check if consistent ID is an instance of UUID and then take > the > > > > appropriate folder. This approach is straightforward, but may affect > > > > current users > > > ? > > > > > > And as a last chance a user will have a possibility to rename a > directory. > > > > > > > > > вт, 12 мар. 2019 г. в 19:33, Alexey Goncharuk < > alexey.goncha...@gmail.com > > > >: > > > > > > > > Igniters, > > > > > > > > I came across the same issue during development and found no sane > > > > workaround for this issue. I believe the solution should be as > simple as > > > > possible because we are already adding a warning to let users know > that > > > it > > > > is good to specify a consistent ID in production deployments. > > > > > > > > As for the current solution, I do not like adding a new configuration > > > > property like 'storageFolder' because it's another way to shoot > yourself > > > in > > > > the leg (e.g. different nodes have different consistent IDs but > > > configured > > > > to have the same storage folder). > > > > Why can't we: > > > > - either check if consistent ID is an instance of UUID and then > take the > > > > appropriate folder. This approach is straightforward, but may affect > > > > current users > > > > - check if consistent ID is set to a string 'nodeXX-UUID'. In this > case > > > > the consistent ID is set to UUID, and the storage folder is chosen > > > > according to the proper rules. This change has a minimal chance to > affect > > > > current users because it's unlikely that somebody is using > auto-generated > > > > folder naming scheme as consistent ID. > > > > > > > > Thoughts? > > > > > > > > вт, 12 мар. 2019 г. в 16:51, Dmitriy Pavlov <dpav...@apache.org>: > > > > > > > > > Hi Igniters, > > > > > > > > > > A full description can be found at wiki page > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-SubfoldersGeneration > > > > > > > > > > I like the idea of introducing a property like nodeStorageName to > > > specify > > > > > exactly name of a directory to use. For example, it could have > higher > > > > > priority then IgniteConfiguration.getConsistentId(). It will > impact the > > > > > mentioned algorithm, but it seems to be easier to understand by > users. > > > > > > > > > > If nobody else minds, I will try to implement this idea. > > > > > > > > > > Sincerely, > > > > > Dmitriy Pavlov > > > > > > > > > > > > > > > сб, 2 мар. 2019 г. в 08:33, Павлухин Иван <vololo...@gmail.com>: > > > > > > > > > > > Dmitriy, > > > > > > > > > > > > You wrote > > > > > > > we need an order to scan and lock random-UUID based folders. > > > > > > > > > > > > It would be great if you provide a discussion about that order to > > > > > > complete the picture. Currently I cannot understand why the > order is > > > > > > important. > > > > > > > > > > > > Also, couple of raw thoughts: > > > > > > 1. Can we extend a directory lookup procedure when consistent id > is > > > > > > specified to check nodeXX-consistentId directories as well? > > > > > > 2. Can we introduce a property like nodeStorageName to specify > exact > > > > > > name of a directory to use? It looks like a straightforward and > > > > > > universal workaround. Or is node index better in some sense? Why? > > > > > > > > > > > > Please, share your thoughts. > > > > > > > > > > > > чт, 28 февр. 2019 г. в 17:43, Dmitriy Pavlov <dpav...@apache.org > >: > > > > > > > > > > > > > > Hi Ivan, > > > > > > > > > > > > > > Yes, you catch me, I'm a little bit cheating with lazy > consensus on > > > > > code > > > > > > > modification without providing a PR because I was expecting > that > > > nobody > > > > > > > comes to discussion. I will prepare PR shortly. And since we > anyway > > > > > have > > > > > > a > > > > > > > discussion, I will not apply anything by lazy approval. > > > > > > > > > > > > > > - storageNodeIndex without consistent ID will not work. > > > > > > > cfg.getDataStorageConfiguration().setNodeIdx() will be > required > > > only > > > > > for > > > > > > > case we have consistent ID. > > > > > > > > > > > > > > Hi Stanislav, > > > > > > > > > > > > > > We can't use only consistent ID because > > > > > > > > > > > > > > 1) we need an order to scan and lock random-UUID based folders. > > > Node > > > > > > index > > > > > > > provides the order of scan. I can find the corresponding > > > discussion, > > > > > but > > > > > > I > > > > > > > guess it is not needed. > > > > > > > 2) we need to separate backward compatible folders from new > > > random-UUID > > > > > > > based folders. Using UUID as folder will not allow us to scan > only > > > new > > > > > > name > > > > > > > format folders. > > > > > > > > > > > > > > I guess specifying node index is a quite rare case and good > JavaDoc > > > > > will > > > > > > > always help. > > > > > > > > > > > > > > DataStorageConfiguration().setNodeIdx() JavaDoc may include > > > following > > > > > > > notes: > > > > > > > Node index used for persistent store folders in case several > nodes > > > > > reuse > > > > > > > one persistent store root folder. > > > > > > > > > > > > > > Sincerely, > > > > > > > Dmitriy Pavlov > > > > > > > > > > > > > > чт, 28 февр. 2019 г. в 08:03, Павлухин Иван < > vololo...@gmail.com>: > > > > > > > > > > > > > > > Dmitiy, > > > > > > > > > > > > > > > > Could please clarify one thing: > > > > > > > > 1. Will it be enough to use only storageNodeIndex in order to > > > reuse > > > > > > > > the same persistence folders when consistentId is > auto-generated? > > > > > E.g. > > > > > > > > I have a configuration with storageNodeIndex=1 and without > > > explicitly > > > > > > > > specified consistentId, will the node after restart use the > same > > > > > > > > persistence folder as before restart? > > > > > > > > > > > > > > > > Also a side note: > > > > > > > > > Please share your vision. I'm going to apply this change by > > > lazy > > > > > > > > consensus > > > > > > > > in 3 days. > > > > > > > > What do you mean by "apply"? I have not seen any PR yet. > > > > > > > > > > > > > > > > ср, 27 февр. 2019 г. в 21:12, Dmitriy Pavlov < > dpav...@apache.org > > > >: > > > > > > > > > > > > > > > > > > Hi Igniters, > > > > > > > > > > > > > > > > > > I would like to fix the issue > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-11432 about > > > > > specifying > > > > > > some > > > > > > > > > previous randomly generated UUID as a new consistent ID. > Folder > > > > > > > > generation > > > > > > > > > algorithm here ( > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood > > > > > > > > ) > > > > > > > > > allows two options > > > > > > > > > -node00+random UUID > > > > > > > > > - consistendId > > > > > > > > > > > > > > > > > > I would like to add to Ignite configuration new property > > > nodeIndex > > > > > in > > > > > > > > > addition to consistent Id. New Property will be named as > > > > > > > > storageNodeIndex, > > > > > > > > > int, zero-based. > > > > > > > > > This will add the third option of subfolders processing: > > > > > > > > > node{storageNodeIndex}+consistentID > > > > > > > > > > > > > > > > > > Please share your vision. I'm going to apply this change by > > > lazy > > > > > > > > consensus > > > > > > > > > in 3 days. > > > > > > > > > > > > > > > > > > Sincerely, > > > > > > > > > Dmitriy Pavlov > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Best regards, > > > > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Best regards, > > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > > > > > > > > -- > > > Best regards, > > > Ivan Pavlukhin > > > > > > > -- > Best regards, > Ivan Pavlukhin >
