Agreed that the probability of collision is low, but it exists and there is no way to check if it kicks in if we don't have that information centralized. For wraparounds, we at least have a way of detecting that we reached the maximum value, but granted that if we reach the maximum value, then we may have duplicates as we reset the counter.
Assuming that we still keep ledger metadata in the metadata service and it is keyed by ledger id, we can check whether the ledger id is taken upon writing the ledger metadata, can't we? -Flavio > On 26 Oct 2016, at 16:41, Venkateswara Rao Jujjuri <jujj...@gmail.com> wrote: > > Flavio, > > In theory you are right. But in practice it is virtually unique. > RFC 4122 (https://www.ietf.org/rfc/rfc4122.txt) talks about multiple ways > to generate virtually unique ID. > Even in the sequence-id we have a theoretical chance of wrapping. So if we > are worried about duplicates, > we need a way to handle it. But in both cases (64 bit long or 128 bit uuid) > collisions are impractical. > > Also I am not advocating to get rid of metadata server completely, Just > saying for this purpose we don't need to. > Thanks, > JV > > > On Wed, Oct 26, 2016 at 3:43 AM, Flavio Junqueira <f...@apache.org> wrote: > >> Hi JV, >> >> Do I understand correctly that you're proposing to generate 128-bit UUIDs >> locally to use as ledger ids? If so, I'm concerned that even if the >> probability of collision is low, there is still a chance of having >> collisions with no way to verify in the case you aren't relying on a >> metadata server. >> >> I feel that I'm missing something here because you can't completely get >> rid of the metadata server without getting in trouble or at least without >> deeper changes. I'd appreciate if you could clarify, please. >> >> -Flavio >> >>> On 26 Oct 2016, at 00:39, Venkateswara Rao Jujjuri <jujj...@gmail.com> >> wrote: >>> >>> We are using 64 bit ledger ids internally right now, but the ledger id is >>> supported by the application/caller. >>> We have extended Hierarchical ledger manger to Long hierarchical ledger >>> manger for this. >>> >>> Ultimately we would like to move to 128 bit UUID as the ledger id. That >>> makes ledgers unique without >>> the need of centralized ZK/metadata server. >>> >>> On Tue, Oct 25, 2016 at 4:11 PM, Sijie Guo <si...@apache.org> wrote: >>> >>>> *Problem: * >>>> >>>> Currently the ledger id is long, which it should be 64-bits. However >>>> currently bookkeeper only can generate 32-bits ledger id as zookeeper's >>>> sequence znode only produce 32-bits. >>>> >>>> This problem was basically raised before at BOOKKEEPER-421. Jiannan has >>>> already done fair amount of work on this and there were several patches >> for >>>> it. >>>> >>>> This email thread is to start the discussion for 64-bits ledger id >> support >>>> in bookkeeper. >>>> >>>> *Discuss*: >>>> >>>> Based on bookkeeper-421, the changes will relatively happen in following >>>> places. Assume the metadata store is ZooKeeper. >>>> >>>> >>>> 1. How to generate 64-bits ledger id? (64 Bits Ledger ID Generation >>>> <https://issues.apache.org/jira/browse/BOOKKEEPER-552>) >>>> 2. How to store the 64-bits ledger id in zookeeper? (New LedgerManager >>>> for 64 Bits Ledger ID Management in ZooKeeper >>>> <https://issues.apache.org/jira/browse/BOOKKEEPER-553>) >>>> 3. How can the garbage collect handle correctly with 64-bits ledger >> id? >>>> ( >>>> https://issues.apache.org/jira/browse/BOOKKEEPER-553? >>>> focusedCommentId=13558192&page=com.atlassian.jira. >>>> plugin.system.issuetabpanels:comment-tabpanel#comment-13558192 >>>> ) >>>> 4. How can we upgrade current HierarchicalLedgerManager to support >>>> 64-bits. [??] >>>> >>>> Feel free to take a look at those tickets and make any proposals. >>>> >>>> - Sijie >>>> >>> >>> >>> >>> -- >>> Jvrao >>> --- >>> First they ignore you, then they laugh at you, then they fight you, then >>> you win. - Mahatma Gandhi >> >> > > > -- > Jvrao > --- > First they ignore you, then they laugh at you, then they fight you, then > you win. - Mahatma Gandhi
