adam-christian-software commented on code in PR #1189: URL: https://github.com/apache/polaris/pull/1189#discussion_r2201651127
########## persistence/nosql/idgen/README.md: ########## @@ -0,0 +1,54 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +# Unique ID generation framework and monotonic clock + +Provides a framework and implementations for unique ID generation, including a monotonically increasing timestamp/clock +source. + +Provides a +[Snowflake-IDs](https://medium.com/@jitenderkmr/demystifying-snowflake-ids-a-unique-identifier-in-distributed-computing-72796a827c9d) +implementation. + +Consuming code must only leverage the `IdGenerator` interface. + +## Snowflake ID source + +The Snowflake ID source is configurable for each backend instance, but cannot be modified for an existing backend +instance to prevent ID conflicts. + +The epoch of these timestamps is 2025-03-01-00:00:00.0 GMT. Timestamps occupy 41 bits at +millisecond precision, which lasts for about 69 years. Node-IDs are 10 bits, which allows 1024 concurrently active +"JVMs running Polaris". 12 bits are used by the sequence number, which then allows each node to generate 4096 IDs per +millisecond. One bit is reserved for future use. + +Node IDs are leased by every "JVM running Polaris" for a period of time. The ID generator implementation guarantees +that no IDs will be generated for a timestamp that exceeds the "lease time". Leases can be extended. The implementation +leverages atomic database operations (CAS) for the lease implementation. + +ID generators must not use timestamps before or after the lease period nor must they re-use an older timestamp. This +requirement is satisfied using a monotonic clock implementation. + +## Code structure Review Comment: We could probably mention mocks here as well. ########## persistence/nosql/async/README.md: ########## @@ -0,0 +1,31 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +# Async execution API + +Provides an abstraction to submit asynchronous tasks, optionally with a delay or delay + repetition and implementations +based on Java's `ThreadPoolExecutor` and Vert.X. + +## Code structure + +The code is structured into multiple modules. Consuming code should almost always pull in only the API module. + +* `polaris-async-api` provides the necessary Java interfaces and immutable types. +* `polaris-async-java` implementation leveraging `CompletableFuture.delayedExecutor` for delayed/scheduled invocations. +* `polaris-async-vertx` implementation leveraging Vert.X for delayed/scheduled invocations. Review Comment: Just checking, is there a reason that we decided to have both the Java & VertX approaches here? ########## persistence/nosql/idgen/README.md: ########## @@ -0,0 +1,54 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +# Unique ID generation framework and monotonic clock + +Provides a framework and implementations for unique ID generation, including a monotonically increasing timestamp/clock +source. + +Provides a +[Snowflake-IDs](https://medium.com/@jitenderkmr/demystifying-snowflake-ids-a-unique-identifier-in-distributed-computing-72796a827c9d) +implementation. + +Consuming code must only leverage the `IdGenerator` interface. + +## Snowflake ID source + +The Snowflake ID source is configurable for each backend instance, but cannot be modified for an existing backend +instance to prevent ID conflicts. + +The epoch of these timestamps is 2025-03-01-00:00:00.0 GMT. Timestamps occupy 41 bits at +millisecond precision, which lasts for about 69 years. Node-IDs are 10 bits, which allows 1024 concurrently active +"JVMs running Polaris". 12 bits are used by the sequence number, which then allows each node to generate 4096 IDs per +millisecond. One bit is reserved for future use. + +Node IDs are leased by every "JVM running Polaris" for a period of time. The ID generator implementation guarantees +that no IDs will be generated for a timestamp that exceeds the "lease time". Leases can be extended. The implementation +leverages atomic database operations (CAS) for the lease implementation. + +ID generators must not use timestamps before or after the lease period nor must they re-use an older timestamp. This +requirement is satisfied using a monotonic clock implementation. + +## Code structure + Review Comment: Just following the pattern here of other modules. Do we need a store-nosql module? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@polaris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
