s0l0ist commented on code in PR #2899:
URL:
https://github.com/apache/datafusion-ballista/pull/2899#discussion_r4644270317
##########
ballista/scheduler/src/scheduler_server/mod.rs:
##########
@@ -98,7 +98,7 @@ pub trait JobIdGenerator: Sync + Send {
/// A monotonically increasing sortable snowflake job id generator
/// which does not capture machine id as part result.
pub(crate) struct DefaultJobGenerator {
- generator: AtomicSnowflakeGenerator<SnowflakeMastodonId, MonotonicClock>,
+ generator: AtomicSnowflakeGenerator<SnowflakeMastodonId, CachedClock>,
Review Comment:
You're right- this is a behavioral change. If the wall clock moves backward,
the generator will async sleep until it catches up. The tradeoff is closer
alignment with wall time across all generated IDs at scale vs the cost of
potential stalls after backward clock corrections.
Agreed, I'll change to `DirectClock`. I didn't considered the throughput
here and chose `CachedClock` because it seemed like the closest replacement for
`MonotonicClock`.
It's not terribly hard for me to create an additional pair of clocks that
are strictly monotonically increasing. LMK if this becomes an issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]