AkshayUpadhyay2411 opened a new pull request, #19820:
URL: https://github.com/apache/pinot/pull/19820
Fixes #19819
## Summary
`SegmentGenerationAndPushTask` ignores an explicitly configured
`append.uuid.to.segment.name`, so deterministic
segment names (`inputFile`, `simple`, `normalizedDate`) cannot be used with
this task.
Two places overrode the configured value (both introduced in #10335):
- **Generator:** unconditionally put top-level
`append.uuid.to.segment.name=true` into every task. It now defaults to
`true` only when the batch config has not set it, either top-level or as
`segmentNameGenerator.configs.append.uuid.to.segment.name` (the prefixed
value is carried into the top-level key).
- **Executor:** unconditionally overwrote the prefixed generator config with
the top-level value (default `false`).
It now only falls back to the top-level key when the prefixed config is
absent.
## Compatibility
No config keys are added, removed or renamed, and the default stays `true`
for generated tasks. The only change is
that an explicit value is now respected:
- unset: unchanged (`true`)
- `segmentNameGenerator.configs.append.uuid.to.segment.name`: now honored;
wins over the top-level key
- top-level `append.uuid.to.segment.name`: now honored when the prefixed key
is absent
- tasks submitted without either key: unchanged (`false`)
Tables that already set `false` (which previously had no effect) will start
producing deterministic segment names. With
`false`, make sure the name generator yields unique names per input,
otherwise segments are overwritten. `FIXED` ignores
the setting.
## Testing
- `SegmentGenerationAndPushTaskGeneratorTest` (new): default, explicit
top-level `false`, explicit prefixed `false`,
and a generator -> executor round trip.
- `SegmentGenerationAndPushTaskExecutorTest`: precedence and fallback.
- Verified the new tests fail without the fix.
- `./mvnw spotless:apply license:format checkstyle:check license:check -pl
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks`
## Related
#16909 (open) changes the same generator line to read a top-level batch key
with a default of `false`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
