clintropolis commented on code in PR #19845:
URL: https://github.com/apache/druid/pull/19845#discussion_r3834650261
##########
server/src/main/java/org/apache/druid/catalog/model/table/DatasourceDefn.java:
##########
@@ -107,18 +106,6 @@ public void validate(ResolvedTable table)
super.validate(table);
final DatasourceBaseTableMetadata baseTable =
table.decodeProperty(BASE_TABLE_PROPERTY);
if (baseTable != null) {
Review Comment:
So yes, this PR opens up a bit of a design question that will need resolved
before we document this stuff. You're right that CompactionTask will skip
downloading the segments and treats baseTable spec if defined in an inline
config as authority if it is set, my current thinking is that this is the
equivalent of a catalog sealed table. I think if we want a way to force it to
scan segments to blend a config we should add an explicit new config option for
it, which could do the same transformation that this PR is doing of adding
additional columns that it discovers.
Mechanically i think it is consistent? Like even this PR the baseTable spec
is still the authority if set, it just allows adding stuff to the catalog
supplied definition.
Catalog driven compaction config currently hasn't been integrated with the
baseTable stuff I think, i would imagine it would want a similar flag to set to
match sealed or not? Also catalog compaction config is not integrated with
reindexing templates, so there are some compaction related designs to finish in
general I think.
I'm not sure any of these things necessarily need resolved in this PR, just
before we document it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
