gianm commented on issue #10283: URL: https://github.com/apache/druid/issues/10283#issuecomment-677859327
This one is interesting. Our SQL metadata is based on doing segment metadata queries. (We look at all the active segments and build a table schema based on what columns are found.) Therefore, as far as the SQL API is concerned, datasources don't exist until they actually have some segments. You should be able to query the datasource `ds` once it has some data, i.e., once the supervisor launches tasks and they read something. It doesn't need to be published. Data that's currently only available from the realtime indexing system is still good enough to update the SQL metadata. Would that be good enough in your case? Beyond that, it would be tough to do a more fundamental fix without making major changes to how SQL metadata works in Druid. You could imagine defining what tables _should be_ rather than having the SQL layer observe what they _are_. This would be in line with what a lot of RDBMSes do (CREATE TABLE, ALTER TABLE, etc). It's something that does have its uses. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
