gianm commented on issue #17481: URL: https://github.com/apache/druid/issues/17481#issuecomment-2498706976
I've reviewed some of the initial PRs and I like the direction this is going. As to this piece: > The more traditional way to achieve this is with materialized views, however we decided to implement projections instead for a number of reasons. Materialized views are typically done as separate physical tables, and in a database like Druid, would bring with them a lot of very complicated synchronization problems to solve. Projections on the other hand, live inside of a regular Druid segment, and as such are constrained to be within the same interval as the segment. My 2ยข is that the 3 most common approaches I've seen to managing materializations (whether they are called "materialized views" or "projections") are: - storing the materialized view in a separate "hidden" table that is refreshed periodically (sometimes in concert with the above) - storing the materialized view in a separate "hidden" table that is incrementally updated using something like triggers - embedding the materialized view within the same storage as the base table (as we're doing here) We're starting with the last one here, which is a good place to start, because it has the nicest operational properties. "Hidden" tables are potentially more flexible, but have operational downsides, such as latency and cost in keeping them up to date. Perhaps we'll want to introduce a feature like that in the future, but it'd be a different user-facing feature as well as being a different internal implementation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
