kbendick commented on issue #2093: URL: https://github.com/apache/iceberg/issues/2093#issuecomment-762566233
I agree that such a query with several joins is likely not practical to be performing regularly. Can you detail more how you intend to use this partition creation / update time info? Is this possibly as a trigger for a batch job - for example, a common pattern would be to continuously write to a table that is partitioned by hour from a streaming job (say, using Apache Flink as the query engine), and then when the new hours partition is committed a batch job would be triggered to process it. I’m not opposed to adding such metadata, though there would be better people than me to ask and it’s likely something that should be brought up on the dev mailing list. However, if you detail your use case (as well as possibly the intended writing situation / query engine), it’s very possible that other users are already handling this use case with some existing pattern that is already supported. Or it’s possible that your use case is new / not currently handled and then adding this metadata might be needed. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
