psolomin commented on issue #26429: URL: https://github.com/apache/beam/issues/26429#issuecomment-1544796438
@mosche > Do you have specific use cases in mind? Yeah, let me name some: - read from a Glue table backed by files in AWS S3 - FileIO will actually read files Glue catalog points to - read from a Glue table backed by AWS RDS (SQL) table - JdbcIO will actually read a table Glue catalog has Writing is more tricky cause in that case Beam will need to edit Glue catalog objects > Glue preparing that data itself "Glue" is actually multiple things: catalog (similar to Hive Metastore), Glue jobs (AWS proprietary version of serverless Spark), Glue crawlers, etc. For now this feature request is about Glue catalog only. > Iceberg That one will be very useful, yes. Trino, Flink and Spark already have Iceberg support. Adding support for Iceberg will be more value added, I would say. And, as I remember, Iceberg can work without catalog or Hive metastore, and use table locations where both data & metadata is stored. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
