georgelza commented on issue #3344: URL: https://github.com/apache/polaris/issues/3344#issuecomment-3713391021
hi there... my view... imagine downloading the NYC Yellow Taxi data set and placing it in <bucket>/NYC/YellowTaxi/year=<year>/month=<month> then pointing the crawler at <bucket>/NYC/YellowTaxi and saying, that table is called YellowTaxy and I want a Iceberg based table compiled from the Parquet files or I want it build as Paimon table. Now build the tree... Then maintain... configure a Lambda even on the S3 bucket that fires every time a Put is executed in any directory in down from the original root provided and update as required. With this now comes normal engine access that can access/write data out updating the table created, or data can be brought in via parquet file drops. G -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
