ozankabak commented on PR #10590: URL: https://github.com/apache/datafusion/pull/10590#issuecomment-2131154079
The aim has been to make the `COPY` and `CREATE EXTERNAL TABLE` statements use the same `OPTIONS` syntax. The PR you mention was actually a part of multi-PR effort to make this so and arrive at a unified, consistent syntax. I should have given the example using the `CREATE EXTERNAL TABLE` statement -- using `COPY` misled you. However I'm not sure if we are at 100% on this yet -- e.g. I don't know if we write a `CREATE EXTERNAL TABLE` query today with `OPTIONS ( 'format.compression::col1' 'zstd(5)')` whether the metadata will be set correctly. Nevertheless, even if it doesn't, it should/will shortly. So, I'm not sure if we are doing > Don't attach metadata directly to columns (i.e., the current DataFusion approach) If so, that's certainly not the end state we want to be in. This is why I referred to the proposed `METADATA` syntax as a shortcut (relative to doing it within `OPTIONS`) in Option 3. We will certainly extend the `OPTIONS` mechanism to cover this case if it doesn't already, but it may also be worthwhile to have a shortcut way (e.g. `METADATA`). I will take a look at how other systems approach this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
