davisp opened a new issue, #10589: URL: https://github.com/apache/datafusion/issues/10589
### Is your feature request related to a problem or challenge? I've been reading and learning the TableProvider APIs and have finally gotten around to taking a serious look at implementing support for `CREATE EXTERNAL TABLE`. I started down the road of using the statement level `OPTIONS` for adding field specific key/value config pairs similar to how the Parquet provider does with the `OPTION::COLUMN.NESTED.PATH` keys. However, my particular use case has quite a few field specific possible options that would be decently more ergonomic with field level options so I went looking for how I might be able to customize the SQL parser to support something of that nature. After many false starts I ended up stumbling over the [BigQuery column options support](https://docs.rs/sqlparser/latest/sqlparser/ast/enum.ColumnOption.html#variant.Options) and thought that would be perfect. While I was a bit disappointed its not directly supported, it turned out to be relatively easy to plumb through so that's what I've done. ### Describe the solution you'd like I'm not tied to anything specific beyond "allow for column specific key/value configuration options". The PR I'll open after filing this issue is one possible implementation, but it doesn't have to be *the* implementation by any means. ### Describe alternatives you've considered My original thought was to basically that I was going to have to go the route used in the [`sql_dialect.rs`](https://github.com/apache/datafusion/blob/main/datafusion-examples/examples/sql_dialect.rs) where I would basically have to reimplement the `CREATE EXTERNAL TABLE` logic and then somehow plumb my extra options around. And the reason I went that direction was my first (hacky) though was to re-parse the `CreateExternalTable::definition` field but it apparently doesn't keep the actual original SQL around (at least via the cli AFAICT). ### Additional context I've got no idea whether the reaction to this will be "Oh, neat!" or "Ewww, gross!". I'm not particularly tied to it because in the end I can always just go to the statement wide options, though given how easy it was to add it seems like it might be acceptable? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
