alamb opened a new issue, #7500: URL: https://github.com/apache/arrow-datafusion/issues/7500
### Is your feature request related to a problem or challenge? DuckDB introduced the `exclude` syntax to select items from the select list except for certain columns: https://duckdb.org/2022/05/04/friendlier-sql.html#select--exclude https://duckdb.org/docs/sql/query_syntax/select.html It would be nice for DataFusion to support the same thing and it came up in the discord chat: https://discord.com/channels/885562378132000778/885562378132000781/1149381021448552558 ### Describe the solution you'd like sqlparser-rs already supports this. It seems like DataFusion currently ignores the syntax Using `traces_sample.parquet'` from [data.zip](https://github.com/apache/arrow-datafusion/files/12552989/data.zip) Run this query: ```sql ❯ select * exclude(span_id) from 'traces_sample.parquet' limit 10; ``` I would expect that to select all columns from traces_sample.parquet **except** `trace_id`. However at the moment datafusion appears to selects them all ``` ❯ select * exclude(span_id) from 'traces_sample.parquet' limit 10; +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+---------------------+--------------+-----------+------------------+----------------------------+----------------------------------+------------------+----------------+ | attributes | duration_nano | end_time_unix_nano | service.name | span.kind | span.name | time | trace_id | otel.status_code | parent_span_id | +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+---------------------+--------------+-----------+------------------+----------------------------+----------------------------------+------------------+----------------+ | {"client-uuid":"7a134d3eca3fc12","component":"net/http","host.name":"8056032ed0f1","http.method":"GET","http.status_code":200,"http.url":"/config?nonse=1.6100127556780492","ip":"172.19.0.4","opencensus.exporterversion":"Jaeger-Go-2.30.0","sampler.param":true,"sampler.type":"const"} | 16000 | 1688748446779372000 | frontend | Server | HTTP GET /config | 2023-07-07T16:47:26.779356 | 0000000000000000000113cdc917af1d | | | | {"client-uuid":"7a134d3eca3fc12","component":"net/http","host.name":"8056032ed0f1","http.method":"GET","http.status_code":200,"http.url":"/config?nonse=1.2346093977208494","ip":"172.19.0.4","opencensus.exporterversion":"Jaeger-Go-2.30.0","sampler.param":true,"sampler.type":"const"} | 10000 | 1688748430404380000 | frontend | Server | HTTP GET /config | 2023-07-07T16:47:10.404370 | 00000000000000000001360bbd2c8721 | | | | {"client-uuid":"7a134d3eca3fc12","component":"net/http","host.name":"8056032ed0f1","http.method":"GET","http.status_code":200,"http.url":"/config?nonse=1.898944744906115","ip":"172.19.0.4","opencensus.exporterversion":"Jaeger-Go-2.30.0","sampler.param":true,"sampler.type":"const"} | 6000 | 1688748422265841000 | frontend | Server | HTTP GET /config | 2023-07-07T16:47:02.265835 | 00000000000000000003ba9c67befc9f | | | | {"client-uuid":"7a134d3eca3fc12","component":"net/http","host.name":"8056032ed0f1","http.method":"GET","http.status_code":200,"http.url":"/config?nonse=1.2523662693007576","ip":"172.19.0.4","opencensus.exporterversion":"Jaeger-Go-2.30.0","sampler.param":true,"sampler.type":"const"} | 39000 | 1688748463776702000 | frontend | Server | HTTP GET /config | 2023-07-07T16:47:43.776663 | 00000000000000000004d2d6c57d572b | | | | {"client-uuid":"7a134d3eca3fc12","component":"net/http","host.name":"8056032ed0f1","http.method":"GET","http.status_code":200,"http.url":"/config?nonse=1.5330469141214835","ip":"172.19.0.4","opencensus.exporterversion":"Jaeger-Go-2.30.0","sampler.param":true,"sampler.type":"const"} | 12000 | 1688748403699926000 | frontend | Server | HTTP GET /config | 2023-07-07T16:46:43.699914 | 000000000000000000055c3b4694884a | | | | {"client-uuid":"7a134d3eca3fc12","component":"net/http","host.name":"8056032ed0f1","http.method":"GET","http.status_code":200,"http.url":"/config?nonse=1.488040998753469","ip":"172.19.0.4","opencensus.exporterversion":"Jaeger-Go-2.30.0","sampler.param":true,"sampler.type":"const"} | 7000 | 1688748392960688000 | frontend | Server | HTTP GET /config | 2023-07-07T16:46:32.960681 | 000000000000000000056fa9dd620e56 | | | | {"client-uuid":"7a134d3eca3fc12","component":"net/http","host.name":"8056032ed0f1","http.method":"GET","http.status_code":200,"http.url":"/config?nonse=1.4673158847657508","ip":"172.19.0.4","opencensus.exporterversion":"Jaeger-Go-2.30.0","sampler.param":true,"sampler.type":"const"} | 8000 | 1688748465953262000 | frontend | Server | HTTP GET /config | 2023-07-07T16:47:45.953254 | 00000000000000000006013370733303 | | | | {"client-uuid":"7a134d3eca3fc12","component":"net/http","host.name":"8056032ed0f1","http.method":"GET","http.status_code":200,"http.url":"/config?nonse=1.7384801849778881","ip":"172.19.0.4","opencensus.exporterversion":"Jaeger-Go-2.30.0","sampler.param":true,"sampler.type":"const"} | 10000 | 1688748413717028000 | frontend | Server | HTTP GET /config | 2023-07-07T16:46:53.717018 | 000000000000000000063870af1fec14 | | | | {"client-uuid":"7a134d3eca3fc12","component":"net/http","host.name":"8056032ed0f1","http.method":"GET","http.status_code":200,"http.url":"/config?nonse=1.6087159145105812","ip":"172.19.0.4","opencensus.exporterversion":"Jaeger-Go-2.30.0","sampler.param":true,"sampler.type":"const"} | 15000 | 1688748412730887000 | frontend | Server | HTTP GET /config | 2023-07-07T16:46:52.730872 | 000000000000000000072072efa8efd9 | | | | {"client-uuid":"7a134d3eca3fc12","component":"net/http","host.name":"8056032ed0f1","http.method":"GET","http.status_code":200,"http.url":"/config?nonse=1.2195245082629549","ip":"172.19.0.4","opencensus.exporterversion":"Jaeger-Go-2.30.0","sampler.param":true,"sampler.type":"const"} | 10000 | 1688748407923796000 | frontend | Server | HTTP GET /config | 2023-07-07T16:46:47.923786 | 0000000000000000000a39e1e7c23340 | | | +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+---------------------+--------------+-----------+------------------+----------------------------+----------------------------------+------------------+----------------+ 10 rows in set. Query took 0.004 seconds. ``` ### Describe alternatives you've considered _No response_ ### Additional context I think this is a fairly straightforward parser extension and doesn't require deep datafusion knowledge to do and thus marking it as a good first issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
