dmitrijoseph opened a new issue, #4428:
URL: https://github.com/apache/arrow-datafusion/issues/4428
```
let test_df = ctx.read_csv("test.csv", CsvReadOptions::new()).await?;
let test_df = test_df.with_column_renamed("id", "renamedID")?;
println!("{:#?}", test_df.explain(true, true)?);
```
```
+---------------+-----------------------------------------------------------------------------------------------------------------+
| plan_type | plan
|
+---------------+-----------------------------------------------------------------------------------------------------------------+
| logical_plan | TableScan: ?table? projection=[id, name]
|
| physical_plan | CsvExec: files=[<>/test.csv], has_header=true, limit=None,
projection=[id, name] |
| |
|
+---------------+-----------------------------------------------------------------------------------------------------------------+
```
This time using `?table?` as a qualifier
```
let test_df = ctx.read_csv("test.csv", CsvReadOptions::new()).await?;
let test_df = test_df.with_column_renamed("?table?.id", "renamedID")?;
test_df.explain(false, false)?.show().await?;
```
```
Error: SchemaError(FieldNotFound { field: Column { relation: None, name:
"?table?.id" }, valid_fields: Some([Column { relation: Some("?table?"), name:
"id" }, Column { relation: Some("?table?"), name: "name" }]) })
```
This time registering table so it has a set table name
```
let test_df = ctx.read_csv("test.csv", CsvReadOptions::new()).await?;
ctx.register_table("t1", test_df)?;
let test_df = ctx.table("t1")?;
let test_df = test_df.with_column_renamed("id", "renamedID")?;
test_df.explain(false, false)?.show().await?;
```
```
+---------------+-------------------------------------------------------------------------------------------------------------------+
| plan_type | plan
|
+---------------+-------------------------------------------------------------------------------------------------------------------+
| logical_plan | Projection: ?table?.id, ?table?.name, alias=t1
|
| | TableScan: ?table? projection=[id, name]
|
| physical_plan | ProjectionExec: expr=[id@0 as id, name@1 as name]
|
| | CsvExec: files=[<>/test.csv], has_header=true,
limit=None, projection=[id, name] |
| |
|
+---------------+-------------------------------------------------------------------------------------------------------------------+
```
This time registering table so it has a set table name and using that table
name as a qualifier
```
let test_df = ctx.read_csv("test.csv", CsvReadOptions::new()).await?;
ctx.register_table("t1", test_df)?;
let test_df = ctx.table("t1")?;
let test_df = test_df.with_column_renamed("t1.id", "renamedID")?;
test_df.explain(false, false)?.show().await?;
```
```
+---------------+---------------------------------------------------------------------------------------------------------------------+
| plan_type | plan
|
+---------------+---------------------------------------------------------------------------------------------------------------------+
| logical_plan | Projection: t1.id AS renamedId, t1.name
|
| | Projection: ?table?.id, ?table?.name, alias=t1
|
| | TableScan: ?table? projection=[id, name]
|
| physical_plan | ProjectionExec: expr=[id@0 as renamedId, name@1 as name]
|
| | ProjectionExec: expr=[id@0 as id, name@1 as name]
|
| | CsvExec: files=[<>/test.csv], has_header=true,
limit=None, projection=[id, name] |
| |
|
+---------------+---------------------------------------------------------------------------------------------------------------------+
```
I have two lines of thinking on this
1. `with_column_renamed` seems to be the only function in the DataFrame API
that always requires the use of the fully qualified column name, this should be
changed to use the unqualified column name.
2. Why are tables being set to `?table?` ? I assume this was chosen because
it isn't valid sql but why not always generate a valid table name from
functions like `ctx.read_csv()`. The current implementation is awkward because
using `?table?` as a qualifier returns an error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
