selvavm commented on issue #1536:
URL:
https://github.com/apache/arrow-datafusion/issues/1536#issuecomment-1008559282
@houqp No. `ctx.table` returns an error. My knowledge on Parquet is limited.
Sorry for that.
Below are the things I tried,
- Changed the `uri` field in register_listing_table as
`&format!("file://{}","C:\\builds\\testdata")` - Still same
- Changed the code like below which now doesn't throw error but returns
empty table
Runtime::new().unwrap().block_on(async {
ctx.register_listing_table(
"my_table",
&format!("file://{}","C:\\builds\\testdata")
listing_options,
None,
)
.await
.unwrap()
});
Runtime::new().unwrap().block_on(async {
let a: Vec<RecordBatch> = df
.select(vec![col("state")])
.unwrap()
.collect()
.await
.unwrap();
pretty::print_batches(&a);
});
- Changed the `table_partition_cols: vec!["country".to_string()],` to
`table_partition_cols: vec![],`. With this, I am getting below error,
`ParquetError(General("Repetition level must be defined for a primitive
type"))'`
I tried viewing my parquet using a viewer tool and the metadata is as
follows,
{
"Version": 1,
"Num_rows": 50,
"Created_by": "parquet-rs version 6.5.0",
"Schema": [
{
"Field_id": 0,
"Name": "state",
"Type": "BYTE_ARRAY",
"Type_length": 0,
"LogicalType": null,
"Scale": 0,
"Precision": 0,
"Repetition_type": "REQUIRED",
"Converted_type": "UTF8"
},
{
"Field_id": 0,
"Name": "population",
"Type": "FLOAT",
"Type_length": 0,
"LogicalType": null,
"Scale": 0,
"Precision": 0,
"Repetition_type": "REQUIRED",
"Converted_type": "UTF8"
},
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
