selvavm edited a comment on issue #1536:
URL:
https://github.com/apache/arrow-datafusion/issues/1536#issuecomment-1008559282
@houqp No. `ctx.table` returns an error. My knowledge on Parquet is limited.
Sorry for that.
Below are the things I tried,
- Changed the `uri` field in register_listing_table as
`&format!("file://{}","C:\\builds\\testdata")` - Still same
- Changed the code like below which now doesn't throw error but returns
below panic
Runtime::new().unwrap().block_on(async {
ctx.register_listing_table(
"my_table",
&format!("file://{}","C:\\builds\\testdata")
listing_options,
None,
)
.await
.unwrap()
});
Runtime::new().unwrap().block_on(async {
let a: Vec<RecordBatch> = df
.select(vec![col("state")])
.unwrap()
.collect()
.await
.unwrap();
pretty::print_batches(&a);
});
`ParquetError(General("Repetition level must be defined for a primitive
type"))'`
I tried viewing my parquet using a viewer tool and the metadata is as
follows,
{
"Version": 1,
"Num_rows": 50,
"Created_by": "parquet-rs version 6.5.0",
"Schema": [
{
"Field_id": 0,
"Name": "state",
"Type": "BYTE_ARRAY",
"Type_length": 0,
"LogicalType": null,
"Scale": 0,
"Precision": 0,
"Repetition_type": "REQUIRED",
"Converted_type": "UTF8"
},
{
"Field_id": 0,
"Name": "population",
"Type": "FLOAT",
"Type_length": 0,
"LogicalType": null,
"Scale": 0,
"Precision": 0,
"Repetition_type": "REQUIRED",
"Converted_type": "UTF8"
},
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
