yjshen opened a new issue #946:
URL: https://github.com/apache/arrow-datafusion/issues/946
**Is your feature request related to a problem or challenge? Please describe
what you are trying to do.**
Currently, we can only handle table input on a file basis. A more flexible
and extensible abstraction is desirable.
**Describe the solution you'd like**
```rust
pub struct PartitionedFile {
/// Path for the file (e.g. URL, filesystem path, etc)
pub file_path: String,
/// Statistics of the file
pub statistics: Statistics,
// Several feasible extensions
// 1. partitioned table
/// Values of partition columns to be appended to each row
pub partition_value: Option<Vec<ScalarValue>>,
// 2. sub-file processing
/// A range of file to process in the current task
pub start: u64,
pub end: u64,
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
