SteveLauC commented on issue #9430:
URL:
https://github.com/apache/arrow-datafusion/issues/9430#issuecomment-1974944527
Copy my comment here:
> I tried this with JSON:
>
> ```shell
> $ cat sql
> create external table test stored as json location '/dev/stdin';
> select * from test;
>
> $ cat 1.json
> { "age": 2, "name": "steve" }
>
> $ cat 1.json |datafusion-cli -f sql
> DataFusion CLI v36.0.0
> 0 rows in set. Query took 0.001 seconds.
>
> 0 rows in set. Query took 0.000 seconds.
> ```
>
> It won't give you error, though it won't print the rows as well..
>
> For parquet, `datafusion-cli` gives you an error because the file size of
a pipe is 0:
>
> > ```shell
> > $ cat src/main.rs
> > fn main() {
> > let stat = nix::sys::stat::fstat(0).unwrap();
> > println!("{}", stat.st_size);
> > }
> >
> > $ cargo b
> > $ cat 1.json | ./target/debug/rust
> > 0
> > ```
>
> ```shell
> $ cat sql
> create external table test stored as parquet location '/dev/stdin';
> select * from test;
>
> $ pqrs cat 2.parquet
>
> ###############
> File: 2.parquet
> ###############
>
> {Age: 2}
>
> $ cat 2.parquet | datafusion-cli -f sql
> DataFusion CLI v36.0.0
> Execution error: file size of 0 is less than footer
> Error during planning: table 'datafusion.public.test' not found
> ```
>
> For CSV, it gives you an error `Object Store error: Generic
LocalFileSystem error: Error seeking file /dev/stdin: Illegal seek (os error
29)` because pipe is not seekable:
>
> ```shell
> $ cat sql
> create external table test stored as csv location '/dev/stdin';
> select * from test;
>
> $ cat username.csv
> Username; Identifier;First name;Last name
> booker12;9012;Rachel;Booker
> grey07;2070;Laura;Grey
> johnson81;4081;Craig;Johnson
> jenkins46;9346;Mary;Jenkins
> smith79;5079;Jamie;Smith
>
>
> $ cat username.csv | datafusion-cli -f sql
> DataFusion CLI v36.0.0
> Object Store error: Generic LocalFileSystem error: Error seeking file
/dev/stdin: Illegal seek (os error 29)
> Error during planning: table 'datafusion.public.test' not found
> ```
I am interested in this, but not quite sure if I can make it as fixing this
needs to work with those arrow-xx crates, but anyway.
I think the first thing I am gonna do is to take a look at the above JSON
example, and figure out why it does not print the data even on success.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
