Blizzara commented on code in PR #13750:
URL: https://github.com/apache/datafusion/pull/13750#discussion_r1884067166
##########
datafusion/core/src/datasource/file_format/csv.rs:
##########
@@ -1259,73 +1259,57 @@ mod tests {
Ok(())
}
- /// Read a single empty csv file in parallel
+ /// Read a single empty csv file
///
/// empty_0_byte.csv:
/// (file is empty)
- #[rstest(n_partitions, case(1), case(2), case(3), case(4))]
#[tokio::test]
- async fn test_csv_parallel_empty_file(n_partitions: usize) -> Result<()> {
Review Comment:
I assume there was no special reason to test parallelism for empty files?
Now that we just skip empty files altogether there is no parallelism, the test
does pass with the repartition settings as well but they seemed unrelevant so I
cleaned them away. Can add back if there's a reason to keep them!
##########
datafusion/core/src/datasource/file_format/json.rs:
##########
@@ -619,13 +619,11 @@ mod tests {
Ok(())
}
- #[rstest(n_partitions, case(1), case(2), case(3), case(4))]
#[tokio::test]
- async fn it_can_read_empty_ndjson_in_parallel(n_partitions: usize) ->
Result<()> {
Review Comment:
same here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
