[
https://issues.apache.org/jira/browse/ARROW-15254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17469324#comment-17469324
]
Antoine Pitrou edited comment on ARROW-15254 at 1/5/22, 1:57 PM:
-----------------------------------------------------------------
I think this is going to be difficult to implement. CSV files are read in a
streaming fashion, with possibly parallel processing of file blocks. So, say
the user asks to skip 10 rows at the footer, but the last block being read only
has 5 rows, how do we handle that situation?
cc [~westonpace] for ideas.
was (Author: pitrou):
I think this is going to be difficult to implement. CSV files are read in a
streaming fashion, with possibly parallel processing of file blocks. So, say
the user asks to skip 10 rows at the footer, but the last block being read only
has 5 rows, how do we handle that situation?
> [C++] Ability to skip CSV footer when reading in dataset
> --------------------------------------------------------
>
> Key: ARROW-15254
> URL: https://issues.apache.org/jira/browse/ARROW-15254
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Nicola Crane
> Priority: Major
>
> In ARROW-15252 a user reports wanting to be able to skip the final row of a
> CSV (the footer) when reading in a dataset of CSVs - is this possible to
> implement?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
