westonpace commented on a change in pull request #9095:
URL: https://github.com/apache/arrow/pull/9095#discussion_r564743436
##########
File path: cpp/src/arrow/csv/reader.cc
##########
@@ -177,14 +182,26 @@ class SerialBlockReader : public BlockReader {
public:
using BlockReader::BlockReader;
- Result<arrow::util::optional<CSVBlock>> Next() {
+ static Iterator<util::optional<CSVBlock>> MakeIterator(
+ Iterator<std::shared_ptr<Buffer>> buffer_iterator,
std::unique_ptr<Chunker> chunker,
+ std::shared_ptr<Buffer> first_buffer) {
+ auto block_reader =
+ std::make_shared<SerialBlockReader>(std::move(chunker), first_buffer);
+ // Wrap shared pointer in callable
+ Transformer<std::shared_ptr<Buffer>, util::optional<CSVBlock>>
block_reader_fn =
+ [block_reader](std::shared_ptr<Buffer> buf) {
+ return (*block_reader)(std::move(buf));
+ };
+ return MakeTransformedIterator(std::move(buffer_iterator),
block_reader_fn);
+ }
+
+ Result<TransformFlow<util::optional<CSVBlock>>> operator()(
Review comment:
`TransformFinish` (line 201) will cause
`IterationTraits<util::optional<CSVBlock>>::End()` to be emitted. So it was
either keeping the `util::optional` or adding
`IterationTraits<CSVBlock>::End()` (and a corresponding equality operator).
The async iterators are still iterators and so they rely on that end token.
One thing we could do is rewire all the async iterator functions so that they
use `util::optional` under the hood (unless the type being iterated is a
pointer) and it doesn't allow users to specify their own end tokens. Then the
`util::optional` could be hidden from the consumer of the iterators API. This
would require all users to rely on `Visit` instead of manually iterating.
However, for async iterator, this is probably a given already.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
