pitrou commented on code in PR #45084: URL: https://github.com/apache/arrow/pull/45084#discussion_r1944368819
########## docs/source/python/json.rst: ########## @@ -115,3 +115,20 @@ and pass it to :func:`read_json`. For example, you can pass an explicit Similarly, you can choose performance settings by passing a :class:`ReadOptions` instance to :func:`read_json`. + + +Incremental reading +------------------- + +For memory-constrained environments, it is also possible to read a JSON file +one batch at a time, using :func:`open_json`. + +There are a few caveats: + +1. For now, the incremental reader is always single-threaded (regardless of + :attr:`ReadOptions.use_threads`) Review Comment: I don't think that's right for JSON, reading at the C++ source code. So we can remove this caveat. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
