This is an automated email from the ASF dual-hosted git repository.
alenka pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow.git
The following commit(s) were added to refs/heads/main by this push:
new 1adc745f67 GH-37560 [Python][Documentation] Replacing confusing batch
size from 128Ki to 128_000 (#37605)
1adc745f67 is described below
commit 1adc745f67791dde1d9e2574b9de52a8d00ca7e8
Author: Arkadiusz Rudny <[email protected]>
AuthorDate: Tue Sep 12 06:27:34 2023 +0200
GH-37560 [Python][Documentation] Replacing confusing batch size from 128Ki
to 128_000 (#37605)
### Rationale for this change
https://github.com/apache/arrow/issues/37560
### Are these changes tested? -> No
### Are there any user-facing changes? -> Documentation
* Closes: #37560
Authored-by: Arkadiusz Rudny <[email protected]>
Signed-off-by: AlenkaF <[email protected]>
---
python/pyarrow/_dataset.pyx | 30 +++++++++++++++---------------
1 file changed, 15 insertions(+), 15 deletions(-)
diff --git a/python/pyarrow/_dataset.pyx b/python/pyarrow/_dataset.pyx
index 8f5688de29..d29fa125e2 100644
--- a/python/pyarrow/_dataset.pyx
+++ b/python/pyarrow/_dataset.pyx
@@ -319,7 +319,7 @@ cdef class Dataset(_Weakrefable):
partition information or internal metadata found in the data
source, e.g. Parquet statistics. Otherwise filters the loaded
RecordBatches before yielding them.
- batch_size : int, default 128Ki
+ batch_size : int, default 131_072
The maximum row count for scanned record batches. If scanned
record batches are overflowing memory then this method can be
called to reduce their size.
@@ -441,7 +441,7 @@ cdef class Dataset(_Weakrefable):
partition information or internal metadata found in the data
source, e.g. Parquet statistics. Otherwise filters the loaded
RecordBatches before yielding them.
- batch_size : int, default 128Ki
+ batch_size : int, default 131_072
The maximum row count for scanned record batches. If scanned
record batches are overflowing memory then this method can be
called to reduce their size.
@@ -519,7 +519,7 @@ cdef class Dataset(_Weakrefable):
partition information or internal metadata found in the data
source, e.g. Parquet statistics. Otherwise filters the loaded
RecordBatches before yielding them.
- batch_size : int, default 128Ki
+ batch_size : int, default 131_072
The maximum row count for scanned record batches. If scanned
record batches are overflowing memory then this method can be
called to reduce their size.
@@ -597,7 +597,7 @@ cdef class Dataset(_Weakrefable):
partition information or internal metadata found in the data
source, e.g. Parquet statistics. Otherwise filters the loaded
RecordBatches before yielding them.
- batch_size : int, default 128Ki
+ batch_size : int, default 131_072
The maximum row count for scanned record batches. If scanned
record batches are overflowing memory then this method can be
called to reduce their size.
@@ -675,7 +675,7 @@ cdef class Dataset(_Weakrefable):
partition information or internal metadata found in the data
source, e.g. Parquet statistics. Otherwise filters the loaded
RecordBatches before yielding them.
- batch_size : int, default 128Ki
+ batch_size : int, default 131_072
The maximum row count for scanned record batches. If scanned
record batches are overflowing memory then this method can be
called to reduce their size.
@@ -730,7 +730,7 @@ cdef class Dataset(_Weakrefable):
partition information or internal metadata found in the data
source, e.g. Parquet statistics. Otherwise filters the loaded
RecordBatches before yielding them.
- batch_size : int, default 128Ki
+ batch_size : int, default 131_072
The maximum row count for scanned record batches. If scanned
record batches are overflowing memory then this method can be
called to reduce their size.
@@ -1411,7 +1411,7 @@ cdef class Fragment(_Weakrefable):
partition information or internal metadata found in the data
source, e.g. Parquet statistics. Otherwise filters the loaded
RecordBatches before yielding them.
- batch_size : int, default 128Ki
+ batch_size : int, default 131_072
The maximum row count for scanned record batches. If scanned
record batches are overflowing memory then this method can be
called to reduce their size.
@@ -1491,7 +1491,7 @@ cdef class Fragment(_Weakrefable):
partition information or internal metadata found in the data
source, e.g. Parquet statistics. Otherwise filters the loaded
RecordBatches before yielding them.
- batch_size : int, default 128Ki
+ batch_size : int, default 131_072
The maximum row count for scanned record batches. If scanned
record batches are overflowing memory then this method can be
called to reduce their size.
@@ -1574,7 +1574,7 @@ cdef class Fragment(_Weakrefable):
partition information or internal metadata found in the data
source, e.g. Parquet statistics. Otherwise filters the loaded
RecordBatches before yielding them.
- batch_size : int, default 128Ki
+ batch_size : int, default 131_072
The maximum row count for scanned record batches. If scanned
record batches are overflowing memory then this method can be
called to reduce their size.
@@ -1653,7 +1653,7 @@ cdef class Fragment(_Weakrefable):
partition information or internal metadata found in the data
source, e.g. Parquet statistics. Otherwise filters the loaded
RecordBatches before yielding them.
- batch_size : int, default 128Ki
+ batch_size : int, default 131_072
The maximum row count for scanned record batches. If scanned
record batches are overflowing memory then this method can be
called to reduce their size.
@@ -1731,7 +1731,7 @@ cdef class Fragment(_Weakrefable):
partition information or internal metadata found in the data
source, e.g. Parquet statistics. Otherwise filters the loaded
RecordBatches before yielding them.
- batch_size : int, default 128Ki
+ batch_size : int, default 131_072
The maximum row count for scanned record batches. If scanned
record batches are overflowing memory then this method can be
called to reduce their size.
@@ -1786,7 +1786,7 @@ cdef class Fragment(_Weakrefable):
partition information or internal metadata found in the data
source, e.g. Parquet statistics. Otherwise filters the loaded
RecordBatches before yielding them.
- batch_size : int, default 128Ki
+ batch_size : int, default 131_072
The maximum row count for scanned record batches. If scanned
record batches are overflowing memory then this method can be
called to reduce their size.
@@ -3436,7 +3436,7 @@ cdef class Scanner(_Weakrefable):
partition information or internal metadata found in the data
source, e.g. Parquet statistics. Otherwise filters the loaded
RecordBatches before yielding them.
- batch_size : int, default 128Ki
+ batch_size : int, default 131_072
The maximum row count for scanned record batches. If scanned
record batches are overflowing memory then this method can be
called to reduce their size.
@@ -3515,7 +3515,7 @@ cdef class Scanner(_Weakrefable):
partition information or internal metadata found in the data
source, e.g. Parquet statistics. Otherwise filters the loaded
RecordBatches before yielding them.
- batch_size : int, default 128Ki
+ batch_size : int, default 131_072
The maximum row count for scanned record batches. If scanned
record batches are overflowing memory then this method can be
called to reduce their size.
@@ -3601,7 +3601,7 @@ cdef class Scanner(_Weakrefable):
partition information or internal metadata found in the data
source, e.g. Parquet statistics. Otherwise filters the loaded
RecordBatches before yielding them.
- batch_size : int, default 128Ki
+ batch_size : int, default 131_072
The maximum row count for scanned record batches. If scanned
record batches are overflowing memory then this method can be
called to reduce their size.
