This is an automated email from the ASF dual-hosted git repository.
nevime pushed a change to branch rust-parquet-arrow-writer
in repository https://gitbox.apache.org/repos/asf/arrow.git.
omit 80a9c02 ARROW-8289: [Rust] Parquet Arrow writer with nested support
add 40ac6e3 ARROW-9205: [Documentation] Fix typos
add 52d0fe6 ARROW-9429: [Python] ChunkedArray.to_numpy
add 811d8f6 ARROW-9402: [C++] Rework portable wrappers for checked
integer arithmetic
add 1c375b5 ARROW-9631: [Rust] Make arrow not depend on flight
add a84e7b0 ARROW-9652: [Rust][DataFusion] Error message rather than
panic for external csv tables with no column defs
add 1e48d6a ARROW-7218: [Python] Conversion from boolean numpy scalars
not working
add 858059f ARROW-9606: [C++][Dataset] Support
`"a"_.In(<>).Assume(<compound>)`
add 37ee600 ARROW-9628: [Rust] Disable artifact caching for Mac OSX builds
add 1b0aebe ARROW-9598: [C++][Parquet] Fix writing nullable structs
add e31e5d4 ARROW-9683: [Rust][DataFusion] Add debug printing to physical
plans and associated types
add 0e15d26 ARROW-9521: [Rust][DataFusion] Handle custom CSV file
extensions
add a0ec4f2 ARROW-9638: [C++][Compute] Implement mode kernel
add 66a8f04 ARROW-9692: [Python] Fix distutils-related warning
add e934a8a ARROW-9602: [R] Improve cmake detection in Linux build
add b1a30e6 ARROW-9653: [Rust][DataFusion] Do not error in planner with
SQL has multiple group by expressions
add aec21b2 ARROW-9696: [Rust] [DataFusion] fix nested binary expressions
add 3fc7fe4 ARROW-9691: [Rust] [DataFusion] Make sql_statement_to_plan
method public
add dd98a9f ARROW-9577: [C++] Ignore EBADF error in posix_madvise()
add ebbe60c ARROW-9684: [C++] Fix undefined behaviour on invalid IPC /
Parquet input
add e1e3188 ARROW-9659: [C++] Fix RecordBatchStreamReader when source is
CudaBufferReader
add faee652 ARROW-9604: [C++] Add aggregate min/max benchmark
add 042998c ARROW-9679: [Rust] [DataFusion] More efficient creation of
final batch from HashAggregateExec
add db9185e ARROW-9695: [Rust] Improve comments on LogicalPlan enum
variants
add 12e31a5 ARROW-9715: [R] changelog/doc updates for 1.0.1
add 796b050 ARROW-9700: [Python] fix create_library_symlinks for macos
add 90d1ab7 ARROW-9721: [Packaging][Python] Update wheel dependency files
add 69d7b2b ARROW-9698: [C++] Remove -DNDEBUG flag leak in .pc file
add b5955d8 ARROW-9713: [Rust] [DataFusion] Remove explicit panics
add 9919e3e ARROW-9712: [Rust] [DataFusion] Fix parquet error handling
and general code improvements
add aaf467a ARROW-9644: [C++][Dataset] Don't apply ignore_prefixes to
partition base_dir
add 898bef8 ARROW-9665: [R] head/tail/take for Datasets
new f11b322 ARROW-8289: [Rust] Parquet Arrow writer with nested support
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (80a9c02)
\
N -- N -- N refs/heads/rust-parquet-arrow-writer (f11b322)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/rust.yml | 8 +-
LICENSE.txt | 16 +
cpp/cmake_modules/SetupCxxFlags.cmake | 6 -
cpp/src/arrow/CMakeLists.txt | 1 +
cpp/src/arrow/array/array_base.h | 17 +-
cpp/src/arrow/array/array_binary.h | 11 +-
cpp/src/arrow/array/array_nested.cc | 71 +-
cpp/src/arrow/array/array_nested.h | 22 +-
cpp/src/arrow/array/array_primitive.h | 5 -
cpp/src/arrow/array/concatenate.cc | 10 +-
cpp/src/arrow/array/data.cc | 2 +-
cpp/src/arrow/array/data.h | 16 +
cpp/src/arrow/array/validate.cc | 29 +-
cpp/src/arrow/arrow.pc.in | 2 +-
cpp/src/arrow/buffer.cc | 2 +-
cpp/src/arrow/compute/api_aggregate.cc | 4 +
cpp/src/arrow/compute/api_aggregate.h | 17 +-
cpp/src/arrow/compute/kernels/aggregate_basic.cc | 5 +-
.../compute/kernels/aggregate_basic_internal.h | 6 +
.../arrow/compute/kernels/aggregate_benchmark.cc | 68 ++
cpp/src/arrow/compute/kernels/aggregate_mode.cc | 179 ++++
cpp/src/arrow/compute/kernels/aggregate_test.cc | 115 ++-
cpp/src/arrow/compute/kernels/codegen_internal.h | 353 +++++--
cpp/src/arrow/compute/kernels/scalar_arithmetic.cc | 148 ++-
.../compute/kernels/scalar_arithmetic_benchmark.cc | 57 +-
.../compute/kernels/scalar_arithmetic_test.cc | 280 +++--
.../arrow/compute/kernels/scalar_cast_boolean.cc | 8 +-
.../arrow/compute/kernels/scalar_cast_numeric.cc | 50 +-
.../arrow/compute/kernels/scalar_cast_temporal.cc | 6 +-
cpp/src/arrow/compute/kernels/scalar_nested.cc | 4 +-
cpp/src/arrow/compute/kernels/scalar_string.cc | 6 +-
cpp/src/arrow/compute/kernels/test_util.cc | 113 ++-
cpp/src/arrow/compute/kernels/test_util.h | 10 +
cpp/src/arrow/dataset/discovery.cc | 12 +-
cpp/src/arrow/dataset/discovery_test.cc | 25 +-
cpp/src/arrow/dataset/filter.cc | 78 +-
cpp/src/arrow/dataset/filter.h | 10 +
cpp/src/arrow/dataset/filter_test.cc | 13 +-
cpp/src/arrow/dataset/partition.cc | 39 +-
cpp/src/arrow/pretty_print.cc | 2 +-
cpp/src/arrow/python/filesystem.cc | 10 +
cpp/src/arrow/python/filesystem.h | 5 +
cpp/src/arrow/python/python_to_arrow.cc | 2 +
cpp/src/arrow/scalar.h | 16 +-
cpp/src/arrow/util/basic_decimal.cc | 2 +-
cpp/src/arrow/util/bit_block_counter.cc | 23 +-
cpp/src/arrow/util/bit_block_counter.h | 186 +++-
cpp/src/arrow/util/bit_block_counter_test.cc | 92 ++
cpp/src/arrow/util/decimal.cc | 2 +-
cpp/src/arrow/util/int_util.h | 94 --
cpp/src/arrow/util/int_util_internal.h | 124 +++
cpp/src/arrow/util/int_util_test.cc | 1 +
cpp/src/arrow/util/io_util.cc | 8 +-
cpp/src/arrow/util/trie.h | 2 -
cpp/src/arrow/util/windows_fixup.h | 11 +
cpp/src/arrow/vendored/portable-snippets/README.md | 10 +
.../arrow/vendored/portable-snippets/safe-math.h | 1071 ++++++++++++++++++++
cpp/src/arrow/visitor_inline.h | 90 +-
cpp/src/parquet/arrow/arrow_reader_writer_test.cc | 17 +
cpp/src/parquet/arrow/reader_internal.cc | 4 +-
cpp/src/parquet/column_reader.cc | 32 +-
cpp/src/parquet/column_writer.cc | 9 +-
cpp/src/parquet/types.h | 1 -
dev/release/00-prepare-test.rb | 16 +-
dev/release/00-prepare.sh | 4 +-
docker-compose.yml | 2 +-
docs/source/cpp/compute.rst | 7 +-
docs/source/format/Columnar.rst | 4 +-
docs/source/format/Flight.rst | 2 +-
python/pyarrow/__init__.py | 2 +-
python/pyarrow/_fs.pyx | 46 +-
python/pyarrow/dataset.py | 10 +-
python/pyarrow/fs.py | 4 +-
python/pyarrow/includes/libarrow_fs.pxd | 2 +
python/pyarrow/table.pxi | 68 +-
python/pyarrow/tests/test_convert_builtin.py | 12 +-
python/pyarrow/tests/test_cuda.py | 54 +
python/pyarrow/tests/test_dataset.py | 6 +-
python/pyarrow/tests/test_extension_type.py | 21 +-
python/pyarrow/tests/test_fs.py | 12 +
python/pyarrow/tests/test_parquet.py | 5 +-
python/pyarrow/tests/test_table.py | 74 +-
python/requirements-wheel-build.txt | 4 +-
python/requirements-wheel-test.txt | 3 +-
python/setup.py | 14 +-
r/NAMESPACE | 7 +
r/NEWS.md | 6 +-
r/R/arrowExports.R | 4 +
r/R/dataset.R | 83 +-
r/R/dplyr.R | 94 +-
r/src/arrowExports.cpp | 17 +
r/src/dataset.cpp | 20 +
r/tests/testthat/test-dataset.R | 115 ++-
r/tests/testthat/test-dplyr.R | 109 ++
r/tools/linuxlibs.R | 33 +-
r/vignettes/install.Rmd | 2 +
rust/arrow/Cargo.toml | 2 +-
rust/datafusion/Cargo.toml | 7 +-
rust/datafusion/src/datasource/csv.rs | 5 +-
rust/datafusion/src/error.rs | 1 -
rust/datafusion/src/execution/context.rs | 126 +--
rust/datafusion/src/execution/physical_plan/csv.rs | 24 +-
.../src/execution/physical_plan/datasource.rs | 20 +-
.../src/execution/physical_plan/expressions.rs | 135 ++-
.../src/execution/physical_plan/hash_aggregate.rs | 436 ++++----
.../src/execution/physical_plan/limit.rs | 2 +
.../src/execution/physical_plan/memory.rs | 2 +
.../src/execution/physical_plan/merge.rs | 17 +-
rust/datafusion/src/execution/physical_plan/mod.rs | 12 +-
.../src/execution/physical_plan/parquet.rs | 103 +-
.../src/execution/physical_plan/projection.rs | 2 +
.../src/execution/physical_plan/selection.rs | 2 +
.../datafusion/src/execution/physical_plan/sort.rs | 2 +
rust/datafusion/src/execution/physical_plan/udf.rs | 23 +
rust/datafusion/src/logicalplan.rs | 99 +-
rust/datafusion/src/optimizer/type_coercion.rs | 1 +
rust/datafusion/src/optimizer/utils.rs | 1 +
rust/datafusion/src/sql/parser.rs | 43 +-
rust/datafusion/src/sql/planner.rs | 154 ++-
rust/datafusion/tests/sql.rs | 38 +
120 files changed, 4388 insertions(+), 1369 deletions(-)
create mode 100644 cpp/src/arrow/compute/kernels/aggregate_mode.cc
create mode 100644 cpp/src/arrow/util/int_util_internal.h
create mode 100644 cpp/src/arrow/vendored/portable-snippets/README.md
create mode 100644 cpp/src/arrow/vendored/portable-snippets/safe-math.h
