This is an automated email from the ASF dual-hosted git repository.
nevime pushed a change to branch rust-parquet-arrow-writer
in repository https://gitbox.apache.org/repos/asf/arrow.git.
discard a953083 ARROW-10168: [Rust] [Parquet] Schema roundtrip - use Arrow
schema from Parquet metadata when available
omit e456dfc ARROW-10191: [Rust] [Parquet] Add roundtrip Arrow -> Parquet
tests for all supported Arrow DataTypes
omit 7f743c2 ARROW-8426: [Rust] [Parquet] Add support for writing
dictionary types
omit 852b5ed ARROW-10095: [Rust] Update rust-parquet-arrow-writer branch's
encode_arrow_schema with ipc changes
omit 5394b3e ARROW-8423: [Rust] [Parquet] Serialize Arrow schema metadata
omit e8ac2bf ARROW-8289: [Rust] Parquet Arrow writer with nested support
add d0579ac ARROW-9753: [Rust] [DataFusion] Replaced Arc<Mutex<>> by Box<>
add 334e33e ARROW-9520: [Rust] [DataFusion] Add support for aliased
aggregate exprs
add b73731b ARROW-10161: [Rust] [DataFusion] DRYed code in tests
add df91582 ARROW-4927: [Rust] Update top level README to describe
current functionality
add 551edca ARROW-10169: [Rust] Pretty print null PrimitiveTypes as empty
strings
add 57f548c ARROW-10155: [Rust] [DataFusion] Improved lib.rs docs
add 46c75ac ARROW-10162: [Rust] Add pretty print support for
DictionaryArray
add a6b30de ARROW-10171: [Rust] [DataFusion] Added
ExecutionContext::From<ExecutionContextState>
add 8c75941 ARROW-9701: [CI][Java] Add a job for s390x Java on TravisCI
add bb553c4 ARROW-10156: [Rust] Added github action to label PRs for rust.
add 4555fb9 ARROW-9786: [R] Unvendor cpp11 before release
add 7f84722 ARROW-9941: [Python] Better string representation for
extension types
add fbb781b ARROW-8394: [JS] Upgrade to TypeScript 4.0.2, fix typings for
TS 3.9+
add 68a7ba5 ARROW-8735: [Rust] [Parquet] Allow arm 32 to use soft hash
implementation
add ecc3ed8 ARROW-10008: [C++][Dataset] Fix filtering/row group
statistics of dict columns
add 72a0e96 ARROW-10121: [C++] Fix emission of new dictionaries in IPC
writer
add 105873e ARROW-10068: [C++] Add bundled external project for
aws-sdk-cpp
add b2842ab ARROW-10147: [Python] Pandas metadata fails if index name not
JSON-serializable
add ffaed97 ARROW-8205: [Rust] [DataFusion] Added check to uniqueness of
column names.
add 88b0cd2 ARROW-10041: [Rust] Added check of data type to
GenericString::from.
add b17cea8 ARROW-10179: [Rust] Fixed error in labeler
add c5d998c ARROW-10189: [Doc] Fixed typo in C-Data interface example
add adcb8ef ARROW-10188: [Rust] [DataFusion] Fixed DataFusion examples.
add a880ae6 ARROW-10167: [Rust] [DataFusion] Support DictionaryArray in
sql.rs tests, by using standard pretty printer
add b1480a6 ARROW-10120: [C++] Add two-level nested Parquet read to Arrow
benchmarks
add c6e0fa4 ARROW-10193: [Python] Segfault when converting to fixed size
binary array
add 814c681 ARROW-10192: [Python] Always decode inner dictionaries when
converting array to Pandas
add bca04cd ARROW-10176: [C++] Avoid using unformattable types for test
parameters
add 03dea9b ARROW-10200: [CI][Java] Fix a job failure for s390x Java on
TravisCI
add 8a03003 ARROW-10201: [C++][CI] Disable S3 in arm64 job on Travis CI
add e9a12fa ARROW-10058: [C++] Improve repeated levels conversion without
BMI2
add e9a0ada ARROW-10202: [CI][Windows] Use sf.net mirror for MSYS2
add 47a019b ARROW-10178: [CI] Remove patch to fix Spark master build
add aeef2a6 ARROW-9147: [C++][Dataset] Support projection from null->any
type
add 1a2d048 ARROW-10151: [Python] Add support for MapArray conversion to
Pandas
add ef08a9d ARROW-9943: [C++] Recursively apply Arrow metadata when
reading from Parquet
add 6a8f84d ARROW-10205: [Java][FlightRPC] Allow disabling server
validation
add 8eea173 ARROW-7960: [C++] Add support fo reading additional types
add a28b1fb ARROW-10093: [R] Add ability to opt-out of int64 -> int
demotion
add eec7277 ARROW-10214: [Python] Allow printing undecodable schema
metadata
add a3b5bcd ARROW-10217: [CI] Run fewer GitHub Actions jobs
add 62269e9 ARROW-10030: [Rust] Add support for `FromIter` and `IntoIter`
for primitive types
add 1c9c718 ARROW-10181: [Rust] Skip compiling one test on 32 bit ARM
architecture
add ad51ac2 ARROW-10196: [C++] Add Future::DeferNotOk
add 25d131b ARROW-9645: [Python] Deprecate pyarrow.filesystem in favor of
pyarrow.fs
add 2a49df6 Fix YAML syntax in dev_labeler
add 0f56c60 ARROW-9964: [C++] Allow reading date types from CSV data
add 806b0ce ARROW-6972: [C#] Support for StructArrays
add 9526558 ARROW-10204: [Rust] Filter kernel should only count bits in
valid range
add 732e333 ARROW-9266: [Python][Packaging] Enable S3 support in macOS
wheels
add 8366799 ARROW-10099: [C++][Dataset] Simplify type inference for
partition columns
add 4bab70f ARROW-10139: [C++] Add support for building arrow_testing
without building tests
add c1dc1f3 ARROW-8289: [Rust] Parquet Arrow writer with nested support
add 92d6132 ARROW-8423: [Rust] [Parquet] Serialize Arrow schema metadata
add 62e6fe6 ARROW-10095: [Rust] Update rust-parquet-arrow-writer branch's
encode_arrow_schema with ipc changes
add a93c4e9 ARROW-8426: [Rust] [Parquet] Add support for writing
dictionary types
add 7b27503 ARROW-10191: [Rust] [Parquet] Add roundtrip Arrow -> Parquet
tests for all supported Arrow DataTypes
add 211d4e9 ARROW-10168: [Rust] [Parquet] Schema roundtrip - use Arrow
schema from Parquet metadata when available
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (a953083)
\
N -- N -- N refs/heads/rust-parquet-arrow-writer (211d4e9)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.env | 2 +-
.github/workflows/cpp.yml | 29 +-
.../Gemfile => .github/workflows/dev_labeler.yml | 21 +-
.../workflows/dev_labeler/labeler.yml | 10 +-
.github/workflows/js.yml | 6 +-
.github/workflows/r.yml | 4 +-
.travis.yml | 63 +-
ci/docker/conda-integration.dockerfile | 2 +-
ci/docker/conda-python-spark.dockerfile | 4 -
ci/docker/debian-10-cpp.dockerfile | 10 +-
ci/docker/debian-10-js.dockerfile | 2 +-
ci/docker/fedora-32-cpp.dockerfile | 11 +-
ci/docker/linux-apt-docs.dockerfile | 2 +-
ci/docker/linux-apt-r.dockerfile | 6 +
ci/docker/linux-r.dockerfile | 4 +
ci/docker/ubuntu-18.04-cpp.dockerfile | 3 +
ci/docker/ubuntu-20.04-cpp.dockerfile | 11 +-
ci/etc/integration_spark_ARROW-9438.patch | 72 -
ci/scripts/cpp_build.sh | 1 +
ci/scripts/cpp_test.sh | 12 -
ci/scripts/install_minio.sh | 14 +-
ci/scripts/java_build.sh | 56 +
ci/scripts/msys2_system_upgrade_phase1.sh | 5 +-
ci/scripts/r_deps.sh | 2 +-
ci/scripts/r_docker_configure.sh | 19 +
ci/scripts/r_test.sh | 7 +
ci/scripts/r_windows_build.sh | 8 +-
cpp/Brewfile | 1 +
cpp/CMakeLists.txt | 1 +
cpp/build-support/run-test.sh | 6 +-
cpp/cmake_modules/DefineOptions.cmake | 5 +
cpp/cmake_modules/ThirdpartyToolchain.cmake | 184 +-
cpp/src/arrow/CMakeLists.txt | 6 +-
cpp/src/arrow/array/builder_binary.h | 4 +
cpp/src/arrow/array/builder_dict.h | 14 +-
cpp/src/arrow/csv/converter.cc | 37 +-
cpp/src/arrow/csv/converter_test.cc | 20 +
cpp/src/arrow/dataset/dataset_test.cc | 25 +-
cpp/src/arrow/dataset/file_base.cc | 6 -
cpp/src/arrow/dataset/file_parquet.cc | 21 +-
cpp/src/arrow/dataset/file_parquet_test.cc | 17 +-
cpp/src/arrow/dataset/filter.cc | 60 +-
cpp/src/arrow/dataset/partition.cc | 142 +-
cpp/src/arrow/dataset/partition.h | 13 +-
cpp/src/arrow/dataset/partition_test.cc | 46 +-
cpp/src/arrow/dataset/projector.cc | 14 +-
cpp/src/arrow/dataset/test_util.h | 3 +-
cpp/src/arrow/filesystem/CMakeLists.txt | 18 +-
cpp/src/arrow/filesystem/s3fs_test.cc | 39 +-
cpp/src/arrow/flight/CMakeLists.txt | 2 +-
cpp/src/arrow/ipc/CMakeLists.txt | 3 +-
cpp/src/arrow/ipc/dictionary.cc | 15 +-
cpp/src/arrow/ipc/dictionary.h | 5 +-
cpp/src/arrow/ipc/read_write_test.cc | 652 +-
cpp/src/arrow/ipc/reader.cc | 141 +-
cpp/src/arrow/ipc/reader.h | 29 +-
cpp/src/arrow/ipc/tensor_test.cc | 506 +
cpp/src/arrow/ipc/writer.cc | 86 +-
cpp/src/arrow/ipc/writer.h | 3 +
cpp/src/arrow/pretty_print.cc | 2 +-
cpp/src/arrow/python/arrow_to_pandas.cc | 247 +-
cpp/src/arrow/python/arrow_to_pandas.h | 3 +
cpp/src/arrow/python/extension_type.cc | 11 +
cpp/src/arrow/python/extension_type.h | 2 +
cpp/src/arrow/python/python_to_arrow.cc | 2 +-
cpp/src/arrow/testing/generator.h | 2 +
cpp/src/arrow/testing/random.cc | 33 +-
cpp/src/arrow/testing/random.h | 17 +-
cpp/src/arrow/util/future.h | 29 +-
cpp/src/arrow/util/thread_pool.h | 4 +-
cpp/src/arrow/util/value_parsing_test.cc | 25 +
cpp/src/parquet/arrow/arrow_reader_writer_test.cc | 66 +-
cpp/src/parquet/arrow/arrow_schema_test.cc | 86 +
cpp/src/parquet/arrow/path_internal.cc | 10 +-
cpp/src/parquet/arrow/path_internal_test.cc | 22 +-
cpp/src/parquet/arrow/reader.cc | 96 +-
cpp/src/parquet/arrow/reader_writer_benchmark.cc | 181 +-
cpp/src/parquet/arrow/schema.cc | 197 +-
cpp/src/parquet/arrow/schema.h | 3 -
cpp/src/parquet/level_conversion.cc | 57 +-
cpp/src/parquet/level_conversion.h | 2 +-
cpp/src/parquet/level_conversion_inc.h | 242 +-
cpp/src/parquet/level_conversion_test.cc | 24 +-
cpp/src/parquet/reader_test.cc | 12 +-
cpp/thirdparty/versions.txt | 5 +-
csharp/README.md | 2 +-
csharp/src/Apache.Arrow/Arrays/StructArray.cs | 29 +-
.../Apache.Arrow/Ipc/ArrowReaderImplementation.cs | 38 +-
csharp/src/Apache.Arrow/Ipc/ArrowStreamWriter.cs | 19 +-
.../Apache.Arrow/Ipc/ArrowTypeFlatbufferBuilder.cs | 19 +-
csharp/src/Apache.Arrow/Ipc/MessageSerializer.cs | 23 +-
csharp/src/Apache.Arrow/Types/ListType.cs | 4 +-
csharp/src/Apache.Arrow/Types/NestedType.cs | 21 +-
csharp/src/Apache.Arrow/Types/StructType.cs | 23 +-
.../test/Apache.Arrow.Tests/ArrowReaderVerifier.cs | 27 +-
csharp/test/Apache.Arrow.Tests/StructArrayTests.cs | 86 +
csharp/test/Apache.Arrow.Tests/TestData.cs | 32 +-
csharp/test/Apache.Arrow.Tests/TypeTests.cs | 18 +-
dev/tasks/homebrew-formulae/travis.osx.r.yml | 2 +-
dev/tasks/python-wheels/osx-build.sh | 21 +-
dev/tasks/python-wheels/travis.osx.yml | 7 +-
dev/tasks/tasks.yml | 58 +-
docker-compose.yml | 9 +-
docs/source/cpp/csv.rst | 4 +
docs/source/developers/python.rst | 5 +
docs/source/format/CDataInterface.rst | 2 +-
docs/source/python/filesystems_deprecated.rst | 6 +-
docs/source/status.rst | 2 +-
.../java/org/apache/arrow/flight/FlightClient.java | 31 +-
.../test/java/org/apache/arrow/flight/TestTls.java | 16 +
js/gulp/arrow-task.js | 8 +-
js/gulp/closure-task.js | 4 +-
js/gulp/test-task.js | 2 +-
js/package-lock.json | 20123 +++++++++++--------
js/package.json | 24 +-
js/src/builder.ts | 8 +-
js/src/column.ts | 15 +-
js/src/data.ts | 5 +-
js/src/interfaces.ts | 196 +-
js/src/io/adapters.ts | 48 +-
js/src/io/node/iterable.ts | 20 +-
js/src/io/stream.ts | 9 +-
js/src/io/whatwg/iterable.ts | 17 +-
js/src/ipc/reader.ts | 4 +-
js/src/type.ts | 1 +
js/src/util/bn.ts | 1 +
js/src/util/buffer.ts | 8 +-
js/src/vector.ts | 6 +-
js/src/vector/bool.ts | 6 +-
js/src/vector/date.ts | 6 +-
js/src/vector/index.ts | 4 +-
js/src/vector/utf8.ts | 6 +-
js/src/visitor.ts | 7 +
js/src/visitor/indexof.ts | 2 +-
js/src/visitor/iterator.ts | 4 +-
js/src/visitor/toarray.ts | 2 +-
js/test/Arrow.ts | 7 +
js/test/inference/column.ts | 29 +-
js/test/inference/nested.ts | 21 +-
js/test/unit/builders/builder-tests.ts | 2 +-
js/test/unit/ipc/helpers.ts | 4 +-
js/test/unit/ipc/reader/streams-node-tests.ts | 3 +-
js/test/unit/visitor-tests.ts | 5 +-
js/tsconfig/tsconfig.base.json | 14 +-
python/pyarrow/__init__.py | 43 +-
python/pyarrow/_dataset.pyx | 69 +-
python/pyarrow/_flight.pyx | 4 +-
python/pyarrow/_parquet.pyx | 7 +-
python/pyarrow/array.pxi | 4 +-
python/pyarrow/filesystem.py | 39 +-
python/pyarrow/hdfs.py | 27 +-
python/pyarrow/includes/libarrow_dataset.pxd | 2 +-
python/pyarrow/pandas_compat.py | 11 +-
python/pyarrow/parquet.py | 5 +-
python/pyarrow/table.pxi | 2 +-
python/pyarrow/tests/test_convert_builtin.py | 9 +
python/pyarrow/tests/test_csv.py | 26 +-
python/pyarrow/tests/test_dataset.py | 94 +-
python/pyarrow/tests/test_extension_type.py | 75 +-
python/pyarrow/tests/test_filesystem.py | 30 +
python/pyarrow/tests/test_hdfs.py | 3 +-
python/pyarrow/tests/test_pandas.py | 99 +
python/pyarrow/tests/test_parquet.py | 50 +-
python/pyarrow/tests/test_schema.py | 11 +
python/pyarrow/types.pxi | 13 +-
python/pyarrow/util.py | 15 +-
r/DESCRIPTION | 1 +
r/NEWS.md | 4 +-
r/R/install-arrow.R | 6 +-
r/README.md | 6 +-
r/configure | 6 +-
r/inst/build_arrow_static.sh | 2 +-
r/inst/include/cpp11.hpp | 25 -
r/inst/include/cpp11/R.hpp | 49 -
r/inst/include/cpp11/altrep.hpp | 44 -
r/inst/include/cpp11/as.hpp | 339 -
r/inst/include/cpp11/attribute_proxy.hpp | 50 -
r/inst/include/cpp11/data_frame.hpp | 102 -
r/inst/include/cpp11/declarations.hpp | 53 -
r/inst/include/cpp11/doubles.hpp | 136 -
r/inst/include/cpp11/environment.hpp | 75 -
r/inst/include/cpp11/external_pointer.hpp | 165 -
r/inst/include/cpp11/function.hpp | 78 -
r/inst/include/cpp11/integers.hpp | 142 -
r/inst/include/cpp11/list.hpp | 138 -
r/inst/include/cpp11/list_of.hpp | 53 -
r/inst/include/cpp11/logicals.hpp | 140 -
r/inst/include/cpp11/matrix.hpp | 111 -
r/inst/include/cpp11/named_arg.hpp | 51 -
r/inst/include/cpp11/protect.hpp | 286 -
r/inst/include/cpp11/r_string.hpp | 92 -
r/inst/include/cpp11/r_vector.hpp | 986 -
r/inst/include/cpp11/raws.hpp | 148 -
r/inst/include/cpp11/sexp.hpp | 79 -
r/inst/include/cpp11/strings.hpp | 187 -
r/src/Makevars.in | 7 +-
r/src/array_to_vector.cpp | 15 +-
r/src/arrow_cpp11.h | 2 +
r/tests/testthat/test-Array.R | 14 +
r/tests/testthat/test-install-arrow.R | 11 +-
r/tools/linuxlibs.R | 43 +-
r/vignettes/fs.Rmd | 7 +-
r/vignettes/install.Rmd | 23 +-
rust/README.md | 53 +-
rust/arrow/benches/cast_kernels.rs | 103 +-
rust/arrow/src/array/array.rs | 125 +-
rust/arrow/src/array/equal.rs | 18 +-
rust/arrow/src/array/iterator.rs | 85 +
rust/arrow/src/array/mod.rs | 6 +
rust/arrow/src/compute/kernels/aggregate.rs | 12 +-
rust/arrow/src/compute/kernels/cast.rs | 99 +-
rust/arrow/src/compute/kernels/comparison.rs | 2 +-
rust/arrow/src/compute/kernels/filter.rs | 3 +-
rust/arrow/src/compute/kernels/substring.rs | 11 +-
rust/arrow/src/compute/kernels/take.rs | 14 +-
rust/arrow/src/csv/reader.rs | 44 +-
rust/arrow/src/json/reader.rs | 25 +-
rust/arrow/src/util/bit_util.rs | 1 +
rust/arrow/src/util/pretty.rs | 140 +-
rust/datafusion/README.md | 53 -
rust/datafusion/examples/dataframe.rs | 4 +-
...{memory_table_api.rs => dataframe_in_memory.rs} | 2 +-
rust/datafusion/examples/flight_client.rs | 3 +
rust/datafusion/examples/flight_server.rs | 7 +-
rust/datafusion/examples/simple_udaf.rs | 1 +
rust/datafusion/src/datasource/memory.rs | 9 +-
rust/datafusion/src/datasource/parquet.rs | 4 +-
rust/datafusion/src/execution/context.rs | 16 +-
rust/datafusion/src/lib.rs | 160 +-
rust/datafusion/src/logical_plan/mod.rs | 89 +-
rust/datafusion/src/physical_plan/common.rs | 11 +-
rust/datafusion/src/physical_plan/csv.rs | 18 +-
rust/datafusion/src/physical_plan/empty.rs | 14 +-
rust/datafusion/src/physical_plan/explain.rs | 19 +-
rust/datafusion/src/physical_plan/expressions.rs | 714 +-
rust/datafusion/src/physical_plan/filter.rs | 17 +-
.../datafusion/src/physical_plan/hash_aggregate.rs | 311 +-
rust/datafusion/src/physical_plan/limit.rs | 40 +-
rust/datafusion/src/physical_plan/memory.rs | 12 +-
rust/datafusion/src/physical_plan/merge.rs | 22 +-
rust/datafusion/src/physical_plan/mod.rs | 11 +-
rust/datafusion/src/physical_plan/parquet.rs | 15 +-
rust/datafusion/src/physical_plan/planner.rs | 38 +-
rust/datafusion/src/physical_plan/projection.rs | 18 +-
rust/datafusion/src/physical_plan/sort.rs | 14 +-
rust/datafusion/src/sql/planner.rs | 1 +
rust/datafusion/tests/sql.rs | 628 +-
rust/datafusion/tests/user_defined_plan.rs | 118 +-
rust/parquet/src/util/hash_util.rs | 2 +-
249 files changed, 17743 insertions(+), 14698 deletions(-)
copy c_glib/Gemfile => .github/workflows/dev_labeler.yml (66%)
copy cpp/src/jni/CMakeLists.txt => .github/workflows/dev_labeler/labeler.yml
(92%)
delete mode 100644 ci/etc/integration_spark_ARROW-9438.patch
create mode 100644 cpp/src/arrow/ipc/tensor_test.cc
create mode 100644 csharp/test/Apache.Arrow.Tests/StructArrayTests.cs
delete mode 100644 r/inst/include/cpp11.hpp
delete mode 100644 r/inst/include/cpp11/R.hpp
delete mode 100644 r/inst/include/cpp11/altrep.hpp
delete mode 100644 r/inst/include/cpp11/as.hpp
delete mode 100644 r/inst/include/cpp11/attribute_proxy.hpp
delete mode 100644 r/inst/include/cpp11/data_frame.hpp
delete mode 100644 r/inst/include/cpp11/declarations.hpp
delete mode 100644 r/inst/include/cpp11/doubles.hpp
delete mode 100644 r/inst/include/cpp11/environment.hpp
delete mode 100644 r/inst/include/cpp11/external_pointer.hpp
delete mode 100644 r/inst/include/cpp11/function.hpp
delete mode 100644 r/inst/include/cpp11/integers.hpp
delete mode 100644 r/inst/include/cpp11/list.hpp
delete mode 100644 r/inst/include/cpp11/list_of.hpp
delete mode 100644 r/inst/include/cpp11/logicals.hpp
delete mode 100644 r/inst/include/cpp11/matrix.hpp
delete mode 100644 r/inst/include/cpp11/named_arg.hpp
delete mode 100644 r/inst/include/cpp11/protect.hpp
delete mode 100644 r/inst/include/cpp11/r_string.hpp
delete mode 100644 r/inst/include/cpp11/r_vector.hpp
delete mode 100644 r/inst/include/cpp11/raws.hpp
delete mode 100644 r/inst/include/cpp11/sexp.hpp
delete mode 100644 r/inst/include/cpp11/strings.hpp
create mode 100644 rust/arrow/src/array/iterator.rs
rename rust/datafusion/examples/{memory_table_api.rs =>
dataframe_in_memory.rs} (96%)
