This is an automated email from the ASF dual-hosted git repository. jorgecarleitao pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/arrow-rs.git
commit 77caed688512d71c074414e5c143edca63202379 Author: Jorge C. Leitao <[email protected]> AuthorDate: Sun Apr 18 14:50:11 2021 +0000 Flatten directory. --- .gitignore | 87 +------ rust/Cargo.toml => Cargo.toml | 0 README.md | 254 ++++++++++++++------- {rust/arrow-flight => arrow-flight}/Cargo.toml | 0 {rust/arrow-flight => arrow-flight}/README.md | 0 {rust/arrow-flight => arrow-flight}/build.rs | 0 .../examples/server.rs | 0 .../src/arrow.flight.protocol.rs | 0 {rust/arrow-flight => arrow-flight}/src/lib.rs | 0 {rust/arrow-flight => arrow-flight}/src/utils.rs | 0 .../.cargo/config | 0 .../.gitignore | 0 .../Cargo.toml | 0 .../README.md | 0 .../pyproject.toml | 0 .../src/lib.rs | 0 .../tests/test_sql.py | 0 {rust/arrow => arrow}/Cargo.toml | 0 {rust/arrow => arrow}/README.md | 0 {rust/arrow => arrow}/benches/aggregate_kernels.rs | 0 .../arrow => arrow}/benches/arithmetic_kernels.rs | 0 {rust/arrow => arrow}/benches/array_from_vec.rs | 0 {rust/arrow => arrow}/benches/array_slice.rs | 0 {rust/arrow => arrow}/benches/bit_length_kernel.rs | 0 {rust/arrow => arrow}/benches/boolean_kernels.rs | 0 {rust/arrow => arrow}/benches/buffer_bit_ops.rs | 0 {rust/arrow => arrow}/benches/buffer_create.rs | 0 {rust/arrow => arrow}/benches/builder.rs | 0 {rust/arrow => arrow}/benches/cast_kernels.rs | 0 .../arrow => arrow}/benches/comparison_kernels.rs | 0 .../arrow => arrow}/benches/concatenate_kernel.rs | 0 {rust/arrow => arrow}/benches/csv_writer.rs | 0 {rust/arrow => arrow}/benches/equal.rs | 0 {rust/arrow => arrow}/benches/filter_kernels.rs | 0 {rust/arrow => arrow}/benches/json_reader.rs | 0 {rust/arrow => arrow}/benches/length_kernel.rs | 0 {rust/arrow => arrow}/benches/mutable_array.rs | 0 {rust/arrow => arrow}/benches/sort_kernel.rs | 0 {rust/arrow => arrow}/benches/take_kernels.rs | 0 {rust/arrow => arrow}/build.rs | 0 {rust/arrow => arrow}/examples/builders.rs | 0 {rust/arrow => arrow}/examples/dynamic_types.rs | 0 {rust/arrow => arrow}/examples/read_csv.rs | 0 .../examples/read_csv_infer_schema.rs | 0 {rust/arrow => arrow}/examples/tensor_builder.rs | 0 {rust/arrow => arrow}/format-0ed34c83.patch | 0 {rust/arrow => arrow}/regen.sh | 0 {rust/arrow => arrow}/src/alloc/alignment.rs | 0 {rust/arrow => arrow}/src/alloc/mod.rs | 0 {rust/arrow => arrow}/src/alloc/types.rs | 0 {rust/arrow => arrow}/src/arch/avx512.rs | 0 {rust/arrow => arrow}/src/arch/mod.rs | 0 {rust/arrow => arrow}/src/array/array.rs | 0 {rust/arrow => arrow}/src/array/array_binary.rs | 0 {rust/arrow => arrow}/src/array/array_boolean.rs | 0 .../arrow => arrow}/src/array/array_dictionary.rs | 0 {rust/arrow => arrow}/src/array/array_list.rs | 0 {rust/arrow => arrow}/src/array/array_primitive.rs | 0 {rust/arrow => arrow}/src/array/array_string.rs | 0 {rust/arrow => arrow}/src/array/array_struct.rs | 0 {rust/arrow => arrow}/src/array/array_union.rs | 0 {rust/arrow => arrow}/src/array/builder.rs | 0 {rust/arrow => arrow}/src/array/cast.rs | 0 {rust/arrow => arrow}/src/array/data.rs | 0 {rust/arrow => arrow}/src/array/equal/boolean.rs | 0 {rust/arrow => arrow}/src/array/equal/decimal.rs | 0 .../arrow => arrow}/src/array/equal/dictionary.rs | 0 .../src/array/equal/fixed_binary.rs | 0 .../arrow => arrow}/src/array/equal/fixed_list.rs | 0 {rust/arrow => arrow}/src/array/equal/list.rs | 0 {rust/arrow => arrow}/src/array/equal/mod.rs | 0 {rust/arrow => arrow}/src/array/equal/null.rs | 0 {rust/arrow => arrow}/src/array/equal/primitive.rs | 0 {rust/arrow => arrow}/src/array/equal/structure.rs | 0 {rust/arrow => arrow}/src/array/equal/utils.rs | 0 .../src/array/equal/variable_size.rs | 0 {rust/arrow => arrow}/src/array/equal_json.rs | 0 {rust/arrow => arrow}/src/array/ffi.rs | 0 {rust/arrow => arrow}/src/array/iterator.rs | 0 {rust/arrow => arrow}/src/array/mod.rs | 0 {rust/arrow => arrow}/src/array/null.rs | 0 {rust/arrow => arrow}/src/array/ord.rs | 0 {rust/arrow => arrow}/src/array/raw_pointer.rs | 0 .../arrow => arrow}/src/array/transform/boolean.rs | 0 .../src/array/transform/fixed_binary.rs | 0 {rust/arrow => arrow}/src/array/transform/list.rs | 0 {rust/arrow => arrow}/src/array/transform/mod.rs | 0 {rust/arrow => arrow}/src/array/transform/null.rs | 0 .../src/array/transform/primitive.rs | 0 .../src/array/transform/structure.rs | 0 {rust/arrow => arrow}/src/array/transform/utils.rs | 0 .../src/array/transform/variable_size.rs | 0 {rust/arrow => arrow}/src/bitmap.rs | 0 {rust/arrow => arrow}/src/buffer/immutable.rs | 0 {rust/arrow => arrow}/src/buffer/mod.rs | 0 {rust/arrow => arrow}/src/buffer/mutable.rs | 0 {rust/arrow => arrow}/src/buffer/ops.rs | 0 {rust/arrow => arrow}/src/bytes.rs | 0 .../src/compute/kernels/aggregate.rs | 0 .../src/compute/kernels/arithmetic.rs | 0 {rust/arrow => arrow}/src/compute/kernels/arity.rs | 0 .../arrow => arrow}/src/compute/kernels/boolean.rs | 0 {rust/arrow => arrow}/src/compute/kernels/cast.rs | 0 .../src/compute/kernels/cast_utils.rs | 0 .../src/compute/kernels/comparison.rs | 0 .../arrow => arrow}/src/compute/kernels/concat.rs | 0 .../arrow => arrow}/src/compute/kernels/filter.rs | 0 .../arrow => arrow}/src/compute/kernels/length.rs | 0 {rust/arrow => arrow}/src/compute/kernels/limit.rs | 0 {rust/arrow => arrow}/src/compute/kernels/mod.rs | 0 .../arrow => arrow}/src/compute/kernels/regexp.rs | 0 {rust/arrow => arrow}/src/compute/kernels/sort.rs | 0 .../src/compute/kernels/substring.rs | 0 {rust/arrow => arrow}/src/compute/kernels/take.rs | 0 .../src/compute/kernels/temporal.rs | 0 .../arrow => arrow}/src/compute/kernels/window.rs | 0 {rust/arrow => arrow}/src/compute/kernels/zip.rs | 0 {rust/arrow => arrow}/src/compute/mod.rs | 0 {rust/arrow => arrow}/src/compute/util.rs | 0 {rust/arrow => arrow}/src/csv/mod.rs | 0 {rust/arrow => arrow}/src/csv/reader.rs | 0 {rust/arrow => arrow}/src/csv/writer.rs | 0 {rust/arrow => arrow}/src/datatypes/datatype.rs | 0 {rust/arrow => arrow}/src/datatypes/field.rs | 0 {rust/arrow => arrow}/src/datatypes/mod.rs | 0 {rust/arrow => arrow}/src/datatypes/native.rs | 0 {rust/arrow => arrow}/src/datatypes/numeric.rs | 0 {rust/arrow => arrow}/src/datatypes/schema.rs | 0 {rust/arrow => arrow}/src/datatypes/types.rs | 0 {rust/arrow => arrow}/src/error.rs | 0 {rust/arrow => arrow}/src/ffi.rs | 0 {rust/arrow => arrow}/src/ipc/convert.rs | 0 {rust/arrow => arrow}/src/ipc/gen/File.rs | 0 {rust/arrow => arrow}/src/ipc/gen/Message.rs | 0 {rust/arrow => arrow}/src/ipc/gen/Schema.rs | 0 {rust/arrow => arrow}/src/ipc/gen/SparseTensor.rs | 0 {rust/arrow => arrow}/src/ipc/gen/Tensor.rs | 0 {rust/arrow => arrow}/src/ipc/gen/mod.rs | 0 {rust/arrow => arrow}/src/ipc/mod.rs | 0 {rust/arrow => arrow}/src/ipc/reader.rs | 0 {rust/arrow => arrow}/src/ipc/writer.rs | 0 {rust/arrow => arrow}/src/json/mod.rs | 0 {rust/arrow => arrow}/src/json/reader.rs | 0 {rust/arrow => arrow}/src/json/writer.rs | 0 {rust/arrow => arrow}/src/lib.rs | 0 {rust/arrow => arrow}/src/record_batch.rs | 0 {rust/arrow => arrow}/src/temporal_conversions.rs | 0 {rust/arrow => arrow}/src/tensor.rs | 0 {rust/arrow => arrow}/src/util/bench_util.rs | 0 .../arrow => arrow}/src/util/bit_chunk_iterator.rs | 0 {rust/arrow => arrow}/src/util/bit_util.rs | 0 {rust/arrow => arrow}/src/util/data_gen.rs | 0 {rust/arrow => arrow}/src/util/display.rs | 0 {rust/arrow => arrow}/src/util/integration_util.rs | 0 {rust/arrow => arrow}/src/util/mod.rs | 0 {rust/arrow => arrow}/src/util/pretty.rs | 0 {rust/arrow => arrow}/src/util/serialization.rs | 0 {rust/arrow => arrow}/src/util/string_writer.rs | 0 {rust/arrow => arrow}/src/util/test_util.rs | 0 {rust/arrow => arrow}/src/util/trusted_len.rs | 0 {rust/arrow => arrow}/src/zz_memory_check.rs | 0 {rust/arrow => arrow}/test/data/arrays.json | 0 {rust/arrow => arrow}/test/data/basic.json | 0 {rust/arrow => arrow}/test/data/basic_nulls.json | 0 {rust/arrow => arrow}/test/data/integration.json | 0 .../test/data/list_string_dict_nested.json | 0 .../test/data/list_string_dict_nested_nulls.json | 0 {rust/arrow => arrow}/test/data/mixed_arrays.json | 0 .../arrow => arrow}/test/data/mixed_arrays.json.gz | Bin .../arrow => arrow}/test/data/nested_structs.json | 0 {rust/arrow => arrow}/test/data/null_test.csv | 0 {rust/arrow => arrow}/test/data/uk_cities.csv | 0 .../test/data/uk_cities_with_headers.csv | 0 {rust/arrow => arrow}/test/data/various_types.csv | 0 .../test/data/various_types_invalid.csv | 0 .../Cargo.toml | 0 .../README.md | 0 .../src/bin/arrow-file-to-stream.rs | 0 .../src/bin/arrow-json-integration-test.rs | 0 .../src/bin/arrow-stream-to-file.rs | 0 .../src/bin/flight-test-integration-client.rs | 0 .../src/bin/flight-test-integration-server.rs | 0 .../src/flight_client_scenarios.rs | 0 .../flight_client_scenarios/auth_basic_proto.rs | 0 .../flight_client_scenarios/integration_test.rs | 0 .../src/flight_client_scenarios/middleware.rs | 0 .../src/flight_server_scenarios.rs | 0 .../flight_server_scenarios/auth_basic_proto.rs | 0 .../flight_server_scenarios/integration_test.rs | 0 .../src/flight_server_scenarios/middleware.rs | 0 .../src/lib.rs | 0 {rust/parquet => parquet}/Cargo.toml | 0 {rust/parquet => parquet}/README.md | 0 {rust/parquet => parquet}/benches/arrow_writer.rs | 0 {rust/parquet => parquet}/build.rs | 0 .../parquet => parquet}/src/arrow/array_reader.rs | 0 .../parquet => parquet}/src/arrow/arrow_reader.rs | 0 .../parquet => parquet}/src/arrow/arrow_writer.rs | 0 {rust/parquet => parquet}/src/arrow/converter.rs | 0 {rust/parquet => parquet}/src/arrow/levels.rs | 0 {rust/parquet => parquet}/src/arrow/mod.rs | 0 .../parquet => parquet}/src/arrow/record_reader.rs | 0 {rust/parquet => parquet}/src/arrow/schema.rs | 0 {rust/parquet => parquet}/src/basic.rs | 0 {rust/parquet => parquet}/src/bin/parquet-read.rs | 0 .../src/bin/parquet-rowcount.rs | 0 .../parquet => parquet}/src/bin/parquet-schema.rs | 0 {rust/parquet => parquet}/src/column/mod.rs | 0 {rust/parquet => parquet}/src/column/page.rs | 0 {rust/parquet => parquet}/src/column/reader.rs | 0 {rust/parquet => parquet}/src/column/writer.rs | 0 {rust/parquet => parquet}/src/compression.rs | 0 {rust/parquet => parquet}/src/data_type.rs | 0 .../parquet => parquet}/src/encodings/decoding.rs | 0 .../parquet => parquet}/src/encodings/encoding.rs | 0 {rust/parquet => parquet}/src/encodings/levels.rs | 0 {rust/parquet => parquet}/src/encodings/mod.rs | 0 {rust/parquet => parquet}/src/encodings/rle.rs | 0 {rust/parquet => parquet}/src/errors.rs | 0 {rust/parquet => parquet}/src/file/footer.rs | 0 {rust/parquet => parquet}/src/file/metadata.rs | 0 {rust/parquet => parquet}/src/file/mod.rs | 0 {rust/parquet => parquet}/src/file/properties.rs | 0 {rust/parquet => parquet}/src/file/reader.rs | 0 .../src/file/serialized_reader.rs | 0 {rust/parquet => parquet}/src/file/statistics.rs | 0 {rust/parquet => parquet}/src/file/writer.rs | 0 {rust/parquet => parquet}/src/lib.rs | 0 {rust/parquet => parquet}/src/record/api.rs | 0 {rust/parquet => parquet}/src/record/mod.rs | 0 {rust/parquet => parquet}/src/record/reader.rs | 0 .../src/record/record_writer.rs | 0 {rust/parquet => parquet}/src/record/triplet.rs | 0 {rust/parquet => parquet}/src/schema/mod.rs | 0 {rust/parquet => parquet}/src/schema/parser.rs | 0 {rust/parquet => parquet}/src/schema/printer.rs | 0 {rust/parquet => parquet}/src/schema/types.rs | 0 {rust/parquet => parquet}/src/schema/visitor.rs | 0 {rust/parquet => parquet}/src/util/bit_packing.rs | 0 {rust/parquet => parquet}/src/util/bit_util.rs | 0 {rust/parquet => parquet}/src/util/cursor.rs | 0 {rust/parquet => parquet}/src/util/hash_util.rs | 0 {rust/parquet => parquet}/src/util/io.rs | 0 {rust/parquet => parquet}/src/util/memory.rs | 0 {rust/parquet => parquet}/src/util/mod.rs | 0 .../src/util/test_common/file_util.rs | 0 .../src/util/test_common/mod.rs | 0 .../src/util/test_common/page_util.rs | 0 .../src/util/test_common/rand_gen.rs | 0 {rust/parquet => parquet}/tests/custom_writer.rs | 0 {rust/parquet_derive => parquet_derive}/Cargo.toml | 0 {rust/parquet_derive => parquet_derive}/README.md | 0 {rust/parquet_derive => parquet_derive}/src/lib.rs | 0 .../src/parquet_field.rs | 0 .../Cargo.toml | 0 .../src/lib.rs | 0 rust/pre-commit.sh => pre-commit.sh | 0 rust/.gitignore | 5 - rust/README.md | 186 --------------- rust/rustfmt.toml => rustfmt.toml | 0 260 files changed, 172 insertions(+), 360 deletions(-) diff --git a/.gitignore b/.gitignore index 6f12336..389f4ab 100644 --- a/.gitignore +++ b/.gitignore @@ -1,82 +1,5 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. - -apache-rat-*.jar -arrow-src.tar -arrow-src.tar.gz - -# Compiled source -*.a -*.dll -*.o -*.py[ocd] -*.so -*.so.* -*.bundle -*.dylib -.build_cache_dir -dependency-reduced-pom.xml -MANIFEST -compile_commands.json -build.ninja - -# Generated Visual Studio files -*.vcxproj -*.vcxproj.* -*.sln -*.iml - -# Linux perf sample data -perf.data -perf.data.old - -cpp/.idea/ -.clangd/ -cpp/.clangd/ -cpp/apidoc/xml/ -docs/example.gz -docs/example1.dat -docs/example3.dat -python/.eggs/ -python/doc/ -# Egg metadata -*.egg-info - -.vscode -.idea/ -.pytest_cache/ -pkgs -docker_cache -.gdb_history -*.orig -.*.swp -.*.swo - -site/ - -# R files -**/.Rproj.user -**/*.Rcheck/ -**/.Rhistory -.Rproj.user - -# macOS -cpp/Brewfile.lock.json -.DS_Store - -# docker volumes used for caching -.docker +Cargo.lock +target +rusty-tags.vi +.history +.flatbuffers/ diff --git a/rust/Cargo.toml b/Cargo.toml similarity index 100% rename from rust/Cargo.toml rename to Cargo.toml diff --git a/README.md b/README.md index 133018c..7fdef29 100644 --- a/README.md +++ b/README.md @@ -17,90 +17,170 @@ under the License. --> -# Apache Arrow - -[](https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/branch/master) -[](https://codecov.io/gh/apache/arrow?branch=master) -[](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened&can=1&q=proj:arrow) -[](https://github.com/apache/arrow/blob/master/LICENSE.txt) -[](https://twitter.com/apachearrow) - -## Powering In-Memory Analytics - -Apache Arrow is a development platform for in-memory analytics. It contains a -set of technologies that enable big data systems to process and move data fast. - -Major components of the project include: - - - [The Arrow Columnar In-Memory Format](https://github.com/apache/arrow/blob/master/docs/source/format/Columnar.rst): - a standard and efficient in-memory representation of various datatypes, plain or nested - - [The Arrow IPC Format](https://github.com/apache/arrow/blob/master/docs/source/format/Columnar.rst#serialization-and-interprocess-communication-ipc): - an efficient serialization of the Arrow format and associated metadata, - for communication between processes and heterogeneous environments - - [The Arrow Flight RPC protocol](https://github.com/apache/arrow/tree/master/format/Flight.proto): - based on the Arrow IPC format, a building block for remote services exchanging - Arrow data with application-defined semantics (for example a storage server or a database) - - [C++ libraries](https://github.com/apache/arrow/tree/master/cpp) - - [C bindings using GLib](https://github.com/apache/arrow/tree/master/c_glib) - - [C# .NET libraries](https://github.com/apache/arrow/tree/master/csharp) - - [Gandiva](https://github.com/apache/arrow/tree/master/cpp/src/gandiva): - an [LLVM](https://llvm.org)-based Arrow expression compiler, part of the C++ codebase - - [Go libraries](https://github.com/apache/arrow/tree/master/go) - - [Java libraries](https://github.com/apache/arrow/tree/master/java) - - [JavaScript libraries](https://github.com/apache/arrow/tree/master/js) - - [Plasma Object Store](https://github.com/apache/arrow/tree/master/cpp/src/plasma): - a shared-memory blob store, part of the C++ codebase - - [Python libraries](https://github.com/apache/arrow/tree/master/python) - - [R libraries](https://github.com/apache/arrow/tree/master/r) - - [Ruby libraries](https://github.com/apache/arrow/tree/master/ruby) - - [Rust libraries](https://github.com/apache/arrow/tree/master/rust) - -Arrow is an [Apache Software Foundation](https://www.apache.org) project. Learn more at -[arrow.apache.org](https://arrow.apache.org). - -## What's in the Arrow libraries? - -The reference Arrow libraries contain many distinct software components: - -- Columnar vector and table-like containers (similar to data frames) supporting - flat or nested types -- Fast, language agnostic metadata messaging layer (using Google's Flatbuffers - library) -- Reference-counted off-heap buffer memory management, for zero-copy memory - sharing and handling memory-mapped files -- IO interfaces to local and remote filesystems -- Self-describing binary wire formats (streaming and batch/file-like) for - remote procedure calls (RPC) and interprocess communication (IPC) -- Integration tests for verifying binary compatibility between the - implementations (e.g. sending data from Java to C++) -- Conversions to and from other in-memory data structures -- Readers and writers for various widely-used file formats (such as Parquet, CSV) - -## Implementation status - -The official Arrow libraries in this repository are in different stages of -implementing the Arrow format and related features. See our current -[feature matrix](https://github.com/apache/arrow/blob/master/docs/source/status.rst) -on git master. - -## How to Contribute - -Please read our latest [project contribution guide][5]. - -## Getting involved - -Even if you do not plan to contribute to Apache Arrow itself or Arrow -integrations in other projects, we'd be happy to have you involved: - -- Join the mailing list: send an email to - [[email protected]][1]. Share your ideas and use cases for the - project. -- [Follow our activity on JIRA][3] -- [Learn the format][2] -- Contribute code to one of the reference implementations - -[1]: mailto:[email protected] -[2]: https://github.com/apache/arrow/tree/master/format -[3]: https://issues.apache.org/jira/browse/ARROW -[4]: https://github.com/apache/arrow -[5]: https://github.com/apache/arrow/blob/master/docs/source/developers/contributing.rst +# Native Rust implementation of Apache Arrow + +[](https://codecov.io/gh/apache/arrow?branch=master) + +Welcome to the implementation of Arrow, the popular in-memory columnar format, in [Rust](https://www.rust-lang.org/). + +This part of the Arrow project is divided in 4 main components: + +| Crate | Description | Documentation | +|-----------|-------------|---------------| +|Arrow | Core functionality (memory layout, arrays, low level computations) | [(README)](arrow/README.md) | +|Parquet | Parquet support | [(README)](parquet/README.md) | +|Arrow-flight | Arrow data between processes | [(README)](arrow-flight/README.md) | +|DataFusion | In-memory query engine with SQL support | [(README)](datafusion/README.md) | +|Ballista | Distributed query execution | [(README)](ballista/README.md) | + +Independently, they support a vast array of functionality for in-memory computations. + +Together, they allow users to write an SQL query or a `DataFrame` (using the `datafusion` crate), run it against a parquet file (using the `parquet` crate), evaluate it in-memory using Arrow's columnar format (using the `arrow` crate), and send to another process (using the `arrow-flight` crate). + +Generally speaking, the `arrow` crate offers functionality to develop code that uses Arrow arrays, and `datafusion` offers most operations typically found in SQL, with the notable exceptions of: + +* `join` +* `window` functions + +There are too many features to enumerate here, but some notable mentions: + +* `Arrow` implements all formats in the specification except certain dictionaries +* `Arrow` supports SIMD operations to some of its vertical operations +* `DataFusion` supports `async` execution +* `DataFusion` supports user-defined functions, aggregates, and whole execution nodes + +You can find more details about each crate in their respective READMEs. + +## Arrow Rust Community + +We use the official [ASF Slack](https://s.apache.org/slack-invite) for informal discussions and coordination. This is +a great place to meet other contributors and get guidance on where to contribute. Join us in the `arrow-rust` channel. + +We use [ASF JIRA](https://issues.apache.org/jira/secure/Dashboard.jspa) as the system of record for new features +and bug fixes and this plays a critical role in the release process. + +For design discussions we generally collaborate on Google documents and file a JIRA linking to the document. + +There is also a bi-weekly Rust-specific sync call for the Arrow Rust community. This is hosted on Google Meet +at https://meet.google.com/ctp-yujs-aee on alternate Wednesday's at 09:00 US/Pacific, 12:00 US/Eastern. During +US daylight savings time this corresponds to 16:00 UTC and at other times this is 17:00 UTC. + +## Developer's guide to Arrow Rust + +### How to compile + +This is a standard cargo project with workspaces. To build it, you need to have `rust` and `cargo`: + +```bash +cd /rust && cargo build +``` + +You can also use rust's official docker image: + +```bash +docker run --rm -v $(pwd)/rust:/rust -it rust /bin/bash -c "cd /rust && cargo build" +``` + +The command above assumes that are in the root directory of the project, not in the same +directory as this README.md. + +You can also compile specific workspaces: + +```bash +cd /rust/arrow && cargo build +``` + +### Git Submodules + +Before running tests and examples, it is necessary to set up the local development environment. + +The tests rely on test data that is contained in git submodules. + +To pull down this data run the following: + +```bash +git submodule update --init +``` + +This populates data in two git submodules: + +- `../cpp/submodules/parquet_testing/data` (sourced from https://github.com/apache/parquet-testing.git) +- `../testing` (sourced from https://github.com/apache/arrow-testing) + +By default, `cargo test` will look for these directories at their +standard location. The following environment variables can be used to override the location: + +```bash +# Optionaly specify a different location for test data +export PARQUET_TEST_DATA=$(cd ../cpp/submodules/parquet-testing/data; pwd) +export ARROW_TEST_DATA=$(cd ../testing/data; pwd) +``` + +From here on, this is a pure Rust project and `cargo` can be used to run tests, benchmarks, docs and examples as usual. + + +### Running the tests + +Run tests using the Rust standard `cargo test` command: + +```bash +# run all tests. +cargo test + + +# run only tests for the arrow crate +cargo test -p arrow +``` + +## Code Formatting + +Our CI uses `rustfmt` to check code formatting. Before submitting a +PR be sure to run the following and check for lint issues: + +```bash +cargo +stable fmt --all -- --check +``` + +## Clippy Lints + +We recommend using `clippy` for checking lints during development. While we do not yet enforce `clippy` checks, we recommend not introducing new `clippy` errors or warnings. + +Run the following to check for clippy lints. + +``` +cargo clippy +``` + +If you use Visual Studio Code with the `rust-analyzer` plugin, you can enable `clippy` to run each time you save a file. See https://users.rust-lang.org/t/how-to-use-clippy-in-vs-code-with-rust-analyzer/41881. + +One of the concerns with `clippy` is that it often produces a lot of false positives, or that some recommendations may hurt readability. We do not have a policy of which lints are ignored, but if you disagree with a `clippy` lint, you may disable the lint and briefly justify it. + +Search for `allow(clippy::` in the codebase to identify lints that are ignored/allowed. We currently prefer ignoring lints on the lowest unit possible. +* If you are introducing a line that returns a lint warning or error, you may disable the lint on that line. +* If you have several lints on a function or module, you may disable the lint on the function or module. +* If a lint is pervasive across multiple modules, you may disable it at the crate level. + +## Git Pre-Commit Hook + +We can use [git pre-commit hook](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks) to automate various kinds of git pre-commit checking/formatting. + +Suppose you are in the root directory of the project. + +First check if the file already exists: + +```bash +ls -l .git/hooks/pre-commit +``` + +If the file already exists, to avoid mistakenly **overriding**, you MAY have to check +the link source or file content. Else if not exist, let's safely soft link [pre-commit.sh](pre-commit.sh) as file `.git/hooks/pre-commit`: + +``` +ln -s ../../rust/pre-commit.sh .git/hooks/pre-commit +``` + +If sometimes you want to commit without checking, just run `git commit` with `--no-verify`: + +```bash +git commit --no-verify -m "... commit message ..." +``` diff --git a/rust/arrow-flight/Cargo.toml b/arrow-flight/Cargo.toml similarity index 100% rename from rust/arrow-flight/Cargo.toml rename to arrow-flight/Cargo.toml diff --git a/rust/arrow-flight/README.md b/arrow-flight/README.md similarity index 100% rename from rust/arrow-flight/README.md rename to arrow-flight/README.md diff --git a/rust/arrow-flight/build.rs b/arrow-flight/build.rs similarity index 100% rename from rust/arrow-flight/build.rs rename to arrow-flight/build.rs diff --git a/rust/arrow-flight/examples/server.rs b/arrow-flight/examples/server.rs similarity index 100% rename from rust/arrow-flight/examples/server.rs rename to arrow-flight/examples/server.rs diff --git a/rust/arrow-flight/src/arrow.flight.protocol.rs b/arrow-flight/src/arrow.flight.protocol.rs similarity index 100% rename from rust/arrow-flight/src/arrow.flight.protocol.rs rename to arrow-flight/src/arrow.flight.protocol.rs diff --git a/rust/arrow-flight/src/lib.rs b/arrow-flight/src/lib.rs similarity index 100% rename from rust/arrow-flight/src/lib.rs rename to arrow-flight/src/lib.rs diff --git a/rust/arrow-flight/src/utils.rs b/arrow-flight/src/utils.rs similarity index 100% rename from rust/arrow-flight/src/utils.rs rename to arrow-flight/src/utils.rs diff --git a/rust/arrow-pyarrow-integration-testing/.cargo/config b/arrow-pyarrow-integration-testing/.cargo/config similarity index 100% rename from rust/arrow-pyarrow-integration-testing/.cargo/config rename to arrow-pyarrow-integration-testing/.cargo/config diff --git a/rust/arrow-pyarrow-integration-testing/.gitignore b/arrow-pyarrow-integration-testing/.gitignore similarity index 100% rename from rust/arrow-pyarrow-integration-testing/.gitignore rename to arrow-pyarrow-integration-testing/.gitignore diff --git a/rust/arrow-pyarrow-integration-testing/Cargo.toml b/arrow-pyarrow-integration-testing/Cargo.toml similarity index 100% rename from rust/arrow-pyarrow-integration-testing/Cargo.toml rename to arrow-pyarrow-integration-testing/Cargo.toml diff --git a/rust/arrow-pyarrow-integration-testing/README.md b/arrow-pyarrow-integration-testing/README.md similarity index 100% rename from rust/arrow-pyarrow-integration-testing/README.md rename to arrow-pyarrow-integration-testing/README.md diff --git a/rust/arrow-pyarrow-integration-testing/pyproject.toml b/arrow-pyarrow-integration-testing/pyproject.toml similarity index 100% rename from rust/arrow-pyarrow-integration-testing/pyproject.toml rename to arrow-pyarrow-integration-testing/pyproject.toml diff --git a/rust/arrow-pyarrow-integration-testing/src/lib.rs b/arrow-pyarrow-integration-testing/src/lib.rs similarity index 100% rename from rust/arrow-pyarrow-integration-testing/src/lib.rs rename to arrow-pyarrow-integration-testing/src/lib.rs diff --git a/rust/arrow-pyarrow-integration-testing/tests/test_sql.py b/arrow-pyarrow-integration-testing/tests/test_sql.py similarity index 100% rename from rust/arrow-pyarrow-integration-testing/tests/test_sql.py rename to arrow-pyarrow-integration-testing/tests/test_sql.py diff --git a/rust/arrow/Cargo.toml b/arrow/Cargo.toml similarity index 100% rename from rust/arrow/Cargo.toml rename to arrow/Cargo.toml diff --git a/rust/arrow/README.md b/arrow/README.md similarity index 100% rename from rust/arrow/README.md rename to arrow/README.md diff --git a/rust/arrow/benches/aggregate_kernels.rs b/arrow/benches/aggregate_kernels.rs similarity index 100% rename from rust/arrow/benches/aggregate_kernels.rs rename to arrow/benches/aggregate_kernels.rs diff --git a/rust/arrow/benches/arithmetic_kernels.rs b/arrow/benches/arithmetic_kernels.rs similarity index 100% rename from rust/arrow/benches/arithmetic_kernels.rs rename to arrow/benches/arithmetic_kernels.rs diff --git a/rust/arrow/benches/array_from_vec.rs b/arrow/benches/array_from_vec.rs similarity index 100% rename from rust/arrow/benches/array_from_vec.rs rename to arrow/benches/array_from_vec.rs diff --git a/rust/arrow/benches/array_slice.rs b/arrow/benches/array_slice.rs similarity index 100% rename from rust/arrow/benches/array_slice.rs rename to arrow/benches/array_slice.rs diff --git a/rust/arrow/benches/bit_length_kernel.rs b/arrow/benches/bit_length_kernel.rs similarity index 100% rename from rust/arrow/benches/bit_length_kernel.rs rename to arrow/benches/bit_length_kernel.rs diff --git a/rust/arrow/benches/boolean_kernels.rs b/arrow/benches/boolean_kernels.rs similarity index 100% rename from rust/arrow/benches/boolean_kernels.rs rename to arrow/benches/boolean_kernels.rs diff --git a/rust/arrow/benches/buffer_bit_ops.rs b/arrow/benches/buffer_bit_ops.rs similarity index 100% rename from rust/arrow/benches/buffer_bit_ops.rs rename to arrow/benches/buffer_bit_ops.rs diff --git a/rust/arrow/benches/buffer_create.rs b/arrow/benches/buffer_create.rs similarity index 100% rename from rust/arrow/benches/buffer_create.rs rename to arrow/benches/buffer_create.rs diff --git a/rust/arrow/benches/builder.rs b/arrow/benches/builder.rs similarity index 100% rename from rust/arrow/benches/builder.rs rename to arrow/benches/builder.rs diff --git a/rust/arrow/benches/cast_kernels.rs b/arrow/benches/cast_kernels.rs similarity index 100% rename from rust/arrow/benches/cast_kernels.rs rename to arrow/benches/cast_kernels.rs diff --git a/rust/arrow/benches/comparison_kernels.rs b/arrow/benches/comparison_kernels.rs similarity index 100% rename from rust/arrow/benches/comparison_kernels.rs rename to arrow/benches/comparison_kernels.rs diff --git a/rust/arrow/benches/concatenate_kernel.rs b/arrow/benches/concatenate_kernel.rs similarity index 100% rename from rust/arrow/benches/concatenate_kernel.rs rename to arrow/benches/concatenate_kernel.rs diff --git a/rust/arrow/benches/csv_writer.rs b/arrow/benches/csv_writer.rs similarity index 100% rename from rust/arrow/benches/csv_writer.rs rename to arrow/benches/csv_writer.rs diff --git a/rust/arrow/benches/equal.rs b/arrow/benches/equal.rs similarity index 100% rename from rust/arrow/benches/equal.rs rename to arrow/benches/equal.rs diff --git a/rust/arrow/benches/filter_kernels.rs b/arrow/benches/filter_kernels.rs similarity index 100% rename from rust/arrow/benches/filter_kernels.rs rename to arrow/benches/filter_kernels.rs diff --git a/rust/arrow/benches/json_reader.rs b/arrow/benches/json_reader.rs similarity index 100% rename from rust/arrow/benches/json_reader.rs rename to arrow/benches/json_reader.rs diff --git a/rust/arrow/benches/length_kernel.rs b/arrow/benches/length_kernel.rs similarity index 100% rename from rust/arrow/benches/length_kernel.rs rename to arrow/benches/length_kernel.rs diff --git a/rust/arrow/benches/mutable_array.rs b/arrow/benches/mutable_array.rs similarity index 100% rename from rust/arrow/benches/mutable_array.rs rename to arrow/benches/mutable_array.rs diff --git a/rust/arrow/benches/sort_kernel.rs b/arrow/benches/sort_kernel.rs similarity index 100% rename from rust/arrow/benches/sort_kernel.rs rename to arrow/benches/sort_kernel.rs diff --git a/rust/arrow/benches/take_kernels.rs b/arrow/benches/take_kernels.rs similarity index 100% rename from rust/arrow/benches/take_kernels.rs rename to arrow/benches/take_kernels.rs diff --git a/rust/arrow/build.rs b/arrow/build.rs similarity index 100% rename from rust/arrow/build.rs rename to arrow/build.rs diff --git a/rust/arrow/examples/builders.rs b/arrow/examples/builders.rs similarity index 100% rename from rust/arrow/examples/builders.rs rename to arrow/examples/builders.rs diff --git a/rust/arrow/examples/dynamic_types.rs b/arrow/examples/dynamic_types.rs similarity index 100% rename from rust/arrow/examples/dynamic_types.rs rename to arrow/examples/dynamic_types.rs diff --git a/rust/arrow/examples/read_csv.rs b/arrow/examples/read_csv.rs similarity index 100% rename from rust/arrow/examples/read_csv.rs rename to arrow/examples/read_csv.rs diff --git a/rust/arrow/examples/read_csv_infer_schema.rs b/arrow/examples/read_csv_infer_schema.rs similarity index 100% rename from rust/arrow/examples/read_csv_infer_schema.rs rename to arrow/examples/read_csv_infer_schema.rs diff --git a/rust/arrow/examples/tensor_builder.rs b/arrow/examples/tensor_builder.rs similarity index 100% rename from rust/arrow/examples/tensor_builder.rs rename to arrow/examples/tensor_builder.rs diff --git a/rust/arrow/format-0ed34c83.patch b/arrow/format-0ed34c83.patch similarity index 100% rename from rust/arrow/format-0ed34c83.patch rename to arrow/format-0ed34c83.patch diff --git a/rust/arrow/regen.sh b/arrow/regen.sh similarity index 100% rename from rust/arrow/regen.sh rename to arrow/regen.sh diff --git a/rust/arrow/src/alloc/alignment.rs b/arrow/src/alloc/alignment.rs similarity index 100% rename from rust/arrow/src/alloc/alignment.rs rename to arrow/src/alloc/alignment.rs diff --git a/rust/arrow/src/alloc/mod.rs b/arrow/src/alloc/mod.rs similarity index 100% rename from rust/arrow/src/alloc/mod.rs rename to arrow/src/alloc/mod.rs diff --git a/rust/arrow/src/alloc/types.rs b/arrow/src/alloc/types.rs similarity index 100% rename from rust/arrow/src/alloc/types.rs rename to arrow/src/alloc/types.rs diff --git a/rust/arrow/src/arch/avx512.rs b/arrow/src/arch/avx512.rs similarity index 100% rename from rust/arrow/src/arch/avx512.rs rename to arrow/src/arch/avx512.rs diff --git a/rust/arrow/src/arch/mod.rs b/arrow/src/arch/mod.rs similarity index 100% rename from rust/arrow/src/arch/mod.rs rename to arrow/src/arch/mod.rs diff --git a/rust/arrow/src/array/array.rs b/arrow/src/array/array.rs similarity index 100% rename from rust/arrow/src/array/array.rs rename to arrow/src/array/array.rs diff --git a/rust/arrow/src/array/array_binary.rs b/arrow/src/array/array_binary.rs similarity index 100% rename from rust/arrow/src/array/array_binary.rs rename to arrow/src/array/array_binary.rs diff --git a/rust/arrow/src/array/array_boolean.rs b/arrow/src/array/array_boolean.rs similarity index 100% rename from rust/arrow/src/array/array_boolean.rs rename to arrow/src/array/array_boolean.rs diff --git a/rust/arrow/src/array/array_dictionary.rs b/arrow/src/array/array_dictionary.rs similarity index 100% rename from rust/arrow/src/array/array_dictionary.rs rename to arrow/src/array/array_dictionary.rs diff --git a/rust/arrow/src/array/array_list.rs b/arrow/src/array/array_list.rs similarity index 100% rename from rust/arrow/src/array/array_list.rs rename to arrow/src/array/array_list.rs diff --git a/rust/arrow/src/array/array_primitive.rs b/arrow/src/array/array_primitive.rs similarity index 100% rename from rust/arrow/src/array/array_primitive.rs rename to arrow/src/array/array_primitive.rs diff --git a/rust/arrow/src/array/array_string.rs b/arrow/src/array/array_string.rs similarity index 100% rename from rust/arrow/src/array/array_string.rs rename to arrow/src/array/array_string.rs diff --git a/rust/arrow/src/array/array_struct.rs b/arrow/src/array/array_struct.rs similarity index 100% rename from rust/arrow/src/array/array_struct.rs rename to arrow/src/array/array_struct.rs diff --git a/rust/arrow/src/array/array_union.rs b/arrow/src/array/array_union.rs similarity index 100% rename from rust/arrow/src/array/array_union.rs rename to arrow/src/array/array_union.rs diff --git a/rust/arrow/src/array/builder.rs b/arrow/src/array/builder.rs similarity index 100% rename from rust/arrow/src/array/builder.rs rename to arrow/src/array/builder.rs diff --git a/rust/arrow/src/array/cast.rs b/arrow/src/array/cast.rs similarity index 100% rename from rust/arrow/src/array/cast.rs rename to arrow/src/array/cast.rs diff --git a/rust/arrow/src/array/data.rs b/arrow/src/array/data.rs similarity index 100% rename from rust/arrow/src/array/data.rs rename to arrow/src/array/data.rs diff --git a/rust/arrow/src/array/equal/boolean.rs b/arrow/src/array/equal/boolean.rs similarity index 100% rename from rust/arrow/src/array/equal/boolean.rs rename to arrow/src/array/equal/boolean.rs diff --git a/rust/arrow/src/array/equal/decimal.rs b/arrow/src/array/equal/decimal.rs similarity index 100% rename from rust/arrow/src/array/equal/decimal.rs rename to arrow/src/array/equal/decimal.rs diff --git a/rust/arrow/src/array/equal/dictionary.rs b/arrow/src/array/equal/dictionary.rs similarity index 100% rename from rust/arrow/src/array/equal/dictionary.rs rename to arrow/src/array/equal/dictionary.rs diff --git a/rust/arrow/src/array/equal/fixed_binary.rs b/arrow/src/array/equal/fixed_binary.rs similarity index 100% rename from rust/arrow/src/array/equal/fixed_binary.rs rename to arrow/src/array/equal/fixed_binary.rs diff --git a/rust/arrow/src/array/equal/fixed_list.rs b/arrow/src/array/equal/fixed_list.rs similarity index 100% rename from rust/arrow/src/array/equal/fixed_list.rs rename to arrow/src/array/equal/fixed_list.rs diff --git a/rust/arrow/src/array/equal/list.rs b/arrow/src/array/equal/list.rs similarity index 100% rename from rust/arrow/src/array/equal/list.rs rename to arrow/src/array/equal/list.rs diff --git a/rust/arrow/src/array/equal/mod.rs b/arrow/src/array/equal/mod.rs similarity index 100% rename from rust/arrow/src/array/equal/mod.rs rename to arrow/src/array/equal/mod.rs diff --git a/rust/arrow/src/array/equal/null.rs b/arrow/src/array/equal/null.rs similarity index 100% rename from rust/arrow/src/array/equal/null.rs rename to arrow/src/array/equal/null.rs diff --git a/rust/arrow/src/array/equal/primitive.rs b/arrow/src/array/equal/primitive.rs similarity index 100% rename from rust/arrow/src/array/equal/primitive.rs rename to arrow/src/array/equal/primitive.rs diff --git a/rust/arrow/src/array/equal/structure.rs b/arrow/src/array/equal/structure.rs similarity index 100% rename from rust/arrow/src/array/equal/structure.rs rename to arrow/src/array/equal/structure.rs diff --git a/rust/arrow/src/array/equal/utils.rs b/arrow/src/array/equal/utils.rs similarity index 100% rename from rust/arrow/src/array/equal/utils.rs rename to arrow/src/array/equal/utils.rs diff --git a/rust/arrow/src/array/equal/variable_size.rs b/arrow/src/array/equal/variable_size.rs similarity index 100% rename from rust/arrow/src/array/equal/variable_size.rs rename to arrow/src/array/equal/variable_size.rs diff --git a/rust/arrow/src/array/equal_json.rs b/arrow/src/array/equal_json.rs similarity index 100% rename from rust/arrow/src/array/equal_json.rs rename to arrow/src/array/equal_json.rs diff --git a/rust/arrow/src/array/ffi.rs b/arrow/src/array/ffi.rs similarity index 100% rename from rust/arrow/src/array/ffi.rs rename to arrow/src/array/ffi.rs diff --git a/rust/arrow/src/array/iterator.rs b/arrow/src/array/iterator.rs similarity index 100% rename from rust/arrow/src/array/iterator.rs rename to arrow/src/array/iterator.rs diff --git a/rust/arrow/src/array/mod.rs b/arrow/src/array/mod.rs similarity index 100% rename from rust/arrow/src/array/mod.rs rename to arrow/src/array/mod.rs diff --git a/rust/arrow/src/array/null.rs b/arrow/src/array/null.rs similarity index 100% rename from rust/arrow/src/array/null.rs rename to arrow/src/array/null.rs diff --git a/rust/arrow/src/array/ord.rs b/arrow/src/array/ord.rs similarity index 100% rename from rust/arrow/src/array/ord.rs rename to arrow/src/array/ord.rs diff --git a/rust/arrow/src/array/raw_pointer.rs b/arrow/src/array/raw_pointer.rs similarity index 100% rename from rust/arrow/src/array/raw_pointer.rs rename to arrow/src/array/raw_pointer.rs diff --git a/rust/arrow/src/array/transform/boolean.rs b/arrow/src/array/transform/boolean.rs similarity index 100% rename from rust/arrow/src/array/transform/boolean.rs rename to arrow/src/array/transform/boolean.rs diff --git a/rust/arrow/src/array/transform/fixed_binary.rs b/arrow/src/array/transform/fixed_binary.rs similarity index 100% rename from rust/arrow/src/array/transform/fixed_binary.rs rename to arrow/src/array/transform/fixed_binary.rs diff --git a/rust/arrow/src/array/transform/list.rs b/arrow/src/array/transform/list.rs similarity index 100% rename from rust/arrow/src/array/transform/list.rs rename to arrow/src/array/transform/list.rs diff --git a/rust/arrow/src/array/transform/mod.rs b/arrow/src/array/transform/mod.rs similarity index 100% rename from rust/arrow/src/array/transform/mod.rs rename to arrow/src/array/transform/mod.rs diff --git a/rust/arrow/src/array/transform/null.rs b/arrow/src/array/transform/null.rs similarity index 100% rename from rust/arrow/src/array/transform/null.rs rename to arrow/src/array/transform/null.rs diff --git a/rust/arrow/src/array/transform/primitive.rs b/arrow/src/array/transform/primitive.rs similarity index 100% rename from rust/arrow/src/array/transform/primitive.rs rename to arrow/src/array/transform/primitive.rs diff --git a/rust/arrow/src/array/transform/structure.rs b/arrow/src/array/transform/structure.rs similarity index 100% rename from rust/arrow/src/array/transform/structure.rs rename to arrow/src/array/transform/structure.rs diff --git a/rust/arrow/src/array/transform/utils.rs b/arrow/src/array/transform/utils.rs similarity index 100% rename from rust/arrow/src/array/transform/utils.rs rename to arrow/src/array/transform/utils.rs diff --git a/rust/arrow/src/array/transform/variable_size.rs b/arrow/src/array/transform/variable_size.rs similarity index 100% rename from rust/arrow/src/array/transform/variable_size.rs rename to arrow/src/array/transform/variable_size.rs diff --git a/rust/arrow/src/bitmap.rs b/arrow/src/bitmap.rs similarity index 100% rename from rust/arrow/src/bitmap.rs rename to arrow/src/bitmap.rs diff --git a/rust/arrow/src/buffer/immutable.rs b/arrow/src/buffer/immutable.rs similarity index 100% rename from rust/arrow/src/buffer/immutable.rs rename to arrow/src/buffer/immutable.rs diff --git a/rust/arrow/src/buffer/mod.rs b/arrow/src/buffer/mod.rs similarity index 100% rename from rust/arrow/src/buffer/mod.rs rename to arrow/src/buffer/mod.rs diff --git a/rust/arrow/src/buffer/mutable.rs b/arrow/src/buffer/mutable.rs similarity index 100% rename from rust/arrow/src/buffer/mutable.rs rename to arrow/src/buffer/mutable.rs diff --git a/rust/arrow/src/buffer/ops.rs b/arrow/src/buffer/ops.rs similarity index 100% rename from rust/arrow/src/buffer/ops.rs rename to arrow/src/buffer/ops.rs diff --git a/rust/arrow/src/bytes.rs b/arrow/src/bytes.rs similarity index 100% rename from rust/arrow/src/bytes.rs rename to arrow/src/bytes.rs diff --git a/rust/arrow/src/compute/kernels/aggregate.rs b/arrow/src/compute/kernels/aggregate.rs similarity index 100% rename from rust/arrow/src/compute/kernels/aggregate.rs rename to arrow/src/compute/kernels/aggregate.rs diff --git a/rust/arrow/src/compute/kernels/arithmetic.rs b/arrow/src/compute/kernels/arithmetic.rs similarity index 100% rename from rust/arrow/src/compute/kernels/arithmetic.rs rename to arrow/src/compute/kernels/arithmetic.rs diff --git a/rust/arrow/src/compute/kernels/arity.rs b/arrow/src/compute/kernels/arity.rs similarity index 100% rename from rust/arrow/src/compute/kernels/arity.rs rename to arrow/src/compute/kernels/arity.rs diff --git a/rust/arrow/src/compute/kernels/boolean.rs b/arrow/src/compute/kernels/boolean.rs similarity index 100% rename from rust/arrow/src/compute/kernels/boolean.rs rename to arrow/src/compute/kernels/boolean.rs diff --git a/rust/arrow/src/compute/kernels/cast.rs b/arrow/src/compute/kernels/cast.rs similarity index 100% rename from rust/arrow/src/compute/kernels/cast.rs rename to arrow/src/compute/kernels/cast.rs diff --git a/rust/arrow/src/compute/kernels/cast_utils.rs b/arrow/src/compute/kernels/cast_utils.rs similarity index 100% rename from rust/arrow/src/compute/kernels/cast_utils.rs rename to arrow/src/compute/kernels/cast_utils.rs diff --git a/rust/arrow/src/compute/kernels/comparison.rs b/arrow/src/compute/kernels/comparison.rs similarity index 100% rename from rust/arrow/src/compute/kernels/comparison.rs rename to arrow/src/compute/kernels/comparison.rs diff --git a/rust/arrow/src/compute/kernels/concat.rs b/arrow/src/compute/kernels/concat.rs similarity index 100% rename from rust/arrow/src/compute/kernels/concat.rs rename to arrow/src/compute/kernels/concat.rs diff --git a/rust/arrow/src/compute/kernels/filter.rs b/arrow/src/compute/kernels/filter.rs similarity index 100% rename from rust/arrow/src/compute/kernels/filter.rs rename to arrow/src/compute/kernels/filter.rs diff --git a/rust/arrow/src/compute/kernels/length.rs b/arrow/src/compute/kernels/length.rs similarity index 100% rename from rust/arrow/src/compute/kernels/length.rs rename to arrow/src/compute/kernels/length.rs diff --git a/rust/arrow/src/compute/kernels/limit.rs b/arrow/src/compute/kernels/limit.rs similarity index 100% rename from rust/arrow/src/compute/kernels/limit.rs rename to arrow/src/compute/kernels/limit.rs diff --git a/rust/arrow/src/compute/kernels/mod.rs b/arrow/src/compute/kernels/mod.rs similarity index 100% rename from rust/arrow/src/compute/kernels/mod.rs rename to arrow/src/compute/kernels/mod.rs diff --git a/rust/arrow/src/compute/kernels/regexp.rs b/arrow/src/compute/kernels/regexp.rs similarity index 100% rename from rust/arrow/src/compute/kernels/regexp.rs rename to arrow/src/compute/kernels/regexp.rs diff --git a/rust/arrow/src/compute/kernels/sort.rs b/arrow/src/compute/kernels/sort.rs similarity index 100% rename from rust/arrow/src/compute/kernels/sort.rs rename to arrow/src/compute/kernels/sort.rs diff --git a/rust/arrow/src/compute/kernels/substring.rs b/arrow/src/compute/kernels/substring.rs similarity index 100% rename from rust/arrow/src/compute/kernels/substring.rs rename to arrow/src/compute/kernels/substring.rs diff --git a/rust/arrow/src/compute/kernels/take.rs b/arrow/src/compute/kernels/take.rs similarity index 100% rename from rust/arrow/src/compute/kernels/take.rs rename to arrow/src/compute/kernels/take.rs diff --git a/rust/arrow/src/compute/kernels/temporal.rs b/arrow/src/compute/kernels/temporal.rs similarity index 100% rename from rust/arrow/src/compute/kernels/temporal.rs rename to arrow/src/compute/kernels/temporal.rs diff --git a/rust/arrow/src/compute/kernels/window.rs b/arrow/src/compute/kernels/window.rs similarity index 100% rename from rust/arrow/src/compute/kernels/window.rs rename to arrow/src/compute/kernels/window.rs diff --git a/rust/arrow/src/compute/kernels/zip.rs b/arrow/src/compute/kernels/zip.rs similarity index 100% rename from rust/arrow/src/compute/kernels/zip.rs rename to arrow/src/compute/kernels/zip.rs diff --git a/rust/arrow/src/compute/mod.rs b/arrow/src/compute/mod.rs similarity index 100% rename from rust/arrow/src/compute/mod.rs rename to arrow/src/compute/mod.rs diff --git a/rust/arrow/src/compute/util.rs b/arrow/src/compute/util.rs similarity index 100% rename from rust/arrow/src/compute/util.rs rename to arrow/src/compute/util.rs diff --git a/rust/arrow/src/csv/mod.rs b/arrow/src/csv/mod.rs similarity index 100% rename from rust/arrow/src/csv/mod.rs rename to arrow/src/csv/mod.rs diff --git a/rust/arrow/src/csv/reader.rs b/arrow/src/csv/reader.rs similarity index 100% rename from rust/arrow/src/csv/reader.rs rename to arrow/src/csv/reader.rs diff --git a/rust/arrow/src/csv/writer.rs b/arrow/src/csv/writer.rs similarity index 100% rename from rust/arrow/src/csv/writer.rs rename to arrow/src/csv/writer.rs diff --git a/rust/arrow/src/datatypes/datatype.rs b/arrow/src/datatypes/datatype.rs similarity index 100% rename from rust/arrow/src/datatypes/datatype.rs rename to arrow/src/datatypes/datatype.rs diff --git a/rust/arrow/src/datatypes/field.rs b/arrow/src/datatypes/field.rs similarity index 100% rename from rust/arrow/src/datatypes/field.rs rename to arrow/src/datatypes/field.rs diff --git a/rust/arrow/src/datatypes/mod.rs b/arrow/src/datatypes/mod.rs similarity index 100% rename from rust/arrow/src/datatypes/mod.rs rename to arrow/src/datatypes/mod.rs diff --git a/rust/arrow/src/datatypes/native.rs b/arrow/src/datatypes/native.rs similarity index 100% rename from rust/arrow/src/datatypes/native.rs rename to arrow/src/datatypes/native.rs diff --git a/rust/arrow/src/datatypes/numeric.rs b/arrow/src/datatypes/numeric.rs similarity index 100% rename from rust/arrow/src/datatypes/numeric.rs rename to arrow/src/datatypes/numeric.rs diff --git a/rust/arrow/src/datatypes/schema.rs b/arrow/src/datatypes/schema.rs similarity index 100% rename from rust/arrow/src/datatypes/schema.rs rename to arrow/src/datatypes/schema.rs diff --git a/rust/arrow/src/datatypes/types.rs b/arrow/src/datatypes/types.rs similarity index 100% rename from rust/arrow/src/datatypes/types.rs rename to arrow/src/datatypes/types.rs diff --git a/rust/arrow/src/error.rs b/arrow/src/error.rs similarity index 100% rename from rust/arrow/src/error.rs rename to arrow/src/error.rs diff --git a/rust/arrow/src/ffi.rs b/arrow/src/ffi.rs similarity index 100% rename from rust/arrow/src/ffi.rs rename to arrow/src/ffi.rs diff --git a/rust/arrow/src/ipc/convert.rs b/arrow/src/ipc/convert.rs similarity index 100% rename from rust/arrow/src/ipc/convert.rs rename to arrow/src/ipc/convert.rs diff --git a/rust/arrow/src/ipc/gen/File.rs b/arrow/src/ipc/gen/File.rs similarity index 100% rename from rust/arrow/src/ipc/gen/File.rs rename to arrow/src/ipc/gen/File.rs diff --git a/rust/arrow/src/ipc/gen/Message.rs b/arrow/src/ipc/gen/Message.rs similarity index 100% rename from rust/arrow/src/ipc/gen/Message.rs rename to arrow/src/ipc/gen/Message.rs diff --git a/rust/arrow/src/ipc/gen/Schema.rs b/arrow/src/ipc/gen/Schema.rs similarity index 100% rename from rust/arrow/src/ipc/gen/Schema.rs rename to arrow/src/ipc/gen/Schema.rs diff --git a/rust/arrow/src/ipc/gen/SparseTensor.rs b/arrow/src/ipc/gen/SparseTensor.rs similarity index 100% rename from rust/arrow/src/ipc/gen/SparseTensor.rs rename to arrow/src/ipc/gen/SparseTensor.rs diff --git a/rust/arrow/src/ipc/gen/Tensor.rs b/arrow/src/ipc/gen/Tensor.rs similarity index 100% rename from rust/arrow/src/ipc/gen/Tensor.rs rename to arrow/src/ipc/gen/Tensor.rs diff --git a/rust/arrow/src/ipc/gen/mod.rs b/arrow/src/ipc/gen/mod.rs similarity index 100% rename from rust/arrow/src/ipc/gen/mod.rs rename to arrow/src/ipc/gen/mod.rs diff --git a/rust/arrow/src/ipc/mod.rs b/arrow/src/ipc/mod.rs similarity index 100% rename from rust/arrow/src/ipc/mod.rs rename to arrow/src/ipc/mod.rs diff --git a/rust/arrow/src/ipc/reader.rs b/arrow/src/ipc/reader.rs similarity index 100% rename from rust/arrow/src/ipc/reader.rs rename to arrow/src/ipc/reader.rs diff --git a/rust/arrow/src/ipc/writer.rs b/arrow/src/ipc/writer.rs similarity index 100% rename from rust/arrow/src/ipc/writer.rs rename to arrow/src/ipc/writer.rs diff --git a/rust/arrow/src/json/mod.rs b/arrow/src/json/mod.rs similarity index 100% rename from rust/arrow/src/json/mod.rs rename to arrow/src/json/mod.rs diff --git a/rust/arrow/src/json/reader.rs b/arrow/src/json/reader.rs similarity index 100% rename from rust/arrow/src/json/reader.rs rename to arrow/src/json/reader.rs diff --git a/rust/arrow/src/json/writer.rs b/arrow/src/json/writer.rs similarity index 100% rename from rust/arrow/src/json/writer.rs rename to arrow/src/json/writer.rs diff --git a/rust/arrow/src/lib.rs b/arrow/src/lib.rs similarity index 100% rename from rust/arrow/src/lib.rs rename to arrow/src/lib.rs diff --git a/rust/arrow/src/record_batch.rs b/arrow/src/record_batch.rs similarity index 100% rename from rust/arrow/src/record_batch.rs rename to arrow/src/record_batch.rs diff --git a/rust/arrow/src/temporal_conversions.rs b/arrow/src/temporal_conversions.rs similarity index 100% rename from rust/arrow/src/temporal_conversions.rs rename to arrow/src/temporal_conversions.rs diff --git a/rust/arrow/src/tensor.rs b/arrow/src/tensor.rs similarity index 100% rename from rust/arrow/src/tensor.rs rename to arrow/src/tensor.rs diff --git a/rust/arrow/src/util/bench_util.rs b/arrow/src/util/bench_util.rs similarity index 100% rename from rust/arrow/src/util/bench_util.rs rename to arrow/src/util/bench_util.rs diff --git a/rust/arrow/src/util/bit_chunk_iterator.rs b/arrow/src/util/bit_chunk_iterator.rs similarity index 100% rename from rust/arrow/src/util/bit_chunk_iterator.rs rename to arrow/src/util/bit_chunk_iterator.rs diff --git a/rust/arrow/src/util/bit_util.rs b/arrow/src/util/bit_util.rs similarity index 100% rename from rust/arrow/src/util/bit_util.rs rename to arrow/src/util/bit_util.rs diff --git a/rust/arrow/src/util/data_gen.rs b/arrow/src/util/data_gen.rs similarity index 100% rename from rust/arrow/src/util/data_gen.rs rename to arrow/src/util/data_gen.rs diff --git a/rust/arrow/src/util/display.rs b/arrow/src/util/display.rs similarity index 100% rename from rust/arrow/src/util/display.rs rename to arrow/src/util/display.rs diff --git a/rust/arrow/src/util/integration_util.rs b/arrow/src/util/integration_util.rs similarity index 100% rename from rust/arrow/src/util/integration_util.rs rename to arrow/src/util/integration_util.rs diff --git a/rust/arrow/src/util/mod.rs b/arrow/src/util/mod.rs similarity index 100% rename from rust/arrow/src/util/mod.rs rename to arrow/src/util/mod.rs diff --git a/rust/arrow/src/util/pretty.rs b/arrow/src/util/pretty.rs similarity index 100% rename from rust/arrow/src/util/pretty.rs rename to arrow/src/util/pretty.rs diff --git a/rust/arrow/src/util/serialization.rs b/arrow/src/util/serialization.rs similarity index 100% rename from rust/arrow/src/util/serialization.rs rename to arrow/src/util/serialization.rs diff --git a/rust/arrow/src/util/string_writer.rs b/arrow/src/util/string_writer.rs similarity index 100% rename from rust/arrow/src/util/string_writer.rs rename to arrow/src/util/string_writer.rs diff --git a/rust/arrow/src/util/test_util.rs b/arrow/src/util/test_util.rs similarity index 100% rename from rust/arrow/src/util/test_util.rs rename to arrow/src/util/test_util.rs diff --git a/rust/arrow/src/util/trusted_len.rs b/arrow/src/util/trusted_len.rs similarity index 100% rename from rust/arrow/src/util/trusted_len.rs rename to arrow/src/util/trusted_len.rs diff --git a/rust/arrow/src/zz_memory_check.rs b/arrow/src/zz_memory_check.rs similarity index 100% rename from rust/arrow/src/zz_memory_check.rs rename to arrow/src/zz_memory_check.rs diff --git a/rust/arrow/test/data/arrays.json b/arrow/test/data/arrays.json similarity index 100% rename from rust/arrow/test/data/arrays.json rename to arrow/test/data/arrays.json diff --git a/rust/arrow/test/data/basic.json b/arrow/test/data/basic.json similarity index 100% rename from rust/arrow/test/data/basic.json rename to arrow/test/data/basic.json diff --git a/rust/arrow/test/data/basic_nulls.json b/arrow/test/data/basic_nulls.json similarity index 100% rename from rust/arrow/test/data/basic_nulls.json rename to arrow/test/data/basic_nulls.json diff --git a/rust/arrow/test/data/integration.json b/arrow/test/data/integration.json similarity index 100% rename from rust/arrow/test/data/integration.json rename to arrow/test/data/integration.json diff --git a/rust/arrow/test/data/list_string_dict_nested.json b/arrow/test/data/list_string_dict_nested.json similarity index 100% rename from rust/arrow/test/data/list_string_dict_nested.json rename to arrow/test/data/list_string_dict_nested.json diff --git a/rust/arrow/test/data/list_string_dict_nested_nulls.json b/arrow/test/data/list_string_dict_nested_nulls.json similarity index 100% rename from rust/arrow/test/data/list_string_dict_nested_nulls.json rename to arrow/test/data/list_string_dict_nested_nulls.json diff --git a/rust/arrow/test/data/mixed_arrays.json b/arrow/test/data/mixed_arrays.json similarity index 100% rename from rust/arrow/test/data/mixed_arrays.json rename to arrow/test/data/mixed_arrays.json diff --git a/rust/arrow/test/data/mixed_arrays.json.gz b/arrow/test/data/mixed_arrays.json.gz similarity index 100% rename from rust/arrow/test/data/mixed_arrays.json.gz rename to arrow/test/data/mixed_arrays.json.gz diff --git a/rust/arrow/test/data/nested_structs.json b/arrow/test/data/nested_structs.json similarity index 100% rename from rust/arrow/test/data/nested_structs.json rename to arrow/test/data/nested_structs.json diff --git a/rust/arrow/test/data/null_test.csv b/arrow/test/data/null_test.csv similarity index 100% rename from rust/arrow/test/data/null_test.csv rename to arrow/test/data/null_test.csv diff --git a/rust/arrow/test/data/uk_cities.csv b/arrow/test/data/uk_cities.csv similarity index 100% rename from rust/arrow/test/data/uk_cities.csv rename to arrow/test/data/uk_cities.csv diff --git a/rust/arrow/test/data/uk_cities_with_headers.csv b/arrow/test/data/uk_cities_with_headers.csv similarity index 100% rename from rust/arrow/test/data/uk_cities_with_headers.csv rename to arrow/test/data/uk_cities_with_headers.csv diff --git a/rust/arrow/test/data/various_types.csv b/arrow/test/data/various_types.csv similarity index 100% rename from rust/arrow/test/data/various_types.csv rename to arrow/test/data/various_types.csv diff --git a/rust/arrow/test/data/various_types_invalid.csv b/arrow/test/data/various_types_invalid.csv similarity index 100% rename from rust/arrow/test/data/various_types_invalid.csv rename to arrow/test/data/various_types_invalid.csv diff --git a/rust/integration-testing/Cargo.toml b/integration-testing/Cargo.toml similarity index 100% rename from rust/integration-testing/Cargo.toml rename to integration-testing/Cargo.toml diff --git a/rust/integration-testing/README.md b/integration-testing/README.md similarity index 100% rename from rust/integration-testing/README.md rename to integration-testing/README.md diff --git a/rust/integration-testing/src/bin/arrow-file-to-stream.rs b/integration-testing/src/bin/arrow-file-to-stream.rs similarity index 100% rename from rust/integration-testing/src/bin/arrow-file-to-stream.rs rename to integration-testing/src/bin/arrow-file-to-stream.rs diff --git a/rust/integration-testing/src/bin/arrow-json-integration-test.rs b/integration-testing/src/bin/arrow-json-integration-test.rs similarity index 100% rename from rust/integration-testing/src/bin/arrow-json-integration-test.rs rename to integration-testing/src/bin/arrow-json-integration-test.rs diff --git a/rust/integration-testing/src/bin/arrow-stream-to-file.rs b/integration-testing/src/bin/arrow-stream-to-file.rs similarity index 100% rename from rust/integration-testing/src/bin/arrow-stream-to-file.rs rename to integration-testing/src/bin/arrow-stream-to-file.rs diff --git a/rust/integration-testing/src/bin/flight-test-integration-client.rs b/integration-testing/src/bin/flight-test-integration-client.rs similarity index 100% rename from rust/integration-testing/src/bin/flight-test-integration-client.rs rename to integration-testing/src/bin/flight-test-integration-client.rs diff --git a/rust/integration-testing/src/bin/flight-test-integration-server.rs b/integration-testing/src/bin/flight-test-integration-server.rs similarity index 100% rename from rust/integration-testing/src/bin/flight-test-integration-server.rs rename to integration-testing/src/bin/flight-test-integration-server.rs diff --git a/rust/integration-testing/src/flight_client_scenarios.rs b/integration-testing/src/flight_client_scenarios.rs similarity index 100% rename from rust/integration-testing/src/flight_client_scenarios.rs rename to integration-testing/src/flight_client_scenarios.rs diff --git a/rust/integration-testing/src/flight_client_scenarios/auth_basic_proto.rs b/integration-testing/src/flight_client_scenarios/auth_basic_proto.rs similarity index 100% rename from rust/integration-testing/src/flight_client_scenarios/auth_basic_proto.rs rename to integration-testing/src/flight_client_scenarios/auth_basic_proto.rs diff --git a/rust/integration-testing/src/flight_client_scenarios/integration_test.rs b/integration-testing/src/flight_client_scenarios/integration_test.rs similarity index 100% rename from rust/integration-testing/src/flight_client_scenarios/integration_test.rs rename to integration-testing/src/flight_client_scenarios/integration_test.rs diff --git a/rust/integration-testing/src/flight_client_scenarios/middleware.rs b/integration-testing/src/flight_client_scenarios/middleware.rs similarity index 100% rename from rust/integration-testing/src/flight_client_scenarios/middleware.rs rename to integration-testing/src/flight_client_scenarios/middleware.rs diff --git a/rust/integration-testing/src/flight_server_scenarios.rs b/integration-testing/src/flight_server_scenarios.rs similarity index 100% rename from rust/integration-testing/src/flight_server_scenarios.rs rename to integration-testing/src/flight_server_scenarios.rs diff --git a/rust/integration-testing/src/flight_server_scenarios/auth_basic_proto.rs b/integration-testing/src/flight_server_scenarios/auth_basic_proto.rs similarity index 100% rename from rust/integration-testing/src/flight_server_scenarios/auth_basic_proto.rs rename to integration-testing/src/flight_server_scenarios/auth_basic_proto.rs diff --git a/rust/integration-testing/src/flight_server_scenarios/integration_test.rs b/integration-testing/src/flight_server_scenarios/integration_test.rs similarity index 100% rename from rust/integration-testing/src/flight_server_scenarios/integration_test.rs rename to integration-testing/src/flight_server_scenarios/integration_test.rs diff --git a/rust/integration-testing/src/flight_server_scenarios/middleware.rs b/integration-testing/src/flight_server_scenarios/middleware.rs similarity index 100% rename from rust/integration-testing/src/flight_server_scenarios/middleware.rs rename to integration-testing/src/flight_server_scenarios/middleware.rs diff --git a/rust/integration-testing/src/lib.rs b/integration-testing/src/lib.rs similarity index 100% rename from rust/integration-testing/src/lib.rs rename to integration-testing/src/lib.rs diff --git a/rust/parquet/Cargo.toml b/parquet/Cargo.toml similarity index 100% rename from rust/parquet/Cargo.toml rename to parquet/Cargo.toml diff --git a/rust/parquet/README.md b/parquet/README.md similarity index 100% rename from rust/parquet/README.md rename to parquet/README.md diff --git a/rust/parquet/benches/arrow_writer.rs b/parquet/benches/arrow_writer.rs similarity index 100% rename from rust/parquet/benches/arrow_writer.rs rename to parquet/benches/arrow_writer.rs diff --git a/rust/parquet/build.rs b/parquet/build.rs similarity index 100% rename from rust/parquet/build.rs rename to parquet/build.rs diff --git a/rust/parquet/src/arrow/array_reader.rs b/parquet/src/arrow/array_reader.rs similarity index 100% rename from rust/parquet/src/arrow/array_reader.rs rename to parquet/src/arrow/array_reader.rs diff --git a/rust/parquet/src/arrow/arrow_reader.rs b/parquet/src/arrow/arrow_reader.rs similarity index 100% rename from rust/parquet/src/arrow/arrow_reader.rs rename to parquet/src/arrow/arrow_reader.rs diff --git a/rust/parquet/src/arrow/arrow_writer.rs b/parquet/src/arrow/arrow_writer.rs similarity index 100% rename from rust/parquet/src/arrow/arrow_writer.rs rename to parquet/src/arrow/arrow_writer.rs diff --git a/rust/parquet/src/arrow/converter.rs b/parquet/src/arrow/converter.rs similarity index 100% rename from rust/parquet/src/arrow/converter.rs rename to parquet/src/arrow/converter.rs diff --git a/rust/parquet/src/arrow/levels.rs b/parquet/src/arrow/levels.rs similarity index 100% rename from rust/parquet/src/arrow/levels.rs rename to parquet/src/arrow/levels.rs diff --git a/rust/parquet/src/arrow/mod.rs b/parquet/src/arrow/mod.rs similarity index 100% rename from rust/parquet/src/arrow/mod.rs rename to parquet/src/arrow/mod.rs diff --git a/rust/parquet/src/arrow/record_reader.rs b/parquet/src/arrow/record_reader.rs similarity index 100% rename from rust/parquet/src/arrow/record_reader.rs rename to parquet/src/arrow/record_reader.rs diff --git a/rust/parquet/src/arrow/schema.rs b/parquet/src/arrow/schema.rs similarity index 100% rename from rust/parquet/src/arrow/schema.rs rename to parquet/src/arrow/schema.rs diff --git a/rust/parquet/src/basic.rs b/parquet/src/basic.rs similarity index 100% rename from rust/parquet/src/basic.rs rename to parquet/src/basic.rs diff --git a/rust/parquet/src/bin/parquet-read.rs b/parquet/src/bin/parquet-read.rs similarity index 100% rename from rust/parquet/src/bin/parquet-read.rs rename to parquet/src/bin/parquet-read.rs diff --git a/rust/parquet/src/bin/parquet-rowcount.rs b/parquet/src/bin/parquet-rowcount.rs similarity index 100% rename from rust/parquet/src/bin/parquet-rowcount.rs rename to parquet/src/bin/parquet-rowcount.rs diff --git a/rust/parquet/src/bin/parquet-schema.rs b/parquet/src/bin/parquet-schema.rs similarity index 100% rename from rust/parquet/src/bin/parquet-schema.rs rename to parquet/src/bin/parquet-schema.rs diff --git a/rust/parquet/src/column/mod.rs b/parquet/src/column/mod.rs similarity index 100% rename from rust/parquet/src/column/mod.rs rename to parquet/src/column/mod.rs diff --git a/rust/parquet/src/column/page.rs b/parquet/src/column/page.rs similarity index 100% rename from rust/parquet/src/column/page.rs rename to parquet/src/column/page.rs diff --git a/rust/parquet/src/column/reader.rs b/parquet/src/column/reader.rs similarity index 100% rename from rust/parquet/src/column/reader.rs rename to parquet/src/column/reader.rs diff --git a/rust/parquet/src/column/writer.rs b/parquet/src/column/writer.rs similarity index 100% rename from rust/parquet/src/column/writer.rs rename to parquet/src/column/writer.rs diff --git a/rust/parquet/src/compression.rs b/parquet/src/compression.rs similarity index 100% rename from rust/parquet/src/compression.rs rename to parquet/src/compression.rs diff --git a/rust/parquet/src/data_type.rs b/parquet/src/data_type.rs similarity index 100% rename from rust/parquet/src/data_type.rs rename to parquet/src/data_type.rs diff --git a/rust/parquet/src/encodings/decoding.rs b/parquet/src/encodings/decoding.rs similarity index 100% rename from rust/parquet/src/encodings/decoding.rs rename to parquet/src/encodings/decoding.rs diff --git a/rust/parquet/src/encodings/encoding.rs b/parquet/src/encodings/encoding.rs similarity index 100% rename from rust/parquet/src/encodings/encoding.rs rename to parquet/src/encodings/encoding.rs diff --git a/rust/parquet/src/encodings/levels.rs b/parquet/src/encodings/levels.rs similarity index 100% rename from rust/parquet/src/encodings/levels.rs rename to parquet/src/encodings/levels.rs diff --git a/rust/parquet/src/encodings/mod.rs b/parquet/src/encodings/mod.rs similarity index 100% rename from rust/parquet/src/encodings/mod.rs rename to parquet/src/encodings/mod.rs diff --git a/rust/parquet/src/encodings/rle.rs b/parquet/src/encodings/rle.rs similarity index 100% rename from rust/parquet/src/encodings/rle.rs rename to parquet/src/encodings/rle.rs diff --git a/rust/parquet/src/errors.rs b/parquet/src/errors.rs similarity index 100% rename from rust/parquet/src/errors.rs rename to parquet/src/errors.rs diff --git a/rust/parquet/src/file/footer.rs b/parquet/src/file/footer.rs similarity index 100% rename from rust/parquet/src/file/footer.rs rename to parquet/src/file/footer.rs diff --git a/rust/parquet/src/file/metadata.rs b/parquet/src/file/metadata.rs similarity index 100% rename from rust/parquet/src/file/metadata.rs rename to parquet/src/file/metadata.rs diff --git a/rust/parquet/src/file/mod.rs b/parquet/src/file/mod.rs similarity index 100% rename from rust/parquet/src/file/mod.rs rename to parquet/src/file/mod.rs diff --git a/rust/parquet/src/file/properties.rs b/parquet/src/file/properties.rs similarity index 100% rename from rust/parquet/src/file/properties.rs rename to parquet/src/file/properties.rs diff --git a/rust/parquet/src/file/reader.rs b/parquet/src/file/reader.rs similarity index 100% rename from rust/parquet/src/file/reader.rs rename to parquet/src/file/reader.rs diff --git a/rust/parquet/src/file/serialized_reader.rs b/parquet/src/file/serialized_reader.rs similarity index 100% rename from rust/parquet/src/file/serialized_reader.rs rename to parquet/src/file/serialized_reader.rs diff --git a/rust/parquet/src/file/statistics.rs b/parquet/src/file/statistics.rs similarity index 100% rename from rust/parquet/src/file/statistics.rs rename to parquet/src/file/statistics.rs diff --git a/rust/parquet/src/file/writer.rs b/parquet/src/file/writer.rs similarity index 100% rename from rust/parquet/src/file/writer.rs rename to parquet/src/file/writer.rs diff --git a/rust/parquet/src/lib.rs b/parquet/src/lib.rs similarity index 100% rename from rust/parquet/src/lib.rs rename to parquet/src/lib.rs diff --git a/rust/parquet/src/record/api.rs b/parquet/src/record/api.rs similarity index 100% rename from rust/parquet/src/record/api.rs rename to parquet/src/record/api.rs diff --git a/rust/parquet/src/record/mod.rs b/parquet/src/record/mod.rs similarity index 100% rename from rust/parquet/src/record/mod.rs rename to parquet/src/record/mod.rs diff --git a/rust/parquet/src/record/reader.rs b/parquet/src/record/reader.rs similarity index 100% rename from rust/parquet/src/record/reader.rs rename to parquet/src/record/reader.rs diff --git a/rust/parquet/src/record/record_writer.rs b/parquet/src/record/record_writer.rs similarity index 100% rename from rust/parquet/src/record/record_writer.rs rename to parquet/src/record/record_writer.rs diff --git a/rust/parquet/src/record/triplet.rs b/parquet/src/record/triplet.rs similarity index 100% rename from rust/parquet/src/record/triplet.rs rename to parquet/src/record/triplet.rs diff --git a/rust/parquet/src/schema/mod.rs b/parquet/src/schema/mod.rs similarity index 100% rename from rust/parquet/src/schema/mod.rs rename to parquet/src/schema/mod.rs diff --git a/rust/parquet/src/schema/parser.rs b/parquet/src/schema/parser.rs similarity index 100% rename from rust/parquet/src/schema/parser.rs rename to parquet/src/schema/parser.rs diff --git a/rust/parquet/src/schema/printer.rs b/parquet/src/schema/printer.rs similarity index 100% rename from rust/parquet/src/schema/printer.rs rename to parquet/src/schema/printer.rs diff --git a/rust/parquet/src/schema/types.rs b/parquet/src/schema/types.rs similarity index 100% rename from rust/parquet/src/schema/types.rs rename to parquet/src/schema/types.rs diff --git a/rust/parquet/src/schema/visitor.rs b/parquet/src/schema/visitor.rs similarity index 100% rename from rust/parquet/src/schema/visitor.rs rename to parquet/src/schema/visitor.rs diff --git a/rust/parquet/src/util/bit_packing.rs b/parquet/src/util/bit_packing.rs similarity index 100% rename from rust/parquet/src/util/bit_packing.rs rename to parquet/src/util/bit_packing.rs diff --git a/rust/parquet/src/util/bit_util.rs b/parquet/src/util/bit_util.rs similarity index 100% rename from rust/parquet/src/util/bit_util.rs rename to parquet/src/util/bit_util.rs diff --git a/rust/parquet/src/util/cursor.rs b/parquet/src/util/cursor.rs similarity index 100% rename from rust/parquet/src/util/cursor.rs rename to parquet/src/util/cursor.rs diff --git a/rust/parquet/src/util/hash_util.rs b/parquet/src/util/hash_util.rs similarity index 100% rename from rust/parquet/src/util/hash_util.rs rename to parquet/src/util/hash_util.rs diff --git a/rust/parquet/src/util/io.rs b/parquet/src/util/io.rs similarity index 100% rename from rust/parquet/src/util/io.rs rename to parquet/src/util/io.rs diff --git a/rust/parquet/src/util/memory.rs b/parquet/src/util/memory.rs similarity index 100% rename from rust/parquet/src/util/memory.rs rename to parquet/src/util/memory.rs diff --git a/rust/parquet/src/util/mod.rs b/parquet/src/util/mod.rs similarity index 100% rename from rust/parquet/src/util/mod.rs rename to parquet/src/util/mod.rs diff --git a/rust/parquet/src/util/test_common/file_util.rs b/parquet/src/util/test_common/file_util.rs similarity index 100% rename from rust/parquet/src/util/test_common/file_util.rs rename to parquet/src/util/test_common/file_util.rs diff --git a/rust/parquet/src/util/test_common/mod.rs b/parquet/src/util/test_common/mod.rs similarity index 100% rename from rust/parquet/src/util/test_common/mod.rs rename to parquet/src/util/test_common/mod.rs diff --git a/rust/parquet/src/util/test_common/page_util.rs b/parquet/src/util/test_common/page_util.rs similarity index 100% rename from rust/parquet/src/util/test_common/page_util.rs rename to parquet/src/util/test_common/page_util.rs diff --git a/rust/parquet/src/util/test_common/rand_gen.rs b/parquet/src/util/test_common/rand_gen.rs similarity index 100% rename from rust/parquet/src/util/test_common/rand_gen.rs rename to parquet/src/util/test_common/rand_gen.rs diff --git a/rust/parquet/tests/custom_writer.rs b/parquet/tests/custom_writer.rs similarity index 100% rename from rust/parquet/tests/custom_writer.rs rename to parquet/tests/custom_writer.rs diff --git a/rust/parquet_derive/Cargo.toml b/parquet_derive/Cargo.toml similarity index 100% rename from rust/parquet_derive/Cargo.toml rename to parquet_derive/Cargo.toml diff --git a/rust/parquet_derive/README.md b/parquet_derive/README.md similarity index 100% rename from rust/parquet_derive/README.md rename to parquet_derive/README.md diff --git a/rust/parquet_derive/src/lib.rs b/parquet_derive/src/lib.rs similarity index 100% rename from rust/parquet_derive/src/lib.rs rename to parquet_derive/src/lib.rs diff --git a/rust/parquet_derive/src/parquet_field.rs b/parquet_derive/src/parquet_field.rs similarity index 100% rename from rust/parquet_derive/src/parquet_field.rs rename to parquet_derive/src/parquet_field.rs diff --git a/rust/parquet_derive_test/Cargo.toml b/parquet_derive_test/Cargo.toml similarity index 100% rename from rust/parquet_derive_test/Cargo.toml rename to parquet_derive_test/Cargo.toml diff --git a/rust/parquet_derive_test/src/lib.rs b/parquet_derive_test/src/lib.rs similarity index 100% rename from rust/parquet_derive_test/src/lib.rs rename to parquet_derive_test/src/lib.rs diff --git a/rust/pre-commit.sh b/pre-commit.sh similarity index 100% rename from rust/pre-commit.sh rename to pre-commit.sh diff --git a/rust/.gitignore b/rust/.gitignore deleted file mode 100644 index 389f4ab..0000000 --- a/rust/.gitignore +++ /dev/null @@ -1,5 +0,0 @@ -Cargo.lock -target -rusty-tags.vi -.history -.flatbuffers/ diff --git a/rust/README.md b/rust/README.md deleted file mode 100644 index 7fdef29..0000000 --- a/rust/README.md +++ /dev/null @@ -1,186 +0,0 @@ -<!--- - Licensed to the Apache Software Foundation (ASF) under one - or more contributor license agreements. See the NOTICE file - distributed with this work for additional information - regarding copyright ownership. The ASF licenses this file - to you under the Apache License, Version 2.0 (the - "License"); you may not use this file except in compliance - with the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - KIND, either express or implied. See the License for the - specific language governing permissions and limitations - under the License. ---> - -# Native Rust implementation of Apache Arrow - -[](https://codecov.io/gh/apache/arrow?branch=master) - -Welcome to the implementation of Arrow, the popular in-memory columnar format, in [Rust](https://www.rust-lang.org/). - -This part of the Arrow project is divided in 4 main components: - -| Crate | Description | Documentation | -|-----------|-------------|---------------| -|Arrow | Core functionality (memory layout, arrays, low level computations) | [(README)](arrow/README.md) | -|Parquet | Parquet support | [(README)](parquet/README.md) | -|Arrow-flight | Arrow data between processes | [(README)](arrow-flight/README.md) | -|DataFusion | In-memory query engine with SQL support | [(README)](datafusion/README.md) | -|Ballista | Distributed query execution | [(README)](ballista/README.md) | - -Independently, they support a vast array of functionality for in-memory computations. - -Together, they allow users to write an SQL query or a `DataFrame` (using the `datafusion` crate), run it against a parquet file (using the `parquet` crate), evaluate it in-memory using Arrow's columnar format (using the `arrow` crate), and send to another process (using the `arrow-flight` crate). - -Generally speaking, the `arrow` crate offers functionality to develop code that uses Arrow arrays, and `datafusion` offers most operations typically found in SQL, with the notable exceptions of: - -* `join` -* `window` functions - -There are too many features to enumerate here, but some notable mentions: - -* `Arrow` implements all formats in the specification except certain dictionaries -* `Arrow` supports SIMD operations to some of its vertical operations -* `DataFusion` supports `async` execution -* `DataFusion` supports user-defined functions, aggregates, and whole execution nodes - -You can find more details about each crate in their respective READMEs. - -## Arrow Rust Community - -We use the official [ASF Slack](https://s.apache.org/slack-invite) for informal discussions and coordination. This is -a great place to meet other contributors and get guidance on where to contribute. Join us in the `arrow-rust` channel. - -We use [ASF JIRA](https://issues.apache.org/jira/secure/Dashboard.jspa) as the system of record for new features -and bug fixes and this plays a critical role in the release process. - -For design discussions we generally collaborate on Google documents and file a JIRA linking to the document. - -There is also a bi-weekly Rust-specific sync call for the Arrow Rust community. This is hosted on Google Meet -at https://meet.google.com/ctp-yujs-aee on alternate Wednesday's at 09:00 US/Pacific, 12:00 US/Eastern. During -US daylight savings time this corresponds to 16:00 UTC and at other times this is 17:00 UTC. - -## Developer's guide to Arrow Rust - -### How to compile - -This is a standard cargo project with workspaces. To build it, you need to have `rust` and `cargo`: - -```bash -cd /rust && cargo build -``` - -You can also use rust's official docker image: - -```bash -docker run --rm -v $(pwd)/rust:/rust -it rust /bin/bash -c "cd /rust && cargo build" -``` - -The command above assumes that are in the root directory of the project, not in the same -directory as this README.md. - -You can also compile specific workspaces: - -```bash -cd /rust/arrow && cargo build -``` - -### Git Submodules - -Before running tests and examples, it is necessary to set up the local development environment. - -The tests rely on test data that is contained in git submodules. - -To pull down this data run the following: - -```bash -git submodule update --init -``` - -This populates data in two git submodules: - -- `../cpp/submodules/parquet_testing/data` (sourced from https://github.com/apache/parquet-testing.git) -- `../testing` (sourced from https://github.com/apache/arrow-testing) - -By default, `cargo test` will look for these directories at their -standard location. The following environment variables can be used to override the location: - -```bash -# Optionaly specify a different location for test data -export PARQUET_TEST_DATA=$(cd ../cpp/submodules/parquet-testing/data; pwd) -export ARROW_TEST_DATA=$(cd ../testing/data; pwd) -``` - -From here on, this is a pure Rust project and `cargo` can be used to run tests, benchmarks, docs and examples as usual. - - -### Running the tests - -Run tests using the Rust standard `cargo test` command: - -```bash -# run all tests. -cargo test - - -# run only tests for the arrow crate -cargo test -p arrow -``` - -## Code Formatting - -Our CI uses `rustfmt` to check code formatting. Before submitting a -PR be sure to run the following and check for lint issues: - -```bash -cargo +stable fmt --all -- --check -``` - -## Clippy Lints - -We recommend using `clippy` for checking lints during development. While we do not yet enforce `clippy` checks, we recommend not introducing new `clippy` errors or warnings. - -Run the following to check for clippy lints. - -``` -cargo clippy -``` - -If you use Visual Studio Code with the `rust-analyzer` plugin, you can enable `clippy` to run each time you save a file. See https://users.rust-lang.org/t/how-to-use-clippy-in-vs-code-with-rust-analyzer/41881. - -One of the concerns with `clippy` is that it often produces a lot of false positives, or that some recommendations may hurt readability. We do not have a policy of which lints are ignored, but if you disagree with a `clippy` lint, you may disable the lint and briefly justify it. - -Search for `allow(clippy::` in the codebase to identify lints that are ignored/allowed. We currently prefer ignoring lints on the lowest unit possible. -* If you are introducing a line that returns a lint warning or error, you may disable the lint on that line. -* If you have several lints on a function or module, you may disable the lint on the function or module. -* If a lint is pervasive across multiple modules, you may disable it at the crate level. - -## Git Pre-Commit Hook - -We can use [git pre-commit hook](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks) to automate various kinds of git pre-commit checking/formatting. - -Suppose you are in the root directory of the project. - -First check if the file already exists: - -```bash -ls -l .git/hooks/pre-commit -``` - -If the file already exists, to avoid mistakenly **overriding**, you MAY have to check -the link source or file content. Else if not exist, let's safely soft link [pre-commit.sh](pre-commit.sh) as file `.git/hooks/pre-commit`: - -``` -ln -s ../../rust/pre-commit.sh .git/hooks/pre-commit -``` - -If sometimes you want to commit without checking, just run `git commit` with `--no-verify`: - -```bash -git commit --no-verify -m "... commit message ..." -``` diff --git a/rust/rustfmt.toml b/rustfmt.toml similarity index 100% rename from rust/rustfmt.toml rename to rustfmt.toml
