[
https://issues.apache.org/jira/browse/ARROW-9600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Grove reopened ARROW-9600:
-------------------------------
Assignee: Andy Grove (was: Andrew Lamb)
> [Rust] When used as a crate dependency, arrow-flight is rebuilt on every
> invocation of cargo build
> --------------------------------------------------------------------------------------------------
>
> Key: ARROW-9600
> URL: https://issues.apache.org/jira/browse/ARROW-9600
> Project: Apache Arrow
> Issue Type: Bug
> Components: Rust
> Affects Versions: 1.0.0
> Reporter: Andrew Lamb
> Assignee: Andy Grove
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.0.0, 1.0.1
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> When used as a crate dependency, arrow-flight is rebuilt on every invocation
> of cargo build
> h1. *Repro*:
> Create a new repo, add `arrow=1.0.0` as a dependency, and then run `cargo
> build`
> *Expected behavior:* After the first successful invocation of `cargo build`,
> arrow-flight will not recompile if no other changes are made.
> *Actual behavior*: After every invocation of `cargo build`, arrow-flight is
> recompiled, even when nothing has changed
> h1. Example
>
> Create a new crate
> {code:java}
> alamb@ip-192-168-0-129 arrow_rebuilds % cargo new too_many_rebuilds --bin
> cargo new too_many_rebuilds --bin
> Created binary (application) `too_many_rebuilds` package
> {code}
> Add arrow as a dependency in Cargo.toml:
> {code:java}
> diff --git a/Cargo.toml b/Cargo.toml
> index a239680..44ed358 100644
> — a/Cargo.toml
> +++ b/Cargo.toml
> @@ -5,3 +5,6 @@ authors = ["alamb <[email protected]>"]
> edition = "2018"
> # See more keys and their definitions at
> [https://doc.rust-lang.org/cargo/reference/manifest.html]
> +
> +[dependencies]
> +arrow = "1.0.0"
> {code}
> Now, all invocations of `cargo build` will rebuild arrow, even though nothing
> in the code has changed:
> {code:java}
> alamb@ip-192-168-0-129 too_many_rebuilds % cargo build
> cargo build
> Compiling arrow-flight v1.0.0
> Compiling arrow v1.0.0
> Compiling too_many_rebuilds v0.1.0
> (/Users/alamb/Software/bugs/arrow_rebuilds/too_many_rebuilds)
> Finished dev [unoptimized + debuginfo] target(s) in 8.70s
> alamb@ip-192-168-0-129 too_many_rebuilds % cargo build
> cargo build
> Compiling arrow-flight v1.0.0
> Compiling arrow v1.0.0
> Compiling too_many_rebuilds v0.1.0
> (/Users/alamb/Software/bugs/arrow_rebuilds/too_many_rebuilds)
> Finished dev [unoptimized + debuginfo] target(s) in 8.65s
> {code}
> You can see what is happening by checking out a fresh copy of arrow/master
> (no Cargo.log) and running `cargo build` – you'll see your local checkout has
> changes in rust/arrow-flight/src/arrow.flight.protocol.rs:
> {code:java}
> alamb@ip-192-168-0-129 arrow % cd rust/arrow
> cd rust/arrow
> alamb@ip-192-168-0-129 arrow % git status
> git status
> On branch master
> Your branch is up to date with 'origin/master'.
> nothing to commit, working tree clean
> alamb@ip-192-168-0-129 arrow % cargo build
> cargo build
> Compiling futures-task v0.3.5
> ...
> Compiling arrow v2.0.0-SNAPSHOT (/Users/alamb/Software/arrow/rust/arrow)
> Finished dev [unoptimized + debuginfo] target(s) in 21.76s
> alamb@ip-192-168-0-129 arrow %
> alamb@ip-192-168-0-129 arrow % git status
> git status
> On branch master
> Your branch is up to date with 'origin/master'.
> Changes not staged for commit:
> (use "git add <file>..." to update what will be committed)
> (use "git restore <file>..." to discard changes in working directory)
> modified: ../arrow-flight/src/arrow.flight.protocol.rs
> no changes added to commit (use "git add" and/or "git commit -a")
> {code}
> # Root Cause Analysis
> The issue is that the build.rs of arrow-flight calls `tonic_build` to auto
> generate `rust/arrow-flight/src/arrow.flight.protocol.rs`, which is also
> checked in (first done in
> [https://github.com/apache/arrow/commit/ec84b7b8102f227295f865c420496830c66a6281]).
> This file and the version of tonic were updated on
> [https://github.com/apache/arrow/commit/7b49cbc23f22ed99eebf85cc0b9acb1f0d3f832f]
> on July 11, 2020
> It turns out that the output of "tonic_build" depends on not only on the
> version of tonic, but also on the version of proc-macro2, and the version of
> proc-macro2 is not specifically pinned.
> `proc-macro2 = "1.0.19"` was released on July 19, 2020
> ([https://crates.io/crates/proc-macro2/1.0.19]) and it appears to subtlety
> changes the resulting output of arrow.flight.protocol.rs; Thus the output no
> longer matches what is checked in. This means that anyone without a
> Cargo.lock file that pins proc-macro2 to 1.0.18 would get 1.0.19 and thus
> also a local modification during build.
> h1. Workaround
> If we pin Cargo.toml to use proc-macro2 1.0.18 the local modification stops.
> {code}
> proc-macro2 = "1.0.18"
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
