[ https://issues.apache.org/jira/browse/ARROW-9600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andy Grove reopened ARROW-9600: ------------------------------- Assignee: Andy Grove (was: Andrew Lamb) > [Rust] When used as a crate dependency, arrow-flight is rebuilt on every > invocation of cargo build > -------------------------------------------------------------------------------------------------- > > Key: ARROW-9600 > URL: https://issues.apache.org/jira/browse/ARROW-9600 > Project: Apache Arrow > Issue Type: Bug > Components: Rust > Affects Versions: 1.0.0 > Reporter: Andrew Lamb > Assignee: Andy Grove > Priority: Major > Labels: pull-request-available > Fix For: 2.0.0, 1.0.1 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > When used as a crate dependency, arrow-flight is rebuilt on every invocation > of cargo build > h1. *Repro*: > Create a new repo, add `arrow=1.0.0` as a dependency, and then run `cargo > build` > *Expected behavior:* After the first successful invocation of `cargo build`, > arrow-flight will not recompile if no other changes are made. > *Actual behavior*: After every invocation of `cargo build`, arrow-flight is > recompiled, even when nothing has changed > h1. Example > > Create a new crate > {code:java} > alamb@ip-192-168-0-129 arrow_rebuilds % cargo new too_many_rebuilds --bin > cargo new too_many_rebuilds --bin > Created binary (application) `too_many_rebuilds` package > {code} > Add arrow as a dependency in Cargo.toml: > {code:java} > diff --git a/Cargo.toml b/Cargo.toml > index a239680..44ed358 100644 > — a/Cargo.toml > +++ b/Cargo.toml > @@ -5,3 +5,6 @@ authors = ["alamb <and...@nerdnetworks.org>"] > edition = "2018" > # See more keys and their definitions at > [https://doc.rust-lang.org/cargo/reference/manifest.html] > + > +[dependencies] > +arrow = "1.0.0" > {code} > Now, all invocations of `cargo build` will rebuild arrow, even though nothing > in the code has changed: > {code:java} > alamb@ip-192-168-0-129 too_many_rebuilds % cargo build > cargo build > Compiling arrow-flight v1.0.0 > Compiling arrow v1.0.0 > Compiling too_many_rebuilds v0.1.0 > (/Users/alamb/Software/bugs/arrow_rebuilds/too_many_rebuilds) > Finished dev [unoptimized + debuginfo] target(s) in 8.70s > alamb@ip-192-168-0-129 too_many_rebuilds % cargo build > cargo build > Compiling arrow-flight v1.0.0 > Compiling arrow v1.0.0 > Compiling too_many_rebuilds v0.1.0 > (/Users/alamb/Software/bugs/arrow_rebuilds/too_many_rebuilds) > Finished dev [unoptimized + debuginfo] target(s) in 8.65s > {code} > You can see what is happening by checking out a fresh copy of arrow/master > (no Cargo.log) and running `cargo build` – you'll see your local checkout has > changes in rust/arrow-flight/src/arrow.flight.protocol.rs: > {code:java} > alamb@ip-192-168-0-129 arrow % cd rust/arrow > cd rust/arrow > alamb@ip-192-168-0-129 arrow % git status > git status > On branch master > Your branch is up to date with 'origin/master'. > nothing to commit, working tree clean > alamb@ip-192-168-0-129 arrow % cargo build > cargo build > Compiling futures-task v0.3.5 > ... > Compiling arrow v2.0.0-SNAPSHOT (/Users/alamb/Software/arrow/rust/arrow) > Finished dev [unoptimized + debuginfo] target(s) in 21.76s > alamb@ip-192-168-0-129 arrow % > alamb@ip-192-168-0-129 arrow % git status > git status > On branch master > Your branch is up to date with 'origin/master'. > Changes not staged for commit: > (use "git add <file>..." to update what will be committed) > (use "git restore <file>..." to discard changes in working directory) > modified: ../arrow-flight/src/arrow.flight.protocol.rs > no changes added to commit (use "git add" and/or "git commit -a") > {code} > # Root Cause Analysis > The issue is that the build.rs of arrow-flight calls `tonic_build` to auto > generate `rust/arrow-flight/src/arrow.flight.protocol.rs`, which is also > checked in (first done in > [https://github.com/apache/arrow/commit/ec84b7b8102f227295f865c420496830c66a6281]). > This file and the version of tonic were updated on > [https://github.com/apache/arrow/commit/7b49cbc23f22ed99eebf85cc0b9acb1f0d3f832f] > on July 11, 2020 > It turns out that the output of "tonic_build" depends on not only on the > version of tonic, but also on the version of proc-macro2, and the version of > proc-macro2 is not specifically pinned. > `proc-macro2 = "1.0.19"` was released on July 19, 2020 > ([https://crates.io/crates/proc-macro2/1.0.19]) and it appears to subtlety > changes the resulting output of arrow.flight.protocol.rs; Thus the output no > longer matches what is checked in. This means that anyone without a > Cargo.lock file that pins proc-macro2 to 1.0.18 would get 1.0.19 and thus > also a local modification during build. > h1. Workaround > If we pin Cargo.toml to use proc-macro2 1.0.18 the local modification stops. > {code} > proc-macro2 = "1.0.18" > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)