[jira] [Commented] (ARROW-2563) [Rust] Poor caching in Travis-CI

2018-05-10 Thread Chao Sun (JIRA) 

[ 
https://issues.apache.org/jira/browse/ARROW-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471046#comment-16471046
 ] 

Chao Sun commented on ARROW-2563:
-

The poor caching was caused by the {{--force}} flag which ignores the cached 
cargo. In the latest PR I removed it and reduced the time to around 6-8 min. 
[~pitrou]: can you take a look? Thanks.

> [Rust] Poor caching in Travis-CI
> 
>
> Key: ARROW-2563
> URL: https://issues.apache.org/jira/browse/ARROW-2563
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Continuous Integration, Rust
>Reporter: Antoine Pitrou
>Assignee: Chao Sun
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Since the Rust project isn't at the repo root, Travis-CI won't compiled cache 
> artifacts by default. This leads to long CI times as all packages get 
> recompiled (see https://docs.travis-ci.com/user/caching/#Rust-Cargo-cache for 
> what gets cached).
> In https://travis-ci.org/pitrou/arrow/jobs/376859806 I tried the following:
> {code}
> export CARGO_TARGET_DIR=$TRAVIS_BUILD_DIR/target
> {code}
> and after a first run, the build time went down to 2 minutes (from 15-18 
> minutes).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2563) [Rust] Poor caching in Travis-CI

2018-05-09 Thread Chao Sun (JIRA) 

[ 
https://issues.apache.org/jira/browse/ARROW-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469904#comment-16469904
 ] 

Chao Sun commented on ARROW-2563:
-

OK. Let me give it a try.

> [Rust] Poor caching in Travis-CI
> 
>
> Key: ARROW-2563
> URL: https://issues.apache.org/jira/browse/ARROW-2563
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Continuous Integration, Rust
>Reporter: Antoine Pitrou
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Since the Rust project isn't at the repo root, Travis-CI won't compiled cache 
> artifacts by default. This leads to long CI times as all packages get 
> recompiled (see https://docs.travis-ci.com/user/caching/#Rust-Cargo-cache for 
> what gets cached).
> In https://travis-ci.org/pitrou/arrow/jobs/376859806 I tried the following:
> {code}
> export CARGO_TARGET_DIR=$TRAVIS_BUILD_DIR/target
> {code}
> and after a first run, the build time went down to 2 minutes (from 15-18 
> minutes).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2563) [Rust] Poor caching in Travis-CI

2018-05-09 Thread Antoine Pitrou (JIRA) 

[ 
https://issues.apache.org/jira/browse/ARROW-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469540#comment-16469540
 ] 

Antoine Pitrou commented on ARROW-2563:
---

{quote}I've seen this before but not sure whether it is related to caching: 
could you restart the job and see if this happens consistently?{quote}

I've restarted it twice already. Feel free to try on your side. The branch I 
tried this on is unrelated, I was just surprised by the Rust compilation times 
and experimented a bit.

> [Rust] Poor caching in Travis-CI
> 
>
> Key: ARROW-2563
> URL: https://issues.apache.org/jira/browse/ARROW-2563
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Continuous Integration, Rust
>Reporter: Antoine Pitrou
>Priority: Major
>
> Since the Rust project isn't at the repo root, Travis-CI won't compiled cache 
> artifacts by default. This leads to long CI times as all packages get 
> recompiled (see https://docs.travis-ci.com/user/caching/#Rust-Cargo-cache for 
> what gets cached).
> In https://travis-ci.org/pitrou/arrow/jobs/376859806 I tried the following:
> {code}
> export CARGO_TARGET_DIR=$TRAVIS_BUILD_DIR/target
> {code}
> and after a first run, the build time went down to 2 minutes (from 15-18 
> minutes).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2563) [Rust] Poor caching in Travis-CI

2018-05-09 Thread Chao Sun (JIRA) 

[ 
https://issues.apache.org/jira/browse/ARROW-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469295#comment-16469295
 ] 

Chao Sun commented on ARROW-2563:
-

I've seen this before but not sure whether it is related to caching: could you 
restart the job and see if this happens consistently?

I can take a look at this.

> [Rust] Poor caching in Travis-CI
> 
>
> Key: ARROW-2563
> URL: https://issues.apache.org/jira/browse/ARROW-2563
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Continuous Integration, Rust
>Reporter: Antoine Pitrou
>Priority: Major
>
> Since the Rust project isn't at the repo root, Travis-CI won't compiled cache 
> artifacts by default. This leads to long CI times as all packages get 
> recompiled (see https://docs.travis-ci.com/user/caching/#Rust-Cargo-cache for 
> what gets cached).
> In https://travis-ci.org/pitrou/arrow/jobs/376859806 I tried the following:
> {code}
> export CARGO_TARGET_DIR=$TRAVIS_BUILD_DIR/target
> {code}
> and after a first run, the build time went down to 2 minutes (from 15-18 
> minutes).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2563) [Rust] Poor caching in Travis-CI

2018-05-09 Thread Antoine Pitrou (JIRA) 

[ 
https://issues.apache.org/jira/browse/ARROW-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469236#comment-16469236
 ] 

Antoine Pitrou commented on ARROW-2563:
---

Yes it's {{cache: cargo}}.

> [Rust] Poor caching in Travis-CI
> 
>
> Key: ARROW-2563
> URL: https://issues.apache.org/jira/browse/ARROW-2563
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Continuous Integration, Rust
>Reporter: Antoine Pitrou
>Priority: Major
>
> Since the Rust project isn't at the repo root, Travis-CI won't compiled cache 
> artifacts by default. This leads to long CI times as all packages get 
> recompiled (see https://docs.travis-ci.com/user/caching/#Rust-Cargo-cache for 
> what gets cached).
> In https://travis-ci.org/pitrou/arrow/jobs/376859806 I tried the following:
> {code}
> export CARGO_TARGET_DIR=$TRAVIS_BUILD_DIR/target
> {code}
> and after a first run, the build time went down to 2 minutes (from 15-18 
> minutes).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2563) [Rust] Poor caching in Travis-CI

2018-05-09 Thread Chao Sun (JIRA) 

[ 
https://issues.apache.org/jira/browse/ARROW-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469233#comment-16469233
 ] 

Chao Sun commented on ARROW-2563:
-

[~pitrou]: which caching is this? `cache: cargo`?

> [Rust] Poor caching in Travis-CI
> 
>
> Key: ARROW-2563
> URL: https://issues.apache.org/jira/browse/ARROW-2563
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Continuous Integration, Rust
>Reporter: Antoine Pitrou
>Priority: Major
>
> Since the Rust project isn't at the repo root, Travis-CI won't compiled cache 
> artifacts by default. This leads to long CI times as all packages get 
> recompiled (see https://docs.travis-ci.com/user/caching/#Rust-Cargo-cache for 
> what gets cached).
> In https://travis-ci.org/pitrou/arrow/jobs/376859806 I tried the following:
> {code}
> export CARGO_TARGET_DIR=$TRAVIS_BUILD_DIR/target
> {code}
> and after a first run, the build time went down to 2 minutes (from 15-18 
> minutes).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2563) [Rust] Poor caching in Travis-CI

2018-05-09 Thread Antoine Pitrou (JIRA) 

[ 
https://issues.apache.org/jira/browse/ARROW-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469214#comment-16469214
 ] 

Antoine Pitrou commented on ARROW-2563:
---

However, it seems that the caching causes a failure in {{cargo coverage}}: see 
https://travis-ci.org/pitrou/arrow/jobs/376900556#L717

I'll let others this up if there's interest, as this is not my area of 
expertise at all. [~andygrove]

> [Rust] Poor caching in Travis-CI
> 
>
> Key: ARROW-2563
> URL: https://issues.apache.org/jira/browse/ARROW-2563
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Continuous Integration, Rust
>Reporter: Antoine Pitrou
>Priority: Major
>
> Since the Rust project isn't at the repo root, Travis-CI won't compiled cache 
> artifacts by default. This leads to long CI times as all packages get 
> recompiled (see https://docs.travis-ci.com/user/caching/#Rust-Cargo-cache for 
> what gets cached).
> In https://travis-ci.org/pitrou/arrow/jobs/376859806 I tried the following:
> {code}
> export CARGO_TARGET_DIR=$TRAVIS_BUILD_DIR/target
> {code}
> and after a first run, the build time went down to 2 minutes (from 15-18 
> minutes).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)