[kudu-CR] build: use thin static libraries on Linux

2018-03-04 Thread Todd Lipcon (Code Review) 
Todd Lipcon has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9472 )

Change subject: build: use thin static libraries on Linux
..

build: use thin static libraries on Linux

Thin static libraries just contain pointers to .o files rather than copying all
of the data. This reduces the amount of IO during the build as well as the 
required
disk space.

For example, comparing a clean build ('ninja clean kudu') before and after this
change, we can see that the build is about 40% faster and about 200x
less disk space in the lib/ directory.

Thick (original):
  real0m24.784s
  user0m58.643s
  sys 0m37.701s

  todd@ve0518:/data/1/todd/kudu$ du -sm build/release/lib/
  594 build/release/lib/

Thin:
  real0m17.422s
  user0m56.092s
  sys 0m34.131s
  todd@ve0518:/data/1/todd/kudu$ du -sm build/release/lib/
  3   build/release/lib/

For comparison, a release build with -DKUDU_LINK=dynamic:

  real0m24.008s
  user2m3.212s
  sys 1m5.243s
  todd@ve0518:/data/1/todd/kudu/build/release-shared$ du -sm lib/
  256 lib/

Of course the dynamic build has a much smaller resulting 'kudu' binary, and if
we are compiling many binaries (eg all the tests), the dynamic build is still
faster and smaller than a thin-static build.

Change-Id: If662cea380e06eaddf45e52617e38e55e4613773
Reviewed-on: http://gerrit.cloudera.org:8080/9472
Reviewed-by: Adar Dembo 
Tested-by: Kudu Jenkins
---
M CMakeLists.txt
1 file changed, 9 insertions(+), 0 deletions(-)

Approvals:
  Adar Dembo: Looks good to me, approved
  Kudu Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/9472
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: If662cea380e06eaddf45e52617e38e55e4613773
Gerrit-Change-Number: 9472
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon

[kudu-CR] build: use thin static libraries on Linux

2018-03-02 Thread Adar Dembo (Code Review) 
Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9472 )

Change subject: build: use thin static libraries on Linux
..


Patch Set 2: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/9472
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If662cea380e06eaddf45e52617e38e55e4613773
Gerrit-Change-Number: 9472
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Sat, 03 Mar 2018 00:57:01 +
Gerrit-HasComments: No

[kudu-CR] build: use thin static libraries on Linux

2018-03-02 Thread Todd Lipcon (Code Review) 
Hello Kudu Jenkins, Adar Dembo,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9472

to look at the new patch set (#2).

Change subject: build: use thin static libraries on Linux
..

build: use thin static libraries on Linux

Thin static libraries just contain pointers to .o files rather than copying all
of the data. This reduces the amount of IO during the build as well as the 
required
disk space.

For example, comparing a clean build ('ninja clean kudu') before and after this
change, we can see that the build is about 40% faster and about 200x
less disk space in the lib/ directory.

Thick (original):
  real0m24.784s
  user0m58.643s
  sys 0m37.701s

  todd@ve0518:/data/1/todd/kudu$ du -sm build/release/lib/
  594 build/release/lib/

Thin:
  real0m17.422s
  user0m56.092s
  sys 0m34.131s
  todd@ve0518:/data/1/todd/kudu$ du -sm build/release/lib/
  3   build/release/lib/

For comparison, a release build with -DKUDU_LINK=dynamic:

  real0m24.008s
  user2m3.212s
  sys 1m5.243s
  todd@ve0518:/data/1/todd/kudu/build/release-shared$ du -sm lib/
  256 lib/

Of course the dynamic build has a much smaller resulting 'kudu' binary, and if
we are compiling many binaries (eg all the tests), the dynamic build is still
faster and smaller than a thin-static build.

Change-Id: If662cea380e06eaddf45e52617e38e55e4613773
---
M CMakeLists.txt
1 file changed, 9 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/72/9472/2
--
To view, visit http://gerrit.cloudera.org:8080/9472
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If662cea380e06eaddf45e52617e38e55e4613773
Gerrit-Change-Number: 9472
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon

[kudu-CR] build: use thin static libraries on Linux

2018-03-02 Thread Todd Lipcon (Code Review) 
Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9472 )

Change subject: build: use thin static libraries on Linux
..


Patch Set 1:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/9472/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/9472/1//COMMIT_MSG@7
PS1, Line 7: build: use thin static libraries on Linux
> Neat trick; how did you discover it?
I read about thin archives somewhere recently... can't recall where, maybe llvm 
weekly?


http://gerrit.cloudera.org:8080/#/c/9472/1//COMMIT_MSG@14
PS1, Line 14: 40% faster
> Out of curiosity, how does this compare to a shared object build?
Added some info to the message


http://gerrit.cloudera.org:8080/#/c/9472/1/CMakeLists.txt
File CMakeLists.txt:

http://gerrit.cloudera.org:8080/#/c/9472/1/CMakeLists.txt@1158
PS1, Line 1158: endif ()
> $ git grep "endif (" | wc -l
Done


http://gerrit.cloudera.org:8080/#/c/9472/1/CMakeLists.txt@1159
PS1, Line 1159:
> Nit: extra empty line here.
Done



--
To view, visit http://gerrit.cloudera.org:8080/9472
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If662cea380e06eaddf45e52617e38e55e4613773
Gerrit-Change-Number: 9472
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Sat, 03 Mar 2018 00:47:03 +
Gerrit-HasComments: Yes

[kudu-CR] build: use thin static libraries on Linux

2018-03-02 Thread Adar Dembo (Code Review) 
Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9472 )

Change subject: build: use thin static libraries on Linux
..


Patch Set 1:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/9472/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/9472/1//COMMIT_MSG@7
PS1, Line 7: build: use thin static libraries on Linux
Neat trick; how did you discover it?


http://gerrit.cloudera.org:8080/#/c/9472/1//COMMIT_MSG@14
PS1, Line 14: 40% faster
Out of curiosity, how does this compare to a shared object build?


http://gerrit.cloudera.org:8080/#/c/9472/1/CMakeLists.txt
File CMakeLists.txt:

http://gerrit.cloudera.org:8080/#/c/9472/1/CMakeLists.txt@1158
PS1, Line 1158: endif ()
$ git grep "endif (" | wc -l
17
$ git grep "endif(" | wc -l
149


http://gerrit.cloudera.org:8080/#/c/9472/1/CMakeLists.txt@1159
PS1, Line 1159:
Nit: extra empty line here.



--
To view, visit http://gerrit.cloudera.org:8080/9472
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If662cea380e06eaddf45e52617e38e55e4613773
Gerrit-Change-Number: 9472
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Sat, 03 Mar 2018 00:29:19 +
Gerrit-HasComments: Yes

[kudu-CR] build: use thin static libraries on Linux

2018-03-02 Thread Todd Lipcon (Code Review) 
Todd Lipcon has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/9472


Change subject: build: use thin static libraries on Linux
..

build: use thin static libraries on Linux

Thin static libraries just contain pointers to .o files rather than copying all
of the data. This reduces the amount of IO during the build as well as the 
required
disk space.

For example, comparing a clean build ('ninja clean kudu') before and after this
change, we can see that the build is about 40% faster and about 200x
less disk space in the lib/ directory.

Thick (original):
  real0m24.784s
  user0m58.643s
  sys 0m37.701s

  todd@ve0518:/data/1/todd/kudu$ du -sm build/release/lib/
  594 build/release/lib/

Thin:
  real0m17.422s
  user0m56.092s
  sys 0m34.131s
  todd@ve0518:/data/1/todd/kudu$ du -sm build/release/lib/
  3   build/release/lib/

Change-Id: If662cea380e06eaddf45e52617e38e55e4613773
---
M CMakeLists.txt
1 file changed, 10 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/72/9472/1
--
To view, visit http://gerrit.cloudera.org:8080/9472
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: If662cea380e06eaddf45e52617e38e55e4613773
Gerrit-Change-Number: 9472
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon