Todd Lipcon has submitted this change and it was merged. Change subject: KUDU-1448. Enable AVX2 bitshuffle at runtime ......................................................................
KUDU-1448. Enable AVX2 bitshuffle at runtime This uses some trickery when building bitshuffle so that we build it twice: once with normal flags and a second time with -mavx2. Enabling AVX2 turns on some faster code paths which have been previously benchmarked to yield a ~30% end-to-end improvement in some Impala scans. In order to avoid multiply-defined symbols, the build uses 'objcopy' to rename the symbols to add a '_avx2' suffix before linking. Then a new wrapper file uses the 'ifunc' attribute to do runtime resolution of the correct variant to call. I'd previously attempted to do this upstream in the bitshuffle library but it was somewhat difficult as the library makes heavy use of macros, etc. In that prior attempt, I gave up after many hours of work, whereas this was comparatively quite simple (only an hour of work!) This also fixes a bug in the cpuid detection which we inherited from Chromium. I filed the bug upstream here: https://bugs.chromium.org/p/chromium/issues/detail?id=667457 I tested on my laptop which has AVX2 and verified that the AVX2 variant functions were getting called. Change-Id: Ied8887c0240a649158085676fdd08ab8628bf7d6 Reviewed-on: http://gerrit.cloudera.org:8080/5166 Tested-by: Kudu Jenkins Reviewed-by: Dan Burkert <[email protected]> --- M src/kudu/cfile/CMakeLists.txt A src/kudu/cfile/bitshuffle_arch_wrapper.cc A src/kudu/cfile/bitshuffle_arch_wrapper.h M src/kudu/cfile/bshuf_block.cc M src/kudu/cfile/bshuf_block.h M src/kudu/gutil/cpu.cc M thirdparty/build-definitions.sh 7 files changed, 178 insertions(+), 12 deletions(-) Approvals: Dan Burkert: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/5166 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ied8887c0240a649158085676fdd08ab8628bf7d6 Gerrit-PatchSet: 5 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Todd Lipcon <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Dan Burkert <[email protected]> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon <[email protected]>
