[kudu-CR] [timestamp] Build and and add boost's date time lib
David Ribeiro Alves has posted comments on this change. Change subject: [timestamp] Build and and add boost's date_time lib .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/5818/1/thirdparty/build-definitions.sh File thirdparty/build-definitions.sh: Line 609: build_boost_libs() { > You need to make a few changes here so that it conforms to the rest of the k, I'll try to pass the flads to ./b2, thanks for the pointers. I'll try to make this work on my env first as to not mess the jenkins slaves. Line 613: ./b2 cflags="-DBOOST_DATE_TIME_POSIX_TIME_STD_CONFIG" > Wait, so we need to provide this cflag when building boost AND when consumi so that the iddef is triggered in the headers and we're able to use the nanosecond resolution methods Line 614: cp stage/lib/* $PREFIX/lib > Can't we do a "make install" or equivalent to install the headers along wit what mike said, we're already "installing" the headers on our own so we only need to copy the .a and .so files. -- To view, visit http://gerrit.cloudera.org:8080/5818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I277c4fda15575e271c426735552762884cb28c43 Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: David Ribeiro AlvesGerrit-Reviewer: Adar Dembo Gerrit-Reviewer: David Ribeiro Alves Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: Yes
[kudu-CR] KUDU-1767. Create a client flush interleave test
Mike Percy has abandoned this change. Change subject: KUDU-1767. Create a client flush interleave test .. Abandoned We decided that, at least for now, this will not be fixed. See KUDU-1767. -- To view, visit http://gerrit.cloudera.org:8080/5256 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: abandon Gerrit-Change-Id: I3886f70246c6ef3f94acda6074d02d4fb199cf50 Gerrit-PatchSet: 4 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike PercyGerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Dan Burkert Gerrit-Reviewer: David Ribeiro Alves Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon
[kudu-CR] KUDU-1767. Add java test for client operation interleaving
Mike Percy has abandoned this change. Change subject: KUDU-1767. Add java test for client operation interleaving .. Abandoned We decided that, at least for now, this will not be fixed. See KUDU-1767. -- To view, visit http://gerrit.cloudera.org:8080/5465 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: abandon Gerrit-Change-Id: Idd30aeed34548b01ba4ad9b08cfd5fae22f967e9 Gerrit-PatchSet: 2 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike PercyGerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Jean-Daniel Cryans Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy
[kudu-CR] [timestamp] Build and and add boost's date time lib
Mike Percy has posted comments on this change. Change subject: [timestamp] Build and and add boost's date_time lib .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/5818/4/thirdparty/build-definitions.sh File thirdparty/build-definitions.sh: Line 622: echo "using clang : $VERSION : $CXX_PATH ;" > $USER_JAMFILE This seemed to work in my local env but I guess the path part is not working. I guess someone is going to have to figure out how to get Jam to pick up a custom compiler path. -- To view, visit http://gerrit.cloudera.org:8080/5818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I277c4fda15575e271c426735552762884cb28c43 Gerrit-PatchSet: 4 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: David Ribeiro AlvesGerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: Yes
[kudu-CR] Add Google Breakpad support to Kudu
Mike Percy has posted comments on this change. Change subject: Add Google Breakpad support to Kudu .. Patch Set 28: Verified+1 Overriding Jenkins failure due to a flaky java test. -- To view, visit http://gerrit.cloudera.org:8080/5620 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I495695cc38b75377f20b0497093a81ed5baa887f Gerrit-PatchSet: 28 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike PercyGerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Dinesh Bhat Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: No
[kudu-CR] thirdparty: Clean up thirdparty prefix from previous builds
Mike Percy has posted comments on this change. Change subject: thirdparty: Clean up thirdparty prefix from previous builds .. Patch Set 1: Verified+1 Overriding flaky java test -- To view, visit http://gerrit.cloudera.org:8080/5826 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: If2a7ceb3dad1acede152775611f549798723016c Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike PercyGerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Mike Percy Gerrit-HasComments: No
[kudu-CR] jenkins: Remove bad boost libs from installation dir
Mike Percy has abandoned this change. Change subject: jenkins: Remove bad boost libs from installation dir .. Abandoned -- To view, visit http://gerrit.cloudera.org:8080/5825 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: abandon Gerrit-Change-Id: I27b92b82e1e381d1a832c664106a267539474c2d Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike PercyGerrit-Reviewer: Kudu Jenkins
[kudu-CR] jenkins: Remove bad boost libs from installation dir
Mike Percy has abandoned this change. Change subject: jenkins: Remove bad boost libs from installation dir .. Abandoned -- To view, visit http://gerrit.cloudera.org:8080/5824 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: abandon Gerrit-Change-Id: I27b92b82e1e381d1a832c664106a267539474c2c Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike PercyGerrit-Reviewer: Kudu Jenkins
[kudu-CR] [timestamp] Build and and add boost's date time lib
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/5818 to look at the new patch set (#3). Change subject: [timestamp] Build and and add boost's date_time lib .. [timestamp] Build and and add boost's date_time lib Change-Id: I277c4fda15575e271c426735552762884cb28c43 --- M CMakeLists.txt A cmake_modules/FindBoostLibs.cmake M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh 4 files changed, 115 insertions(+), 28 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/18/5818/3 -- To view, visit http://gerrit.cloudera.org:8080/5818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I277c4fda15575e271c426735552762884cb28c43 Gerrit-PatchSet: 3 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: David Ribeiro AlvesGerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Todd Lipcon
[kudu-CR] jenkins: Remove bad boost libs from installation dir
Mike Percy has abandoned this change. Change subject: jenkins: Remove bad boost libs from installation dir .. Abandoned -- To view, visit http://gerrit.cloudera.org:8080/5823 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: abandon Gerrit-Change-Id: I27b92b82e1e381d1a832c664106a267539474c2b Gerrit-PatchSet: 4 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike PercyGerrit-Reviewer: Kudu Jenkins
[kudu-CR] thirdparty: Clean up thirdparty prefix from previous builds
Mike Percy has uploaded a new change for review. http://gerrit.cloudera.org:8080/5826 Change subject: thirdparty: Clean up thirdparty prefix from previous builds .. thirdparty: Clean up thirdparty prefix from previous builds If we are rebuilding thirdparty anyway, let's remove the artifacts from previous builds that were previously installed, since we are going to reinstall them anyway. This should not affect build times, since we are not wiping out the build directory, and this will protect us against bad artifacts left over from previous failed builds. Change-Id: If2a7ceb3dad1acede152775611f549798723016c --- M thirdparty/build-thirdparty.sh 1 file changed, 10 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/26/5826/1 -- To view, visit http://gerrit.cloudera.org:8080/5826 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: If2a7ceb3dad1acede152775611f549798723016c Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike Percy
[kudu-CR] jenkins: Remove bad boost libs from installation dir
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/5823 to look at the new patch set (#4). Change subject: jenkins: Remove bad boost libs from installation dir .. jenkins: Remove bad boost libs from installation dir This is a temporary fix for Jenkins and should not be merged. Change-Id: I27b92b82e1e381d1a832c664106a267539474c2b --- M thirdparty/build-thirdparty.sh 1 file changed, 6 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/5823/4 -- To view, visit http://gerrit.cloudera.org:8080/5823 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I27b92b82e1e381d1a832c664106a267539474c2b Gerrit-PatchSet: 4 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike PercyGerrit-Reviewer: Kudu Jenkins
[kudu-CR] jenkins: Remove bad boost libs from installation dir
Mike Percy has uploaded a new change for review. http://gerrit.cloudera.org:8080/5825 Change subject: jenkins: Remove bad boost libs from installation dir .. jenkins: Remove bad boost libs from installation dir This is a temporary fix for Jenkins and should not be merged. Change-Id: I27b92b82e1e381d1a832c664106a267539474c2d --- M thirdparty/build-thirdparty.sh 1 file changed, 6 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/25/5825/1 -- To view, visit http://gerrit.cloudera.org:8080/5825 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I27b92b82e1e381d1a832c664106a267539474c2d Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike Percy
[kudu-CR] jenkins: Remove bad boost libs from installation dir
Mike Percy has uploaded a new change for review. http://gerrit.cloudera.org:8080/5824 Change subject: jenkins: Remove bad boost libs from installation dir .. jenkins: Remove bad boost libs from installation dir This is a temporary fix for Jenkins and should not be merged. Change-Id: I27b92b82e1e381d1a832c664106a267539474c2c --- M thirdparty/build-thirdparty.sh 1 file changed, 6 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/24/5824/1 -- To view, visit http://gerrit.cloudera.org:8080/5824 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I27b92b82e1e381d1a832c664106a267539474c2c Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike Percy
[kudu-CR] [timestamp] Build and and add boost's date time lib
Mike Percy has posted comments on this change. Change subject: [timestamp] Build and and add boost's date_time lib .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/5818/1/thirdparty/build-definitions.sh File thirdparty/build-definitions.sh: Line 614: cp stage/lib/* $PREFIX/lib > Can't we do a "make install" or equivalent to install the headers along wit When I tried to do this it installed tons and tons of headers, even though we only built the datetime lib. But maybe that's OK and maybe it's what we want, I'm not sure. -- To view, visit http://gerrit.cloudera.org:8080/5818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I277c4fda15575e271c426735552762884cb28c43 Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: David Ribeiro AlvesGerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: Yes
[kudu-CR] [timestamp] Build and and add boost's date time lib
Mike Percy has posted comments on this change. Change subject: [timestamp] Build and and add boost's date_time lib .. Patch Set 2: I started investigating this because the build servers were broken. Boost's build system is pretty nasty. I made some WIP changes to try and fix the TSAN issues, and this appears to work but is still pretty crufty. For example, I couldn't figure out how to get an out-of-tree build to work so I am just copying the source to the build dir and then running an in-tree build. I leave the rest to David but hope this helps. -- To view, visit http://gerrit.cloudera.org:8080/5818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I277c4fda15575e271c426735552762884cb28c43 Gerrit-PatchSet: 2 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: David Ribeiro AlvesGerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: No
[kudu-CR] [timestamp] Build and and add boost's date time lib
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/5818 to look at the new patch set (#2). Change subject: [timestamp] Build and and add boost's date_time lib .. [timestamp] Build and and add boost's date_time lib Change-Id: I277c4fda15575e271c426735552762884cb28c43 --- M CMakeLists.txt A cmake_modules/FindBoostLibs.cmake M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh 4 files changed, 116 insertions(+), 28 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/18/5818/2 -- To view, visit http://gerrit.cloudera.org:8080/5818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I277c4fda15575e271c426735552762884cb28c43 Gerrit-PatchSet: 2 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: David Ribeiro AlvesGerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Todd Lipcon
[kudu-CR] jenkins: Remove bad boost libs from installation dir
Mike Percy has uploaded a new patch set (#2). Change subject: jenkins: Remove bad boost libs from installation dir .. jenkins: Remove bad boost libs from installation dir This is a temporary fix for Jenkins and should not be merged Change-Id: I27b92b82e1e381d1a832c664106a267539474c2b --- M thirdparty/build-thirdparty.sh 1 file changed, 8 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/5823/2 -- To view, visit http://gerrit.cloudera.org:8080/5823 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I27b92b82e1e381d1a832c664106a267539474c2b Gerrit-PatchSet: 2 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike PercyGerrit-Reviewer: Kudu Jenkins
[kudu-CR] jenkins: Remove bad boost libs from installation dir
Mike Percy has uploaded a new change for review. http://gerrit.cloudera.org:8080/5823 Change subject: jenkins: Remove bad boost libs from installation dir .. jenkins: Remove bad boost libs from installation dir This is a temporary fix for Jenkins and should not be merged Change-Id: I27b92b82e1e381d1a832c664106a267539474c2b --- M thirdparty/build-thirdparty.sh 1 file changed, 8 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/5823/1 -- To view, visit http://gerrit.cloudera.org:8080/5823 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I27b92b82e1e381d1a832c664106a267539474c2b Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike Percy
[kudu-CR] [security] sign/verify data using RSA key pair
Alexey Serbin has posted comments on this change. Change subject: [security] sign/verify data using RSA key pair .. Patch Set 8: Verified+1 unrelated breakage: boost_date_time issues -- To view, visit http://gerrit.cloudera.org:8080/5805 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3fa04bb7d09aa416363998e2f8d7ccbdea625e4f Gerrit-PatchSet: 8 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Alexey SerbinGerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Dan Burkert Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: No
[kudu-CR] [security] do actual token signing/verification
Alexey Serbin has posted comments on this change. Change subject: [security] do actual token signing/verification .. Patch Set 4: Verified+1 unrelated breakage: boost_datetime issues -- To view, visit http://gerrit.cloudera.org:8080/5812 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Icf035c64032320a450731ae921e92615bf2efd98 Gerrit-PatchSet: 4 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Alexey SerbinGerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Dan Burkert Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: No
[kudu-CR] [security] introduced crypto-test
Alexey Serbin has posted comments on this change. Change subject: [security] introduced crypto-test .. Patch Set 8: Verified+1 unrelated breakage: boost_datetime issues -- To view, visit http://gerrit.cloudera.org:8080/5798 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ibace36b6336b650879cf3f9139ccf562f82765a7 Gerrit-PatchSet: 8 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Alexey SerbinGerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Dan Burkert Gerrit-Reviewer: Tidy Bot Gerrit-HasComments: No
[kudu-CR] [security] method to extract public part of RSA key
Alexey Serbin has posted comments on this change. Change subject: [security] method to extract public part of RSA key .. Patch Set 8: Verified+1 unrelated breakage: boost_datetime issues -- To view, visit http://gerrit.cloudera.org:8080/5783 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I8ac61a08c8165152d0046468c0fa655131ff8fef Gerrit-PatchSet: 8 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Alexey SerbinGerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Dan Burkert Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: No
[kudu-CR] [timestamp] Build and and add boost's date time lib
Adar Dembo has posted comments on this change. Change subject: [timestamp] Build and and add boost's date_time lib .. Patch Set 1: (8 comments) The TSAN failure shows that the boost libraries are not built against the TSAN-instrumented libc++, but rather against the system's libstdc++. That can be fixed via the CFLAGS/CXXFLAGS stuff I mentioned in one of the comments. http://gerrit.cloudera.org:8080/#/c/5818/1//COMMIT_MSG Commit Message: Line 7: [timestamp] Build and and add boost's date_time lib Since starting to use boost libraries (again) is somewhat of a shift from our previous "let's try to get rid of all boost usage, eventually" attitude, it would be nice to explain why we need to use date_time. What specifically is it about the TIMESTAMP type that makes it much more difficult to implement without date_time? http://gerrit.cloudera.org:8080/#/c/5818/1/CMakeLists.txt File CMakeLists.txt: Line 140: set(CXX_COMMON_FLAGS "-fno-strict-aliasing -msse4.2 -Wall -Wno-sign-compare -Wno-deprecated -pthread -DBOOST_DATE_TIME_POSIX_TIME_STD_CONFIG") Since this line is getting quite long, could you break it up? Something like this: # -fno-strict-aliasing: ... set(CXX_COMMON_FLAGS "-fno-strict-aliasing) # -msse4.2: ... set(CXX_COMMON_FLAGS "${CXX_COMMON_FLAGS} -msse4.2") ... Line 986: find_package(BoostLibs REQUIRED) Why can't we reuse find_package(Boost) for this? ISTR you can provide certain libraries and it'll find them too. We used to do that. http://gerrit.cloudera.org:8080/#/c/5818/1/cmake_modules/FindBoostLibs.cmake File cmake_modules/FindBoostLibs.cmake: PS1, Line 26: snappy Copy/pasta http://gerrit.cloudera.org:8080/#/c/5818/1/thirdparty/build-definitions.sh File thirdparty/build-definitions.sh: Line 609: build_boost_libs() { You need to make a few changes here so that it conforms to the rest of the thirdparty build: 1. Build boost in a BOOST_BDIR that includes MODE_SUFFIX in its name, to isolate the TSAN build directory from the regular build directory. 2. Make sure the build incorporates CFLAGS, CXXFLAGS, LDFLAGS, and LIBS, so that it'll build against libc++ when in TSAN mode. 3. Also make sure CC/CXX have an effect if they've been set (i.e. to build with clang in TSAN mode); maybe this is automatic. Line 613: ./b2 cflags="-DBOOST_DATE_TIME_POSIX_TIME_STD_CONFIG" Wait, so we need to provide this cflag when building boost AND when consuming boost? Why both? Line 614: cp stage/lib/* $PREFIX/lib Can't we do a "make install" or equivalent to install the headers along with the libraries? I think "./b2 install" should do the trick. http://gerrit.cloudera.org:8080/#/c/5818/1/thirdparty/build-thirdparty.sh File thirdparty/build-thirdparty.sh: PS1, Line 93: "boost_headers") F_BOOST_HEADERS=1 ;; : "boost_libs") F_BOOST_LIBS=1 ;; I don't think we should separate them like this. If we're going to start building the libraries, we should treat the headers as part of the libraries and install the whole tree into uninstrumented/tsan. -- To view, visit http://gerrit.cloudera.org:8080/5818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I277c4fda15575e271c426735552762884cb28c43 Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: David Ribeiro AlvesGerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: Yes
[kudu-CR] KUDU-1845: Kerberos client keytab should be periodically renewed
Adar Dembo has posted comments on this change. Change subject: KUDU-1845: Kerberos client keytab should be periodically renewed .. Patch Set 1: (2 comments) Just passing through... http://gerrit.cloudera.org:8080/#/c/5820/1/src/kudu/security/init.cc File src/kudu/security/init.cc: Line 38: DEFINE_int32(kerberos_reinit_interval, 60, "Duration in minutes before which an attempt to " I don't have much context on the security work going on in Kudu, but I was wondering: would it be possible to use the expiration of the Kerberos credentials to dictate how long to sleep? That is, if we know our tickets are expiring in 6 hours, we can sleep until then, rather than waking up periodically? Ideally we wouldn't even need a thread, but schedule the future work as a task. Unfortunately, that facility is provided by the messenger in krpc, so using it might create a circular dependency. Line 86: SleepFor(MonoDelta::FromSeconds(FLAGS_kerberos_reinit_interval * 60)); Shouldn't this thread participate in the rest of the server lifecycle though? That is, if the server is shutdown, shouldn't we explicitly stop this thread? -- To view, visit http://gerrit.cloudera.org:8080/5820 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic4c072c1210216369e60eac88be4a20d9b166b2d Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Sailesh MukilGerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tidy Bot Gerrit-HasComments: Yes
[kudu-CR] Control mutex stack walking in DEBUG mode with a gflag
Adar Dembo has posted comments on this change. Change subject: Control mutex stack walking in DEBUG mode with a gflag .. Patch Set 10: (2 comments) http://gerrit.cloudera.org:8080/#/c/5741/10//COMMIT_MSG Commit Message: PS10, Line 19: excluvisity Nit: exclusivity http://gerrit.cloudera.org:8080/#/c/5741/10/src/kudu/util/mutex.cc File src/kudu/util/mutex.cc: Line 100: DCHECK_EQ(0, rv) << ". " << strerror(rv) << ". " << GetOwnerThreadInfo(); I think moving the DCHECK_EQ() into the NDEBUG check has the effect of causing an unused variable warning to show up in release builds. Can you double check that? -- To view, visit http://gerrit.cloudera.org:8080/5741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ie4593cf7173867ce2f6151e03df0be94f97d95d2 Gerrit-PatchSet: 10 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike PercyGerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: Yes
[kudu-CR] Add Google Breakpad support to Kudu
Adar Dembo has posted comments on this change. Change subject: Add Google Breakpad support to Kudu .. Patch Set 27: (3 comments) http://gerrit.cloudera.org:8080/#/c/5620/27/src/kudu/client/symbols.map File src/kudu/client/symbols.map: Line 26: # breakpad Can we wildcard some of these? Maybe ConvertUTF* and my_*? http://gerrit.cloudera.org:8080/#/c/5620/27/src/kudu/util/minidump.cc File src/kudu/util/minidump.cc: Line 217: google::InstallFailureFunction(); Do you even need the redirection through AbortFailureFunction? abort() has the same signature, so maybe you can pass it directly here? http://gerrit.cloudera.org:8080/#/c/5620/27/src/kudu/util/subprocess.cc File src/kudu/util/subprocess.cc: Line 432: ResetSigPipeHandlerToDefault(); Why do we only reset SIGPIPE to SIG_DFL and not every signal? What about SIGUSR1? -- To view, visit http://gerrit.cloudera.org:8080/5620 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I495695cc38b75377f20b0497093a81ed5baa887f Gerrit-PatchSet: 27 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike PercyGerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Dinesh Bhat Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: Yes
[kudu-CR] KUDU-1835 (part 2): enable WAL compression
David Ribeiro Alves has posted comments on this change. Change subject: KUDU-1835 (part 2): enable WAL compression .. Patch Set 9: just a tiny nit, feel free to ignore or fix and keep the +2 -- To view, visit http://gerrit.cloudera.org:8080/5736 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I0fc790ddc72a519a82bbb6b71a902418051b35a5 Gerrit-PatchSet: 9 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Todd LipconGerrit-Reviewer: David Ribeiro Alves Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: No
[kudu-CR] KUDU-1835 (part 2): enable WAL compression
David Ribeiro Alves has posted comments on this change. Change subject: KUDU-1835 (part 2): enable WAL compression .. Patch Set 9: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/5736/9/src/kudu/consensus/log-test.cc File src/kudu/consensus/log-test.cc: PS9, Line 151: extra lines -- To view, visit http://gerrit.cloudera.org:8080/5736 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I0fc790ddc72a519a82bbb6b71a902418051b35a5 Gerrit-PatchSet: 9 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Todd LipconGerrit-Reviewer: David Ribeiro Alves Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: Yes
[kudu-CR] WIP: Add a new TIMESTAMP type
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/5819 to look at the new patch set (#3). Change subject: WIP: Add a new TIMESTAMP type .. WIP: Add a new TIMESTAMP type This adds a new timestamp type that matches Impala's. The new type can be used as a key and has all the facilities of the other types except on-disk encodings. This might still need some cleanup but should mostly be gtg. Change-Id: I63271adf6b87f048e30bf2d6d04c758884752afc --- M src/kudu/cfile/type_encodings.cc M src/kudu/client/scan_batch.cc M src/kudu/client/scan_batch.h M src/kudu/client/scanner-internal.cc M src/kudu/client/schema.cc M src/kudu/client/schema.h M src/kudu/client/value-internal.h M src/kudu/client/value.cc M src/kudu/client/value.h M src/kudu/common/common.proto M src/kudu/common/encoded_key-test.cc M src/kudu/common/key_encoder.cc M src/kudu/common/key_encoder.h M src/kudu/common/key_util-test.cc M src/kudu/common/key_util.cc M src/kudu/common/partial_row.cc M src/kudu/common/partial_row.h M src/kudu/common/partition.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/integration-tests/all_types-itest.cc M src/kudu/util/CMakeLists.txt A src/kudu/util/timestamp_value-test.cc A src/kudu/util/timestamp_value.cc A src/kudu/util/timestamp_value.h 25 files changed, 843 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/19/5819/3 -- To view, visit http://gerrit.cloudera.org:8080/5819 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I63271adf6b87f048e30bf2d6d04c758884752afc Gerrit-PatchSet: 3 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: David Ribeiro AlvesGerrit-Reviewer: Jean-Daniel Cryans Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon
[kudu-CR] [python] KUDU-1563. Add support for INSERT IGNORE
Jordan Birdsell has posted comments on this change. Change subject: [python] KUDU-1563. Add support for INSERT IGNORE .. Patch Set 10: Is this still going on? -- To view, visit http://gerrit.cloudera.org:8080/4522 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I6c45a50d4b87d8f7c4f0f83fbc72932d056d3a79 Gerrit-PatchSet: 10 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Brock NolandGerrit-Reviewer: Brock Noland Gerrit-Reviewer: David Ribeiro Alves Gerrit-Reviewer: Jordan Birdsell Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: No
[kudu-CR] [python] - Enable Session.apply for list of WriteOperations
Jordan Birdsell has abandoned this change. Change subject: [python] - Enable Session.apply for list of WriteOperations .. Abandoned -- To view, visit http://gerrit.cloudera.org:8080/5231 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: abandon Gerrit-Change-Id: I637e08547e2f710dded8248a18652b575b239cbe Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Jordan BirdsellGerrit-Reviewer: Jordan Birdsell Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon
[kudu-CR] [python] - Enable Session.apply for list of WriteOperations
Jordan Birdsell has posted comments on this change. Change subject: [python] - Enable Session.apply for list of WriteOperations .. Patch Set 1: After thinking about it a bit further, i agree. Going to withdraw this. -- To view, visit http://gerrit.cloudera.org:8080/5231 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I637e08547e2f710dded8248a18652b575b239cbe Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Jordan BirdsellGerrit-Reviewer: Jordan Birdsell Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: No
[kudu-CR] KUDU-1691 - [python] Updated Column Encoding Types
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/4824 to look at the new patch set (#3). Change subject: KUDU-1691 - [python] Updated Column Encoding Types .. KUDU-1691 - [python] Updated Column Encoding Types GROUP_VARINT encoding has been depricated and as thus been removed from the Python client. Additionally, bit shuffle and dictionary encoding have been added. This patch includes updated unit tests. Change-Id: I18a611e746d48879d6e9988abcee42e0354dcfdc --- M python/kudu/__init__.py M python/kudu/libkudu_client.pxd M python/kudu/schema.pyx M python/kudu/tests/test_schema.py 4 files changed, 17 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/24/4824/3 -- To view, visit http://gerrit.cloudera.org:8080/4824 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I18a611e746d48879d6e9988abcee42e0354dcfdc Gerrit-PatchSet: 3 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: g.kocu...@gmail.com Gerrit-Reviewer: David Ribeiro AlvesGerrit-Reviewer: Jordan Birdsell Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: g.kocu...@gmail.com
[kudu-CR] [security] do actual token signing/verification
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/5812 to look at the new patch set (#3). Change subject: [security] do actual token signing/verification .. [security] do actual token signing/verification Replaced temporary stub sign/verify calls with calls of actual cryptographic functions. Change-Id: Icf035c64032320a450731ae921e92615bf2efd98 --- M src/kudu/security/token_signing_key.cc M src/kudu/security/token_signing_key.h 2 files changed, 23 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/12/5812/3 -- To view, visit http://gerrit.cloudera.org:8080/5812 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icf035c64032320a450731ae921e92615bf2efd98 Gerrit-PatchSet: 3 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Alexey SerbinGerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Dan Burkert Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon
[kudu-CR] [security] sign/verify data using RSA key pair
Hello Dan Burkert, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/5805 to look at the new patch set (#7). Change subject: [security] sign/verify data using RSA key pair .. [security] sign/verify data using RSA key pair Added functionality to make a signature of a data chunk and verify signature of a data chunk using RSA keys. As for now, supporting SHA1, SHA256 and SHA512 message digests with test coverage for SHA512. Change-Id: I3fa04bb7d09aa416363998e2f8d7ccbdea625e4f --- M src/kudu/security/crypto-test.cc M src/kudu/security/crypto.cc M src/kudu/security/crypto.h M src/kudu/security/test/test_certs.cc M src/kudu/security/test/test_certs.h 5 files changed, 315 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/05/5805/7 -- To view, visit http://gerrit.cloudera.org:8080/5805 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I3fa04bb7d09aa416363998e2f8d7ccbdea625e4f Gerrit-PatchSet: 7 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Alexey SerbinGerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Dan Burkert Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon
[kudu-CR] [security] method to extract public part of RSA key
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/5783 to look at the new patch set (#7). Change subject: [security] method to extract public part of RSA key .. [security] method to extract public part of RSA key Added a method to get the public part of an RSA keypair. Renamed Key --> RSAPrivateKey. Reorganized the layout of files in the src/security sub-directory: separated the common crypto stuff from openssl_util.h into crypto.h. Change-Id: I8ac61a08c8165152d0046468c0fa655131ff8fef --- M src/kudu/master/master_cert_authority.cc M src/kudu/master/master_cert_authority.h M src/kudu/security/CMakeLists.txt R src/kudu/security/ca/cert_management-test.cc M src/kudu/security/ca/cert_management.cc M src/kudu/security/ca/cert_management.h A src/kudu/security/cert.cc A src/kudu/security/cert.h A src/kudu/security/crypto.cc A src/kudu/security/crypto.h M src/kudu/security/openssl_util.cc M src/kudu/security/openssl_util.h A src/kudu/security/openssl_util_bio.h M src/kudu/security/server_cert_manager.cc M src/kudu/security/server_cert_manager.h M src/kudu/security/test/test_certs.cc M src/kudu/security/test/test_certs.h M src/kudu/security/tls_handshake-test.cc M src/kudu/security/tls_handshake.h M src/kudu/security/tls_socket.cc M src/kudu/security/token_signer.cc M src/kudu/security/token_signing_key.cc M src/kudu/security/token_signing_key.h 23 files changed, 725 insertions(+), 341 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/83/5783/7 -- To view, visit http://gerrit.cloudera.org:8080/5783 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8ac61a08c8165152d0046468c0fa655131ff8fef Gerrit-PatchSet: 7 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Alexey SerbinGerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Dan Burkert Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon
[kudu-CR] [security] introduced crypto-test
Hello Dan Burkert, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/5798 to look at the new patch set (#7). Change subject: [security] introduced crypto-test .. [security] introduced crypto-test The new crypto-test module is for test scenarios involving common non-CA crypto entities in the Kudu security library. The CA-related scenarios are kept in the cert_management-test. Change-Id: Ibace36b6336b650879cf3f9139ccf562f82765a7 --- M src/kudu/security/CMakeLists.txt M src/kudu/security/ca/cert_management-test.cc A src/kudu/security/crypto-test.cc M src/kudu/security/test/test_certs.cc M src/kudu/security/test/test_certs.h 5 files changed, 209 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/98/5798/7 -- To view, visit http://gerrit.cloudera.org:8080/5798 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ibace36b6336b650879cf3f9139ccf562f82765a7 Gerrit-PatchSet: 7 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Alexey SerbinGerrit-Reviewer: Dan Burkert Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tidy Bot
[kudu-CR] [security] do actual token signing/verification
Alexey Serbin has posted comments on this change. Change subject: [security] do actual token signing/verification .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/5812/2/src/kudu/security/token_signing_key.cc File src/kudu/security/token_signing_key.cc: Line 35: CHECK(pb_.has_rsa_key_der()); > These CHECKs in the ctors are a little fishy. Maybe we could add an Init f Why CHECK is fishy in constructors? Do you mean it's better to remove CHECK() at all or you are OK with CHECK() anywhere but constructor? -- To view, visit http://gerrit.cloudera.org:8080/5812 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Icf035c64032320a450731ae921e92615bf2efd98 Gerrit-PatchSet: 2 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Alexey SerbinGerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Dan Burkert Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: Yes
[kudu-CR] KUDU-1851 - [python] TableAlterer direct instantiation causes SIGSEGV
Jordan Birdsell has uploaded a new change for review. http://gerrit.cloudera.org:8080/5822 Change subject: KUDU-1851 - [python] TableAlterer direct instantiation causes SIGSEGV .. KUDU-1851 - [python] TableAlterer direct instantiation causes SIGSEGV The python client is currently crashing when a TableAlterer is directly instantiated as opposed to instantiating via the new_table_alterer method in the Client class. This patch addresses this issue and includes a new unit test. Change-Id: I074737c8f6992a7cd21f72f711337e915ffe2e82 --- M python/kudu/client.pyx M python/kudu/tests/test_client.py 2 files changed, 24 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/22/5822/1 -- To view, visit http://gerrit.cloudera.org:8080/5822 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I074737c8f6992a7cd21f72f711337e915ffe2e82 Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Jordan Birdsell
[kudu-CR] Control mutex stack walking in DEBUG mode with a gflag
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/5741 to look at the new patch set (#10). Change subject: Control mutex stack walking in DEBUG mode with a gflag .. Control mutex stack walking in DEBUG mode with a gflag This patch disables the Mutex owner stack trace collection on DEBUG builds by default, only enabling it when a certain gflag is set. In DEBUG mode, our Mutex implementation collects a stack trace of the owning thread each time the Mutex is acquired. It does this by calling google::GetStackTrace() from glog, which in the context of the Kudu build environment calls into libunwind to collect the stack trace. At the time of writing, google::GetStackTrace() only allows access by one thread at a time. If more than one thread attempts to invoke this function simultaneously, there is a CAS that determines excluvisity. The "loser" of this contest gets a short-circuit return along with an empty stack trace, indicating a failure to collect the stack trace. NB: I have filed a glog issue about that behavior upstream. For more information, see https://github.com/google/glog/issues/160 This situation becomes a problem when there are one or more Mutexes constantly being acquired. When that happens, there is always a thread collecting a stack trace, and so the probability of being able to successfully collect a stack trace at any given moment is greatly reduced. One important caller of google::GetStackTrace() is the glog failure function and SIGABRT signal handler that is called when a CHECK() fails or a LOG(FATAL) call is invoked. I have observed that this crash handler will often print an empty stack trace in DEBUG mode. Investigating this issue led me to discover that we had a thread (our AsyncLogger thread) constantly acquiring a Mutex and racing on the above-mentioned CAS check inside google::GetStackTrace(). Depriving our DEBUG builds of stack traces on LOG(FATAL) or CHECK failures, especially on Jenkins runs, is counterproductive. One simple solution to this problem is to disable this behavior by default. Change-Id: Ie4593cf7173867ce2f6151e03df0be94f97d95d2 --- M src/kudu/util/mutex.cc M src/kudu/util/mutex.h 2 files changed, 37 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/5741/10 -- To view, visit http://gerrit.cloudera.org:8080/5741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie4593cf7173867ce2f6151e03df0be94f97d95d2 Gerrit-PatchSet: 10 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike PercyGerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Todd Lipcon
[kudu-CR] Add Google Breakpad support to Kudu
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/5620 to look at the new patch set (#27). Change subject: Add Google Breakpad support to Kudu .. Add Google Breakpad support to Kudu Breakpad creates minidumps upon crash, which are small files that include stack-based information for each thread. These do not include heap information, so they complement, not replace, core files for debugging purposes. This patch enables Google Breakpad for the kudu-tserver and kudu-master processes. Breakpad is part of the crash-reporting infrastructure open-sourced by Google. It is also used by Mozilla Firefox and Google Chrome. When a process crashes, Breakpad writes a small dump file called a minidump to disk. Since these files are typically only a few MB in size, this allows users to share critical crash information with developers for offline analysis. For more information, see the breakpad getting started docs at http://chromium.googlesource.com/breakpad/breakpad/+/master/docs/getting_started_with_breakpad.md By default, on crash, minidump files are written to a subdirectory of the log directory for a given Kudu daemon process. This was chosen for the following reasons: 1. The minidump files are relatively small, potentially comparable in size to log files, and the log directory is a place an administrator would look in when debugging. 2. This convention is consistent with what Apache Impala (incubating) does for its minidump files. Changes in this patch: * Add breakpad to thirdparty. * Add breakpad source archive creation script (Breakpad does not do releases). * Add a "hack" to the breakpad thirdparty installation routine to forcibly add a breakpad/ prefix to the breakpad header files. This frees us from having to modify our include path when building. * Pull in (heavily modified) util/minidump.{cc,h} from Apache Impala (incubating). The modifications consist of removing use of boost::filesystem, conformance to the Kudu code style guidelines, and some significant refactoring. * Enable breakpad support for the kudu-tserver and kudu-master processes. * By default, invoke previously-installed signal handler if installed. * Handle SIGUSR1 by safely using sigwait() to create minidumps on demand. * Add #ifdefs and CMake logic for Linux. * Add test for all the deadly signals to ensure we get both a stack trace and a minidump, except for the case of SIGTERM, where we should not get a minidump (by design). * Change a few library unit tests that relied on SIGUSR1 to use another signal for their testing, such as SIGHUP or SIGUSR2. * Remove unused includes from mini_tablet_server.cc * Clean up signal handling logic when forking a subprocess. * Hide breakpad symbols from libkuduclient.so * Install failure function that simply calls abort() instead of first printing a stacktrace. Change-Id: I495695cc38b75377f20b0497093a81ed5baa887f --- M CMakeLists.txt M build-support/jenkins/build-and-test.sh A cmake_modules/FindBreakpadClient.cmake M java/kudu-client/src/test/java/org/apache/kudu/client/MiniKuduCluster.java M src/kudu/client/client_samples-test.sh M src/kudu/client/symbols.map M src/kudu/integration-tests/CMakeLists.txt M src/kudu/integration-tests/external_mini_cluster.cc M src/kudu/integration-tests/external_mini_cluster.h A src/kudu/integration-tests/minidump_generation-itest.cc M src/kudu/master/mini_master.cc M src/kudu/server/server_base.cc M src/kudu/server/server_base.h M src/kudu/tserver/mini_tablet_server.cc M src/kudu/util/CMakeLists.txt M src/kudu/util/debug-util-test.cc M src/kudu/util/env_posix.cc M src/kudu/util/logging.cc A src/kudu/util/minidump-test.cc A src/kudu/util/minidump.cc A src/kudu/util/minidump.h M src/kudu/util/subprocess-test.cc M src/kudu/util/subprocess.cc M src/kudu/util/test_main.cc M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh M thirdparty/preflight.py A thirdparty/scripts/make-breakpad-src-archive.sh M thirdparty/vars.sh 30 files changed, 1,012 insertions(+), 84 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/20/5620/27 -- To view, visit http://gerrit.cloudera.org:8080/5620 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I495695cc38b75377f20b0497093a81ed5baa887f Gerrit-PatchSet: 27 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike PercyGerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Dinesh Bhat Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon