Re: Monitoring Use Cases for Kudu

Hi Senthil, Kudu masters and tservers expose internal metrics in JSON format at the /metrics endpoint of the web UI. You could start by scraping this endpoint every 10 seconds or so, and recording that. I think that's more or less how Cloudera Manager exposes Kudu metrics. Our docs

Re: TokenPB contents

// presence of unknown fields. >> optional bytes token_contents = 2; >> >> // The cryptographic signature of 'token_contents'. >> optional bytes signature = 3; >> >> // The sequence number of the key which produced 'signature'. >> optional int64 s

Re: TokenPB contents

ng, but that's a good reason not to do it that way. Our normal required feature flags mechanism should work in that case, where the field is not critical/required. > > > Best regards, > > Alexey > > On Wed, Jan 25, 2017 at 3:14 PM, Dan Burkert <danburk...@apache.org>

Re: TokenPB contents

ional bytes token_contents = 2; > > // The cryptographic signature of 'token_contents'. > optional bytes signature = 3; > > // The sequence number of the key which produced 'signature'. > optional int64 signing_key_seq_num = 4; > }; > > This looks great to me. > &g

Re: TokenPB contents

I think it must go in the 'token_contents' itself, otherwise it can be modified by a malicious client. Other than that, looks good. - Dan On Wed, Jan 25, 2017 at 12:37 PM, Todd Lipcon wrote: > Hey folks > > I'm working on the token signing/verification stuff at the moment.

Re: Proposal for 1.3 branch and release

+1 SGTM, thanks! - Dan On Tue, Feb 21, 2017 at 11:19 AM, Hao Hao wrote: > +1 Thanks a lot Todd for putting this together and volunteered to be the > RM. > > Best, > Hao > > On Tue, Feb 21, 2017 at 10:40 AM, Dave Wang wrote: > > > Strong +1 on your

Re: TLS for localhost connections

A couple thoughts: * You had to explicitly turn on the ADH and AECDH ciphers, right? We shouldn't be using those in any circumstances, but I don't think it would change the results of your test. * We've discussed adding checksums to the RPC system in the past, but punted since we would get it

Re: TLS for localhost connections

On Fri, Feb 10, 2017 at 10:02 AM, Todd Lipcon wrote: > > Yea, I figured that both sides would check the remote peer, and if they > both agree that the other side is local, they'd offer TLS_NEGOTIATION_ONLY > or somesuch. This could also be used in a scenario where a user needs

Re: [VOTE] Apache Kudu 1.2.0 RC1

Little bit late to the party, but release LGTM. Compiled and passed all tests on OS X 10.10. - Dan On Wed, Jan 18, 2017 at 8:48 AM, William Berkeley wrote: > Yeah I'm ok with passing it. Just a thought since it would be a quick fix. > > -Will > > On Wed, Jan 18, 2017 at

Re: Use of boost non-header-only functionality

I think for a long time we resisted vendoring boost, and so avoiding the dynamically linked parts of boost was advantageous to avoid incompatibilities in system boost versions. I recall we are vendoring boost now, so that should be less of a concern, I would think. - Dan On Thu, Jan 19, 2017 at

Re: Use of boost non-header-only functionality

Is there an equivalent library for the JVM? - Dan On Thu, Jan 19, 2017 at 4:33 PM, Dan Burkert <danburk...@apache.org> wrote: > I think for a long time we resisted vendoring boost, and so avoiding the > dynamically linked parts of boost was advantageous to avoid > incompatibil

Re: Deploying to the ASF maven repo

Both the staging and nightly repos look good to me, I was able to use them from a thirdparty maven project. apache.snapshots Apache Snapshot Repository https://repository.apache.org/snapshots apache.kudu.staging Apache Kudu Staging Repository

Re: [ANNOUNCE] Two new Kudu committer/PMC members

Congrats! On Mon, Sep 12, 2016 at 4:27 PM, Jordan Birdsell wrote: > Congrats > > On Mon, Sep 12, 2016 at 7:09 PM Brock Noland wrote: > >> Congratulations!! >> >> On Mon, Sep 12, 2016 at 6:06 PM, David Alves >> wrote: >> >>>

[VOTE] Apache Kudu 1.0.1 RC1

Hi, We're happy to announce the first release candidate for Apache Kudu 1.0.1. This release includes bug fixes and documentation updates since the 1.0.0 release. The is a source-only release. The artifacts were staged here: https://dist.apache.org/repos/dist/dev/kudu/1.0.1-RC1/ It was built

Re: Kudu Build: Unsupported options error

Valencia, If these are just issues with building libstdc++ in thirdparty, then the effect on Kudu should be negligible. libstdc++ is *only* used (linked-to) when building with thread sanitizer (TSAN) support. Right now we unconditionally build thirdparty libraries for TSAN when compiling on

Re: Kudu 1.0.1 patch release

> https://gerrit.cloudera.org/#/c/4535/ >> >> This fixes a crash when running on single-core systems. Again not too >> common (even most VMs have two cores these days) but it can be a blocker if >> you need to run on a single-core. >> >> -Todd >> >> On

Re: Fault tolerant scanner returning out-of-order partitions

The first three rows (including the out of order row) all fall in the same range partition, so the issue is likely that the intra-tablet scan returned out of order results (as opposed to the client scanning tablets out of order). I'm under the same impression about SetFaultTolerant(), which is why

Re: [VOTE] Apache Kudu 1.0.1 RC1

ral million of rows and it passed. > > > Best regards, > > Alexey > > > On Fri, Oct 7, 2016 at 2:00 PM, Dan Burkert <danburk...@apache.org> wrote: > > > Hi, > > > > We're happy to announce the first release candidate for Apache Kudu > 1

Heads Up: new (test-only) system dependencies

Hi all, As you can see in the recent commit history, work has gotten started on adding security features to Kudu. As part of this, we are now relying on MIT Kerberos (krb5), OpenSSL, and the GSSAPI SASL plugin. Since these are security-sensitive dependencies, we are relying on the system to

Re: Patches that need release notes for Kudu 1.1.0

'pruning fails with IS NOT NULL' and 'DNS resolves killing tservers' were both fixed in 1.0.1. I'll write up notes for IN LIST. - Dan On Tue, Nov 8, 2016 at 9:58 AM, Jean-Daniel Cryans wrote: > Hey devs, > > I'm doing a pass over the stuff committed since 1.0.1 >

Re: Plan for upcoming releases

Sounds good to me, thanks for volunteering. - Dan On Mon, Oct 17, 2016 at 5:27 PM, Jordan Birdsell wrote: > Seems reasonable to me, have a few things for Python left I'd like to get > in for 1.1: range partition implementation and alter table capabilities, > that

Re: [VOTE] Apache Kudu 1.1.0 RC1

+1 Built and tested on OS X 10.10 in debug mode, and checked hashes and signature. A few of the tests failed, but these are expected for this release (and have already been fixed on master): * client_samples-test * external_mini_cluster-test * sasl_rpc-test * pstack_watcher-test sasl_rpc-test

Re: checkstyle on console during precommit builds

I'm looking into this a bit. It does get disabled on generated sources when I run it locally, and there is a line in the pom.xml calling that out. I haven't been able to pinpoint what is different on the build slaves yet. The

Re: checkstyle on console during precommit builds

.org> wrote: > > > > I am familiar with checkstyle on Maven and could potentially take a look > at > > this, but it won't be before Tuesday. > > > > Mike > > > > On Wed, Nov 23, 2016 at 8:08 PM, Dan Burkert <d...@cloudera.com> wrote: >

Re: [VOTE] Apache Kudu 1.1.0 RC1

n bug a regression or also present in 1.0? If it's not > a regression i think it's reasonable to continue with the release and try > to fix in a 1.1.1 along with a couple other late discovered bugs. > > Todd > > On Nov 17, 2016 6:50 PM, "Dan Burkert" <danburk...@apache.org&

Re: [VOTE] Apache Kudu 1.1.0 RC1

Dimitris Tsirogiannis just uncovered a bug that allows unintended range partitions to be dropped: KUDU-1750 . The bug allows a range partition to be dropped if either the lower or upper bound range partition key matches, instead of requiring both

Re: [VOTE] Apache Kudu 1.1.0 RC1

robably by mid-afternoon tomorrow PST. > > -Todd > > On Thu, Nov 17, 2016 at 8:26 PM, Jordan Birdsell < > jordantbirds...@gmail.com> > wrote: > > > Agreed with proceeding and doing a 1.1.1 release. > > > > On Thu, Nov 17, 2016, 5:11 PM Dan Burkert &l

Re: 1.1.0 rc coming today

Sounds good. On Mon, Nov 14, 2016 at 3:09 PM, Jordan Birdsell wrote: > Sounds reasonable to me > > On Mon, Nov 14, 2016, 5:06 PM Todd Lipcon wrote: > > > Hey folks, > > > > I'm wrapping up the creation of 1.1.0 RC0 in the next few minutes. I > >

[RESULT] [VOTE] Apache Kudu 1.0.1 RC1

tes over 100K rows, looks good. > > J-D > > On Fri, Oct 7, 2016 at 2:00 PM, Dan Burkert <danburk...@apache.org> wrote: > > > Hi, > > > > We're happy to announce the first release candidate for Apache Kudu > 1.0.1. > > > > T

Re: initial draft of security scoping document/roadmap

gt; > On Mon, Oct 17, 2016 at 6:48 PM, William Berkeley <wdberke...@cloudera.com > > > wrote: > > > Fixed link: > > https://docs.google.com/document/d/1Yu4iuIhaERwug1vS95yWDd_ > > WzrNRIKvvVGUb31y-_mY/edit# > > > > -Will > > > > On

Multi-word Flag Style

Hi all, As of Kudu 1.3, multi-word flags can use a dash '-' separator in lieu of the underscore '_' separator. For example, --memory_limit_hard_bytes can now be specified as --memory-limit-hard-bytes, or even --memory_limit-hard_bytes. Of the people I've talked to, most seem to prefer dashes

Re: Multi-word Flag Style

nd regards, > > > > Alexey > > > > > > > > On 4/10/17 10:42 AM, William Berkeley wrote: > >> > >> I agree, for the reason you gave: dashes are the norm in Unix, so they > >> "feel right" for flag names. > >> > >> -

Re: [VOTE] Apache Kudu 1.3.0 RC1

+0 - env_util-test.cc fails to build on OS X 10.10 (fixed on master in 97831ead ) - external_mini_cluster-test and security-itest fail on OS X 10.10 (fixed on master in c5ec0ddb0

Re: [VOTE] Apache Kudu 1.3.0 RC1

t test code, but AFAIK we have yet to ship a > > release whose default build configuration (i.e. without NO_TESTS=1) > > fails to build on a commonly used platform, and I'd like to avoid > > setting that precedent. Between this and the macOS build breakages Dan > > pointed out, I

Re: [VOTE] Apache Kudu 1.3.1 RC1

+1 Built and ran tests on OS X 10.10. delete_tablet-itest fails, but it appears to fail on master as well, all they way back to when it was introduced about a month ago. Not a blocker IMO. - Dan On Mon, Apr 17, 2017 at 4:32 PM, Todd Lipcon wrote: > +1 > > Built the

Re: Java 7 and Spark Support

I'll preface my response by saying I don't think there are any hard and fast rules here, but I'd like us to try and continue following SemVer rules as much as possible. On Tue, Aug 15, 2017 at 2:03 PM, Grant Henke wrote: > > >- Should/can we drop Spark 1 support in the

Re: Preconditions vs assert in the Kudu Java client

My general rule of thumb is that I use 'assert' whenever I'm claiming that something must be true given static (non-runtime) knowledge local to the context (usually a method or class). If the assert fails, it indicates that there is irrefutably a bug in that context. I use the Preconditions

Re: Preconditions vs assert in the Kudu Java client

situations, but which fail catastrophically and unpredictably due to failure to handle black-swan error cases. - Dan On Tue, Jul 11, 2017 at 1:24 PM, Dan Burkert <danburk...@apache.org> wrote: > My general rule of thumb is that I use 'assert' whenever I'm claiming that > something

Re: Multi-word Flag Style

Apr 10, 2017 at 1:24 PM, Alexey Serbin <aser...@cloudera.com> wrote: > Oops, sure -- we cannot use dashes when referring to the variables in our > C++ code. > > > On 4/10/17 12:40 PM, Dan Burkert wrote: > >> On Mon, Apr 10, 2017 at 11:56 AM, Adar Dembo <a...@cloude

Re: [VOTE] Apache Kudu 1.4.0 RC2

+1 Built and ran tests on OS X 10.10. - Dan On Mon, Jun 12, 2017 at 9:49 AM, Brock Noland wrote: > +1 (non-binding) > > Built and ran tests > > On Mon, Jun 12, 2017 at 9:49 AM, Jean-Daniel Cryans > wrote: > > Err sorry I meant to vote on Friday but then

Re: RFC: Canceling RPC in mid-transmission

I'm still hung up on the premise that if the network is down, cancellation should still be expected to complete in bounded time. If the network is partitioned or congested to the point of non-delivery, how is the coordinator going to deliver the cancellation messages to the fragments in the first

Re: [VOTE] Apache Kudu 1.5.0 RC3

+1 Built and ran tests on macos, no unexpected failures. - Dan On Thu, Sep 7, 2017 at 8:24 PM, Jean-Daniel Cryans <jdcry...@apache.org> wrote: > +1 > > Built on CentOS 7.3.1611 in release mode, ran all unit tests, also ran the > Java unit tests. > > J-D > > On Tue

[VOTE] Apache Kudu 1.5.0 RC3

Hi, The Apache Kudu team is happy to announce the third release candidate for Apache Kudu 1.5.0. Apache Kudu 1.5.0 is a minor release which offers many improvements and fixes since the prior release. The is a source-only release. The artifacts are staged here:

Re: [VOTE] Apache Kudu 1.5.0 RC2

> > > > > --tablet_copy_idle_timeout_ms). > > > > > Apologize for any inconvenience! > > > > > > > > > > Best, > > > > > Hao > > > > > > > > > > On Fri, Sep 1, 2017 at 6:44 PM, Mike Percy <mpe...@apache.org&g

Upcoming 1.5 release

Hey everyone, I think it's time to get the ball rolling on a 1.5 release. It's been a bit over 2 months since 1.4, and we've had ~250 commits to master since then. I volunteer to RM, unless someone else wants the honor. I propose we branch 1.5 this coming Monday, August 28th, with the aim to

Re: IWYU configuration for gerrit pre-commit checks

Thanks, Alexey! - Dan On Thu, Aug 24, 2017 at 4:46 PM, Mike Percy wrote: > Thanks for setting this up, Alexey! > > Mike > > On Thu, Aug 24, 2017 at 11:18 AM, Alexey Serbin > wrote: > > > Hi, > > > > Today I enabled the IWYU (include-what-you-use)

[VOTE] Apache Kudu 1.5.0 RC1

Hi, The Apache Kudu team is happy to announce the second release candidate for Apache Kudu 1.5.0. Apache Kudu 1.5.0 is a minor release which offers many improvements and fixes since the prior release. The is a source-only release. The artifacts are staged here:

Re: [VOTE] Apache Kudu 1.5.0 RC1

from > elsewhere. We need to check that. If the former is the case, we simply > change to the proper header. If the latter is the case, the files need to > be added to the LICENSE.txt file. > > Mike > > [1] https://www.apache.org/legal/src-headers.html > > On Fri, Sep 1, 2017 at 9

Re: INT128 Column Support Interest

I think it would be useful. As far as I've seen the main costs in carrying data types are in writing performant encoders, and updating integrations to work with them. I'm guessing with 128 bit integers there would be some integrations that can't or won't support it, which might be a cause for

Re: [VOTE] Apache Kudu 1.6.0 RC1

+1 built and ran tests on macos. - Dan On Tue, Dec 5, 2017 at 7:06 PM, Mike Percy wrote: > +1 > > I built 1.6.0-RC1 on RHEL 7 in RELEASE mode. > > - Checksums and sigs match > - README.txt and LICENSE.txt look good > - RAT check passed > - All tests passed except for one

Re: [VOTE] Apache Kudu 1.7.1 RC2

+1 Built on macOS 10.12, all tests passed. - Dan On Tue, Jun 5, 2018 at 5:21 PM, William Berkeley wrote: > +1 > > Built on el7. > Ran tests. Passed. > Put on a 4-node cluster with some existing data written by 1.5. Was able to > scan the data back (same results as before). > > -Will > > On

[ANNOUNCE] Recognizing the newest Apache Kudu committers

Hi all, I'm pleased to announce that the Kudu PMC has voted to add Attila Bukor and Sailesh Mukil as committers and PMC members. Attila has contributed many supportability, build, docs, and quality of life improvements. In addition, Attila has been very active helping users on our Slack and

KUDU-428 Sentry Integration Design Doc

Hi Kudu developers, I've drafted a design doc for KUDU-428 Sentry Integration [1]. This is the next major step under the security umbrella. The high level goal is table and column level access control and authorization in Kudu by integrating with Apache Sentry. Please take a look at the doc --

Re: Adding examples

Sounds great to me. Having examples in the repository proper means they can be hooked into the build, and prevented from going stale. Having that kind of integration with CI is critical to making sure things don't bit-rot. - Dan On Tue, Mar 27, 2018 at 2:45 PM, William Berkeley

Re: [VOTE] Apache Kudu 1.8.0-RC2

+1 Built and ran C++ tests on MacOS. Had three failures (fs_manager-test, kudu-tool-test, and trace-test), but I believe they are all MacOS-specific, and shouldn't be release blockers. - Dan On Fri, Oct 19, 2018 at 11:48 PM Andrew Wong wrote: > +1 > > Built and ran all C++ tests on Centos

Re: Proposal: Gradle as the primary build tool

Sounds great, thanks for chasing this down! - Dan On Wed, Oct 3, 2018 at 7:20 PM Grant Henke wrote: > Hi Kudu dev community, > > I wanted to provide a progress and plan update to switching over to Gradle > as the primary build tool. > > First, the following steps have already been completed: >

Re: Kudu 1.8.0 Release

Sounds good to me. - Dan On Fri, Sep 21, 2018 at 11:14 AM Andrew Wong wrote: > I've been discussing with members of the dev community and I think it makes > sense to push out branching, to let some in-progress work finish and bake, > and to let some devs come back from time off. So I'm now