[jira] [Comment Edited] (KUDU-2427) Add support for Ubuntu 18.04
[
https://issues.apache.org/jira/browse/KUDU-2427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868166#comment-16868166
]
Alexey Serbin edited comment on KUDU-2427 at 6/20/19 1:53 AM:
--
Many tests fail if running Kudu at Ubuntu18.04LTS with the following
configuration (at least if built with third-party clang6.0):
{noformat}
# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.1 LTS"
{noformat}
{noformat}
# openssl version
OpenSSL 1.1.1 11 Sep 2018
{noformat}
{noformat}
# apt show libc6:amd64
Package: libc6
Version: 2.27-3ubuntu1
Priority: required
Section: libs
Source: glibc
Origin: Ubuntu
Maintainer: Ubuntu Developers
Original-Maintainer: GNU Libc Maintainers
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 12.2 MB
Depends: libgcc1
Suggests: glibc-doc, debconf | debconf-2.0, locales
Conflicts: openrc (<< 0.27-2~)
Breaks: hurd (<< 1:0.5.git20140203-1), libtirpc1 (<< 0.2.3), locales (<< 2.27),
locales-all (<< 2.27), nscd (<< 2.27)
Replaces: libc6-amd64
Homepage: https://www.gnu.org/software/libc/libc.html
Task: minimal
Supported: 5y
Download-Size: 2824 kB
APT-Manual-Installed: yes
APT-Sources: http://us-west-2.ec2.archive.ubuntu.com/ubuntu bionic/main amd64
Packages
Description: GNU C Library: Shared libraries
Contains the standard libraries that are used by nearly all programs on
the system. This package includes shared versions of the standard C library
and the standard math library, as well as many others.
{noformat}
{noformat}
# apt show libsasl2-2:amd64
Package: libsasl2-2
Version: 2.1.27~101-g0780600+dfsg-3ubuntu2
Priority: optional
Section: libs
Source: cyrus-sasl2
Origin: Ubuntu
Maintainer: Ubuntu Developers
Original-Maintainer: Debian Cyrus SASL Team
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 152 kB
Depends: libsasl2-modules-db (>= 2.1.27~101-g0780600+dfsg-3ubuntu2), libc6 (>=
2.15)
Recommends: libsasl2-modules (>= 2.1.27~101-g0780600+dfsg-3ubuntu2)
Breaks: postfix (<= 2.8.3-1), slapd (<= 2.4.25-3)
Replaces: libsasl2
Homepage: https://www.cyrusimap.org/
Task: ubuntu-desktop, mail-server, print-server, cloud-image, samba-server,
server, postgresql-server, lamp-server, kubuntu-desktop, xubuntu-core,
xubuntu-desktop, lubuntu-live-share, lubuntu-desktop-share,
lubuntu-gtk-desktop, lubuntu-desktop, lubuntu-live, lubuntu-qt-core,
lubuntu-qt-desktop, lubuntu-qt-desktop, lubuntu-live-qt, lubuntu-live-gtk,
ubuntustudio-desktop-core, ubuntustudio-desktop, ubuntukylin-desktop,
ubuntu-mate-core, ubuntu-mate-desktop, ubuntu-budgie-desktop
Supported: 5y
Download-Size: 49.2 kB
APT-Manual-Installed: yes
APT-Sources: http://us-west-2.ec2.archive.ubuntu.com/ubuntu bionic/main amd64
Packages
{noformat}
For example, the {{negotiation-test}} crashes with the following stack:
{noformat}
[ RUN ] NegotiationCombinations/TestNegotiation.TestNegotiation/10
Loading random data
Initializing database
'/tmp/kudutest-0/negotiation-test.NegotiationCombinations_TestNegotiation.Test
Negotiation_10.1560993441102421-13933/krb5kdc/principal' for realm
'KRBTEST.COM',
master key name 'K/m...@krbtest.com'
Jun 19 18:17:23 xxx.xxx.com krb5kdc[14040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:17:23 xxx.xxx.com krb5kdc[14040](info): set up 2 sockets
Jun 19 18:17:23 xxx.xxx.com krb5kdc[14040](info): commencing operation
krb5kdc: starting...
Authenticating as principal root/ad...@krbtest.com with password.
WARNING: no policy specified for client-gss...@krbtest.com; defaulting to no
policy
Principal "client-gss...@krbtest.com" created.
Jun 19 18:17:23 xxx.xxx.com krb5kdc[14040](info): AS_REQ (2 etypes {17 16})
127.0.0
.1: ISSUE: authtime 1560993443, etypes {rep=17 tkt=17 ses=17},
client-gss...@krbtest.com for krbtgt/
krbtest@krbtest.com
Password for client-gss...@krbtest.com:
Authenticating as principal client-gssapi/ad...@krbtest.com with password.
WARNING: no policy specified for kudu/127.0@krbtest.com; defaulting to no
policy
Principal "kudu/127.0@krbtest.com" created.
Authenticating as principal client-gssapi/ad...@krbtest.com with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type
aes256-cts-hmac-sha1-96 added to key
tab
WRFILE:/tmp/kudutest-0/negotiation-test.NegotiationCombinations_TestNegotiation.TestNegotiation_
10.1560993441102421-13933/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type
aes128-cts-hmac-sha1-96 added to key
tab
WRFILE:/tmp/kudutest-0/negotiation-test.NegotiationCombinations_TestNegotiation.TestNegotiation_
10.1560993441102421-13933/krb5kdc/kudu_127.0.0.1.keytab.
F0619 18:17:23.980032 14049 sasl_helper.cc:119] Server: Illegal state: Received
illegal call-id during negotiation; expected: -33, received: -3
*** Check failure stack trace: ***
*** Aborted at
[jira] [Comment Edited] (KUDU-2427) Add support for Ubuntu 18.04
[
https://issues.apache.org/jira/browse/KUDU-2427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868166#comment-16868166
]
Alexey Serbin edited comment on KUDU-2427 at 6/20/19 1:37 AM:
--
Many tests fail if running Kudu at Ubuntu18.04LTS with the following
configuration (at least if built with third-party clang6.0):
{noformat}
# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.1 LTS"
{noformat}
{noformat}
# openssl version
OpenSSL 1.1.1 11 Sep 2018
{noformat}
{noformat}
# apt show libc6:amd64
Package: libc6
Version: 2.27-3ubuntu1
Priority: required
Section: libs
Source: glibc
Origin: Ubuntu
Maintainer: Ubuntu Developers
Original-Maintainer: GNU Libc Maintainers
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 12.2 MB
Depends: libgcc1
Suggests: glibc-doc, debconf | debconf-2.0, locales
Conflicts: openrc (<< 0.27-2~)
Breaks: hurd (<< 1:0.5.git20140203-1), libtirpc1 (<< 0.2.3), locales (<< 2.27),
locales-all (<< 2.27), nscd (<< 2.27)
Replaces: libc6-amd64
Homepage: https://www.gnu.org/software/libc/libc.html
Task: minimal
Supported: 5y
Download-Size: 2824 kB
APT-Manual-Installed: yes
APT-Sources: http://us-west-2.ec2.archive.ubuntu.com/ubuntu bionic/main amd64
Packages
Description: GNU C Library: Shared libraries
Contains the standard libraries that are used by nearly all programs on
the system. This package includes shared versions of the standard C library
and the standard math library, as well as many others.
{noformat}
For example, the {{negotiation-test}} crashes with the following stack:
{noformat}
[ RUN ] NegotiationCombinations/TestNegotiation.TestNegotiation/10
Loading random data
Initializing database
'/tmp/kudutest-0/negotiation-test.NegotiationCombinations_TestNegotiation.Test
Negotiation_10.1560993441102421-13933/krb5kdc/principal' for realm
'KRBTEST.COM',
master key name 'K/m...@krbtest.com'
Jun 19 18:17:23 xxx.xxx.com krb5kdc[14040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:17:23 xxx.xxx.com krb5kdc[14040](info): set up 2 sockets
Jun 19 18:17:23 xxx.xxx.com krb5kdc[14040](info): commencing operation
krb5kdc: starting...
Authenticating as principal root/ad...@krbtest.com with password.
WARNING: no policy specified for client-gss...@krbtest.com; defaulting to no
policy
Principal "client-gss...@krbtest.com" created.
Jun 19 18:17:23 xxx.xxx.com krb5kdc[14040](info): AS_REQ (2 etypes {17 16})
127.0.0
.1: ISSUE: authtime 1560993443, etypes {rep=17 tkt=17 ses=17},
client-gss...@krbtest.com for krbtgt/
krbtest@krbtest.com
Password for client-gss...@krbtest.com:
Authenticating as principal client-gssapi/ad...@krbtest.com with password.
WARNING: no policy specified for kudu/127.0@krbtest.com; defaulting to no
policy
Principal "kudu/127.0@krbtest.com" created.
Authenticating as principal client-gssapi/ad...@krbtest.com with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type
aes256-cts-hmac-sha1-96 added to key
tab
WRFILE:/tmp/kudutest-0/negotiation-test.NegotiationCombinations_TestNegotiation.TestNegotiation_
10.1560993441102421-13933/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type
aes128-cts-hmac-sha1-96 added to key
tab
WRFILE:/tmp/kudutest-0/negotiation-test.NegotiationCombinations_TestNegotiation.TestNegotiation_
10.1560993441102421-13933/krb5kdc/kudu_127.0.0.1.keytab.
F0619 18:17:23.980032 14049 sasl_helper.cc:119] Server: Illegal state: Received
illegal call-id during negotiation; expected: -33, received: -3
*** Check failure stack trace: ***
*** Aborted at 1560993443 (unix time) try "date -d @1560993443" if you are
using GNU date ***
PC: @ 0x7ff0be0cbe97 gsignal
*** SIGABRT (@0x366d) received by PID 13933 (TID 0x7ff0bae1d700) from PID
13933; stack trace: ***
@ 0x7ff0be490890 (unknown)
@ 0x7ff0be0cbe97 gsignal
@ 0x7ff0be0cd801 abort
@ 0x7ff0c0357d59 google::logging_fail()
@ 0x7ff0c0359d6d google::LogMessage::Fail()
@ 0x7ff0c035bd44 google::LogMessage::SendToLog()
@ 0x7ff0c035988d google::LogMessage::Flush()
@ 0x7ff0c0359ae1 google::LogMessage::~LogMessage()
@ 0x7ff0c2c11878 kudu::rpc::SaslHelper::CheckNegotiateCallId()
@ 0x7ff0c2c163ff kudu::rpc::ServerNegotiation::RecvNegotiatePB()
@ 0x7ff0c2c152c9 kudu::rpc::ServerNegotiation::Negotiate()
@ 0x459975
kudu::rpc::TestNegotiation_TestNegotiation_Test::TestBody()::$_10::operator()()
@ 0x4598bd
_ZSt13__invoke_implIvZN4kudu3rpc36TestNegotiation_TestNegotiation_Test8TestBodyEvE4$_10JEET_St14__invoke_otherOT0_DpOT1_
@ 0x45984d
_ZSt8__invokeIZN4kudu3rpc36TestNegotiation_TestNegotiation_Test8TestBodyEvE4$_10JEENSt15__invoke_resultIT_JDpT0_EE4typeEOS5_DpOS6_
@ 0x459825
[jira] [Reopened] (KUDU-2427) Add support for Ubuntu 18.04
[
https://issues.apache.org/jira/browse/KUDU-2427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Serbin reopened KUDU-2427:
-
Many tests fail if running Kudu at Ubuntu18.04LTS with the following
configuration (at least if built with third-party clang6.0):
{noformat}
# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.1 LTS"
{noformat}
{noformat}
# openssl version
OpenSSL 1.1.1 11 Sep 2018
{noformat}
{noformat}
root@aserbin-u18:~/Projects/kudu/build/master.debug# apt show libc6:amd64
Package: libc6
Version: 2.27-3ubuntu1
Priority: required
Section: libs
Source: glibc
Origin: Ubuntu
Maintainer: Ubuntu Developers
Original-Maintainer: GNU Libc Maintainers
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 12.2 MB
Depends: libgcc1
Suggests: glibc-doc, debconf | debconf-2.0, locales
Conflicts: openrc (<< 0.27-2~)
Breaks: hurd (<< 1:0.5.git20140203-1), libtirpc1 (<< 0.2.3), locales (<< 2.27),
locales-all (<< 2.27), nscd (<< 2.27)
Replaces: libc6-amd64
Homepage: https://www.gnu.org/software/libc/libc.html
Task: minimal
Supported: 5y
Download-Size: 2824 kB
APT-Manual-Installed: yes
APT-Sources: http://us-west-2.ec2.archive.ubuntu.com/ubuntu bionic/main amd64
Packages
Description: GNU C Library: Shared libraries
Contains the standard libraries that are used by nearly all programs on
the system. This package includes shared versions of the standard C library
and the standard math library, as well as many others.
{noformat}
For example, the {{negotiation-test}} crashes with the following stack:
{noformat}
[ RUN ] NegotiationCombinations/TestNegotiation.TestNegotiation/10
Loading random data
Initializing database
'/tmp/kudutest-0/negotiation-test.NegotiationCombinations_TestNegotiation.Test
Negotiation_10.1560993441102421-13933/krb5kdc/principal' for realm
'KRBTEST.COM',
master key name 'K/m...@krbtest.com'
Jun 19 18:17:23 aserbin-u18.vpc.cloudera.com krb5kdc[14040](info): setting up
network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:17:23 aserbin-u18.vpc.cloudera.com krb5kdc[14040](info): set up 2
sockets
Jun 19 18:17:23 aserbin-u18.vpc.cloudera.com krb5kdc[14040](info): commencing
operation
krb5kdc: starting...
Authenticating as principal root/ad...@krbtest.com with password.
WARNING: no policy specified for client-gss...@krbtest.com; defaulting to no
policy
Principal "client-gss...@krbtest.com" created.
Jun 19 18:17:23 aserbin-u18.vpc.cloudera.com krb5kdc[14040](info): AS_REQ (2
etypes {17 16}) 127.0.0
.1: ISSUE: authtime 1560993443, etypes {rep=17 tkt=17 ses=17},
client-gss...@krbtest.com for krbtgt/
krbtest@krbtest.com
Password for client-gss...@krbtest.com:
Authenticating as principal client-gssapi/ad...@krbtest.com with password.
WARNING: no policy specified for kudu/127.0@krbtest.com; defaulting to no
policy
Principal "kudu/127.0@krbtest.com" created.
Authenticating as principal client-gssapi/ad...@krbtest.com with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type
aes256-cts-hmac-sha1-96 added to key
tab
WRFILE:/tmp/kudutest-0/negotiation-test.NegotiationCombinations_TestNegotiation.TestNegotiation_
10.1560993441102421-13933/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type
aes128-cts-hmac-sha1-96 added to key
tab
WRFILE:/tmp/kudutest-0/negotiation-test.NegotiationCombinations_TestNegotiation.TestNegotiation_
10.1560993441102421-13933/krb5kdc/kudu_127.0.0.1.keytab.
F0619 18:17:23.980032 14049 sasl_helper.cc:119] Server: Illegal state: Received
illegal call-id during negotiation; expected: -33, received: -3
*** Check failure stack trace: ***
*** Aborted at 1560993443 (unix time) try "date -d @1560993443" if you are
using GNU date ***
PC: @ 0x7ff0be0cbe97 gsignal
*** SIGABRT (@0x366d) received by PID 13933 (TID 0x7ff0bae1d700) from PID
13933; stack trace: ***
@ 0x7ff0be490890 (unknown)
@ 0x7ff0be0cbe97 gsignal
@ 0x7ff0be0cd801 abort
@ 0x7ff0c0357d59 google::logging_fail()
@ 0x7ff0c0359d6d google::LogMessage::Fail()
@ 0x7ff0c035bd44 google::LogMessage::SendToLog()
@ 0x7ff0c035988d google::LogMessage::Flush()
@ 0x7ff0c0359ae1 google::LogMessage::~LogMessage()
@ 0x7ff0c2c11878 kudu::rpc::SaslHelper::CheckNegotiateCallId()
@ 0x7ff0c2c163ff kudu::rpc::ServerNegotiation::RecvNegotiatePB()
@ 0x7ff0c2c152c9 kudu::rpc::ServerNegotiation::Negotiate()
@ 0x459975
kudu::rpc::TestNegotiation_TestNegotiation_Test::TestBody()::$_10::operator()()
@ 0x4598bd
_ZSt13__invoke_implIvZN4kudu3rpc36TestNegotiation_TestNegotiation_Test8TestBodyEvE4$_10JEET_St14__invoke_otherOT0_DpOT1_
@ 0x45984d
[jira] [Updated] (KUDU-2870) Checksum scan fails with "Not authorized" error when authz enabled
[
https://issues.apache.org/jira/browse/KUDU-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wong updated KUDU-2870:
--
Code Review: https://gerrit.cloudera.org/c/13681/
> Checksum scan fails with "Not authorized" error when authz enabled
> --
>
> Key: KUDU-2870
> URL: https://issues.apache.org/jira/browse/KUDU-2870
> Project: Kudu
> Issue Type: Bug
> Components: authz
>Affects Versions: 1.10.0
>Reporter: Mike Percy
>Assignee: Andrew Wong
>Priority: Critical
>
> While testing a Kudu 1.10.0 RC build with authorization enabled, I tried a
> checksum scan and it failed:
> {code:java}
> [mpercy@mpercy-c63s-0619-1 ~]$ kudu cluster ksck
> mpercy-c63s-0619-1.vpc.cloudera.com
> -tables=default.loadgen_auto_b527de07b2d842f3a3c82c5f85eb2854 -checksum_scan
> -sections=CHECKSUM_RESULTS
> Checksum finished in 0s: 0/8 replicas remaining (0B from disk, 0 rows summed)
> Checksum Summary
> ---
> default.loadgen_auto_b527de07b2d842f3a3c82c5f85eb2854
> ---
> T 09d0df0ca48c41bf94c6a3a03533b811 P 7d31913f6bbf4355a974c76e4f82c72a
> (mpercy-c63s-0619-4.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T 16cafc1e2e814b5fb988b22554ac306b P 11edb01b3b184a2da8586fa5cffda90c
> (mpercy-c63s-0619-3.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T 37d40c90b0614b5d9515d1458e31657c P de47be31840f4b349f970cf759097cec
> (mpercy-c63s-0619-2.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T 5da264a95d31474ea4b0b2e464a5b261 P de47be31840f4b349f970cf759097cec
> (mpercy-c63s-0619-2.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T 6acad06c945942b5af696f7f59b4d2ea P 7d31913f6bbf4355a974c76e4f82c72a
> (mpercy-c63s-0619-4.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T 949f20e82db1467fa9f968853c901f11 P 11edb01b3b184a2da8586fa5cffda90c
> (mpercy-c63s-0619-3.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T 9eda0d8c267c44efb9d76cf8fb911f93 P 921f4e7e28274a9189c978162d604f2e
> (mpercy-c63s-0619-5.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T a1ec4565447c4628baa6a1c5f9765c7a P 11edb01b3b184a2da8586fa5cffda90c
> (mpercy-c63s-0619-3.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> ==
> Warnings:
> ==
> Some masters have unsafe, experimental, or hidden flags set
> Some tablet servers have unsafe, experimental, or hidden flags set
> ==
> Errors:
> ==
> Aborted: checksum scan error: 8 errors were detected
> FAILED
> Runtime error: ksck discovered errors{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KUDU-2870) Checksum scan fails with "Not authorized" error when authz enabled
[
https://issues.apache.org/jira/browse/KUDU-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867967#comment-16867967
]
Andrew Wong commented on KUDU-2870:
---
This is an artifact of the fact that the ksck checksum scan doesn't use a
KuduClient to send its checksum requests. Instead, it uses a proxy directly.
There are a few things we could do to work around this:
# Rather than doing fine-grained authz checking for Checksum, require
coarse-grained super-user privileges via the --super-user-acl flag. While
somewhat complicating the mental model for authorization, this seems relatively
straight-forward to implement. Given Checksum doesn't seem to be exposed as a
part of any client's public API, this seem's like the easiest path forward.
# Refactor the ksck checksummer to collect tokens from any OpenTable calls it
makes before checksumming, and then passing in the appropriate authz tokens
along with the requests. Extra plumbing would be required to handle retriable
token-related errors (e.g. authz token expired).
# Refactor the ksck checksummer to use a real client instead of a proxy, so it
can leverage the client's handling of token-related errors. This seems like a
larger piece to chew than #2, but seems more correct.
# Introduce the concept of a --trusted-user-acl to the tablet servers to
forego all fine-grained privilege checking, and perhaps add the
--super-user-acl to the --trusted-user-acl. In this way, tooling would be able
to do anything without relying on authz tokens. This may be desirable in
reasoning about the internals of authorization, since it standardizes semantics
for trusted-user-acls more than it is today (i.e. today, being a part of the
--trusted-user-acl means the Master will return to the user with an authz token
containing all privileges).
For now, I will implement #1 given it seems easy to implement and we can think
about the others moving forward.
> Checksum scan fails with "Not authorized" error when authz enabled
> --
>
> Key: KUDU-2870
> URL: https://issues.apache.org/jira/browse/KUDU-2870
> Project: Kudu
> Issue Type: Bug
> Components: authz
>Affects Versions: 1.10.0
>Reporter: Mike Percy
>Assignee: Andrew Wong
>Priority: Critical
>
> While testing a Kudu 1.10.0 RC build with authorization enabled, I tried a
> checksum scan and it failed:
> {code:java}
> [mpercy@mpercy-c63s-0619-1 ~]$ kudu cluster ksck
> mpercy-c63s-0619-1.vpc.cloudera.com
> -tables=default.loadgen_auto_b527de07b2d842f3a3c82c5f85eb2854 -checksum_scan
> -sections=CHECKSUM_RESULTS
> Checksum finished in 0s: 0/8 replicas remaining (0B from disk, 0 rows summed)
> Checksum Summary
> ---
> default.loadgen_auto_b527de07b2d842f3a3c82c5f85eb2854
> ---
> T 09d0df0ca48c41bf94c6a3a03533b811 P 7d31913f6bbf4355a974c76e4f82c72a
> (mpercy-c63s-0619-4.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T 16cafc1e2e814b5fb988b22554ac306b P 11edb01b3b184a2da8586fa5cffda90c
> (mpercy-c63s-0619-3.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T 37d40c90b0614b5d9515d1458e31657c P de47be31840f4b349f970cf759097cec
> (mpercy-c63s-0619-2.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T 5da264a95d31474ea4b0b2e464a5b261 P de47be31840f4b349f970cf759097cec
> (mpercy-c63s-0619-2.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T 6acad06c945942b5af696f7f59b4d2ea P 7d31913f6bbf4355a974c76e4f82c72a
> (mpercy-c63s-0619-4.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T 949f20e82db1467fa9f968853c901f11 P 11edb01b3b184a2da8586fa5cffda90c
> (mpercy-c63s-0619-3.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T 9eda0d8c267c44efb9d76cf8fb911f93 P 921f4e7e28274a9189c978162d604f2e
> (mpercy-c63s-0619-5.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T a1ec4565447c4628baa6a1c5f9765c7a P 11edb01b3b184a2da8586fa5cffda90c
> (mpercy-c63s-0619-3.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> ==
> Warnings:
> ==
> Some masters have unsafe, experimental, or hidden flags set
> Some tablet servers have unsafe, experimental, or hidden flags set
> ==
> Errors:
> ==
> Aborted: checksum scan error: 8 errors were detected
> FAILED
> Runtime error: ksck discovered errors{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KUDU-2870) Checksum scan fails with "Not authorized" error when authz enabled
[
https://issues.apache.org/jira/browse/KUDU-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wong reassigned KUDU-2870:
-
Assignee: Andrew Wong
> Checksum scan fails with "Not authorized" error when authz enabled
> --
>
> Key: KUDU-2870
> URL: https://issues.apache.org/jira/browse/KUDU-2870
> Project: Kudu
> Issue Type: Bug
> Components: authz
>Affects Versions: 1.10.0
>Reporter: Mike Percy
>Assignee: Andrew Wong
>Priority: Critical
>
> While testing a Kudu 1.10.0 RC build with authorization enabled, I tried a
> checksum scan and it failed:
> {code:java}
> [mpercy@mpercy-c63s-0619-1 ~]$ kudu cluster ksck
> mpercy-c63s-0619-1.vpc.cloudera.com
> -tables=default.loadgen_auto_b527de07b2d842f3a3c82c5f85eb2854 -checksum_scan
> -sections=CHECKSUM_RESULTS
> Checksum finished in 0s: 0/8 replicas remaining (0B from disk, 0 rows summed)
> Checksum Summary
> ---
> default.loadgen_auto_b527de07b2d842f3a3c82c5f85eb2854
> ---
> T 09d0df0ca48c41bf94c6a3a03533b811 P 7d31913f6bbf4355a974c76e4f82c72a
> (mpercy-c63s-0619-4.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T 16cafc1e2e814b5fb988b22554ac306b P 11edb01b3b184a2da8586fa5cffda90c
> (mpercy-c63s-0619-3.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T 37d40c90b0614b5d9515d1458e31657c P de47be31840f4b349f970cf759097cec
> (mpercy-c63s-0619-2.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T 5da264a95d31474ea4b0b2e464a5b261 P de47be31840f4b349f970cf759097cec
> (mpercy-c63s-0619-2.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T 6acad06c945942b5af696f7f59b4d2ea P 7d31913f6bbf4355a974c76e4f82c72a
> (mpercy-c63s-0619-4.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T 949f20e82db1467fa9f968853c901f11 P 11edb01b3b184a2da8586fa5cffda90c
> (mpercy-c63s-0619-3.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T 9eda0d8c267c44efb9d76cf8fb911f93 P 921f4e7e28274a9189c978162d604f2e
> (mpercy-c63s-0619-5.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> T a1ec4565447c4628baa6a1c5f9765c7a P 11edb01b3b184a2da8586fa5cffda90c
> (mpercy-c63s-0619-3.vpc.cloudera.com:7050): Error: Remote error: Not
> authorized: no authorization token presented
> ==
> Warnings:
> ==
> Some masters have unsafe, experimental, or hidden flags set
> Some tablet servers have unsafe, experimental, or hidden flags set
> ==
> Errors:
> ==
> Aborted: checksum scan error: 8 errors were detected
> FAILED
> Runtime error: ksck discovered errors{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KUDU-2870) Checksum scan fails with "Not authorized" error when authz enabled
Mike Percy created KUDU-2870:
Summary: Checksum scan fails with "Not authorized" error when
authz enabled
Key: KUDU-2870
URL: https://issues.apache.org/jira/browse/KUDU-2870
Project: Kudu
Issue Type: Bug
Components: authz
Affects Versions: 1.10.0
Reporter: Mike Percy
While testing a Kudu 1.10.0 RC build with authorization enabled, I tried a
checksum scan and it failed:
{code:java}
[mpercy@mpercy-c63s-0619-1 ~]$ kudu cluster ksck
mpercy-c63s-0619-1.vpc.cloudera.com
-tables=default.loadgen_auto_b527de07b2d842f3a3c82c5f85eb2854 -checksum_scan
-sections=CHECKSUM_RESULTS
Checksum finished in 0s: 0/8 replicas remaining (0B from disk, 0 rows summed)
Checksum Summary
---
default.loadgen_auto_b527de07b2d842f3a3c82c5f85eb2854
---
T 09d0df0ca48c41bf94c6a3a03533b811 P 7d31913f6bbf4355a974c76e4f82c72a
(mpercy-c63s-0619-4.vpc.cloudera.com:7050): Error: Remote error: Not
authorized: no authorization token presented
T 16cafc1e2e814b5fb988b22554ac306b P 11edb01b3b184a2da8586fa5cffda90c
(mpercy-c63s-0619-3.vpc.cloudera.com:7050): Error: Remote error: Not
authorized: no authorization token presented
T 37d40c90b0614b5d9515d1458e31657c P de47be31840f4b349f970cf759097cec
(mpercy-c63s-0619-2.vpc.cloudera.com:7050): Error: Remote error: Not
authorized: no authorization token presented
T 5da264a95d31474ea4b0b2e464a5b261 P de47be31840f4b349f970cf759097cec
(mpercy-c63s-0619-2.vpc.cloudera.com:7050): Error: Remote error: Not
authorized: no authorization token presented
T 6acad06c945942b5af696f7f59b4d2ea P 7d31913f6bbf4355a974c76e4f82c72a
(mpercy-c63s-0619-4.vpc.cloudera.com:7050): Error: Remote error: Not
authorized: no authorization token presented
T 949f20e82db1467fa9f968853c901f11 P 11edb01b3b184a2da8586fa5cffda90c
(mpercy-c63s-0619-3.vpc.cloudera.com:7050): Error: Remote error: Not
authorized: no authorization token presented
T 9eda0d8c267c44efb9d76cf8fb911f93 P 921f4e7e28274a9189c978162d604f2e
(mpercy-c63s-0619-5.vpc.cloudera.com:7050): Error: Remote error: Not
authorized: no authorization token presented
T a1ec4565447c4628baa6a1c5f9765c7a P 11edb01b3b184a2da8586fa5cffda90c
(mpercy-c63s-0619-3.vpc.cloudera.com:7050): Error: Remote error: Not
authorized: no authorization token presented
==
Warnings:
==
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==
Errors:
==
Aborted: checksum scan error: 8 errors were detected
FAILED
Runtime error: ksck discovered errors{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KUDU-2851) Have table scan tool surface errors instead of crashing
[
https://issues.apache.org/jira/browse/KUDU-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wong reassigned KUDU-2851:
-
Assignee: Hannah Nguyen
> Have table scan tool surface errors instead of crashing
> ---
>
> Key: KUDU-2851
> URL: https://issues.apache.org/jira/browse/KUDU-2851
> Project: Kudu
> Issue Type: Improvement
> Components: CLI
>Affects Versions: 1.10.0
>Reporter: Andrew Wong
>Assignee: Hannah Nguyen
>Priority: Major
> Labels: newbie
>
> Currently the `kudu table scan` tool crashes if it hits an error in any of
> its scan threads. It'd be nicer usability-wise to instead collect the errors
> and display them at the end.
>
> {{$ kudu table scan awong-1.vpc.cloudera.com:7051
> --table-name=default.loadgen_auto}}
> {{F0425 20:07:08.283156 22044 table_scanner.cc:475] Check failed: _s.ok() Bad
> status: Remote error: Not authorized: not authorized to Scan}}
> {{*** Check failure stack trace: ***}}
> {{ @ 0x108a38d google::LogMessage::Fail()}}
> {{F0425 20:07:08.285526 22043 table_scanner.cc:475] Check failed: _s.ok() Bad
> status: Remote error: Not authorized: not authorized to Scan}}
> {{*** Check failure stack trace: ***}}
> {{ @ 0x108c2e3 google::LogMessage::SendToLog()}}
> {{ @ 0x108a38d google::LogMessage::Fail()}}
> {{ @ 0x108c2e3 google::LogMessage::SendToLog()}}
> {{ @ 0x1089ee9 google::LogMessage::Flush()}}
> {{ @ 0x1089ee9 google::LogMessage::Flush()}}
> {{ @ 0x108cc6f google::LogMessageFatal::~LogMessageFatal()}}
> {{ @ 0x108cc6f google::LogMessageFatal::~LogMessageFatal()}}
> {{ @ 0x10d7807 kudu::tools::TableScanner::ScanData()}}
> {{ @ 0x10d7807 kudu::tools::TableScanner::ScanData()}}
> {{ @ 0x10d79c9 kudu::tools::TableScanner::ScanTask()}}
> {{ @ 0x10d79c9 kudu::tools::TableScanner::ScanTask()}}
> {{ @ 0xe754cd kudu::FunctionRunnable::Run()}}
> {{ @ 0xe754cd kudu::FunctionRunnable::Run()}}
> {{ @ 0xe724bf kudu::ThreadPool::DispatchThread()}}
> {{ @ 0xe724bf kudu::ThreadPool::DispatchThread()}}
> {{ @ 0xe69134 kudu::Thread::SuperviseThread()}}
> {{ @ 0x7ff76335add5 start_thread}}
> {{ @ 0x7ff761630ead __clone}}
> {{Aborted}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KUDU-2851) Have table scan tool surface errors instead of crashing
[
https://issues.apache.org/jira/browse/KUDU-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wong reassigned KUDU-2851:
-
Assignee: (was: Andrew Wong)
> Have table scan tool surface errors instead of crashing
> ---
>
> Key: KUDU-2851
> URL: https://issues.apache.org/jira/browse/KUDU-2851
> Project: Kudu
> Issue Type: Improvement
> Components: CLI
>Affects Versions: 1.10.0
>Reporter: Andrew Wong
>Priority: Major
> Labels: newbie
>
> Currently the `kudu table scan` tool crashes if it hits an error in any of
> its scan threads. It'd be nicer usability-wise to instead collect the errors
> and display them at the end.
>
> {{$ kudu table scan awong-1.vpc.cloudera.com:7051
> --table-name=default.loadgen_auto}}
> {{F0425 20:07:08.283156 22044 table_scanner.cc:475] Check failed: _s.ok() Bad
> status: Remote error: Not authorized: not authorized to Scan}}
> {{*** Check failure stack trace: ***}}
> {{ @ 0x108a38d google::LogMessage::Fail()}}
> {{F0425 20:07:08.285526 22043 table_scanner.cc:475] Check failed: _s.ok() Bad
> status: Remote error: Not authorized: not authorized to Scan}}
> {{*** Check failure stack trace: ***}}
> {{ @ 0x108c2e3 google::LogMessage::SendToLog()}}
> {{ @ 0x108a38d google::LogMessage::Fail()}}
> {{ @ 0x108c2e3 google::LogMessage::SendToLog()}}
> {{ @ 0x1089ee9 google::LogMessage::Flush()}}
> {{ @ 0x1089ee9 google::LogMessage::Flush()}}
> {{ @ 0x108cc6f google::LogMessageFatal::~LogMessageFatal()}}
> {{ @ 0x108cc6f google::LogMessageFatal::~LogMessageFatal()}}
> {{ @ 0x10d7807 kudu::tools::TableScanner::ScanData()}}
> {{ @ 0x10d7807 kudu::tools::TableScanner::ScanData()}}
> {{ @ 0x10d79c9 kudu::tools::TableScanner::ScanTask()}}
> {{ @ 0x10d79c9 kudu::tools::TableScanner::ScanTask()}}
> {{ @ 0xe754cd kudu::FunctionRunnable::Run()}}
> {{ @ 0xe754cd kudu::FunctionRunnable::Run()}}
> {{ @ 0xe724bf kudu::ThreadPool::DispatchThread()}}
> {{ @ 0xe724bf kudu::ThreadPool::DispatchThread()}}
> {{ @ 0xe69134 kudu::Thread::SuperviseThread()}}
> {{ @ 0x7ff76335add5 start_thread}}
> {{ @ 0x7ff761630ead __clone}}
> {{Aborted}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KUDU-2854) Short circuit predicates on dictionary-coded columns
[ https://issues.apache.org/jira/browse/KUDU-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867570#comment-16867570 ] ZhangYao commented on KUDU-2854: After reading related code as saying in Description, if no entries in the dictionary match the predicate, we can short circuit at a few layers. But currently we don't have a quick way to judge if there is any delta for the whole column(cfile) or the whole data block(part of cfile). Although base data's dictionary may not hit the predicate but after applying delta things may change. Cfile reads data batch by batch so we can only judge if there is any deltas for the batch and can short circuit the following data copy if on entries hit the predicate and no delta for this batch. This has been implemented in BinaryDictBlockDecoder::CopyNextAndEval. Is there any way we can easily judge if a column contain deltas or if a data block contain deltas?(?) > Short circuit predicates on dictionary-coded columns > > > Key: KUDU-2854 > URL: https://issues.apache.org/jira/browse/KUDU-2854 > Project: Kudu > Issue Type: Improvement > Components: cfile, perf, tserver >Reporter: Todd Lipcon >Priority: Major > > In the common case that a column has no updates in a given DRS, if we see > that no entries in the dictionary match the predicate, we can short circuit > at a few layers: > - we can store a flag in the cfile footer that indicates that all blocks are > dict-coded (ie there are no fallbacks). In that case, we can skip the whole > rowset > - if a cfile is partially dict-encoded, we can skip any dict-coded blocks > without decoding the dictionary words -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KUDU-2868) Support TTL in kudu
[ https://issues.apache.org/jira/browse/KUDU-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867554#comment-16867554 ] ZhangYao commented on KUDU-2868: Here is the design doc: [https://docs.google.com/document/d/1Eap2Hr1kp_Covg3qv64gTb46hSddyVvkzPC9rGlzZ5o/edit?usp=sharing] Looking forward to your suggestions . > Support TTL in kudu > --- > > Key: KUDU-2868 > URL: https://issues.apache.org/jira/browse/KUDU-2868 > Project: Kudu > Issue Type: New Feature >Reporter: ZhangYao >Priority: Major > Labels: roadmap-candidate > > In development scenario we usually want to use TTL feature to discard rows > that have not been updated for a while and kudu do not support that yet. > Currently kudu may can use range partition to drop partition out of date to > handle TTL style case but this can not completely cover our needs. Drop > partition can not handle partial updates such as when out of date partition > contains rows been updated recently it can't be dropped. Those long-term > non-updated rows are not needed but still occupying storage space. It's > meaningful for kudu to support row level TTL. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KUDU-2869) Internal compiler error compiling with devtoolset-7 on el7
Todd Lipcon created KUDU-2869:
-
Summary: Internal compiler error compiling with devtoolset-7 on el7
Key: KUDU-2869
URL: https://issues.apache.org/jira/browse/KUDU-2869
Project: Kudu
Issue Type: Bug
Affects Versions: 1.10.0
Reporter: Todd Lipcon
Trying to compile 1.10.0-rc0 with devtoolset-7 I hit the following error:
{code}
In file included from
/data/1/todd/apache-kudu-1.10.0/src/kudu/gutil/macros.h:14:0,
from
/data/1/todd/apache-kudu-1.10.0/src/kudu/common/types.h:34,
from
/data/1/todd/apache-kudu-1.10.0/src/kudu/common/columnblock.h:28,
from
/data/1/todd/apache-kudu-1.10.0/src/kudu/common/columnblock.cc:18:
/data/1/todd/apache-kudu-1.10.0/src/kudu/gutil/port.h: In substitution of
‘template::type*
, bool USE_REINTERPRET> T UnalignedLoad(const void*) [with T =
__int128; typename port_internal::enable_if_numeric::type* =
; bool USE_REINTERPRET = ]’:
/data/1/todd/apache-kudu-1.10.0/src/kudu/common/types.h:361:55: required from
here
/data/1/todd/apache-kudu-1.10.0/src/kudu/gutil/port.h:1196:73: internal
compiler error: unexpected expression ‘LoadByReinterpretCast<__int128>’ of kind
template_id_expr
bool USE_REINTERPRET = port_internal::LoadByReinterpretCast()>
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
