[jira] [Commented] (KUDU-2167) C++ client crashes when server filters out all data in a scan RPC
[
https://issues.apache.org/jira/browse/KUDU-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194050#comment-16194050
]
Adar Dembo commented on KUDU-2167:
--
I did some git spelunking and AFAICT the client-side code has always been
vulnerable to this, and the server-side code was changed to omit the 'data'
field when there's no data in commit 822f6b6. That commit was part of the first
Kudu release, so it's also more or less "since forever".
> C++ client crashes when server filters out all data in a scan RPC
> -
>
> Key: KUDU-2167
> URL: https://issues.apache.org/jira/browse/KUDU-2167
> Project: Kudu
> Issue Type: Bug
> Components: client
>Affects Versions: 1.3.1, 1.4.0, 1.5.0, 1.2.1
>Reporter: Adar Dembo
>Assignee: Adar Dembo
>Priority: Critical
>
> (this may affect older versions too; I only checked as far back as 1.4.0).
> A crash like this was observed in a test cluster:
> {noformat}
> *** Aborted at 1507079547 (unix time) try "date -d @1507079547" if you are
> using GNU date ***
> PC: @ 0x7f41942410ad std::swap<>()
> *** SIGSEGV (@0x18) received by PID 12187 (TID 0x7f418d067840) from PID 24;
> stack trace: ***
> @ 0x7f4192d82390 (unknown)
> @ 0x7f41942410ad std::swap<>()
> @ 0x7f419120b51d kudu::RowwiseRowBlockPB::InternalSwap()
> @ 0x7f419120b4ed kudu::RowwiseRowBlockPB::Swap()
> @ 0x7f419400cf15 kudu::client::KuduScanBatch::Data::Reset()
> @ 0x7f4193f928f2 kudu::client::KuduScanner::NextBatch()
> {noformat}
> I've tracked this down to the client scanner code which assumes that if the
> previous scan request indicated that more results await, the next scan
> request must return data. There's at least one case where that isn't true:
> when the server filters out all data in the next scan request due to a
> predicate.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KUDU-2167) C++ client crashes when server filters out all data in a scan RPC
[
https://issues.apache.org/jira/browse/KUDU-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adar Dembo updated KUDU-2167:
-
Affects Version/s: 1.2.1
1.3.1
> C++ client crashes when server filters out all data in a scan RPC
> -
>
> Key: KUDU-2167
> URL: https://issues.apache.org/jira/browse/KUDU-2167
> Project: Kudu
> Issue Type: Bug
> Components: client
>Affects Versions: 1.3.1, 1.4.0, 1.5.0, 1.2.1
>Reporter: Adar Dembo
>Assignee: Adar Dembo
>Priority: Critical
>
> (this may affect older versions too; I only checked as far back as 1.4.0).
> A crash like this was observed in a test cluster:
> {noformat}
> *** Aborted at 1507079547 (unix time) try "date -d @1507079547" if you are
> using GNU date ***
> PC: @ 0x7f41942410ad std::swap<>()
> *** SIGSEGV (@0x18) received by PID 12187 (TID 0x7f418d067840) from PID 24;
> stack trace: ***
> @ 0x7f4192d82390 (unknown)
> @ 0x7f41942410ad std::swap<>()
> @ 0x7f419120b51d kudu::RowwiseRowBlockPB::InternalSwap()
> @ 0x7f419120b4ed kudu::RowwiseRowBlockPB::Swap()
> @ 0x7f419400cf15 kudu::client::KuduScanBatch::Data::Reset()
> @ 0x7f4193f928f2 kudu::client::KuduScanner::NextBatch()
> {noformat}
> I've tracked this down to the client scanner code which assumes that if the
> previous scan request indicated that more results await, the next scan
> request must return data. There's at least one case where that isn't true:
> when the server filters out all data in the next scan request due to a
> predicate.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KUDU-2175) Do not hold the RaftConsensus lock while performing I/O
[ https://issues.apache.org/jira/browse/KUDU-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated KUDU-2175: -- Component/s: consensus > Do not hold the RaftConsensus lock while performing I/O > --- > > Key: KUDU-2175 > URL: https://issues.apache.org/jira/browse/KUDU-2175 > Project: Kudu > Issue Type: Improvement > Components: consensus >Reporter: Mike Percy > > Do not hold the RaftConsensus lock while performing I/O. This optimization > would improve stability and reliability as it could make a tablet server less > sensitive to disk latency spikes and election storms. On the other hand, we > would need to carefully ensure that the ordering invariants currently > enforced by this lock continue to hold. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (KUDU-2175) Do not hold the RaftConsensus lock while performing I/O
Mike Percy created KUDU-2175: Summary: Do not hold the RaftConsensus lock while performing I/O Key: KUDU-2175 URL: https://issues.apache.org/jira/browse/KUDU-2175 Project: Kudu Issue Type: Improvement Reporter: Mike Percy Do not hold the RaftConsensus lock while performing I/O. This optimization would improve stability and reliability as it could make a tablet server less sensitive to disk latency spikes and election storms. On the other hand, we would need to carefully ensure that the ordering invariants currently enforced by this lock continue to hold. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KUDU-2173) Partitions are pruned incorrectly when range-partitioned on a PK prefix
[
https://issues.apache.org/jira/browse/KUDU-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193928#comment-16193928
]
Todd Lipcon commented on KUDU-2173:
---
The reason we missed this bug is that partition_pruner-test wasn't calling
OptimizeScanSpec and thus didn't produce the same behavior as the client. Repro
test code here: https://gist.github.com/24f8bcb78187a18c91e04c6c36d165a2
> Partitions are pruned incorrectly when range-partitioned on a PK prefix
> ---
>
> Key: KUDU-2173
> URL: https://issues.apache.org/jira/browse/KUDU-2173
> Project: Kudu
> Issue Type: Bug
> Components: client
>Affects Versions: 1.2.0, 1.3.1, 1.4.0, 1.5.0
>Reporter: Todd Lipcon
>Assignee: Dan Burkert
>Priority: Blocker
>
> Given a schema:
> {code}
> Schema [
> 0:a[int8 NOT NULL],
> 1:b[int8 NOT NULL]
> ]
> PRIMARY KEY (a,b)
> {code}
> and a partition:
> {code}
> RANGE (a) PARTITION VALUES >= 10
> {code}
> ... the partition pruner incorrectly handles the following scan spec:
> {code}
> `a` < 11 AND `b` < 11
> {code}
> ... and prunes the partition despite the possibility of it having a row like
> (10, 1)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Closed] (KUDU-2171) Add IWYU to kudu/build-support/jenkins/build-and-test.sh
[ https://issues.apache.org/jira/browse/KUDU-2171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Fancher closed KUDU-2171. Resolution: Not A Bug My mistake. I was running on an older version. > Add IWYU to kudu/build-support/jenkins/build-and-test.sh > > > Key: KUDU-2171 > URL: https://issues.apache.org/jira/browse/KUDU-2171 > Project: Kudu > Issue Type: Improvement > Components: build >Affects Versions: 1.4.0 >Reporter: Edward Fancher >Assignee: Edward Fancher >Priority: Minor > > IWYU isn't present in build and test which makes it difficult to run > consistently from Jenkins. > It should be added in a way similar to lint builds. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KUDU-2172) Fractional voting
[
https://issues.apache.org/jira/browse/KUDU-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Percy updated KUDU-2172:
-
Description:
Fractional voting is a concept where different voters in a Raft config get
different voting weights. This can tip the balance in certain cases in which it
would otherwise be impossible to achieve a majority.
Example: With equally-weighted votes -- the standard scenario -- in a 4-voter
config \{A, B, C, D\}, if a network partition bifurcated this config into two
equal groups \{A, B\} and \{C, D\} then no one side could achieve a strict
majority (greater than 2 out of 4 votes) and thus the configuration (as a
whole) would be unavailable for writes. With fractional voting, if some (but
not all) voters got an additional fractional weight to their vote, perhaps
\{A=1.1, B=1.1, C=1.1, D=1.0\} it would be possible for some group of two nodes
to achieve a majority of votes. In this case, the \{A, B\} group would have a
combined weight of 2.2 out of 4.3, which is a strict majority. Therefore the
\{A, B\} group would be able to elect a leader between them and accept writes
to only the two nodes.
There is a durability tradeoff, in exchange for availability, in the above
scenario.
It is worth noting that there is a utility overlap in certain scenarios between
fractional voting and config change cancellation (KUDU-1194).
This idea was suggested by Todd during a discussion of the [3-4-3
re-replication
design|https://docs.google.com/document/d/1_gQ3BONhKVR2hFlDTShtdx1vHkbVTcLE3b-nrNFwGDI].
was:
Fractional voting is a concept where different voters in a Raft config get
different voting weights. This can tip the balance in certain cases in which it
would otherwise be impossible to achieve a majority.
Example: With equally-weighted votes -- the standard scenario -- in a 4-voter
config \{A, B, C, D\}, if a network partition bifurcated this config into two
equal groups \{A, B\} and \{C, D\} then no one side could achieve a strict
majority (greater than 2 out of 4 votes) and thus the configuration (as a
whole) would be unavailable for writes. With fractional voting, if some (but
not all) voters got an additional fractional weight to their vote, perhaps
\{A=1.1, B=1.1, C=1.1, D=1.0\} it would be possible for some group of two nodes
to achieve a majority of votes. In this case, the \{A, B\} group would have a
combined weight of 2.2 out of 4.3, which is a strict majority. Therefore the
\{A, B\} group would be able to elect a leader between them and accept writes
to only the two nodes.
There is a durability tradeoff, in exchange for availability, in the above
scenario.
It is worth noting that there is a utility overlap in certain scenarios between
fractional voting and config change cancellation (KUDU-1729).
This idea was suggested by Todd during a discussion of the [3-4-3
re-replication
design|https://docs.google.com/document/d/1_gQ3BONhKVR2hFlDTShtdx1vHkbVTcLE3b-nrNFwGDI].
> Fractional voting
> -
>
> Key: KUDU-2172
> URL: https://issues.apache.org/jira/browse/KUDU-2172
> Project: Kudu
> Issue Type: Bug
> Components: consensus
>Reporter: Mike Percy
>
> Fractional voting is a concept where different voters in a Raft config get
> different voting weights. This can tip the balance in certain cases in which
> it would otherwise be impossible to achieve a majority.
> Example: With equally-weighted votes -- the standard scenario -- in a 4-voter
> config \{A, B, C, D\}, if a network partition bifurcated this config into two
> equal groups \{A, B\} and \{C, D\} then no one side could achieve a strict
> majority (greater than 2 out of 4 votes) and thus the configuration (as a
> whole) would be unavailable for writes. With fractional voting, if some (but
> not all) voters got an additional fractional weight to their vote, perhaps
> \{A=1.1, B=1.1, C=1.1, D=1.0\} it would be possible for some group of two
> nodes to achieve a majority of votes. In this case, the \{A, B\} group would
> have a combined weight of 2.2 out of 4.3, which is a strict majority.
> Therefore the \{A, B\} group would be able to elect a leader between them and
> accept writes to only the two nodes.
> There is a durability tradeoff, in exchange for availability, in the above
> scenario.
> It is worth noting that there is a utility overlap in certain scenarios
> between fractional voting and config change cancellation (KUDU-1194).
> This idea was suggested by Todd during a discussion of the [3-4-3
> re-replication
> design|https://docs.google.com/document/d/1_gQ3BONhKVR2hFlDTShtdx1vHkbVTcLE3b-nrNFwGDI].
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (KUDU-2172) Fractional voting
Mike Percy created KUDU-2172:
Summary: Fractional voting
Key: KUDU-2172
URL: https://issues.apache.org/jira/browse/KUDU-2172
Project: Kudu
Issue Type: Bug
Components: consensus
Reporter: Mike Percy
Fractional voting is a concept where different voters in a Raft config get
different voting weights. This can tip the balance in certain cases in which it
would otherwise be impossible to achieve a majority.
Example: With equally-weighted votes -- the standard scenario -- in a 4-voter
config \{A, B, C, D\}, if a network partition bifurcated this config into two
equal groups \{A, B\} and \{C, D\} then no one side could achieve a strict
majority (greater than 2 out of 4 votes) and thus the configuration (as a
whole) would be unavailable for writes. With fractional voting, if some (but
not all) voters got an additional fractional weight to their vote, perhaps
\{A=1.1, B=1.1, C=1.1, D=1.0\} it would be possible for some group of two nodes
to achieve a majority of votes. In this case, the \{A, B\} group would have a
combined weight of 2.2 out of 4.3, which is a strict majority. Therefore the
\{A, B\} group would be able to elect a leader between them and accept writes
to only the two nodes.
There is a durability tradeoff, in exchange for availability, in the above
scenario.
It is worth noting that there is a utility overlap in certain scenarios between
fractional voting and config change cancellation (KUDU-1729).
This idea was suggested by Todd during a discussion of the [3-4-3
re-replication
design|https://docs.google.com/document/d/1_gQ3BONhKVR2hFlDTShtdx1vHkbVTcLE3b-nrNFwGDI].
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (KUDU-2170) Masters can start up with duplicate masters specified
Andrew Wong created KUDU-2170: - Summary: Masters can start up with duplicate masters specified Key: KUDU-2170 URL: https://issues.apache.org/jira/browse/KUDU-2170 Project: Kudu Issue Type: Bug Components: master Reporter: Andrew Wong In a multi-master deployment, when a master starts up, it contacts the other masters to gather UUIDs, but doesn't check whether there are any duplicates. This can lead to some scary-looking issues, like duplicate masters showing up in the web UI. It would be nice if this were disallowed. Also worth noting: having all unique entries in `master_addresses` doesn't necessarily imply they are all unique masters (e.g. some masters may be specified via DNS aliases). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KUDU-2171) Add IWYU to kudu/build-support/jenkins/build-and-test.sh
[ https://issues.apache.org/jira/browse/KUDU-2171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Fancher updated KUDU-2171: - Description: IWYU isn't present in build and test which makes it difficult to run consistently from Jenkins. It should be added in a way similar to lint builds. > Add IWYU to kudu/build-support/jenkins/build-and-test.sh > > > Key: KUDU-2171 > URL: https://issues.apache.org/jira/browse/KUDU-2171 > Project: Kudu > Issue Type: Improvement > Components: build >Affects Versions: 1.4.0 >Reporter: Edward Fancher >Assignee: Edward Fancher >Priority: Minor > > IWYU isn't present in build and test which makes it difficult to run > consistently from Jenkins. > It should be added in a way similar to lint builds. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KUDU-2032) Kerberos authentication fails with rdns disabled in krb5.conf
[ https://issues.apache.org/jira/browse/KUDU-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Will Berkeley updated KUDU-2032: Target Version/s: (was: 1.4.1) Fix Version/s: 1.4.1 1.3.2 [~r1pp3rj4ck] That's right! I'm resolving this as a result. > Kerberos authentication fails with rdns disabled in krb5.conf > - > > Key: KUDU-2032 > URL: https://issues.apache.org/jira/browse/KUDU-2032 > Project: Kudu > Issue Type: Bug > Components: security >Affects Versions: 1.3.1, 1.4.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Critical > Fix For: 1.3.2, 1.5.0, 1.4.1 > > > Currently if 'rnds = false' is configured in krb5.conf, Kudu ends up using > the IP addresses of remote hosts instead of the hostnames. This means that it > will look for krb5 principals by IP, even if actual hostnames have been > passed instead. > This prevents krb5 from working properly in most environments where > rdns=false is set. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KUDU-2032) Kerberos authentication fails with rdns disabled in krb5.conf
[ https://issues.apache.org/jira/browse/KUDU-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193677#comment-16193677 ] Attila Bukor commented on KUDU-2032: it seems this has been merged into branch-1.4.x and branch-1.3.x now > Kerberos authentication fails with rdns disabled in krb5.conf > - > > Key: KUDU-2032 > URL: https://issues.apache.org/jira/browse/KUDU-2032 > Project: Kudu > Issue Type: Bug > Components: security >Affects Versions: 1.3.1, 1.4.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Critical > Fix For: 1.5.0 > > > Currently if 'rnds = false' is configured in krb5.conf, Kudu ends up using > the IP addresses of remote hosts instead of the hostnames. This means that it > will look for krb5 principals by IP, even if actual hostnames have been > passed instead. > This prevents krb5 from working properly in most environments where > rdns=false is set. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KUDU-871) Allow tombstoned tablets to vote
[ https://issues.apache.org/jira/browse/KUDU-871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Percy resolved KUDU-871. - Resolution: Fixed Fix Version/s: 1.5.0 Target Version/s: 1.5.0 (was: 1.4.0) Merged in 5bca7d8ba185d62952fb3e3163cbe88d20453da0 > Allow tombstoned tablets to vote > > > Key: KUDU-871 > URL: https://issues.apache.org/jira/browse/KUDU-871 > Project: Kudu > Issue Type: Sub-task > Components: consensus >Affects Versions: Public beta >Reporter: Mike Percy >Assignee: Mike Percy >Priority: Critical > Fix For: 1.5.0 > > > We may need to support voting for tombstoned tablets to avoid availability > issues in certain scenarios. For more discussion, see: > https://github.com/apache/kudu/blob/master/docs/design-docs/raft-tablet-copy.md#should-a-server-be-allowed-to-vote-if-it-does_not_exist-or-is-deleted -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KUDU-2163) names of "rpc_authentication" do not match in documents
[ https://issues.apache.org/jira/browse/KUDU-2163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193071#comment-16193071 ] Dan Burkert commented on KUDU-2163: --- relevant: https://lists.apache.org/thread.html/14cfce88cb6af7b51b5ed801f9b7212b7e7ff34b7858eec88cff0cd8@%3Cdev.kudu.apache.org%3E > names of "rpc_authentication" do not match in documents > --- > > Key: KUDU-2163 > URL: https://issues.apache.org/jira/browse/KUDU-2163 > Project: Kudu > Issue Type: Bug > Components: documentation >Reporter: Jiahongchao >Priority: Minor > Fix For: n/a > > > in > [http://kudu.apache.org/docs/configuration_reference.html#kudu-master_rpc_authentication],it > is "rpc_authentication" > in [http://kudu.apache.org/docs/security.html],it is "rpc-authentication" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
