[
https://issues.apache.org/jira/browse/KUDU-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18091162#comment-18091162
]
ASF subversion and git services commented on KUDU-3785:
-------------------------------------------------------
Commit 59f7715aeebb34e0dd1e254dfb5ae1c3fad4c3e8 in kudu's branch
refs/heads/master from Marton Greber
[ https://gitbox.apache.org/repos/asf?p=kudu.git;h=59f7715ae ]
KUDU-3785: Fix CLI flag validator failure
A customer issue was discovered during rolling restarts where
the tserver flagfile is passed to CLI commands (e.g. kudu fs
dump --flagfile=tserver.flags). The CLI previously inflated
rpc_max_message_size to ~2GB via SET_FLAGS_DEFAULT in tool_main.cc
to accommodate large RPC responses. This caused the
GROUP_FLAG_VALIDATOR in op_tracker.cc to reject any flagfile that
set tablet_transaction_memory_limit_mb below 2GB (e.g. 256MB <
2GB -> failure).
The root cause is that inflating rpc_max_message_size as a global
flag is a layering violation: it's a client-side concern that
pollutes process-wide state and triggers global validators.
The fix moves the large message size into BuildMessenger(), which
is the single chokepoint for all CLI RPC messengers. If the user
has not explicitly set --rpc_max_message_size, the messenger gets
min(INT32_MAX, available_memory); otherwise their explicit value
is honored. The global flag is never mutated, so the validator
sees the compiled default (50MB) and passes regardless of flagfile
contents. Server subcommands are unaffected since they build their
own messengers through the normal server path.
A regression test verifies the flagfile scenario, and a unit test
confirms BuildMessenger produces a messenger with a large max
message size.
Change-Id: Idd24ec605bd14ce90ed8a5705230ff96d77d0d64
Reviewed-on: http://gerrit.cloudera.org:8080/24475
Reviewed-by: Ashwani Raina <[email protected]>
Tested-by: Marton Greber <[email protected]>
Reviewed-by: Alexey Serbin <[email protected]>
Reviewed-by: Abhishek Chennaka <[email protected]>
Reviewed-by: Zoltan Chovan <[email protected]>
> [CLI] Flag validation failure when flagfile sets
> tablet_transaction_memory_limit_mb
> -----------------------------------------------------------------------------------
>
> Key: KUDU-3785
> URL: https://issues.apache.org/jira/browse/KUDU-3785
> Project: Kudu
> Issue Type: Bug
> Reporter: Marton Greber
> Assignee: Marton Greber
> Priority: Major
>
> The kudu CLI tool fails with "Detected inconsistency in command-line flags"
> when invoked with a tserver flagfile that sets
> --tablet_transaction_memory_limit_mb
> to a value lower than the CLI's internal --rpc_max_message_size default
> (~2GB).
> This occurs during rolling restarts when the restart script passes the
> tserver's
> flagfile to commands like `kudu fs dump uuid`.
> Fix: set tablet_transaction_memory_limit_mb=-1 (disabled) after flag parsing
> using SET_FLAGS_VALUE. The CLI doesn't host tablets, so transaction memory
> tracking is irrelevant. The validator already skips negative values.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
