[
https://issues.apache.org/jira/browse/YARN-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15968218#comment-15968218
]
Carlo Curino commented on YARN-3663:
------------------------------------
Thanks [~giovanni.fumarola] for the updated patch.
Conf:
# You set
{{YarnConfiguration.DEFAULT_FEDERATION_STATESTORE_SQL_MAXCONNECTIONS}} to 1.
Isn't this too small? What values you commonly use? I would put a conservative
but not overly tight value, otherwise users are forced to learn/modify more
params.
# We should omit {{yarn.federation.state-store.sql.max-connections} from
yarn-default.xml like you do for all other params in this patch.
In {{SQLFederationStateStore}}
# For most fields except {{CALL_SP_GET_ALL_POLICY_CONFIGURATIONS}} you simply
use the plural to differentiate the getter of one or all items. Be consistent
(e.g., remove the ALL here, or add it everywhere else)
# Minor: userNameDB --> userName, passwordDB --> password
# When you throw the exceptions (e.g., subcluster registration), it might be
nice in the message to include the sub-clusterID / ip or any other info one can
use to debug.
# Can you comment on why we are using: {{FederationStateStoreErrorCode}}?
They don't seem to be connected to SQL error codes, and they are not used
anywhere else (we normally use named exception, which are easier to
understand/track).
# at line 277-278: formatting
# We should try to remove redundance, e.g., you have lots of things that look
like this:
{code}
try {
FederationMembershipStateStoreInputValidator
.validateGetSubClusterInfoRequest(subClusterRequest);
} catch (FederationStateStoreInvalidInputException e) {
FederationStateStoreUtils.logAndThrowInvalidInputException(LOG,
"Unable to obtain the infomation about the SubCluster. "
+ e.getMessage());
}
{code}
They could be factored out to
{{FederationMembershipStateStoreInputValidator.validate(subClusterRequest)}}
where the type of input param is used to differentiate the method, and the
logAndThrowInvalidInputException is done on that side. Same goes for
{{checkSubClusterInfo}}.
# Similarly to the above we should try to factor out the very repetitive code
to create connection/statements, set params, run, and throw. I don't have a
specific advise on this, but the code is mostly copy and paste, which we should
avoid.
# Move the {{fromStringToSubClusterState}} to the SubclusterState class (and
call it fromString().
# Why {{getSubCluster}} and {{getSubClusters}} use different mechanics for
return values? (registered params vs ResultSet)? Might be worth to be
consistent (probably using ResultSet).
# Line 540: Is this behavior (overwrite existing) consistent with general
YARN? (I think so, but want to check)
# Some of the log are a bit vague {{LOG.debug("Got the information about the
specified application }} say spefically what info where gotten
# if you use {{LOG.debug}} consider to prefix it with a check if we are in
debug mode (save time/objects creations for the String that are then not used).
# You have several {{ if (cstmt.getInt(2) != 1) }} ROWCOUNT checks. This
mix the no tuple where changed to multiple tuple where changed. Distinguishing
the two cases, might help debug (do we have duplicates in DB, or the entry was
not found). (Not mandatory, just somethign to consider)
# {{setPolicyConfiguration}} You are doing {{cstmt.setBytes(3, new
byte[policyConf.getParams().remaining()]);}} which adds an empty byte[] instead
of what is coming in input.
# {{getCurrentVersion}} and {{loadVersion}} throw a NotSupportedException or
something of the sort, a silent return null is easy to confuse people. (I know
the full version will be in V2, let's just have a clear breakage if someone try
to use this methods).
(.... to be continued ...)
> Federation State and Policy Store (DBMS implementation)
> -------------------------------------------------------
>
> Key: YARN-3663
> URL: https://issues.apache.org/jira/browse/YARN-3663
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
> Affects Versions: YARN-2915
> Reporter: Giovanni Matteo Fumarola
> Assignee: Giovanni Matteo Fumarola
> Attachments: YARN-3663-YARN-2915.v1.patch,
> YARN-3663-YARN-2915.v2.patch
>
>
> This JIRA tracks a SQL-based implementation of the Federation State and
> Policy Store, which implements YARN-3662 APIs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
