[
https://issues.apache.org/jira/browse/HDFS-16129?focusedWorklogId=633413&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-633413
]
ASF GitHub Bot logged work on HDFS-16129:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 04/Aug/21 10:29
Start Date: 04/Aug/21 10:29
Worklog Time Spent: 10m
Work Description: brumi1024 commented on a change in pull request #3209:
URL: https://github.com/apache/hadoop/pull/3209#discussion_r682480203
##########
File path:
hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/HttpFSAuthenticationFilter.java
##########
@@ -69,27 +72,11 @@
@Override
protected Properties getConfiguration(String configPrefix,
FilterConfig filterConfig) throws ServletException{
- Properties props = new Properties();
+ System.out.println("getConfiguration1");
Configuration conf = HttpFSServerWebApp.get().getConfig();
-
- props.setProperty(AuthenticationFilter.COOKIE_PATH, "/");
- for (Map.Entry<String, String> entry : conf) {
- String name = entry.getKey();
- if (name.startsWith(HADOOP_HTTP_CONF_PREFIX)) {
- name = name.substring(HADOOP_HTTP_CONF_PREFIX.length());
- props.setProperty(name, entry.getValue());
- }
- }
-
- // Replace Hadoop Http Authentication Configs with HttpFS specific Configs
- for (Map.Entry<String, String> entry : conf) {
- String name = entry.getKey();
- if (name.startsWith(CONF_PREFIX)) {
- String value = conf.get(name);
- name = name.substring(CONF_PREFIX.length());
- props.setProperty(name, value);
- }
- }
+ Properties props = HttpServer2.getFilterProperties(conf,
+ new ArrayList<>(Arrays.asList(CONF_PREFIXES)));
+ System.out.println("getConfiguration2");
Review comment:
Nit: I think this could be removed.
##########
File path:
hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/HttpFSAuthenticationFilter.java
##########
@@ -98,6 +85,7 @@ protected Properties getConfiguration(String configPrefix,
}
if (!isRandomSecret(filterConfig)) {
+ System.out.println("FILE: " + signatureSecretFile);
Review comment:
Nit: I think this could be removed.
##########
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java
##########
@@ -243,7 +243,9 @@
private String hostName;
private boolean disallowFallbackToRandomSignerSecretProvider;
- private String authFilterConfigurationPrefix =
"hadoop.http.authentication.";
+ private final ArrayList<String> authFilterConfigurationPrefixes =
+ new ArrayList<>(Collections.singletonList(
+ "hadoop.http.authentication."));
Review comment:
Nit: can you use a common final variable here as well?
##########
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java
##########
@@ -811,18 +822,26 @@ private static SignerSecretProvider
constructSecretProvider(final Builder b,
throws Exception {
final Configuration conf = b.conf;
Properties config = getFilterProperties(conf,
- b.authFilterConfigurationPrefix);
+ b.authFilterConfigurationPrefixes);
return AuthenticationFilter.constructSecretProvider(
ctx, config, b.disallowFallbackToRandomSignerSecretProvider);
}
- private static Properties getFilterProperties(Configuration conf, String
- prefix) {
- Properties prop = new Properties();
- Map<String, String> filterConfig = AuthenticationFilterInitializer
- .getFilterConfigMap(conf, prefix);
- prop.putAll(filterConfig);
- return prop;
+ public static Properties getFilterProperties(Configuration conf,
+ ArrayList<String> prefixes) {
+ Properties props = new Properties();
+ prefixes.forEach(prefix -> {
+ Map<String, String> filterConfigMap =
+ AuthenticationFilterInitializer.getFilterConfigMap(conf, prefix);
+ filterConfigMap.forEach((key, value) -> {
+ Object previous = props.setProperty(key, value);
+ if (previous != null && !previous.equals(value)) {
+ LOG.warn("Overwriting configuration for key='{}' with value='{}' " +
+ "previous_value='{}'", key, value, previous);
+ }
+ });
+ });
Review comment:
I generally don't have anything against using lambdas, but this doesn't
seem like an improvement over a simple for or for-each cycle. Performance wise
(doesn't matter too much in this case) it's similar, but this I think is a bit
harder to read, and debugging it is harder (because the unusual stacktrace that
comes from the lambda), and in the future if for some reason someone wants to
throw a checked exception when parsing the config he/she will need to rewrite
this to a for cycle.
##########
File path:
hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/HttpFSAuthenticationFilter.java
##########
@@ -69,27 +72,11 @@
@Override
protected Properties getConfiguration(String configPrefix,
FilterConfig filterConfig) throws ServletException{
- Properties props = new Properties();
+ System.out.println("getConfiguration1");
Review comment:
Nit: I think this could be removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 633413)
Time Spent: 1h 40m (was: 1.5h)
> HttpFS signature secret file misusage
> -------------------------------------
>
> Key: HDFS-16129
> URL: https://issues.apache.org/jira/browse/HDFS-16129
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: httpfs
> Affects Versions: 3.4.0
> Reporter: Tamas Domok
> Assignee: Tamas Domok
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> I started to work on the YARN-10814 issue, and found this bug in the HttpFS.
> I investigated the problem and I already have some fix for it.
>
> If the deprecated *httpfs.authentication.signature.secret.file* is not set in
> the configuration (e.g.: httpfs-site.xml) then the new
> *hadoop.http.authentication.signature.secret.file* config option won't be
> used, it will fallback to the random secret provider silently.
> The _HttpFSServerWebServer_ sets an _authFilterConfigurationPrefix_ when
> building the server for the old path (*httpfs.authentication.*). Later the
> _AuthenticationFilter.constructSecretProvider_ will immediately fallback to
> +random+, because the config won't contain the file. If the old path was set
> too, then it handled the file, and the provider was set to +file+ type.
> The configuration should be based on both the old and the new prefix filter,
> merging the two. The new config option should win in my opinion.
>
> There is another issue in the _HttpFSAuthenticationFilter_, it is closely
> related.
> If both config option is set then the _HttpFSAuthenticationFilter_ will fail
> with an impossible file path (e.g.:
> *${httpfs.config.dir}/httpfs-signature.secret*).
> _HttpFSAuthenticationFilter_ constructs the configuration, filtering first
> the new config prefix then the old prefix. The old prefix code works
> correctly, it uses the _conf.get(key)_
> instead of the _entry.getValue()_ which gives back the file path mentioned
> earlier. The code duplication can be eliminated and I think it would be
> better to change the order, first adding the config options from the old path
> then the new, and the new should overwrite the old values, with a warning log
> message.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
