[
https://issues.apache.org/jira/browse/HDFS-16129?focusedWorklogId=633462&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-633462
]
ASF GitHub Bot logged work on HDFS-16129:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 04/Aug/21 12:29
Start Date: 04/Aug/21 12:29
Worklog Time Spent: 10m
Work Description: tomicooler commented on a change in pull request #3209:
URL: https://github.com/apache/hadoop/pull/3209#discussion_r682568677
##########
File path:
hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServerWebServer.java
##########
@@ -71,53 +82,195 @@ public static void beforeClass() throws Exception {
System.setProperty("httpfs.home.dir", homeDir.getAbsolutePath());
System.setProperty("httpfs.log.dir", logsDir.getAbsolutePath());
System.setProperty("httpfs.config.dir", confDir.getAbsolutePath());
- FileUtils.writeStringToFile(new File(confDir, "httpfs-signature.secret"),
- "foo", StandardCharsets.UTF_8);
+ secretFile = new File(System.getProperty("httpfs.config.dir"),
+ "httpfs-signature-custom.secret");
}
- @Before
- public void setUp() throws Exception {
- Configuration conf = new Configuration();
- conf.set(HttpFSServerWebServer.HTTP_HOSTNAME_KEY, "localhost");
- conf.setInt(HttpFSServerWebServer.HTTP_PORT_KEY, 0);
- conf.set(AuthenticationFilter.SIGNATURE_SECRET_FILE,
- "httpfs-signature.secret");
- Configuration sslConf = new Configuration();
- webServer = new HttpFSServerWebServer(conf, sslConf);
+ @After
+ public void teardown() throws Exception {
+ if (webServer != null) {
+ webServer.stop();
+ }
}
@Test
public void testStartStop() throws Exception {
+ webServer = createWebServer(createConfigurationWithRandomSecret());
webServer.start();
- String user = HadoopUsersConfTestHelper.getHadoopUsers()[0];
- URL url = new URL(webServer.getUrl(), MessageFormat.format(
- "/webhdfs/v1/?user.name={0}&op=liststatus", user));
- HttpURLConnection conn = (HttpURLConnection) url.openConnection();
- Assert.assertEquals(HttpURLConnection.HTTP_OK, conn.getResponseCode());
- BufferedReader reader = new BufferedReader(
- new InputStreamReader(conn.getInputStream()));
- reader.readLine();
- reader.close();
webServer.stop();
}
@Test
public void testJustStop() throws Exception {
+ webServer = createWebServer(createConfigurationWithRandomSecret());
webServer.stop();
}
@Test
public void testDoubleStop() throws Exception {
+ webServer = createWebServer(createConfigurationWithRandomSecret());
webServer.start();
webServer.stop();
webServer.stop();
}
@Test
public void testDoubleStart() throws Exception {
+ webServer = createWebServer(createConfigurationWithRandomSecret());
+ webServer.start();
+ webServer.start();
+ webServer.stop();
+ }
+
+ @Test
+ public void testServiceWithSecretFile() throws Exception {
+ createSecretFile("foo");
+ webServer = createWebServer(createConfigurationWithSecretFile());
+ webServer.start();
+ assertServiceRespondsWithOK(webServer.getUrl());
+ assertSignerSecretProviderType(webServer.getHttpServer(),
+ FileSignerSecretProvider.class);
+ webServer.stop();
+ }
+
+ @Test
+ public void testServiceWithSecretFileWithDeprecatedConfigOnly()
+ throws Exception {
+ createSecretFile("foo");
+ Configuration conf = createConfiguration();
+ setDeprecatedSecretFile(conf, secretFile.getAbsolutePath());
+ webServer = createWebServer(conf);
+
+ // HttpFSAuthenticationFilter gets the configuration from the webapp, the
defaults are loaded
+ // so the custom file location is overwritten with a non-existing default
one.
+ Exception exception =
+ Assert.assertThrows(IOException.class, new ThrowingRunnable() {
+ @Override
+ public void run() throws Throwable {
+ webServer.start();
+ }
+ });
+ Assert.assertTrue(exception.getMessage().startsWith(
+ "Unable to initialize"));
+ }
+
+ @Test
+ public void testServiceWithSecretFileWithBothConfigOption() throws Exception
{
+ createSecretFile("foo");
+ Configuration conf = createConfigurationWithSecretFile();
+ setDeprecatedSecretFile(conf, secretFile.getAbsolutePath());
+ webServer = createWebServer(conf);
+ webServer.start();
+ assertServiceRespondsWithOK(webServer.getUrl());
+ assertSignerSecretProviderType(webServer.getHttpServer(),
+ FileSignerSecretProvider.class);
+ webServer.stop();
+ }
+
+ @Test
+ public void testServiceWithSecretFileWithBothConfigOptionOverWriteCheck()
Review comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 633462)
Time Spent: 3h 50m (was: 3h 40m)
> HttpFS signature secret file misusage
> -------------------------------------
>
> Key: HDFS-16129
> URL: https://issues.apache.org/jira/browse/HDFS-16129
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: httpfs
> Affects Versions: 3.4.0
> Reporter: Tamas Domok
> Assignee: Tamas Domok
> Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> I started to work on the YARN-10814 issue, and found this bug in the HttpFS.
> I investigated the problem and I already have some fix for it.
>
> If the deprecated *httpfs.authentication.signature.secret.file* is not set in
> the configuration (e.g.: httpfs-site.xml) then the new
> *hadoop.http.authentication.signature.secret.file* config option won't be
> used, it will fallback to the random secret provider silently.
> The _HttpFSServerWebServer_ sets an _authFilterConfigurationPrefix_ when
> building the server for the old path (*httpfs.authentication.*). Later the
> _AuthenticationFilter.constructSecretProvider_ will immediately fallback to
> +random+, because the config won't contain the file. If the old path was set
> too, then it handled the file, and the provider was set to +file+ type.
> The configuration should be based on both the old and the new prefix filter,
> merging the two. The new config option should win in my opinion.
>
> There is another issue in the _HttpFSAuthenticationFilter_, it is closely
> related.
> If both config option is set then the _HttpFSAuthenticationFilter_ will fail
> with an impossible file path (e.g.:
> *${httpfs.config.dir}/httpfs-signature.secret*).
> _HttpFSAuthenticationFilter_ constructs the configuration, filtering first
> the new config prefix then the old prefix. The old prefix code works
> correctly, it uses the _conf.get(key)_
> instead of the _entry.getValue()_ which gives back the file path mentioned
> earlier. The code duplication can be eliminated and I think it would be
> better to change the order, first adding the config options from the old path
> then the new, and the new should overwrite the old values, with a warning log
> message.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
