[jira] [Updated] (HADOOP-13597) Switch KMS from Tomcat to Jetty

[John Zhuge (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Zhuge updated HADOOP-13597:

Release Note: 


The following environment variables are deprecated. Set the corresponding
configuration properties instead.

Environment Variable | Configuration Property   | Configuration File
-|--|
KMS_HTTP_PORT| hadoop.kms.http.port | kms-site.xml
KMS_MAX_HTTP_HEADER_SIZE | hadoop.http.max.request.header.size and 
hadoop.http.max.response.header.size | kms-site.xml
KMS_MAX_THREADS  | hadoop.http.max.threads  | kms-site.xml
KMS_SSL_ENABLED  | hadoop.kms.ssl.enabled   | kms-site.xml
KMS_SSL_KEYSTORE_FILE| ssl.server.keystore.location | ssl-server.xml
KMS_SSL_KEYSTORE_PASS| ssl.server.keystore.password | ssl-server.xml
KMS_TEMP | hadoop.http.temp.dir | kms-site.xml

These default HTTP Services have been added.

Name   | Description
---|
/conf  | Display configuration properties
/jmx   | Java JMX management interface
/logLevel  | Get or set log level per class
/logs  | Display log files
/stacks| Display JVM stacks
/static/index.html | The static home page

The JMX path has been changed from /kms/jmx to /jmx.

Script kms.sh has been deprecated, use `hadoop kms` instead. The new scripts 
are based on the Hadoop shell scripting framework. `hadoop daemonlog` is 
supported. SSL configurations are read from ssl-server.xml.

  was:


The following environment variables are deprecated. Set the corresponding
configuration properties instead.

Environment Variable | Configuration Property   | Configuration File
-|--|
KMS_HTTP_PORT| hadoop.kms.http.port | kms-site.xml
KMS_MAX_HTTP_HEADER_SIZE | hadoop.http.max.request.header.size and 
hadoop.http.max.response.header.size | kms-site.xml
KMS_MAX_THREADS  | hadoop.http.max.threads  | kms-site.xml
KMS_SSL_ENABLED  | hadoop.kms.ssl.enabled   | kms-site.xml
KMS_SSL_KEYSTORE_FILE| ssl.server.keystore.location | ssl-server.xml
KMS_SSL_KEYSTORE_PASS| ssl.server.keystore.password | ssl-server.xml
KMS_TEMP | hadoop.http.temp.dir | kms-site.xml

These default HTTP Services have been added.

Name   | Description
---|
/conf  | Display configuration properties
/jmx   | Java JMX management interface
/logLevel  | Get or set log level per class
/logs  | Display log files
/stacks| Display JVM stacks
/static/index.html | The static home page

Script kms.sh has been deprecated, use `hadoop kms` instead. The new scripts 
are based on the Hadoop shell scripting framework. `hadoop daemonlog` is 
supported. SSL configurations are read from ssl-server.xml.


> Switch KMS from Tomcat to Jetty
> ---
>
> Key: HADOOP-13597
> URL: https://issues.apache.org/jira/browse/HADOOP-13597
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: kms
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13597.001.patch, HADOOP-13597.002.patch, 
> HADOOP-13597.003.patch, HADOOP-13597.004.patch, HADOOP-13597.005.patch, 
> HADOOP-13597.006.patch, HADOOP-13597.007.patch
>
>
> The Tomcat 6 we are using will reach EOL at the end of 2017. While there are 
> other good options, I would propose switching to {{Jetty 9}} for the 
> following reasons:
> * Easier migration. Both Tomcat and Jetty are based on {{Servlet 
> Containers}}, so we don't have change client code that much. It would require 
> more work to switch to {{JAX-RS}}.
> * Well established.
> * Good performance and scalability.
> Other alternatives:
> * Jersey + Grizzly
> * Tomcat 8
> Your opinions will be greatly appreciated.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-13597) Switch KMS from Tomcat to Jetty

[John Zhuge (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Zhuge updated HADOOP-13597:

Release Note: 


The following environment variables are deprecated. Set the corresponding
configuration properties instead.

Environment Variable | Configuration Property   | Configuration File
-|--|
KMS_HTTP_PORT| hadoop.kms.http.port | kms-site.xml
KMS_MAX_HTTP_HEADER_SIZE | hadoop.http.max.request.header.size and 
hadoop.http.max.response.header.size | kms-site.xml
KMS_MAX_THREADS  | hadoop.http.max.threads  | kms-site.xml
KMS_SSL_ENABLED  | hadoop.kms.ssl.enabled   | kms-site.xml
KMS_SSL_KEYSTORE_FILE| ssl.server.keystore.location | ssl-server.xml
KMS_SSL_KEYSTORE_PASS| ssl.server.keystore.password | ssl-server.xml
KMS_TEMP | hadoop.http.temp.dir | kms-site.xml

These default HTTP Services have been added.

Name   | Description
---|
/conf  | Display configuration properties
/jmx   | Java JMX management interface
/logLevel  | Get or set log level per class
/logs  | Display log files
/stacks| Display JVM stacks
/static/index.html | The static home page

Script kms.sh has been deprecated, use `hadoop kms` instead. The new scripts 
are based on the Hadoop shell scripting framework. `hadoop daemonlog` is 
supported. SSL configurations are read from ssl-server.xml.

  was:


The following environment variables are deprecated. Set the corresponding
configuration properties instead.

Environment Variable | Configuration Property   | Configuration File
-|--|
KMS_HTTP_PORT| hadoop.kms.http.port | kms-site.xml
KMS_MAX_HTTP_HEADER_SIZE | hadoop.http.max.request.header.size and 
hadoop.http.max.response.header.size | kms-site.xml
KMS_MAX_THREADS  | hadoop.http.max.threads  | kms-site.xml
KMS_SSL_ENABLED  | hadoop.kms.ssl.enabled   | kms-site.xml
KMS_SSL_KEYSTORE_FILE| ssl.server.keystore.location | ssl-server.xml
KMS_SSL_KEYSTORE_PASS| ssl.server.keystore.password | ssl-server.xml
KMS_TEMP | hadoop.http.temp.dir | kms-site.xml

These default HTTP Services have been added.

Name   | Description
---|
/conf  | Display configuration properties
/jmx   | Java JMX management interface
/logLevel  | Get or set log level per class
/logs  | Display log files
/stacks| Display JVM stacks
/static/index.html | The static home page

Script kms.sh has been deprecated, use 'hadoop kms' instead. Conform to the 
Hadoop shell scripting framework. Support 'hadoop daemonlog'. Read SSL 
configurations from ssl-server.xml, like many other Hadoop components.


> Switch KMS from Tomcat to Jetty
> ---
>
> Key: HADOOP-13597
> URL: https://issues.apache.org/jira/browse/HADOOP-13597
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: kms
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13597.001.patch, HADOOP-13597.002.patch, 
> HADOOP-13597.003.patch, HADOOP-13597.004.patch, HADOOP-13597.005.patch, 
> HADOOP-13597.006.patch, HADOOP-13597.007.patch
>
>
> The Tomcat 6 we are using will reach EOL at the end of 2017. While there are 
> other good options, I would propose switching to {{Jetty 9}} for the 
> following reasons:
> * Easier migration. Both Tomcat and Jetty are based on {{Servlet 
> Containers}}, so we don't have change client code that much. It would require 
> more work to switch to {{JAX-RS}}.
> * Well established.
> * Good performance and scalability.
> Other alternatives:
> * Jersey + Grizzly
> * Tomcat 8
> Your opinions will be greatly appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-13597) Switch KMS from Tomcat to Jetty

[John Zhuge (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Zhuge updated HADOOP-13597:

Release Note: 


The following environment variables are deprecated. Set the corresponding
configuration properties instead.

Environment Variable | Configuration Property   | Configuration File
-|--|
KMS_HTTP_PORT| hadoop.kms.http.port | kms-site.xml
KMS_MAX_HTTP_HEADER_SIZE | hadoop.http.max.request.header.size and 
hadoop.http.max.response.header.size | kms-site.xml
KMS_MAX_THREADS  | hadoop.http.max.threads  | kms-site.xml
KMS_SSL_ENABLED  | hadoop.kms.ssl.enabled   | kms-site.xml
KMS_SSL_KEYSTORE_FILE| ssl.server.keystore.location | ssl-server.xml
KMS_SSL_KEYSTORE_PASS| ssl.server.keystore.password | ssl-server.xml
KMS_TEMP | hadoop.http.temp.dir | kms-site.xml

These default HTTP Services have been added.

Name   | Description
---|
/conf  | Display configuration properties
/jmx   | Java JMX management interface
/logLevel  | Get or set log level per class
/logs  | Display log files
/stacks| Display JVM stacks
/static/index.html | The static home page

Script kms.sh has been deprecated, use 'hadoop kms' instead. Conform to the 
Hadoop shell scripting framework. Support 'hadoop daemonlog'. Read SSL 
configurations from ssl-server.xml, like many other Hadoop components.

  was:
The following environment variables are deprecated. Set the corresponding
configuration properties instead.

Environment Variable | Configuration Property   | Configuration File
-|--|
KMS_HTTP_PORT| hadoop.kms.http.port | kms-site.xml
KMS_MAX_HTTP_HEADER_SIZE | hadoop.http.max.request.header.size and 
hadoop.http.max.response.header.size | kms-site.xml
KMS_MAX_THREADS  | hadoop.http.max.threads  | kms-site.xml
KMS_SSL_ENABLED  | hadoop.kms.ssl.enabled   | kms-site.xml
KMS_SSL_KEYSTORE_FILE| ssl.server.keystore.location | ssl-server.xml
KMS_SSL_KEYSTORE_PASS| ssl.server.keystore.password | ssl-server.xml
KMS_TEMP | hadoop.http.temp.dir | kms-site.xml

These default HTTP Services have been added.

Name   | Description
---|
/conf  | Display configuration properties
/jmx   | Java JMX management interface
/logLevel  | Get or set log level per class
/logs  | Display log files
/stacks| Display JVM stacks
/static/index.html | The static home page

Script kms.sh has been deprecated, use 'hadoop kms' instead. Conform to the 
Hadoop shell scripting framework. Support 'hadoop daemonlog'. Read SSL 
configurations from ssl-server.xml, like many other Hadoop components.


> Switch KMS from Tomcat to Jetty
> ---
>
> Key: HADOOP-13597
> URL: https://issues.apache.org/jira/browse/HADOOP-13597
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: kms
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13597.001.patch, HADOOP-13597.002.patch, 
> HADOOP-13597.003.patch, HADOOP-13597.004.patch, HADOOP-13597.005.patch, 
> HADOOP-13597.006.patch, HADOOP-13597.007.patch
>
>
> The Tomcat 6 we are using will reach EOL at the end of 2017. While there are 
> other good options, I would propose switching to {{Jetty 9}} for the 
> following reasons:
> * Easier migration. Both Tomcat and Jetty are based on {{Servlet 
> Containers}}, so we don't have change client code that much. It would require 
> more work to switch to {{JAX-RS}}.
> * Well established.
> * Good performance and scalability.
> Other alternatives:
> * Jersey + Grizzly
> * Tomcat 8
> Your opinions will be greatly appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-13597) Switch KMS from Tomcat to Jetty

[John Zhuge (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Zhuge updated HADOOP-13597:

Release Note: 
The following environment variables are deprecated. Set the corresponding
configuration properties instead.

Environment Variable | Configuration Property   | Configuration File
-|--|
KMS_HTTP_PORT| hadoop.kms.http.port | kms-site.xml
KMS_MAX_HTTP_HEADER_SIZE | hadoop.http.max.request.header.size and 
hadoop.http.max.response.header.size | kms-site.xml
KMS_MAX_THREADS  | hadoop.http.max.threads  | kms-site.xml
KMS_SSL_ENABLED  | hadoop.kms.ssl.enabled   | kms-site.xml
KMS_SSL_KEYSTORE_FILE| ssl.server.keystore.location | ssl-server.xml
KMS_SSL_KEYSTORE_PASS| ssl.server.keystore.password | ssl-server.xml
KMS_TEMP | hadoop.http.temp.dir | kms-site.xml

These default HTTP Services have been added.

Name   | Description
---|
/conf  | Display configuration properties
/jmx   | Java JMX management interface
/logLevel  | Get or set log level per class
/logs  | Display log files
/stacks| Display JVM stacks
/static/index.html | The static home page

Script kms.sh has been deprecated, use 'hadoop kms' instead. Conform to the 
Hadoop shell scripting framework. Support 'hadoop daemonlog'. Read SSL 
configurations from ssl-server.xml, like many other Hadoop components.

> Switch KMS from Tomcat to Jetty
> ---
>
> Key: HADOOP-13597
> URL: https://issues.apache.org/jira/browse/HADOOP-13597
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: kms
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13597.001.patch, HADOOP-13597.002.patch, 
> HADOOP-13597.003.patch, HADOOP-13597.004.patch, HADOOP-13597.005.patch, 
> HADOOP-13597.006.patch, HADOOP-13597.007.patch
>
>
> The Tomcat 6 we are using will reach EOL at the end of 2017. While there are 
> other good options, I would propose switching to {{Jetty 9}} for the 
> following reasons:
> * Easier migration. Both Tomcat and Jetty are based on {{Servlet 
> Containers}}, so we don't have change client code that much. It would require 
> more work to switch to {{JAX-RS}}.
> * Well established.
> * Good performance and scalability.
> Other alternatives:
> * Jersey + Grizzly
> * Tomcat 8
> Your opinions will be greatly appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-13597) Switch KMS from Tomcat to Jetty

[Xiao Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiao Chen updated HADOOP-13597:
---
   Resolution: Fixed
 Hadoop Flags: Incompatible change,Reviewed  (was: Incompatible change)
Fix Version/s: 3.0.0-alpha2
   Status: Resolved  (was: Patch Available)

Committed to trunk.

Thanks [~jzhuge] for the contribution, and all reviewers for the discussion and 
review!

John, could you also add a release note to the jira?

> Switch KMS from Tomcat to Jetty
> ---
>
> Key: HADOOP-13597
> URL: https://issues.apache.org/jira/browse/HADOOP-13597
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: kms
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13597.001.patch, HADOOP-13597.002.patch, 
> HADOOP-13597.003.patch, HADOOP-13597.004.patch, HADOOP-13597.005.patch, 
> HADOOP-13597.006.patch, HADOOP-13597.007.patch
>
>
> The Tomcat 6 we are using will reach EOL at the end of 2017. While there are 
> other good options, I would propose switching to {{Jetty 9}} for the 
> following reasons:
> * Easier migration. Both Tomcat and Jetty are based on {{Servlet 
> Containers}}, so we don't have change client code that much. It would require 
> more work to switch to {{JAX-RS}}.
> * Well established.
> * Good performance and scalability.
> Other alternatives:
> * Jersey + Grizzly
> * Tomcat 8
> Your opinions will be greatly appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-13597) Switch KMS from Tomcat to Jetty

[John Zhuge (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Zhuge updated HADOOP-13597:

Attachment: HADOOP-13597.007.patch

Patch 007
- Update CommandsManual.md and findbugsExcludeFile.xml

TESTING DONE
- KMS Bats regression tests https://github.com/jzhuge/hadoop-regression-tests 
in insecure and ssl mode
- Verify CommandsManual.html
- /jmx, /logLevel, /conf, /stack, /logs, and /static/index.html

> Switch KMS from Tomcat to Jetty
> ---
>
> Key: HADOOP-13597
> URL: https://issues.apache.org/jira/browse/HADOOP-13597
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: kms
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
> Attachments: HADOOP-13597.001.patch, HADOOP-13597.002.patch, 
> HADOOP-13597.003.patch, HADOOP-13597.004.patch, HADOOP-13597.005.patch, 
> HADOOP-13597.006.patch, HADOOP-13597.007.patch
>
>
> The Tomcat 6 we are using will reach EOL at the end of 2017. While there are 
> other good options, I would propose switching to {{Jetty 9}} for the 
> following reasons:
> * Easier migration. Both Tomcat and Jetty are based on {{Servlet 
> Containers}}, so we don't have change client code that much. It would require 
> more work to switch to {{JAX-RS}}.
> * Well established.
> * Good performance and scalability.
> Other alternatives:
> * Jersey + Grizzly
> * Tomcat 8
> Your opinions will be greatly appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-13597) Switch KMS from Tomcat to Jetty

[John Zhuge (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Zhuge updated HADOOP-13597:

Attachment: HADOOP-13597.006.patch

Patch 006
- Xiao's comments

TESTING DONE
- Automated regression tests for KMS and HttpFS 
https://github.com/jzhuge/hadoop-regression-tests in unsecure and ssl mode
- View docs

> Switch KMS from Tomcat to Jetty
> ---
>
> Key: HADOOP-13597
> URL: https://issues.apache.org/jira/browse/HADOOP-13597
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: kms
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
> Attachments: HADOOP-13597.001.patch, HADOOP-13597.002.patch, 
> HADOOP-13597.003.patch, HADOOP-13597.004.patch, HADOOP-13597.005.patch, 
> HADOOP-13597.006.patch
>
>
> The Tomcat 6 we are using will reach EOL at the end of 2017. While there are 
> other good options, I would propose switching to {{Jetty 9}} for the 
> following reasons:
> * Easier migration. Both Tomcat and Jetty are based on {{Servlet 
> Containers}}, so we don't have change client code that much. It would require 
> more work to switch to {{JAX-RS}}.
> * Well established.
> * Good performance and scalability.
> Other alternatives:
> * Jersey + Grizzly
> * Tomcat 8
> Your opinions will be greatly appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-13597) Switch KMS from Tomcat to Jetty

[John Zhuge (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Zhuge updated HADOOP-13597:

Attachment: HADOOP-13597.005.patch

Patch 005
- Set {{HttpServer2.Builder#authFilterConfigurationPrefix}} to integrate with 
HttpServer2’s secret provider
- Rename {{AuthenticationFilter#isInitializedByTomcat}} to 
{{destroySecretProvider}}
- Update docs: index.md.vm and CommandsManual.md 

TESTING DONE
- hadoop key list/create/delete/roll in non-secure and SSL setup
- hadoop —daemon start|stop kms
- kms.sh run
- hadoop daemonlog
- KMS unit tests
- dist-test for module hadoop-common and hadoop-hdfs: 
http://dist-test.cloudera.org/job?job_id=hadoop.jzhuge.1481879379.20252, 10 
unrelated test failures.
- Shell script unit tests
- /jmx, /logLevel, /conf, /stack, /logs, and /static/index.html

TODO in new JIRAs:
- Update doc CredentialProviderAPI.md and SecureMode.md for both KMS and HttpFS
- HADOOP-13875: Full SSL server configuration: 
includeProtocols/excludeProtocols/includeCipherSuites/excludeCipherSuites, etc.
- Share web apps code in Common, HDFS, and YARN

[~aw], [~xiaochen], [~jojochuang], [~rkanter], please review. I believe all 
review comments are either addressed or to be followed up in future JIRAs. Let 
me know otherwise.

> Switch KMS from Tomcat to Jetty
> ---
>
> Key: HADOOP-13597
> URL: https://issues.apache.org/jira/browse/HADOOP-13597
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: kms
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
> Attachments: HADOOP-13597.001.patch, HADOOP-13597.002.patch, 
> HADOOP-13597.003.patch, HADOOP-13597.004.patch, HADOOP-13597.005.patch
>
>
> The Tomcat 6 we are using will reach EOL at the end of 2017. While there are 
> other good options, I would propose switching to {{Jetty 9}} for the 
> following reasons:
> * Easier migration. Both Tomcat and Jetty are based on {{Servlet 
> Containers}}, so we don't have change client code that much. It would require 
> more work to switch to {{JAX-RS}}.
> * Well established.
> * Good performance and scalability.
> Other alternatives:
> * Jersey + Grizzly
> * Tomcat 8
> Your opinions will be greatly appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-13597) Switch KMS from Tomcat to Jetty

[John Zhuge (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Zhuge updated HADOOP-13597:

Attachment: HADOOP-13597.004.patch

Patch 004
- Add unit tests for shell scripts
- Add {{HttpServer2}} Builder API {{setSSLConf}} to provide an alternative to 
keyStore/keyPassword/trustStore APIs.
- Revert skipSecretProvider and setContextPath
- Revert {{Configuration#getPasswordString}}
- Remove {{AuthenticationFilter#isInitializedByTomcat}} and related code
- Rename {{AccessLoggingConfiguration}} to {{ConfigurationWithLogging}}

TESTING DONE
- hadoop key list/create/delete/roll in non-secure and SSL setup
- hadoop kms
- hadoop —daemon start|status|stop kms
- kms.sh run|start|status|stop
- hadoop daemonlog
- KMS unit tests
- Shell script unit tests
- /jmx, /logLevel, /conf, /stack, /logs, and /static/index.html

TODO
- Update docs: index.md.vm, CommandsManual.md, and SecureMode.md

TODO in new JIRAs:
- HADOOP-13875: Full SSL server configuration: 
includeProtocols/excludeProtocols/includeCipherSuites/excludeCipherSuites, etc.
- Share web apps code in Common, HDFS, and YARN

Private branch: https://github.com/jzhuge/hadoop/tree/HADOOP-13597.004

> Switch KMS from Tomcat to Jetty
> ---
>
> Key: HADOOP-13597
> URL: https://issues.apache.org/jira/browse/HADOOP-13597
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: kms
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
> Attachments: HADOOP-13597.001.patch, HADOOP-13597.002.patch, 
> HADOOP-13597.003.patch, HADOOP-13597.004.patch
>
>
> The Tomcat 6 we are using will reach EOL at the end of 2017. While there are 
> other good options, I would propose switching to {{Jetty 9}} for the 
> following reasons:
> * Easier migration. Both Tomcat and Jetty are based on {{Servlet 
> Containers}}, so we don't have change client code that much. It would require 
> more work to switch to {{JAX-RS}}.
> * Well established.
> * Good performance and scalability.
> Other alternatives:
> * Jersey + Grizzly
> * Tomcat 8
> Your opinions will be greatly appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-13597) Switch KMS from Tomcat to Jetty

[John Zhuge (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Zhuge updated HADOOP-13597:

Attachment: HADOOP-13597.003.patch

Patch 003
- Add dependencySets to pom.xml
- Add shell script hadoop_using_envvar and hadoop_mkdir
- Extend hadoop_deprecate_envvar
- Enhance AccessLoggingConfiguration to inherit from Configuration
- Move the configuration keys out of SSLConfig and HttpConfig
- HttpServer2 reads tempDirectory and header sizes from configuration instead 
of builder

TESTING DONE
- Run “hadoop key list/create/delete/roll” in non-secure and SSL setup
- All KMS unit tests that actually exercise the full-blown KMS
- Script: hadoop kms, hadoop —daemon start|status|stop kms
- Script: kms.sh run|start|status|stop
- /kms/jmx, /kms/logLevel, /kms/conf, /kms/stack, /logs, and /static

TODO
- Update docs: index.md.vm

TODO in new JIRAs:
- Full SSL server configuration: 
includeProtocols/excludeProtocols/includeCipherSuites/excludeCipherSuites, etc.
- Share web apps code in Common, HDFS, and YARN

> Switch KMS from Tomcat to Jetty
> ---
>
> Key: HADOOP-13597
> URL: https://issues.apache.org/jira/browse/HADOOP-13597
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: kms
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
> Attachments: HADOOP-13597.001.patch, HADOOP-13597.002.patch, 
> HADOOP-13597.003.patch
>
>
> The Tomcat 6 we are using will reach EOL at the end of 2017. While there are 
> other good options, I would propose switching to {{Jetty 9}} for the 
> following reasons:
> * Easier migration. Both Tomcat and Jetty are based on {{Servlet 
> Containers}}, so we don't have change client code that much. It would require 
> more work to switch to {{JAX-RS}}.
> * Well established.
> * Good performance and scalability.
> Other alternatives:
> * Jersey + Grizzly
> * Tomcat 8
> Your opinions will be greatly appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-13597) Switch KMS from Tomcat to Jetty

[John Zhuge (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Zhuge updated HADOOP-13597:

Attachment: HADOOP-13597.002.patch

Patch 002
- Add “hadoop kms” sub-command. kms.sh is now just a wrapper.
- Read SSL configuration from ssl-server.xml
- Put common SSL config keys in SSLConfig
- Put common HTTP config keys in HttpConfig
- Support all deprecated environment variables and give warning of deprecation
- Enhanced web page /static/index.html, not /index.html due to HttpServer2 
limitation

TESTING DONE
- Run “hadoop key list/create/delete/roll” in non-secure and SSL setup
- All KMS unit tests that actually exercise the full-blown KMS
- Script: hadoop kms, hadoop —daemon start|status|stop kms
- Script: kms.sh run|start|status|stop
- /kms/jmx, /kms/logLevel, /kms/conf, /kms/stack, /logs, and /static

TODO
- Integration testing
- Update docs: index.md.vm

TODO in new JIRAs:
- Full SSL server configuration: 
includeProtocols/excludeProtocols/includeCipherSuites/excludeCipherSuites, etc.
- Share web apps code in Common, HDFS, and YARN


> Switch KMS from Tomcat to Jetty
> ---
>
> Key: HADOOP-13597
> URL: https://issues.apache.org/jira/browse/HADOOP-13597
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: kms
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
> Attachments: HADOOP-13597.001.patch, HADOOP-13597.002.patch
>
>
> The Tomcat 6 we are using will reach EOL at the end of 2017. While there are 
> other good options, I would propose switching to {{Jetty 9}} for the 
> following reasons:
> * Easier migration. Both Tomcat and Jetty are based on {{Servlet 
> Containers}}, so we don't have change client code that much. It would require 
> more work to switch to {{JAX-RS}}.
> * Well established.
> * Good performance and scalability.
> Other alternatives:
> * Jersey + Grizzly
> * Tomcat 8
> Your opinions will be greatly appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-13597) Switch KMS from Tomcat to Jetty

[John Zhuge (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Zhuge updated HADOOP-13597:

Status: Patch Available  (was: In Progress)

> Switch KMS from Tomcat to Jetty
> ---
>
> Key: HADOOP-13597
> URL: https://issues.apache.org/jira/browse/HADOOP-13597
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: kms
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
> Attachments: HADOOP-13597.001.patch
>
>
> The Tomcat 6 we are using will reach EOL at the end of 2017. While there are 
> other good options, I would propose switching to {{Jetty 9}} for the 
> following reasons:
> * Easier migration. Both Tomcat and Jetty are based on {{Servlet 
> Containers}}, so we don't have change client code that much. It would require 
> more work to switch to {{JAX-RS}}.
> * Well established.
> * Good performance and scalability.
> Other alternatives:
> * Jersey + Grizzly
> * Tomcat 8
> Your opinions will be greatly appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-13597) Switch KMS from Tomcat to Jetty

[John Zhuge (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Zhuge updated HADOOP-13597:

Attachment: HADOOP-13597.001.patch

Patch 001
- KMSHttpServer based on HttpServer2
- Redirect MiniKMS to KMSHttpServer
- Add kms-default.xml
- Add Jetty properties including SSL properties
- Convert hadoop-kms from war to jar
- Rewrite kms.sh to use Hadoop shell script framework

TESTING DONE
- All hadoop-kms unit tests which exercise the full KMS instead of MiniKMS
- Non-secure REST APIs
- Non-secure “hadoop key” commands
- SSL REST APIs
- kms.sh run/start/stop/status

TODO
- Still need admin port?
- Still need /index.html?
- More ad-hoc testing
- Integration testing
- Update docs: index.md.vm

TODO in new JIRAs:
- Integrate with Hadoop SSL server configuration
- Full SSL server configuration: 
includeProtocols/excludeProtocols/includeCipherSuites/excludeCipherSuites, etc.
- Design common Http server configuration. Common properties in 
“-site.xml” with config prefix, e.g., “hadoop.kms.”.
- Design HttpServer2 configuration-based builder
- JMX not working, existing issue
- Share web apps code in Common, HDFS, and YARN

My private branch: https://github.com/jzhuge/hadoop/tree/HADOOP-13597.001

> Switch KMS from Tomcat to Jetty
> ---
>
> Key: HADOOP-13597
> URL: https://issues.apache.org/jira/browse/HADOOP-13597
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: kms
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
> Attachments: HADOOP-13597.001.patch
>
>
> The Tomcat 6 we are using will reach EOL at the end of 2017. While there are 
> other good options, I would propose switching to {{Jetty 9}} for the 
> following reasons:
> * Easier migration. Both Tomcat and Jetty are based on {{Servlet 
> Containers}}, so we don't have change client code that much. It would require 
> more work to switch to {{JAX-RS}}.
> * Well established.
> * Good performance and scalability.
> Other alternatives:
> * Jersey + Grizzly
> * Tomcat 8
> Your opinions will be greatly appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-13597) Switch KMS from Tomcat to Jetty

[John Zhuge (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Zhuge updated HADOOP-13597:

Hadoop Flags: Incompatible change

> Switch KMS from Tomcat to Jetty
> ---
>
> Key: HADOOP-13597
> URL: https://issues.apache.org/jira/browse/HADOOP-13597
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: kms
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
>
> The Tomcat 6 we are using will reach EOL at the end of 2017. While there are 
> other good options, I would propose switching to {{Jetty 9}} for the 
> following reasons:
> * Easier migration. Both Tomcat and Jetty are based on {{Servlet 
> Containers}}, so we don't have change client code that much. It would require 
> more work to switch to {{JAX-RS}}.
> * Well established.
> * Good performance and scalability.
> Other alternatives:
> * Jersey + Grizzly
> * Tomcat 8
> Your opinions will be greatly appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-13597) Switch KMS from Tomcat to Jetty

[John Zhuge (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Zhuge updated HADOOP-13597:

Summary: Switch KMS from Tomcat to Jetty  (was: Switch KMS to use Jetty)

> Switch KMS from Tomcat to Jetty
> ---
>
> Key: HADOOP-13597
> URL: https://issues.apache.org/jira/browse/HADOOP-13597
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: kms
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
>
> The Tomcat 6 we are using will reach EOL at the end of 2017. While there are 
> other good options, I would propose switching to {{Jetty 9}} for the 
> following reasons:
> * Easier migration. Both Tomcat and Jetty are based on {{Servlet 
> Containers}}, so we don't have change client code that much. It would require 
> more work to switch to {{JAX-RS}}.
> * Well established.
> * Good performance and scalability.
> Other alternatives:
> * Jersey + Grizzly
> * Tomcat 8
> Your opinions will be greatly appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org