GitHub user cestella reopened a pull request: https://github.com/apache/incubator-metron/pull/495
METRON-797: Pass security.protocol and enable auto-renew for the storm topologies ## Contributor Comments METRON-793 migrated the storm topologies to the storm-kafka-client spout, which supports kerberos. To complete the kerberos work on the existing topologies, we need to be able to enable the spouts and kafka writers to use security protocols other than PLAINTEXT. Also, enabling auto-renew plugins for storm will enable the topologies to run for extended durations in a kerberized cluster. NOTE: This does not encompass MPack changes to enable kerberos or fix the sensors to work with a kerberized kafka. That would be follow-on work. This work was inspired by a portion of the investigatory work done at https://github.com/dlyle65535/incubator-metron/tree/kerb-testing?files=1 by: * @dlyle65535 * @merrimanr * @mmiklavc This carves out a specific piece of that functionality with the following differences: The mpack work is not included, but the properties are set up to enable it as a follow-on It presumes METRON-793, so it uses storm-kafka-client rather than storm-kafka It adds a flag when starting the parsers to pass the security protocol and sets up the writers and the spout automatically rather than relying on the set of extra kafka configs (though both approaches would work here). ## Pull Request Checklist Thank you for submitting a contribution to Apache Metron (Incubating). Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions. Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides. In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? ### For code changes: - [x] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [x] Have you included steps or a guide to how the change may be verified and tested manually? - [x] Have you ensured that the full suite of tests and checks have been executed in the root incubating-metron folder via: ``` mvn -q clean integration-test install && build_utils/verify_licenses.sh ``` - [x] Have you written or updated unit tests and or integration tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`: ``` cd site-book bin/generate-md.sh mvn site:site ``` #### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. It is also recommened that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request. You can merge this pull request into a Git repository by running: $ git pull https://github.com/cestella/incubator-metron METRON-797-merged Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-metron/pull/495.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #495 ---- commit dae102b0228b969d4e685a81dd6df25e59f63cb5 Author: cstella <ceste...@gmail.com> Date: 2017-03-29T12:17:52Z METRON-797: Pass security.protocol and enable auto-renew for the storm topologies commit e03636d8d3fa18bbd3ed53cd55e66b81eb84603d Author: justinleet <justinjl...@gmail.com> Date: 2017-03-29T13:16:34Z METRON-797: Pass security.protocol and enable auto-renew for the storm topologies commit 440fa1fb1db6b70699c0ca5cee5c602042543bd4 Author: mmiklavc <michael.miklav...@gmail.com> Date: 2017-03-29T13:23:09Z METRON-797: Pass security.protocol and enable auto-renew for the storm topologies ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---