[
https://issues.apache.org/jira/browse/KNOX-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16204117#comment-16204117
]
Phil Zampino edited comment on KNOX-1010 at 10/13/17 8:17 PM:
--------------------------------------------------------------
Attached KNOX-1010.patch
*N.B. This is only the MONITORING piece of this functionality. The mechanism(s)
for populating the ZooKeeper znodes is another JIRA.*
You can review the tests to see how it works:
#
gateway-server/src/test/java/org/apache/hadoop/gateway/topology/monitor/zk/ZooKeeperConfigMonitorTest.java
Exercises the ZooKeeperConfigMonitor WITHOUT any znode ACLs or client
authentication.
#
gateway-test/src/test/java/org/apache/hadoop/gateway/topology/monitor/RemoteConfigurationMonitorTest.java
Exercises the ZooKeeperConfigMonitor WITH znode ACLs and associated SASL client
authentication.
To manually test:
# Ensure access to a ZooKeeper host or ensemble (e.g. sandbox works)
# Ensure access to ZooKeeper client
# Create znodes (/knox/config/shared-providers, /knox/config/descriptors)
# Add the remote monitor address property to gateway-site.xml:
gateway.remote.config.monitor.address=localhost:2181 (or whatever your
zookeeper connection string is)
# Start the gateway
# Add a provider config to the shared-providers znode (e.g., attached
sandbox-providers.xml)
(You can use the attached zkupload script to make this easier: zkupload.sh
localhost:2181 create /knox/config/shared-providers/sandbox-providers.xml
./sandbox-providers.xml)
Check the gateway log for message about the config file being downloaded
Check the local conf/shared-providers for your config file
# Add a descriptor to the descriptors znode (e.g., attached docker-sandbox.json)
(You can use the attached zkupload script to make this easier: zkupload.sh
localhost:2181 create /knox/config/descriptors/docker-sandbox.json
./docker-sandbox.json)
Check the gateway log for message about the descriptor file being downloaded
Check the local conf/shared-providers for your descriptor
# Delete the descriptors child znode for the descriptor you added (e.g.,
zkCli.sh delete /knox/shared-providers/sandbox-providers.xml)
Note the gateway log message about the descriptor being deleted
Note that the descriptor has been deleted from conf/descriptors
If your descriptor had resulted in a successful topology deployment, then
you'll also notice the undeployment/removal of the topology)
# Delete the shared-providers child znode for the provider config you added
(e.g., zkCli.sh delete /knox/descriptors/docker-sandbox.json)
Note the gateway log message about the config file being deleted
Note that the provider config has been deleted from conf/shared-providers
If you want to test with SASL auth and ACLs, you should ask me, and we can
discuss it together.
was (Author: pzampino):
Attached KNOX-1010.patch
You can review the tests to see how it works:
#
gateway-server/src/test/java/org/apache/hadoop/gateway/topology/monitor/zk/ZooKeeperConfigMonitorTest.java
Exercises the ZooKeeperConfigMonitor WITHOUT any znode ACLs or client
authentication.
#
gateway-test/src/test/java/org/apache/hadoop/gateway/topology/monitor/RemoteConfigurationMonitorTest.java
Exercises the ZooKeeperConfigMonitor WITH znode ACLs and associated SASL client
authentication.
To manually test:
# Ensure access to a ZooKeeper host or ensemble (e.g. sandbox works)
# Ensure access to ZooKeeper client
# Create znodes (/knox/config/shared-providers, /knox/config/descriptors)
# Add the remote monitor address property to gateway-site.xml:
gateway.remote.config.monitor.address=localhost:2181 (or whatever your
zookeeper connection string is)
# Start the gateway
# Add a provider config to the shared-providers znode (e.g., attached
sandbox-providers.xml)
(You can use the attached zkupload script to make this easier: zkupload.sh
localhost:2181 create /knox/config/shared-providers/sandbox-providers.xml
./sandbox-providers.xml)
Check the gateway log for message about the config file being downloaded
Check the local conf/shared-providers for your config file
# Add a descriptor to the descriptors znode (e.g., attached docker-sandbox.json)
(You can use the attached zkupload script to make this easier: zkupload.sh
localhost:2181 create /knox/config/descriptors/docker-sandbox.json
./docker-sandbox.json)
Check the gateway log for message about the descriptor file being downloaded
Check the local conf/shared-providers for your descriptor
# Delete the descriptors child znode for the descriptor you added (e.g.,
zkCli.sh delete /knox/shared-providers/sandbox-providers.xml)
Note the gateway log message about the descriptor being deleted
Note that the descriptor has been deleted from conf/descriptors
If your descriptor had resulted in a successful topology deployment, then
you'll also notice the undeployment/removal of the topology)
# Delete the shared-providers child znode for the provider config you added
(e.g., zkCli.sh delete /knox/descriptors/docker-sandbox.json)
Note the gateway log message about the config file being deleted
Note that the provider config has been deleted from conf/shared-providers
If you want to test with SASL auth and ACLs, you should ask me, and we can
discuss it together.
> Remote Discovery of Knox Topology Configuration
> -----------------------------------------------
>
> Key: KNOX-1010
> URL: https://issues.apache.org/jira/browse/KNOX-1010
> Project: Apache Knox
> Issue Type: Sub-task
> Components: Server
> Reporter: Phil Zampino
> Assignee: Phil Zampino
> Labels: kip-8
> Fix For: 0.14.0
>
> Attachments: KNOX-1010.patch, docker-sandbox.json,
> sandbox-providers.xml, zkupload.sh
>
>
> To support HA deployments, Knox should be able to discover simple topology
> descriptors and provider configuration remotely.
> - Define the ZooKeeper structure for remote config (simple desc, externalized
> provider) discovery
> - Determine the best way to interact with ZooKeeper (REST, or some other
> client binding)
> - Simple descriptor discovery
> - External provider config discovery
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
