smolnar82 opened a new pull request #285: KNOX-2284 - Handling CM descriptors after Knox shared-provider/descriptor/topology monitors are started URL: https://github.com/apache/knox/pull/285 ## What changes were proposed in this pull request? Handling CM descriptors after Knox shared-provider/descriptor/topology monitors are started ## How was this patch tested? Tested manually: 1. redeployed Knox with my changes in place; brand new installation, only the OOTB topologies existed 2. produced a CM descriptor (cm-topologies.cm) with the following content: ``` <configuration> <property> <name>topology1</name> <value> providerConfigRef=default-providers; app:knoxauth:param1.name=param1.value; HIVE:url=http://localhost:456; HIVE:version=1.0; HIVE:httpclient.connectionTimeout=5m; HIVE:httpclient.socketTimeout=100m </value> </property> <property> <name>topology2</name> <value> providerConfigRef=default-providers; HDFS:url=http://localhost:456; HDFS:httpclient.connectionTimeout=5m; HDFS:httpclient.socketTimeout=100m </value> </property> </configuration> ``` 3. started Knox and confirmed (from logs and using the Admin UI): - `$KNOX_HOME/conf/descriptors/topology1.json` and `$KNOX_HOME/conf/descriptors/topology2.json` got persisted - Knox picked them up and generated `$KNOX_HOME/data/deployments/topology1.topo.170c48f42c0` and `$KNOX_HOME/data/deployments/topology2.topo.170c48f42c0` Relevant logs: ``` 2020-03-10 14:07:50,348 INFO knox.gateway (DefaultTopologyService.java:init(648)) - Monitoring simple descriptors in directory: /Users/smolnar/test/knoxGateway/conf/descriptors 2020-03-10 14:07:50,577 INFO knox.gateway (DefaultTopologyService.java:init(653)) - Monitoring shared provider configurations in directory: /Users/smolnar/test/knoxGateway/conf/shared-providers ... 2020-03-10 14:08:02,899 INFO knox.gateway (GatewayServer.java:start(700)) - Monitoring topologies in directory: /Users/smolnar/test/knoxGateway/conf/topologies 2020-03-10 14:08:02,920 INFO discovery.advanced (AdvancedServiceDiscoveryConfigurationMonitor.java:setupMonitor(73)) - Monitoring /Users/smolnar/test/knoxGateway/conf/auto-discovery-advanced-configuration-* for advanced service discovery configuration changes. 2020-03-10 14:08:02,925 INFO knox.gateway (ClouderaManagerDescriptorMonitor.java:setupMonitor(66)) - Monitoring Cloudera Manager descriptors in /Users/smolnar/test/knoxGateway/conf/descriptors ... 2020-03-10 14:08:02,925 INFO knox.gateway (ClouderaManagerDescriptorParser.java:parse(79)) - Parsing Cloudera Manager descriptor /Users/smolnar/test/knoxGateway/conf/descriptors/cm-topologies.cm. Looking up all topologies... 2020-03-10 14:08:02,925 INFO knox.gateway (GatewayServer.java:startGateway(392)) - Started gateway on port 8,443. 2020-03-10 14:08:02,936 INFO knox.gateway (ClouderaManagerDescriptorParser.java:parse(84)) - Found Knox descriptors topology1, topology2 in /Users/smolnar/test/knoxGateway/conf/descriptors/cm-topologies.cm 2020-03-10 14:08:03,125 INFO knox.gateway (ClouderaManagerDescriptorMonitor.java:lambda$1(100)) - Saved Knox descriptor /Users/smolnar/test/knoxGateway/conf/descriptors/topology1.json 2020-03-10 14:08:03,125 INFO knox.gateway (ClouderaManagerDescriptorMonitor.java:lambda$1(100)) - Saved Knox descriptor /Users/smolnar/test/knoxGateway/conf/descriptors/topology2.json 2020-03-10 14:08:08,011 INFO knox.gateway (DefaultTopologyService.java:onFileChange(872)) - Generated topology topology1.xml because the associated descriptor topology1.json changed. ... 2020-03-10 14:08:14,278 INFO knox.gateway (GatewayServer.java:handleCreateDeployment(1013)) - Deploying topology topology1 to /Users/smolnar/test/knoxGateway/data/deployments/topology1.topo.170c48f42c0 2020-03-10 14:08:14,278 INFO knox.gateway (GatewayServer.java:internalDeactivateTopology(932)) - Deactivating topology topology1 2020-03-10 14:08:14,450 INFO knox.gateway (DefaultGatewayServices.java:initializeContribution(161)) - Creating credential store for the cluster: topology1 2020-03-10 14:08:14,753 INFO knox.gateway (GatewayServer.java:internalActivateTopology(898)) - Activating topology topology1 2020-03-10 14:08:14,753 INFO knox.gateway (GatewayServer.java:internalActivateArchive(908)) - Activating topology topology1 archive %2Fknoxauth 2020-03-10 14:08:14,833 INFO knox.gateway (GatewayServer.java:handleCreateDeployment(1013)) - Deploying topology topology2 to /Users/smolnar/test/knoxGateway/data/deployments/topology2.topo.170c48f42c0 2020-03-10 14:08:14,833 INFO knox.gateway (GatewayServer.java:internalDeactivateTopology(932)) - Deactivating topology topology2 2020-03-10 14:08:14,940 INFO knox.gateway (GatewayServer.java:internalActivateTopology(898)) - Activating topology topology2 ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
