> On Nov. 10, 2014, 3:48 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostComponentLogResourceDefinition.java, > > line 36 > > <https://reviews.apache.org/r/27802/diff/2/?file=756631#file756631line36> > > > > You might want to make these names more unique since they are for host > > components.
Renamed to "host_components_logs" > On Nov. 10, 2014, 3:48 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/api/services/HostComponentService.java, > > line 111 > > <https://reviews.apache.org/r/27802/diff/2/?file=756635#file756635line111> > > > > These should not be on individual lines since they easily fit next to > > each other. This is due to auto formatting. Fixed. > On Nov. 10, 2014, 3:48 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/api/services/HostComponentService.java, > > line 317 > > <https://reviews.apache.org/r/27802/diff/2/?file=756635#file756635line317> > > > > downlaod (spelling) Good catch. Thanks. > On Nov. 10, 2014, 3:48 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/api/services/HostComponentService.java, > > line 401 > > <https://reviews.apache.org/r/27802/diff/2/?file=756635#file756635line401> > > > > Are these other types needed if you're just returning the stream to the > > HDFS log file? It is not required. Removing it. > On Nov. 10, 2014, 3:48 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/api/services/HostComponentService.java, > > line 414 > > <https://reviews.apache.org/r/27802/diff/2/?file=756635#file756635line414> > > > > Instead of trying to request "HostComponentLog:1" by key, why not just > > request only the Resource.Type.HostComponentLog instead of the other > > Resource.Types above and then use the first and only child? Fixed. > On Nov. 10, 2014, 3:48 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/api/services/HostComponentService.java, > > line 427 > > <https://reviews.apache.org/r/27802/diff/2/?file=756635#file756635line427> > > > > That's an odd application type; is this the correct one to use for this > > web request? Changed to 'application/octet-stream' > On Nov. 10, 2014, 3:48 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java, > > line 209 > > <https://reviews.apache.org/r/27802/diff/2/?file=756636#file756636line209> > > > > Should these HDFS property names belong here if they aren't actually > > retrieved via the Configuration instance? Moved these constants to HostComponentLogResourceProvider class. > On Nov. 10, 2014, 3:48 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/HostComponentLogResponse.java, > > line 61 > > <https://reviews.apache.org/r/27802/diff/2/?file=756637#file756637line61> > > > > I personally hate the use of "this." when assiging members in a > > constructor/method. However if the class already does it I try to be > > consistent. In this case, the constructor has a mixture of both. Fixed. > On Nov. 10, 2014, 3:48 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentLogResourceProvider.java, > > line 76 > > <https://reviews.apache.org/r/27802/diff/2/?file=756639#file756639line76> > > > > These are not exposed in other places, are they? If not, no need to be > > public. Made it as protected. > On Nov. 10, 2014, 3:48 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentLogResourceProvider.java, > > line 262 > > <https://reviews.apache.org/r/27802/diff/2/?file=756639#file756639line262> > > > > This method will fail in HA mode for HDFS; in this mode, the http and > > https properties are scoped by two more parameters, such as > > hdfs-site/dfs.nameservices and hdfs-site/dfs.X.namenodes.Y where X and Y > > are parameterized namenode aliases. > > > > Example: > > dfs.namenode.http-address.hacluster.nn1 Thanks very much for pointing this. Somehow i missed this scenario. This is handled in the new patch now. After talking with Nicolos (webHdfs team), he suggested to use WebHdfsFileSystem. Next patch will use WebHdfsFileSystem which will automatically handle the fail-over. - Cabir ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27802/#review60602 ----------------------------------------------------------- On Nov. 10, 2014, 9:04 a.m., Cabir Zounaidou wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/27802/ > ----------------------------------------------------------- > > (Updated Nov. 10, 2014, 9:04 a.m.) > > > Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, John > Speidel, Mahadev Konar, Nate Cole, Sid Wagle, and Yusaku Sako. > > > Bugs: AMBARI-4083 > https://issues.apache.org/jira/browse/AMBARI-4083 > > > Repository: ambari > > > Description > ------- > > The host component log implements the following two rest api's. > > 1. To retrieve the log entries. > /clusters/{cluster}/hosts/{hostname}/host_components/{component}/logs > This api will retrieve the logs if available for the host component from > HDFS. It can fetch only maximum of 5120 bytes. The window can be adjusted > using the query parameters 'offset' and 'length'. It also provides simple > filtering using 'level' query parameter. > Sample response will look like below: > { > "href" : > "http://c6501.ambari.apache.org:8080/api/v1/clusters/cl1/hosts/c6503.ambari.apache.org/host_components/HBASE_CLIENT/logs";, > "items" : [ > { > "href" : > "http://c6501.ambari.apache.org:8080/api/v1/clusters/cl1/hosts/c6503.ambari.apache.org/host_components/HBASE_CLIENT/logs/HBASE_CLIENT";, > "length" : 5120, > "level" : null, > "offset" : 153920, > "size" : 159040, > "HostComponentLog" : { > "cluster_name" : "cl1", > "component_name" : "HBASE_CLIENT", > "entries" : [ > { > "timestamp" : "2014-11-01 17:46:38,456", > "level" : "DEBUG", > "thread" : "main-EventThread", > "message" : "master.SplitLogManager: task not yet acquired > /hbase-unsecure/splitWAL/WALs%2Fc6503.ambari.apache.org%2C60020%2C1414856039721-splitting%2Fc6503.ambari.apache.org%252C60020%252C1414856039721.1414856049592.meta > ver = 0" > }, > { > "timestamp" : "2014-11-01 17:46:38,459", > "level" : "DEBUG", > "thread" : "main-EventThread", > "message" : "master.SplitLogManager: put up splitlog task at > znode > /hbase-unsecure/splitWAL/WALs%2Fc6503.ambari.apache.org%2C60020%2C1414856039721-splitting%2Fc6503.ambari.apache.org%252C60020%252C1414856039721.1414856050193.meta" > }, > { > "timestamp" : "2014-11-01 17:46:38,460", > "level" : "DEBUG", > "thread" : "main-EventThread", > "message" : "master.SplitLogManager: put up splitlog task at > znode > /hbase-unsecure/splitWAL/WALs%2Fc6503.ambari.apache.org%2C60020%2C1414856039721-splitting%2Fc6503.ambari.apache.org%252C60020%252C1414856039721.1414856105560.meta" > }, > : > : > }] > } > ] > } > 2. To download the log file from HDFS use the following the api > > /clusters/{cluster}/hosts/{hostname}/host_components/{component}/logs?download=true > The response will automatically download the file with the HDFS file name. > > > Diffs > ----- > > ambari-agent/src/main/python/ambari_agent/Controller.py dc3a1cf > > ambari-server/src/main/java/org/apache/ambari/server/agent/AgentCommand.java > e2f013d > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java > e99e39f > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatResponse.java > 56b4f18 > > ambari-server/src/main/java/org/apache/ambari/server/agent/LogConfigCommand.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/agent/RegistrationResponse.java > 8a24560 > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostComponentLogResourceDefinition.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostComponentResourceDefinition.java > 6dc9e2d > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java > 9ad37ec > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > bb4c569 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/HostComponentService.java > 4990ad7 > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > 4f53544 > > ambari-server/src/main/java/org/apache/ambari/server/controller/HostComponentLogResponse.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java > ae20f56 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentLogResourceProvider.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java > dbac906 > ambari-server/src/main/java/org/apache/ambari/server/state/LogConfig.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/state/LogConfigHandler.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/state/LogDefinition.java > PRE-CREATION > ambari-server/src/main/java/org/apache/ambari/server/state/LogEntry.java > PRE-CREATION > ambari-server/src/main/resources/key_properties.json c1a6636 > ambari-server/src/main/resources/log_handler_config.json PRE-CREATION > ambari-server/src/main/resources/properties.json 36cff96 > > ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java > e7b946d > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostComponentLogResourceProviderTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/27802/diff/ > > > Testing > ------- > > 1. Ran the unit tests. > 2. Manually ran the retrieve host component logs without any query parameters. > 3. Manually ran the retrieve host component logs with offset and length. > 4. Manually ran the retrieve host component logs with level filter. > 5. Manually ran the retrieve host component logs with webHDFS disabled. > 6. Manually ran the download host component log file with webHDFS enabled. > 7. Manually ran the download host component log file with webHDFS disabled. > > > Thanks, > > Cabir Zounaidou > >
