> On Nov. 10, 2014, 4:28 p.m., Nate Cole wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentLogResourceProvider.java, > > lines 114-115 > > <https://reviews.apache.org/r/27802/diff/2/?file=756639#file756639line114> > > > > I think pattern picks up log lines? If yes, then a change to > > log4j.properties may end up with no matching lines when it's pumped into > > HDFS. I'm not 100% clear how the communication will look
Assumption is that the format of the file pushed to hdfs will be in standard format following this regular expression. If the component log file is different, it should be converted using change_log_format=true, log_pattern=<COMP_LOG_FORMAT>, log_pattern_map and the conversion will be handled by the agent. - Cabir ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27802/#review60613 ----------------------------------------------------------- On Nov. 10, 2014, 9:04 a.m., Cabir Zounaidou wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/27802/ > ----------------------------------------------------------- > > (Updated Nov. 10, 2014, 9:04 a.m.) > > > Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, John > Speidel, Mahadev Konar, Nate Cole, Sid Wagle, and Yusaku Sako. > > > Bugs: AMBARI-4083 > https://issues.apache.org/jira/browse/AMBARI-4083 > > > Repository: ambari > > > Description > ------- > > The host component log implements the following two rest api's. > > 1. To retrieve the log entries. > /clusters/{cluster}/hosts/{hostname}/host_components/{component}/logs > This api will retrieve the logs if available for the host component from > HDFS. It can fetch only maximum of 5120 bytes. The window can be adjusted > using the query parameters 'offset' and 'length'. It also provides simple > filtering using 'level' query parameter. > Sample response will look like below: > { > "href" : > "http://c6501.ambari.apache.org:8080/api/v1/clusters/cl1/hosts/c6503.ambari.apache.org/host_components/HBASE_CLIENT/logs";, > "items" : [ > { > "href" : > "http://c6501.ambari.apache.org:8080/api/v1/clusters/cl1/hosts/c6503.ambari.apache.org/host_components/HBASE_CLIENT/logs/HBASE_CLIENT";, > "length" : 5120, > "level" : null, > "offset" : 153920, > "size" : 159040, > "HostComponentLog" : { > "cluster_name" : "cl1", > "component_name" : "HBASE_CLIENT", > "entries" : [ > { > "timestamp" : "2014-11-01 17:46:38,456", > "level" : "DEBUG", > "thread" : "main-EventThread", > "message" : "master.SplitLogManager: task not yet acquired > /hbase-unsecure/splitWAL/WALs%2Fc6503.ambari.apache.org%2C60020%2C1414856039721-splitting%2Fc6503.ambari.apache.org%252C60020%252C1414856039721.1414856049592.meta > ver = 0" > }, > { > "timestamp" : "2014-11-01 17:46:38,459", > "level" : "DEBUG", > "thread" : "main-EventThread", > "message" : "master.SplitLogManager: put up splitlog task at > znode > /hbase-unsecure/splitWAL/WALs%2Fc6503.ambari.apache.org%2C60020%2C1414856039721-splitting%2Fc6503.ambari.apache.org%252C60020%252C1414856039721.1414856050193.meta" > }, > { > "timestamp" : "2014-11-01 17:46:38,460", > "level" : "DEBUG", > "thread" : "main-EventThread", > "message" : "master.SplitLogManager: put up splitlog task at > znode > /hbase-unsecure/splitWAL/WALs%2Fc6503.ambari.apache.org%2C60020%2C1414856039721-splitting%2Fc6503.ambari.apache.org%252C60020%252C1414856039721.1414856105560.meta" > }, > : > : > }] > } > ] > } > 2. To download the log file from HDFS use the following the api > > /clusters/{cluster}/hosts/{hostname}/host_components/{component}/logs?download=true > The response will automatically download the file with the HDFS file name. > > > Diffs > ----- > > ambari-agent/src/main/python/ambari_agent/Controller.py dc3a1cf > > ambari-server/src/main/java/org/apache/ambari/server/agent/AgentCommand.java > e2f013d > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java > e99e39f > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatResponse.java > 56b4f18 > > ambari-server/src/main/java/org/apache/ambari/server/agent/LogConfigCommand.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/agent/RegistrationResponse.java > 8a24560 > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostComponentLogResourceDefinition.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostComponentResourceDefinition.java > 6dc9e2d > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java > 9ad37ec > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > bb4c569 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/HostComponentService.java > 4990ad7 > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > 4f53544 > > ambari-server/src/main/java/org/apache/ambari/server/controller/HostComponentLogResponse.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java > ae20f56 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentLogResourceProvider.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java > dbac906 > ambari-server/src/main/java/org/apache/ambari/server/state/LogConfig.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/state/LogConfigHandler.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/state/LogDefinition.java > PRE-CREATION > ambari-server/src/main/java/org/apache/ambari/server/state/LogEntry.java > PRE-CREATION > ambari-server/src/main/resources/key_properties.json c1a6636 > ambari-server/src/main/resources/log_handler_config.json PRE-CREATION > ambari-server/src/main/resources/properties.json 36cff96 > > ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java > e7b946d > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostComponentLogResourceProviderTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/27802/diff/ > > > Testing > ------- > > 1. Ran the unit tests. > 2. Manually ran the retrieve host component logs without any query parameters. > 3. Manually ran the retrieve host component logs with offset and length. > 4. Manually ran the retrieve host component logs with level filter. > 5. Manually ran the retrieve host component logs with webHDFS disabled. > 6. Manually ran the download host component log file with webHDFS enabled. > 7. Manually ran the download host component log file with webHDFS disabled. > > > Thanks, > > Cabir Zounaidou > >
