yes, there are rpc and mapred logs in log file hadoop-metrics.properties is like this, and hadoop version is 0.2 # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License.
chukwaAgent.class=org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext chukwaAgent.period=60 chukwaAgent.directory=/tmp/chukwa/log/metrics/ chukwaHttpSender.class=org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext chukwaHttpSender.period=60 chukwaHttpSender.directory=/tmp/chukwa/log/metrics/ chunkQueue.class=org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext chunkQueue.period=60 chunkQueue.directory=/tmp/chukwa/log/metrics/ dfs.class=org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext dfs.period=60 dfs.directory=/tmp/chukwa/log/metrics/ dfs.uuid=true jvm.class=org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext jvm.period=60 jvm.directory=/tmp/chukwa/log/metrics/ jvm.uuid=true mapred.class=org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext mapred.period=60 mapred.directory=/tmp/chukwa/log/metrics/ mapred.uuid=true rpc.class=org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext rpc.period=60 rpc.directory=/tmp/chukwa/log/metrics/ rpc.uuid=true At 2011-07-02 11:39:32，"Eric Yang" <[email protected]> wrote: >Check the log directory and see if you see any log file with rpc or >mapred in the file name? >What does the hadoop-metrics.properties look like? What version of >hadoop are you running? > >regards, >Eric > >2011/7/1 孙长 <[email protected]>: >> i know where the problem is , i didn't put log4j and hadoop-metrics file >> into all nodes' hadoop conf dir. >> Now the problem disapears >> but for hadoop metrics, i only got dfs data in mysql, no mapreduce, jvm, rpc >> data, why? >> thank you~ >> >> >> At 2011-07-01 22:59:41，"Eric Yang" <[email protected]> wrote: >> >>>Looks like mapreduce can not find json.jar in its class path. Which >>>version of Hadoop are you using? >>> >>>regards, >>>Eric >>> >>>2011/7/1 孙长 <[email protected]>: >>>> hi,i have copied chukwa-hadoop-*-client.jar and json.jar to >>>> HADOOP_HOME/lib, and restarted hadoop, but the MR job lanched by demux runs >>>> error: >>>> task error is as follows: >>>> >>>> Error: java.lang.ClassNotFoundException: org.json.JSONException >>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:200) >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:188) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:307) >>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:252) >>>> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) >>>> at java.lang.Class.forName0(Native Method) >>>> at java.lang.Class.forName(Class.java:169) >>>> at >>>> org.apache.hadoop.chukwa.extraction.demux.processor.mapper.MapProcessorFactory.getProcessor(MapProcessorFactory.java:40) >>>> at >>>> org.apache.hadoop.chukwa.extraction.demux.Demux$MapClass.map(Demux.java:95) >>>> at >>>> org.apache.hadoop.chukwa.extraction.demux.Demux$MapClass.map(Demux.java:64) >>>> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) >>>> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358) >>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) >>>> at org.apache.hadoop.mapred.Child.main(Child.java:170) >>>> >>>> At 2011-07-01 13:17:42，"Eric Yang" <[email protected]> wrote: >>>> >>>>>Did you copy chukwa-hadoop-*-client.jar and json.jar to >>>>>HADOOP_HOME/lib, and restart hadoop? >>>>> >>>>>regards, >>>>>Eric >>>>> >>>>>2011/6/30 孙长 <[email protected]>: >>>>>> Hi, I am using chukwa 0.4 , and i start a collector and a agent on two >>>>>> nodes, but i only got system metrics in mysql, no hadoop metrics like >>>>>> jvm, >>>>>> dfs, mapred >>>>>> dbAdmin.sh is running background, log4j.properties >>>>>> and hadoop-metrics.properties in hadoop conf dir are set according to >>>>>> user >>>>>> guide >>>>>> no dfs, mapred logs in chukwa/log/metrics dir. >>>>>> thank you ~ >>>>>> >>>>>> >>>> >>>> >>>> >> >> >>
