Quantifying HDFS Client Latency to understand performance and scalability
-------------------------------------------------------------------------
Key: HDFS-1324
URL: https://issues.apache.org/jira/browse/HDFS-1324
Project: Hadoop HDFS
Issue Type: Test
Components: benchmarks
Affects Versions: 0.20-append
Environment: An HDFS cluster of 9 nodes (same rack; 1 rack = 40 nodes
to one rack switch) was deployed on Yahoo!'s R&D grid cluster. It comprised of
1 Namenode, 1 JobTracker and 7 Datanodes. client assigned to a separate compute
node not part of this HDFS. The characteristics of the testbed are as follows:
• Hadoop 0.20.1xx - latest Yahoo! Hadoop Security version
• OS - GNU/Linux x86_64 kernel version 2.6.18
• Java 1.6
• Processor - 2 SMP Quad-core Intel Xeon @ 2.5Ghz per node
• Memory - 16GB RAM per node
Reporter: Mona Chitnis
JAR to measure HDFS client latencies. Runs a process comprising of HDFS
operations with helper scripts to specify number of iterations, number of
concurrent instances and base directory. Compiles statistics in 2 text files
inside base directory which can be read/manipulated using data charts
applications (e.g. MS Excel).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
