You can run that code w/o running hadoop. As long as you have the libraries in
classpath and the code is doing FileSystem operations, it will execute in
standalone mode just fine. The last I remember the conf values from the client
would be default values of the HDFS version and will not get picked up from the
namenode installation. This might have changed, or may not. So it is in your
best interest to verify what values is your client Conf object using. e.g. your
innstallation might say rep factor 2, but default rep factor on Configuration
object is 3, which can give you wrong perf numbers.
-Ayon
________________________________
From: Harsh J <qwertyman...@gmail.com>
To: hdfs-user@hadoop.apache.org; pei...@gmail.com; zhengda1...@gmail.com
Sent: Fri, January 28, 2011 9:53:17 PM
Subject: Re: Latency and speed of HDFS
Moving discussion to hdfs-user mailing list: hdfs-u...@hadoop.apache.com
On Sat, Jan 29, 2011 at 5:06 AM, Pei HE <pei...@gmail.com> wrote:
> Hi all,
> I want to know the detailed performance of Hadoop.
>
> I am writing a client code to test the latency, transfer speed of
> HDFS, and the time of initiating a JVM for a task.
>
> Does anyone know that which classes can be called without running a
> whole map/reduce job to find out the performance of Hadoop?
>
> I am trying to create a DFSClient in the following way.
> Configuration conf = new Configuration();
> DFSClient client = new DFSClient(conf);
> client.exists("hdfs://localhost:9900/home/");
> But, a RemoteException was thrown.
Would help to know what the cause for the throw of a RemoteException was.
--
Harsh J
www.harshj.com
