Sorry for bothering everyone, I accidentally configured my dfs.data.dir and
mapred.local.dir to the same directory... Bad copy/paste job.
Thanks for everyone's help!
So I pulled everything of NFS and I'm still getting the original error with a
FileNotFoundException for current/VERSION.
I only have 4 slaves and scp'ed the Hadoop directory to all 4 slaves.
Any other ideas?
On May 14, 2010, at 7:41 PM, Hemanth Yamijala wrote:
> Andrew,
>
>> Just to be clear,
Yeah, I tried some more experiments today and the error messages were more
helpful. It does seem that some of the values were defaulting to ones very
different from what I had configured.
I have been looking into Puppet but figured with 4 slaves, it shouldn't be a
problem to use NFS. Guess
Andrew,
> Just to be clear, I'm only sharing the Hadoop binaries and config files via
> NFS. I don't see how this would cause a conflict - do you have any
> additional information?
FWIW, we had an experience where we were storing config files on NFS
on a large cluster. Randomly, (and we guess
My hdfs-site.xml file:
11
12
13 dfs.replication
14 3
15
16
17 dfs.name.dir
18 /srv/hadoop/dfs.name.dir
19
20
21 dfs.data.dir
22 /srv/hadoop/dfs/1
23
24
Here is my /srv/hadoop/hadoop directory listing:
total 5068
drwxr-xr-x 2 hadoop hadoo
I'm pretty sure I just set my dfs.data.dir to be /srv/hadoop/dfs/1
dfs.data.dir
/srv/hadoop/dfs/1
I don't have hadoop.tmp.dir set to anything so it's whatever the default is.
I don't have access to the cluster right now but will update with the exact
settings when I get a chance.
I have 4 sl
On May 14, 2010, at 8:53 AM, Andrew Nguyen wrote:
> Just to be clear, I'm only sharing the Hadoop binaries and config files via
> NFS. I don't see how this would cause a conflict - do you have any
> additional information?
>
> The referenced path in the error below (/srv/hadoop/dfs/1) is not
Just to be clear, I'm only sharing the Hadoop binaries and config files via
NFS. I don't see how this would cause a conflict - do you have any additional
information?
The referenced path in the error below (/srv/hadoop/dfs/1) is not being shared
via NFS...
Thanks,
Andrew
On May 13, 2010, at
It is not suggested to deploy hadoop on NFS, there will be conflict
between data nodes, because NFS share the same namespace of file
system.
On Thu, May 13, 2010 at 9:52 PM, Andrew Nguyen wrote:
>
> Yes, in this deployment, I'm attempting to share the hadoop files via NFS.
> The log and pid d
Yes, in this deployment, I'm attempting to share the hadoop files via NFS. The
log and pid directories are local.
Thanks!
--Andrew
On May 12, 2010, at 7:40 PM, Jeff Zhang wrote:
> These 4 nodes share NFS ？
>
>
> On Thu, May 13, 2010 at 8:19 AM, Andrew Nguyen
> wrote:
>> I'm working on brin
These 4 nodes share NFS ？
On Thu, May 13, 2010 at 8:19 AM, Andrew Nguyen
wrote:
> I'm working on bringing up a second test cluster and am getting these
> intermittent errors on the DataNodes:
>
> 2010-05-12 17:17:15,094 ERROR
> org.apache.hadoop.hdfs.server.datanode.DataNode:
> java.io.FileNo
I'm working on bringing up a second test cluster and am getting these
intermittent errors on the DataNodes:
2010-05-12 17:17:15,094 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode:
java.io.FileNotFoundException: /srv/hadoop/dfs/1/current/VERSION (No such file
or directory)
at java
12 matches
Mail list logo
