Chris K Wensel wrote:
you cannot have underscores in a bucket name. it freaks out java.net.URI.
freaks out DNS, too, which is why the java.net classes whine. minus
signs should work
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action
block with their consoles going 'NFS Server not responding for 30s to
make you wish you weren't using NFS. Because NFS IO is done in the
kernel, there's no way to put policy into the apps about retry,
timeouts and behaviour on failure.
just say no to NFS.
--
Steve Loughran
for users,
that may a better area to focus on.
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http://antbook.org/
artifacts yourself, but that yum is essentially not what you
want if you want to stay in control of your machine state.
Khalil -how many machines do you have to look after?
-steve
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http
it was ready.
Thanks, pete
Seems to me something that should be remotely available and visible via
JMX...always good to use. Absolute time info, if sent as a time_t long
should be in UTC to avoid confusion when talking to servers in different
time zones...
--
Steve Loughran http
custom hadoop EC2 images,
something like
-bring up the image
-push out new RPMs and ssh keys, including JVM versions.
-create the new AMI
-set the AMI access rights up.
-delete the old one.
Like I said, on the todo list.
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Allen Wittenauer wrote:
On 5/1/08 5:00 PM, Bradford Stephens [EMAIL PROTECTED] wrote:
*Very* cool information. As someone who's leading the transition to
open-source and cluster-orientation at a company of about 50 people,
finding good tools for the IT staff to use is essential. Thanks so much
to spare, others do, so design your changes to
test in that world, and to run on bigger clusters.
Note that Append is not something you are ever going to see on S3 files;
it's not part of the S3 REST API. So if you assume append everywhere,
your app wont be so happy on the EC2 farms.
--
Steve
Bryan Duxbury wrote:
Nobody has any ideas about this?
-Bryan
On May 13, 2008, at 11:27 AM, Bryan Duxbury wrote:
I'm trying to create a java application that writes to HDFS. I have it
set up such that hadoop-0.16.3 is on my machine, and the env variables
HADOOP_HOME and HADOOP_CONF_DIR point
Allen Wittenauer wrote:
On 5/15/08 5:05 AM, Steve Loughran [EMAIL PROTECTED] wrote:
I have a question for users: how do they ensure their client apps have
configuration XML file that are kept up to date?
We control the client as well as the servers, so it all gets pushed at
once. :)
yes
.
For Hadoop, something that prints out the config and looks for good/bad
problems, maybe even does nslookup() on the hosts, checks the ports are
open, etc, would be nice, especially if it provides hints when things
are screwed up.
-Steve
--
Steve Loughran http://www.1060
Alejandro Abdelnur wrote:
A while ago I've opened an issue related to this topic
https://issues.apache.org/jira/browse/HADOOP-3287
My take is a little different, when submitting a job, the clients
should only send to the jobtracker the configuration they explicitly
set, then the job tracker
Haijun Cao wrote:
Hi,
I noticed that the org.apache.hadoop.conf.Configuration constructor will
log a message like below if DEBUG is enabled:
2008-05-19 15:59:43,237 DEBUG [main] conf.Configuration
java.io.IOException: config()
at
Christophe Taton wrote:
Actually Hadoop could be made more friendly to such realtime Map/Reduce
jobs.
For instance, we could consider running all tasks inside the task tracker
jvm as separate threads, which could be implemented as another personality
of the TaskRunner.
I have been looking into
Alejandro Abdelnur wrote:
Yes you would have to do it with classloaders (not 'hello world' but not
'rocket science' either).
That's where we differ.
I do actually think that classloaders are incredibly hard to get right,
and I say that as someone who has single stepped through the Axis2 code
can help it.
The x-trace team are trying to instrument hadoop for better debugging
http://radlab.cs.berkeley.edu/wiki/Projects/X-Trace_on_Hadoop
this looks really interesting
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http
worry/address the risk of someone like me bringing up a machine
in the EC2 farm that then portscans all the near-neighbours in the
address space for open hdfs data node/name node ports, and strikes up a
conversation with your filesystem?
--
Steve Loughran http://www.1060.org
Colin Freas wrote:
I've wondered about this using single or dual quad-core machines with one
spindle per core, and partitioning them out into 2, 4, 8, whatever virtual
machines, possibly marking each physical box as a rack.
Or just host VM images with multi-cpu support and make it one
novice user wrote:
Hi,
I am getting the below error when I was using some one's code where they
are using hadoop-17 and have the method FileInputFormat.setInputPaths for
setting input paths for the job. The exact error is given below.
java.lang.NoSuchMethodError:
-with-space
I'd go for option 2, edit /etc/passwd.
-I havent tried this, I use linux, etc, etc. Consider filing a bugrep
against Hadoop Hadoop assumes user names do not contain spaces
steve
Thanks and regards,
Ravi
- Original Message - From: Steve Loughran [EMAIL PROTECTED]
To: core-user
Steve Loughran wrote:
Ravi Shankar (Google) wrote:
Hi Steve,
Thanks for reply. whoami gives me
Ravi Shankar
How can I make it a single word? Because this name is from
installation of my windows
a Quick search for whoami cygwin shows this article
http://cygwin.com/faq/faq.setup.html
Satoshi YAMADA wrote:
From hadoop doc, only Linux and Windows are supported platforms. Is
it possible to run
hadoop on Solaris? Is hadoop implemented in pure java? What kinds of
problems are there in
order to port to Solaris? Thanks in advance.
hi,
no one seems to reply to the previous
Daniel Blaisdell wrote:
I ran into some similar issues with firewalls and ended up completely
turning them off. That took care of some of the problems but allowed me to
figure out that if DNS / HOST files aren't configured correctly, weird
things will happen during the communication between
in
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http://antbook.org/
Rutuja Joshi wrote:
Hello,
I am new to Hadoop and am trying to run
HadoopDfsReadWriteExamplehttp://wiki.apache.org/hadoop/HadoopDfsReadWriteExample?action=fullsearchvalue=linkto%3A%22HadoopDfsReadWriteExample%22context=180
from eclipse on Windows XP.
I have added following files in the build
Kylie McCormick wrote:
Hello!
My name is Kylie McCormick, and I'm currently working on creating a
distributed information retrieval package with Hadoop based on my previous
work with other middlewares like OGSA-DAI. I've been developing a design
that works with the structures of the other
Robert Krüger wrote:
Hi,
I'm trying to run a program (basically a simple thruput benchmark, I've
already tested on Linux) in Eclipse on Windows. The HDFS is located on a
Linux server, though.
The exception I get is the following (probably the same Rutuja Joshi
posted earlier this month):
Lincoln Ritter wrote:
Why not use jruby?
Indeed! I'm basically working from the JRuby wiki page on Java
integration (http://wiki.jruby.org/wiki/Java_Integration). I'm taking
this one step at a time and, while I would love tighter integration,
the recommended way is through the scripting
Lincoln Ritter wrote:
This is a bit scattered but I wanted to post this in case it might
help someone...
Here's a little more detail on the loading problems I've been having.
(snip)
Poking around the JRubyScriptEngine source
Gopal Gandhi wrote:
Hi folks,
Does anybody has a comment on that? Why we let reducer fetch local data
through HTTP not SSH?
presumably because its way more efficent. And, being fairly stateless,
scales well. SSH has an expensive connection overhead as well as the
encrypt/decrypt costs.
, as then your
failover tools have to deal with the risk that there are now two
machines that think they are in charge. This is why building
High-Availability and fault-tolerant systems are tricky.
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action
Himanshu Sharma wrote:
The NFS seems to be having problem as NFS locking causes namenode hangup.
Can't be there any other way, say if namenode starts writing synchronously
to secondary namenode apart from local directories, then in case of namenode
failover, we can start the primary namenode
sudha sadhasivam wrote:
Respected Sir
We are ME students working on Hadoop for bioinformatics.
We have installed Hadoop on Linux and it is working well.
We have installed biojava (an extension of java for bioinformatics) for DNA sequence searching.
To install biojava, we have to copy the jar
to say you can do
some things here and point to the full docs at SVN or the hadoop site
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http://antbook.org/
Kevin wrote:
Thank you for the suggestion. I looked at DFSClient. It appears that
chooseDataNode method decides which data node to connect to. Currently
it chooses the first non-dead data node returned by namenode, which
have sorted the nodes by proximity to the client. However,
chooseDataNode
Allen Wittenauer wrote:
On 8/6/08 11:52 AM, Otis Gospodnetic [EMAIL PROTECTED] wrote:
You can put the same hadoop-site.xml on all machines. Yes, you do want a
secondary NN - a single NN is a SPOF. Browser the archives a few days back to
find an email from Paul about DRBD (disk replication) to
Mork0075 wrote:
Hello,
can someone please explain oder point me to some documentation or
papers, where i can read well proven facts, why scaling a relational db
is so hard and scaling a document oriented db isnt?
http://labs.google.com/papers/bigtable.html
relational dbs are great for
Piotr Kozikowski wrote:
Hi there:
We would like to know what are the most likely causes of this sort of
error:
Exception closing
file
/data1/hdfs/tmp/person_url_pipe_59984_3405334/_temporary/_task_200807311534_0055_m_22_0/part-00022
java.io.IOException: Could not get block locations.
Joel Welling wrote:
Hi folks;
I'm new to Hadoop, and I'm trying to set it up on a cluster for which
almost all the disk is mounted via the Lustre filesystem. That
filesystem is visible to all the nodes, so I don't actually need HDFS to
implement a shared filesystem. (I know the philosophical
Karl Anderson wrote:
I'm getting NotReplicatedYet exceptions when I try to put a file on DFS
for a newly created cluster. If I wait a while, the put works.
Is there a way to tell if the DFS is ready from the master node?
This is something I'm working on
)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http://antbook.org/
Joel Welling wrote:
Thanks, Steve and Arun. I'll definitely try to write something based on
the KFS interface. I think that for our applications putting the mapper
on the right rack is not going to be that useful. A lot of our
calculations are going to be disordered stuff based on 3D spatial
Dmitry Pushkarev wrote:
Dear hadoop users,
Our lab in slowly switching from SGE to hadoop, however not everything seems
to be easy and obvious. We are in no way computer scientists, we're just
physicists, biologist and couple of statisticians trying to solve our
computational problems,
Dmitry Pushkarev wrote:
Probably, but the current idea is to bypass writing small files to HDFS by
creating my own local har archive and uploading it. (small files lower
transfer speed from 40-70MB/s to hundreds ok kbps :(
If there is an entry point to do this on the command line, it could be
Tenaali Ram wrote:
Hi,
I am new to hadoop. What I have understood so far is- hadoop is used to
process huge data using map-reduce paradigm.
I am working on problem where I need to perform large number of
computations, most computations can be done independently of each other (so
I think each
Allen Wittenauer wrote:
On 9/5/08 5:53 AM, Andreas Kostyrka [EMAIL PROTECTED] wrote:
Another idea would be a tool or namenode startup mode that would make it
ignore EOFExceptions to recover as much of the edits as possible.
We clearly need to change the how to configure docs to make
Owen O'Malley wrote:
Currently there isn't a way to do that. In Hadoop 0.19, there will be a way
to have a clean up method that runs at the end of the job. See
HADOOP-3150https://issues.apache.org/jira/browse/HADOOP-3150
.
another bit of feature creep would be an expires: attribute on files,
Julien Nioche wrote:
I have tried using *slave.host.name* and give it the public address of my
data node. I can now see the node listed with its public address on the
dfshealth.jsp, however when I try to send a file to the HDFS from my
external server I still get :
*08/09/08 15:58:41 INFO
叶双明 wrote:
Thanks for paying attention to my tentative idea!
What I thought isn't how to store the meradata, but the final (or last) way
to recover valuable data in the cluster when something worst (which destroy
the metadata in all multiple NameNode) happen. i.e. terrorist attack or
natural
James Moore wrote:
On Thu, Sep 11, 2008 at 5:46 AM, Allen Wittenauer [EMAIL PROTECTED] wrote:
On 9/11/08 2:39 AM, Alex Loddengaard [EMAIL PROTECTED] wrote:
I've never dealt with a large cluster, though I'd imagine it is managed the
same way as small clusters:
Maybe. :)
Depends how often
Paco NATHAN wrote:
We use an EC2 image onto which we install Java, Ant, Hadoop, etc. To
make it simple, pull those from S3 buckets. That provides a flexible
pattern for managing the frameworks involved, more so than needing to
re-do an EC2 image whenever you want to add a patch to Hadoop.
Given
zheng daqi wrote:
Hello,
I got a problem when I compiled hadoop's pipes' example(under cygwin).
and I searched a lot, and find nobody have met the same problem.
could you please give me some suggestions, thanks very much.
BUILD FAILED
?
thanks and regards,
Prasad Pingali,
IIIT Hyderabad.
--
Sorry for my english!! 明
Please help me to correct my english expression and error in syntax
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http://antbook.org/
Owen O'Malley wrote:
On Sep 24, 2008, at 1:50 AM, Trinh Tuan Cuong wrote:
We are developing a project and we are intend to use Hadoop to handle
the processing vast amount of data. But to convince our customers
about the using of Hadoop in our project, we must show them the
advantages ( and
Gerardo Velez wrote:
Hi everybody!
I'm a newbie on hadoop and after follow up some hadoop examples and studied
them. I will start my own application but I got a question.
Is there anyway I could debug my own hadoop application?
Actually I've been working on IntelliJ IDE, but I'm feeling
Shirley Cohen wrote:
Hi,
I'm trying to figure out which log files are used by the job tracker's
web interface to display the following information:
Job Name: my job
Job File:
hdfs://localhost:9000/tmp/hadoop-scohen/mapred/system/job_200809260816_0001/job.xml
Status: Succeeded
Started at:
Jason Rutherglen wrote:
I implemented an RMI protocol using Hadoop IPC and implemented basic
HMAC signing. It is I believe faster than public key private key
because it uses a secret key and does not require public key
provisioning like PKI would. Perhaps it would be a baseline way to
sign the
.
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http://antbook.org/
of signed jars
myself but I never implemented it.
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http://antbook.org/
Allen Wittenauer wrote:
On 10/6/08 6:39 AM, Steve Loughran [EMAIL PROTECTED] wrote:
Edward Capriolo wrote:
You bring up some valid points. This would be a great topic for a
white paper.
-a wiki page would be a start too
I was thinking about doing Deploying Hadoop Securely
Edward J. Yoon wrote:
If we have a group blog of the hadoop user/dev group such as a Y!
developer network, we can easily share/introduce our experience and
outcomes from our research. So, I thought about a group blog, I guess
there are plenty of contributors.
What do you think about it?
existing web page health
checking code to pull in all the hadoop services. The best bit: when it
fails, the ops team can point their browser at the same URL and see what
is up. And if you are a standalone developer -you are the ops team!
-steve
--
Steve Loughran http://www
is to be saved) as local
filesystem.. i'm able to save the chart..
but if i set path to be hdfs, then i'm unable to...
so what changes do i need to make..
You'll need to copy the local file to HDFS after it is rendered.
Thanks
Chandravadana.S
Steve Loughran wrote:
Alex Loddengaard wrote:
Hadoop
chandravadana wrote:
ya.. will write up in hadoop wiki..
is there a way other than copying from local filesystem to hdfs...
like writing directly to hdfs...?
Can you patch jfreechart to write directly to HDFS files? That is the
only way to do it right now, unless you can mount the HFDS
Amit k. Saha wrote:
On Wed, Oct 15, 2008 at 9:09 AM, David Wei [EMAIL PROTECTED] wrote:
It seems that we need to restart the whole hadoop system in order to add new
nodes inside the cluster. Any solution for us that no need for the
rebooting?
From what I know so far, you have to start the
David Wei wrote:
Error initializing attempt_200810220716_0004_m_01_0:
java.lang.IllegalArgumentException: Wrong FS:
hdfs://192.168.52.129:9000/tmp/hadoop-root/mapred/system/job_200810220716_0004/job.xml,
expected: hdfs://datacenter5:9000 at
David Wei wrote:
Steve Loughran ??:
David Wei wrote:
Error initializing attempt_200810220716_0004_m_01_0:
java.lang.IllegalArgumentException: Wrong FS:
hdfs://192.168.52.129:9000/tmp/hadoop-root/mapred/system/job_200810220716_0004/job.xml,
expected: hdfs://datacenter5:9000
Alex Loddengaard wrote:
Just to be clear, you want to persist a configuration change to your entire
cluster without bringing it down, and you're hoping to use the Configuration
API to do so. Did I get your question correct?
I don't know of a way to do this without restarting the cluster,
woody zhou wrote:
Hi everyone,
I have a problem about Hadoop startup.
I failed to startup the namenode and I got the following exception in the
namenode log file:
2008-10-23 21:54:51,232 ERROR org.apache.hadoop.dfs.NameNode:
java.lang.OutOfMemoryError: unable to create new native thread
/~stevel/slides/deploying_hadoop_with_smartfrog.pdf
I'm putting in for an apachecon eu talk on the topic, Dynamic Hadoop
Clusters.
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http://antbook.org/
Paco NATHAN wrote:
Hi Karl,
Rather than using separate key pairs, you can use EC2 security groups
to keep track of different clusters.
Effectively, that requires a new security group for every cluster --
so just allocate a bunch of different ones in a config file, then have
the launch scripts
chaitanya krishna wrote:
Hi,
If the problem is due to the OS-level limit on the number of active
threads, then why is the error showing outofmemory exception? Is it an issue
of the heap size available for hadoop?Won't increasing heap size fix this
problem?
It's not saying out of memory, it
wmitchell wrote:
Hi All,
Ive been working michael nolls multi-node cluster setup example
(Running_Hadoop_On_Ubuntu_Linux) for hadoop and I have a working setup. I
then on my slave machine -- which is currently running a datanode killed the
process in an effort to try to simulate some sort of
Alex Loddengaard wrote:
I'd like my log messages to display the hostname of the node that they were
outputted on. Sure, this information can be grabbed from the log filename,
but I would like each log message to also have the hostname. I don't think
log4j provides support to include the
Alex Loddengaard wrote:
Thanks, Steve. I'll look in to this patch. As a temporary solution I use a
log4j variable to manually set a hostname private field in the Appender.
This solution is rather annoying, but it'll work fro now.
Thanks again.
what about having the task tracker pass down a
to a DOM, things should work
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http://antbook.org/
Pete Wyckoff wrote:
It has come a long way since 0.18 and facebook keeps our (0.17) dfs mounted via
fuse and uses that for some operations.
There have recently been some problems with fuse-dfs when used in a
multithreaded environment, but those have been fixed in 0.18.2 and 0.19. (do
not use
localhost 9000 but not remotely, you have a firewall in the way
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http://antbook.org/
C G wrote:
I've got a grid which has been up and running for some time. It's been using a 32
bit JVM. I am hitting the wall on memory within NameNode and need to specify max
heap size 4G. Is it possible to switch seemlessly from 32bit JVM to 64bit?
I've tried this on a small test grid and
Raghu Angadi wrote:
How often is safe depends on what probabilities you are willing to accept.
I just checked on one of clusters with 4PB of data, the scanner fixes
about 1 block a day. Assuming avg size of 64MB per block (pretty high),
probability that 3 copies of one replica go bad in 3
Kevin Peterson wrote:
I'm trying to import Hadoop Core into our local repository using piston
( http://piston.rubyforge.org/index.html ).
I can't seem to access svn.apache.org though. I've also tried the EU
mirror. No errors, nothing but eventual timeout. Traceroute fails at
Bhupesh Bansal wrote:
Hey folks,
I re-started my cluster after some node failures and saw couple of
tasktrackers not being up (they finally did after abt 20 Mins)
In the logs below check the blue timestamp to Red timestamp.
I was just curious what do we do while starting tasktracker that
Mithila Nagendra wrote:
I tried dropping the jar files into the lib. It still doesnt work.. The
following is how the lib looks after the new files were put in:
[EMAIL PROTECTED] hadoop-0.17.2.1]$ cd bin
[EMAIL PROTECTED] bin]$ ls
hadoophadoop-daemon.sh rccstart-all.sh
Scott Whitecross wrote:
Thanks Brian. So you have had luck w/ log4j?
We grab logs off machines by not using lo4j and routing to our own
logging infrastructure that can feed events to other boxes via RMI and
queues. This stuff slots in behind commons-logging, with a custom
commons-logging
Mithila Nagendra wrote:
Hey Steve
Out of the following which one do I remove - just making sure.. I got rid
of commons-logging-1.0.4.jar
commons-logging-api-1.0.4.jar
commons-logging-1.1.1-sources.jar commons-logging-1.1.1-sources.jar
Hadoop is currently built with
Mithila Nagendra wrote:
Hey Steve
I deleted what ever I needed to.. still no luck..
You said that the classpath might be messed up.. Is there some way I can
reset it? For the root user? What path do I set it to.
Let's start with what kind of machine is this? Windows? or Linux. If
Linux,
Mithila Nagendra wrote:
Hey steve
The version is: Linux enpc3740.eas.asu.edu 2.6.9-67.0.20.EL #1 Wed Jun 18
12:23:46 EDT 2008 i686 i686 i386 GNU/Linux, this is what I got when I used
the command uname -a
On Tue, Nov 25, 2008 at 1:50 PM, Steve Loughran [EMAIL PROTECTED] wrote:
Mithila Nagendra
lohit wrote:
As Amareshwari said, you can almost safely stop TaskTracker process on node. Task(s) running on that would be considered failed and would be re-executed by JobTracker on another node. Reason why we decomission DataNode is to protect against data loss. DataNode stores HDFS blocks, by
Ricky Ho wrote:
Does Hadoop support the environment where nodes join and leave without a preconfigured
file like hadoop-site.xml ? The characteristic is that none of the IP
addresses and node names of any machines are stable. They will change after the machine
is reboot after crash.
Before
Sandeep Dhawan, Noida wrote:
Hello,
I am currently using hadoop-0.18.0. I am not able to append files in
DFS. I came across a fix which was done on version 0.19.0
(http://issues.apache.org/jira/browse/HADOOP-1700). But I cannot migrate
to 0.19.0 version because it runs on JDK 1.6 and I have
Robert Goodman wrote:
I have some code where I create my own Hadoop job and the use the JobClient
to submit the Hadoop job. I noticed that the JobClient class has a
killJob() method. I was planning to play around and try to kill a running
Hadoop job. Does anybody know status the killJob method?
Here's a presentation on datacentres and MR my colleague and I gave to
the local university this week, My other computer is a datacentre.
MapReduce was demoed on a different dataset from normal (scanned
bluetooth devices from a static location) and implemented in Erlang,
because the students
Allen, Jeffrey wrote:
Hi,
I have an existing enterprise system using web services. I'd like to have an
event in the web service eventually result in a map/reduce being performed. It
would be very desirable to be able to package up the map reduce classes into a
jar that gets deployed inside
Leeau wrote:
Dear,
I want to config a 4-site hadoop cluster. but failed. Who can help me to
know why? and how can i start it? thanks.
you need to learn to read stack traces
2008-12-04 17:59:11,674 INFO org.apache.hadoop.ipc.Server: Stopping server
on 9000
2008-12-04 17:59:11,730 ERROR
Brian Bockelman wrote:
On Dec 4, 2008, at 6:58 AM, Steve Loughran wrote:
Leeau wrote:
Dear,
I want to config a 4-site hadoop cluster. but failed. Who can help me to
know why? and how can i start it? thanks.
you need to learn to read stack traces
Ah, the common claim of the Java
Bryan Duxbury wrote:
If you are considering using it as a conventional filesystem from a few
clients, then it most resembles NAS. However, I don't think it makes
sense to try and classify it as SAN or NAS. HDFS is a distributed
filesystem designed to be consumed in a massively distributed
Doug Cutting wrote:
Brian Bockelman wrote:
To some extent, this whole issue is caused because we only have enough
space for 2 replicas; I'd imagine that at 3 replicas, the issue would
be much harder to trigger.
The unfortunate reality is that if you run a configuration that's
different than
Christian Kunz wrote:
Is there support to tell hadoop servers (tasktracker, datanode, )
to re-read configuration, or fully reset, or to shut down (without an
external kill)?
Some of the lifecycle stuff that Is now targeted at 0.21 will be able to
do this.
Doug Cutting wrote:
Steve Loughran wrote:
Alternatively, why we should be exploring the configuration space
more widely
Are you volunteering?
Doug
Not yet. I think we have a fair bit of what is needed, and it would make
for some interesting uses of the Yahoo!-HP-Intel cirrus testbed
Runping Qi wrote:
If you may have turned on ipv6 on your hadoop cluster, it may cause severe
performance hit!
Suggested Workaround:
Either set -Djava.net.preferIPv4Stack=true for the child process option,
which forces Java to use IPv4
instead, or you disable IPv6 entirely in the system.
1 - 100 of 202 matches
Mail list logo
