Hi folks,
I'm experiencing the exact symptoms of HDFS-770
(https://issues.apache.org/jira/browse/HDFS-770) using Spark and a basic HDFS
deployment. Everything is running locally on a single machine. I'm using Hadoop
2.7.3. My HDFS deployment consists of a single 8 TB disk with replication
disa
Hi all,
Is it possible to monitor MR Job counters while a job is running? I'm using
Hadoop 2.7.1 and YARN. I can view counters for past jobs using the history
server, but can't find a way to view them in real time.
Thanks
Hi!
Can anyone tell me what exactly YarnChild is and how I can control the
quantity of child JVMs running in each container? In this case I'm concerned
with the map phase of my MR job. I'm having issues with my containers
running beyond *physical* memory limits and I'm trying to determine the
c
I'm using Hadoop 2.7.1.
I'm running on MR job on 9 nodes. Everything was working smoothly until it
reached (map 99%, reduce 10%). Here's the relevant lines from my
ResourceManager logs:
2016-04-13 14:19:07,930 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySch
edule
I believe I've encountered some curious HDFS behavior using Hadoop 2.7.1.
Unfortunately I'm in a situation where I need to manually migrate the
contents of two volumes used by HDFS to a new volume, on each node. After
doing so there are a few file conflicts coming from the two original
volumes, spe
l possible?
Thanks again,
Joe
-Original Message-
From: Anu Engineer [mailto:aengin...@hortonworks.com]
Sent: Friday, March 04, 2016 4:25 PM
To: Joseph Naegele ; user@hadoop.apache.org
Subject: Re: change HDFS disks on each node
Hi Joe,
As long as you copy all the data in the old di
Hi all,
Each of our N datanodes has attached two 3TB disks. I want to attach new
*replacement* storage to each node, move the HDFS contents to the new
storage, and remove the old volumes. We're using Hadoop 2.7.1.
1. What is the simplest, correct way to do this? Does hot-swapping move data
from o
