Problems installing Hadoop on Windows 7 enterprise N

Hi, Currently I'm trying to install Hadoop 2.6.0 on a Windows 7 enterprise computer. I used the following guide http://www.srccodes.com/p/article/38/build-install-configure-run-apache-hadoop-2.2.0-microsoft-windows-os I already fixed some previous problems by installing a correct version of

which file map and reduce in reading or writing?

Hi, I am looking to YARN MapReduce internals, and I would like know if it possible to know which file a map/reduce function is reading or writing from inside a map or reduce function defined by the user, or simply by the client? Thanks, -- --

Re: which file map and reduce in reading or writing?

Hi, the context object (passed to map(), reduce() and setup()) contains information about the input split, such as the file name. From the top of my head: String fileName = ((FileSplit) context.getInputSplit()).getPath().getName(); Kai Am 09.03.2015 um 16:39 schrieb xeonmailinglist-gmail

How reduce tasks know which partition they should read?

Hi, I am looking to the Yarn mapreduce internals to try to understand how reduce tasks know which partition of the map output they should read. Even, when they re-execute after a crash? I am also looking to the mapreduce source code. Is there any class that I should look to try to

Re: which file map and reduce in reading or writing?

Reducer doesn't have the ((FileSplit) context.getInputSplit()). Just the mapper. I would like to know from java which files a reducer is reading, and to where it is writing. Can I do this? Thanks, On 09-03-2015 15:44, Kai Voigt wrote: Hi, the context object (passed to map(), reduce() and

Forcing kerberos principals to lowercase

Hey everyone, I have a situation where case sensitivity of principals is causing some pains. Here's the set up AD Centrify (Centrify.com) hadoop on RHEL The logins to the cluster are all using LDAP/PAM . When I do something like id then it returns the lowercase version of the user. env return

Don't delete temp files

Hi, During a mapreduce execution, there are some temp configuration files that are created. I want that these temp files won't be removed when a job ends. How I configure this? Thanks -- --

Re: Forcing kerberos principals to lowercase

Sorry - I forgot to mention that I've seen the configuration with Hue, but I want plain CLI access to work. -Matt On Mon, Mar 9, 2015 at 12:01 PM, Matt Davies m...@mattdavies.net wrote: Hey everyone, I have a situation where case sensitivity of principals is causing some pains. Here's the

Re: Problems installing Hadoop on Windows 7 enterprise N

Hey, You might need to skip all the unit tests to finish your build. By doing that, you could add -DskipTests Thanks From: Daniël van Dam daniel.van...@ortec-finance.commailto:daniel.van...@ortec-finance.com Reply-To: user@hadoop.apache.orgmailto:user@hadoop.apache.org

Not able to ping AWS host

Hi, I am trying to setup Hadoop cluster on AWS . After creating an instance, I got the public ip and dns. But I tried to ping it from my windows machine I am not able to ping it. I am not able to logon to machine using putty . It is saying Network timed out. Security group in the AWS cluster

Re: What skills to Learn to become Hadoop Admin

Hi Jay, Is there a blog or anything that talks about setting up this big pet store application? as i looked at the GIT readme file and was a little bit lost. Maybe thats becuase i am new to Hadoop. On Sat, Mar 7, 2015 at 10:34 AM, jay vyas jayunit100.apa...@gmail.com wrote: Setting up vendor

Tasks stuck in UNASSIGNED state - Hadoop 1.2.1

Hi, We have a hadoop 1.2.1 cluster running on EC2. The job tracker is an On Demand box while we task trackers are stop instances all being setup using an AMI. There are normally 100 instances running with about 1500 map and reduce slots. We run hundreds of hive queries everyday, there are no

Re: How reduce tasks know which partition they should read?

The reducers(Fetcher.java) simply ask the Shuffle Service (ShuffleHandler.java) to give them output corresponding to a specific map. The partitioning detail is hidden from the reducers. Thanks, +Vinod On Mar 9, 2015, at 7:56 AM, xeonmailinglist-gmail xeonmailingl...@gmail.com wrote: Hi,

Re: Not able to ping AWS host

Yes security group has all open ports to 0.0.0.0 and yes cluster is under VPC On Mon, Mar 9, 2015 at 5:15 PM, max scalf oracle.bl...@gmail.com wrote: when you say the security group has all open ports, is that open to public (0.0.0.0) or to your specific IP(if so is ur ip correct)? also are