Re: Hadoop and RAID 5

On Sun, Oct 5, 2014 at 4:17 PM, Ulul wrote: > Hi Travis > > Thank you for your detailed answer and for honoring my question with a > blog entry :-) > No problem. I had been meaning to write something up. Thanks for the prod. :-) > > I will look into bus quiescing with admins but I'm under t

Re: Reduce fails always

It was the space issue. It works perfectly after adding this to mapped.site xml file. mapred.local.dir /mnt Thanks for the help. :) Thanks & Regards, Abdul Navaz Research Assistant University of Houston Main Campus, Houston TX Ph: 281-685-0388 From: Ulul Reply-To: Date: Monday, Oct

Re: [Blog] Doubts On CCD-410 Sample Dumps on Ecosystem Projects

Hi Pradeep You are right. Updated the right answers in the blog. This may help anyone thinking about investing in that particular test package. On Tue, Oct 7, 2014 at 9:25 AM, Pradeep Gollakota wrote: > That's not exactly what the question is asking for... It's saying that you > have a bunch

Re: [Blog] Doubts On CCD-410 Sample Dumps on Ecosystem Projects

That's not exactly what the question is asking for... It's saying that you have a bunch of weblogs in HDFS that you want to join with user profile data that is stored in your OLTP database, how do you do the join? First, you export your OLTP database into HDFS using Sqoop. Then you can use Pig/Hive

Re: [Blog] Doubts On CCD-410 Sample Dumps on Ecosystem Projects

What I feel like is For question ​ 5​ it says, the weblogs are already in HDFS (so no need to import anything).Also these are log files, NOT database files with a specific schema. So ​ I think​ Pig is the best way to access and process this data. On Tue, Oct 7, 2014 at 4:10 AM, Pradeep Gollakota

Re: [Blog] Doubts On CCD-410 Sample Dumps on Ecosystem Projects

I agree with the answers suggested above. 3. B 4. D 5. C On Mon, Oct 6, 2014 at 2:58 PM, Ulul wrote: > Hi > > No, Pig is a data manipulation language for data already in Hadoop. > The question is about importing data from OLTP DB (eg Oracle, MySQL...) to > Hadoop, this is what Sqoop is for (SQ

Re: ext4 on a hadoop cluster datanodes

For filesystem creation, we use the following with mkfs.ext4 mkfs.ext4 -T largefile -m 1 -O dir_index,extent,sparse_super -L $HDFS_LABEL /dev/${DEV}1 By default, mkfs creates way too many inodes, so we tune it a bit with the "largefile" option, which modifies the inode_ratio. This gives us ~2 mi

Re: [Blog] Doubts On CCD-410 Sample Dumps on Ecosystem Projects

Hi No, Pig is a data manipulation language for data already in Hadoop. The question is about importing data from OLTP DB (eg Oracle, MySQL...) to Hadoop, this is what Sqoop is for (SQL to Hadoop) I'm not certain certification guys are happy with their exam questions ending up on blogs and mai

ext4 on a hadoop cluster datanodes

Hi, I'm trying to figure out what are more ideal settings for using ext4 on hadoop cluster datanodes. From the hadoop site its recommended nodelalloc option is chosen in the fstab. Is that still a preferred option? I read elsewhere to disable the ext4 journal, and use data=writeback. http://feni

Re: Reduce fails always

Hello Did you check you don't have a job.setNumReduceTasks(1); in your job driver ? And you should check the number of slots available on the jobtracker web interface Ulul Le 06/10/2014 20:34, Abdul Navaz a écrit : Hello, I have 8 Datanodes and each having storage capacity of only 3GB. I

Re: Reduce phase of wordcount

Nice ! mapred.reduce.tasks affects the job (the group of tasks) so it should be at least equal to mapred.tasktracker.reduce.tasks.maximum * nodes> With your setup you allow each of your 7 tasktrackers to launch 8 reducers (that would be 56) but you limit the total number of reducers at 7...

Re: How to login a user with password to Kerberos Hadoop instead of ticket cache or key tab file ?

Larry, Thanks and you have a nice day! Best regards, Sophia On Mon, Oct 6, 2014 at 12:08 PM, Larry McCay wrote: > Well, it seems to be committed to branch-2 - so I assume it will make it > into the next 2.x release. > > > On Mon, Oct 6, 2014 at 2:51 PM, Xiaohua Chen wrote: >> >> Hi Larry, >>

Re: Reduce phase of wordcount

Hi folks, just as a feeback: increasing mapred.tasktracker.reduce.tasks.maximum had no effect (it was already set to 8) and the job created only 1 reducer (my original scenario). However, adding mapred.reduce.tasks and setting to some higher than 1 value (I´ve set to 7) made hadoop spawn that

Re: How to login a user with password to Kerberos Hadoop instead of ticket cache or key tab file ?

Well, it seems to be committed to branch-2 - so I assume it will make it into the next 2.x release. On Mon, Oct 6, 2014 at 2:51 PM, Xiaohua Chen wrote: > Hi Larry, > > Thanks! This is the very right approach I am looking for. Currently > I am using Hadoop 2.3.0 , seems this API > UserGroupInfo

Re: How to login a user with password to Kerberos Hadoop instead of ticket cache or key tab file ?

Hi Larry, Thanks! This is the very right approach I am looking for. Currently I am using Hadoop 2.3.0 , seems this API UserGroupInformation.getUGIFromSubject(subject) is only available from Hadoop 3.0.0 , which seems is not released yet. So when can I expect to get the downloadable for Hadoop 3.0

Re: Reduce fails always

Hello, I have 8 Datanodes and each having storage capacity of only 3GB. I am running word count on 1GB of text file. Initially df ­h shows it has 2.8GB after HDFS write. When Shuffling Starts it goes on consuming the disc space of only one node. I think it is the reducer. Finally df ­h shows 2MB

Hadoop 2.5 datanode error: IncorrectVersionException: Unexpected version of storage directory ... Reported: -57. Expecting = -55

Hi List I have a hadoop 2.5 namenode communicating with a single datanode: When I run start-hdfs.sh on the name node, I see the datanode process initially start up on the node, then fail with the following exception: --- 2014-10-06 21:12:39,835 FATAL org.apache.hadoop.hdfs.server.datanode.DataN

Re: How to login a user with password to Kerberos Hadoop instead of ticket cache or key tab file ?

You may find this approach interesting. https://issues.apache.org/jira/browse/HADOOP-10342 The idea is that you preauthenticate using JAAS/krb5 or something in your application and then leverage the resulting java Subject to assert the authenticated identity. On Mon, Oct 6, 2014 at 1:51 PM, Xiaoh

How to login a user with password to Kerberos Hadoop instead of ticket cache or key tab file ?

Hi Experts, We have a use case which needs to login user into Kerberos hadoop using the kerberos user's name and password. I have searched around and only found that 1) one can login a user from ticket cache ( this is the default one) or 2) login a user from this user's keytab file e.g. UserGr

Re: Unsubscribe

Thank you. On Sat, Oct 4, 2014 at 10:38 AM, Ted Yu wrote: > send email to user-unsubscr...@hadoop.apache.org please. > > On Sat, Oct 4, 2014 at 4:56 AM, Daneille Miller > wrote: > >> Unsubscribe >> > >

Datanode volume full, but not moving to free volume

All, I have a small hadoop cluster (2.5.0) with 4 datanodes and 3 data disks per node. Lately some of the volumes have been filling, but instead of moving to other configured volumes that *have* free space, it's giving errors in the datanode logs: 2014-10-03 11:52:44,989 ERROR org.apache.had

Re: ExportSnapshot webhdfs problems

I’ll try hftp next time. This is an intermittent problem. Mapper retries will typically complete eventually. I got past the problem by tolerating more mapper failures before killing the job. On Oct 6, 2014, at 9:24 AM, Artem Ervits wrote: > Does it work with a small table? I prefer to use hf

Re: ExportSnapshot webhdfs problems

Does it work with a small table? I prefer to use hftp instead of webhdfs. From: Brian Jeltema [mailto:brian.jelt...@digitalenvoy.net] Sent: Friday, October 03, 2014 11:01 AM To: user@hadoop.apache.org Subject: ExportSnapshot webhdfs problems I posted this on users@hbase, but got not response, so

Re: [Blog] Doubts On CCD-410 Sample Dumps on Ecosystem Projects

what about the last one? The answer is correct. Pig. Is nt it? On Mon, Oct 6, 2014 at 4:29 PM, adarsh deshratnam < adarsh.deshrat...@gmail.com> wrote: > For question 3 answer should be B and for question 4 answer should be D. > > Thanks, > Adarsh D > > Consultant - BigData and Cloud > > [image: V

Re: [Blog] Doubts On CCD-410 Sample Dumps on Ecosystem Projects

For question 3 answer should be B and for question 4 answer should be D. Thanks, Adarsh D Consultant - BigData and Cloud [image: View my profile on LinkedIn] On Mon, Oct 6, 2014 at 2:25 PM, unmesha sreeveni wrote: > Hi > > 5 th question can it be S

RE: Reduce fails always

Hi How much data does wordcount job is processing? What is the disk space ("df -h" ) available in the node where it always fail? The point I didn't understand is why it uses only one datanode disc space? >> For reducers task running, containers can be allocated at any node. I >> think, in your

Re: [Blog] Doubts On CCD-410 Sample Dumps on Ecosystem Projects

Hi 5 th question can it be SQOOP? On Mon, Oct 6, 2014 at 1:24 PM, unmesha sreeveni wrote: > Yes > > On Mon, Oct 6, 2014 at 1:22 PM, Santosh Kumar > wrote: > >> Are you preparing g for Cloudera certification exam? >> >> >> >> >> >> Thanks and Regards, >> >> Santosh Kumar SINHA

[Blog] Doubts On CCD-410 Sample Dumps on Ecosystem Projects

http://www.unmeshasreeveni.blogspot.in/2014/09/what-do-you-think-of-these-three.html -- *Thanks & Regards * *Unmesha Sreeveni U.B* *Hadoop, Bigdata Developer* *Center for Cyber Security | Amrita Vishwa Vidyapeetham* http://www.unmeshasreeveni.blogspot.in/