Re: New to hadoop..plz help

2015-04-18 Thread Marco Shaw
Hi, We are usually here to help, but not really to do this all for you. If you need someone to write this for you, there are some freelance sites on the Internet where you can post a job or request and have people bid on it. Marco On Sat, Apr 18, 2015 at 7:13 PM, shanthi k wrote: > I need ma

Re: Self Study Project in Hadoop -----

2015-01-03 Thread Marco Shaw
Proof Of Concept > On Jan 3, 2015, at 4:12 PM, Publius wrote: > > What is POC, please? > > > > > > > > KMG 365 > > Putin for President :O > > The most wretched device devised by man ..is a Computer :) > > Fight back SPAM: http://www.spamcop.net > > From: Abhishek Singh <23singhabhi

Re: New to this group.

2015-01-02 Thread Marco Shaw
5, at 3:58 PM, Krish Donald wrote: > > I know shell scripting , and I believe I should be able to lean Pig atleast > ... > >> On Fri, Jan 2, 2015 at 11:52 AM, Marco Shaw wrote: >> One thing you might have to realize is that you will likely need to get some >> kind of p

Re: New to this group.

2015-01-02 Thread Marco Shaw
One thing you might have to realize is that you will likely need to get some kind of programming knowledge. You don't have to be an expert, but I don't know that many orgs have dedicated Hadoop administrators that don't have to touch some code at some point, even if to integrate components togethe

Re: Need some tutorials for Mapreduce written in Python

2014-08-27 Thread Marco Shaw
You might want to consider the Hadoop course on udacity.com. I think it provides a decent foundation to Hadoop/MapReduce with a focus on Python (using the streaming API like Sebastiano mentions). Marco On Wed, Aug 27, 2014 at 3:13 PM, Amar Singh wrote: > Hi Users, > I am new to big data world

Re: Hadoop virtual machine

2014-07-06 Thread Marco Shaw
Note that the CDH link is for Cloudera which only provides Hadoop for Linux. HDP has "pre-built VMs" for both Linux and Windows hosts. You can also search for "HDInsight emulator" which runs on Windows and is based on HDP. Marco > On Jul 6, 2014, at 12:38 AM, Gavin Yue wrote: > > http://h

Re: The future of MapReduce

2014-07-01 Thread Marco Shaw
to burn things down to their essentials. If > MapReduce isn’t going to be part of the ecosystem in the future, one does > not want to spend hours learning how to write MapReduce jobs. > > B. > > *From:* Marco Shaw > *Sent:* Tuesday, July 01, 2014 3:50 PM > *To:* user > *

Re: The future of MapReduce

2014-07-01 Thread Marco Shaw
, 2014 at 3:07 PM, Adaryl "Bob" Wakefield, MBA < adaryl.wakefi...@hotmail.com> wrote: > From your answer, it sounds like you need to be able to do both. > > *From:* Marco Shaw > *Sent:* Tuesday, July 01, 2014 10:24 AM > *To:* user > *Subject:* Re: The future

Re: The future of MapReduce

2014-07-01 Thread Marco Shaw
It depends... It seems most are evolving from needing "lots of data crunched", to "lots of data crunched right now". Most are looking for *real-time* fraud detection or recommendations, for example, which MapReduce is not ideal for. Marco On Tue, Jul 1, 2014 at 12:00 PM, Adaryl "Bob" Wakefield

Re: MapReduce scalability study

2014-05-22 Thread Marco Shaw
I would consider the timeframe that you are looking for to determine if you should focus on Hadoop 2.x (with YARN) or older. 2.x should scale much better than 1.x. Keep in mind that 2.x was only "officially" released late last year. Marco > On May 22, 2014, at 5:17 PM, Sylvain Gault wrote:

Re: running YARN in Production

2014-04-24 Thread Marco Shaw
A bit too many mentions of "Cloudera" for this list... Please consider going to a Cloudera list and asking this there asking for specific examples. Marco On Thu, Apr 24, 2014 at 5:17 PM, Matt K wrote: > We run a number of mission-critical MapReduce jobs daily in our production > cluster, most

Re: Is hdinsights a C# version of hadoop or is it in java.

2014-03-13 Thread Marco Shaw
It is based on Java (uses Hortonworks), however, Microsoft provides a .NET SDK: http://hadoopsdk.codeplex.com Marco > On Mar 14, 2014, at 2:32 AM, Sai Sai wrote: > > Is hdinsights a C# version of hadoop or is it in java. > Please let me know. > Thanks > Sai

Re: Best class for currency

2014-02-26 Thread Marco Shaw
> I believe there must be a BigDecimalWritable in hadoop. > > regards > > >> On Wed, Feb 26, 2014 at 2:58 PM, Marco Shaw wrote: >> (If code is required, I can send it along later.) >> >> I'm a beginner and I'm having issues with MR when trying

Best class for currency

2014-02-26 Thread Marco Shaw
(If code is required, I can send it along later.) I'm a beginner and I'm having issues with MR when trying to read values of the form 99.99. I'm reading up as much as I can. I wanted to try to use Java's types to determine whether to use a DoubleWritable or FloatWritable, but after some resea

Old vs new MR API

2014-02-26 Thread Marco Shaw
Tom White's Hadoop book refers to old vs new MR APIs. Is this strictly a MR1 vs MR2 (YARN) comparison, or something before MR2?

Re: How to learn hadoop follow Tom White

2014-01-22 Thread Marco Shaw
I'm pretty sure that site is illegal... > On Jan 23, 2014, at 3:22 AM, Cooleaf wrote: > > thanks all for comment on my concern. by the way, are those books in the > website are copy right free? > > > > 2014/1/23 Abirami V >> From the below site we can download hadoop related books. Hope thi

Re: Question about Yarn

2014-01-19 Thread Marco Shaw
Can you clarify? What tutorial and specific sections are you referring to? Marco > On Jan 19, 2014, at 9:26 AM, chandler song wrote: > > hi all > > I have some question about yarn when I read the tutorial on the website. > > 1) the contain is physical or logic? for example, there are thr

Re: Hadoop- Network ( protocols) information

2014-01-18 Thread Marco Shaw
Hi. I think you are asking for quite a bit without any indication that you have tried to lookup this information yourself. Start with your favourite search engine. and look for "hadoop firewall", for example. One example http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-1.2.0/bk_reference/

Re: Expressions in MapReduce

2014-01-12 Thread Marco Shaw
I'm not sure on your exact details, but it is always possible to do just about anything with your MR logic. Now, to suggests that there may or may not be a performance hit is a different story... Marco On Sat, Jan 11, 2014 at 2:45 PM, unmesha sreeveni wrote: > Are we able to do expresions in M

Re: Monitor network traffic in hadoop

2013-12-08 Thread Marco Shaw
Try something like: http://tutorialgenius.blogspot.ca/2012/03/tcpdump-localhost.html After you launch tcpdump it mentions "eth0", while your mapred-site.xml is referring to "localhost" or "lo". Perhaps... (Sorry, I don't have a VM running and can't remember if it's "lo" = "elle-o".) On Sun, De

Re: Writing to remote HDFS using C# on Windows

2013-12-06 Thread Marco Shaw
Does this help? http://blogs.msdn.com/b/data_otaku/archive/2013/08/26/hadoop-for-net-developers-programmatically-loading-data-to-hdfs.aspx I'm assuming you can use Microsoft's .NET SDK to write to a non-HDInsight cluster... Marco On Fri, Dec 6, 2013 at 1:58 AM, Fengyun RAO wrote: > Thanks! >

Re: Cloudera Vs Hortonworks Vs MapR

2013-09-12 Thread Marco Shaw
Hi I don't this is the approprite place to discuss this.  This list should be a vendor-neutral service.  You arr encouraged to do your own research or look through the popular search engines for others who may have already done such an anlys

Re: There are no dumb questions

2013-08-15 Thread Marco Shaw
Compleltely agree. For basic questions that show no interest or effort, I refer users to sites like elance.com.  When a user comes to a free site with a laundry list of items that they need "right away" or to "impress their boss" then I usually leave that community when this b

Re: Experimental Hadoop Cluster - Linux Windows machines

2013-06-01 Thread Marco Shaw
Hi, If you can't install 64-bit, simply install 32-bit. Basically, it means you have access to a smaller amount of RAM, even if you're using a computer with. For example http://jeffhoogland.blogspot.ca/2010/08/32-bit-vs-64-bit-linux-which-to-choose.html Honestly, getting Linux on your computers

Re: Experimental Hadoop Cluster - Linux Windows machines

2013-06-01 Thread Marco Shaw
complicated procedure? Some kind of drivers > and issues, is it common to expect ? > > > On Sat, Jun 1, 2013 at 4:05 PM, Marco Shaw wrote: > >> If you're running XP, be aware of the others suggesting 64-bit. That >> depends on the exact proc you're running. >

Re: Experimental Hadoop Cluster - Linux Windows machines

2013-06-01 Thread Marco Shaw
If you're running XP, be aware of the others suggesting 64-bit. That depends on the exact proc you're running. You sort of need to break this down into first determining how to get Linux on your systems. RHEL is pretty costly for a test, unless you've got that covered. Go with CentOS for a proo

Re: R for Hadoop

2013-05-19 Thread Marco Shaw
You can try to search for "Rhadoop" using your favourite search engine.  I think you are going to have to put in a bit more effort on your own.  Marco

Re: Urgent Requirement: How to copy File from One cluster to another cluster using java client(throught java Program)

2013-03-01 Thread Marco Shaw
Hi Samir, I may be alone here, but I would prefer you not use "urgent" when asking for free help from a mailing list. My recommendation is that if this is really urgent and you need instant support for your Hadoop installation, that you consider getting a proper support contract to help you when

Re: mapr videos question

2013-02-23 Thread Marco Shaw
Sorry. Can you provide some specific links? Marco On 2013-02-23, at 5:37 AM, Sai Sai wrote: > > Hi > Could some one please verify if the mapr videos are meant for learning hadoop > or is it for learning mapr. If we r interested in learning hadoop only then > will they help. As a starter woul

Re: Hadoop admin training recommendation?

2013-02-21 Thread Marco Shaw
Check out the Cloudera (.com) and Hortonworks (.com) web sites, for examples of their admin training courses. Marco On Thu, Feb 21, 2013 at 12:34 PM, Guy Matz wrote: > Hello! Anyone in the NYC area recommend any of the hadoop training classes > for administrators? > > Thanks a lot! > Guy

Book - Pro Hadoop 2?

2013-01-29 Thread Marco Shaw
http://www.amazon.com/Pro-Hadoop-2-Jason-Venner/dp/1430248637/ Anyone have any inside information on this book? Amazon has a date of March 2013, while Apress doesn't list the 2nd edition at all... Marco

Re: Hadoop Tutorial help

2012-12-09 Thread Marco Shaw
I typically suggest Cloudera to get packaged VMs: https://ccp.cloudera.com/display/SUPPORT/Demo+VMs On 2012-12-09, at 11:53 AM, Jennifer Lopez wrote: > I am going through the tutorial presented @ > http://developer.yahoo.com/hadoop/tutorial/module3.html#vm-jobs > > I have installed vmware and

Re: Reg: No space left on device Exception

2012-12-07 Thread Marco Shaw
Another tip... Is this a 32-bit Linux system? Check for any files that have grown to 2GB. Marco On Fri, Dec 7, 2012 at 1:38 AM, Manoj Babu wrote: > Hi All, > > I am getting the exception as below but the job continues to running how > to avoid this and whats the cause for the issue? > > 12/1

Re: Sample questions for taking Cloudera CDH3 exam for Hadoop developer

2012-11-07 Thread Marco Shaw
ams > > > On Wed, Nov 7, 2012 at 6:54 PM, Marco Shaw wrote: > >> Keep in mind that v3 exams will be retired at the end of this year... >> >> http://university.cloudera.com/certification.html >> >> On Wed, Nov 7, 2012 at 8:50 AM, Ramasubramanian Narayanan <

Re: Sample questions for taking Cloudera CDH3 exam for Hadoop developer

2012-11-07 Thread Marco Shaw
Keep in mind that v3 exams will be retired at the end of this year... http://university.cloudera.com/certification.html On Wed, Nov 7, 2012 at 8:50 AM, Ramasubramanian Narayanan < ramasubramanian.naraya...@gmail.com> wrote: > Hi, > > Can anyone suggest sample model questions for taking the Cloud