RE: Grunt Shell hangs on Cygwin.

2013-08-02 Thread Sudhir N
I have same problem, I could not find a solution, seems grunt doesn't work on Cygwin, I stopped trying.. I run scripts Sudhir N -Original Message- From: Darpan R [mailto:darpa...@gmail.com] Sent: Friday, August 02, 2013 6:59 PM To: user@pig.apache.org Subject: Grunt Shell hangs on Cygw

Re: debugging java.lang.IndexOutOfBoundsException

2013-08-02 Thread Serega Sheypak
Looks like one of your files is not parsed. By default pig storage thinks that your file is tab delimited. 03.08.2013 2:49 пользователь "Jesse Jaggars" написал: > Hey folks, > > I'm a brand new user and I'm working on my first 'real' script. The idea is > to count web traffic hits by day, user, a

debugging java.lang.IndexOutOfBoundsException

2013-08-02 Thread Jesse Jaggars
Hey folks, I'm a brand new user and I'm working on my first 'real' script. The idea is to count web traffic hits by day, user, and url. At the end I want to join some account information for each user. I'm running into an issue and I'm not sure how to go about debugging my work. The sso_to_accoun

Re: Problem with using CROSS in PIG

2013-08-02 Thread Mehmet Tepedelenlioglu
Looks like a bug. On Aug 2, 2013, at 1:51 AM, Simonffy Szilvia wrote: > Yes, I read your problem with cross. > But for me doesn't goes away, if I use more reducers in cross. (I don't use > join!) > > Changed: > > D = CROSS C, sequence_number parallel 8; > > Execution results after five times

Re: Replace join with custom implementation

2013-08-02 Thread Serega Sheypak
No problem. I've created a bug. It's a root cause: https://issues.apache.org/jira/browse/PIG-3409 2013/8/2 Pradeep Gollakota > Oh... sorry... I missed the part where you were saying that you want to > reimplement the replicated join algorithm > > > On Fri, Aug 2, 2013 at 9:13 AM, Pradeep Gollak

Grunt Shell hangs on Cygwin.

2013-08-02 Thread Darpan R
Hi Guys, I am running Hadoop on local mode on my windows 7 machine (32 Bit). I've installed HIVE/PIG/Hadoop/Java6 all on the C: drive. I am using Cygwin version : 2.819. PIG Version I tried with 0.11 and 0.10 (both I am facing issue) Hadoop Version : 1.1.2 Hive Version : 0.10 Java version : 1.6 mi

Re: During running "store" command, output data file part-m-00000 is missing

2013-08-02 Thread Amit
Hello, Check the HADOOP_CONF_DIR and/or PIG_CLASSPATH environment variables. It might be that you are looking at incorrect HDFS location.   Regards, Amit From: lulynn_2008 To: user Sent: Thursday, August 1, 2013 11:02 PM Subject: During running "store" comm

Re: Replace join with custom implementation

2013-08-02 Thread Pradeep Gollakota
Oh... sorry... I missed the part where you were saying that you want to reimplement the replicated join algorithm On Fri, Aug 2, 2013 at 9:13 AM, Pradeep Gollakota wrote: > join BIG by key, SMALL by key using 'replicated'; > > > On Fri, Aug 2, 2013 at 5:29 AM, Serega Sheypak > wrote: > >> Hi. I

Re: Replace join with custom implementation

2013-08-02 Thread Pradeep Gollakota
join BIG by key, SMALL by key using 'replicated'; On Fri, Aug 2, 2013 at 5:29 AM, Serega Sheypak wrote: > Hi. I've met a problem wth replicated join in pig 0.11 > I have two relations: > BIG (3-6GB) and SMALL (100MB) > I do join them on four integer fields. > It takes up to 30 minutes to join t

Replace join with custom implementation

2013-08-02 Thread Serega Sheypak
Hi. I've met a problem wth replicated join in pig 0.11 I have two relations: BIG (3-6GB) and SMALL (100MB) I do join them on four integer fields. It takes up to 30 minutes to join them. Join runs on 18 reducers: -Xmx=3072mb for Java, 128 GB in total 32 cores on each TaskTracker. So our hardware

Re: Fwd: Problem with using CROSS in PIG

2013-08-02 Thread Simonffy Szilvia
Yes, I read your problem with cross. But for me doesn't goes away, if I use more reducers in cross. (I don't use join!) Changed: D = CROSS C, sequence_number parallel 8; Execution results after five times running: 1. Successfully stored 1 records 2. Successfully stored 2 records 3. Successful