Re: COALESCE UDF?

2013-09-04 Thread Serega Sheypak
Use simple jython UDF. Its 3 lines of code 04.09.2013 18:04 пользователь Sajid Raza windcl...@gmail.com написал: For my two cents' worth I agree, it's nicer to have coalesce than the conditional operator. On Sep 4, 2013, at 8:50 AM, Something Something mailinglist...@gmail.com wrote: What

Re: COALESCE UDF?

2013-09-04 Thread Something Something
What if you've 10 fields? On Wed, Sep 4, 2013 at 4:30 AM, Ruslan Al-Fakikh metarus...@gmail.comwrote: Hi, I think you could mimic it with an expression like this: b = foreach a generate ((field1 is null) ? ((field2 is null) ? null : field2) : field1); Hope that helps, Ruslan On Wed,

Re: COALESCE UDF?

2013-09-04 Thread Russell Jurney
On the other hand, if you use a java UDF, consider contributing it. Sent from my iPhone On Sep 4, 2013, at 9:10 AM, Serega Sheypak serega.shey...@gmail.com wrote: Use simple jython UDF. Its 3 lines of code 04.09.2013 18:04 пользователь Sajid Raza windcl...@gmail.com написал: For my two

Pig 0.11.1 OutOfMemory error

2013-09-04 Thread Shubham Chopra
Hi, I have a relatively large pig scripts (around 1.5k lines, 85 assignments). Around 150 columns are getting projected, joined, grouped and aggregated ending in multiple stores. Pig 0.11.1 fails with the following error even before any jobs are fired: Pig Stack Trace --- ERROR 2998:

Re: COALESCE UDF?

2013-09-04 Thread Something Something
Serega - I think you missed this line: I know it will be very (very) easy to write this, but just don't want to create one if one already exists. There's a saying, don't code something that's already been coded! Code Reuse for the win! Anyway, seems like the simple answer is NO, this

Join Question

2013-09-04 Thread F. Jerrell Schivers
Howdy folks, Let's say I have a set of data that looks like this: X, (X1, X2) Y, (Y1, Y2, Y3) So there could be an unknown number of members of each tuple per row. I also have a second set of data that looks like this: X1, 4, 5, 6 X2, 3, 7, 3 I'd like to join these such that I get: X, (X1,

Re: Join Question

2013-09-04 Thread Pradeep Gollakota
I think there's probably some convoluted way to do this. First thing you'll have to do is flatten your data. data1 = A, B _ X, X1 X, X2 Y, Y1 Y, Y2 Y, Y3 Then do a join by B onto you second dataset. This should produce the following data2 = data1::A, data1::B, data2::A, data2::B, data2::C

Re: SchemaTuple doesn't seem to work on YARN

2013-09-04 Thread Jonathan Coveney
Hello! I implemented the SchemaTuple stuff. Glad to hear you're trying it out! I did not test it with YARN at all. It looks like the way that the filesystem and distributed cache work have changed. I myself am not super up on that, but perhaps there is known documentation on how it differs? The