Matthew Hayes created DATAFU-12:
-----------------------------------
Summary: Implement Lead UDF based on version from SQL
Key: DATAFU-12
URL: https://issues.apache.org/jira/browse/DATAFU-12
Project: DataFu
Issue Type: Improvement
Reporter: Matthew Hayes
Min Zhou has provided this suggestion ([Issue #88 on
GitHub|https://github.com/linkedin/datafu/pull/88]):
Lead is an analytic function like Oracle's Lead function. It provides access to
more than one tuple of a bag at the same time without a self join. Given a bag
of tuple returned from a query, LEAD provides access to a tuple at a given
physical offset beyond that position. Generates pairs of all items in a bag.
If you do not specify offset, then its default is 1. Null is returned if the
offset goes beyond the scope of the bag.
Example 1:
{noformat}
register ba-pig-0.1.jar
define Lead datafu.pig.bags.Lead('2');
-- INPUT: ({(1),(2),(3),(4)})
data = LOAD 'input' AS (data: bag {T: tuple(v:INT)});
describe data;
-- OUTPUT: ({((1),(2),(3)),((2),(3),(4)),((3),(4),),((4),,)})
-- OUTPUT SCHEMA: data2: {lead_data: {(elem0: (v: int),elem1: (v:
int),elem2: (v: int))}}
data2 = FOREACH data GENERATE Lead(data);
describe data2;
DUMP data2;
{noformat}
Example 2
{noformat}
register ba-pig-0.1.jar
define Lead datafu.pig.bags.Lead();
-- INPUT:
({(10,{(1),(2),(3)}),(20,{(4),(5),(6)}),(30,{(7),(8)}),(40,{(9),(10),(11)}),(50,{(12),(13),(14),(15)})})
data = LOAD 'input' AS (data: bag {T: tuple(v1:INT,B: bag{T:
tuple(v2:INT)})});
--describe data;
-- OUPUT:
({((10,{(1),(2),(3)}),(20,{(4),(5),(6)})),((20,{(4),(5),(6)}),(30,{(7),(8)})),((30,{(7),(8)}),(40,{(9),(10),(11)})),((40,{(9),(10),(11)}),(50,{(12),(13),(14),(15)})),((50,{(12),(13),(14),(15)}),)})
data2 = FOREACH data GENERATE Lead(data);
--describe data2;
DUMP data2;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
