[
https://issues.apache.org/jira/browse/PIG-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liyunzhang_intel updated PIG-4198:
----------------------------------
Description:
how to use:
CACHE alias
You can use "CACHE" liking following way:
1. hadoop fs -cat /user/root/passwd1
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
2./pig -x spark
grunt> A = load 'passwd1' using PigStorage(':');
CACHE A;
store A into 'id1.out';
3. change the content of /user/root/passwd1
hadoop fs -cat /user/root/passwd1
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
4. store A into id2.out. Although the input file has changed. but the output of
store is the same as previous.
grunt>store A into 'id2.out';
5. compare the output in the id1.out/ and id2.out/. they are same
hadoop fs -cat /user/root/id1.out/*
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
hadoop fs -cat /user/root/id2.out/*
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
> Add operator to pig latin for performing caching
> ------------------------------------------------
>
> Key: PIG-4198
> URL: https://issues.apache.org/jira/browse/PIG-4198
> Project: Pig
> Issue Type: New Feature
> Reporter: liyunzhang_intel
>
> how to use:
> CACHE alias
> You can use "CACHE" liking following way:
> 1. hadoop fs -cat /user/root/passwd1
> root:x:0:0:root:/root:/bin/bash
> bin:x:1:1:bin:/bin:/sbin/nologin
> 2./pig -x spark
> grunt> A = load 'passwd1' using PigStorage(':');
> CACHE A;
> store A into 'id1.out';
> 3. change the content of /user/root/passwd1
> hadoop fs -cat /user/root/passwd1
> daemon:x:2:2:daemon:/sbin:/sbin/nologin
> adm:x:3:4:adm:/var/adm:/sbin/nologin
> 4. store A into id2.out. Although the input file has changed. but the output
> of store is the same as previous.
> grunt>store A into 'id2.out';
> 5. compare the output in the id1.out/ and id2.out/. they are same
> hadoop fs -cat /user/root/id1.out/*
> root:x:0:0:root:/root:/bin/bash
> bin:x:1:1:bin:/bin:/sbin/nologin
> hadoop fs -cat /user/root/id2.out/*
> root:x:0:0:root:/root:/bin/bash
> bin:x:1:1:bin:/bin:/sbin/nologin
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
