I have implemented to class below to test the udf cache, and it fails in
local mode with the error below. That cache should work in local mode as
well, right?
------------
org.apache.pig.backend.executionengine.ExecException: ERROR 2078: Caught
error from UDF: datafu.pig.text.Udfcachetest [./passwd (No such file or
directory)]
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:358)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNextString(POUserFunc.java:432)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:315)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:378)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:298)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:282)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
Caused by: java.io.FileNotFoundException: ./passwd (No such file or
directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:146)
at java.io.FileInputStream.<init>(FileInputStream.java:101)
at java.io.FileReader.<init>(FileReader.java:58)
at datafu.pig.text.Udfcachetest.exec(Udfcachetest.java:22)
at datafu.pig.text.Udfcachetest.exec(Udfcachetest.java:19)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:330)
-----------------------
package datafu.pig.text;
import org.apache.pig.EvalFunc;
import org.apache.pig.data.Tuple;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
/**
* Created with IntelliJ IDEA.
* User: rjurney
* Date: 1/5/14
* Time: 8:32 PM
* To change this template use File | Settings | File Templates.
*/
public class Udfcachetest extends EvalFunc<String> {
public String exec(Tuple input) throws IOException {
FileReader fr = new FileReader("./passwd");
BufferedReader d = new BufferedReader(fr);
return d.readLine();
}
public List<String> getCacheFiles() {
List<String> list = new ArrayList<String>(1);
list.add("/etc/passwd");
return list;
}
}
--
Russell Jurney twitter.com/rjurney [email protected] datasyndrome.com
