1. I've also given it an absolute local path. I don't know what you mean by an absolute cache path. How do I know what that is? The examples use ./link to access the cached file. 2. Because all examples do so. What paths should I use to access the distributed cache from inside exec?
Exception does say that passed is missing. But as I read the examples, it should be there. On Monday, January 6, 2014, Serega Sheypak wrote: > Yes it works. Exception clearly says that ./passwd is missing. > 1. Try to give absolute path to file, see if it works. It should. > 2. Then give relative path. Looks like you incorrectly provide relative > path. why do you put "./" before filename? > > > 2014/1/6 Russell Jurney <[email protected] <javascript:;>> > > > I have implemented to class below to test the udf cache, and it fails in > > local mode with the error below. That cache should work in local mode as > > well, right? > > > > ------------ > > > > org.apache.pig.backend.executionengine.ExecException: ERROR 2078: Caught > > error from UDF: datafu.pig.text.Udfcachetest [./passwd (No such file or > > directory)] > > > > at > > > > > org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:358) > > > > at > > > > > org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNextString(POUserFunc.java:432) > > > > at > > > > > org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:315) > > > > at > > > > > org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:378) > > > > at > > > > > org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:298) > > > > at > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:282) > > > > at > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277) > > > > at > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64) > > > > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) > > > > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) > > > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) > > > > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212) > > > > Caused by: java.io.FileNotFoundException: ./passwd (No such file or > > directory) > > > > at java.io.FileInputStream.open(Native Method) > > > > at java.io.FileInputStream.<init>(FileInputStream.java:146) > > > > at java.io.FileInputStream.<init>(FileInputStream.java:101) > > > > at java.io.FileReader.<init>(FileReader.java:58) > > > > at datafu.pig.text.Udfcachetest.exec(Udfcachetest.java:22) > > > > at datafu.pig.text.Udfcachetest.exec(Udfcachetest.java:19) > > > > at > > > > > org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:330) > > ----------------------- > > > > package datafu.pig.text; > > > > import org.apache.pig.EvalFunc; > > import org.apache.pig.data.Tuple; > > > > import java.io.BufferedReader; > > import java.io.FileReader; > > import java.io.IOException; > > import java.util.ArrayList; > > import java.util.List; > > > > /** > > * Created with IntelliJ IDEA. > > * User: rjurney > > * Date: 1/5/14 > > * Time: 8:32 PM > > * To change this template use File | Settings | File Templates. > > */ > > public class Udfcachetest extends EvalFunc<String> { > > > > public String exec(Tuple input) throws IOException { > > FileReader fr = new FileReader("./passwd"); > > BufferedReader d = new BufferedReader(fr); > > return d.readLine(); > > } > > > > public List<String> getCacheFiles() { > > List<String> list = new ArrayList<String>(1); > > list.add("/etc/passwd"); > > return list; > > } > > } > > > > -- > > Russell Jurney twitter.com/rjurney [email protected]<javascript:;> > > datasyndrome.com > > > -- Russell Jurney twitter.com/rjurney [email protected] datasyndrome.com
