[ https://issues.apache.org/jira/browse/PIG-756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Viraj Bhat resolved PIG-756. ---------------------------- Resolution: Fixed Fix Version/s: 0.7.0 https://issues.apache.org/jira/browse/PIG-1053 fixes this issue. > UDFs should have API for transparently opening and reading files from HDFS or > from local file system with only relative path > ---------------------------------------------------------------------------------------------------------------------------- > > Key: PIG-756 > URL: https://issues.apache.org/jira/browse/PIG-756 > Project: Pig > Issue Type: Bug > Reporter: David Ciemiewicz > Fix For: 0.7.0 > > > I have a utility function util.INSETFROMFILE() that I pass a file name during > initialization. > {code} > define inQuerySet util.INSETFROMFILE(analysis/queries); > A = load 'logs' using PigStorage() as ( date int, query chararray ); > B = filter A by inQuerySet(query); > {code} > This provides a computationally inexpensive way to effect map-side joins for > small sets plus functions of this style provide the ability to encapsulate > more complex matching rules. > For rapid development and debugging purposes, I want this code to run without > modification on both my local file system when I do pig -exectype local and > on HDFS. > Pig needs to provide an API for UDFs which allow them to either: > 1) "know" when they are in local or HDFS mode and let them open and read > from files as appropriate > 2) just provide a file name and read statements and have pig transparently > manage local or HDFS opens and reads for the UDF > UDFs need to read configuration information off the filesystem and it > simplifies the process if one can just flip the switch of -exectype local. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.