[
https://issues.apache.org/jira/browse/PIG-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12587209#action_12587209
]
Benjamin Reed commented on PIG-86:
----------------------------------
I believe PIG-55 takes care of this. For a database you would use a LoadFunc
which implements SlicerFactory. Your LoadFunc would then be able to validate
the "filename", which in this case would probably be a spec to a table.
> load and store function should be file independent
> --------------------------------------------------
>
> Key: PIG-86
> URL: https://issues.apache.org/jira/browse/PIG-86
> Project: Pig
> Issue Type: Improvement
> Reporter: Stefan Groschupf
> Priority: Critical
>
> Currently pig is very file system centric for input LoadFunc and output
> StoreFunc.
> However in many buzz usecases the data are not in the dfs directly but come
> from a remote service. Also it is common to store the results into a database
> for a webbased ui.
> There for I suggest to make filepath elements in the gramar optional.
> For example:
> A = load 'mydata' using PigStorage() // " 'myData' " would be optional
> STORE A into 'output.bz2' USING myStore(); // " into 'output.bz2' " would be
> optional.
> Current workaround for implementing Load and Store functions that work
> without file system access is to create temporary dummy files since pig is
> checking for this files, even if the load and store functions do not need
> those file.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
