[
https://issues.apache.org/jira/browse/PIG-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12587184#action_12587184
]
Pi Song commented on PIG-86:
----------------------------
There is a demand to be able to read data from databases or external services.
I think we need to modify the load syntax a bit or introduce a new syntax
(whatever suitable) instead of passing a dummy file.
> load and store function should be file independent
> --------------------------------------------------
>
> Key: PIG-86
> URL: https://issues.apache.org/jira/browse/PIG-86
> Project: Pig
> Issue Type: Improvement
> Reporter: Stefan Groschupf
> Priority: Critical
>
> Currently pig is very file system centric for input LoadFunc and output
> StoreFunc.
> However in many buzz usecases the data are not in the dfs directly but come
> from a remote service. Also it is common to store the results into a database
> for a webbased ui.
> There for I suggest to make filepath elements in the gramar optional.
> For example:
> A = load 'mydata' using PigStorage() // " 'myData' " would be optional
> STORE A into 'output.bz2' USING myStore(); // " into 'output.bz2' " would be
> optional.
> Current workaround for implementing Load and Store functions that work
> without file system access is to create temporary dummy files since pig is
> checking for this files, even if the load and store functions do not need
> those file.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
