[jira] Commented: (PIG-891) Fixing dfs statement for Pig
[ https://issues.apache.org/jira/browse/PIG-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12757951#action_12757951 ] Daniel Dai commented on PIG-891: Patch committed. Thanks Jeff for contributing! > Fixing dfs statement for Pig > > > Key: PIG-891 > URL: https://issues.apache.org/jira/browse/PIG-891 > Project: Pig > Issue Type: Bug >Affects Versions: 0.4.0 >Reporter: Daniel Dai >Assignee: Jeff Zhang >Priority: Minor > Fix For: 0.6.0 > > Attachments: Pig_891.patch > > > Several hadoop dfs commands are not support or restrictive on current Pig. We > need to fix that. These include: > 1. Several commands do not supported: lsr, dus, count, rmr, expunge, put, > moveFromLocal, get, getmerge, text, moveToLocal, mkdir, touchz, test, stat, > tail, chmod, chown, chgrp. A reference for these command can be found in > http://hadoop.apache.org/common/docs/current/hdfs_shell.html > 2. All existing dfs commands do not support globing. > 3. Pig should provide a programmatic way to perform dfs commands. Several of > them exist in PigServer, but not all of them. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-891) Fixing dfs statement for Pig
[ https://issues.apache.org/jira/browse/PIG-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755576#action_12755576 ] Daniel Dai commented on PIG-891: Hi, Jeff, I get response from the Giri who implement this release audit checking. We can ignore the release audit warnings in this case. I will commit the patch shortly. > Fixing dfs statement for Pig > > > Key: PIG-891 > URL: https://issues.apache.org/jira/browse/PIG-891 > Project: Pig > Issue Type: Bug >Affects Versions: 0.4.0 >Reporter: Daniel Dai >Assignee: Jeff Zhang >Priority: Minor > Fix For: 0.4.0 > > Attachments: Pig_891.patch > > > Several hadoop dfs commands are not support or restrictive on current Pig. We > need to fix that. These include: > 1. Several commands do not supported: lsr, dus, count, rmr, expunge, put, > moveFromLocal, get, getmerge, text, moveToLocal, mkdir, touchz, test, stat, > tail, chmod, chown, chgrp. A reference for these command can be found in > http://hadoop.apache.org/common/docs/current/hdfs_shell.html > 2. All existing dfs commands do not support globing. > 3. Pig should provide a programmatic way to perform dfs commands. Several of > them exist in PigServer, but not all of them. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-891) Fixing dfs statement for Pig
[ https://issues.apache.org/jira/browse/PIG-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755081#action_12755081 ] Daniel Dai commented on PIG-891: Not quite sure about it now. But I will figure out and let you know. Thanks. > Fixing dfs statement for Pig > > > Key: PIG-891 > URL: https://issues.apache.org/jira/browse/PIG-891 > Project: Pig > Issue Type: Bug >Affects Versions: 0.4.0 >Reporter: Daniel Dai >Assignee: Jeff Zhang >Priority: Minor > Fix For: 0.4.0 > > Attachments: Pig_891.patch > > > Several hadoop dfs commands are not support or restrictive on current Pig. We > need to fix that. These include: > 1. Several commands do not supported: lsr, dus, count, rmr, expunge, put, > moveFromLocal, get, getmerge, text, moveToLocal, mkdir, touchz, test, stat, > tail, chmod, chown, chgrp. A reference for these command can be found in > http://hadoop.apache.org/common/docs/current/hdfs_shell.html > 2. All existing dfs commands do not support globing. > 3. Pig should provide a programmatic way to perform dfs commands. Several of > them exist in PigServer, but not all of them. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-891) Fixing dfs statement for Pig
[ https://issues.apache.org/jira/browse/PIG-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754519#action_12754519 ] Jeff Zhang commented on PIG-891: Daniel, Do you know what causes the release audit failed ? I do not understand the message in [http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/26/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt] > Fixing dfs statement for Pig > > > Key: PIG-891 > URL: https://issues.apache.org/jira/browse/PIG-891 > Project: Pig > Issue Type: Bug >Affects Versions: 0.4.0 >Reporter: Daniel Dai >Assignee: Jeff Zhang >Priority: Minor > Fix For: 0.4.0 > > Attachments: Pig_891.patch > > > Several hadoop dfs commands are not support or restrictive on current Pig. We > need to fix that. These include: > 1. Several commands do not supported: lsr, dus, count, rmr, expunge, put, > moveFromLocal, get, getmerge, text, moveToLocal, mkdir, touchz, test, stat, > tail, chmod, chown, chgrp. A reference for these command can be found in > http://hadoop.apache.org/common/docs/current/hdfs_shell.html > 2. All existing dfs commands do not support globing. > 3. Pig should provide a programmatic way to perform dfs commands. Several of > them exist in PigServer, but not all of them. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-891) Fixing dfs statement for Pig
[ https://issues.apache.org/jira/browse/PIG-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754513#action_12754513 ] Hadoop QA commented on PIG-891: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12419372/Pig_891.patch against trunk revision 814075. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 278 release audit warnings (more than the trunk's current 276 warnings). +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/26/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/26/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/26/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/26/console This message is automatically generated. > Fixing dfs statement for Pig > > > Key: PIG-891 > URL: https://issues.apache.org/jira/browse/PIG-891 > Project: Pig > Issue Type: Bug >Affects Versions: 0.4.0 >Reporter: Daniel Dai >Assignee: Jeff Zhang >Priority: Minor > Fix For: 0.4.0 > > Attachments: Pig_891.patch > > > Several hadoop dfs commands are not support or restrictive on current Pig. We > need to fix that. These include: > 1. Several commands do not supported: lsr, dus, count, rmr, expunge, put, > moveFromLocal, get, getmerge, text, moveToLocal, mkdir, touchz, test, stat, > tail, chmod, chown, chgrp. A reference for these command can be found in > http://hadoop.apache.org/common/docs/current/hdfs_shell.html > 2. All existing dfs commands do not support globing. > 3. Pig should provide a programmatic way to perform dfs commands. Several of > them exist in PigServer, but not all of them. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-891) Fixing dfs statement for Pig
[
https://issues.apache.org/jira/browse/PIG-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754233#action_12754233
]
Daniel Dai commented on PIG-891:
Tested in local mode also, the patch even works well in local mode. We have
discussed issues in my previous comment, the suggestions are:
1. We can keep existing file system commands for now
2. We shall use "fs" instead of "dfs" to indicate a file system command as
latest hadoop does
Jeff, can you make this little change ("dfs"->"fs") and submit again? Thanks!
> Fixing dfs statement for Pig
>
>
> Key: PIG-891
> URL: https://issues.apache.org/jira/browse/PIG-891
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.4.0
>Reporter: Daniel Dai
>Assignee: Jeff Zhang
>Priority: Minor
> Fix For: 0.4.0
>
> Attachments: Pig_891.patch
>
>
> Several hadoop dfs commands are not support or restrictive on current Pig. We
> need to fix that. These include:
> 1. Several commands do not supported: lsr, dus, count, rmr, expunge, put,
> moveFromLocal, get, getmerge, text, moveToLocal, mkdir, touchz, test, stat,
> tail, chmod, chown, chgrp. A reference for these command can be found in
> http://hadoop.apache.org/common/docs/current/hdfs_shell.html
> 2. All existing dfs commands do not support globing.
> 3. Pig should provide a programmatic way to perform dfs commands. Several of
> them exist in PigServer, but not all of them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-891) Fixing dfs statement for Pig
[ https://issues.apache.org/jira/browse/PIG-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753944#action_12753944 ] Daniel Dai commented on PIG-891: +1. Patch is concise and clear. For the syntax, I need opinion from other developers / users: 1. After this patch, shall we remove existing hadoop file system commands since they are duplicate with the new one 2. Shall we allow both fs/dfs to indicate a dfs command, just like hadoop does? > Fixing dfs statement for Pig > > > Key: PIG-891 > URL: https://issues.apache.org/jira/browse/PIG-891 > Project: Pig > Issue Type: Bug >Affects Versions: 0.4.0 >Reporter: Daniel Dai >Assignee: Jeff Zhang >Priority: Minor > Fix For: 0.4.0 > > Attachments: Pig_891.patch > > > Several hadoop dfs commands are not support or restrictive on current Pig. We > need to fix that. These include: > 1. Several commands do not supported: lsr, dus, count, rmr, expunge, put, > moveFromLocal, get, getmerge, text, moveToLocal, mkdir, touchz, test, stat, > tail, chmod, chown, chgrp. A reference for these command can be found in > http://hadoop.apache.org/common/docs/current/hdfs_shell.html > 2. All existing dfs commands do not support globing. > 3. Pig should provide a programmatic way to perform dfs commands. Several of > them exist in PigServer, but not all of them. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-891) Fixing dfs statement for Pig
[ https://issues.apache.org/jira/browse/PIG-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753935#action_12753935 ] Daniel Dai commented on PIG-891: I am reviewing the patch. > Fixing dfs statement for Pig > > > Key: PIG-891 > URL: https://issues.apache.org/jira/browse/PIG-891 > Project: Pig > Issue Type: Bug >Affects Versions: 0.4.0 >Reporter: Daniel Dai >Assignee: Jeff Zhang >Priority: Minor > Fix For: 0.4.0 > > Attachments: Pig_891.patch > > > Several hadoop dfs commands are not support or restrictive on current Pig. We > need to fix that. These include: > 1. Several commands do not supported: lsr, dus, count, rmr, expunge, put, > moveFromLocal, get, getmerge, text, moveToLocal, mkdir, touchz, test, stat, > tail, chmod, chown, chgrp. A reference for these command can be found in > http://hadoop.apache.org/common/docs/current/hdfs_shell.html > 2. All existing dfs commands do not support globing. > 3. Pig should provide a programmatic way to perform dfs commands. Several of > them exist in PigServer, but not all of them. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-891) Fixing dfs statement for Pig
[ https://issues.apache.org/jira/browse/PIG-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12739652#action_12739652 ] Alan Gates commented on PIG-891: +1 to Pradeep's suggestion. I think it would be better for Pig dump its shell command implementations and have grunt dispatch shell commands to Hadoop. This avoids Pig needing to implement these commands. It also means Pig will automatically be in sync with Hadoop on how shell commands work. > Fixing dfs statement for Pig > > > Key: PIG-891 > URL: https://issues.apache.org/jira/browse/PIG-891 > Project: Pig > Issue Type: Bug >Reporter: Daniel Dai >Assignee: Jeff Zhang >Priority: Minor > > Several hadoop dfs commands are not support or restrictive on current Pig. We > need to fix that. These include: > 1. Several commands do not supported: lsr, dus, count, rmr, expunge, put, > moveFromLocal, get, getmerge, text, moveToLocal, mkdir, touchz, test, stat, > tail, chmod, chown, chgrp. A reference for these command can be found in > http://hadoop.apache.org/common/docs/current/hdfs_shell.html > 2. All existing dfs commands do not support globing. > 3. Pig should provide a programmatic way to perform dfs commands. Several of > them exist in PigServer, but not all of them. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-891) Fixing dfs statement for Pig
[ https://issues.apache.org/jira/browse/PIG-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12739068#action_12739068 ] Pradeep Kamath commented on PIG-891: There have been some discussions around whether hadoop commands should be supported through hadoop fsShell - see https://issues.apache.org/jira/browse/PIG-79. I think that option should be explored before expanding pig's grunt shell - it would cleaner to not have that code in pig since truely hadoop commands should be handled by hadoop's utility code (fsShell). > Fixing dfs statement for Pig > > > Key: PIG-891 > URL: https://issues.apache.org/jira/browse/PIG-891 > Project: Pig > Issue Type: Bug >Reporter: Daniel Dai >Assignee: Jeff Zhang >Priority: Minor > > Several hadoop dfs commands are not support or restrictive on current Pig. We > need to fix that. These include: > 1. Several commands do not supported: lsr, dus, count, rmr, expunge, put, > moveFromLocal, get, getmerge, text, moveToLocal, mkdir, touchz, test, stat, > tail, chmod, chown, chgrp. A reference for these command can be found in > http://hadoop.apache.org/common/docs/current/hdfs_shell.html > 2. All existing dfs commands do not support globing. > 3. Pig should provide a programmatic way to perform dfs commands. Several of > them exist in PigServer, but not all of them. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-891) Fixing dfs statement for Pig
[ https://issues.apache.org/jira/browse/PIG-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738192#action_12738192 ] Daniel Dai commented on PIG-891: Hi Jeff, I am not working on it. We appreciate and need your help on this. > Fixing dfs statement for Pig > > > Key: PIG-891 > URL: https://issues.apache.org/jira/browse/PIG-891 > Project: Pig > Issue Type: Bug >Reporter: Daniel Dai >Priority: Minor > > Several hadoop dfs commands are not support or restrictive on current Pig. We > need to fix that. These include: > 1. Several commands do not supported: lsr, dus, count, rmr, expunge, put, > moveFromLocal, get, getmerge, text, moveToLocal, mkdir, touchz, test, stat, > tail, chmod, chown, chgrp. A reference for these command can be found in > http://hadoop.apache.org/common/docs/current/hdfs_shell.html > 2. All existing dfs commands do not support globing. > 3. Pig should provide a programmatic way to perform dfs commands. Several of > them exist in PigServer, but not all of them. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-891) Fixing dfs statement for Pig
[ https://issues.apache.org/jira/browse/PIG-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738189#action_12738189 ] Jeff Zhang commented on PIG-891: Daniel, are you working on this right now ? If not, I can help resolve this issue. > Fixing dfs statement for Pig > > > Key: PIG-891 > URL: https://issues.apache.org/jira/browse/PIG-891 > Project: Pig > Issue Type: Bug >Reporter: Daniel Dai >Priority: Minor > > Several hadoop dfs commands are not support or restrictive on current Pig. We > need to fix that. These include: > 1. Several commands do not supported: lsr, dus, count, rmr, expunge, put, > moveFromLocal, get, getmerge, text, moveToLocal, mkdir, touchz, test, stat, > tail, chmod, chown, chgrp. A reference for these command can be found in > http://hadoop.apache.org/common/docs/current/hdfs_shell.html > 2. All existing dfs commands do not support globing. > 3. Pig should provide a programmatic way to perform dfs commands. Several of > them exist in PigServer, but not all of them. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
