[jira] [Commented] (PIG-4202) UDFContext.isFrontend return true for OutputCommitter in local mode

2014-09-29 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151612#comment-14151612 ] Rohini Palaniswamy commented on PIG-4202: - AM (job commit). All static

[jira] [Updated] (PIG-4192) Make ruby udfs work with Spark

2014-09-29 Thread Praveen Rachabattuni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Praveen Rachabattuni updated PIG-4192: -- Summary: Make ruby udfs work with Spark (was: Make ruby udf's work with Spark) Make

[jira] [Created] (PIG-4207) Make python udfs work with Spark

2014-09-29 Thread Praveen Rachabattuni (JIRA)
Praveen Rachabattuni created PIG-4207: - Summary: Make python udfs work with Spark Key: PIG-4207 URL: https://issues.apache.org/jira/browse/PIG-4207 Project: Pig Issue Type: Sub-task

[jira] [Commented] (PIG-4206) e2e tests for Spark can not work in hadoop env

2014-09-29 Thread Praveen Rachabattuni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151687#comment-14151687 ] Praveen Rachabattuni commented on PIG-4206: --- That doesn't seem to fix it.

[jira] [Created] (PIG-4208) Make merge-sparse join work with Spark

2014-09-29 Thread Praveen Rachabattuni (JIRA)
Praveen Rachabattuni created PIG-4208: - Summary: Make merge-sparse join work with Spark Key: PIG-4208 URL: https://issues.apache.org/jira/browse/PIG-4208 Project: Pig Issue Type:

[jira] [Created] (PIG-4209) Make stream work with Spark

2014-09-29 Thread Praveen Rachabattuni (JIRA)
Praveen Rachabattuni created PIG-4209: - Summary: Make stream work with Spark Key: PIG-4209 URL: https://issues.apache.org/jira/browse/PIG-4209 Project: Pig Issue Type: Sub-task

[jira] [Updated] (PIG-4207) Make python udfs work with Spark

2014-09-29 Thread Praveen Rachabattuni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Praveen Rachabattuni updated PIG-4207: -- Description: Related e2e tests: Scripting_[1,3,5,6-8], StreamingPythonUDFs_[1-12] Make

[jira] [Commented] (PIG-3411) pig skewed join with a big table causes “Split metadata size exceeded 10000000”

2014-09-29 Thread Mass Dosage (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151713#comment-14151713 ] Mass Dosage commented on PIG-3411: -- If this is still an issue you might want to try

[jira] [Updated] (PIG-4202) UDFContext.isFrontend return true for OutputCommitter in local mode

2014-09-29 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-4202: Attachment: PIG-4202-2.patch Workaround patch till we have a API in Tez to cleanup for

[jira] [Commented] (PIG-4202) UDFContext.isFrontend return true for OutputCommitter in local mode

2014-09-29 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151903#comment-14151903 ] Daniel Dai commented on PIG-4202: - Why we create a diverge in tez? We can do the clean up in

[jira] [Commented] (PIG-4202) UDFContext.isFrontend return true for OutputCommitter in local mode

2014-09-29 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151905#comment-14151905 ] Rohini Palaniswamy commented on PIG-4202: - hmm. Thought will keep it separate if

Re: Branching Pig 0.14

2014-09-29 Thread Daniel Dai
I will go ahead and create the branch. Please hold off any commit. I will send out another email once I've done. Thanks, Daniel On Wed, Sep 24, 2014 at 9:59 AM, Daniel Dai da...@hortonworks.com wrote: @Prashant, I just posted my review comments @Russell, Pig on Spark is still in development

[jira] [Commented] (PIG-4202) UDFContext.isFrontend return true for OutputCommitter in local mode

2014-09-29 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151916#comment-14151916 ] Bikas Saha commented on PIG-4202: - To be clear, committer are per vertex in Tez and not per

[jira] [Created] (PIG-4210) Drop support for JDK 6 from Pig 0.14

2014-09-29 Thread Rohini Palaniswamy (JIRA)
Rohini Palaniswamy created PIG-4210: --- Summary: Drop support for JDK 6 from Pig 0.14 Key: PIG-4210 URL: https://issues.apache.org/jira/browse/PIG-4210 Project: Pig Issue Type: Improvement

[RESULT][VOTE] Drop support for JDK 6 from Pig 0.14

2014-09-29 Thread Rohini Palaniswamy
My +1 as well. With 6 binding +1s, 5 non-binding +1s and no -1s this vote passes. Created PIG-4210 to take care of the build changes for that. Regards, Rohini On Thu, Sep 18, 2014 at 8:47 AM, Ashutosh Chauhan hashut...@apache.org wrote: +1 On Wed, Sep 17, 2014 at 7:02 PM, Daniel Dai

Re: Branching Pig 0.14

2014-09-29 Thread Daniel Dai
Branch created. You can continue to commit code now. On Mon, Sep 29, 2014 at 10:11 AM, Daniel Dai da...@hortonworks.com wrote: I will go ahead and create the branch. Please hold off any commit. I will send out another email once I've done. Thanks, Daniel On Wed, Sep 24, 2014 at 9:59 AM,

Re: [RESULT] [VOTE] Drop support for Hadoop 0.20 from Pig 0.14

2014-09-29 Thread Rohini Palaniswamy
My +1 as well. With 6 binding +1s, 8 non-binding +1s and no -1s this vote passes. Nothing special to address for this. PIG-3507 which went into Pig 0.14 used UserGroupInformation class without reflection and so Pig 0.14 is already incompatible with Hadoop 0.20. Regards, Rohini On Mon, Sep 22,

[jira] [Commented] (PIG-4202) UDFContext.isFrontend return true for OutputCommitter in local mode

2014-09-29 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151948#comment-14151948 ] Daniel Dai commented on PIG-4202: - We do ask for a vertex level hook not a DAG level. Pig

[jira] [Commented] (PIG-4202) UDFContext.isFrontend return true for OutputCommitter in local mode

2014-09-29 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151964#comment-14151964 ] Bikas Saha commented on PIG-4202: - To be clear, none of my above comment was for processors.

[jira] [Commented] (PIG-4202) UDFContext.isFrontend return true for OutputCommitter in local mode

2014-09-29 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151995#comment-14151995 ] Daniel Dai commented on PIG-4202: - Pig has a single PigOutputCommitter derived from

[jira] [Updated] (PIG-4184) UDF backward compatibility issue after POStatus.STATUS_NULL refactory

2014-09-29 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4184: Status: Patch Available (was: Open) UDF backward compatibility issue after POStatus.STATUS_NULL refactory

[jira] [Updated] (PIG-4175) PIG CROSS operation follow by STORE produces non-deterministic results each run

2014-09-29 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4175: Fix Version/s: 0.14.0 Assignee: Daniel Dai Status: Patch Available (was: Open) PIG

[jira] [Updated] (PIG-4202) UDFContext.isFrontend return true for OutputCommitter in local mode

2014-09-29 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-4202: Attachment: PIG-4202-3.patch [~daijy], I am getting errors with MR in local mode in

[jira] [Updated] (PIG-4210) Drop support for JDK 6 from Pig 0.14

2014-09-29 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-4210: Description: Change default javac.version to 1.7 and publish jdk7 compiled jars to maven from

[jira] [Commented] (PIG-4202) UDFContext.isFrontend return true for OutputCommitter in local mode

2014-09-29 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152267#comment-14152267 ] Daniel Dai commented on PIG-4202: - +1. It is a little weird we get errors if we reuse

[jira] [Updated] (PIG-4202) Reset UDFContext state before OutputCommitter invocations in Tez

2014-09-29 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-4202: Summary: Reset UDFContext state before OutputCommitter invocations in Tez (was:

[jira] [Updated] (PIG-4202) Reset UDFContext state before OutputCommitter invocations in Tez

2014-09-29 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-4202: Issue Type: Sub-task (was: Bug) Parent: PIG-3446 Reset UDFContext state before

[jira] [Resolved] (PIG-4202) Reset UDFContext state before OutputCommitter invocations in Tez

2014-09-29 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy resolved PIG-4202. - Resolution: Fixed Assignee: Rohini Palaniswamy (was: Daniel Dai) Hadoop

[jira] [Created] (PIG-4211) Just register of hive jars for OrcStorage gives CNFE

2014-09-29 Thread Rohini Palaniswamy (JIRA)
Rohini Palaniswamy created PIG-4211: --- Summary: Just register of hive jars for OrcStorage gives CNFE Key: PIG-4211 URL: https://issues.apache.org/jira/browse/PIG-4211 Project: Pig Issue

Build failed in Jenkins: Pig-trunk #1669

2014-09-29 Thread Apache Jenkins Server
See https://builds.apache.org/job/Pig-trunk/1669/changes Changes: [rohini] PIG-4202: Reset UDFContext state before OutputCommitter invocations in Tez (rohini) [daijy] Preparing for 0.15.0 development [daijy] Preparing for release 0.14.0 --

[jira] [Commented] (PIG-4173) Move to Spark 1.x

2014-09-29 Thread Praveen Rachabattuni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152753#comment-14152753 ] Praveen Rachabattuni commented on PIG-4173: --- Looks fine on my end. Had to add

[jira] [Resolved] (PIG-4167) Initial implementation of Pig on Spark

2014-09-29 Thread Praveen Rachabattuni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Praveen Rachabattuni resolved PIG-4167. --- Resolution: Fixed Initial implementation of Pig on Spark