[jira] [Reopened] (PIG-5453) FLATTEN shifting fields incorrectly

2024-05-17 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi reopened PIG-5453: --- While tracking multiple jiras, I missed that this patch was not put through full unit/e2e tests.   (Thus

[jira] [Commented] (PIG-5453) FLATTEN shifting fields incorrectly

2024-05-16 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847095#comment-17847095 ] Koji Noguchi commented on PIG-5453: --- Sorry, original patch had extra comma causing the compile error for

[jira] [Updated] (PIG-5453) FLATTEN shifting fields incorrectly

2024-05-16 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5453: -- Attachment: pig-5453-v02.patch > FLATTEN shifting fields incorrectly >

[jira] [Resolved] (PIG-5453) FLATTEN shifting fields incorrectly

2024-05-14 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi resolved PIG-5453. --- Fix Version/s: 0.19.0 Hadoop Flags: Reviewed Resolution: Fixed Thanks for the review

[jira] [Resolved] (PIG-5452) Null handling of FLATTEN with user defined schema (as clause)

2024-05-14 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi resolved PIG-5452. --- Fix Version/s: 0.19.0 Resolution: Fixed Thanks for the review Daniel!  Committed to trunk. >

[jira] [Resolved] (PIG-5450) Pig-on-Spark3 E2E ORC test failing with java.lang.VerifyError: Bad return type

2024-05-14 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi resolved PIG-5450. --- Fix Version/s: 0.19.0 Resolution: Fixed Thanks for the review Rohini!  Committed to trunk. >

[jira] [Resolved] (PIG-5446) Tez TestPigProgressReporting.testProgressReportingWithStatusMessage failing

2024-05-14 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi resolved PIG-5446. --- Fix Version/s: 0.19.0 Hadoop Flags: Reviewed Resolution: Fixed Thanks for the review

[jira] [Resolved] (PIG-5448) All TestHBaseStorage tests failing on pig-on-spark3

2024-05-14 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi resolved PIG-5448. --- Fix Version/s: 0.19.0 Hadoop Flags: Reviewed Resolution: Fixed Thanks for the review

[jira] [Resolved] (PIG-5447) Pig-on-Spark TestSkewedJoin.testSkewedJoinOuter failing with NoSuchElementException

2024-05-14 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi resolved PIG-5447. --- Hadoop Flags: Reviewed Resolution: Fixed Thanks for the review Rohini!  Committed to trunk. >

[jira] [Updated] (PIG-5447) Pig-on-Spark TestSkewedJoin.testSkewedJoinOuter failing with NoSuchElementException

2024-05-14 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5447: -- Fix Version/s: 0.19.0 > Pig-on-Spark TestSkewedJoin.testSkewedJoinOuter failing with >

[jira] [Resolved] (PIG-5439) Support Spark 3 and drop SparkShim

2024-05-14 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi resolved PIG-5439. --- Hadoop Flags: Reviewed Resolution: Fixed Thanks for the review Rohini! Committed to trunk >

[jira] [Updated] (PIG-5438) Update SparkCounter.Accumulator to AccumulatorV2

2024-05-14 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5438: -- Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) Thanks

[jira] [Resolved] (PIG-5416) Spark unit tests failing randomly with "java.lang.RuntimeException: Unexpected job execution status RUNNING"

2024-05-14 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi resolved PIG-5416. --- Fix Version/s: 0.19.0 Hadoop Flags: Reviewed Resolution: Fixed Thanks for the review

[jira] [Created] (PIG-5454) Make ParallelGC the default Garbage Collection

2024-04-18 Thread Koji Noguchi (Jira)
Koji Noguchi created PIG-5454: - Summary: Make ParallelGC the default Garbage Collection Key: PIG-5454 URL: https://issues.apache.org/jira/browse/PIG-5454 Project: Pig Issue Type: Bug

[jira] [Updated] (PIG-5453) FLATTEN shifting fields incorrectly

2024-04-18 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5453: -- Attachment: pig-5453-v01.patch Uploading the patch that uses a new field introduced as part of PIG-5201, 

[jira] [Created] (PIG-5453) FLATTEN shifting fields incorrectly

2024-04-15 Thread Koji Noguchi (Jira)
Koji Noguchi created PIG-5453: - Summary: FLATTEN shifting fields incorrectly Key: PIG-5453 URL: https://issues.apache.org/jira/browse/PIG-5453 Project: Pig Issue Type: Bug Components:

[jira] [Updated] (PIG-5452) Null handling of FLATTEN with user defined schema (as clause)

2024-04-15 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5452: -- Description: Follow up from PIG-5201,  {code:java} A = load 'input' as (a1:chararray); B = FOREACH A

[jira] [Updated] (PIG-5452) Null handling of FLATTEN with user defined schema (as clause)

2024-04-12 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5452: -- Attachment: pig-5452-v01.patch Instead of relying on innerfield schema, using the output schema which

[jira] [Created] (PIG-5452) Null handling of FLATTEN with user defined schema (as clause)

2024-04-12 Thread Koji Noguchi (Jira)
Koji Noguchi created PIG-5452: - Summary: Null handling of FLATTEN with user defined schema (as clause) Key: PIG-5452 URL: https://issues.apache.org/jira/browse/PIG-5452 Project: Pig Issue Type:

[jira] [Assigned] (PIG-5416) Spark unit tests failing randomly with "java.lang.RuntimeException: Unexpected job execution status RUNNING"

2024-04-12 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi reassigned PIG-5416: - Assignee: Koji Noguchi > Spark unit tests failing randomly with "java.lang.RuntimeException: >

[jira] [Commented] (PIG-5451) Pig-on-Spark3 E2E Orc_Pushdown_5 failing

2024-03-29 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17832323#comment-17832323 ] Koji Noguchi commented on PIG-5451: --- This was caused by conflict of orc.version.  

[jira] [Created] (PIG-5451) Pig-on-Spark3 E2E Orc_Pushdown_5 failing

2024-03-29 Thread Koji Noguchi (Jira)
Koji Noguchi created PIG-5451: - Summary: Pig-on-Spark3 E2E Orc_Pushdown_5 failing Key: PIG-5451 URL: https://issues.apache.org/jira/browse/PIG-5451 Project: Pig Issue Type: Bug

[jira] [Commented] (PIG-5451) Pig-on-Spark3 E2E Orc_Pushdown_5 failing

2024-03-29 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17832320#comment-17832320 ] Koji Noguchi commented on PIG-5451: --- Full stack trace. {noformat} 2024-03-29 10:57:31,787

[jira] [Updated] (PIG-5450) Pig-on-Spark3 E2E ORC test failing with java.lang.VerifyError: Bad return type

2024-03-29 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5450: -- Attachment: pig-5450-v01.patch It turns out the weird error was coming from conflicting jar. 

[jira] [Commented] (PIG-5450) Pig-on-Spark3 E2E ORC test failing with java.lang.VerifyError: Bad return type

2024-03-29 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17832318#comment-17832318 ] Koji Noguchi commented on PIG-5450: --- Weird full trace. {noformat} 024-03-27 10:50:40,088

[jira] [Created] (PIG-5450) Pig-on-Spark3 E2E ORC test failing with java.lang.VerifyError: Bad return type

2024-03-29 Thread Koji Noguchi (Jira)
Koji Noguchi created PIG-5450: - Summary: Pig-on-Spark3 E2E ORC test failing with java.lang.VerifyError: Bad return type Key: PIG-5450 URL: https://issues.apache.org/jira/browse/PIG-5450 Project: Pig

[jira] [Updated] (PIG-5410) Support Python 3 for streaming_python

2024-03-29 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5410: -- Attachment: pig-5410-v02.patch Testing the patch, it was failing with {noformat} Caused by:

[jira] [Comment Edited] (PIG-5410) Support Python 3 for streaming_python

2024-03-29 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17832317#comment-17832317 ] Koji Noguchi edited comment on PIG-5410 at 3/29/24 9:10 PM: Testing the patch,

[jira] [Updated] (PIG-5449) TestEmptyInputDir failing on pig-on-spark3

2024-03-22 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5449: -- Attachment: pig-5449-v01.patch Before (in spark2 land), this used to work by checking empty list

[jira] [Created] (PIG-5449) TestEmptyInputDir failing on pig-on-spark3

2024-03-22 Thread Koji Noguchi (Jira)
Koji Noguchi created PIG-5449: - Summary: TestEmptyInputDir failing on pig-on-spark3 Key: PIG-5449 URL: https://issues.apache.org/jira/browse/PIG-5449 Project: Pig Issue Type: Bug

[jira] [Updated] (PIG-5448) All TestHBaseStorage tests failing on pig-on-spark3

2024-03-19 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5448: -- Attachment: pig-5448-v01.patch {quote}No task metrics available for jobId 0 {quote} This is actually

[jira] [Updated] (PIG-5439) Support Spark 3 and drop SparkShim

2024-03-19 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5439: -- Attachment: pig-5439-v02.patch Adding missing spark-scala.version. (pig-5439-v02.patch) > Support Spark

[jira] [Created] (PIG-5448) All TestHBaseStorage tests failing on pig-on-spark3

2024-03-19 Thread Koji Noguchi (Jira)
Koji Noguchi created PIG-5448: - Summary: All TestHBaseStorage tests failing on pig-on-spark3 Key: PIG-5448 URL: https://issues.apache.org/jira/browse/PIG-5448 Project: Pig Issue Type: Bug

[jira] [Updated] (PIG-5447) Pig-on-Spark TestSkewedJoin.testSkewedJoinOuter failing with NoSuchElementException

2024-03-06 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5447: -- Attachment: pig-5447-v01.patch There is no simple way to implement hasNext() for this implementation.  I

[jira] [Commented] (PIG-5447) Pig-on-Spark TestSkewedJoin.testSkewedJoinOuter failing with NoSuchElementException

2024-03-06 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824136#comment-17824136 ] Koji Noguchi commented on PIG-5447: --- > However, inside {{{}next(){}}}, it sometimes recursively traverses

[jira] [Commented] (PIG-5447) Pig-on-Spark TestSkewedJoin.testSkewedJoinOuter failing with NoSuchElementException

2024-03-06 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824097#comment-17824097 ] Koji Noguchi commented on PIG-5447: --- I don't see how this ever worked. Iterator under

[jira] [Commented] (PIG-5447) Pig-on-Spark TestSkewedJoin.testSkewedJoinOuter failing with NoSuchElementException

2024-03-06 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824093#comment-17824093 ] Koji Noguchi commented on PIG-5447: --- Full stack trace. {noformat}

[jira] [Created] (PIG-5447) Pig-on-Spark TestSkewedJoin.testSkewedJoinOuter failing with NoSuchElementException

2024-03-06 Thread Koji Noguchi (Jira)
Koji Noguchi created PIG-5447: - Summary: Pig-on-Spark TestSkewedJoin.testSkewedJoinOuter failing with NoSuchElementException Key: PIG-5447 URL: https://issues.apache.org/jira/browse/PIG-5447 Project: Pig

[jira] [Commented] (PIG-5416) Spark unit tests failing randomly with "java.lang.RuntimeException: Unexpected job execution status RUNNING"

2024-02-08 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815770#comment-17815770 ] Koji Noguchi commented on PIG-5416: --- Issue seems to be on the Spark side. For now, added a silly

[jira] [Updated] (PIG-5416) Spark unit tests failing randomly with "java.lang.RuntimeException: Unexpected job execution status RUNNING"

2024-02-08 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5416: -- Attachment: pig-5416-v01.patch > Spark unit tests failing randomly with "java.lang.RuntimeException: >

[jira] [Updated] (PIG-5446) Tez TestPigProgressReporting.testProgressReportingWithStatusMessage failing

2024-02-06 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5446: -- Attachment: pig-5446-v01.patch > Tez TestPigProgressReporting.testProgressReportingWithStatusMessage

[jira] [Updated] (PIG-5446) Tez TestPigProgressReporting.testProgressReportingWithStatusMessage failing

2024-02-06 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5446: -- It seems like "reporter.progress()" is a no-op in Tez pig. Test started failing after upgrading the

[jira] [Commented] (PIG-5446) Tez TestPigProgressReporting.testProgressReportingWithStatusMessage failing

2024-02-06 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17814970#comment-17814970 ] Koji Noguchi commented on PIG-5446: --- From  {code:title=testProgressReportingWithStatusMessage}

[jira] [Created] (PIG-5446) Tez TestPigProgressReporting.testProgressReportingWithStatusMessage failing

2024-02-06 Thread Koji Noguchi (Jira)
Koji Noguchi created PIG-5446: - Summary: Tez TestPigProgressReporting.testProgressReportingWithStatusMessage failing Key: PIG-5446 URL: https://issues.apache.org/jira/browse/PIG-5446 Project: Pig

[jira] [Updated] (PIG-5445) TestTezCompiler.testMergeCogroup fails whenever config is updated

2024-02-05 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5445: -- Attachment: pig-5445-v01.patch I have no understanding of how the cogroup are implemented, but checking

[jira] [Commented] (PIG-5445) TestTezCompiler.testMergeCogroup fails whenever config is updated

2024-02-05 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17814456#comment-17814456 ] Koji Noguchi commented on PIG-5445: --- {quote}Basically 3rd argument to MergeJoinIndexer differed. {quote}

[jira] [Created] (PIG-5445) TestTezCompiler.testMergeCogroup fails whenever config is updated

2024-02-05 Thread Koji Noguchi (Jira)
Koji Noguchi created PIG-5445: - Summary: TestTezCompiler.testMergeCogroup fails whenever config is updated Key: PIG-5445 URL: https://issues.apache.org/jira/browse/PIG-5445 Project: Pig Issue

[jira] [Updated] (PIG-5444) TestFRJoin.testFRJoinOut7 and testFRJoinOut8 failing with Edge already defined error on Tez

2024-01-18 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5444: -- Attachment: pig-5444-v03.patch pig-5444-v02.patch worked on the failing test but it would not pick up

[jira] [Updated] (PIG-5444) TestFRJoin.testFRJoinOut7 and testFRJoinOut8 failing with Edge already defined error on Tez

2024-01-11 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5444: -- Attachment: pig-5444-v02.patch > TestFRJoin.testFRJoinOut7 and testFRJoinOut8 failing with Edge already

[jira] [Updated] (PIG-5444) TestFRJoin.testFRJoinOut7 and testFRJoinOut8 failing with Edge already defined error on Tez

2024-01-11 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5444: -- Attachment: (was: pig-5444-v02.patch) > TestFRJoin.testFRJoinOut7 and testFRJoinOut8 failing with

[jira] [Updated] (PIG-5444) TestFRJoin.testFRJoinOut7 and testFRJoinOut8 failing with Edge already defined error on Tez

2024-01-11 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5444: -- Attachment: (was: pig-5444-v01.patch) > TestFRJoin.testFRJoinOut7 and testFRJoinOut8 failing with

[jira] [Updated] (PIG-5444) TestFRJoin.testFRJoinOut7 and testFRJoinOut8 failing with Edge already defined error on Tez

2024-01-11 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5444: -- Attachment: pig-5444-v02.patch > TestFRJoin.testFRJoinOut7 and testFRJoinOut8 failing with Edge already

[jira] [Updated] (PIG-5444) TestFRJoin.testFRJoinOut7 and testFRJoinOut8 failing with Edge already defined error on Tez

2024-01-11 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5444: -- Attachment: pig-5444-v01.patch > TestFRJoin.testFRJoinOut7 and testFRJoinOut8 failing with Edge already

[jira] [Updated] (PIG-5444) TestFRJoin.testFRJoinOut7 and testFRJoinOut8 failing with Edge already defined error on Tez

2024-01-11 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5444: -- Attaching the patch which basically does two things. > [~rohini] , should I add a predecessor check to

[jira] [Commented] (PIG-5444) TestFRJoin.testFRJoinOut7 and testFRJoinOut8 failing with Edge already defined error on Tez

2023-11-14 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786019#comment-17786019 ] Koji Noguchi commented on PIG-5444: --- This is how it looks when testFRJoinOut8 is run by itself when

[jira] [Commented] (PIG-5444) TestFRJoin.testFRJoinOut7 and testFRJoinOut8 failing with Edge already defined error on Tez

2023-11-13 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17785674#comment-17785674 ] Koji Noguchi commented on PIG-5444: --- [~rohini] , should I add a predecessor check to prevent this merge ?

[jira] [Comment Edited] (PIG-5444) TestFRJoin.testFRJoinOut7 and testFRJoinOut8 failing with Edge already defined error on Tez

2023-11-13 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17785673#comment-17785673 ] Koji Noguchi edited comment on PIG-5444 at 11/13/23 9:44 PM: - Issue seems to be

[jira] [Commented] (PIG-5444) TestFRJoin.testFRJoinOut7 and testFRJoinOut8 failing with Edge already defined error on Tez

2023-11-13 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17785673#comment-17785673 ] Koji Noguchi commented on PIG-5444: --- Issue seems to be happening inside MultiQueryOptimizerTez.java.

[jira] [Created] (PIG-5444) TestFRJoin.testFRJoinOut7 and testFRJoinOut8 failing with Edge already defined error on Tez

2023-11-11 Thread Koji Noguchi (Jira)
Koji Noguchi created PIG-5444: - Summary: TestFRJoin.testFRJoinOut7 and testFRJoinOut8 failing with Edge already defined error on Tez Key: PIG-5444 URL: https://issues.apache.org/jira/browse/PIG-5444

[jira] [Commented] (PIG-5441) Pig skew join tez grace reducer fails to find shuffle data

2023-05-24 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725786#comment-17725786 ] Koji Noguchi commented on PIG-5441: --- It would be nice if you can add a unit test. (However, if you

[jira] [Commented] (PIG-5440) Extra jars needed for hive3

2023-05-12 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722278#comment-17722278 ] Koji Noguchi commented on PIG-5440: --- bq. Can you add space between "orc-shims","aircompressor" before

[jira] [Updated] (PIG-5440) Extra jars needed for hive3

2023-05-12 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5440: -- Attachment: pig-5440-v02.patch > Extra jars needed for hive3 > --- > >

[jira] [Updated] (PIG-5440) Extra jars needed for hive3

2023-05-12 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5440: -- Attachment: pig-5440-v01.patch > Extra jars needed for hive3 > --- > >

[jira] [Created] (PIG-5440) Extra jars needed for hive3

2023-05-12 Thread Koji Noguchi (Jira)
Koji Noguchi created PIG-5440: - Summary: Extra jars needed for hive3 Key: PIG-5440 URL: https://issues.apache.org/jira/browse/PIG-5440 Project: Pig Issue Type: Improvement Reporter:

[jira] [Assigned] (PIG-5437) Add lib and idea folder to .gitignore

2023-05-11 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi reassigned PIG-5437: - Assignee: Alagappan Maruthappan > Add lib and idea folder to .gitignore >

[jira] [Updated] (PIG-5439) Support Spark 3 and drop SparkShim

2023-05-11 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5439: -- Attachment: pig-5439-v01.patch > Support Spark 3 and drop SparkShim > --

[jira] [Created] (PIG-5439) Support Spark 3 and drop SparkShim

2023-05-11 Thread Koji Noguchi (Jira)
Koji Noguchi created PIG-5439: - Summary: Support Spark 3 and drop SparkShim Key: PIG-5439 URL: https://issues.apache.org/jira/browse/PIG-5439 Project: Pig Issue Type: Improvement

[jira] [Updated] (PIG-5438) Update SparkCounter.Accumulator to AccumulatorV2

2023-05-11 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5438: -- Status: Patch Available (was: Open) Since it's a replacement, existing tests should cover the changes

[jira] [Updated] (PIG-5438) Update SparkCounter.Accumulator to AccumulatorV2

2023-05-11 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5438: -- Attachment: pig-5438-v01.patch > Update SparkCounter.Accumulator to AccumulatorV2 >

[jira] [Assigned] (PIG-5438) Update SparkCounter.Accumulator to AccumulatorV2

2023-05-11 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi reassigned PIG-5438: - Assignee: Koji Noguchi > Update SparkCounter.Accumulator to AccumulatorV2 >

[jira] [Assigned] (PIG-5437) Add lib and idea folder to .gitignore

2023-05-11 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi reassigned PIG-5437: - Assignee: Koji Noguchi > Add lib and idea folder to .gitignore >

[jira] [Assigned] (PIG-5437) Add lib and idea folder to .gitignore

2023-05-11 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi reassigned PIG-5437: - Assignee: (was: Koji Noguchi) > Add lib and idea folder to .gitignore >

[jira] [Created] (PIG-5438) Update SparkCounter.Accumulator to AccumulatorV2

2023-05-11 Thread Koji Noguchi (Jira)
Koji Noguchi created PIG-5438: - Summary: Update SparkCounter.Accumulator to AccumulatorV2 Key: PIG-5438 URL: https://issues.apache.org/jira/browse/PIG-5438 Project: Pig Issue Type: Improvement

[jira] [Updated] (PIG-5436) update owasp version

2023-01-16 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5436: -- Attachment: pig-5436-v01.patch > update owasp version > > > Key:

[jira] [Created] (PIG-5436) update owasp version

2023-01-16 Thread Koji Noguchi (Jira)
Koji Noguchi created PIG-5436: - Summary: update owasp version Key: PIG-5436 URL: https://issues.apache.org/jira/browse/PIG-5436 Project: Pig Issue Type: Test Reporter: Koji Noguchi

[jira] [Commented] (PIG-5434) Migrate from log4j to reload4j

2023-01-16 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677555#comment-17677555 ] Koji Noguchi commented on PIG-5434: --- 1 > Migrate from log4j to reload4j > --

[jira] [Comment Edited] (PIG-5434) Migrate from log4j to reload4j

2023-01-16 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677555#comment-17677555 ] Koji Noguchi edited comment on PIG-5434 at 1/17/23 12:21 AM: - +1 was (Author:

[jira] [Commented] (PIG-5417) Replace guava's Files.createTempDir()

2023-01-16 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677553#comment-17677553 ] Koji Noguchi commented on PIG-5417: --- +1 > Replace guava's Files.createTempDir() >

[jira] [Commented] (PIG-5435) pig.exec.reducers.max does not take effect for skewed join

2023-01-16 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677554#comment-17677554 ] Koji Noguchi commented on PIG-5435: --- +1 > pig.exec.reducers.max does not take effect for skewed join >

[jira] [Commented] (PIG-5431) Date datatype is different between Hive 1.x and Hive 3.x

2023-01-15 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677101#comment-17677101 ] Koji Noguchi commented on PIG-5431: --- +1 > Date datatype is different between Hive 1.x and Hive 3.x >

[jira] [Commented] (PIG-5433) Fix test failures with TestHBaseStorage and htrace dependency

2023-01-15 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677100#comment-17677100 ] Koji Noguchi commented on PIG-5433: --- +1 > Fix test failures with TestHBaseStorage and htrace dependency

[jira] [Resolved] (PIG-5424) Upgrade hbase/zookeeper dependencies

2022-10-17 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi resolved PIG-5424. --- Resolution: Duplicate With many dependency update, I missed this one and created the same change in

[jira] [Updated] (PIG-5430) TestTezGraceParallelism failing due to tez log change

2022-10-16 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5430: -- Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) Thanks

[jira] [Resolved] (PIG-5429) Update hbase version from 2.0.0 to 2.4.14

2022-10-16 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi resolved PIG-5429. --- Fix Version/s: 0.18.0 Hadoop Flags: Reviewed Resolution: Fixed Thanks for the review

[jira] [Assigned] (PIG-5430) TestTezGraceParallelism failing due to tez log change

2022-10-13 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi reassigned PIG-5430: - Attachment: pig-5430-v01.patch Fix Version/s: 0.18.0 Assignee: Koji Noguchi

[jira] [Created] (PIG-5430) TestTezGraceParallelism failing due to tez log change

2022-10-13 Thread Koji Noguchi (Jira)
Koji Noguchi created PIG-5430: - Summary: TestTezGraceParallelism failing due to tez log change Key: PIG-5430 URL: https://issues.apache.org/jira/browse/PIG-5430 Project: Pig Issue Type: Test

[jira] [Updated] (PIG-5430) TestTezGraceParallelism failing due to tez log change

2022-10-13 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5430: -- Status: Patch Available (was: Open) > TestTezGraceParallelism failing due to tez log change >

[jira] [Updated] (PIG-5429) Update hbase version from 2.0.0 to 2.4.14

2022-10-12 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5429: -- Attachment: pig-5429-v01.patch Uploading pig-5429-v01.patch changing hbase version from 2.0.0 to 2.4.14.

[jira] [Created] (PIG-5429) Update hbase version from 2.0.0 to 2.4.14

2022-10-12 Thread Koji Noguchi (Jira)
Koji Noguchi created PIG-5429: - Summary: Update hbase version from 2.0.0 to 2.4.14 Key: PIG-5429 URL: https://issues.apache.org/jira/browse/PIG-5429 Project: Pig Issue Type: Improvement

[jira] [Updated] (PIG-5428) Update hadoop2,3 and tez to recent versions

2022-10-12 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5428: -- Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) Thanks

[jira] [Updated] (PIG-5428) Update hadoop2,3 and tez to recent versions

2022-10-12 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5428: -- Status: Patch Available (was: Open) > Update hadoop2,3 and tez to recent versions >

[jira] [Resolved] (PIG-5423) Upgrade hadoop/tez dependency

2022-10-12 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi resolved PIG-5423. --- Resolution: Duplicate Closing this Jira as a duplicate of PIG-5428.  After PIG-5253, some of the

[jira] [Commented] (PIG-5428) Update hadoop2,3 and tez to recent versions

2022-10-12 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17616576#comment-17616576 ] Koji Noguchi commented on PIG-5428: --- For tez 0.10.2, somehow hundreds of test failures in local shuffle.

[jira] [Commented] (PIG-5428) Update hadoop2,3 and tez to recent versions

2022-10-12 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17616575#comment-17616575 ] Koji Noguchi commented on PIG-5428: --- Upgrading to hadoop 3.3.4 resulted over 100

[jira] [Comment Edited] (PIG-5428) Update hadoop2,3 and tez to recent versions

2022-10-12 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17616574#comment-17616574 ] Koji Noguchi edited comment on PIG-5428 at 10/12/22 4:31 PM: - Uploaded

[jira] [Commented] (PIG-5428) Update hadoop2,3 and tez to recent versions

2022-10-12 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17616574#comment-17616574 ] Koji Noguchi commented on PIG-5428: --- Uploaded pig-5428-v01.patch. For hadoop2 * hadoop 2.7.3 -> 2.10.2 *

[jira] [Updated] (PIG-5428) Update hadoop2,3 and tez to recent versions

2022-10-12 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5428: -- Attachment: pig-5428-v01.patch > Update hadoop2,3 and tez to recent versions >

[jira] [Created] (PIG-5428) Update hadoop2,3 and tez to recent versions

2022-10-12 Thread Koji Noguchi (Jira)
Koji Noguchi created PIG-5428: - Summary: Update hadoop2,3 and tez to recent versions Key: PIG-5428 URL: https://issues.apache.org/jira/browse/PIG-5428 Project: Pig Issue Type: Improvement

[jira] [Updated] (PIG-5424) Upgrade hbase/zookeeper dependencies

2022-04-13 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5424: -- Attachment: pig-5424-v02.patch Missed that there were two more releases after the last time I looked.

[jira] [Updated] (PIG-5422) Upgrade guava/groovy dependency

2022-04-13 Thread Koji Noguchi (Jira)
[ https://issues.apache.org/jira/browse/PIG-5422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-5422: -- Attachment: pig-5422-v02.patch bq. Updating guava version will cause issues with hadoop unless we shade

  1   2   3   4   5   6   7   8   9   10   >