[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222964#comment-14222964
]
Rui Li commented on HIVE-8536:
--
With the latest patch, the previous example query plan is:
[
https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8834:
-
Attachment: HIVE-8834.6-spark.patch
Address RB comments. Thanks [~vanzin] for the review!
enable job progress
[
https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224045#comment-14224045
]
Rui Li commented on HIVE-8834:
--
Thanks guys for the review!
enable job progress monitoring
[
https://issues.apache.org/jira/browse/HIVE-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li reassigned HIVE-8956:
Assignee: Rui Li
Hive hangs while some error/exception happens beyond job execution[Spark
Branch]
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224122#comment-14224122
]
Rui Li commented on HIVE-8536:
--
Hi [~szehon],
{{skewjoin_noskew}} failed to run because our
Rui Li created HIVE-8958:
Summary: Make sure map join tasks created by runtime skew join can
fit into memory [Spark Branch]
Key: HIVE-8958
URL: https://issues.apache.org/jira/browse/HIVE-8958
Project: Hive
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224182#comment-14224182
]
Rui Li commented on HIVE-8536:
--
OK I'll take a look.
Enable SkewJoinResolver for spark
[
https://issues.apache.org/jira/browse/HIVE-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8956:
-
Attachment: HIVE-8956.1-spark.patch
Add timeout for waiting a spark job to be submitted.
Hive hangs while some
[
https://issues.apache.org/jira/browse/HIVE-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8956:
-
Status: Patch Available (was: Open)
Hive hangs while some error/exception happens beyond job execution[Spark
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224299#comment-14224299
]
Rui Li commented on HIVE-8536:
--
It seems the dependency task is created in
Rui Li created HIVE-8961:
Summary: Remove unnecessary dependency collection task [Spark
Branch]
Key: HIVE-8961
URL: https://issues.apache.org/jira/browse/HIVE-8961
Project: Hive
Issue Type: Test
[
https://issues.apache.org/jira/browse/HIVE-8961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8961:
-
Component/s: Spark
Remove unnecessary dependency collection task [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-8961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8961:
-
Issue Type: Sub-task (was: Test)
Parent: HIVE-7292
Remove unnecessary dependency collection task [Spark
[
https://issues.apache.org/jira/browse/HIVE-8961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8961:
-
Attachment: HIVE-8961.1-spark.patch
Changes to golden files are all in query plan. No results changed.
Remove
[
https://issues.apache.org/jira/browse/HIVE-8961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8961:
-
Status: Patch Available (was: Open)
Remove unnecessary dependency collection task [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225565#comment-14225565
]
Rui Li commented on HIVE-8536:
--
Hi [~csun] thanks for your input.
I'm removing the dependency
[
https://issues.apache.org/jira/browse/HIVE-8961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225570#comment-14225570
]
Rui Li commented on HIVE-8961:
--
Thanks [~xuefuz] for taking care of this. I'll rebase and run
[
https://issues.apache.org/jira/browse/HIVE-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225614#comment-14225614
]
Rui Li commented on HIVE-8956:
--
Thanks [~vanzin] for your input!
Adding {{JobReceived}} and
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225677#comment-14225677
]
Rui Li commented on HIVE-8536:
--
Thanks [~csun], I can verify this later. For now, let's stick
[
https://issues.apache.org/jira/browse/HIVE-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225721#comment-14225721
]
Rui Li commented on HIVE-8836:
--
Hi [~brocknoland], this may be related to HIVE-8956. Do you
[
https://issues.apache.org/jira/browse/HIVE-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225751#comment-14225751
]
Rui Li commented on HIVE-8836:
--
OK I'll try out the failed tests.
Enable automatic tests
Rui Li created HIVE-8972:
Summary: Implement more fine-grained remote client-level events
[Spark Branch]
Key: HIVE-8972
URL: https://issues.apache.org/jira/browse/HIVE-8972
Project: Hive
Issue
[
https://issues.apache.org/jira/browse/HIVE-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225763#comment-14225763
]
Rui Li commented on HIVE-8956:
--
Yeah I created HIVE-8972 as a follow up.
As for retry failed
[
https://issues.apache.org/jira/browse/HIVE-8961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225826#comment-14225826
]
Rui Li commented on HIVE-8961:
--
Thanks [~xuefuz] for the work.
Remove unnecessary dependency
[
https://issues.apache.org/jira/browse/HIVE-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225860#comment-14225860
]
Rui Li commented on HIVE-8836:
--
Shall we use same #reducers as before for now? So that we can
[
https://issues.apache.org/jira/browse/HIVE-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227107#comment-14227107
]
Rui Li commented on HIVE-8956:
--
OK I see. Thanks [~vanzin] for the explanation.
Hive hangs
[
https://issues.apache.org/jira/browse/HIVE-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227118#comment-14227118
]
Rui Li commented on HIVE-8836:
--
Thanks [~brocknoland] and [~xuefuz] for the work. Maybe we can
[
https://issues.apache.org/jira/browse/HIVE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8984:
-
Issue Type: Sub-task (was: Improvement)
Parent: HIVE-8548
Enable job metrics/statistics gathering for
Rui Li created HIVE-8984:
Summary: Enable job metrics/statistics gathering for remote spark
context [Spark Branch]
Key: HIVE-8984
URL: https://issues.apache.org/jira/browse/HIVE-8984
Project: Hive
[
https://issues.apache.org/jira/browse/HIVE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li reassigned HIVE-8984:
Assignee: Rui Li
Enable job metrics/statistics gathering for remote spark context [Spark
Branch]
[
https://issues.apache.org/jira/browse/HIVE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8984:
-
Description: stats_counter fails due to this issue
Enable job metrics/statistics gathering for remote spark
[
https://issues.apache.org/jira/browse/HIVE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8984:
-
Attachment: HIVE-8984.1-spark.patch
The refactor in this patch is mainly moving spark counter related classes to
[
https://issues.apache.org/jira/browse/HIVE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8984:
-
Status: Patch Available (was: Open)
Enable job metrics/statistics gathering for remote spark context [Spark
[
https://issues.apache.org/jira/browse/HIVE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227708#comment-14227708
]
Rui Li commented on HIVE-8984:
--
[~xuefuz] - sorry for the lack of descriptions. The patch
[
https://issues.apache.org/jira/browse/HIVE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14228039#comment-14228039
]
Rui Li commented on HIVE-8984:
--
Thanks [~xuefuz] for the review!
Enable job
Rui Li created HIVE-8991:
Summary: Fix custom_input_output_format
Key: HIVE-8991
URL: https://issues.apache.org/jira/browse/HIVE-8991
Project: Hive
Issue Type: Bug
Reporter: Rui Li
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8991:
-
Issue Type: Sub-task (was: Bug)
Parent: HIVE-8548
Fix custom_input_output_format
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8991:
-
Summary: Fix custom_input_output_format [Spark Branch] (was: Fix
custom_input_output_format)
Fix
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8991:
-
Component/s: Spark
Fix custom_input_output_format [Spark Branch]
-
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li reassigned HIVE-8991:
Assignee: Rui Li
Fix custom_input_output_format [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8991:
-
Status: Patch Available (was: Open)
Fix custom_input_output_format [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8991:
-
Attachment: HIVE-8991.1-spark.patch
This patch can fix the test on my machine.
Strange thing is that, if I add
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14228335#comment-14228335
]
Rui Li commented on HIVE-8991:
--
[~vanzin] could you help look at this? Thanks!
Fix
[
https://issues.apache.org/jira/browse/HIVE-8972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li reassigned HIVE-8972:
Assignee: Rui Li
Implement more fine-grained remote client-level events [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14230811#comment-14230811
]
Rui Li commented on HIVE-8991:
--
Hi [~vanzin], just as [~xuefuz] said, this JIRA is only meant
[
https://issues.apache.org/jira/browse/HIVE-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14230812#comment-14230812
]
Rui Li commented on HIVE-8995:
--
OK I'll have a look.
Find thread leak in RSC Tests
[
https://issues.apache.org/jira/browse/HIVE-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14230872#comment-14230872
]
Rui Li commented on HIVE-8995:
--
Hi [~brocknoland], I checked the code. It seems we initialize
[
https://issues.apache.org/jira/browse/HIVE-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8995:
-
Attachment: HIVE-8995.1-spark.patch
I tried the tests [~brocknoland] mentioned with this patch. All the tests
[
https://issues.apache.org/jira/browse/HIVE-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8995:
-
Status: Patch Available (was: Open)
Find thread leak in RSC Tests
-
[
https://issues.apache.org/jira/browse/HIVE-8972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8972:
-
Attachment: HIVE-8972.1-spark.patch
Submit a patch to let the test run
Implement more fine-grained remote
[
https://issues.apache.org/jira/browse/HIVE-8972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8972:
-
Status: Patch Available (was: Open)
Implement more fine-grained remote client-level events [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8995:
-
Attachment: without-patch.PNG
Find thread leak in RSC Tests
-
Key:
[
https://issues.apache.org/jira/browse/HIVE-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8995:
-
Attachment: with-patch.PNG
Find thread leak in RSC Tests
-
Key:
[
https://issues.apache.org/jira/browse/HIVE-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14231528#comment-14231528
]
Rui Li commented on HIVE-8995:
--
Further verified by monitoring the surefirebooter JVM. Seems
[
https://issues.apache.org/jira/browse/HIVE-8972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8972:
-
Attachment: HIVE-8972.1-spark.patch
Reload the same patch to run test again.
Implement more fine-grained remote
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8536:
-
Attachment: HIVE-8536.3-spark.patch
Now {{skewjoin_noskew.q}} should be able to run, with diff in query plan.
[
https://issues.apache.org/jira/browse/HIVE-8972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8972:
-
Attachment: HIVE-8972.2-spark.patch
Fix some bugs and try again.
Implement more fine-grained remote client-level
[
https://issues.apache.org/jira/browse/HIVE-8972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8972:
-
Attachment: HIVE-8972.3-spark.patch
Try again as I can't reproduce the failures on my side.
Implement more
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14233870#comment-14233870
]
Rui Li commented on HIVE-8536:
--
The failures and diff in query plan is expected.
We can
[
https://issues.apache.org/jira/browse/HIVE-8972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8972:
-
Attachment: (was: HIVE-8972.1-spark.patch)
Implement more fine-grained remote client-level events [Spark
Rui Li created HIVE-9017:
Summary: Clean up temp files after unit tests [Spark Branch]
Key: HIVE-9017
URL: https://issues.apache.org/jira/browse/HIVE-9017
Project: Hive
Issue Type: Test
[
https://issues.apache.org/jira/browse/HIVE-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9017:
-
Issue Type: Sub-task (was: Test)
Parent: HIVE-7292
Clean up temp files after unit tests [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14233993#comment-14233993
]
Rui Li commented on HIVE-8991:
--
I looked a little more into this. It seems hive-exec is
Rui Li created HIVE-9019:
Summary: Avoid using SPARK_JAVA_OPTS [Spark Branch]
Key: HIVE-9019
URL: https://issues.apache.org/jira/browse/HIVE-9019
Project: Hive
Issue Type: Test
Components:
[
https://issues.apache.org/jira/browse/HIVE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9019:
-
Issue Type: Sub-task (was: Test)
Parent: HIVE-7292
Avoid using SPARK_JAVA_OPTS [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14234082#comment-14234082
]
Rui Li commented on HIVE-8991:
--
Not sure if it's because how we add hive-exec:
If added
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14237359#comment-14237359
]
Rui Li commented on HIVE-8536:
--
Hi [~xuefuz], my uncertainty is mainly about the # map join
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8536:
-
Attachment: HIVE-8536.4-spark.patch
Disable for now.
Enable SkewJoinResolver for spark [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li reassigned HIVE-9019:
Assignee: Rui Li
Avoid using SPARK_JAVA_OPTS [Spark Branch]
--
[
https://issues.apache.org/jira/browse/HIVE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9019:
-
Description:
SPARK_JAVA_OPTS has been deprecated, see {{SparkConf.validateSettings}}.
Using it together with
[
https://issues.apache.org/jira/browse/HIVE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9019:
-
Attachment: HIVE-9019.1-spark.patch
Avoid using SPARK_JAVA_OPTS [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9019:
-
Attachment: (was: HIVE-9019.1-spark.patch)
Avoid using SPARK_JAVA_OPTS [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9019:
-
Attachment: HIVE-9019.1-spark.patch
Avoid using SPARK_JAVA_OPTS [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9019:
-
Status: Patch Available (was: Open)
Avoid using SPARK_JAVA_OPTS [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14237833#comment-14237833
]
Rui Li commented on HIVE-9019:
--
Hi [~brocknoland], could you help to verify if this patch
[
https://issues.apache.org/jira/browse/HIVE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238795#comment-14238795
]
Rui Li commented on HIVE-9019:
--
Thanks [~brocknoland] for the review.
Avoid using
[
https://issues.apache.org/jira/browse/HIVE-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9017:
-
Description:
Currently RSC will leave a lot of temp files in {{/tmp}}, including {{*_lock}},
{{*_cache}},
[
https://issues.apache.org/jira/browse/HIVE-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9017:
-
Summary: Clean up temp files of RSC [Spark Branch] (was: Clean up temp
files after unit tests [Spark Branch])
[
https://issues.apache.org/jira/browse/HIVE-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9063:
-
Issue Type: Sub-task (was: Bug)
Parent: HIVE-7292
NPE in RemoteSparkJobStatus.getSparkStatistics [Spark
Rui Li created HIVE-9063:
Summary: NPE in RemoteSparkJobStatus.getSparkStatistics [Spark
Branch]
Key: HIVE-9063
URL: https://issues.apache.org/jira/browse/HIVE-9063
Project: Hive
Issue Type: Bug
[
https://issues.apache.org/jira/browse/HIVE-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9063:
-
Status: Patch Available (was: Open)
NPE in RemoteSparkJobStatus.getSparkStatistics [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9063:
-
Attachment: HIVE-9063.1-spark.patch
cc [~brocknoland]
NPE in RemoteSparkJobStatus.getSparkStatistics [Spark
[
https://issues.apache.org/jira/browse/HIVE-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8913:
-
Attachment: HIVE-8913.1-spark.patch
The patch only supports runtime skew join for simple queries where join is the
[
https://issues.apache.org/jira/browse/HIVE-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8913:
-
Status: Patch Available (was: Open)
Make SparkMapJoinResolver handle runtime skew join [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8913:
-
Attachment: HIVE-8913.2-spark.patch
Fix concurrent modification exception.
Update to {{bucket_map_join_tez1.q}}
[
https://issues.apache.org/jira/browse/HIVE-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14241026#comment-14241026
]
Rui Li commented on HIVE-8913:
--
Since we only enable runtime skew join for simple cases where
[
https://issues.apache.org/jira/browse/HIVE-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14241984#comment-14241984
]
Rui Li commented on HIVE-9063:
--
[~xuefuz] - that seems to be an auto optimize of IntelliJ.
[
https://issues.apache.org/jira/browse/HIVE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14241993#comment-14241993
]
Rui Li commented on HIVE-9019:
--
OK... let me know if you find other problems with this :-)
[
https://issues.apache.org/jira/browse/HIVE-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242020#comment-14242020
]
Rui Li commented on HIVE-8913:
--
I think the failure is not related because it passes on my
[
https://issues.apache.org/jira/browse/HIVE-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242035#comment-14242035
]
Rui Li commented on HIVE-8913:
--
Just quick thought, maybe {{IOContext.inputNameIOContextMap}}
[
https://issues.apache.org/jira/browse/HIVE-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242087#comment-14242087
]
Rui Li commented on HIVE-8913:
--
Another possible issue is that we have to make sure the
[
https://issues.apache.org/jira/browse/HIVE-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242145#comment-14242145
]
Rui Li commented on HIVE-8913:
--
I think the call path should be {{ExecMapperContext.clear -
[
https://issues.apache.org/jira/browse/HIVE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243589#comment-14243589
]
Rui Li commented on HIVE-7816:
--
OK I'm on it.
Enable map-join tests which Tez executes
[
https://issues.apache.org/jira/browse/HIVE-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8913:
-
Attachment: HIVE-8913.3-spark.patch
Address RB comments
Make SparkMapJoinResolver handle runtime skew join
[
https://issues.apache.org/jira/browse/HIVE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243892#comment-14243892
]
Rui Li commented on HIVE-7816:
--
We already have golden files for {{filter_join_breaktask.q}}
[
https://issues.apache.org/jira/browse/HIVE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-7816:
-
Attachment: HIVE-7816.1-spark.patch
This patch implements a spark-specific cross product checker
[
https://issues.apache.org/jira/browse/HIVE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-7816:
-
Status: Patch Available (was: Open)
Enable map-join tests which Tez executes [Spark Branch]
Rui Li created HIVE-9097:
Summary: Support runtime skew join for more queries [Spark Branch]
Key: HIVE-9097
URL: https://issues.apache.org/jira/browse/HIVE-9097
Project: Hive
Issue Type: Bug
[
https://issues.apache.org/jira/browse/HIVE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246268#comment-14246268
]
Rui Li commented on HIVE-7816:
--
[~xuefuz] - shall we wait a little bit? That's just a couple
[
https://issues.apache.org/jira/browse/HIVE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246273#comment-14246273
]
Rui Li commented on HIVE-7816:
--
That's OK. No worries, I'll take care of that.
Enable
301 - 400 of 670 matches
Mail list logo
