[jira] [Created] (HIVE-5831) filter input files for bucketed tables

Rui Li created HIVE-5831: Summary: filter input files for bucketed tables Key: HIVE-5831 URL: https://issues.apache.org/jira/browse/HIVE-5831 Project: Hive Issue Type: Improvement

[jira] [Updated] (HIVE-5831) filter input files for bucketed tables

[ https://issues.apache.org/jira/browse/HIVE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-5831: - Description: When the users query a bucketed table and use the bucketed column in the predicate, only the

[jira] [Updated] (HIVE-5831) filter input files for bucketed tables

[ https://issues.apache.org/jira/browse/HIVE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-5831: - Attachment: hive-5831.patch This implementation supports filtering bucketed table with single bucket key. The

[jira] [Created] (HIVE-5871) Use multiple-characters as field delimiter

Rui Li created HIVE-5871: Summary: Use multiple-characters as field delimiter Key: HIVE-5871 URL: https://issues.apache.org/jira/browse/HIVE-5871 Project: Hive Issue Type: Improvement

[jira] [Updated] (HIVE-5871) Use multiple-characters as field delimiter

[ https://issues.apache.org/jira/browse/HIVE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-5871: - Attachment: HIVE-5871.patch This implementation mainly relies on LazySimpleSerDe for serialization and

[jira] [Updated] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7527: - Attachment: HIVE-7527-spark.patch Support order by and sort by on Spark -

[jira] [Updated] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7527: - Attachment: HIVE-7527-spark.patch Support order by and sort by on Spark -

[jira] [Updated] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7527: - Attachment: (was: HIVE-7527-spark.patch) Support order by and sort by on Spark

[jira] [Updated] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7527: - Status: Patch Available (was: Open) Support order by and sort by on Spark

[jira] [Commented] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085642#comment-14085642 ] Rui Li commented on HIVE-7527: -- Hi [~brocknoland], how to specify reviewer/group to publish a

[jira] [Commented] (HIVE-7540) NotSerializableException encountered when using sortByKey transformation

[ https://issues.apache.org/jira/browse/HIVE-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085662#comment-14085662 ] Rui Li commented on HIVE-7540: -- Hi [~brocknoland], we're using BytesWritable because the key

[jira] [Commented] (HIVE-7540) NotSerializableException encountered when using sortByKey transformation

[ https://issues.apache.org/jira/browse/HIVE-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085670#comment-14085670 ] Rui Li commented on HIVE-7540: -- OK got it. Let me try it out. NotSerializableException

[jira] [Commented] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085675#comment-14085675 ] Rui Li commented on HIVE-7527: -- Thanks [~brocknoland], I've created the request at:

[jira] [Commented] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085680#comment-14085680 ] Rui Li commented on HIVE-7527: -- I also noticed that the numPartitions stuff has some conflicts

[jira] [Updated] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7527: - Attachment: HIVE-7527.2-spark.patch Support order by and sort by on Spark -

[jira] [Commented] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085817#comment-14085817 ] Rui Li commented on HIVE-7527: -- Hi [~brocknoland] I've updated the patch. Please help to

[jira] [Commented] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085836#comment-14085836 ] Rui Li commented on HIVE-7527: -- Thanks [~brocknoland] for the review :-) Support order by

[jira] [Commented] (HIVE-7540) NotSerializableException encountered when using sortByKey transformation

[ https://issues.apache.org/jira/browse/HIVE-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086012#comment-14086012 ] Rui Li commented on HIVE-7540: -- Hi [~sandyr], I set spark.serializer to KryoSerializer when I

[jira] [Commented] (HIVE-7540) NotSerializableException encountered when using sortByKey transformation

[ https://issues.apache.org/jira/browse/HIVE-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086059#comment-14086059 ] Rui Li commented on HIVE-7540: -- Oh sure. It's great if that's already fixed.

[jira] [Commented] (HIVE-7540) NotSerializableException encountered when using sortByKey transformation

[ https://issues.apache.org/jira/browse/HIVE-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086121#comment-14086121 ] Rui Li commented on HIVE-7540: -- I've tested with spark 1.1 branch, and verified the

[jira] [Created] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

Rui Li created HIVE-7624: Summary: Reduce operator initialization failed when running multiple MR query on spark Key: HIVE-7624 URL: https://issues.apache.org/jira/browse/HIVE-7624 Project: Hive

[jira] [Assigned] (HIVE-7540) NotSerializableException encountered when using sortByKey transformation

[ https://issues.apache.org/jira/browse/HIVE-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li reassigned HIVE-7540: Assignee: Rui Li NotSerializableException encountered when using sortByKey transformation

[jira] [Updated] (HIVE-7540) NotSerializableException encountered when using sortByKey transformation

[ https://issues.apache.org/jira/browse/HIVE-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7540: - Attachment: HIVE-7540-spark.patch Use spark-1.1.0-SNAPSHOT to solve this issue. SortByShuffler is changed

[jira] [Commented] (HIVE-7540) NotSerializableException encountered when using sortByKey transformation

[ https://issues.apache.org/jira/browse/HIVE-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087169#comment-14087169 ] Rui Li commented on HIVE-7540: -- Hi [~brocknoland], I've uploaded a patch. But I'm afraid it

[jira] [Updated] (HIVE-7540) NotSerializableException encountered when using sortByKey transformation

[ https://issues.apache.org/jira/browse/HIVE-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7540: - Attachment: HIVE-7540.2-spark.patch Hi [~brocknoland] I updated the patch to remove some unused code. Sorry about

[jira] [Updated] (HIVE-5871) Use multiple-characters as field delimiter

[ https://issues.apache.org/jira/browse/HIVE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-5871: - Status: Open (was: Patch Available) Use multiple-characters as field delimiter

[jira] [Updated] (HIVE-5871) Use multiple-characters as field delimiter

[ https://issues.apache.org/jira/browse/HIVE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-5871: - Attachment: HIVE-5871.2.patch Use multiple-characters as field delimiter

[jira] [Updated] (HIVE-5871) Use multiple-characters as field delimiter

[ https://issues.apache.org/jira/browse/HIVE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-5871: - Status: Patch Available (was: Open) Use multiple-characters as field delimiter

[jira] [Commented] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087272#comment-14087272 ] Rui Li commented on HIVE-7624: -- Thanks [~brocknoland] let me try this. Reduce operator

[jira] [Commented] (HIVE-5871) Use multiple-characters as field delimiter

[ https://issues.apache.org/jira/browse/HIVE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087293#comment-14087293 ] Rui Li commented on HIVE-5871: -- Hi [~brocknoland], when users initially required this feature,

[jira] [Updated] (HIVE-5871) Use multiple-characters as field delimiter

[ https://issues.apache.org/jira/browse/HIVE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-5871: - Attachment: HIVE-5871.3.patch Update the patch for latest code base Use multiple-characters as field delimiter

[jira] [Commented] (HIVE-5871) Use multiple-characters as field delimiter

[ https://issues.apache.org/jira/browse/HIVE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087356#comment-14087356 ] Rui Li commented on HIVE-5871: -- [~brocknoland] I updated the patch because the old one won't

[jira] [Updated] (HIVE-5871) Use multiple-characters as field delimiter

[ https://issues.apache.org/jira/browse/HIVE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-5871: - Attachment: HIVE-5871.4.patch Use multiple-characters as field delimiter

[jira] [Assigned] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li reassigned HIVE-7624: Assignee: Rui Li Reduce operator initialization failed when running multiple MR query on spark

[jira] [Commented] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14089001#comment-14089001 ] Rui Li commented on HIVE-7624: -- Thanks very much [~csun]. After some debugging, I found this

[jira] [Updated] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7624: - Attachment: HIVE-7624.patch Reduce operator initialization failed when running multiple MR query on spark

[jira] [Commented] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14089181#comment-14089181 ] Rui Li commented on HIVE-7624: -- This patch solves the reducesinkkey0 problem. Map work and

[jira] [Commented] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090152#comment-14090152 ] Rui Li commented on HIVE-7624: -- [~xuefuz] currently the second R does end with a ReduceSink.

[jira] [Updated] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7624: - Attachment: HIVE-7624.2-spark.patch Reduce operator initialization failed when running multiple MR query on

[jira] [Commented] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090261#comment-14090261 ] Rui Li commented on HIVE-7624: -- Hi [~csun] I updated the patch based on latest code. But the

[jira] [Updated] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7624: - Attachment: HIVE-7624.3-spark.patch Reduce operator initialization failed when running multiple MR query on

[jira] [Updated] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7624: - Status: Patch Available (was: Open) Reduce operator initialization failed when running multiple MR query on

[jira] [Commented] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090452#comment-14090452 ] Rui Li commented on HIVE-7624: -- Finally I found this is because we don't set output collector

[jira] [Updated] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7624: - Attachment: HIVE-7624.4-spark.patch Reduce operator initialization failed when running multiple MR query on

[jira] [Commented] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090509#comment-14090509 ] Rui Li commented on HIVE-7624: -- Some change may bypass HIVE-7597. Remove it. Reduce operator

[jira] [Created] (HIVE-7659) Unnecessary sort in query plan

Rui Li created HIVE-7659: Summary: Unnecessary sort in query plan Key: HIVE-7659 URL: https://issues.apache.org/jira/browse/HIVE-7659 Project: Hive Issue Type: Improvement Components:

[jira] [Updated] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7624: - Attachment: HIVE-7624.5-spark.patch Reduce operator initialization failed when running multiple MR query on

[jira] [Commented] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093618#comment-14093618 ] Rui Li commented on HIVE-7624: -- Hi [~brocknoland], [~szehon] I'll rebase the patch. Reduce

[jira] [Updated] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7624: - Attachment: HIVE-7624.6-spark.patch I rebased with latest code. Reduce operator initialization failed when

[jira] [Commented] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093796#comment-14093796 ] Rui Li commented on HIVE-7624: -- Thanks for the review : -) Reduce operator initialization

[jira] [Commented] (HIVE-7333) Create RDD translator, translating Hive Tables into Spark RDDs

[ https://issues.apache.org/jira/browse/HIVE-7333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095022#comment-14095022 ] Rui Li commented on HIVE-7333: -- Sorry I forgot to put these here: I tested the following

[jira] [Commented] (HIVE-7333) Create RDD translator, translating Hive Tables into Spark RDDs

[ https://issues.apache.org/jira/browse/HIVE-7333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095064#comment-14095064 ] Rui Li commented on HIVE-7333: -- Oh sorry if I was being confusing. This is to test if the

[jira] [Resolved] (HIVE-7333) Create RDD translator, translating Hive Tables into Spark RDDs

[ https://issues.apache.org/jira/browse/HIVE-7333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li resolved HIVE-7333. -- Resolution: Done Close this as most tables can be represented as spark RDD intrinsically Create RDD

[jira] [Updated] (HIVE-5871) Use multiple-characters as field delimiter

[ https://issues.apache.org/jira/browse/HIVE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-5871: - Attachment: HIVE-5871.5.patch Use multiple-characters as field delimiter

[jira] [Commented] (HIVE-5871) Use multiple-characters as field delimiter

[ https://issues.apache.org/jira/browse/HIVE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095096#comment-14095096 ] Rui Li commented on HIVE-5871: -- Hi [~brocknoland] I updated the patch accordingly. Use

[jira] [Assigned] (HIVE-7659) Unnecessary sort in query plan

[ https://issues.apache.org/jira/browse/HIVE-7659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li reassigned HIVE-7659: Assignee: Rui Li Unnecessary sort in query plan -- Key:

[jira] [Updated] (HIVE-7659) Unnecessary sort in query plan

[ https://issues.apache.org/jira/browse/HIVE-7659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7659: - Attachment: HIVE-7659-spark.patch Unnecessary sort in query plan --

[jira] [Commented] (HIVE-7659) Unnecessary sort in query plan

[ https://issues.apache.org/jira/browse/HIVE-7659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096774#comment-14096774 ] Rui Li commented on HIVE-7659: -- After some research, I found the unnecessary sort is mainly

[jira] [Updated] (HIVE-7659) Unnecessary sort in query plan

[ https://issues.apache.org/jira/browse/HIVE-7659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7659: - Status: Patch Available (was: Open) Unnecessary sort in query plan --

[jira] [Created] (HIVE-7731) Incorrect result returned when a map work has multiple downstream reduce works

Rui Li created HIVE-7731: Summary: Incorrect result returned when a map work has multiple downstream reduce works Key: HIVE-7731 URL: https://issues.apache.org/jira/browse/HIVE-7731 Project: Hive

[jira] [Commented] (HIVE-7731) Incorrect result returned when a map work has multiple downstream reduce works

[ https://issues.apache.org/jira/browse/HIVE-7731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096899#comment-14096899 ] Rui Li commented on HIVE-7731: -- Some quick thoughts: I suspect we hit the output collector

[jira] [Updated] (HIVE-7659) Unnecessary sort in query plan [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7659: - Summary: Unnecessary sort in query plan [Spark Branch] (was: Spark: Unnecessary sort in query plan)

[jira] [Commented] (HIVE-7731) Incorrect result returned when a map work has multiple downstream reduce works [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098004#comment-14098004 ] Rui Li commented on HIVE-7731: -- Thanks [~thejas] and [~brocknoland] for correcting the title.

[jira] [Commented] (HIVE-5871) Use multiple-characters as field delimiter

[ https://issues.apache.org/jira/browse/HIVE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098025#comment-14098025 ] Rui Li commented on HIVE-5871: -- Hi [~brocknoland], I made the change because MultiDelimitSerde

[jira] [Updated] (HIVE-7659) Unnecessary sort in query plan [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7659: - Attachment: HIVE-7659.2-spark.patch Update patch according to review Unnecessary sort in query plan [Spark

[jira] [Commented] (HIVE-7528) Support cluster by and distributed by

[ https://issues.apache.org/jira/browse/HIVE-7528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098360#comment-14098360 ] Rui Li commented on HIVE-7528: -- I've tried simple distribute/cluster by queries and they can

[jira] [Reopened] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li reopened HIVE-7624: -- Reopen this as we've refactored to use SparkRecordHandler instead of ExecMapper and ExecReducer Reduce operator

[jira] [Updated] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7624: - Status: Patch Available (was: Reopened) Reduce operator initialization failed when running multiple MR query on

[jira] [Updated] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7624: - Attachment: HIVE-7624.7-spark.patch Reduce operator initialization failed when running multiple MR query on

[jira] [Updated] (HIVE-7528) Support cluster by and distributed by

[ https://issues.apache.org/jira/browse/HIVE-7528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7528: - Attachment: HIVE-7528.spark.patch Support cluster by and distributed by -

[jira] [Commented] (HIVE-7528) Support cluster by and distributed by

[ https://issues.apache.org/jira/browse/HIVE-7528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14100467#comment-14100467 ] Rui Li commented on HIVE-7528: -- Distribute/cluster by should work with the sort shuffler in

[jira] [Updated] (HIVE-7528) Support cluster by and distributed by

[ https://issues.apache.org/jira/browse/HIVE-7528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7528: - Status: Patch Available (was: Open) Support cluster by and distributed by

[jira] [Commented] (HIVE-7624) Reduce operator initialization failed when running multiple MR query on spark

[ https://issues.apache.org/jira/browse/HIVE-7624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14101680#comment-14101680 ] Rui Li commented on HIVE-7624: -- [~brocknoland] Got it, thanks! Reduce operator

[jira] [Created] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]

Rui Li created HIVE-7772: Summary: Add tests for order/sort/distribute/cluster by query [Spark Branch] Key: HIVE-7772 URL: https://issues.apache.org/jira/browse/HIVE-7772 Project: Hive Issue Type:

[jira] [Commented] (HIVE-7528) Support cluster by and distributed by [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14101687#comment-14101687 ] Rui Li commented on HIVE-7528: -- Thanks [~brocknoland] I've created HIVE-7772 for it. Support

[jira] [Created] (HIVE-7773) Union all query finished with errors [Spark Branch]

Rui Li created HIVE-7773: Summary: Union all query finished with errors [Spark Branch] Key: HIVE-7773 URL: https://issues.apache.org/jira/browse/HIVE-7773 Project: Hive Issue Type: Bug

[jira] [Updated] (HIVE-7773) Union all query finished with errors [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7773: - Attachment: HIVE-7773.spark.patch I found the problem is that IOContext is used to store and retrieve input path

[jira] [Commented] (HIVE-7773) Union all query finished with errors [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103208#comment-14103208 ] Rui Li commented on HIVE-7773: -- Thank you [~brocknoland] Union all query finished with

[jira] [Updated] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7772: - Attachment: HIVE-7772-spark.patch Add tests for order/sort/distribute/cluster by query [Spark Branch]

[jira] [Updated] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7772: - Status: Patch Available (was: Open) Add tests for order/sort/distribute/cluster by query [Spark Branch]

[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14105327#comment-14105327 ] Rui Li commented on HIVE-7772: -- This patch adds some simple cases. Other cases require join or

[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14106297#comment-14106297 ] Rui Li commented on HIVE-7772: -- Thanks [~brocknoland], let me rebase my branch and see if I

[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14106731#comment-14106731 ] Rui Li commented on HIVE-7772: -- Hi [~brocknoland], I tested other cases with latest code again

[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14108704#comment-14108704 ] Rui Li commented on HIVE-7772: -- Add more tests when counter is ready. Add tests for

[jira] [Created] (HIVE-7893) Find a way to get a job identifier when submitting a spark job [Spark Branch]

Rui Li created HIVE-7893: Summary: Find a way to get a job identifier when submitting a spark job [Spark Branch] Key: HIVE-7893 URL: https://issues.apache.org/jira/browse/HIVE-7893 Project: Hive

[jira] [Commented] (HIVE-7916) Snappy-java error when running hive query on spark [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14117050#comment-14117050 ] Rui Li commented on HIVE-7916: -- Hi [~xuefuz], I tried on my cluster but cannot reproduce the

[jira] [Commented] (HIVE-7916) Snappy-java error when running hive query on spark [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14117057#comment-14117057 ] Rui Li commented on HIVE-7916: -- I noted this may be related to SPARK-2881. Snappy-java is

[jira] [Commented] (HIVE-7916) Snappy-java error when running hive query on spark [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14117911#comment-14117911 ] Rui Li commented on HIVE-7916: -- Hi [~xuefuz], do you use the latest code of spark 1.1 branch?

[jira] [Created] (HIVE-7956) When inserting into a bucketed table, all data goes to a single bucket [Spark Branch]

Rui Li created HIVE-7956: Summary: When inserting into a bucketed table, all data goes to a single bucket [Spark Branch] Key: HIVE-7956 URL: https://issues.apache.org/jira/browse/HIVE-7956 Project: Hive

[jira] [Commented] (HIVE-7956) When inserting into a bucketed table, all data goes to a single bucket [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14120807#comment-14120807 ] Rui Li commented on HIVE-7956: -- Yes [~brocknoland], with {{set hive.enforce.bucketing =

[jira] [Commented] (HIVE-7956) When inserting into a bucketed table, all data goes to a single bucket [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14120926#comment-14120926 ] Rui Li commented on HIVE-7956: -- [~xuefuz], I copied the extra fields

[jira] [Assigned] (HIVE-7956) When inserting into a bucketed table, all data goes to a single bucket [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li reassigned HIVE-7956: Assignee: Rui Li When inserting into a bucketed table, all data goes to a single bucket [Spark Branch]

[jira] [Commented] (HIVE-7956) When inserting into a bucketed table, all data goes to a single bucket [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122735#comment-14122735 ] Rui Li commented on HIVE-7956: -- Hi [~brocknoland] [~xuefuz], The problem is {{RowContainer}}

[jira] [Commented] (HIVE-7956) When inserting into a bucketed table, all data goes to a single bucket [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14124268#comment-14124268 ] Rui Li commented on HIVE-7956: -- Thanks [~brocknoland], [~xuefuz] for the comments. I also

[jira] [Commented] (HIVE-5871) Use multiple-characters as field delimiter

[ https://issues.apache.org/jira/browse/HIVE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14124272#comment-14124272 ] Rui Li commented on HIVE-5871: -- Thanks [~brocknoland] for the patient review! Use

[jira] [Commented] (HIVE-7956) When inserting into a bucketed table, all data goes to a single bucket [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14124295#comment-14124295 ] Rui Li commented on HIVE-7956: -- [~xuefuz], do you mean we have to add

[jira] [Created] (HIVE-8017) Use HiveKey instead of Byteswritable as key type of the pair RDD [Spark Branch]

Rui Li created HIVE-8017: Summary: Use HiveKey instead of Byteswritable as key type of the pair RDD [Spark Branch] Key: HIVE-8017 URL: https://issues.apache.org/jira/browse/HIVE-8017 Project: Hive

[jira] [Updated] (HIVE-8017) Use HiveKey instead of Byteswritable as key type of the pair RDD [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-8017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-8017: - Issue Type: Sub-task (was: Bug) Parent: HIVE-7292 Use HiveKey instead of Byteswritable as key type of

[jira] [Updated] (HIVE-8017) Use HiveKey instead of BytesWritable as key type of the pair RDD [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-8017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-8017: - Summary: Use HiveKey instead of BytesWritable as key type of the pair RDD [Spark Branch] (was: Use HiveKey

[jira] [Commented] (HIVE-7956) When inserting into a bucketed table, all data goes to a single bucket [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-7956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14124742#comment-14124742 ] Rui Li commented on HIVE-7956: -- [~xuefuz] that's great! I created HIVE-8017 and will do the

  1   2   3   4   5   6   7   >