[jira] [Commented] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085817#comment-14085817 ] Rui Li commented on HIVE-7527: -- Hi [~brocknoland] I've updated the patch. Pleas

[jira] [Updated] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7527: - Attachment: HIVE-7527.2-spark.patch > Support order by and sort by on Sp

[jira] [Commented] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085680#comment-14085680 ] Rui Li commented on HIVE-7527: -- I also noticed that the numPartitions stuff has

[jira] [Commented] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085675#comment-14085675 ] Rui Li commented on HIVE-7527: -- Thanks [~brocknoland], I've created the request

[jira] [Commented] (HIVE-7540) NotSerializableException encountered when using sortByKey transformation

[ https://issues.apache.org/jira/browse/HIVE-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085670#comment-14085670 ] Rui Li commented on HIVE-7540: -- OK got it. Let me try it out. > NotSerializableEx

[jira] [Commented] (HIVE-7540) NotSerializableException encountered when using sortByKey transformation

[ https://issues.apache.org/jira/browse/HIVE-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085662#comment-14085662 ] Rui Li commented on HIVE-7540: -- Hi [~brocknoland], we're using BytesWritable becaus

[jira] [Commented] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085642#comment-14085642 ] Rui Li commented on HIVE-7527: -- Hi [~brocknoland], how to specify reviewer/group to publi

[jira] [Updated] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7527: - Status: Patch Available (was: Open) > Support order by and sort by on Sp

[jira] [Updated] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7527: - Attachment: (was: HIVE-7527-spark.patch) > Support order by and sort by on Sp

[jira] [Updated] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7527: - Attachment: HIVE-7527-spark.patch > Support order by and sort by on Sp

[jira] [Updated] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7527: - Attachment: HIVE-7527-spark.patch > Support order by and sort by on Sp

[jira] [Commented] (HIVE-7526) Research to use groupby transformation to replace Hive existing partitionByKey and SparkCollector combination

[ https://issues.apache.org/jira/browse/HIVE-7526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14084450#comment-14084450 ] Rui Li commented on HIVE-7526: -- Hi [~xuefuz] [~csun], it seems in SparkShuffler, we lost

[jira] [Commented] (HIVE-7526) Research to use groupby transformation to replace Hive existing partitionByKey and SparkCollector combination

[ https://issues.apache.org/jira/browse/HIVE-7526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14080728#comment-14080728 ] Rui Li commented on HIVE-7526: -- Hi [~xuefuz] do you mean you committed patch #5? I che

[jira] [Commented] (HIVE-7334) Create SparkShuffler, shuffling data between map-side data processing and reduce-side processing

[ https://issues.apache.org/jira/browse/HIVE-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14078936#comment-14078936 ] Rui Li commented on HIVE-7334: -- Thanks [~xuefuz] this is much clearer. >

[jira] [Commented] (HIVE-7334) Create SparkShuffler, shuffling data between map-side data processing and reduce-side processing

[ https://issues.apache.org/jira/browse/HIVE-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14078764#comment-14078764 ] Rui Li commented on HIVE-7334: -- Just some initial ground work. Submitted for re

[jira] [Updated] (HIVE-7334) Create SparkShuffler, shuffling data between map-side data processing and reduce-side processing

[ https://issues.apache.org/jira/browse/HIVE-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7334: - Attachment: HIVE-7334.patch > Create SparkShuffler, shuffling data between map-side data processing and >

[jira] [Created] (HIVE-7540) NotSerializableException encountered when using sortByKey transformation

Rui Li created HIVE-7540: Summary: NotSerializableException encountered when using sortByKey transformation Key: HIVE-7540 URL: https://issues.apache.org/jira/browse/HIVE-7540 Project: Hive Issue

[jira] [Commented] (HIVE-7527) Support order by and sort by on Spark

[ https://issues.apache.org/jira/browse/HIVE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14076187#comment-14076187 ] Rui Li commented on HIVE-7527: -- Hi [~xuefuz], I tried to run order by queries using spa

[jira] [Commented] (HIVE-7467) When querying HBase table, task fails with exception: java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString

[ https://issues.apache.org/jira/browse/HIVE-7467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069696#comment-14069696 ] Rui Li commented on HIVE-7467: -- The workaround mentioned in HBASE-10877 can solve the pro

[jira] [Created] (HIVE-7467) When querying HBase table, task fails with exception: java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString

Rui Li created HIVE-7467: Summary: When querying HBase table, task fails with exception: java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Key: HIVE-7467 URL: https://issues.apache.org/jira/browse

[jira] [Commented] (HIVE-7431) When run on spark cluster, some spark tasks may fail

[ https://issues.apache.org/jira/browse/HIVE-7431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068216#comment-14068216 ] Rui Li commented on HIVE-7431: -- Hi [~xuefuz], I've updated the patch. > When

[jira] [Updated] (HIVE-7431) When run on spark cluster, some spark tasks may fail

[ https://issues.apache.org/jira/browse/HIVE-7431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7431: - Attachment: HIVE-7431.2.patch > When run on spark cluster, some spark tasks may f

[jira] [Commented] (HIVE-7431) When run on spark cluster, some spark tasks may fail

[ https://issues.apache.org/jira/browse/HIVE-7431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068118#comment-14068118 ] Rui Li commented on HIVE-7431: -- [~xuefuz] I see. In HiveMapFunction.call, MapWor

[jira] [Commented] (HIVE-7431) When run on spark cluster, some spark tasks may fail

[ https://issues.apache.org/jira/browse/HIVE-7431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068100#comment-14068100 ] Rui Li commented on HIVE-7431: -- [~xuefuz], thanks for the review. I'll clean up the

[jira] [Updated] (HIVE-7431) When run on spark cluster, some spark tasks may fail

[ https://issues.apache.org/jira/browse/HIVE-7431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7431: - Assignee: Rui Li Status: Patch Available (was: Open) > When run on spark cluster, some spark tasks

[jira] [Commented] (HIVE-7431) When run on spark cluster, some spark tasks may fail

[ https://issues.apache.org/jira/browse/HIVE-7431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14066188#comment-14066188 ] Rui Li commented on HIVE-7431: -- Since each spark task is a thread running in the same JVM

[jira] [Updated] (HIVE-7431) When run on spark cluster, some spark tasks may fail

[ https://issues.apache.org/jira/browse/HIVE-7431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-7431: - Attachment: HIVE-7431.1.patch > When run on spark cluster, some spark tasks may f

[jira] [Commented] (HIVE-7431) When run on spark cluster, some spark tasks may fail

[ https://issues.apache.org/jira/browse/HIVE-7431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14066120#comment-14066120 ] Rui Li commented on HIVE-7431: -- I also noted that when running on Tez, MapWork is cache

[jira] [Commented] (HIVE-7431) When run on spark cluster, some spark tasks may fail

[ https://issues.apache.org/jira/browse/HIVE-7431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14066114#comment-14066114 ] Rui Li commented on HIVE-7431: -- [~xuefuz] This failure happens when I run select c

[jira] [Commented] (HIVE-7431) When run on spark cluster, some spark tasks may fail

[ https://issues.apache.org/jira/browse/HIVE-7431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065958#comment-14065958 ] Rui Li commented on HIVE-7431: -- It seems that some malformed op tree caused this i

[jira] [Commented] (HIVE-7431) When run on spark cluster, some spark tasks may fail

[ https://issues.apache.org/jira/browse/HIVE-7431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065872#comment-14065872 ] Rui Li commented on HIVE-7431: -- The original exception is: {q

[jira] [Created] (HIVE-7431) When run on spark cluster, some spark tasks may fail

Rui Li created HIVE-7431: Summary: When run on spark cluster, some spark tasks may fail Key: HIVE-7431 URL: https://issues.apache.org/jira/browse/HIVE-7431 Project: Hive Issue Type: Bug

[jira] [Commented] (HIVE-7387) Guava version conflict between hadoop and spark [Spark-Branch]

[ https://issues.apache.org/jira/browse/HIVE-7387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058438#comment-14058438 ] Rui Li commented on HIVE-7387: -- Seems that hive (spark branch) also depends on guava-14.

[jira] [Updated] (HIVE-5871) Use multiple-characters as field delimiter

[ https://issues.apache.org/jira/browse/HIVE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-5871: - Description: By default, hive only allows user to use single character as field delimiter. Although there&#

[jira] [Updated] (HIVE-5831) filter input files for bucketed tables

[ https://issues.apache.org/jira/browse/HIVE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-5831: - Status: Patch Available (was: Open) > filter input files for bucketed tab

[jira] [Updated] (HIVE-5871) Use multiple-characters as field delimiter

[ https://issues.apache.org/jira/browse/HIVE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-5871: - Status: Patch Available (was: Open) > Use multiple-characters as field delimi

[jira] [Updated] (HIVE-5871) Use multiple-characters as field delimiter

[ https://issues.apache.org/jira/browse/HIVE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-5871: - Attachment: HIVE-5871-v2.patch Fix previous implementation. > Use multiple-characters as field delimi

[jira] [Updated] (HIVE-5871) Use multiple-characters as field delimiter

[ https://issues.apache.org/jira/browse/HIVE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-5871: - Attachment: HIVE-5871.patch This implementation mainly relies on LazySimpleSerDe for serialization and

[jira] [Created] (HIVE-5871) Use multiple-characters as field delimiter

Rui Li created HIVE-5871: Summary: Use multiple-characters as field delimiter Key: HIVE-5871 URL: https://issues.apache.org/jira/browse/HIVE-5871 Project: Hive Issue Type: Improvement

[jira] [Updated] (HIVE-5831) filter input files for bucketed tables

[ https://issues.apache.org/jira/browse/HIVE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-5831: - Attachment: hive-5831.patch This implementation supports filtering bucketed table with single bucket key. The

[jira] [Updated] (HIVE-5831) filter input files for bucketed tables

[ https://issues.apache.org/jira/browse/HIVE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-5831: - Description: When the users query a bucketed table and use the bucketed column in the predicate, only the

[jira] [Created] (HIVE-5831) filter input files for bucketed tables

Rui Li created HIVE-5831: Summary: filter input files for bucketed tables Key: HIVE-5831 URL: https://issues.apache.org/jira/browse/HIVE-5831 Project: Hive Issue Type: Improvement

<    3   4   5   6   7   8