[jira] [Commented] (HIVE-14240) HoS itests shouldn't depend on a Spark distribution

[ https://issues.apache.org/jira/browse/HIVE-14240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15508363#comment-15508363 ] liyunzhang_intel commented on HIVE-14240: - [~Ferd]: bq. In Pig, they don't require Spark

[jira] [Commented] (HIVE-15259) The deserialization time of HOS20 is longer than what in HOS16

[ https://issues.apache.org/jira/browse/HIVE-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15688947#comment-15688947 ] liyunzhang_intel commented on HIVE-15259: - [~ruili]: thanks for your reply. In yarn mode, after i

[jira] [Commented] (HIVE-14825) Figure out the minimum set of required jars for Hive on Spark after bumping up to Spark 2.0.0

[ https://issues.apache.org/jira/browse/HIVE-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15692463#comment-15692463 ] liyunzhang_intel commented on HIVE-14825: - [~ruili]: Will spark load all the jars in

[jira] [Commented] (HIVE-14825) Figure out the minimum set of required jars for Hive on Spark after bumping up to Spark 2.0.0

[ https://issues.apache.org/jira/browse/HIVE-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15688540#comment-15688540 ] liyunzhang_intel commented on HIVE-14825: - [~lirui]: i have not found the necessary libs on

[jira] [Updated] (HIVE-15259) The deserialization time of HOS20 is longer than what in HOS16

[ https://issues.apache.org/jira/browse/HIVE-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-15259: Attachment: Deserialization_HOS20.PNG Deserialization_HOS16.PNG [~xuefuz] ,

[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.

[ https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15624357#comment-15624357 ] liyunzhang_intel commented on HIVE-13517: - [~szehon]: currently i view the driver and executor

[jira] [Updated] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.

[ https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-13517: Attachment: executor-driver-log.PNG > Hive logs in Spark Executor and Driver should show

[jira] [Assigned] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.

[ https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel reassigned HIVE-13517: --- Assignee: liyunzhang_intel (was: Szehon Ho) > Hive logs in Spark Executor and

[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.

[ https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15622634#comment-15622634 ] liyunzhang_intel commented on HIVE-13517: - [~szehon]: {quote} It would be great if there could

[jira] [Comment Edited] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.

[ https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15622634#comment-15622634 ] liyunzhang_intel edited comment on HIVE-13517 at 10/31/16 4:38 PM: ---

[jira] [Updated] (HIVE-15313) Add export spark.yarn.archive or spark.yarn.jars variable in Hive on Spark document

[ https://issues.apache.org/jira/browse/HIVE-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-15313: Description: According to

[jira] [Commented] (HIVE-15432) java.lang.ClassCastException is thrown when setting "hive.input.format" as "org.apache.hadoop.hive.ql.io.CombineHiveInputFormat" in hive on spark

[ https://issues.apache.org/jira/browse/HIVE-15432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750760#comment-15750760 ] liyunzhang_intel commented on HIVE-15432: - [~lirui] : do we support

[jira] [Resolved] (HIVE-15432) java.lang.ClassCastException is thrown when setting "hive.input.format" as "org.apache.hadoop.hive.ql.io.CombineHiveInputFormat" in hive on spark

[ https://issues.apache.org/jira/browse/HIVE-15432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel resolved HIVE-15432. - Resolution: Duplicate duplicated with HIVE-8722 > java.lang.ClassCastException is thrown

[jira] [Commented] (HIVE-15302) Relax the requirement that HoS needs Spark built w/o Hive

[ https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15713661#comment-15713661 ] liyunzhang_intel commented on HIVE-15302: - [~lirui]: {quote} We only care about yarn-client and

[jira] [Commented] (HIVE-8373) OOM for a simple query with spark.master=local [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-8373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15714427#comment-15714427 ] liyunzhang_intel commented on HIVE-8373: [~xuefuz] and [~lirui]: I have met same OOM error when

[jira] [Updated] (HIVE-15313) Add export spark.yarn.archive or spark.yarn.jars variable in Hive on Spark document

[ https://issues.apache.org/jira/browse/HIVE-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-15313: Description: According to

[jira] [Updated] (HIVE-15313) Add export spark.yarn.archive or spark.yarn.jars variable in Hive on Spark document

[ https://issues.apache.org/jira/browse/HIVE-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-15313: Description: According to

[jira] [Commented] (HIVE-15259) The deserialization time of HOS20 is longer than what in HOS16

[ https://issues.apache.org/jira/browse/HIVE-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15707422#comment-15707422 ] liyunzhang_intel commented on HIVE-15259: - [~lirui]: Something update for this jira: The

[jira] [Resolved] (HIVE-15259) The deserialization time of HOS20 is longer than what in HOS16

[ https://issues.apache.org/jira/browse/HIVE-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel resolved HIVE-15259. - Resolution: Not A Bug > The deserialization time of HOS20 is longer than what in HOS16 >

[jira] [Updated] (HIVE-15313) Add export spark.yarn.archive or spark.yarn.jars variable in Hive on Spark document

[ https://issues.apache.org/jira/browse/HIVE-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-15313: Attachment: performance.improvement.after.set.spark.yarn.archive.PNG

[jira] [Commented] (HIVE-15302) Relax the requirement that HoS needs Spark built w/o Hive

[ https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15711352#comment-15711352 ] liyunzhang_intel commented on HIVE-15302: - [~lirui]: understand the requirement. My question:

[jira] [Commented] (HIVE-15302) Relax the requirement that HoS needs Spark built w/o Hive

[ https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15710852#comment-15710852 ] liyunzhang_intel commented on HIVE-15302: - [~lirui]: The idea is good. so the flow of your idea

[jira] [Commented] (HIVE-15527) Memory usage is unbound in SortByShuffler for Spark

[ https://issues.apache.org/jira/browse/HIVE-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15793967#comment-15793967 ] liyunzhang_intel commented on HIVE-15527: - [~xuefuz] and [~lirui]: HiveKVResultCache will write

[jira] [Comment Edited] (HIVE-15527) Memory usage is unbound in SortByShuffler for Spark

[ https://issues.apache.org/jira/browse/HIVE-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15793967#comment-15793967 ] liyunzhang_intel edited comment on HIVE-15527 at 1/3/17 2:53 AM: -

[jira] [Commented] (HIVE-9153) Perf enhancement on CombineHiveInputFormat and HiveInputFormat

[ https://issues.apache.org/jira/browse/HIVE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763251#comment-15763251 ] liyunzhang_intel commented on HIVE-9153: [~lirui]: in [hive on spark

[jira] [Commented] (HIVE-8373) OOM for a simple query with spark.master=local [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-8373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782254#comment-15782254 ] liyunzhang_intel commented on HIVE-8373: [~asears] : I have tested the command provided by you and

[jira] [Commented] (HIVE-8373) OOM for a simple query with spark.master=local [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-8373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15769284#comment-15769284 ] liyunzhang_intel commented on HIVE-8373: [~xuefuz] and [~lirui]: i have run spark.master=local

[jira] [Assigned] (HIVE-8373) OOM for a simple query with spark.master=local [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-8373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel reassigned HIVE-8373: -- Assignee: liyunzhang_intel > OOM for a simple query with spark.master=local [Spark

[jira] [Commented] (HIVE-8373) OOM for a simple query with spark.master=local [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-8373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15769353#comment-15769353 ] liyunzhang_intel commented on HIVE-8373: [~lirui]: yes, this problem only happens on jdk7. after

[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.

[ https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15939618#comment-15939618 ] liyunzhang_intel commented on HIVE-13517: - [~stakiar]: LGTM, but what i am confused {quote} In a

[jira] [Commented] (HIVE-14919) Improve the performance of Hive on Spark 2.0.0

[ https://issues.apache.org/jira/browse/HIVE-14919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15934145#comment-15934145 ] liyunzhang_intel commented on HIVE-14919: - [~stakiar]: i think it is a good point to integration

[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.

[ https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933954#comment-15933954 ] liyunzhang_intel commented on HIVE-13517: - [~stakiar]: it is ok to assign it to you. what i am

[jira] [Commented] (HIVE-14919) Improve the performance of Hive on Spark 2.0.0

[ https://issues.apache.org/jira/browse/HIVE-14919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929407#comment-15929407 ] liyunzhang_intel commented on HIVE-14919: - [~lirui]: I guess you mean to set

[jira] [Assigned] (HIVE-11297) Combine op trees for partition info generating tasks [Spark branch]

[ https://issues.apache.org/jira/browse/HIVE-11297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel reassigned HIVE-11297: --- Assignee: liyunzhang_intel > Combine op trees for partition info generating tasks

[jira] [Commented] (HIVE-14919) Improve the performance of Hive on Spark 2.0.0

[ https://issues.apache.org/jira/browse/HIVE-14919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923492#comment-15923492 ] liyunzhang_intel commented on HIVE-14919: - [~lirui]: {quote} One thing I noted is the Xms flag was

[jira] [Commented] (HIVE-16046) Broadcasting small table for Hive on Spark

[ https://issues.apache.org/jira/browse/HIVE-16046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15974264#comment-15974264 ] liyunzhang_intel commented on HIVE-16046: - [~xuefuz]: in

[jira] [Commented] (HIVE-17321) HoS: analyze ORC table doesn't compute raw data size when noscan/partialscan is not specified

[ https://issues.apache.org/jira/browse/HIVE-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128375#comment-16128375 ] liyunzhang_intel commented on HIVE-17321: - [~lirui]: understand, but i am very curious why the raw

[jira] [Comment Edited] (HIVE-17321) HoS: analyze ORC table doesn't compute raw data size when noscan/partialscan is not specified

[ https://issues.apache.org/jira/browse/HIVE-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128187#comment-16128187 ] liyunzhang_intel edited comment on HIVE-17321 at 8/16/17 5:51 AM: --

[jira] [Commented] (HIVE-17321) HoS: analyze ORC table doesn't compute raw data size when noscan/partialscan is not specified

[ https://issues.apache.org/jira/browse/HIVE-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128187#comment-16128187 ] liyunzhang_intel commented on HIVE-17321: - [~lirui]: for orc, we need not compute raw data size by

[jira] [Updated] (HIVE-17287) HoS can not deal with skewed data group by

[ https://issues.apache.org/jira/browse/HIVE-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-17287: Attachment: compare_groupby_groupby_rollup.png > HoS can not deal with skewed data group by

[jira] [Commented] (HIVE-17287) HoS can not deal with skewed data group by

[ https://issues.apache.org/jira/browse/HIVE-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126964#comment-16126964 ] liyunzhang_intel commented on HIVE-17287: - [~lirui] ,[~gopalv]: some update about skewed data

[jira] [Commented] (HIVE-17287) HoS can not deal with skewed data group by

[ https://issues.apache.org/jira/browse/HIVE-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122897#comment-16122897 ] liyunzhang_intel commented on HIVE-17287: - [~gopalv],[~lirui]： the result why the output of join

[jira] [Updated] (HIVE-17287) HoS can not deal with skewed data group by

[ https://issues.apache.org/jira/browse/HIVE-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-17287: Attachment: query67-groupby_shuffle_metric.png > HoS can not deal with skewed data group by

[jira] [Commented] (HIVE-17287) HoS can not deal with skewed data group by

[ https://issues.apache.org/jira/browse/HIVE-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123021#comment-16123021 ] liyunzhang_intel commented on HIVE-17287: - [~lirui]: attached is the

[jira] [Updated] (HIVE-17287) HoS can not deal with skewed data group by

[ https://issues.apache.org/jira/browse/HIVE-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-17287: Attachment: query67-fail-at-groupby.png the attached query67-fail-at-groupby.png shows that

[jira] [Commented] (HIVE-17287) HoS can not deal with skewed data group by

[ https://issues.apache.org/jira/browse/HIVE-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122955#comment-16122955 ] liyunzhang_intel commented on HIVE-17287: - [~lirui]: thanks for comments

[jira] [Comment Edited] (HIVE-17287) HoS can not deal with skewed data group by

[ https://issues.apache.org/jira/browse/HIVE-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121023#comment-16121023 ] liyunzhang_intel edited comment on HIVE-17287 at 8/10/17 4:02 AM: --

[jira] [Commented] (HIVE-17287) HoS can not deal with skewed data group by

[ https://issues.apache.org/jira/browse/HIVE-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121023#comment-16121023 ] liyunzhang_intel commented on HIVE-17287: - [~gopalv]: thanks for your comments {quote} As little

[jira] [Assigned] (HIVE-17287) HoS can not deal with skewed data group by

[ https://issues.apache.org/jira/browse/HIVE-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel reassigned HIVE-17287: --- Assignee: liyunzhang_intel > HoS can not deal with skewed data group by >

[jira] [Commented] (HIVE-17261) Hive use deprecated ParquetInputSplit constructor which blocked parquet dictionary filter

[ https://issues.apache.org/jira/browse/HIVE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121062#comment-16121062 ] liyunzhang_intel commented on HIVE-17261: - [~junjie]: GTM from my side. [~ferd] and [~csun]:

[jira] [Commented] (HIVE-17287) HoS can not deal with skewed data group by

[ https://issues.apache.org/jira/browse/HIVE-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123123#comment-16123123 ] liyunzhang_intel commented on HIVE-17287: - [~lirui] : bq.You can run some statistics on the group

[jira] [Commented] (HIVE-17287) HoS can not deal with skewed data group by

[ https://issues.apache.org/jira/browse/HIVE-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16125125#comment-16125125 ] liyunzhang_intel commented on HIVE-17287: - [~xuefuz]: the memory related error is {noformat}

[jira] [Commented] (HIVE-17287) HoS can not deal with skewed data group by

[ https://issues.apache.org/jira/browse/HIVE-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16125155#comment-16125155 ] liyunzhang_intel commented on HIVE-17287: - [~lirui]: bq.Have you tried

[jira] [Commented] (HIVE-17287) HoS can not deal with skewed data group by

[ https://issues.apache.org/jira/browse/HIVE-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16125415#comment-16125415 ] liyunzhang_intel commented on HIVE-17287: - [~lirui]: When i viewed all tasks, saw that 1 task was

[jira] [Updated] (HIVE-17287) HoS can not deal with skewed data group by

[ https://issues.apache.org/jira/browse/HIVE-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-17287: Attachment: not_stages_completed_but_job_completed.PNG > HoS can not deal with skewed data

[jira] [Commented] (HIVE-17287) HoS can not deal with skewed data group by

[ https://issues.apache.org/jira/browse/HIVE-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16125326#comment-16125326 ] liyunzhang_intel commented on HIVE-17287: - [~lirui]: in current case, i have not set

[jira] [Updated] (HIVE-16948) Invalid explain when running dynamic partition pruning query in Hive On Spark

[ https://issues.apache.org/jira/browse/HIVE-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-16948: Attachment: HIVE-16948.6.patch [~lirui]: attached is HIVE-16948.6.patch. Update code with

[jira] [Commented] (HIVE-17321) HoS: analyze ORC table doesn't compute raw data size when noscan/partialscan is not specified

[ https://issues.apache.org/jira/browse/HIVE-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129989#comment-16129989 ] liyunzhang_intel commented on HIVE-17321: - [~lirui]: +1 > HoS: analyze ORC table doesn't compute

[jira] [Commented] (HIVE-17321) HoS: analyze ORC table doesn't compute raw data size when noscan/partialscan is not specified

[ https://issues.apache.org/jira/browse/HIVE-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129923#comment-16129923 ] liyunzhang_intel commented on HIVE-17321: - [~lirui]: patch looks good. But I have 1 question， why

[jira] [Comment Edited] (HIVE-17321) HoS: analyze ORC table doesn't compute raw data size when noscan/partialscan is not specified

[ https://issues.apache.org/jira/browse/HIVE-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129923#comment-16129923 ] liyunzhang_intel edited comment on HIVE-17321 at 8/17/17 5:06 AM: --

[jira] [Comment Edited] (HIVE-17321) HoS: analyze ORC table doesn't compute raw data size when noscan/partialscan is not specified

[ https://issues.apache.org/jira/browse/HIVE-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129923#comment-16129923 ] liyunzhang_intel edited comment on HIVE-17321 at 8/17/17 5:06 AM: --

[jira] [Commented] (HIVE-17287) HoS can not deal with skewed data group by

[ https://issues.apache.org/jira/browse/HIVE-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122840#comment-16122840 ] liyunzhang_intel commented on HIVE-17287: - [~gopalv] or [~lirui]: after enable

[jira] [Commented] (HIVE-17018) Small table is converted to map join even the total size of small tables exceeds the threshold(hive.auto.convert.join.noconditionaltask.size)

[ https://issues.apache.org/jira/browse/HIVE-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083480#comment-16083480 ] liyunzhang_intel commented on HIVE-17018: - [~csun]: {quote}Another way is to specify A as the

[jira] [Comment Edited] (HIVE-17018) Small table is converted to map join even the total size of small tables exceeds the threshold(hive.auto.convert.join.noconditionaltask.size)

[ https://issues.apache.org/jira/browse/HIVE-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16084774#comment-16084774 ] liyunzhang_intel edited comment on HIVE-17018 at 7/14/17 1:19 AM: --

[jira] [Commented] (HIVE-17108) Parquet file does not gather statistic such as "RAW DATA SIZE" automatically

[ https://issues.apache.org/jira/browse/HIVE-17108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16089491#comment-16089491 ] liyunzhang_intel commented on HIVE-17108: - [~csun] or [~xuefuz]: can you help to view it, thanks!

[jira] [Commented] (HIVE-17018) Small table is converted to map join even the total size of small tables exceeds the threshold(hive.auto.convert.join.noconditionaltask.size)

[ https://issues.apache.org/jira/browse/HIVE-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16078791#comment-16078791 ] liyunzhang_intel commented on HIVE-17018: - [~csun]: please spend some time to check whether this

[jira] [Commented] (HIVE-17018) Small table is converted to map join even the total size of small tables exceeds the threshold(hive.auto.convert.join.noconditionaltask.size)

[ https://issues.apache.org/jira/browse/HIVE-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083058#comment-16083058 ] liyunzhang_intel commented on HIVE-17018: - [~csun]: {quote}A better way might be to have a

[jira] [Commented] (HIVE-17010) Fix the overflow problem of Long type in SetSparkReducerParallelism

[ https://issues.apache.org/jira/browse/HIVE-17010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079823#comment-16079823 ] liyunzhang_intel commented on HIVE-17010: - [~Ferd]: as [~lirui] finished review, please help

[jira] [Commented] (HIVE-17018) Small table is converted to map join even the total size of small tables exceeds the threshold(hive.auto.convert.join.noconditionaltask.size)

[ https://issues.apache.org/jira/browse/HIVE-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081694#comment-16081694 ] liyunzhang_intel commented on HIVE-17018: - [~csun]: {quote} Are you trying to explain that HoS is

[jira] [Commented] (HIVE-17108) Parquet file does not gather statistic such as "RAW DATA SIZE" automatically

[ https://issues.apache.org/jira/browse/HIVE-17108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091284#comment-16091284 ] liyunzhang_intel commented on HIVE-17108: - [~pxiong]: when I view the code about [orc_analyze.q

[jira] [Commented] (HIVE-17108) Parquet file does not gather statistic such as "RAW DATA SIZE" automatically

[ https://issues.apache.org/jira/browse/HIVE-17108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090965#comment-16090965 ] liyunzhang_intel commented on HIVE-17108: - [~csun], [~xuefuz]: If we must use "ANALYZE TABLE

[jira] [Commented] (HIVE-17018) Small table is converted to map join even the total size of small tables exceeds the threshold(hive.auto.convert.join.noconditionaltask.size)

[ https://issues.apache.org/jira/browse/HIVE-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16089220#comment-16089220 ] liyunzhang_intel commented on HIVE-17018: - [~cartershanklin]: {quote}Maybe a new variable like

[jira] [Commented] (HIVE-17114) HoS: Possible skew in shuffling when data is not really skewed

[ https://issues.apache.org/jira/browse/HIVE-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094384#comment-16094384 ] liyunzhang_intel commented on HIVE-17114: - [~lirui]: {quote} It happens when the data has the

[jira] [Commented] (HIVE-17114) HoS: Possible skew in shuffling when data is not really skewed

[ https://issues.apache.org/jira/browse/HIVE-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094329#comment-16094329 ] liyunzhang_intel commented on HIVE-17114: - [~lirui]: thanks for your explanation, but i also want

[jira] [Comment Edited] (HIVE-17114) HoS: Possible skew in shuffling when data is not really skewed

[ https://issues.apache.org/jira/browse/HIVE-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094329#comment-16094329 ] liyunzhang_intel edited comment on HIVE-17114 at 7/20/17 8:40 AM: --

[jira] [Updated] (HIVE-17010) Fix the overflow problem of Long type in SetSparkReducerParallelism

[ https://issues.apache.org/jira/browse/HIVE-17010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-17010: Description: We use

[jira] [Commented] (HIVE-17114) HoS: Possible skew in shuffling when data is not really skewed

[ https://issues.apache.org/jira/browse/HIVE-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16092529#comment-16092529 ] liyunzhang_intel commented on HIVE-17114: - [~lirui]: can you provide detail example to explain the

[jira] [Commented] (HIVE-17114) HoS: Possible skew in shuffling when data is not really skewed

[ https://issues.apache.org/jira/browse/HIVE-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16092780#comment-16092780 ] liyunzhang_intel commented on HIVE-17114: - [~lirui]: several questions: 1.{quote} Spark decides

[jira] [Commented] (HIVE-17087) Remove unnecessary HoS DPP trees during map-join conversion

[ https://issues.apache.org/jira/browse/HIVE-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095886#comment-16095886 ] liyunzhang_intel commented on HIVE-17087: - [~stakiar]: 1 question about the patch 1. {noformat}

[jira] [Commented] (HIVE-16948) Invalid explain when running dynamic partition pruning query in HOS

[ https://issues.apache.org/jira/browse/HIVE-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099424#comment-16099424 ] liyunzhang_intel commented on HIVE-16948: - the reason why Map4 does not exist in the explain is

[jira] [Updated] (HIVE-16948) Invalid explain when running dynamic partition pruning query in HOS

[ https://issues.apache.org/jira/browse/HIVE-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-16948: Status: Patch Available (was: Open) > Invalid explain when running dynamic partition

[jira] [Updated] (HIVE-16948) Invalid explain when running dynamic partition pruning query in HOS

[ https://issues.apache.org/jira/browse/HIVE-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-16948: Description: in

[jira] [Updated] (HIVE-16948) Invalid explain when running dynamic partition pruning query in HOS

[ https://issues.apache.org/jira/browse/HIVE-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-16948: Attachment: HIVE-16948.patch > Invalid explain when running dynamic partition pruning query

[jira] [Commented] (HIVE-16948) Invalid explain when running dynamic partition pruning query in Hive On Spark

[ https://issues.apache.org/jira/browse/HIVE-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102849#comment-16102849 ] liyunzhang_intel commented on HIVE-16948: - [~lirui]: {quote} Is it possible that the DPP work

[jira] [Updated] (HIVE-17182) Invalid statistics like "RAW DATA SIZE" info for parquet file

[ https://issues.apache.org/jira/browse/HIVE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-17182: Description: on TPC-DS 200g scale store_sales use "describe formatted store_sales" to view

[jira] [Updated] (HIVE-17182) Invalid statistics like "RAW DATA SIZE" info for parquet file

[ https://issues.apache.org/jira/browse/HIVE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-17182: Description: on TPC-DS 200g scale store_sales use "describe formatted store_sales" to view

[jira] [Commented] (HIVE-16948) Invalid explain when running dynamic partition pruning query in Hive On Spark

[ https://issues.apache.org/jira/browse/HIVE-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104637#comment-16104637 ] liyunzhang_intel commented on HIVE-16948: - [~lirui]: thanks for your catch, it needs to remove

[jira] [Updated] (HIVE-16948) Invalid explain when running dynamic partition pruning query in Hive On Spark

[ https://issues.apache.org/jira/browse/HIVE-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-16948: Attachment: HIVE-16948.2.patch > Invalid explain when running dynamic partition pruning

[jira] [Commented] (HIVE-16948) Invalid explain when running dynamic partition pruning query in Hive On Spark

[ https://issues.apache.org/jira/browse/HIVE-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104700#comment-16104700 ] liyunzhang_intel commented on HIVE-16948: - {quote} Thinking more about this, I find a bug in

[jira] [Commented] (HIVE-17087) Remove unnecessary HoS DPP trees during map-join conversion

[ https://issues.apache.org/jira/browse/HIVE-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102338#comment-16102338 ] liyunzhang_intel commented on HIVE-17087: - [~stakiar]: GTM +1, meanwhile can you spend some time

[jira] [Comment Edited] (HIVE-17087) Remove unnecessary HoS DPP trees during map-join conversion

[ https://issues.apache.org/jira/browse/HIVE-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102338#comment-16102338 ] liyunzhang_intel edited comment on HIVE-17087 at 7/26/17 9:53 PM: --

[jira] [Updated] (HIVE-16948) Invalid explain when running dynamic partition pruning query in Hive On Spark

[ https://issues.apache.org/jira/browse/HIVE-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-16948: Attachment: HIVE-16948_1.patch upload HIVE-16948_1.patch to trigger HiveQA. > Invalid

[jira] [Commented] (HIVE-16948) Invalid explain when running dynamic partition pruning query in HOS

[ https://issues.apache.org/jira/browse/HIVE-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099616#comment-16099616 ] liyunzhang_intel commented on HIVE-16948: - One thing need to be mentioned here: why remove

[jira] [Commented] (HIVE-17087) Remove unnecessary HoS DPP trees during map-join conversion

[ https://issues.apache.org/jira/browse/HIVE-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099620#comment-16099620 ] liyunzhang_intel commented on HIVE-17087: - [~stakiar]: I think the 3rd patch is more clean to

[jira] [Commented] (HIVE-17122) spark_vectorized_dynamic_partition_pruning.q is continuously failing

[ https://issues.apache.org/jira/browse/HIVE-17122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094071#comment-16094071 ] liyunzhang_intel commented on HIVE-17122: - [~stakiar]: I also found the same problem on

[jira] [Commented] (HIVE-17108) Parquet file does not gather statistic such as "RAW DATA SIZE" automatically

[ https://issues.apache.org/jira/browse/HIVE-17108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094128#comment-16094128 ] liyunzhang_intel commented on HIVE-17108: - the detail reason why parquet file does not gather

[jira] [Commented] (HIVE-17087) Remove unnecessary HoS DPP trees during map-join conversion

[ https://issues.apache.org/jira/browse/HIVE-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098064#comment-16098064 ] liyunzhang_intel commented on HIVE-17087: - [~stakiar] {quote} It's actually suppose to be

[jira] [Commented] (HIVE-17018) Small table is converted to map join even the total size of small tables exceeds the threshold(hive.auto.convert.join.noconditionaltask.size)

[ https://issues.apache.org/jira/browse/HIVE-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16084774#comment-16084774 ] liyunzhang_intel commented on HIVE-17018: - [~csun]: {quote} Yes. I think we don't need to change

[jira] [Updated] (HIVE-17108) Parquet file does not gather statistic such as "RAW DATA SIZE" automatically

[ https://issues.apache.org/jira/browse/HIVE-17108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-17108: Attachment: (was: HIVE-17018.patch) > Parquet file does not gather statistic such as

[jira] [Updated] (HIVE-17108) Parquet file does not gather statistic such as "RAW DATA SIZE" automatically

[ https://issues.apache.org/jira/browse/HIVE-17108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated HIVE-17108: Attachment: HIVE-17018.patch > Parquet file does not gather statistic such as "RAW DATA

  1   2   3   4   >