[jira] [Updated] (HIVE-23004) Support Decimal64 operations across multiple vertices
[ https://issues.apache.org/jira/browse/HIVE-23004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-23004: Status: Open (was: Patch Available) > Support Decimal64 operations across multiple vertices > - > > Key: HIVE-23004 > URL: https://issues.apache.org/jira/browse/HIVE-23004 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-23004.1.patch, HIVE-23004.2.patch, > HIVE-23004.4.patch > > > Support Decimal64 operations across multiple vertices -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22997) Copy external table to target during Repl Dump operation
[ https://issues.apache.org/jira/browse/HIVE-22997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PRAVIN KUMAR SINHA updated HIVE-22997: -- Attachment: HIVE-22997.02.patch > Copy external table to target during Repl Dump operation > > > Key: HIVE-22997 > URL: https://issues.apache.org/jira/browse/HIVE-22997 > Project: Hive > Issue Type: Task >Reporter: PRAVIN KUMAR SINHA >Assignee: PRAVIN KUMAR SINHA >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22997.02.patch, HIVE-22997.1.patch, > HIVE-22997.2.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23004) Support Decimal64 operations across multiple vertices
[ https://issues.apache.org/jira/browse/HIVE-23004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-23004: Attachment: HIVE-23004.4.patch Status: Patch Available (was: Open) > Support Decimal64 operations across multiple vertices > - > > Key: HIVE-23004 > URL: https://issues.apache.org/jira/browse/HIVE-23004 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-23004.1.patch, HIVE-23004.2.patch, > HIVE-23004.4.patch > > > Support Decimal64 operations across multiple vertices -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[ https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22990: --- Status: In Progress (was: Patch Available) > Build acknowledgement mechanism for repl dump and load > -- > > Key: HIVE-22990 > URL: https://issues.apache.org/jira/browse/HIVE-22990 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch, > HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch, > HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch, > HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.11.patch, > HIVE-22990.12.patch, HIVE-22990.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[ https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22990: --- Attachment: HIVE-22990.12.patch Status: Patch Available (was: In Progress) > Build acknowledgement mechanism for repl dump and load > -- > > Key: HIVE-22990 > URL: https://issues.apache.org/jira/browse/HIVE-22990 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch, > HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch, > HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch, > HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.11.patch, > HIVE-22990.12.patch, HIVE-22990.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21778) CBO: "Struct is not null" gets evaluated as `nullable` always causing filter miss in the query
[
https://issues.apache.org/jira/browse/HIVE-21778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058431#comment-17058431
]
Hive QA commented on HIVE-21778:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21094/dev-support/hive-personality.sh
|
| git revision | master / 109ecb9 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21094/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21094/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CBO: "Struct is not null" gets evaluated as `nullable` always causing filter
> miss in the query
> --
>
> Key: HIVE-21778
> URL: https://issues.apache.org/jira/browse/HIVE-21778
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 4.0.0, 2.3.5
>Reporter: Rajesh Balamohan
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21778.1.patch, HIVE-21778.2.patch,
> HIVE-21778.3.patch, HIVE-21778.4.patch, HIVE-21778.5.patch,
> HIVE-21778.6.patch, HIVE-21778.7.patch, HIVE-21778.8.patch, test_null.q,
> test_null.q.out
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> {noformat}
> drop table if exists test_struct;
> CREATE external TABLE test_struct
> (
> f1 string,
> demo_struct struct,
> datestr string
> );
> set hive.cbo.enable=true;
> explain select * from etltmp.test_struct where datestr='2019-01-01' and
> demo_struct is not null;
> STAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: test_struct
> filterExpr: (datestr = '2019-01-01') (type: boolean) <- Note
> that demo_struct filter is not added here
> Filter Operator
> predicate: (datestr = '2019-01-01') (type: boolean)
> Select Operator
> expressions: f1 (type: string), demo_struct (type:
> struct), '2019-01-01' (type:
[jira] [Updated] (HIVE-22998) Dump partition info if hive.repl.dump.metadata.only.for.external.table conf is enabled
[ https://issues.apache.org/jira/browse/HIVE-22998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-22998: --- Resolution: Fixed Status: Resolved (was: Patch Available) [^HIVE-22998.12.patch] committed to master. Thanks [~aasha] . > Dump partition info if hive.repl.dump.metadata.only.for.external.table conf > is enabled > --- > > Key: HIVE-22998 > URL: https://issues.apache.org/jira/browse/HIVE-22998 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22998.01.patch, HIVE-22998.02.patch, > HIVE-22998.03.patch, HIVE-22998.04.patch, HIVE-22998.05.patch, > HIVE-22998.06.patch, HIVE-22998.07.patch, HIVE-22998.08.patch, > HIVE-22998.09.patch, HIVE-22998.10.patch, HIVE-22998.11.patch, > HIVE-22998.12.patch, HIVE-22998.patch > > Time Spent: 2h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23015) Fix HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED definition
[
https://issues.apache.org/jira/browse/HIVE-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058418#comment-17058418
]
Hive QA commented on HIVE-23015:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996552/HIVE-23015.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 18098 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21093/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21093/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21093/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12996552 - PreCommit-HIVE-Build
> Fix HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED definition
> ---
>
> Key: HIVE-23015
> URL: https://issues.apache.org/jira/browse/HIVE-23015
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Varga
>Priority: Minor
> Attachments: HIVE-23015.patch
>
>
> The _HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED_ is definied like this:
> {code}
> HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED("hive.vectorized.groupby.complex.types.enabled",
> true,
> "This flag should be set to true to enable group by vectorization\n" +
> "of aggregations that use complex types.\n",
> "For example, AVG uses a complex type (STRUCT) for partial aggregation
> results" +
> "The default value is true."),
> {code}
> Notice that the end of the "of aggregations that use complex types.\n" line
> is a "," instead of "+".
> So this will end up using the following constructor using the second sentence
> as an "altName" :)
> {code}
> ConfVars(String varname, Object defaultVal, String description, String
> altName) {
> this(varname, defaultVal, null, description, true, false, altName);
> }
> {code}
> We should fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21939) protoc:2.5.0 dependence has broken building on aarch64
[ https://issues.apache.org/jira/browse/HIVE-21939?focusedWorklogId=402645=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-402645 ] ASF GitHub Bot logged work on HIVE-21939: - Author: ASF GitHub Bot Created on: 13/Mar/20 03:33 Start Date: 13/Mar/20 03:33 Worklog Time Spent: 10m Work Description: ZhengZhenyu commented on pull request #953: HIVE-21939: Bump protoc version in standalone metadata URL: https://github.com/apache/hive/pull/953 Standalone metadata package currently depends on proroc 2.5.0 which does not support Aarch64 platform and blocks building Hive on Aarch64. This patch proposed to bump protoc used in standalone metadata to 2.6.1, and also added a new profile that detacts hardware architecture, override the default protoc package with package from com.github.os72(it has aarch64 support) if the hardware arch is Aarch64. By doing this, we will keep the affects to current X86 users to the minimum. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 402645) Time Spent: 40m (was: 0.5h) > protoc:2.5.0 dependence has broken building on aarch64 > --- > > Key: HIVE-21939 > URL: https://issues.apache.org/jira/browse/HIVE-21939 > Project: Hive > Issue Type: Bug >Reporter: liusheng >Assignee: Zhenyu Zheng >Priority: Blocker > Labels: pull-request-available > Attachments: HIVE-21939.1.patch, HIVE-21939.2.patch, > hive_build_with_protobuf_371.log > > Time Spent: 40m > Remaining Estimate: 0h > > When I try to build master of Hive from source code on "aarch64" server, I > met following error: > [ERROR] Failed to execute goal > com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on project > hive-standalone-metastore-common: Error resolving artifact: > com.google.protobuf:protoc:2.5.0: Could not find artifact > com.google.protobuf:protoc:exe:linux-aarch_64:2.5.0 in central > ([https://repo.maven.apache.org/maven2)] > that is because Hive using the "com.google.protobuf:protoc:2.5.0" as required > artifact, which does not have released package for "aarch64" platform. > Hadoop has the same problem and after some discussion, we upgraded protobuf > to version 3.7.1(https://issues.apache.org/jira/browse/HADOOP-13363) and the > work is mostly done. We can do the same in Hive. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23015) Fix HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED definition
[
https://issues.apache.org/jira/browse/HIVE-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058400#comment-17058400
]
Hive QA commented on HIVE-23015:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21093/dev-support/hive-personality.sh
|
| git revision | master / 9be51fd |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21093/yetus/patch-asflicense-problems.txt
|
| modules | C: common U: common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21093/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED definition
> ---
>
> Key: HIVE-23015
> URL: https://issues.apache.org/jira/browse/HIVE-23015
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Varga
>Priority: Minor
> Attachments: HIVE-23015.patch
>
>
> The _HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED_ is definied like this:
> {code}
> HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED("hive.vectorized.groupby.complex.types.enabled",
> true,
> "This flag should be set to true to enable group by vectorization\n" +
> "of aggregations that use complex types.\n",
> "For example, AVG uses a complex type (STRUCT) for partial aggregation
> results" +
> "The default value is true."),
> {code}
> Notice that the end of the "of aggregations that use complex types.\n" line
> is a "," instead of "+".
> So this will end up using the following constructor using the second sentence
> as an "altName" :)
> {code}
> ConfVars(String varname, Object defaultVal, String description, String
> altName) {
> this(varname, defaultVal, null, description, true, false, altName);
> }
> {code}
> We should fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[ https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22990: --- Attachment: HIVE-22990.11.patch Status: Patch Available (was: In Progress) > Build acknowledgement mechanism for repl dump and load > -- > > Key: HIVE-22990 > URL: https://issues.apache.org/jira/browse/HIVE-22990 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch, > HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch, > HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch, > HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.11.patch, > HIVE-22990.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[ https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22990: --- Status: In Progress (was: Patch Available) > Build acknowledgement mechanism for repl dump and load > -- > > Key: HIVE-22990 > URL: https://issues.apache.org/jira/browse/HIVE-22990 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch, > HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch, > HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch, > HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.11.patch, > HIVE-22990.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22785) Update/delete/merge statements not optimized through CBO
[
https://issues.apache.org/jira/browse/HIVE-22785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058386#comment-17058386
]
Hive QA commented on HIVE-22785:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996531/HIVE-22785.6.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 104 failed/errored test(s), 18098 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[map_join_on_filter]
(batchId=309)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_view_delete]
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_view_disable_cbo_1]
(batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join0] (batchId=102)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join15] (batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join20]
(batchId=103)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join21] (batchId=94)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join23] (batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join28] (batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join29] (batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join31] (batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join0]
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer14]
(batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas] (batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas_char] (batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas_date] (batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas_varchar]
(batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[identity_project_remove_skip]
(batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input4_limit]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input_part7] (batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join0] (batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join15] (batchId=94)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join20] (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join21] (batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join23] (batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join40] (batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_filter_on_outerjoin]
(batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_test_outer]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[no_hooks] (batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parallel_join0]
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pcs] (batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join4] (batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[print_header] (batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_ppr] (batchId=22)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_no_buckets]
(batchId=187)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_vectorization_original]
(batchId=190)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_join0]
(batchId=193)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_join21]
(batchId=191)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_join29]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_join30]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[check_constraint]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ctas]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[delete_where_non_partitioned]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_3]
(batchId=189)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization_acid]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[enforce_constraint_notnull]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[identity_project_remove_skip]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_into_default_keyword]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join0]
(batchId=184)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
[jira] [Commented] (HIVE-22785) Update/delete/merge statements not optimized through CBO
[
https://issues.apache.org/jira/browse/HIVE-22785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058353#comment-17058353
]
Hive QA commented on HIVE-22785:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 13 new + 516 unchanged - 4
fixed = 529 total (was 520) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
51s{color} | {color:red} ql generated 1 new + 1531 unchanged - 0 fixed = 1532
total (was 1531) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 14s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Dead store to fields in
org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveSortExchange,
ImmutableBitSet, Set) At
HiveRelFieldTrimmer.java:org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveSortExchange,
ImmutableBitSet, Set) At HiveRelFieldTrimmer.java:[line 933] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21092/dev-support/hive-personality.sh
|
| git revision | master / 9be51fd |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21092/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21092/yetus/new-findbugs-ql.html
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21092/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21092/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Update/delete/merge statements not optimized through CBO
>
>
> Key: HIVE-22785
> URL: https://issues.apache.org/jira/browse/HIVE-22785
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Critical
> Attachments: HIVE-22785.1.patch, HIVE-22785.2.patch,
> HIVE-22785.2.patch, HIVE-22785.3.patch, HIVE-22785.4.patch,
> HIVE-22785.5.patch, HIVE-22785.6.patch
>
>
> Currently, CBO is bypassed for update/delete/merge statements.
> To support optimizing these statements through CBO, we need to complete three
> main tasks: 1) support for sort in Calcite planner, 2) support for SORT in
> AST converter, and 3) {{RewriteSemanticAnalyzer}} should extend
> {{CalcitePlanner}} instead
[jira] [Updated] (HIVE-21939) protoc:2.5.0 dependence has broken building on aarch64
[ https://issues.apache.org/jira/browse/HIVE-21939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenyu Zheng updated HIVE-21939: Attachment: HIVE-21939.2.patch Status: Patch Available (was: Open) > protoc:2.5.0 dependence has broken building on aarch64 > --- > > Key: HIVE-21939 > URL: https://issues.apache.org/jira/browse/HIVE-21939 > Project: Hive > Issue Type: Bug >Reporter: liusheng >Assignee: Zhenyu Zheng >Priority: Blocker > Labels: pull-request-available > Attachments: HIVE-21939.1.patch, HIVE-21939.2.patch, > hive_build_with_protobuf_371.log > > Time Spent: 0.5h > Remaining Estimate: 0h > > When I try to build master of Hive from source code on "aarch64" server, I > met following error: > [ERROR] Failed to execute goal > com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on project > hive-standalone-metastore-common: Error resolving artifact: > com.google.protobuf:protoc:2.5.0: Could not find artifact > com.google.protobuf:protoc:exe:linux-aarch_64:2.5.0 in central > ([https://repo.maven.apache.org/maven2)] > that is because Hive using the "com.google.protobuf:protoc:2.5.0" as required > artifact, which does not have released package for "aarch64" platform. > Hadoop has the same problem and after some discussion, we upgraded protobuf > to version 3.7.1(https://issues.apache.org/jira/browse/HADOOP-13363) and the > work is mostly done. We can do the same in Hive. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21939) protoc:2.5.0 dependence has broken building on aarch64
[ https://issues.apache.org/jira/browse/HIVE-21939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenyu Zheng updated HIVE-21939: Status: Open (was: Patch Available) > protoc:2.5.0 dependence has broken building on aarch64 > --- > > Key: HIVE-21939 > URL: https://issues.apache.org/jira/browse/HIVE-21939 > Project: Hive > Issue Type: Bug >Reporter: liusheng >Assignee: Zhenyu Zheng >Priority: Blocker > Labels: pull-request-available > Attachments: HIVE-21939.1.patch, HIVE-21939.2.patch, > hive_build_with_protobuf_371.log > > Time Spent: 0.5h > Remaining Estimate: 0h > > When I try to build master of Hive from source code on "aarch64" server, I > met following error: > [ERROR] Failed to execute goal > com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on project > hive-standalone-metastore-common: Error resolving artifact: > com.google.protobuf:protoc:2.5.0: Could not find artifact > com.google.protobuf:protoc:exe:linux-aarch_64:2.5.0 in central > ([https://repo.maven.apache.org/maven2)] > that is because Hive using the "com.google.protobuf:protoc:2.5.0" as required > artifact, which does not have released package for "aarch64" platform. > Hadoop has the same problem and after some discussion, we upgraded protobuf > to version 3.7.1(https://issues.apache.org/jira/browse/HADOOP-13363) and the > work is mostly done. We can do the same in Hive. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21939) protoc:2.5.0 dependence has broken building on aarch64
[ https://issues.apache.org/jira/browse/HIVE-21939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenyu Zheng updated HIVE-21939: Attachment: (was: HIVE-21939.2.patch) > protoc:2.5.0 dependence has broken building on aarch64 > --- > > Key: HIVE-21939 > URL: https://issues.apache.org/jira/browse/HIVE-21939 > Project: Hive > Issue Type: Bug >Reporter: liusheng >Assignee: Zhenyu Zheng >Priority: Blocker > Labels: pull-request-available > Attachments: HIVE-21939.1.patch, hive_build_with_protobuf_371.log > > Time Spent: 0.5h > Remaining Estimate: 0h > > When I try to build master of Hive from source code on "aarch64" server, I > met following error: > [ERROR] Failed to execute goal > com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on project > hive-standalone-metastore-common: Error resolving artifact: > com.google.protobuf:protoc:2.5.0: Could not find artifact > com.google.protobuf:protoc:exe:linux-aarch_64:2.5.0 in central > ([https://repo.maven.apache.org/maven2)] > that is because Hive using the "com.google.protobuf:protoc:2.5.0" as required > artifact, which does not have released package for "aarch64" platform. > Hadoop has the same problem and after some discussion, we upgraded protobuf > to version 3.7.1(https://issues.apache.org/jira/browse/HADOOP-13363) and the > work is mostly done. We can do the same in Hive. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21939) protoc:2.5.0 dependence has broken building on aarch64
[ https://issues.apache.org/jira/browse/HIVE-21939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenyu Zheng updated HIVE-21939: Attachment: HIVE-21939.2.patch > protoc:2.5.0 dependence has broken building on aarch64 > --- > > Key: HIVE-21939 > URL: https://issues.apache.org/jira/browse/HIVE-21939 > Project: Hive > Issue Type: Bug >Reporter: liusheng >Assignee: Zhenyu Zheng >Priority: Blocker > Labels: pull-request-available > Attachments: HIVE-21939.1.patch, HIVE-21939.2.patch, > hive_build_with_protobuf_371.log > > Time Spent: 0.5h > Remaining Estimate: 0h > > When I try to build master of Hive from source code on "aarch64" server, I > met following error: > [ERROR] Failed to execute goal > com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on project > hive-standalone-metastore-common: Error resolving artifact: > com.google.protobuf:protoc:2.5.0: Could not find artifact > com.google.protobuf:protoc:exe:linux-aarch_64:2.5.0 in central > ([https://repo.maven.apache.org/maven2)] > that is because Hive using the "com.google.protobuf:protoc:2.5.0" as required > artifact, which does not have released package for "aarch64" platform. > Hadoop has the same problem and after some discussion, we upgraded protobuf > to version 3.7.1(https://issues.apache.org/jira/browse/HADOOP-13363) and the > work is mostly done. We can do the same in Hive. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22964) MM table split computation is very slow
[
https://issues.apache.org/jira/browse/HIVE-22964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058336#comment-17058336
]
Hive QA commented on HIVE-22964:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996515/HIVE-22964.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 18098 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21091/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21091/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21091/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12996515 - PreCommit-HIVE-Build
> MM table split computation is very slow
> ---
>
> Key: HIVE-22964
> URL: https://issues.apache.org/jira/browse/HIVE-22964
> Project: Hive
> Issue Type: Improvement
>Reporter: Aditya Shah
>Assignee: Aditya Shah
>Priority: Major
> Attachments: HIVE-22964.1.patch, HIVE-22964.2.patch,
> HIVE-22964.3.patch, HIVE-22964.patch
>
>
> Since for MM table we process the paths prior to inputFormat.getSplits() we
> end up doing listing on the whole table at once. This could be optimized.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22964) MM table split computation is very slow
[
https://issues.apache.org/jira/browse/HIVE-22964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058319#comment-17058319
]
Hive QA commented on HIVE-22964:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
57s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
44s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 1 new + 184 unchanged - 0
fixed = 185 total (was 184) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21091/dev-support/hive-personality.sh
|
| git revision | master / 9be51fd |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21091/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21091/yetus/patch-asflicense-problems.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21091/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> MM table split computation is very slow
> ---
>
> Key: HIVE-22964
> URL: https://issues.apache.org/jira/browse/HIVE-22964
> Project: Hive
> Issue Type: Improvement
>Reporter: Aditya Shah
>Assignee: Aditya Shah
>Priority: Major
> Attachments: HIVE-22964.1.patch, HIVE-22964.2.patch,
> HIVE-22964.3.patch, HIVE-22964.patch
>
>
> Since for MM table we process the paths prior to inputFormat.getSplits() we
> end up doing listing on the whole table at once. This could be optimized.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23002) Optimise LazyBinaryUtils.writeVLong
[
https://issues.apache.org/jira/browse/HIVE-23002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058306#comment-17058306
]
Hive QA commented on HIVE-23002:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996499/HIVE-23002.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 18098 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.beeline.TestBeeLineWithArgs.testRowsAffected (batchId=286)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21090/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21090/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21090/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12996499 - PreCommit-HIVE-Build
> Optimise LazyBinaryUtils.writeVLong
> ---
>
> Key: HIVE-23002
> URL: https://issues.apache.org/jira/browse/HIVE-23002
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-23002.1.patch, HIVE-23002.2.patch, Screenshot
> 2020-03-10 at 5.01.34 AM.jpg
>
>
> [https://github.com/apache/hive/blob/master/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java#L420]
> It would be good to add a method which accepts scratch bytes.
>
> !Screenshot 2020-03-10 at 5.01.34 AM.jpg|width=452,height=321!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23002) Optimise LazyBinaryUtils.writeVLong
[
https://issues.apache.org/jira/browse/HIVE-23002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058284#comment-17058284
]
Hive QA commented on HIVE-23002:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} serde in master has 197 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} serde: The patch generated 2 new + 55 unchanged - 0
fixed = 57 total (was 55) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21090/dev-support/hive-personality.sh
|
| git revision | master / c77ef8e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21090/yetus/diff-checkstyle-serde.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21090/yetus/patch-asflicense-problems.txt
|
| modules | C: serde U: serde |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21090/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Optimise LazyBinaryUtils.writeVLong
> ---
>
> Key: HIVE-23002
> URL: https://issues.apache.org/jira/browse/HIVE-23002
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-23002.1.patch, HIVE-23002.2.patch, Screenshot
> 2020-03-10 at 5.01.34 AM.jpg
>
>
> [https://github.com/apache/hive/blob/master/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java#L420]
> It would be good to add a method which accepts scratch bytes.
>
> !Screenshot 2020-03-10 at 5.01.34 AM.jpg|width=452,height=321!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23006) Compiler support for Probe MapJoin
[ https://issues.apache.org/jira/browse/HIVE-23006?focusedWorklogId=402532=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-402532 ] ASF GitHub Bot logged work on HIVE-23006: - Author: ASF GitHub Bot Created on: 12/Mar/20 22:52 Start Date: 12/Mar/20 22:52 Worklog Time Spent: 10m Work Description: pgaref commented on pull request #952: HIVE-23006 ProbeDecode compiler support URL: https://github.com/apache/hive/pull/952 This patch adds an extra optimisation step with the goal of finding Table Scan operators that could reduce the number of rows decoded at runtime using extra available information. It currently looks for all the available MapJoin operators that could use the smaller HashTable on the probing side (where TS is) to filter-out rows that would never match. To do so the HashTable information is pushed down to the TS properties and then propagated as part of MapWork. If the a single TS is used by multiple operators (shared-word), this rule can not be applied. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 402532) Remaining Estimate: 0h Time Spent: 10m > Compiler support for Probe MapJoin > -- > > Key: HIVE-23006 > URL: https://issues.apache.org/jira/browse/HIVE-23006 > Project: Hive > Issue Type: Sub-task >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The decision of pushing down information to the Record reader (potentially > reducing decoding time by row-level filtering) should be done at query > compilation time. > This patch adds an extra optimisation step with the goal of finding Table > Scan operators that could reduce the number of rows decoded at runtime using > extra available information. > It currently looks for all the available MapJoin operators that could use the > smaller HashTable on the probing side (where TS is) to filter-out rows that > would never match. > To do so the HashTable information is pushed down to the TS properties and > then propagated as part of MapWork. > If the a single TS is used by multiple operators (shared-word), this rule can > not be applied. > This rule can be extended to support static filter expressions like: > _select * from sales where sold_state = 'PR';_ > This optimisation manly targets the Tez execution engine running on Llap. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23006) Compiler support for Probe MapJoin
[ https://issues.apache.org/jira/browse/HIVE-23006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-23006: -- Labels: pull-request-available (was: ) > Compiler support for Probe MapJoin > -- > > Key: HIVE-23006 > URL: https://issues.apache.org/jira/browse/HIVE-23006 > Project: Hive > Issue Type: Sub-task >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > > The decision of pushing down information to the Record reader (potentially > reducing decoding time by row-level filtering) should be done at query > compilation time. > This patch adds an extra optimisation step with the goal of finding Table > Scan operators that could reduce the number of rows decoded at runtime using > extra available information. > It currently looks for all the available MapJoin operators that could use the > smaller HashTable on the probing side (where TS is) to filter-out rows that > would never match. > To do so the HashTable information is pushed down to the TS properties and > then propagated as part of MapWork. > If the a single TS is used by multiple operators (shared-word), this rule can > not be applied. > This rule can be extended to support static filter expressions like: > _select * from sales where sold_state = 'PR';_ > This optimisation manly targets the Tez execution engine running on Llap. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23006) Compiler support for Probe MapJoin
[ https://issues.apache.org/jira/browse/HIVE-23006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23006: -- Description: The decision of pushing down information to the Record reader (potentially reducing decoding time by row-level filtering) should be done at query compilation time. This patch adds an extra optimisation step with the goal of finding Table Scan operators that could reduce the number of rows decoded at runtime using extra available information. It currently looks for all the available MapJoin operators that could use the smaller HashTable on the probing side (where TS is) to filter-out rows that would never match. To do so the HashTable information is pushed down to the TS properties and then propagated as part of MapWork. If the a single TS is used by multiple operators (shared-word), this rule can not be applied. This rule can be extended to support static filter expressions like: _select * from sales where sold_state = 'PR';_ This optimisation manly targets the Tez execution engine running on Llap. > Compiler support for Probe MapJoin > -- > > Key: HIVE-23006 > URL: https://issues.apache.org/jira/browse/HIVE-23006 > Project: Hive > Issue Type: Sub-task >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > > The decision of pushing down information to the Record reader (potentially > reducing decoding time by row-level filtering) should be done at query > compilation time. > This patch adds an extra optimisation step with the goal of finding Table > Scan operators that could reduce the number of rows decoded at runtime using > extra available information. > It currently looks for all the available MapJoin operators that could use the > smaller HashTable on the probing side (where TS is) to filter-out rows that > would never match. > To do so the HashTable information is pushed down to the TS properties and > then propagated as part of MapWork. > If the a single TS is used by multiple operators (shared-word), this rule can > not be applied. > This rule can be extended to support static filter expressions like: > _select * from sales where sold_state = 'PR';_ > This optimisation manly targets the Tez execution engine running on Llap. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22998) Dump partition info if hive.repl.dump.metadata.only.for.external.table conf is enabled
[
https://issues.apache.org/jira/browse/HIVE-22998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058278#comment-17058278
]
Hive QA commented on HIVE-22998:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996509/HIVE-22998.12.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 18098 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21089/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21089/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21089/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12996509 - PreCommit-HIVE-Build
> Dump partition info if hive.repl.dump.metadata.only.for.external.table conf
> is enabled
> ---
>
> Key: HIVE-22998
> URL: https://issues.apache.org/jira/browse/HIVE-22998
> Project: Hive
> Issue Type: Bug
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22998.01.patch, HIVE-22998.02.patch,
> HIVE-22998.03.patch, HIVE-22998.04.patch, HIVE-22998.05.patch,
> HIVE-22998.06.patch, HIVE-22998.07.patch, HIVE-22998.08.patch,
> HIVE-22998.09.patch, HIVE-22998.10.patch, HIVE-22998.11.patch,
> HIVE-22998.12.patch, HIVE-22998.patch
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21851) FireEventResponse should include event id when available
[
https://issues.apache.org/jira/browse/HIVE-21851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058276#comment-17058276
]
Vihang Karajgaonkar commented on HIVE-21851:
Patch merged into master. I will attach a branch-3 patch.
> FireEventResponse should include event id when available
>
>
> Key: HIVE-21851
> URL: https://issues.apache.org/jira/browse/HIVE-21851
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Attachments: HIVE-21851.01.patch, HIVE-21851.02.patch,
> HIVE-21851.03.patch, HIVE-21851.04.patch, HIVE-21851.05.patch,
> HIVE-21851.06.patch
>
>
> The metastore API {{fire_listener_event}} gives clients the ability to fire a
> INSERT event on DML operations. However, the returned response is empty
> struct. It would be useful to sent back the event id information in the
> response so that clients can take actions based of the event id.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22998) Dump partition info if hive.repl.dump.metadata.only.for.external.table conf is enabled
[
https://issues.apache.org/jira/browse/HIVE-22998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058259#comment-17058259
]
Hive QA commented on HIVE-22998:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21089/dev-support/hive-personality.sh
|
| git revision | master / 6838d95 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21089/yetus/patch-asflicense-problems.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21089/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Dump partition info if hive.repl.dump.metadata.only.for.external.table conf
> is enabled
> ---
>
> Key: HIVE-22998
> URL: https://issues.apache.org/jira/browse/HIVE-22998
> Project: Hive
> Issue Type: Bug
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22998.01.patch, HIVE-22998.02.patch,
> HIVE-22998.03.patch, HIVE-22998.04.patch, HIVE-22998.05.patch,
> HIVE-22998.06.patch, HIVE-22998.07.patch, HIVE-22998.08.patch,
> HIVE-22998.09.patch, HIVE-22998.10.patch, HIVE-22998.11.patch,
> HIVE-22998.12.patch, HIVE-22998.patch
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22997) Copy external table to target during Repl Dump operation
[ https://issues.apache.org/jira/browse/HIVE-22997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PRAVIN KUMAR SINHA updated HIVE-22997: -- Attachment: HIVE-22997.2.patch > Copy external table to target during Repl Dump operation > > > Key: HIVE-22997 > URL: https://issues.apache.org/jira/browse/HIVE-22997 > Project: Hive > Issue Type: Task >Reporter: PRAVIN KUMAR SINHA >Assignee: PRAVIN KUMAR SINHA >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22997.1.patch, HIVE-22997.2.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23019) Fix TestTxnCommandsForMmTable test case
[ https://issues.apache.org/jira/browse/HIVE-23019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Varga updated HIVE-23019: --- Attachment: HIVE-23019.patch > Fix TestTxnCommandsForMmTable test case > --- > > Key: HIVE-23019 > URL: https://issues.apache.org/jira/browse/HIVE-23019 > Project: Hive > Issue Type: Bug > Components: Test >Reporter: Peter Varga >Assignee: Peter Varga >Priority: Minor > Attachments: HIVE-23019.patch > > > TestTxnCommandsForMmTable.testInsertOverwriteForPartitionedMmTable was fixed > in HIVE-19084 to avoid being dependent on the order of the element returned > by FileSystem.listStatus. > However the fix introduced a new bug, as now the assertion for the base > directory name doesn't run for the second partition, instead it runs twice > for the first one. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23011) Shared work optimizer should check residual predicates when comparing joins
[ https://issues.apache.org/jira/browse/HIVE-23011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-23011: --- Attachment: HIVE-23011.patch > Shared work optimizer should check residual predicates when comparing joins > --- > > Key: HIVE-23011 > URL: https://issues.apache.org/jira/browse/HIVE-23011 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-23011.patch, HIVE-23011.patch, HIVE-23011.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23019) Fix TestTxnCommandsForMmTable test case
[ https://issues.apache.org/jira/browse/HIVE-23019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Varga updated HIVE-23019: --- Status: Patch Available (was: In Progress) > Fix TestTxnCommandsForMmTable test case > --- > > Key: HIVE-23019 > URL: https://issues.apache.org/jira/browse/HIVE-23019 > Project: Hive > Issue Type: Bug > Components: Test >Reporter: Peter Varga >Assignee: Peter Varga >Priority: Minor > Attachments: HIVE-23019.patch > > > TestTxnCommandsForMmTable.testInsertOverwriteForPartitionedMmTable was fixed > in HIVE-19084 to avoid being dependent on the order of the element returned > by FileSystem.listStatus. > However the fix introduced a new bug, as now the assertion for the base > directory name doesn't run for the second partition, instead it runs twice > for the first one. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23011) Shared work optimizer should check residual predicates when comparing joins
[
https://issues.apache.org/jira/browse/HIVE-23011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058243#comment-17058243
]
Hive QA commented on HIVE-23011:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996493/HIVE-23011.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 18099 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.beeline.TestBeeLineWithArgs.testRowsAffected (batchId=286)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21088/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21088/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21088/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12996493 - PreCommit-HIVE-Build
> Shared work optimizer should check residual predicates when comparing joins
> ---
>
> Key: HIVE-23011
> URL: https://issues.apache.org/jira/browse/HIVE-23011
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-23011.patch, HIVE-23011.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23019) Fix TestTxnCommandsForMmTable test case
[ https://issues.apache.org/jira/browse/HIVE-23019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Varga reassigned HIVE-23019: -- > Fix TestTxnCommandsForMmTable test case > --- > > Key: HIVE-23019 > URL: https://issues.apache.org/jira/browse/HIVE-23019 > Project: Hive > Issue Type: Bug > Components: Test >Reporter: Peter Varga >Assignee: Peter Varga >Priority: Minor > > TestTxnCommandsForMmTable.testInsertOverwriteForPartitionedMmTable was fixed > in HIVE-19084 to avoid being dependent on the order of the element returned > by FileSystem.listStatus. > However the fix introduced a new bug, as now the assertion for the base > directory name doesn't run for the second partition, instead it runs twice > for the first one. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-23019) Fix TestTxnCommandsForMmTable test case
[ https://issues.apache.org/jira/browse/HIVE-23019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-23019 started by Peter Varga. -- > Fix TestTxnCommandsForMmTable test case > --- > > Key: HIVE-23019 > URL: https://issues.apache.org/jira/browse/HIVE-23019 > Project: Hive > Issue Type: Bug > Components: Test >Reporter: Peter Varga >Assignee: Peter Varga >Priority: Minor > > TestTxnCommandsForMmTable.testInsertOverwriteForPartitionedMmTable was fixed > in HIVE-19084 to avoid being dependent on the order of the element returned > by FileSystem.listStatus. > However the fix introduced a new bug, as now the assertion for the base > directory name doesn't run for the second partition, instead it runs twice > for the first one. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22940) Make the datasketches functions available as predefined functions
[
https://issues.apache.org/jira/browse/HIVE-22940?focusedWorklogId=402471=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-402471
]
ASF GitHub Bot logged work on HIVE-22940:
-
Author: ASF GitHub Bot
Created on: 12/Mar/20 20:38
Start Date: 12/Mar/20 20:38
Worklog Time Spent: 10m
Work Description: jcamachor commented on pull request #930: HIVE-22940
datasketches functions
URL: https://github.com/apache/hive/pull/930#discussion_r391881328
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/DataSketchesFunctions.java
##
@@ -0,0 +1,218 @@
+package org.apache.hadoop.hive.ql.exec;
+
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDAFResolver2;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDTF;
+
+public class DataSketchesFunctions {
+
+ private static final String DATA_TO_SKETCH = "datatosketch";
+ private static final String SKETCH_TO_ESTIMATE_WITH_ERROR_BOUNDS =
"sketchToEstimateWithErrorBounds";
+ // FIXME: consider to rename it to simply "estimate" or "evaluate" - in case
of the counting sketches the "sketchto..." doesnt add value
+ private static final String SKETCH_TO_ESTIMATE = "sketchToEstimate";
+ private static final String SKETCH_TO_STRING = "sketchToString";
+ // FIXME: probably use simply "union" instead unionSketch?
+ private static final String UNION_SKETCH = "unionSketch";
+ private static final String GET_N = "getN";
+ private static final String GET_CDF = "getCdf";
+ private static final String GET_PMF = "getPmf";
+ private static final String GET_QUANTILES = "GetQuantiles";
+ private static final String GET_QUANTILE = "GetQuantile";
+ private static final String GET_RANK = "getRank";
+ private static final String INTERSECT_SKETCH = "intersection";
+ private static final String EXCLUDE_SKETCH = "exclude";
+ private static final String GET_K = "getK";
+ private static final String GET_FREQUENT_ITEMS = "getFrequentItems";
+ private static final String T_TEST = "TTest";
+ private static final String SKETCH_TO_MEANS = "sketchtomeans";
+ private static final String SKETCH_TO_NUMBER_OF_RETAINED_ENTRIES =
"sketchtonumberofretainedentries";
+ private static final String SKETCH_TO_QUANTILES_SKETCH =
"sketchToQuantilesSketch";
+ private static final String SKETCH_TO_VALUES = "sketchToValues";
+ private static final String SKETCH_TO_VARIANCES = "sketchToVariances";
+ private static final String SKETCH_TO_PERCENTILE = "sketchToPercentile";
+ private static final String UNION_SKETCH1 = "unionSketch1";
+ private static final String INTERSECT_SKETCH1 = "intersect";
+
+ private final Registry system;
+
+ public DataSketchesFunctions(Registry system) {
+this.system = system;
+ }
+
+ public static void register(Registry system) {
+DataSketchesFunctions dsf = new DataSketchesFunctions(system);
+// FIXME: what this should be approx, ds ... other?
+String prefix = "ds";
Review comment:
I like `ds`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 402471)
Time Spent: 2.5h (was: 2h 20m)
> Make the datasketches functions available as predefined functions
> --
>
> Key: HIVE-22940
> URL: https://issues.apache.org/jira/browse/HIVE-22940
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22940.01.patch, HIVE-22940.02.patch
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22940) Make the datasketches functions available as predefined functions
[
https://issues.apache.org/jira/browse/HIVE-22940?focusedWorklogId=402469=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-402469
]
ASF GitHub Bot logged work on HIVE-22940:
-
Author: ASF GitHub Bot
Created on: 12/Mar/20 20:37
Start Date: 12/Mar/20 20:37
Worklog Time Spent: 10m
Work Description: jcamachor commented on pull request #930: HIVE-22940
datasketches functions
URL: https://github.com/apache/hive/pull/930#discussion_r391880759
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/DataSketchesFunctions.java
##
@@ -0,0 +1,218 @@
+package org.apache.hadoop.hive.ql.exec;
+
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDAFResolver2;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDTF;
+
+public class DataSketchesFunctions {
+
+ private static final String DATA_TO_SKETCH = "datatosketch";
+ private static final String SKETCH_TO_ESTIMATE_WITH_ERROR_BOUNDS =
"sketchToEstimateWithErrorBounds";
+ // FIXME: consider to rename it to simply "estimate" or "evaluate" - in case
of the counting sketches the "sketchto..." doesnt add value
+ private static final String SKETCH_TO_ESTIMATE = "sketchToEstimate";
+ private static final String SKETCH_TO_STRING = "sketchToString";
+ // FIXME: probably use simply "union" instead unionSketch?
+ private static final String UNION_SKETCH = "unionSketch";
+ private static final String GET_N = "getN";
+ private static final String GET_CDF = "getCdf";
+ private static final String GET_PMF = "getPmf";
+ private static final String GET_QUANTILES = "GetQuantiles";
+ private static final String GET_QUANTILE = "GetQuantile";
+ private static final String GET_RANK = "getRank";
+ private static final String INTERSECT_SKETCH = "intersection";
+ private static final String EXCLUDE_SKETCH = "exclude";
+ private static final String GET_K = "getK";
+ private static final String GET_FREQUENT_ITEMS = "getFrequentItems";
+ private static final String T_TEST = "TTest";
+ private static final String SKETCH_TO_MEANS = "sketchtomeans";
+ private static final String SKETCH_TO_NUMBER_OF_RETAINED_ENTRIES =
"sketchtonumberofretainedentries";
Review comment:
LGTM, if you remove `get` in all of them, use `n_retained`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 402469)
Time Spent: 2h 20m (was: 2h 10m)
> Make the datasketches functions available as predefined functions
> --
>
> Key: HIVE-22940
> URL: https://issues.apache.org/jira/browse/HIVE-22940
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22940.01.patch, HIVE-22940.02.patch
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22940) Make the datasketches functions available as predefined functions
[
https://issues.apache.org/jira/browse/HIVE-22940?focusedWorklogId=402468=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-402468
]
ASF GitHub Bot logged work on HIVE-22940:
-
Author: ASF GitHub Bot
Created on: 12/Mar/20 20:36
Start Date: 12/Mar/20 20:36
Worklog Time Spent: 10m
Work Description: jcamachor commented on pull request #930: HIVE-22940
datasketches functions
URL: https://github.com/apache/hive/pull/930#discussion_r391880374
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/DataSketchesFunctions.java
##
@@ -0,0 +1,218 @@
+package org.apache.hadoop.hive.ql.exec;
+
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDAFResolver2;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDTF;
+
+public class DataSketchesFunctions {
+
+ private static final String DATA_TO_SKETCH = "datatosketch";
+ private static final String SKETCH_TO_ESTIMATE_WITH_ERROR_BOUNDS =
"sketchToEstimateWithErrorBounds";
Review comment:
+1
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 402468)
Time Spent: 2h 10m (was: 2h)
> Make the datasketches functions available as predefined functions
> --
>
> Key: HIVE-22940
> URL: https://issues.apache.org/jira/browse/HIVE-22940
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22940.01.patch, HIVE-22940.02.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22940) Make the datasketches functions available as predefined functions
[
https://issues.apache.org/jira/browse/HIVE-22940?focusedWorklogId=402467=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-402467
]
ASF GitHub Bot logged work on HIVE-22940:
-
Author: ASF GitHub Bot
Created on: 12/Mar/20 20:36
Start Date: 12/Mar/20 20:36
Worklog Time Spent: 10m
Work Description: jcamachor commented on pull request #930: HIVE-22940
datasketches functions
URL: https://github.com/apache/hive/pull/930#discussion_r391880266
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/DataSketchesFunctions.java
##
@@ -0,0 +1,218 @@
+package org.apache.hadoop.hive.ql.exec;
+
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDAFResolver2;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDTF;
+
+public class DataSketchesFunctions {
+
+ private static final String DATA_TO_SKETCH = "datatosketch";
Review comment:
LGTM, I think they should easy enough to identify, and it is good we follow
same pattern.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 402467)
Time Spent: 2h (was: 1h 50m)
> Make the datasketches functions available as predefined functions
> --
>
> Key: HIVE-22940
> URL: https://issues.apache.org/jira/browse/HIVE-22940
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22940.01.patch, HIVE-22940.02.patch
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22940) Make the datasketches functions available as predefined functions
[
https://issues.apache.org/jira/browse/HIVE-22940?focusedWorklogId=402466=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-402466
]
ASF GitHub Bot logged work on HIVE-22940:
-
Author: ASF GitHub Bot
Created on: 12/Mar/20 20:36
Start Date: 12/Mar/20 20:36
Worklog Time Spent: 10m
Work Description: jcamachor commented on pull request #930: HIVE-22940
datasketches functions
URL: https://github.com/apache/hive/pull/930#discussion_r391880118
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/DataSketchesFunctions.java
##
@@ -0,0 +1,218 @@
+package org.apache.hadoop.hive.ql.exec;
+
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDAFResolver2;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDTF;
+
+public class DataSketchesFunctions {
+
+ private static final String DATA_TO_SKETCH = "datatosketch";
+ private static final String SKETCH_TO_ESTIMATE_WITH_ERROR_BOUNDS =
"sketchToEstimateWithErrorBounds";
+ // FIXME: consider to rename it to simply "estimate" or "evaluate" - in case
of the counting sketches the "sketchto..." doesnt add value
+ private static final String SKETCH_TO_ESTIMATE = "sketchToEstimate";
Review comment:
makes sense, +1
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 402466)
Time Spent: 1h 50m (was: 1h 40m)
> Make the datasketches functions available as predefined functions
> --
>
> Key: HIVE-22940
> URL: https://issues.apache.org/jira/browse/HIVE-22940
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22940.01.patch, HIVE-22940.02.patch
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23011) Shared work optimizer should check residual predicates when comparing joins
[
https://issues.apache.org/jira/browse/HIVE-23011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058227#comment-17058227
]
Hive QA commented on HIVE-23011:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21088/dev-support/hive-personality.sh
|
| git revision | master / 48bc9e3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21088/yetus/whitespace-eol.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21088/yetus/patch-asflicense-problems.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21088/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Shared work optimizer should check residual predicates when comparing joins
> ---
>
> Key: HIVE-23011
> URL: https://issues.apache.org/jira/browse/HIVE-23011
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-23011.patch, HIVE-23011.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21218) KafkaSerDe doesn't support topics created via Confluent Avro serializer
[ https://issues.apache.org/jira/browse/HIVE-21218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058218#comment-17058218 ] David McGinnis commented on HIVE-21218: --- Thanks, [~bslim]! I'm not seeing any documentation on the Hive wiki for Kafka support. It looks like based on HIVE-20377 that it won't be officially released until Hive 4, but I assumed there would be some documentation written at this point. Is there any existing documentation I should add this information to? If not, are there any JIRAs to write such documentation around for Hive 4? Or should we have a task to take care of that? Being the creator of the feature, you'd obviously be the best person to do that work if not, but I'm happy to help if necessary. > KafkaSerDe doesn't support topics created via Confluent Avro serializer > --- > > Key: HIVE-21218 > URL: https://issues.apache.org/jira/browse/HIVE-21218 > Project: Hive > Issue Type: Bug > Components: kafka integration, Serializers/Deserializers >Affects Versions: 3.1.1 >Reporter: Milan Baran >Assignee: David McGinnis >Priority: Major > Labels: pull-request-available > Attachments: > 0001-HIVE-21818-Adding-ability-for-Kafka-Handler-to-proce.patch, > HIVE-21218.10.patch, HIVE-21218.11.patch, HIVE-21218.12.patch, > HIVE-21218.13.patch, HIVE-21218.2.patch, HIVE-21218.3.patch, > HIVE-21218.4.patch, HIVE-21218.5.patch, HIVE-21218.6.patch, > HIVE-21218.7.patch, HIVE-21218.8.patch, HIVE-21218.9.patch, HIVE-21218.patch > > Time Spent: 15h > Remaining Estimate: 0h > > According to [Google > groups|https://groups.google.com/forum/#!topic/confluent-platform/JYhlXN0u9_A] > the Confluent avro serialzier uses propertiary format for kafka value - > <4 bytes of schema ID> conforms to schema>. > This format does not cause any problem for Confluent kafka deserializer which > respect the format however for hive kafka handler its bit a problem to > correctly deserialize kafka value, because Hive uses custom deserializer from > bytes to objects and ignores kafka consumer ser/deser classes provided via > table property. > It would be nice to support Confluent format with magic byte. > Also it would be great to support Schema registry as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23009) SEL operator created by DynamicPartitionPruningOptimization does not populate colExprMap
[ https://issues.apache.org/jira/browse/HIVE-23009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-23009: --- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. > SEL operator created by DynamicPartitionPruningOptimization does not populate > colExprMap > > > Key: HIVE-23009 > URL: https://issues.apache.org/jira/browse/HIVE-23009 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer, Statistics >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-23009.01.patch, HIVE-23009.patch > > > This can lead to incorrect column stats propagation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21218) KafkaSerDe doesn't support topics created via Confluent Avro serializer
[ https://issues.apache.org/jira/browse/HIVE-21218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Slim Bouguerra updated HIVE-21218: -- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks [~mcginnda] https://git-wip-us.apache.org/repos/asf?p=hive.git;a=commit;h=48bc9e389c9b88a6e25bec3fd19ed29130f3f156 > KafkaSerDe doesn't support topics created via Confluent Avro serializer > --- > > Key: HIVE-21218 > URL: https://issues.apache.org/jira/browse/HIVE-21218 > Project: Hive > Issue Type: Bug > Components: kafka integration, Serializers/Deserializers >Affects Versions: 3.1.1 >Reporter: Milan Baran >Assignee: David McGinnis >Priority: Major > Labels: pull-request-available > Attachments: > 0001-HIVE-21818-Adding-ability-for-Kafka-Handler-to-proce.patch, > HIVE-21218.10.patch, HIVE-21218.11.patch, HIVE-21218.12.patch, > HIVE-21218.13.patch, HIVE-21218.2.patch, HIVE-21218.3.patch, > HIVE-21218.4.patch, HIVE-21218.5.patch, HIVE-21218.6.patch, > HIVE-21218.7.patch, HIVE-21218.8.patch, HIVE-21218.9.patch, HIVE-21218.patch > > Time Spent: 15h > Remaining Estimate: 0h > > According to [Google > groups|https://groups.google.com/forum/#!topic/confluent-platform/JYhlXN0u9_A] > the Confluent avro serialzier uses propertiary format for kafka value - > <4 bytes of schema ID> conforms to schema>. > This format does not cause any problem for Confluent kafka deserializer which > respect the format however for hive kafka handler its bit a problem to > correctly deserialize kafka value, because Hive uses custom deserializer from > bytes to objects and ignores kafka consumer ser/deser classes provided via > table property. > It would be nice to support Confluent format with magic byte. > Also it would be great to support Schema registry as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23009) SEL operator created by DynamicPartitionPruningOptimization does not populate colExprMap
[
https://issues.apache.org/jira/browse/HIVE-23009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058202#comment-17058202
]
Hive QA commented on HIVE-23009:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996479/HIVE-23009.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 18098 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21087/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21087/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21087/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12996479 - PreCommit-HIVE-Build
> SEL operator created by DynamicPartitionPruningOptimization does not populate
> colExprMap
>
>
> Key: HIVE-23009
> URL: https://issues.apache.org/jira/browse/HIVE-23009
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer, Statistics
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-23009.01.patch, HIVE-23009.patch
>
>
> This can lead to incorrect column stats propagation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23018) Provide a bulk API to fire multiple listener events
[
https://issues.apache.org/jira/browse/HIVE-23018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058172#comment-17058172
]
Vihang Karajgaonkar commented on HIVE-23018:
{noformat}
struct InsertEventRequestData {
1: optional bool replace,
2: required list filesAdded,
// Checksum of files (hex string of checksum byte payload)
3: optional list filesAddedChecksum,
// Used by acid operation to create the sub directory
4: optional list subDirectoryList,
}
union FireEventRequestData {
1: InsertEventRequestData insertData
}
struct FireEventRequest {
1: required bool successful,
2: required FireEventRequestData data
// dbname, tablename, and partition vals are included as optional in the
top level event rather than placed in each type of
// subevent as I assume they'll be used across most event types.
3: optional string dbName,
4: optional string tableName,
5: optional list partitionVals,
6: optional string catName,
}
struct FireEventResponse {
// NOP for now, this is just a place holder for future responses
}
{noformat}
The thrift structures above can be reused to have a new API to fire multiple
events. Most of the thrift structures above can be reused. I propose to add a
field to the {{FireEventRequestData}} which takes a
{{list}} so that multiple events can be fired. Since
{{FireEventRequestData}} is defined as union only one field is allowed to be
set so existing clients can continue to use the same API to fire event one by
one while newer clients can make use of the bulk events in one RPC call.
> Provide a bulk API to fire multiple listener events
> ---
>
> Key: HIVE-23018
> URL: https://issues.apache.org/jira/browse/HIVE-23018
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
>
> Metastore provides a API to fire a listener event (currently only supports
> INSERT event). The problem with that API is that it only takes in one
> partition at a time. A typical query may insert data into multiple partitions
> at a time. In such a case query engines like HS2 or Impala will have to issue
> multiple RPCs to metastore sequentially to fire these events. This can show
> up as a slowdown to the user if the query engines do not return the prompt to
> the user until all the events are fired (In case of HS2 and Impala). It would
> be great if we have bulk API which takes in multiple partitions for a table
> so that metastore can generate many such events in one RPC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23009) SEL operator created by DynamicPartitionPruningOptimization does not populate colExprMap
[
https://issues.apache.org/jira/browse/HIVE-23009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058169#comment-17058169
]
Hive QA commented on HIVE-23009:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
47s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21087/dev-support/hive-personality.sh
|
| git revision | master / 812a626 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21087/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21087/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> SEL operator created by DynamicPartitionPruningOptimization does not populate
> colExprMap
>
>
> Key: HIVE-23009
> URL: https://issues.apache.org/jira/browse/HIVE-23009
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer, Statistics
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-23009.01.patch, HIVE-23009.patch
>
>
> This can lead to incorrect column stats propagation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22853) Allow JDBC FetchSize to Be Set in Beeline
[ https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058161#comment-17058161 ] David Mollitor commented on HIVE-22853: --- [~ngangam] Can you please review this patch? :) > Allow JDBC FetchSize to Be Set in Beeline > - > > Key: HIVE-22853 > URL: https://issues.apache.org/jira/browse/HIVE-22853 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, > HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.6.patch, > HIVE-22853.7.patch, HIVE-22853.patch > > > Currently beeline uses a hard coded default of 1000 rows for fetchSize. This > default value is different from what the server has set. While the beeline > user can reset the value via set command, its cumbersome to change the > workloads. > Rather it should default to the server-side value and set should be used to > override within the session. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21218) KafkaSerDe doesn't support topics created via Confluent Avro serializer
[ https://issues.apache.org/jira/browse/HIVE-21218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058163#comment-17058163 ] David McGinnis commented on HIVE-21218: --- [~bslim]: Looks like everything passed in the last test run. > KafkaSerDe doesn't support topics created via Confluent Avro serializer > --- > > Key: HIVE-21218 > URL: https://issues.apache.org/jira/browse/HIVE-21218 > Project: Hive > Issue Type: Bug > Components: kafka integration, Serializers/Deserializers >Affects Versions: 3.1.1 >Reporter: Milan Baran >Assignee: David McGinnis >Priority: Major > Labels: pull-request-available > Attachments: > 0001-HIVE-21818-Adding-ability-for-Kafka-Handler-to-proce.patch, > HIVE-21218.10.patch, HIVE-21218.11.patch, HIVE-21218.12.patch, > HIVE-21218.13.patch, HIVE-21218.2.patch, HIVE-21218.3.patch, > HIVE-21218.4.patch, HIVE-21218.5.patch, HIVE-21218.6.patch, > HIVE-21218.7.patch, HIVE-21218.8.patch, HIVE-21218.9.patch, HIVE-21218.patch > > Time Spent: 15h > Remaining Estimate: 0h > > According to [Google > groups|https://groups.google.com/forum/#!topic/confluent-platform/JYhlXN0u9_A] > the Confluent avro serialzier uses propertiary format for kafka value - > <4 bytes of schema ID> conforms to schema>. > This format does not cause any problem for Confluent kafka deserializer which > respect the format however for hive kafka handler its bit a problem to > correctly deserialize kafka value, because Hive uses custom deserializer from > bytes to objects and ignores kafka consumer ser/deser classes provided via > table property. > It would be nice to support Confluent format with magic byte. > Also it would be great to support Schema registry as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23018) Provide a bulk API to fire multiple listener events
[ https://issues.apache.org/jira/browse/HIVE-23018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar reassigned HIVE-23018: -- > Provide a bulk API to fire multiple listener events > --- > > Key: HIVE-23018 > URL: https://issues.apache.org/jira/browse/HIVE-23018 > Project: Hive > Issue Type: Improvement >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > > Metastore provides a API to fire a listener event (currently only supports > INSERT event). The problem with that API is that it only takes in one > partition at a time. A typical query may insert data into multiple partitions > at a time. In such a case query engines like HS2 or Impala will have to issue > multiple RPCs to metastore sequentially to fire these events. This can show > up as a slowdown to the user if the query engines do not return the prompt to > the user until all the events are fired (In case of HS2 and Impala). It would > be great if we have bulk API which takes in multiple partitions for a table > so that metastore can generate many such events in one RPC. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22959) Extend storage-api to expose FilterContext
[ https://issues.apache.org/jira/browse/HIVE-22959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-22959: -- Attachment: HIVE-22959.2.patch > Extend storage-api to expose FilterContext > -- > > Key: HIVE-22959 > URL: https://issues.apache.org/jira/browse/HIVE-22959 > Project: Hive > Issue Type: Sub-task > Components: storage-api >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22959.1.patch, HIVE-22959.2.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > To enable row-level filtering at the ORC level ORC-577, or as an extension > ProDecode MapJoin HIVE-22731 we need a common context class that will hold > all the needed information for the filter. > I propose this class to be part of the storage-api – similar to > VectorizedRowBatch class and hold the information below: > * A boolean variable showing if the filter is enabled > * A int array storing the row Ids that are actually selected (passing the > filter) > * An int variable storing the the number or rows that passed the filter > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22959) Extend storage-api to expose FilterContext
[ https://issues.apache.org/jira/browse/HIVE-22959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-22959: -- Status: Patch Available (was: Open) > Extend storage-api to expose FilterContext > -- > > Key: HIVE-22959 > URL: https://issues.apache.org/jira/browse/HIVE-22959 > Project: Hive > Issue Type: Sub-task > Components: storage-api >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22959.1.patch, HIVE-22959.2.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > To enable row-level filtering at the ORC level ORC-577, or as an extension > ProDecode MapJoin HIVE-22731 we need a common context class that will hold > all the needed information for the filter. > I propose this class to be part of the storage-api – similar to > VectorizedRowBatch class and hold the information below: > * A boolean variable showing if the filter is enabled > * A int array storing the row Ids that are actually selected (passing the > filter) > * An int variable storing the the number or rows that passed the filter > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22959) Extend storage-api to expose FilterContext
[ https://issues.apache.org/jira/browse/HIVE-22959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-22959: -- Status: Open (was: Patch Available) > Extend storage-api to expose FilterContext > -- > > Key: HIVE-22959 > URL: https://issues.apache.org/jira/browse/HIVE-22959 > Project: Hive > Issue Type: Sub-task > Components: storage-api >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22959.1.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > To enable row-level filtering at the ORC level ORC-577, or as an extension > ProDecode MapJoin HIVE-22731 we need a common context class that will hold > all the needed information for the filter. > I propose this class to be part of the storage-api – similar to > VectorizedRowBatch class and hold the information below: > * A boolean variable showing if the filter is enabled > * A int array storing the row Ids that are actually selected (passing the > filter) > * An int variable storing the the number or rows that passed the filter > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22959) Extend storage-api to expose FilterContext
[ https://issues.apache.org/jira/browse/HIVE-22959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-22959: -- Status: Patch Available (was: In Progress) > Extend storage-api to expose FilterContext > -- > > Key: HIVE-22959 > URL: https://issues.apache.org/jira/browse/HIVE-22959 > Project: Hive > Issue Type: Sub-task > Components: storage-api >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22959.1.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > To enable row-level filtering at the ORC level ORC-577, or as an extension > ProDecode MapJoin HIVE-22731 we need a common context class that will hold > all the needed information for the filter. > I propose this class to be part of the storage-api – similar to > VectorizedRowBatch class and hold the information below: > * A boolean variable showing if the filter is enabled > * A int array storing the row Ids that are actually selected (passing the > filter) > * An int variable storing the the number or rows that passed the filter > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[
https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058151#comment-17058151
]
Hive QA commented on HIVE-22990:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996565/HIVE-22990.08.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 46 failed/errored test(s), 18098 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[repl_load_requires_admin]
(batchId=107)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesBootstrapWithJsonMessage.testRetryAcidTablesBootstrapFromDifferentDump
(batchId=259)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testAbortTxnEvent
(batchId=277)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testAcidBootstrapReplLoadRetryAfterFailure
(batchId=277)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testAcidTablesBootstrapWithConcurrentDropTable
(batchId=277)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testOpenTxnEvent
(batchId=277)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testBootstrapLoadRetryAfterFailureForAlterTable
(batchId=268)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testBootstrapReplLoadRetryAfterFailureForFunctions
(batchId=268)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testBootstrapReplLoadRetryAfterFailureForPartitions
(batchId=268)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testBootstrapReplLoadRetryAfterFailureForTablesAndConstraints
(batchId=268)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testCreateFunctionIncrementalReplication
(batchId=268)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testDropFunctionIncrementalReplication
(batchId=268)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testIncrementalDumpEmptyDumpDirectory
(batchId=268)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testMoveOptimizationBootstrapReplLoadRetryAfterFailure
(batchId=268)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testMoveOptimizationIncrementalFailureAfterCopy
(batchId=268)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testMoveOptimizationIncrementalFailureAfterCopyReplace
(batchId=268)
org.apache.hadoop.hive.ql.parse.TestReplTableMigrationWithJsonFormat.testIncrementalLoadMigrationManagedToAcidAllOp
(batchId=275)
org.apache.hadoop.hive.ql.parse.TestReplTableMigrationWithJsonFormat.testIncrementalLoadMigrationManagedToAcidFailure
(batchId=275)
org.apache.hadoop.hive.ql.parse.TestReplTableMigrationWithJsonFormat.testIncrementalLoadMigrationManagedToAcidFailurePart
(batchId=275)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAbortTxnEvent
(batchId=279)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAcidBootstrapReplLoadRetryAfterFailure
(batchId=279)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAcidTablesBootstrapWithConcurrentDropTable
(batchId=279)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testOpenTxnEvent
(batchId=279)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTablesBootstrap.testRetryAcidTablesBootstrapFromDifferentDump
(batchId=257)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testBootstrapLoadRetryAfterFailureForAlterTable
(batchId=273)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testBootstrapReplLoadRetryAfterFailureForFunctions
(batchId=273)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testBootstrapReplLoadRetryAfterFailureForPartitions
(batchId=273)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testBootstrapReplLoadRetryAfterFailureForTablesAndConstraints
(batchId=273)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testCreateFunctionIncrementalReplication
(batchId=273)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testDropFunctionIncrementalReplication
(batchId=273)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testIncrementalDumpEmptyDumpDirectory
(batchId=273)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testMoveOptimizationBootstrapReplLoadRetryAfterFailure
(batchId=273)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testMoveOptimizationIncrementalFailureAfterCopy
(batchId=273)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testMoveOptimizationIncrementalFailureAfterCopyReplace
(batchId=273)
[jira] [Updated] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[ https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22990: --- Attachment: HIVE-22990.10.patch Status: Patch Available (was: In Progress) > Build acknowledgement mechanism for repl dump and load > -- > > Key: HIVE-22990 > URL: https://issues.apache.org/jira/browse/HIVE-22990 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch, > HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch, > HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch, > HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[ https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22990: --- Status: In Progress (was: Patch Available) > Build acknowledgement mechanism for repl dump and load > -- > > Key: HIVE-22990 > URL: https://issues.apache.org/jira/browse/HIVE-22990 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch, > HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch, > HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch, > HIVE-22990.09.patch, HIVE-22990.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[ https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22990: --- Status: In Progress (was: Patch Available) > Build acknowledgement mechanism for repl dump and load > -- > > Key: HIVE-22990 > URL: https://issues.apache.org/jira/browse/HIVE-22990 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch, > HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch, > HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch, > HIVE-22990.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[ https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22990: --- Attachment: HIVE-22990.09.patch Status: Patch Available (was: In Progress) > Build acknowledgement mechanism for repl dump and load > -- > > Key: HIVE-22990 > URL: https://issues.apache.org/jira/browse/HIVE-22990 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch, > HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch, > HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch, > HIVE-22990.09.patch, HIVE-22990.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[
https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058117#comment-17058117
]
Hive QA commented on HIVE-22990:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
46s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
43s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
24s{color} | {color:red} itests/hive-unit: The patch generated 3 new + 693
unchanged - 3 fixed = 696 total (was 696) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21085/dev-support/hive-personality.sh
|
| git revision | master / 812a626 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21085/yetus/diff-checkstyle-itests_hive-unit.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21085/yetus/patch-asflicense-problems.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21085/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Build acknowledgement mechanism for repl dump and load
> --
>
> Key: HIVE-22990
> URL: https://issues.apache.org/jira/browse/HIVE-22990
> Project: Hive
> Issue Type: Bug
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch,
> HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch,
> HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch,
> HIVE-22990.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22126) hive-exec packaging should shade guava
[
https://issues.apache.org/jira/browse/HIVE-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058101#comment-17058101
]
Eugene Chung commented on HIVE-22126:
-
I don't see any relevance between the patch and failed tests. If this patch has
some problems, it causes java.lang.NoClassDefFoundError or method signature
error.
1. TestMiniLlapLocalCliDriver.testCliDriver[schq_ingest] is just the query
result difference.
2. Other two tests show TCP related errors.
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schq_ingest]
{code:java}
java.lang.AssertionError:
Client Execution succeeded but contained differences (error code = 1) after
executing schq_ingest.q
114d113
< FINISHED NULL
at org.junit.Assert.fail(Assert.java:88)
at org.apache.hadoop.hive.ql.QTestUtil.failedDiff(QTestUtil.java:1012)
at
org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:182)
at org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:157)
at
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver(TestMiniLlapLocalCliDriver.java:59)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:135)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at
org.apache.hadoop.hive.cli.control.CliAdapter$1$1.evaluate(CliAdapter.java:95)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
{code}
org.apache.hive.jdbc.TestJdbcWithMiniHS2.org.apache.hive.jdbc.TestJdbcWithMiniHS2
{code:java}
2020-03-11T04:35:21,077 ERROR [main] service.CompositeService: Error starting
services HiveServer2
java.lang.RuntimeException: Failed to init thrift server
at
org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.initServer(ThriftBinaryCLIService.java:162)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
org.apache.hive.service.cli.thrift.ThriftCLIService.start(ThriftCLIService.java:218)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hive.service.CompositeService.start(CompositeService.java:70)
[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:644)
[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hive.jdbc.miniHS2.MiniHS2.start(MiniHS2.java:377)
[hive-it-util-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
org.apache.hive.jdbc.TestJdbcWithMiniHS2.startMiniHS2(TestJdbcWithMiniHS2.java:229)
[jira] [Updated] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[
https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23005:
--
Attachment: HIVE-23005.4.patch
> Consider Default JDBC Fetch Size From HS2
> -
>
> Key: HIVE-23005
> URL: https://issues.apache.org/jira/browse/HIVE-23005
> Project: Hive
> Issue Type: Sub-task
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23005.1.patch, HIVE-23005.2.patch,
> HIVE-23005.3.patch, HIVE-23005.4.patch
>
>
> # Create two fetch sizes: and "init" fetch size and a "default" fetch size.
> The "init" fetch size comes from the JDBC connection string _fetchSize_ (if
> present) and the "default" fetch size comes from
> _HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
> open session request.
> # When a {{Statement}} is created, its starting fetch size is the "init"
> fetch size (may be 0)
> # Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
> size to be the server default
> Setting to zero defaults to the server's instructed default and adheres to:
> bq. If the value specified is zero, then the hint is ignored. The default
> value is zero.
> That is to say, if the fetch size is 0, the default from the server is used,
> otherwise the user can pass a 'hint' and that will be the number of rows
> fetched instead.
> https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21218) KafkaSerDe doesn't support topics created via Confluent Avro serializer
[
https://issues.apache.org/jira/browse/HIVE-21218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058079#comment-17058079
]
Hive QA commented on HIVE-21218:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996474/HIVE-21218.13.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 18098 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21084/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21084/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21084/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12996474 - PreCommit-HIVE-Build
> KafkaSerDe doesn't support topics created via Confluent Avro serializer
> ---
>
> Key: HIVE-21218
> URL: https://issues.apache.org/jira/browse/HIVE-21218
> Project: Hive
> Issue Type: Bug
> Components: kafka integration, Serializers/Deserializers
>Affects Versions: 3.1.1
>Reporter: Milan Baran
>Assignee: David McGinnis
>Priority: Major
> Labels: pull-request-available
> Attachments:
> 0001-HIVE-21818-Adding-ability-for-Kafka-Handler-to-proce.patch,
> HIVE-21218.10.patch, HIVE-21218.11.patch, HIVE-21218.12.patch,
> HIVE-21218.13.patch, HIVE-21218.2.patch, HIVE-21218.3.patch,
> HIVE-21218.4.patch, HIVE-21218.5.patch, HIVE-21218.6.patch,
> HIVE-21218.7.patch, HIVE-21218.8.patch, HIVE-21218.9.patch, HIVE-21218.patch
>
> Time Spent: 15h
> Remaining Estimate: 0h
>
> According to [Google
> groups|https://groups.google.com/forum/#!topic/confluent-platform/JYhlXN0u9_A]
> the Confluent avro serialzier uses propertiary format for kafka value -
> <4 bytes of schema ID> conforms to schema>.
> This format does not cause any problem for Confluent kafka deserializer which
> respect the format however for hive kafka handler its bit a problem to
> correctly deserialize kafka value, because Hive uses custom deserializer from
> bytes to objects and ignores kafka consumer ser/deser classes provided via
> table property.
> It would be nice to support Confluent format with magic byte.
> Also it would be great to support Schema registry as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-21778) CBO: "Struct is not null" gets evaluated as `nullable` always causing filter miss in the query
[
https://issues.apache.org/jira/browse/HIVE-21778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-21778:
---
Status: Open (was: Patch Available)
> CBO: "Struct is not null" gets evaluated as `nullable` always causing filter
> miss in the query
> --
>
> Key: HIVE-21778
> URL: https://issues.apache.org/jira/browse/HIVE-21778
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 2.3.5, 4.0.0
>Reporter: Rajesh Balamohan
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21778.1.patch, HIVE-21778.2.patch,
> HIVE-21778.3.patch, HIVE-21778.4.patch, HIVE-21778.5.patch,
> HIVE-21778.6.patch, HIVE-21778.7.patch, HIVE-21778.8.patch, test_null.q,
> test_null.q.out
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> {noformat}
> drop table if exists test_struct;
> CREATE external TABLE test_struct
> (
> f1 string,
> demo_struct struct,
> datestr string
> );
> set hive.cbo.enable=true;
> explain select * from etltmp.test_struct where datestr='2019-01-01' and
> demo_struct is not null;
> STAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: test_struct
> filterExpr: (datestr = '2019-01-01') (type: boolean) <- Note
> that demo_struct filter is not added here
> Filter Operator
> predicate: (datestr = '2019-01-01') (type: boolean)
> Select Operator
> expressions: f1 (type: string), demo_struct (type:
> struct), '2019-01-01' (type: string)
> outputColumnNames: _col0, _col1, _col2
> ListSink
> set hive.cbo.enable=false;
> explain select * from etltmp.test_struct where datestr='2019-01-01' and
> demo_struct is not null;
> STAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: test_struct
> filterExpr: ((datestr = '2019-01-01') and demo_struct is not null)
> (type: boolean) <- Note that demo_struct filter is added when CBO is
> turned off
> Filter Operator
> predicate: ((datestr = '2019-01-01') and demo_struct is not null)
> (type: boolean)
> Select Operator
> expressions: f1 (type: string), demo_struct (type:
> struct), '2019-01-01' (type: string)
> outputColumnNames: _col0, _col1, _col2
> ListSink
> {noformat}
> In CalcitePlanner::genFilterRelNode, the following code misses to evaluate
> this filter.
> {noformat}
> RexNode factoredFilterExpr = RexUtil
> .pullFactors(cluster.getRexBuilder(), convertedFilterExpr);
> {noformat}
> Note that even if we add `demo_struct.f1` it would end up pushing the filter
> correctly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-21778) CBO: "Struct is not null" gets evaluated as `nullable` always causing filter miss in the query
[
https://issues.apache.org/jira/browse/HIVE-21778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-21778:
---
Status: Patch Available (was: Open)
> CBO: "Struct is not null" gets evaluated as `nullable` always causing filter
> miss in the query
> --
>
> Key: HIVE-21778
> URL: https://issues.apache.org/jira/browse/HIVE-21778
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 2.3.5, 4.0.0
>Reporter: Rajesh Balamohan
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21778.1.patch, HIVE-21778.2.patch,
> HIVE-21778.3.patch, HIVE-21778.4.patch, HIVE-21778.5.patch,
> HIVE-21778.6.patch, HIVE-21778.7.patch, HIVE-21778.8.patch, test_null.q,
> test_null.q.out
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> {noformat}
> drop table if exists test_struct;
> CREATE external TABLE test_struct
> (
> f1 string,
> demo_struct struct,
> datestr string
> );
> set hive.cbo.enable=true;
> explain select * from etltmp.test_struct where datestr='2019-01-01' and
> demo_struct is not null;
> STAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: test_struct
> filterExpr: (datestr = '2019-01-01') (type: boolean) <- Note
> that demo_struct filter is not added here
> Filter Operator
> predicate: (datestr = '2019-01-01') (type: boolean)
> Select Operator
> expressions: f1 (type: string), demo_struct (type:
> struct), '2019-01-01' (type: string)
> outputColumnNames: _col0, _col1, _col2
> ListSink
> set hive.cbo.enable=false;
> explain select * from etltmp.test_struct where datestr='2019-01-01' and
> demo_struct is not null;
> STAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: test_struct
> filterExpr: ((datestr = '2019-01-01') and demo_struct is not null)
> (type: boolean) <- Note that demo_struct filter is added when CBO is
> turned off
> Filter Operator
> predicate: ((datestr = '2019-01-01') and demo_struct is not null)
> (type: boolean)
> Select Operator
> expressions: f1 (type: string), demo_struct (type:
> struct), '2019-01-01' (type: string)
> outputColumnNames: _col0, _col1, _col2
> ListSink
> {noformat}
> In CalcitePlanner::genFilterRelNode, the following code misses to evaluate
> this filter.
> {noformat}
> RexNode factoredFilterExpr = RexUtil
> .pullFactors(cluster.getRexBuilder(), convertedFilterExpr);
> {noformat}
> Note that even if we add `demo_struct.f1` it would end up pushing the filter
> correctly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-21778) CBO: "Struct is not null" gets evaluated as `nullable` always causing filter miss in the query
[
https://issues.apache.org/jira/browse/HIVE-21778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-21778:
---
Attachment: HIVE-21778.8.patch
> CBO: "Struct is not null" gets evaluated as `nullable` always causing filter
> miss in the query
> --
>
> Key: HIVE-21778
> URL: https://issues.apache.org/jira/browse/HIVE-21778
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 4.0.0, 2.3.5
>Reporter: Rajesh Balamohan
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21778.1.patch, HIVE-21778.2.patch,
> HIVE-21778.3.patch, HIVE-21778.4.patch, HIVE-21778.5.patch,
> HIVE-21778.6.patch, HIVE-21778.7.patch, HIVE-21778.8.patch, test_null.q,
> test_null.q.out
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> {noformat}
> drop table if exists test_struct;
> CREATE external TABLE test_struct
> (
> f1 string,
> demo_struct struct,
> datestr string
> );
> set hive.cbo.enable=true;
> explain select * from etltmp.test_struct where datestr='2019-01-01' and
> demo_struct is not null;
> STAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: test_struct
> filterExpr: (datestr = '2019-01-01') (type: boolean) <- Note
> that demo_struct filter is not added here
> Filter Operator
> predicate: (datestr = '2019-01-01') (type: boolean)
> Select Operator
> expressions: f1 (type: string), demo_struct (type:
> struct), '2019-01-01' (type: string)
> outputColumnNames: _col0, _col1, _col2
> ListSink
> set hive.cbo.enable=false;
> explain select * from etltmp.test_struct where datestr='2019-01-01' and
> demo_struct is not null;
> STAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: test_struct
> filterExpr: ((datestr = '2019-01-01') and demo_struct is not null)
> (type: boolean) <- Note that demo_struct filter is added when CBO is
> turned off
> Filter Operator
> predicate: ((datestr = '2019-01-01') and demo_struct is not null)
> (type: boolean)
> Select Operator
> expressions: f1 (type: string), demo_struct (type:
> struct), '2019-01-01' (type: string)
> outputColumnNames: _col0, _col1, _col2
> ListSink
> {noformat}
> In CalcitePlanner::genFilterRelNode, the following code misses to evaluate
> this filter.
> {noformat}
> RexNode factoredFilterExpr = RexUtil
> .pullFactors(cluster.getRexBuilder(), convertedFilterExpr);
> {noformat}
> Note that even if we add `demo_struct.f1` it would end up pushing the filter
> correctly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21218) KafkaSerDe doesn't support topics created via Confluent Avro serializer
[
https://issues.apache.org/jira/browse/HIVE-21218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058052#comment-17058052
]
Hive QA commented on HIVE-21218:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
23s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} serde in master has 197 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
14s{color} | {color:red} kafka-handler in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 54m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile findbugs
checkstyle |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21084/dev-support/hive-personality.sh
|
| git revision | master / 812a626 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21084/yetus/patch-mvninstall-kafka-handler.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21084/yetus/patch-asflicense-problems.txt
|
| modules | C: serde kafka-handler . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21084/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> KafkaSerDe doesn't support topics created via Confluent Avro serializer
> ---
>
> Key: HIVE-21218
> URL: https://issues.apache.org/jira/browse/HIVE-21218
> Project: Hive
> Issue Type: Bug
> Components: kafka integration, Serializers/Deserializers
>Affects Versions: 3.1.1
>Reporter: Milan Baran
>Assignee: David McGinnis
>Priority: Major
> Labels: pull-request-available
> Attachments:
> 0001-HIVE-21818-Adding-ability-for-Kafka-Handler-to-proce.patch,
> HIVE-21218.10.patch, HIVE-21218.11.patch, HIVE-21218.12.patch,
> HIVE-21218.13.patch, HIVE-21218.2.patch, HIVE-21218.3.patch,
> HIVE-21218.4.patch, HIVE-21218.5.patch, HIVE-21218.6.patch,
> HIVE-21218.7.patch, HIVE-21218.8.patch, HIVE-21218.9.patch, HIVE-21218.patch
>
> Time Spent: 15h
> Remaining Estimate: 0h
>
> According to [Google
>
[jira] [Updated] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[ https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22990: --- Status: In Progress (was: Patch Available) > Build acknowledgement mechanism for repl dump and load > -- > > Key: HIVE-22990 > URL: https://issues.apache.org/jira/browse/HIVE-22990 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch, > HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch, > HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch, > HIVE-22990.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[ https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22990: --- Attachment: HIVE-22990.08.patch Status: Patch Available (was: In Progress) > Build acknowledgement mechanism for repl dump and load > -- > > Key: HIVE-22990 > URL: https://issues.apache.org/jira/browse/HIVE-22990 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch, > HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch, > HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch, > HIVE-22990.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23014) ORC reading performance
[
https://issues.apache.org/jira/browse/HIVE-23014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058019#comment-17058019
]
Panagiotis Garefalakis commented on HIVE-23014:
---
I believe this was addressed by HIVE-18738 and is now part of Hive 3.0 – not
sure if back-porting it to 2.3.6 would make sense (?)
> ORC reading performance
> ---
>
> Key: HIVE-23014
> URL: https://issues.apache.org/jira/browse/HIVE-23014
> Project: Hive
> Issue Type: Bug
> Components: ORC
>Affects Versions: 2.3.6
>Reporter: Peter Toth
>Priority: Major
> Attachments: OrcReadBenchmark-results.txt.hive-1.2.1,
> OrcReadBenchmark-results.txt.hive-2.3.6
>
>
> Spark 3 adds support for using Hive 2.3.6 besides the old Hive 1.2.1 version.
> Some of the ORC reading benchmark shows that there is a huge performance
> difference in ORC reading between the 2 versions. I measured that
> {{org.apache.hadoop.hive.ql.io.orc.ReaderImpl}} in hive-exec-2.3.6-core.jar
> is ~3-5 times slower than in hive-exec-1.2.1.spark2.jar.
> I'm not sure if more recent Hive versions still suffer from this performance
> regression.
> Please see some details here: SPARK-30565
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[ https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22990: --- Attachment: HIVE-22990.07.patch Status: Patch Available (was: In Progress) > Build acknowledgement mechanism for repl dump and load > -- > > Key: HIVE-22990 > URL: https://issues.apache.org/jira/browse/HIVE-22990 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch, > HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch, > HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[ https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22990: --- Status: In Progress (was: Patch Available) > Build acknowledgement mechanism for repl dump and load > -- > > Key: HIVE-22990 > URL: https://issues.apache.org/jira/browse/HIVE-22990 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch, > HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch, > HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-23014) ORC reading performance
[
https://issues.apache.org/jira/browse/HIVE-23014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058005#comment-17058005
]
Panagiotis Garefalakis edited comment on HIVE-23014 at 3/12/20, 3:12 PM:
-
Thanks for the extra details [~petertoth]
I have a feeling that the included columns Options is not properly set for the
OrcReader and it ends up reading the whole dataset.
For instance, for 200columns the runtime is 2x compared to reading 100 columns
and in a similar manner reading 300columns is 3x (while it should read just 1
column each time).
I can also see that there are some major changes in getIncludedColumns method
in 2.3.6 that could be an issue –
[https://github.com/apache/hive/blob/2c2fdd524e8783f6e1f3ef15281cc2d5ed08728f/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L396]
cc: [~gopalv] [~ashutoshc] [~omalley]
was (Author: pgaref):
Thanks for the extra details [~petertoth]
I have a feeling that the included columns Options is not properly set for the
OrcReader and it ends up reading the whole dataset.
For instance, for 200columns the runtime is 2x compared to reading 100 columns
and in a similar manner reading 300columns is 3x (while it should read just 1
column each time).
I can also see that there are some major changes in getIncludedColumns method
in 2.3.6 –
[https://github.com/apache/hive/blob/2c2fdd524e8783f6e1f3ef15281cc2d5ed08728f/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L396]
cc: [~gopalv] [~ashutoshc] [~omalley]
> ORC reading performance
> ---
>
> Key: HIVE-23014
> URL: https://issues.apache.org/jira/browse/HIVE-23014
> Project: Hive
> Issue Type: Bug
> Components: ORC
>Affects Versions: 2.3.6
>Reporter: Peter Toth
>Priority: Major
> Attachments: OrcReadBenchmark-results.txt.hive-1.2.1,
> OrcReadBenchmark-results.txt.hive-2.3.6
>
>
> Spark 3 adds support for using Hive 2.3.6 besides the old Hive 1.2.1 version.
> Some of the ORC reading benchmark shows that there is a huge performance
> difference in ORC reading between the 2 versions. I measured that
> {{org.apache.hadoop.hive.ql.io.orc.ReaderImpl}} in hive-exec-2.3.6-core.jar
> is ~3-5 times slower than in hive-exec-1.2.1.spark2.jar.
> I'm not sure if more recent Hive versions still suffer from this performance
> regression.
> Please see some details here: SPARK-30565
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23014) ORC reading performance
[
https://issues.apache.org/jira/browse/HIVE-23014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058005#comment-17058005
]
Panagiotis Garefalakis commented on HIVE-23014:
---
Thanks for the extra details [~petertoth]
I have a feeling that the included columns Options is not properly set for the
OrcReader and it ends up reading the whole dataset.
For instance, for 200columns the runtime is 2x compared to reading 100 columns
and in a similar manner reading 300columns is 3x (while it should read just 1
column each time).
I can also see that there are some major changes in getIncludedColumns method
in 2.3.6 –
[https://github.com/apache/hive/blob/2c2fdd524e8783f6e1f3ef15281cc2d5ed08728f/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L396]
cc: [~gopalv] [~ashutoshc] [~omalley]
> ORC reading performance
> ---
>
> Key: HIVE-23014
> URL: https://issues.apache.org/jira/browse/HIVE-23014
> Project: Hive
> Issue Type: Bug
> Components: ORC
>Affects Versions: 2.3.6
>Reporter: Peter Toth
>Priority: Major
> Attachments: OrcReadBenchmark-results.txt.hive-1.2.1,
> OrcReadBenchmark-results.txt.hive-2.3.6
>
>
> Spark 3 adds support for using Hive 2.3.6 besides the old Hive 1.2.1 version.
> Some of the ORC reading benchmark shows that there is a huge performance
> difference in ORC reading between the 2 versions. I measured that
> {{org.apache.hadoop.hive.ql.io.orc.ReaderImpl}} in hive-exec-2.3.6-core.jar
> is ~3-5 times slower than in hive-exec-1.2.1.spark2.jar.
> I'm not sure if more recent Hive versions still suffer from this performance
> regression.
> Please see some details here: SPARK-30565
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22853) Allow JDBC FetchSize to Be Set in Beeline
[
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058003#comment-17058003
]
Hive QA commented on HIVE-22853:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996426/HIVE-22853.7.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 18098 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21083/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21083/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21083/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12996426 - PreCommit-HIVE-Build
> Allow JDBC FetchSize to Be Set in Beeline
> -
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch,
> HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.6.patch,
> HIVE-22853.7.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This
> default value is different from what the server has set. While the beeline
> user can reset the value via set command, its cumbersome to change the
> workloads.
> Rather it should default to the server-side value and set should be used to
> override within the session.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23016) Extract JdbcConnectionParams from Utils Class
[ https://issues.apache.org/jira/browse/HIVE-23016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23016: -- Description: And make it its own class. https://github.com/apache/hive/blob/4700e210ef7945278c4eb313c9ebd810b0224da1/jdbc/src/java/org/apache/hive/jdbc/Utils.java#L72 was:And make it its own class. > Extract JdbcConnectionParams from Utils Class > - > > Key: HIVE-23016 > URL: https://issues.apache.org/jira/browse/HIVE-23016 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Priority: Minor > Labels: n00b, newbie, noob > > And make it its own class. > https://github.com/apache/hive/blob/4700e210ef7945278c4eb313c9ebd810b0224da1/jdbc/src/java/org/apache/hive/jdbc/Utils.java#L72 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22974) Metastore's table location check should be applied when location changed
[ https://issues.apache.org/jira/browse/HIVE-22974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057955#comment-17057955 ] Naveen Gangam commented on HIVE-22974: -- +1. Looks good to me. Thank you [~amagyar] > Metastore's table location check should be applied when location changed > > > Key: HIVE-22974 > URL: https://issues.apache.org/jira/browse/HIVE-22974 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Reporter: Attila Magyar >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22974.2.patch, HIVE-22974.3.patch, > HIVE-22974.4.patch > > > In HIVE-22189 a check was introduced to make sure managed and external tables > are located at the proper space. This condition cannot be satisfied during an > upgrade. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22853) Allow JDBC FetchSize to Be Set in Beeline
[
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057948#comment-17057948
]
Hive QA commented on HIVE-22853:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} beeline in master has 48 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21083/dev-support/hive-personality.sh
|
| git revision | master / 812a626 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21083/yetus/patch-asflicense-problems.txt
|
| modules | C: beeline U: beeline |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21083/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Allow JDBC FetchSize to Be Set in Beeline
> -
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch,
> HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.6.patch,
> HIVE-22853.7.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This
> default value is different from what the server has set. While the beeline
> user can reset the value via set command, its cumbersome to change the
> workloads.
> Rather it should default to the server-side value and set should be used to
> override within the session.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23014) ORC reading performance
[
https://issues.apache.org/jira/browse/HIVE-23014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057945#comment-17057945
]
Peter Toth commented on HIVE-23014:
---
I've attached the output of the benchmark runs with the 2 Hive profiles. Based
on those only "Single Column Scans From Wide" benchmark is affected.
The 2 profiles are here:
[https://github.com/apache/spark/blob/master/pom.xml#L3029-L3048] vs
[https://github.com/apache/spark/blob/master/pom.xml#L3050-L3053]. They do some
other changes as well, but I measured this particular call on Hive's ORC
reader:
[https://github.com/apache/spark/blob/master/sql/hive/src/main/java/org/apache/hadoop/hive/ql/io/orc/SparkOrcNewRecordReader.java#L85]
and the performance gap clearly comes from here.
> ORC reading performance
> ---
>
> Key: HIVE-23014
> URL: https://issues.apache.org/jira/browse/HIVE-23014
> Project: Hive
> Issue Type: Bug
> Components: ORC
>Affects Versions: 2.3.6
>Reporter: Peter Toth
>Priority: Major
> Attachments: OrcReadBenchmark-results.txt.hive-1.2.1,
> OrcReadBenchmark-results.txt.hive-2.3.6
>
>
> Spark 3 adds support for using Hive 2.3.6 besides the old Hive 1.2.1 version.
> Some of the ORC reading benchmark shows that there is a huge performance
> difference in ORC reading between the 2 versions. I measured that
> {{org.apache.hadoop.hive.ql.io.orc.ReaderImpl}} in hive-exec-2.3.6-core.jar
> is ~3-5 times slower than in hive-exec-1.2.1.spark2.jar.
> I'm not sure if more recent Hive versions still suffer from this performance
> regression.
> Please see some details here: SPARK-30565
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23007) Cleanup Default Fetch Size If One Is Not Sent By Client
[ https://issues.apache.org/jira/browse/HIVE-23007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23007: -- Description: * Remove superfluous null check * Use a Collection's Singleton Map instead of creating a full-sized HashMap * Code simplification > Cleanup Default Fetch Size If One Is Not Sent By Client > --- > > Key: HIVE-23007 > URL: https://issues.apache.org/jira/browse/HIVE-23007 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23007.1.patch, HIVE-23007.2.patch > > > * Remove superfluous null check > * Use a Collection's Singleton Map instead of creating a full-sized HashMap > * Code simplification -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23007) Cleanup Default Fetch Size If One Is Not Sent By Client
[ https://issues.apache.org/jira/browse/HIVE-23007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23007: -- Summary: Cleanup Default Fetch Size If One Is Not Sent By Client (was: Server Should Return Default Fetch Size If One Is Not Sent By Client) > Cleanup Default Fetch Size If One Is Not Sent By Client > --- > > Key: HIVE-23007 > URL: https://issues.apache.org/jira/browse/HIVE-23007 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23007.1.patch, HIVE-23007.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23014) ORC reading performance
[
https://issues.apache.org/jira/browse/HIVE-23014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Toth updated HIVE-23014:
--
Attachment: OrcReadBenchmark-results.txt.hive-1.2.1
> ORC reading performance
> ---
>
> Key: HIVE-23014
> URL: https://issues.apache.org/jira/browse/HIVE-23014
> Project: Hive
> Issue Type: Bug
> Components: ORC
>Affects Versions: 2.3.6
>Reporter: Peter Toth
>Priority: Major
> Attachments: OrcReadBenchmark-results.txt.hive-1.2.1,
> OrcReadBenchmark-results.txt.hive-2.3.6
>
>
> Spark 3 adds support for using Hive 2.3.6 besides the old Hive 1.2.1 version.
> Some of the ORC reading benchmark shows that there is a huge performance
> difference in ORC reading between the 2 versions. I measured that
> {{org.apache.hadoop.hive.ql.io.orc.ReaderImpl}} in hive-exec-2.3.6-core.jar
> is ~3-5 times slower than in hive-exec-1.2.1.spark2.jar.
> I'm not sure if more recent Hive versions still suffer from this performance
> regression.
> Please see some details here: SPARK-30565
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23014) ORC reading performance
[
https://issues.apache.org/jira/browse/HIVE-23014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Toth updated HIVE-23014:
--
Attachment: OrcReadBenchmark-results.txt.hive-2.3.6
> ORC reading performance
> ---
>
> Key: HIVE-23014
> URL: https://issues.apache.org/jira/browse/HIVE-23014
> Project: Hive
> Issue Type: Bug
> Components: ORC
>Affects Versions: 2.3.6
>Reporter: Peter Toth
>Priority: Major
> Attachments: OrcReadBenchmark-results.txt.hive-1.2.1,
> OrcReadBenchmark-results.txt.hive-2.3.6
>
>
> Spark 3 adds support for using Hive 2.3.6 besides the old Hive 1.2.1 version.
> Some of the ORC reading benchmark shows that there is a huge performance
> difference in ORC reading between the 2 versions. I measured that
> {{org.apache.hadoop.hive.ql.io.orc.ReaderImpl}} in hive-exec-2.3.6-core.jar
> is ~3-5 times slower than in hive-exec-1.2.1.spark2.jar.
> I'm not sure if more recent Hive versions still suffer from this performance
> regression.
> Please see some details here: SPARK-30565
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23013) Fix UnitTestPropertiesParser creation log message
[
https://issues.apache.org/jira/browse/HIVE-23013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057930#comment-17057930
]
Hive QA commented on HIVE-23013:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996422/HIVE-23013.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 18098 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21082/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21082/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21082/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12996422 - PreCommit-HIVE-Build
> Fix UnitTestPropertiesParser creation log message
> -
>
> Key: HIVE-23013
> URL: https://issues.apache.org/jira/browse/HIVE-23013
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23013.01.patch
>
>
> UnitTestPropertiesParser has a bug preventing from logging it's creation info
> appropriately. Instead of string concatenation ( + ) it has a ( , ), thus the
> end of the message is considered as the first argument, making a mess.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22974) Metastore's table location check should be applied when location changed
[ https://issues.apache.org/jira/browse/HIVE-22974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057909#comment-17057909 ] Attila Magyar commented on HIVE-22974: -- [~ngangam], tests finally passed. Please review the change and push it to master if everything is ok. > Metastore's table location check should be applied when location changed > > > Key: HIVE-22974 > URL: https://issues.apache.org/jira/browse/HIVE-22974 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Reporter: Attila Magyar >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22974.2.patch, HIVE-22974.3.patch, > HIVE-22974.4.patch > > > In HIVE-22189 a check was introduced to make sure managed and external tables > are located at the proper space. This condition cannot be satisfied during an > upgrade. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23015) Fix HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED definition
[
https://issues.apache.org/jira/browse/HIVE-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Varga updated HIVE-23015:
---
Status: Patch Available (was: In Progress)
> Fix HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED definition
> ---
>
> Key: HIVE-23015
> URL: https://issues.apache.org/jira/browse/HIVE-23015
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Varga
>Priority: Minor
> Attachments: HIVE-23015.patch
>
>
> The _HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED_ is definied like this:
> {code}
> HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED("hive.vectorized.groupby.complex.types.enabled",
> true,
> "This flag should be set to true to enable group by vectorization\n" +
> "of aggregations that use complex types.\n",
> "For example, AVG uses a complex type (STRUCT) for partial aggregation
> results" +
> "The default value is true."),
> {code}
> Notice that the end of the "of aggregations that use complex types.\n" line
> is a "," instead of "+".
> So this will end up using the following constructor using the second sentence
> as an "altName" :)
> {code}
> ConfVars(String varname, Object defaultVal, String description, String
> altName) {
> this(varname, defaultVal, null, description, true, false, altName);
> }
> {code}
> We should fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work started] (HIVE-23015) Fix HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED definition
[
https://issues.apache.org/jira/browse/HIVE-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-23015 started by Peter Varga.
--
> Fix HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED definition
> ---
>
> Key: HIVE-23015
> URL: https://issues.apache.org/jira/browse/HIVE-23015
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Varga
>Priority: Minor
> Attachments: HIVE-23015.patch
>
>
> The _HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED_ is definied like this:
> {code}
> HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED("hive.vectorized.groupby.complex.types.enabled",
> true,
> "This flag should be set to true to enable group by vectorization\n" +
> "of aggregations that use complex types.\n",
> "For example, AVG uses a complex type (STRUCT) for partial aggregation
> results" +
> "The default value is true."),
> {code}
> Notice that the end of the "of aggregations that use complex types.\n" line
> is a "," instead of "+".
> So this will end up using the following constructor using the second sentence
> as an "altName" :)
> {code}
> ConfVars(String varname, Object defaultVal, String description, String
> altName) {
> this(varname, defaultVal, null, description, true, false, altName);
> }
> {code}
> We should fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23015) Fix HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED definition
[
https://issues.apache.org/jira/browse/HIVE-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Varga updated HIVE-23015:
---
Attachment: HIVE-23015.patch
> Fix HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED definition
> ---
>
> Key: HIVE-23015
> URL: https://issues.apache.org/jira/browse/HIVE-23015
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Varga
>Priority: Minor
> Attachments: HIVE-23015.patch
>
>
> The _HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED_ is definied like this:
> {code}
> HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED("hive.vectorized.groupby.complex.types.enabled",
> true,
> "This flag should be set to true to enable group by vectorization\n" +
> "of aggregations that use complex types.\n",
> "For example, AVG uses a complex type (STRUCT) for partial aggregation
> results" +
> "The default value is true."),
> {code}
> Notice that the end of the "of aggregations that use complex types.\n" line
> is a "," instead of "+".
> So this will end up using the following constructor using the second sentence
> as an "altName" :)
> {code}
> ConfVars(String varname, Object defaultVal, String description, String
> altName) {
> this(varname, defaultVal, null, description, true, false, altName);
> }
> {code}
> We should fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23013) Fix UnitTestPropertiesParser creation log message
[
https://issues.apache.org/jira/browse/HIVE-23013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057884#comment-17057884
]
Hive QA commented on HIVE-23013:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
22s{color} | {color:blue} testutils/ptest2 in master has 24 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 5 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21082/dev-support/hive-personality.sh
|
| git revision | master / 812a626 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21082/yetus/patch-asflicense-problems.txt
|
| modules | C: testutils/ptest2 U: testutils/ptest2 |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21082/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix UnitTestPropertiesParser creation log message
> -
>
> Key: HIVE-23013
> URL: https://issues.apache.org/jira/browse/HIVE-23013
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23013.01.patch
>
>
> UnitTestPropertiesParser has a bug preventing from logging it's creation info
> appropriately. Instead of string concatenation ( + ) it has a ( , ), thus the
> end of the message is considered as the first argument, making a mess.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-23014) ORC reading performance
[
https://issues.apache.org/jira/browse/HIVE-23014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057875#comment-17057875
]
Panagiotis Garefalakis edited comment on HIVE-23014 at 3/12/20, 12:33 PM:
--
[~petertoth] thanks for reporting this.
It seems that the particular regression is observed on Single Column Scans
>From Wide Columns:
[https://github.com/apache/spark/blob/f5118f81e395bde0cd8253dbef6a9e6455c3958a/sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala#L267]
Did you observe similar ReaderImpl behaviour in other cases? The only change
was Hive version from 1.2.1 to 2.3.6 ?
was (Author: pgaref):
[~petertoth] thanks for reporting this.
It seems that the particular regression is observed on Single Column Scans
>From Wide Columns:
[https://github.com/apache/spark/blob/f5118f81e395bde0cd8253dbef6a9e6455c3958a/sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala#L267]
Did you observe similar ReaderImpl behaviour in other cases?
> ORC reading performance
> ---
>
> Key: HIVE-23014
> URL: https://issues.apache.org/jira/browse/HIVE-23014
> Project: Hive
> Issue Type: Bug
> Components: ORC
>Affects Versions: 2.3.6
>Reporter: Peter Toth
>Priority: Major
>
> Spark 3 adds support for using Hive 2.3.6 besides the old Hive 1.2.1 version.
> Some of the ORC reading benchmark shows that there is a huge performance
> difference in ORC reading between the 2 versions. I measured that
> {{org.apache.hadoop.hive.ql.io.orc.ReaderImpl}} in hive-exec-2.3.6-core.jar
> is ~3-5 times slower than in hive-exec-1.2.1.spark2.jar.
> I'm not sure if more recent Hive versions still suffer from this performance
> regression.
> Please see some details here: SPARK-30565
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23014) ORC reading performance
[
https://issues.apache.org/jira/browse/HIVE-23014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057875#comment-17057875
]
Panagiotis Garefalakis commented on HIVE-23014:
---
[~petertoth] thanks for reporting this.
It seems that the particular regression is observed on Single Column Scans
>From Wide Columns:
[https://github.com/apache/spark/blob/f5118f81e395bde0cd8253dbef6a9e6455c3958a/sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala#L267]
Did you observe similar ReaderImpl behaviour in other cases?
> ORC reading performance
> ---
>
> Key: HIVE-23014
> URL: https://issues.apache.org/jira/browse/HIVE-23014
> Project: Hive
> Issue Type: Bug
> Components: ORC
>Affects Versions: 2.3.6
>Reporter: Peter Toth
>Priority: Major
>
> Spark 3 adds support for using Hive 2.3.6 besides the old Hive 1.2.1 version.
> Some of the ORC reading benchmark shows that there is a huge performance
> difference in ORC reading between the 2 versions. I measured that
> {{org.apache.hadoop.hive.ql.io.orc.ReaderImpl}} in hive-exec-2.3.6-core.jar
> is ~3-5 times slower than in hive-exec-1.2.1.spark2.jar.
> I'm not sure if more recent Hive versions still suffer from this performance
> regression.
> Please see some details here: SPARK-30565
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22974) Metastore's table location check should be applied when location changed
[
https://issues.apache.org/jira/browse/HIVE-22974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057868#comment-17057868
]
Hive QA commented on HIVE-22974:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996423/HIVE-22974.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 18099 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21081/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21081/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21081/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12996423 - PreCommit-HIVE-Build
> Metastore's table location check should be applied when location changed
>
>
> Key: HIVE-22974
> URL: https://issues.apache.org/jira/browse/HIVE-22974
> Project: Hive
> Issue Type: Sub-task
> Components: Metastore
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22974.2.patch, HIVE-22974.3.patch,
> HIVE-22974.4.patch
>
>
> In HIVE-22189 a check was introduced to make sure managed and external tables
> are located at the proper space. This condition cannot be satisfied during an
> upgrade.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[ https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22990: --- Attachment: HIVE-22990.06.patch Status: Patch Available (was: In Progress) > Build acknowledgement mechanism for repl dump and load > -- > > Key: HIVE-22990 > URL: https://issues.apache.org/jira/browse/HIVE-22990 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch, > HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch, > HIVE-22990.06.patch, HIVE-22990.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[ https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22990: --- Status: In Progress (was: Patch Available) > Build acknowledgement mechanism for repl dump and load > -- > > Key: HIVE-22990 > URL: https://issues.apache.org/jira/browse/HIVE-22990 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch, > HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch, > HIVE-22990.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22974) Metastore's table location check should be applied when location changed
[
https://issues.apache.org/jira/browse/HIVE-22974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057819#comment-17057819
]
Hive QA commented on HIVE-22974:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
21s{color} | {color:blue} standalone-metastore/metastore-server in master has
185 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
22s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 2 new + 375 unchanged - 0 fixed = 377 total (was 375) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} itests/hive-unit: The patch generated 1 new + 190
unchanged - 0 fixed = 191 total (was 190) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21081/dev-support/hive-personality.sh
|
| git revision | master / 812a626 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21081/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21081/yetus/diff-checkstyle-itests_hive-unit.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21081/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore/metastore-server itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21081/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Metastore's table location check should be applied when location changed
>
>
> Key: HIVE-22974
> URL: https://issues.apache.org/jira/browse/HIVE-22974
> Project: Hive
> Issue Type: Sub-task
> Components: Metastore
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22974.2.patch,
[jira] [Assigned] (HIVE-23015) Fix HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED definition
[
https://issues.apache.org/jira/browse/HIVE-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Varga reassigned HIVE-23015:
--
Assignee: Peter Varga
> Fix HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED definition
> ---
>
> Key: HIVE-23015
> URL: https://issues.apache.org/jira/browse/HIVE-23015
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Varga
>Priority: Minor
>
> The _HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED_ is definied like this:
> {code}
> HIVE_VECTORIZATION_GROUPBY_COMPLEX_TYPES_ENABLED("hive.vectorized.groupby.complex.types.enabled",
> true,
> "This flag should be set to true to enable group by vectorization\n" +
> "of aggregations that use complex types.\n",
> "For example, AVG uses a complex type (STRUCT) for partial aggregation
> results" +
> "The default value is true."),
> {code}
> Notice that the end of the "of aggregations that use complex types.\n" line
> is a "," instead of "+".
> So this will end up using the following constructor using the second sentence
> as an "altName" :)
> {code}
> ConfVars(String varname, Object defaultVal, String description, String
> altName) {
> this(varname, defaultVal, null, description, true, false, altName);
> }
> {code}
> We should fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23005) Consider Default JDBC Fetch Size From HS2
[
https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057795#comment-17057795
]
Hive QA commented on HIVE-23005:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996427/HIVE-23005.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 18098 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestHiveStatement.testSetFetchSize1 (batchId=313)
org.apache.hive.jdbc.TestHiveStatement.testSetFetchSize2 (batchId=313)
org.apache.hive.jdbc.TestHiveStatement.testSetFetchSize3 (batchId=313)
org.apache.hive.jdbc.TestHiveStatement.testaddBatch (batchId=313)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testFetchSize (batchId=293)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21080/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21080/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21080/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12996427 - PreCommit-HIVE-Build
> Consider Default JDBC Fetch Size From HS2
> -
>
> Key: HIVE-23005
> URL: https://issues.apache.org/jira/browse/HIVE-23005
> Project: Hive
> Issue Type: Sub-task
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23005.1.patch, HIVE-23005.2.patch,
> HIVE-23005.3.patch
>
>
> # Create two fetch sizes: and "init" fetch size and a "default" fetch size.
> The "init" fetch size comes from the JDBC connection string _fetchSize_ (if
> present) and the "default" fetch size comes from
> _HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
> open session request.
> # When a {{Statement}} is created, its starting fetch size is the "init"
> fetch size (may be 0)
> # Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
> size to be the server default
> Setting to zero defaults to the server's instructed default and adheres to:
> bq. If the value specified is zero, then the hint is ignored. The default
> value is zero.
> That is to say, if the fetch size is 0, the default from the server is used,
> otherwise the user can pass a 'hint' and that will be the number of rows
> fetched instead.
> https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23008) UDAFExampleMaxMinNUtil.sortedMerge must be able to handle all inputs
[ https://issues.apache.org/jira/browse/HIVE-23008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057789#comment-17057789 ] Miklos Gergely commented on HIVE-23008: --- +1 > UDAFExampleMaxMinNUtil.sortedMerge must be able to handle all inputs > > > Key: HIVE-23008 > URL: https://issues.apache.org/jira/browse/HIVE-23008 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-23008.01.patch > > > in case both inputs contain the same input values; the merge may stop with an > infinite loop > TestContribCliDriver may trigger this -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23008) UDAFExampleMaxMinNUtil.sortedMerge must be able to handle all inputs
[ https://issues.apache.org/jira/browse/HIVE-23008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057781#comment-17057781 ] Zoltan Haindrich commented on HIVE-23008: - [~mgergely] could you please take a look? > UDAFExampleMaxMinNUtil.sortedMerge must be able to handle all inputs > > > Key: HIVE-23008 > URL: https://issues.apache.org/jira/browse/HIVE-23008 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-23008.01.patch > > > in case both inputs contain the same input values; the merge may stop with an > infinite loop > TestContribCliDriver may trigger this -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22785) Update/delete/merge statements not optimized through CBO
[
https://issues.apache.org/jira/browse/HIVE-22785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22785:
--
Status: Patch Available (was: Open)
> Update/delete/merge statements not optimized through CBO
>
>
> Key: HIVE-22785
> URL: https://issues.apache.org/jira/browse/HIVE-22785
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Critical
> Attachments: HIVE-22785.1.patch, HIVE-22785.2.patch,
> HIVE-22785.2.patch, HIVE-22785.3.patch, HIVE-22785.4.patch,
> HIVE-22785.5.patch, HIVE-22785.6.patch
>
>
> Currently, CBO is bypassed for update/delete/merge statements.
> To support optimizing these statements through CBO, we need to complete three
> main tasks: 1) support for sort in Calcite planner, 2) support for SORT in
> AST converter, and 3) {{RewriteSemanticAnalyzer}} should extend
> {{CalcitePlanner}} instead of {{SemanticAnalyzer}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
