[jira] [Commented] (HIVE-22774) Add toggle to disable scheduled query service in HMS
[
https://issues.apache.org/jira/browse/HIVE-22774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023447#comment-17023447
]
Hive QA commented on HIVE-22774:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
8s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
33s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
20s{color} | {color:blue} standalone-metastore/metastore-common in master has
37 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
30s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
7s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
1s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
40s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
31s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} standalone-metastore/metastore-common: The patch
generated 2 new + 84 unchanged - 0 fixed = 86 total (was 84) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 1 new + 14 unchanged - 0 fixed
= 15 total (was 14) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 8m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20334/dev-support/hive-personality.sh
|
| git revision | master / 037eace |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20334/yetus/diff-checkstyle-standalone-metastore_metastore-common.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20334/yetus/diff-checkstyle-ql.txt
|
| modules | C: standalone-metastore/metastore-common common
standalone-metastore/metastore-server ql itests/hcatalog-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20334/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add toggle to disable scheduled query service in HMS
>
>
> Key: HIVE-22774
> URL: https://issues.apache.org/jira/browse/HIVE-22774
> Project: Hive
> Issue
[jira] [Updated] (HIVE-22779) Hive build fails with Spark 3
[ https://issues.apache.org/jira/browse/HIVE-22779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated HIVE-22779: - Attachment: HIVE-22779.patch Status: Patch Available (was: Open) > Hive build fails with Spark 3 > - > > Key: HIVE-22779 > URL: https://issues.apache.org/jira/browse/HIVE-22779 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 3.1.2 >Reporter: Igor Dvorzhak >Priority: Major > Attachments: HIVE-22779.patch > > > Hive build with Spark 3 fails because of deletion of few deprecated APIs in > Spark 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22779) Hive build fails with Spark 3
[ https://issues.apache.org/jira/browse/HIVE-22779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated HIVE-22779: - Attachment: (was: HIVE-22779.patch) > Hive build fails with Spark 3 > - > > Key: HIVE-22779 > URL: https://issues.apache.org/jira/browse/HIVE-22779 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 3.1.2 >Reporter: Igor Dvorzhak >Priority: Major > > Hive build with Spark 3 fails because of deletion of few deprecated APIs in > Spark 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22779) Hive build fails with Spark 3
[ https://issues.apache.org/jira/browse/HIVE-22779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated HIVE-22779: - Status: Open (was: Patch Available) > Hive build fails with Spark 3 > - > > Key: HIVE-22779 > URL: https://issues.apache.org/jira/browse/HIVE-22779 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 3.1.2 >Reporter: Igor Dvorzhak >Priority: Major > > Hive build with Spark 3 fails because of deletion of few deprecated APIs in > Spark 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22775) Use the qt:authorizer option in qtests
[
https://issues.apache.org/jira/browse/HIVE-22775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023437#comment-17023437
]
Hive QA commented on HIVE-22775:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991787/HIVE-22775.01.patch
{color:green}SUCCESS:{color} +1 due to 86 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 17946 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_1_sql_std]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_alter_table_exchange_partition]
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_create_temp_table]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_with_auth]
(batchId=92)
org.apache.hive.spark.client.rpc.TestRpc.testClientTimeout (batchId=362)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20333/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20333/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20333/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991787 - PreCommit-HIVE-Build
> Use the qt:authorizer option in qtests
> --
>
> Key: HIVE-22775
> URL: https://issues.apache.org/jira/browse/HIVE-22775
> Project: Hive
> Issue Type: Improvement
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22775.01.patch
>
>
> qt:authorizer sets up things for auth tests; [~mgergely] suggested to use it
> in existing tests
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22775) Use the qt:authorizer option in qtests
[
https://issues.apache.org/jira/browse/HIVE-22775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023436#comment-17023436
]
Hive QA commented on HIVE-22775:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
10s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
11s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
25s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 66m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense xml javac javadoc findbugs checkstyle
compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20333/dev-support/hive-personality.sh
|
| git revision | master / 037eace |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql . itests/util U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20333/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Use the qt:authorizer option in qtests
> --
>
> Key: HIVE-22775
> URL: https://issues.apache.org/jira/browse/HIVE-22775
> Project: Hive
> Issue Type: Improvement
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22775.01.patch
>
>
> qt:authorizer sets up things for auth tests; [~mgergely] suggested to use it
> in existing tests
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023426#comment-17023426
]
Shivam edited comment on HIVE-22771 at 1/25/20 6:17 AM:
Patch to fix regular expression to de-scratchify the final partition location:
HIVE-22771
was (Author: shivam-mohan):
Patch to fix regular expression to de-scratchify the final partition locatio:
HIVE-22771
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> +In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And,
> in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#FF}\\d\\.?\\d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shivam updated HIVE-22771:
--
Status: Patch Available (was: Open)
Patch to fix regular expression to de-scratchify the final partition locatio:
HIVE-22771
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> +In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And,
> in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#FF}\\d\\.?\\d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22723) SHOW CREATE TABLE omits backtics, results in syntax error
[
https://issues.apache.org/jira/browse/HIVE-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023411#comment-17023411
]
Hive QA commented on HIVE-22723:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991775/HIVE-22723.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17946 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20332/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20332/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20332/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991775 - PreCommit-HIVE-Build
> SHOW CREATE TABLE omits backtics, results in syntax error
> -
>
> Key: HIVE-22723
> URL: https://issues.apache.org/jira/browse/HIVE-22723
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Environment: Run SHOW CREATE TABLE via AWS Athena
>Reporter: Alex Yankelevich
>Assignee: Miklos Gergely
>Priority: Minor
> Attachments: HIVE-22723.01.patch, HIVE-22723.02.patch
>
>
> {code}
> CREATE EXTERNAL TABLE `alerts1`(
> `_id` string COMMENT 'from deserializer', `org` string COMMENT 'from
> deserializer', `collector` string COMMENT 'from deserializer', `source`
> string COMMENT 'from deserializer', `entitytype` string COMMENT 'from
> deserializer', `entity` string COMMENT 'from deserializer', `rule` string
> COMMENT 'from deserializer', `findingdescid` string COMMENT 'from
> deserializer',
> `details`
> struct>,findingtype:string,producttype:string,entityname:string,extra:struct,entityparams:struct,version:string,build:string>>
> COMMENT 'from deserializer', `seenat` array>
> COMMENT 'from deserializer', `execution` array>
> COMMENT 'from deserializer', `lastmodified` struct<__numberint:string>
> COMMENT 'from deserializer'
> ) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' WITH SERDEPROPERTIES
> (
> 'paths'='_id,collector,details,entity,entityType,execution,findingDescId,lastModified,org,rule,seenAt,source')
> STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat'
> OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
> LOCATION 's3://redshift-poc-ay/alerts' TBLPROPERTIES (
> 'CrawlerSchemaDeserializerVersion'='1.0',
> 'CrawlerSchemaSerializerVersion'='1.0',
> 'UPDATED_BY_CRAWLER'='redshift-poc-alerts-ay', 'averageRecordSize'='9619',
> 'classification'='json', 'compressionType'='none', 'objectCount'='1',
> 'recordCount'='21028', 'sizeKey'='202270843',
> 'transient_lastDdlTime'='1578946186', 'typeOfData'='file')
> {code}
> __numberint, __numberlong columns are not generated enclosed in backticks
> even though they were explicitly created with them. Therefore, the generated
> DDL is syntactically incorrect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22779) Hive build fails with Spark 3
[ https://issues.apache.org/jira/browse/HIVE-22779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated HIVE-22779: - Attachment: (was: HIVE-22779.patch) > Hive build fails with Spark 3 > - > > Key: HIVE-22779 > URL: https://issues.apache.org/jira/browse/HIVE-22779 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 3.1.2 >Reporter: Igor Dvorzhak >Priority: Major > Attachments: HIVE-22779.patch > > > Hive build with Spark 3 fails because of deletion of few deprecated APIs in > Spark 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22779) Hive build fails with Spark 3
[ https://issues.apache.org/jira/browse/HIVE-22779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated HIVE-22779: - Attachment: HIVE-22779.patch > Hive build fails with Spark 3 > - > > Key: HIVE-22779 > URL: https://issues.apache.org/jira/browse/HIVE-22779 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 3.1.2 >Reporter: Igor Dvorzhak >Priority: Major > Attachments: HIVE-22779.patch > > > Hive build with Spark 3 fails because of deletion of few deprecated APIs in > Spark 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22779) Hive build fails with Spark 3
[ https://issues.apache.org/jira/browse/HIVE-22779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated HIVE-22779: - Attachment: HIVE-22779.patch Status: Patch Available (was: Open) > Hive build fails with Spark 3 > - > > Key: HIVE-22779 > URL: https://issues.apache.org/jira/browse/HIVE-22779 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 3.1.2 >Reporter: Igor Dvorzhak >Priority: Major > Attachments: HIVE-22779.patch > > > Hive build with Spark 3 fails because of deletion of few deprecated APIs in > Spark 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22779) Hive build fails with Spark 3
[ https://issues.apache.org/jira/browse/HIVE-22779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated HIVE-22779: - Description: Hive build with Spark 3 fails because of deletion of few deprecated APIs in Spark 3. (was: Because of deletion of few deprecated APIs in Spark 3 Hive build fails.) > Hive build fails with Spark 3 > - > > Key: HIVE-22779 > URL: https://issues.apache.org/jira/browse/HIVE-22779 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 3.1.2 >Reporter: Igor Dvorzhak >Priority: Major > > Hive build with Spark 3 fails because of deletion of few deprecated APIs in > Spark 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22723) SHOW CREATE TABLE omits backtics, results in syntax error
[
https://issues.apache.org/jira/browse/HIVE-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023408#comment-17023408
]
Hive QA commented on HIVE-22723:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
4s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20332/dev-support/hive-personality.sh
|
| git revision | master / 037eace |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20332/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20332/yetus/whitespace-eol.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20332/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> SHOW CREATE TABLE omits backtics, results in syntax error
> -
>
> Key: HIVE-22723
> URL: https://issues.apache.org/jira/browse/HIVE-22723
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Environment: Run SHOW CREATE TABLE via AWS Athena
>Reporter: Alex Yankelevich
>Assignee: Miklos Gergely
>Priority: Minor
> Attachments: HIVE-22723.01.patch, HIVE-22723.02.patch
>
>
> {code}
> CREATE EXTERNAL TABLE `alerts1`(
> `_id` string COMMENT 'from deserializer', `org` string COMMENT 'from
> deserializer', `collector` string COMMENT 'from deserializer', `source`
> string COMMENT 'from deserializer', `entitytype` string COMMENT 'from
> deserializer', `entity` string COMMENT 'from deserializer', `rule` string
> COMMENT 'from deserializer', `findingdescid` string COMMENT 'from
> deserializer',
> `details`
> struct>,findingtype:string,producttype:string,entityname:string,extra:struct,entityparams:struct,version:string,build:string>>
> COMMENT 'from deserializer', `seenat` array>
> COMMENT 'from deserializer', `execution` array>
> COMMENT 'from deserializer', `lastmodified` struct<__numberint:string>
> COMMENT 'from deserializer'
> ) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' WITH
[jira] [Commented] (HIVE-10362) Support Type check/conversion in dynamic partition column
[
https://issues.apache.org/jira/browse/HIVE-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023407#comment-17023407
]
Hive QA commented on HIVE-10362:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991772/HIVE-10362.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 235 failed/errored test(s), 17946 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_dynamic_partitions]
(batchId=306)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions]
(batchId=306)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_insert_overwrite]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats2] (batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_subquery]
(batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_directories_test]
(batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alterColumnStatsPart]
(batchId=99)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_coltype]
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_cascade]
(batchId=101)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_partition_drop]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_6]
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[combine2] (batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_transactional_full_acid]
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas] (batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[drop_partitions_filter4]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynamic_partition_insert]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynamic_partition_skip_default]
(batchId=92)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynpart_sort_opt_bucketing]
(batchId=99)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynpart_sort_optimization_acid2]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explain_locks]
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[filter_numeric]
(batchId=91)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[implicit_cast_during_insert]
(batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_bucket_sort_dyn_part]
(batchId=42)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_bucket_sort_num_buckets]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert0] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_acid_dynamic_partition]
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_into6]
(batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_into_with_schema2]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_values_dynamic_partitioned]
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_dml_6]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_dml_7]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_dml_8]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[load_dyn_part12]
(batchId=94)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[load_dyn_part13]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[lock3] (batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[lock4] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge4] (batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_all] (batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_buckets] (batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[multi_insert_partitioned]
(batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge10] (batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge1] (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge2] (batchId=103)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_diff_fs]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_incompat2]
(batchId=96)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_decode_name]
(batchId=92)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_discovery]
(batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_params_postgres]
(batchId=94)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_special_char]
(batchId=50)
[jira] [Commented] (HIVE-10362) Support Type check/conversion in dynamic partition column
[
https://issues.apache.org/jira/browse/HIVE-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023400#comment-17023400
]
Hive QA commented on HIVE-10362:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
58s{color} | {color:red} ql generated 4 new + 1533 unchanged - 0 fixed = 1537
total (was 1533) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 36s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Null pointer dereference of destinationTable in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(String, QB,
Operator) Dereferenced at SemanticAnalyzer.java:in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(String, QB,
Operator) Dereferenced at SemanticAnalyzer.java:[line 7679] |
| | Null pointer dereference of dpCtx in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(String, QB,
Operator) Dereferenced at SemanticAnalyzer.java:in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(String, QB,
Operator) Dereferenced at SemanticAnalyzer.java:[line 7812] |
| | Null pointer dereference of dpCtx in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(String, QB,
Operator) Dereferenced at SemanticAnalyzer.java:in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(String, QB,
Operator) Dereferenced at SemanticAnalyzer.java:[line 7817] |
| | Load of known null value in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(String, QB,
Operator) At SemanticAnalyzer.java:in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(String, QB,
Operator) At SemanticAnalyzer.java:[line 7801] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20331/dev-support/hive-personality.sh
|
| git revision | master / 037eace |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20331/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20331/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20331/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support Type check/conversion in dynamic partition column
>
[jira] [Commented] (HIVE-22772) Log opType and session level information for each operation
[
https://issues.apache.org/jira/browse/HIVE-22772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023394#comment-17023394
]
Hive QA commented on HIVE-22772:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991770/HIVE-22772.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20330/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20330/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20330/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12991770/HIVE-22772.1.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991770 - PreCommit-HIVE-Build
> Log opType and session level information for each operation
>
>
> Key: HIVE-22772
> URL: https://issues.apache.org/jira/browse/HIVE-22772
> Project: Hive
> Issue Type: Improvement
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22772.1.patch
>
>
> if would be useful to have this info available for issue troubleshooting.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22518) SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
[
https://issues.apache.org/jira/browse/HIVE-22518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023393#comment-17023393
]
Hive QA commented on HIVE-22518:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991754/HIVE-22518.06.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17946 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20329/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20329/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20329/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991754 - PreCommit-HIVE-Build
> SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
>
>
> Key: HIVE-22518
> URL: https://issues.apache.org/jira/browse/HIVE-22518
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22518.01.patch, HIVE-22518.01.patch,
> HIVE-22518.02.patch, HIVE-22518.02.patch, HIVE-22518.03.patch,
> HIVE-22518.03.patch, HIVE-22518.04.patch, HIVE-22518.05.patch,
> HIVE-22518.05.patch, HIVE-22518.06.patch, HIVE-22518.06.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22518) SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
[
https://issues.apache.org/jira/browse/HIVE-22518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023388#comment-17023388
]
Hive QA commented on HIVE-22518:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
54s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
3s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
43s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
48s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
34s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 67m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense xml javac javadoc findbugs checkstyle
compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20329/dev-support/hive-personality.sh
|
| git revision | master / 037eace |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql . itests/util U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20329/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
>
>
> Key: HIVE-22518
> URL: https://issues.apache.org/jira/browse/HIVE-22518
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22518.01.patch, HIVE-22518.01.patch,
> HIVE-22518.02.patch, HIVE-22518.02.patch, HIVE-22518.03.patch,
> HIVE-22518.03.patch, HIVE-22518.04.patch, HIVE-22518.05.patch,
> HIVE-22518.05.patch, HIVE-22518.06.patch, HIVE-22518.06.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22746) Make TypeCheckProcFactory generic
[
https://issues.apache.org/jira/browse/HIVE-22746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023380#comment-17023380
]
Vineet Garg commented on HIVE-22746:
LGTM +1 pending review comments
> Make TypeCheckProcFactory generic
> -
>
> Key: HIVE-22746
> URL: https://issues.apache.org/jira/browse/HIVE-22746
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22746.01.patch, HIVE-22746.01.patch,
> HIVE-22746.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {{TypeCheckProcFactory}} is responsible for processing AST nodes and
> generating ExprNode objects from them. When we generate the expressions for
> Calcite planning, we go through a {{AST node -> ExprNode -> RexNode}}
> transformation. We would like to avoid the overhead of going through the
> ExprNode, and thus generate directly the RexNode from the AST.
> To do that, the first step is to make {{TypeCheckProcFactory}} generic, so it
> can receive an expression factory and create expressions in different realms.
> For the time being, the only factory implementation is the ExprNode factory.
> Thus, this patch focuses mainly on refactoring {{TypeCheckProcFactory}}
> without breaking anything that is already working.
> In a follow-up patch, we will create a {{RexNode}} factory and use it when we
> parse the query in CalcitePlanner.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22765) Quote table names in CompactionTxnHandler
[
https://issues.apache.org/jira/browse/HIVE-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023368#comment-17023368
]
Hive QA commented on HIVE-22765:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991746/HIVE-22765.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17946 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.service.server.TestInformationSchemaWithPrivilege.test
(batchId=286)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20328/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20328/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20328/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991746 - PreCommit-HIVE-Build
> Quote table names in CompactionTxnHandler
> -
>
> Key: HIVE-22765
> URL: https://issues.apache.org/jira/browse/HIVE-22765
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Laszlo Pinter
>Assignee: Zoltan Chovan
>Priority: Major
> Attachments: HIVE-22765.patch
>
>
> When I run a metastore backed up by a postgres db, I get the following
> exception:
> {code:java}
> MetaException(message:Unable to connect to transaction database
> org.postgresql.util.PSQLException: ERROR: relation "compaction_queue" does
> not existMetaException(message:Unable to connect to transaction database
> org.postgresql.util.PSQLException: ERROR: relation "compaction_queue" does
> not exist Position: 83 at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2468)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2211)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:309)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:446) at
> org.postgresql.jdbc.PgStatement.execute(PgStatement.java:370) at
> org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:311) at
> org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:297) at
> org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:274) at
> org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:225) at
> org.apache.hive.com.zaxxer.hikari.pool.ProxyStatement.executeQuery(ProxyStatement.java:108)
> at
> org.apache.hive.com.zaxxer.hikari.pool.HikariProxyStatement.executeQuery(HikariProxyStatement.java)
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findNextToCompact(CompactionTxnHandler.java:157)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.find_next_compact(HiveMetaStore.java:8061)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy28.find_next_compact(Unknown Source) at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.findNextCompact(HiveMetaStoreClient.java:4046)
> at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:208)
> at com.sun.proxy.$Proxy29.findNextCompact(Unknown Source) at
> org.apache.hadoop.hive.ql.txn.compactor.Worker.run(Worker.java:102){code}
> This is due to missing quotes around table names.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22765) Quote table names in CompactionTxnHandler
[
https://issues.apache.org/jira/browse/HIVE-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023344#comment-17023344
]
Hive QA commented on HIVE-22765:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
18s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 24 new + 55 unchanged - 25 fixed = 79 total (was 80) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20328/dev-support/hive-personality.sh
|
| git revision | master / 037eace |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20328/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20328/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Quote table names in CompactionTxnHandler
> -
>
> Key: HIVE-22765
> URL: https://issues.apache.org/jira/browse/HIVE-22765
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Laszlo Pinter
>Assignee: Zoltan Chovan
>Priority: Major
> Attachments: HIVE-22765.patch
>
>
> When I run a metastore backed up by a postgres db, I get the following
> exception:
> {code:java}
> MetaException(message:Unable to connect to transaction database
> org.postgresql.util.PSQLException: ERROR: relation "compaction_queue" does
> not existMetaException(message:Unable to connect to transaction database
> org.postgresql.util.PSQLException: ERROR: relation "compaction_queue" does
> not exist Position: 83 at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2468)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2211)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:309)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:446) at
> org.postgresql.jdbc.PgStatement.execute(PgStatement.java:370) at
> org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:311) at
>
[jira] [Commented] (HIVE-22772) Log opType and session level information for each operation
[
https://issues.apache.org/jira/browse/HIVE-22772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023335#comment-17023335
]
Hive QA commented on HIVE-22772:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991770/HIVE-22772.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17946 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20327/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20327/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20327/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991770 - PreCommit-HIVE-Build
> Log opType and session level information for each operation
>
>
> Key: HIVE-22772
> URL: https://issues.apache.org/jira/browse/HIVE-22772
> Project: Hive
> Issue Type: Improvement
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22772.1.patch
>
>
> if would be useful to have this info available for issue troubleshooting.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22366) Multiple metastore calls for same table and constraints during planning
[
https://issues.apache.org/jira/browse/HIVE-22366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023323#comment-17023323
]
Vineet Garg commented on HIVE-22366:
[~jcamachorodriguez] Can you take a look?
> Multiple metastore calls for same table and constraints during planning
> ---
>
> Key: HIVE-22366
> URL: https://issues.apache.org/jira/browse/HIVE-22366
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22366.1.patch, HIVE-22366.2.patch,
> HIVE-22366.3.patch, HIVE-22366.4.patch, HIVE-22366.5.patch,
> HIVE-22366.6.patch, HIVE-22366.7.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> e.g. for a simple query like
> {code:sql}
> explain select count(i) from test2 group by j limit 3;
> {code}
> planner makes the following calls:
> {code}
> 2019-10-17T22:44:49,892 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:49,908 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:49,910 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:49,911 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,913 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,979 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_partitions : tbl=hive.default.test2
> 2019-10-17T22:44:49,997 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_aggr_stats_for: table=hive.default.test2
> 2019-10-17T22:44:50,014 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:50,023 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:50,024 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:50,025 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:50,026 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22653) Remove commons-lang leftovers
[ https://issues.apache.org/jira/browse/HIVE-22653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22653: Resolution: Fixed Status: Resolved (was: Patch Available) I've run mvn install under 'testutils/ptest2' as well :) pushed to master (second try) > Remove commons-lang leftovers > - > > Key: HIVE-22653 > URL: https://issues.apache.org/jira/browse/HIVE-22653 > Project: Hive > Issue Type: Bug >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22653.01.patch, HIVE-22653.01.patch, > HIVE-22653.02.patch, HIVE-22653.03.patch, HIVE-22653.04.patch, > HIVE-22653.04.patch, HIVE-22653.04.patch, HIVE-22653.05.patch, > HIVE-22653.06.patch, HIVE-22653.07.patch, HIVE-22653.07.patch, > HIVE-22653.08.patch, HIVE-22653.08.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > HIVE-7145 removed commons-lang - in favor of commons-lang3 - as a direct > dependency, however a high number of files still refer to commons-lang, which > is transitively brought in either way. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22774) Add toggle to disable scheduled query service in HMS
[ https://issues.apache.org/jira/browse/HIVE-22774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22774: Attachment: HIVE-22774.03.patch > Add toggle to disable scheduled query service in HMS > > > Key: HIVE-22774 > URL: https://issues.apache.org/jira/browse/HIVE-22774 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22774.01.patch, HIVE-22774.02.patch, > HIVE-22774.03.patch > > > * currently it's enabled by default -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22777) Sorted dynamic partition optimization doesn't work if plan require implicit cast
[
https://issues.apache.org/jira/browse/HIVE-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22777:
---
Status: Patch Available (was: Open)
> Sorted dynamic partition optimization doesn't work if plan require implicit
> cast
>
>
> Key: HIVE-22777
> URL: https://issues.apache.org/jira/browse/HIVE-22777
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22777.1.patch
>
>
> *Repro*
> set hive.stats.autogather=false;
> set hive.optimize.sort.dynamic.partition.threshold=1;
> set hive.optimize.bucketingsorting = true;
> {code:sql}
> drop table if exists t1_staging;
> create table t1_staging(
>a string,
>b int,
>c int,
>d string)
> partitioned by (e decimal(18,0))
> clustered by(a)
> into 256 buckets STORED AS TEXTFILE;
> load data local inpath '../../data/files/sortdp/00_0' overwrite into
> table t1_staging partition (e=100);
> drop table t1_n147;
> create table t1_n147(
> a string,
> b decimal(6,0),
> c int,
> d string)
> partitioned by (e decimal(3,0))
> clustered by(a,b)
> into 10 buckets STORED AS ORC TBLPROPERTIES ('transactional'='true');
> set hive.stats.autogather=false;
> set hive.optimize.bucketingsorting = true;
> explain insert overwrite table t1_n147 partition(e) select a,b,c,d,e from
> t1_staging;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22777) Sorted dynamic partition optimization doesn't work if plan require implicit cast
[
https://issues.apache.org/jira/browse/HIVE-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22777:
---
Attachment: HIVE-22777.1.patch
> Sorted dynamic partition optimization doesn't work if plan require implicit
> cast
>
>
> Key: HIVE-22777
> URL: https://issues.apache.org/jira/browse/HIVE-22777
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22777.1.patch
>
>
> *Repro*
> set hive.stats.autogather=false;
> set hive.optimize.sort.dynamic.partition.threshold=1;
> set hive.optimize.bucketingsorting = true;
> {code:sql}
> drop table if exists t1_staging;
> create table t1_staging(
>a string,
>b int,
>c int,
>d string)
> partitioned by (e decimal(18,0))
> clustered by(a)
> into 256 buckets STORED AS TEXTFILE;
> load data local inpath '../../data/files/sortdp/00_0' overwrite into
> table t1_staging partition (e=100);
> drop table t1_n147;
> create table t1_n147(
> a string,
> b decimal(6,0),
> c int,
> d string)
> partitioned by (e decimal(3,0))
> clustered by(a,b)
> into 10 buckets STORED AS ORC TBLPROPERTIES ('transactional'='true');
> set hive.stats.autogather=false;
> set hive.optimize.bucketingsorting = true;
> explain insert overwrite table t1_n147 partition(e) select a,b,c,d,e from
> t1_staging;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22393) HiveStreamingConnection: Exception in beginTransaction causes AbstractRecordWriter to throw NPE, covering up real exception
[
https://issues.apache.org/jira/browse/HIVE-22393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated HIVE-22393:
Release Note: Submitting patch again due to flaky tests (nothing that was
changed in the patch)
Status: Patch Available (was: In Progress)
> HiveStreamingConnection: Exception in beginTransaction causes
> AbstractRecordWriter to throw NPE, covering up real exception
> ---
>
> Key: HIVE-22393
> URL: https://issues.apache.org/jira/browse/HIVE-22393
> Project: Hive
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 3.1.2
>Reporter: Alex Parrill
>Assignee: Matt Burgess
>Priority: Major
> Attachments: HIVE-22393.1.patch, HIVE-22393.2.patch
>
>
> Getting an error in my code that does some basic stuff with
> {{HiveStreamingConnection}}:
> {code}
> 19/10/23 15:12:44 ERROR yarn_logger.Main: Thread worker-0 uncaught exception
> java.lang.RuntimeException: org.apache.hive.streaming.StreamingException:
> Unable to close
> at com.datto.yarn_logger.WorkerThread.run(WorkerThread.java:51)
> Caused by: org.apache.hive.streaming.StreamingException: Unable to close
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.close(HiveStreamingConnection.java:973)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.markDead(HiveStreamingConnection.java:833)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.(HiveStreamingConnection.java:677)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.(HiveStreamingConnection.java:596)
> at
> org.apache.hive.streaming.HiveStreamingConnection.createNewTransactionBatch(HiveStreamingConnection.java:485)
> at
> org.apache.hive.streaming.HiveStreamingConnection.beginNextTransaction(HiveStreamingConnection.java:466)
> at
> org.apache.hive.streaming.HiveStreamingConnection.beginTransaction(HiveStreamingConnection.java:507)
> at com.datto.yarn_logger.WorkerThread.run(WorkerThread.java:49)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hive.streaming.AbstractRecordWriter.logStats(AbstractRecordWriter.java:547)
> at
> org.apache.hive.streaming.AbstractRecordWriter.close(AbstractRecordWriter.java:352)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.closeImpl(HiveStreamingConnection.java:979)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.close(HiveStreamingConnection.java:970)
> ... 7 more
> {code}
> Digging through the stack trace... {{TransactionBatch}} will try to catch
> exception in its constructor, and calls the close method if an exception is
> thrown (which is definitely happening; that's line 677 in
> {{HiveStreamingConnection.java}}). This eventually calls
> {{AbstractRecordWriter::close}} which, calls {{logStats}}, which tries to use
> {{heapMemoryMonitor}}, which is null because presumably {{init}} had never
> been called on the writer yet.
> Easy fix would to just check in {{logStats}} if {{heapMemoryMonitor}} is
> null, and do the same thing if the method it calls returns null in it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22777) Sorted dynamic partition optimization doesn't work if plan require implicit cast
[
https://issues.apache.org/jira/browse/HIVE-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023316#comment-17023316
]
Vineet Garg commented on HIVE-22777:
Root cause of HIVE-20910 is same as this
> Sorted dynamic partition optimization doesn't work if plan require implicit
> cast
>
>
> Key: HIVE-22777
> URL: https://issues.apache.org/jira/browse/HIVE-22777
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
>
> *Repro*
> set hive.stats.autogather=false;
> set hive.optimize.sort.dynamic.partition.threshold=1;
> set hive.optimize.bucketingsorting = true;
> {code:sql}
> drop table if exists t1_staging;
> create table t1_staging(
>a string,
>b int,
>c int,
>d string)
> partitioned by (e decimal(18,0))
> clustered by(a)
> into 256 buckets STORED AS TEXTFILE;
> load data local inpath '../../data/files/sortdp/00_0' overwrite into
> table t1_staging partition (e=100);
> drop table t1_n147;
> create table t1_n147(
> a string,
> b decimal(6,0),
> c int,
> d string)
> partitioned by (e decimal(3,0))
> clustered by(a,b)
> into 10 buckets STORED AS ORC TBLPROPERTIES ('transactional'='true');
> set hive.stats.autogather=false;
> set hive.optimize.bucketingsorting = true;
> explain insert overwrite table t1_n147 partition(e) select a,b,c,d,e from
> t1_staging;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22777) Sorted dynamic partition optimization doesn't work if plan require implicit cast
[
https://issues.apache.org/jira/browse/HIVE-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reassigned HIVE-22777:
--
> Sorted dynamic partition optimization doesn't work if plan require implicit
> cast
>
>
> Key: HIVE-22777
> URL: https://issues.apache.org/jira/browse/HIVE-22777
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
>
> *Repro*
> set hive.stats.autogather=false;
> set hive.optimize.sort.dynamic.partition.threshold=1;
> set hive.optimize.bucketingsorting = true;
> {code:sql}
> drop table if exists t1_staging;
> create table t1_staging(
>a string,
>b int,
>c int,
>d string)
> partitioned by (e decimal(18,0))
> clustered by(a)
> into 256 buckets STORED AS TEXTFILE;
> load data local inpath '../../data/files/sortdp/00_0' overwrite into
> table t1_staging partition (e=100);
> drop table t1_n147;
> create table t1_n147(
> a string,
> b decimal(6,0),
> c int,
> d string)
> partitioned by (e decimal(3,0))
> clustered by(a,b)
> into 10 buckets STORED AS ORC TBLPROPERTIES ('transactional'='true');
> set hive.stats.autogather=false;
> set hive.optimize.bucketingsorting = true;
> explain insert overwrite table t1_n147 partition(e) select a,b,c,d,e from
> t1_staging;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22393) HiveStreamingConnection: Exception in beginTransaction causes AbstractRecordWriter to throw NPE, covering up real exception
[
https://issues.apache.org/jira/browse/HIVE-22393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated HIVE-22393:
Attachment: HIVE-22393.2.patch
> HiveStreamingConnection: Exception in beginTransaction causes
> AbstractRecordWriter to throw NPE, covering up real exception
> ---
>
> Key: HIVE-22393
> URL: https://issues.apache.org/jira/browse/HIVE-22393
> Project: Hive
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 3.1.2
>Reporter: Alex Parrill
>Assignee: Matt Burgess
>Priority: Major
> Attachments: HIVE-22393.1.patch, HIVE-22393.2.patch
>
>
> Getting an error in my code that does some basic stuff with
> {{HiveStreamingConnection}}:
> {code}
> 19/10/23 15:12:44 ERROR yarn_logger.Main: Thread worker-0 uncaught exception
> java.lang.RuntimeException: org.apache.hive.streaming.StreamingException:
> Unable to close
> at com.datto.yarn_logger.WorkerThread.run(WorkerThread.java:51)
> Caused by: org.apache.hive.streaming.StreamingException: Unable to close
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.close(HiveStreamingConnection.java:973)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.markDead(HiveStreamingConnection.java:833)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.(HiveStreamingConnection.java:677)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.(HiveStreamingConnection.java:596)
> at
> org.apache.hive.streaming.HiveStreamingConnection.createNewTransactionBatch(HiveStreamingConnection.java:485)
> at
> org.apache.hive.streaming.HiveStreamingConnection.beginNextTransaction(HiveStreamingConnection.java:466)
> at
> org.apache.hive.streaming.HiveStreamingConnection.beginTransaction(HiveStreamingConnection.java:507)
> at com.datto.yarn_logger.WorkerThread.run(WorkerThread.java:49)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hive.streaming.AbstractRecordWriter.logStats(AbstractRecordWriter.java:547)
> at
> org.apache.hive.streaming.AbstractRecordWriter.close(AbstractRecordWriter.java:352)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.closeImpl(HiveStreamingConnection.java:979)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.close(HiveStreamingConnection.java:970)
> ... 7 more
> {code}
> Digging through the stack trace... {{TransactionBatch}} will try to catch
> exception in its constructor, and calls the close method if an exception is
> thrown (which is definitely happening; that's line 677 in
> {{HiveStreamingConnection.java}}). This eventually calls
> {{AbstractRecordWriter::close}} which, calls {{logStats}}, which tries to use
> {{heapMemoryMonitor}}, which is null because presumably {{init}} had never
> been called on the writer yet.
> Easy fix would to just check in {{logStats}} if {{heapMemoryMonitor}} is
> null, and do the same thing if the method it calls returns null in it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22393) HiveStreamingConnection: Exception in beginTransaction causes AbstractRecordWriter to throw NPE, covering up real exception
[
https://issues.apache.org/jira/browse/HIVE-22393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated HIVE-22393:
Status: Open (was: Patch Available)
> HiveStreamingConnection: Exception in beginTransaction causes
> AbstractRecordWriter to throw NPE, covering up real exception
> ---
>
> Key: HIVE-22393
> URL: https://issues.apache.org/jira/browse/HIVE-22393
> Project: Hive
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 3.1.2
>Reporter: Alex Parrill
>Assignee: Matt Burgess
>Priority: Major
> Attachments: HIVE-22393.1.patch
>
>
> Getting an error in my code that does some basic stuff with
> {{HiveStreamingConnection}}:
> {code}
> 19/10/23 15:12:44 ERROR yarn_logger.Main: Thread worker-0 uncaught exception
> java.lang.RuntimeException: org.apache.hive.streaming.StreamingException:
> Unable to close
> at com.datto.yarn_logger.WorkerThread.run(WorkerThread.java:51)
> Caused by: org.apache.hive.streaming.StreamingException: Unable to close
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.close(HiveStreamingConnection.java:973)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.markDead(HiveStreamingConnection.java:833)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.(HiveStreamingConnection.java:677)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.(HiveStreamingConnection.java:596)
> at
> org.apache.hive.streaming.HiveStreamingConnection.createNewTransactionBatch(HiveStreamingConnection.java:485)
> at
> org.apache.hive.streaming.HiveStreamingConnection.beginNextTransaction(HiveStreamingConnection.java:466)
> at
> org.apache.hive.streaming.HiveStreamingConnection.beginTransaction(HiveStreamingConnection.java:507)
> at com.datto.yarn_logger.WorkerThread.run(WorkerThread.java:49)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hive.streaming.AbstractRecordWriter.logStats(AbstractRecordWriter.java:547)
> at
> org.apache.hive.streaming.AbstractRecordWriter.close(AbstractRecordWriter.java:352)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.closeImpl(HiveStreamingConnection.java:979)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.close(HiveStreamingConnection.java:970)
> ... 7 more
> {code}
> Digging through the stack trace... {{TransactionBatch}} will try to catch
> exception in its constructor, and calls the close method if an exception is
> thrown (which is definitely happening; that's line 677 in
> {{HiveStreamingConnection.java}}). This eventually calls
> {{AbstractRecordWriter::close}} which, calls {{logStats}}, which tries to use
> {{heapMemoryMonitor}}, which is null because presumably {{init}} had never
> been called on the writer yet.
> Easy fix would to just check in {{logStats}} if {{heapMemoryMonitor}} is
> null, and do the same thing if the method it calls returns null in it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work started] (HIVE-22393) HiveStreamingConnection: Exception in beginTransaction causes AbstractRecordWriter to throw NPE, covering up real exception
[
https://issues.apache.org/jira/browse/HIVE-22393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-22393 started by Matt Burgess.
---
> HiveStreamingConnection: Exception in beginTransaction causes
> AbstractRecordWriter to throw NPE, covering up real exception
> ---
>
> Key: HIVE-22393
> URL: https://issues.apache.org/jira/browse/HIVE-22393
> Project: Hive
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 3.1.2
>Reporter: Alex Parrill
>Assignee: Matt Burgess
>Priority: Major
> Attachments: HIVE-22393.1.patch
>
>
> Getting an error in my code that does some basic stuff with
> {{HiveStreamingConnection}}:
> {code}
> 19/10/23 15:12:44 ERROR yarn_logger.Main: Thread worker-0 uncaught exception
> java.lang.RuntimeException: org.apache.hive.streaming.StreamingException:
> Unable to close
> at com.datto.yarn_logger.WorkerThread.run(WorkerThread.java:51)
> Caused by: org.apache.hive.streaming.StreamingException: Unable to close
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.close(HiveStreamingConnection.java:973)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.markDead(HiveStreamingConnection.java:833)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.(HiveStreamingConnection.java:677)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.(HiveStreamingConnection.java:596)
> at
> org.apache.hive.streaming.HiveStreamingConnection.createNewTransactionBatch(HiveStreamingConnection.java:485)
> at
> org.apache.hive.streaming.HiveStreamingConnection.beginNextTransaction(HiveStreamingConnection.java:466)
> at
> org.apache.hive.streaming.HiveStreamingConnection.beginTransaction(HiveStreamingConnection.java:507)
> at com.datto.yarn_logger.WorkerThread.run(WorkerThread.java:49)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hive.streaming.AbstractRecordWriter.logStats(AbstractRecordWriter.java:547)
> at
> org.apache.hive.streaming.AbstractRecordWriter.close(AbstractRecordWriter.java:352)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.closeImpl(HiveStreamingConnection.java:979)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.close(HiveStreamingConnection.java:970)
> ... 7 more
> {code}
> Digging through the stack trace... {{TransactionBatch}} will try to catch
> exception in its constructor, and calls the close method if an exception is
> thrown (which is definitely happening; that's line 677 in
> {{HiveStreamingConnection.java}}). This eventually calls
> {{AbstractRecordWriter::close}} which, calls {{logStats}}, which tries to use
> {{heapMemoryMonitor}}, which is null because presumably {{init}} had never
> been called on the writer yet.
> Easy fix would to just check in {{logStats}} if {{heapMemoryMonitor}} is
> null, and do the same thing if the method it calls returns null in it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22772) Log opType and session level information for each operation
[
https://issues.apache.org/jira/browse/HIVE-22772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023308#comment-17023308
]
Hive QA commented on HIVE-22772:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} service in master has 51 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} service: The patch generated 8 new + 21 unchanged - 0
fixed = 29 total (was 21) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20327/dev-support/hive-personality.sh
|
| git revision | master / a344426 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20327/yetus/diff-checkstyle-service.txt
|
| modules | C: service U: service |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20327/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Log opType and session level information for each operation
>
>
> Key: HIVE-22772
> URL: https://issues.apache.org/jira/browse/HIVE-22772
> Project: Hive
> Issue Type: Improvement
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22772.1.patch
>
>
> if would be useful to have this info available for issue troubleshooting.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22774) Add toggle to disable scheduled query service in HMS
[ https://issues.apache.org/jira/browse/HIVE-22774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22774: Attachment: HIVE-22774.02.patch > Add toggle to disable scheduled query service in HMS > > > Key: HIVE-22774 > URL: https://issues.apache.org/jira/browse/HIVE-22774 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22774.01.patch, HIVE-22774.02.patch > > > * currently it's enabled by default -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023300#comment-17023300
]
Hive QA commented on HIVE-22736:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991727/HIVE-22736.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20326/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20326/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20326/
Messages:
{noformat}
This message was trimmed, see log for full details
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/ServletRequest.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/ServletResponse.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/annotation/WebFilter.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/http/HttpServletRequest.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/http/HttpServletResponse.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/classification/target/hive-classification-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/common/classification/InterfaceAudience$LimitedPrivate.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/classification/target/hive-classification-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/common/classification/InterfaceStability$Unstable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/ByteArrayOutputStream.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/OutputStream.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/Closeable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/lang/AutoCloseable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/Flushable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(javax/xml/bind/annotation/XmlRootElement.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/commons/commons-exec/1.1/commons-exec-1.1.jar(org/apache/commons/exec/ExecuteException.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/security/PrivilegedExceptionAction.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/ExecutionException.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/TimeoutException.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/3.1.0/hadoop-common-3.1.0.jar(org/apache/hadoop/fs/FileSystem.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/HadoopShimsSecure.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/ShimLoader.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/HadoopShims.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/HadoopShims$WebHCatJTShim.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/3.1.0/hadoop-common-3.1.0.jar(org/apache/hadoop/util/ToolRunner.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/CancellationException.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/RejectedExecutionException.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/SynchronousQueue.class)]]
[loading
[jira] [Updated] (HIVE-22774) Add toggle to disable scheduled query service in HMS
[ https://issues.apache.org/jira/browse/HIVE-22774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22774: Attachment: HIVE-22774.01.patch > Add toggle to disable scheduled query service in HMS > > > Key: HIVE-22774 > URL: https://issues.apache.org/jira/browse/HIVE-22774 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22774.01.patch > > > * currently it's enabled by default -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22774) Add toggle to disable scheduled query service in HMS
[ https://issues.apache.org/jira/browse/HIVE-22774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22774: Status: Patch Available (was: Open) > Add toggle to disable scheduled query service in HMS > > > Key: HIVE-22774 > URL: https://issues.apache.org/jira/browse/HIVE-22774 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22774.01.patch > > > * currently it's enabled by default -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22767) beeline doesn't parse semicolons in comments properly
[
https://issues.apache.org/jira/browse/HIVE-22767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023291#comment-17023291
]
Hive QA commented on HIVE-22767:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991728/HIVE-22767.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17947 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20325/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20325/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20325/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991728 - PreCommit-HIVE-Build
> beeline doesn't parse semicolons in comments properly
> -
>
> Key: HIVE-22767
> URL: https://issues.apache.org/jira/browse/HIVE-22767
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Zoltan Matyus
>Assignee: Zoltan Matyus
>Priority: Major
> Attachments: HIVE-22767.01.patch, HIVE-22767.02.patch
>
>
> HIVE-12646 fixed the handling of semicolons in quoted strings, but leaves the
> problem of semicolons in comments. E.g. with beeline connected to any
> database...
> this works: {code:sql}select 1; select /* */ 2; select /* */ 3;{code}
> this doesn't work: {code:sql}select 1; select /* ; */ 2; select /* ; */
> 3;{code}
> This has been fixed and reintroduced before (possibly multiple times).
> Ideally, there should be a single utility method somewhere to separate
> comments, strings and commands -- with the proper testing in place (q files).
> However, I'm trying to make this fix back-portable, so a light touch is
> needed. I'm focusing on beeline for now, and only writing (very thorough)
> unit tests, as I cannot exclude any new q files from TestCliDriver (which
> would break, since it's using a different parsing method).
>
> P.S. excerpt of the error message:
> {noformat}
> 0: jdbc:hive2://...> select 1; select /* ; */ 2; select /* ; */ 3;
> INFO : Compiling command(queryId=...): select 1
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0,
> type:int, comment:null)], properties:null)
> INFO : Completed compiling command(queryId=...); Time taken: 0.38 seconds
> INFO : Executing command(queryId=...): select 1
> INFO : Completed executing command(queryId=...); Time taken: 0.004 seconds
> INFO : OK
> +--+
> | _c0 |
> +--+
> | 1|
> +--+
> 1 row selected (2.007 seconds)
> INFO : Compiling command(queryId=...): select /*
> ERROR : FAILED: ParseException line 1:9 cannot recognize input near ''
> '' '' in select clause
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 cannot recognize
> input near '' '' '' in select clause
> at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:233)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:79)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:72)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:598)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1505)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1452)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1447)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at ...
> {noformat}
>
> Similarly, the following query also fails:
> {code:sql}select /* ' */ 1; select /* ' */ 2;{code}
> I suspect line comments are also not handled properly but I cannot reproduce
> this in interactive beeline...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22767) beeline doesn't parse semicolons in comments properly
[
https://issues.apache.org/jira/browse/HIVE-22767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023265#comment-17023265
]
Hive QA commented on HIVE-22767:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} beeline in master has 48 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20325/dev-support/hive-personality.sh
|
| git revision | master / a344426 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: beeline U: beeline |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20325/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> beeline doesn't parse semicolons in comments properly
> -
>
> Key: HIVE-22767
> URL: https://issues.apache.org/jira/browse/HIVE-22767
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Zoltan Matyus
>Assignee: Zoltan Matyus
>Priority: Major
> Attachments: HIVE-22767.01.patch, HIVE-22767.02.patch
>
>
> HIVE-12646 fixed the handling of semicolons in quoted strings, but leaves the
> problem of semicolons in comments. E.g. with beeline connected to any
> database...
> this works: {code:sql}select 1; select /* */ 2; select /* */ 3;{code}
> this doesn't work: {code:sql}select 1; select /* ; */ 2; select /* ; */
> 3;{code}
> This has been fixed and reintroduced before (possibly multiple times).
> Ideally, there should be a single utility method somewhere to separate
> comments, strings and commands -- with the proper testing in place (q files).
> However, I'm trying to make this fix back-portable, so a light touch is
> needed. I'm focusing on beeline for now, and only writing (very thorough)
> unit tests, as I cannot exclude any new q files from TestCliDriver (which
> would break, since it's using a different parsing method).
>
> P.S. excerpt of the error message:
> {noformat}
> 0: jdbc:hive2://...> select 1; select /* ; */ 2; select /* ; */ 3;
> INFO : Compiling command(queryId=...): select 1
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0,
> type:int, comment:null)], properties:null)
> INFO : Completed compiling
[jira] [Commented] (HIVE-22729) Provide a failure reason for failed compactions
[
https://issues.apache.org/jira/browse/HIVE-22729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023244#comment-17023244
]
Hive QA commented on HIVE-22729:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991726/HIVE-22729.09.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17947 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20324/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20324/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20324/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991726 - PreCommit-HIVE-Build
> Provide a failure reason for failed compactions
> ---
>
> Key: HIVE-22729
> URL: https://issues.apache.org/jira/browse/HIVE-22729
> Project: Hive
> Issue Type: Improvement
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-22729.01.patch, HIVE-22729.02.patch,
> HIVE-22729.03.patch, HIVE-22729.04.patch, HIVE-22729.05.patch,
> HIVE-22729.06.patch, HIVE-22729.07.patch, HIVE-22729.08.patch,
> HIVE-22729.09.patch
>
>
> We should provide a compaction failure reason as easily accessible as
> possible. Like in the result of the {{SHOW COMPACTIONS}} command.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-22776) Add mssql to test against metastore schema updates
[ https://issues.apache.org/jira/browse/HIVE-22776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich resolved HIVE-22776. - Resolution: Duplicate we already have this via HIVE-21954; the test is named ITestSqlServer > Add mssql to test against metastore schema updates > -- > > Key: HIVE-22776 > URL: https://issues.apache.org/jira/browse/HIVE-22776 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > > there is a docker container which runs mssql - we could utilize it to run our > metastore upgrade tests -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shivam updated HIVE-22771:
--
Attachment: HIVE-22771.patch
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> +In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And,
> in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#FF}\\d\\.?\\d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22775) Use the qt:authorizer option in qtests
[ https://issues.apache.org/jira/browse/HIVE-22775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22775: Status: Patch Available (was: Open) > Use the qt:authorizer option in qtests > -- > > Key: HIVE-22775 > URL: https://issues.apache.org/jira/browse/HIVE-22775 > Project: Hive > Issue Type: Improvement > Components: Tests >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22775.01.patch > > > qt:authorizer sets up things for auth tests; [~mgergely] suggested to use it > in existing tests -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22775) Use the qt:authorizer option in qtests
[ https://issues.apache.org/jira/browse/HIVE-22775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22775: Attachment: HIVE-22775.01.patch > Use the qt:authorizer option in qtests > -- > > Key: HIVE-22775 > URL: https://issues.apache.org/jira/browse/HIVE-22775 > Project: Hive > Issue Type: Improvement > Components: Tests >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22775.01.patch > > > qt:authorizer sets up things for auth tests; [~mgergely] suggested to use it > in existing tests -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22729) Provide a failure reason for failed compactions
[
https://issues.apache.org/jira/browse/HIVE-22729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023223#comment-17023223
]
Hive QA commented on HIVE-22729:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
29s{color} | {color:blue} standalone-metastore/metastore-common in master has
37 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
10s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
3s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
23s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 5 new + 694 unchanged - 8 fixed = 699 total (was 702) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 2 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 8m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20324/dev-support/hive-personality.sh
|
| git revision | master / a344426 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20324/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20324/yetus/whitespace-tabs.txt
|
| modules | C: standalone-metastore/metastore-common
standalone-metastore/metastore-server ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20324/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Provide a failure reason for failed compactions
> ---
>
> Key: HIVE-22729
> URL: https://issues.apache.org/jira/browse/HIVE-22729
> Project: Hive
> Issue Type: Improvement
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-22729.01.patch, HIVE-22729.02.patch,
> HIVE-22729.03.patch, HIVE-22729.04.patch, HIVE-22729.05.patch,
> HIVE-22729.06.patch, HIVE-22729.07.patch, HIVE-22729.08.patch,
> HIVE-22729.09.patch
>
>
> We should provide a compaction failure reason as easily
[jira] [Commented] (HIVE-22753) Fix gradual mem leak: Operationlog related appenders should be cleared up on errors
[
https://issues.apache.org/jira/browse/HIVE-22753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023201#comment-17023201
]
Ashutosh Chauhan commented on HIVE-22753:
-
[~rajesh.balamohan] Will {{IdlePurgePolicy}} fix this?
Timeout for cache should be lower. 1s i would suggest.
> Fix gradual mem leak: Operationlog related appenders should be cleared up on
> errors
>
>
> Key: HIVE-22753
> URL: https://issues.apache.org/jira/browse/HIVE-22753
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22753.1.patch, HIVE-22753.2.patch,
> image-2020-01-21-11-14-37-911.png, image-2020-01-21-11-17-59-279.png,
> image-2020-01-21-11-18-37-294.png
>
>
> In case of exception in SQLOperation, operational log does not get cleared
> up. This causes gradual build up of HushableRandomAccessFileAppender causing
> HS2 to OOM after some time.
> !image-2020-01-21-11-14-37-911.png|width=431,height=267!
>
> Allocation tree
> !image-2020-01-21-11-18-37-294.png|width=425,height=178!
>
> Prod instance mem
> !image-2020-01-21-11-17-59-279.png|width=698,height=209!
>
> Each HushableRandomAccessFileAppender holds internal ref to
> RandomAccessFileAppender which holds a 256 KB bytebuffer, causing the mem
> leak.
> Related ticket: HIVE-18820
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22726) TopN Key optimizer should use array instead of priority queue
[
https://issues.apache.org/jira/browse/HIVE-22726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023185#comment-17023185
]
Hive QA commented on HIVE-22726:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991724/HIVE-22726.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20323/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20323/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20323/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-01-24 19:05:34.638
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-20323/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-01-24 19:05:34.659
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at a344426 HIVE-22712: ReExec Driver execute submit the query in
default queue irrespective of user defined queue (Rajkumar Singh via Zoltan
Haindrich)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at a344426 HIVE-22712: ReExec Driver execute submit the query in
default queue irrespective of user defined queue (Rajkumar Singh via Zoltan
Haindrich)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-01-24 19:05:41.147
+ rm -rf ../yetus_PreCommit-HIVE-Build-20323
+ mkdir ../yetus_PreCommit-HIVE-Build-20323
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-20323
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-20323/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
error: a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyFilter.java: does
not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java: does
not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorTopNKeyOperator.java:
does not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/wrapper/VectorHashKeyWrapperBatch.java:
does not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/wrapper/VectorHashKeyWrapperGeneralComparator.java:
does not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java: does not
exist in index
Trying to apply the patch with -p1
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java:38
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java' with
conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java:58
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java'
with conflicts.
Going to apply patch with: git apply -p1
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java:38
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java' with
conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyProcessor.java:58
Falling back to three-way merge...
Applied patch to
[jira] [Commented] (HIVE-22770) Skip interning of MapWork fields for tez during deserialization
[
https://issues.apache.org/jira/browse/HIVE-22770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023167#comment-17023167
]
Hive QA commented on HIVE-22770:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
17s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
25s{color} | {color:red} ql generated 2 new + 1532 unchanged - 0 fixed = 1534
total (was 1532) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 24s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Unchecked/unconfirmed cast from com.esotericsoftware.kryo.Kryo to
org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks in
org.apache.hadoop.hive.ql.exec.SerializationUtilities.releaseKryo(Kryo) At
SerializationUtilities.java:org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks
in org.apache.hadoop.hive.ql.exec.SerializationUtilities.releaseKryo(Kryo) At
SerializationUtilities.java:[line 307] |
| | Unchecked/unconfirmed cast from com.esotericsoftware.kryo.Kryo to
org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks in
org.apache.hadoop.hive.ql.exec.SerializationUtilities$MapWorkSerializer.read(Kryo,
Input, Class) At
SerializationUtilities.java:org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks
in
org.apache.hadoop.hive.ql.exec.SerializationUtilities$MapWorkSerializer.read(Kryo,
Input, Class) At SerializationUtilities.java:[line 600] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20322/dev-support/hive-personality.sh
|
| git revision | master / a344426 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20322/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20322/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Skip interning of MapWork fields for tez during deserialization
> ---
>
> Key: HIVE-22770
> URL: https://issues.apache.org/jira/browse/HIVE-22770
> Project: Hive
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-22770.01.patch
>
>
> HIVE-19937 introduced some interning logic into mapwork deserialization
> process, but it's only related to spark, maybe we should skip this
[jira] [Commented] (HIVE-22518) SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
[ https://issues.apache.org/jira/browse/HIVE-22518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023145#comment-17023145 ] Miklos Gergely commented on HIVE-22518: --- +1 > SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests > > > Key: HIVE-22518 > URL: https://issues.apache.org/jira/browse/HIVE-22518 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22518.01.patch, HIVE-22518.01.patch, > HIVE-22518.02.patch, HIVE-22518.02.patch, HIVE-22518.03.patch, > HIVE-22518.03.patch, HIVE-22518.04.patch, HIVE-22518.05.patch, > HIVE-22518.05.patch, HIVE-22518.06.patch, HIVE-22518.06.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023137#comment-17023137
]
Mithun Radhakrishnan edited comment on HIVE-22771 at 1/24/20 5:44 PM:
--
Hmm... This looks like a good catch to me. It looks like the
static-partitioning case was missed/mishandled before.
The dynamic-partitioning case seems to be handled better, with a more greedy
regex:
[https://github.com/apache/hive/blob/706c1d44f7734ed36900a02c7255c1d0ce0ad45a/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java#L867]
[~shivam-mohan], you might want to post the patch here, for {{master}} as well,
and have CI run tests. The fix seems good to me, on the face of it.
was (Author: mithun):
Hmm... This looks like a good catch to me. It looks like the
static-partitioning case was missed/mishandled before.
The dynamic-partitioning case seems to be handled better, with a more greedy
regex:
[https://github.com/apache/hive/blob/706c1d44f7734ed36900a02c7255c1d0ce0ad45a/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java#L867]
[~shivam-mohan], you might want to post the patch here, for {{master}} as well,
and have CI run tests. The fix seems good to me, on the face of it.
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> +In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And,
> in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#FF}\\d\\.?\\d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023137#comment-17023137
]
Mithun Radhakrishnan commented on HIVE-22771:
-
Hmm... This looks like a good catch to me. It looks like the
static-partitioning case was missed/mishandled before.
The dynamic-partitioning case seems to be handled better, with a more greedy
regex:
[https://github.com/apache/hive/blob/706c1d44f7734ed36900a02c7255c1d0ce0ad45a/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java#L867]
[~shivam-mohan], you might want to post the patch here, for {{master}} as well,
and have CI run tests. The fix seems good to me, on the face of it.
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> +In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And,
> in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#FF}\\d\\.?\\d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22518) SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
[ https://issues.apache.org/jira/browse/HIVE-22518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023132#comment-17023132 ] Zoltan Haindrich commented on HIVE-22518: - [~mgergely]: I've removed the user.name set from the qtoption; will migrate other tests in HIVE-22775 could you please take another look? > SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests > > > Key: HIVE-22518 > URL: https://issues.apache.org/jira/browse/HIVE-22518 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22518.01.patch, HIVE-22518.01.patch, > HIVE-22518.02.patch, HIVE-22518.02.patch, HIVE-22518.03.patch, > HIVE-22518.03.patch, HIVE-22518.04.patch, HIVE-22518.05.patch, > HIVE-22518.05.patch, HIVE-22518.06.patch, HIVE-22518.06.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22723) SHOW CREATE TABLE omits backtics, results in syntax error
[
https://issues.apache.org/jira/browse/HIVE-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely updated HIVE-22723:
--
Attachment: HIVE-22723.02.patch
> SHOW CREATE TABLE omits backtics, results in syntax error
> -
>
> Key: HIVE-22723
> URL: https://issues.apache.org/jira/browse/HIVE-22723
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Environment: Run SHOW CREATE TABLE via AWS Athena
>Reporter: Alex Yankelevich
>Assignee: Miklos Gergely
>Priority: Minor
> Attachments: HIVE-22723.01.patch, HIVE-22723.02.patch
>
>
> {code}
> CREATE EXTERNAL TABLE `alerts1`(
> `_id` string COMMENT 'from deserializer', `org` string COMMENT 'from
> deserializer', `collector` string COMMENT 'from deserializer', `source`
> string COMMENT 'from deserializer', `entitytype` string COMMENT 'from
> deserializer', `entity` string COMMENT 'from deserializer', `rule` string
> COMMENT 'from deserializer', `findingdescid` string COMMENT 'from
> deserializer',
> `details`
> struct>,findingtype:string,producttype:string,entityname:string,extra:struct,entityparams:struct,version:string,build:string>>
> COMMENT 'from deserializer', `seenat` array>
> COMMENT 'from deserializer', `execution` array>
> COMMENT 'from deserializer', `lastmodified` struct<__numberint:string>
> COMMENT 'from deserializer'
> ) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' WITH SERDEPROPERTIES
> (
> 'paths'='_id,collector,details,entity,entityType,execution,findingDescId,lastModified,org,rule,seenAt,source')
> STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat'
> OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
> LOCATION 's3://redshift-poc-ay/alerts' TBLPROPERTIES (
> 'CrawlerSchemaDeserializerVersion'='1.0',
> 'CrawlerSchemaSerializerVersion'='1.0',
> 'UPDATED_BY_CRAWLER'='redshift-poc-alerts-ay', 'averageRecordSize'='9619',
> 'classification'='json', 'compressionType'='none', 'objectCount'='1',
> 'recordCount'='21028', 'sizeKey'='202270843',
> 'transient_lastDdlTime'='1578946186', 'typeOfData'='file')
> {code}
> __numberint, __numberlong columns are not generated enclosed in backticks
> even though they were explicitly created with them. Therefore, the generated
> DDL is syntactically incorrect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22126) hive-exec packaging should shade guava
[
https://issues.apache.org/jira/browse/HIVE-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023109#comment-17023109
]
Hive QA commented on HIVE-22126:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
30s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
1s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 1 new + 45 unchanged - 1 fixed
= 46 total (was 46) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
0s{color} | {color:red} ql generated 1 new + 1531 unchanged - 1 fixed = 1532
total (was 1532) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 19s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| |
org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveAggregate.deriveRowType(RelDataTypeFactory,
RelDataType, boolean, ImmutableBitSet, List) concatenates strings using + in a
loop At HiveAggregate.java:concatenates strings using + in a loop At
HiveAggregate.java:[line 104] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20321/dev-support/hive-personality.sh
|
| git revision | master / a344426 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20321/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20321/yetus/new-findbugs-ql.html
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20321/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> hive-exec packaging should shade guava
> --
>
> Key: HIVE-22126
> URL: https://issues.apache.org/jira/browse/HIVE-22126
> Project: Hive
> Issue Type: Bug
>Reporter: Vihang Karajgaonkar
>Assignee: Eugene Chung
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22126.01.patch, HIVE-22126.02.patch,
>
[jira] [Assigned] (HIVE-22776) Add mssql to test against metastore schema updates
[ https://issues.apache.org/jira/browse/HIVE-22776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich reassigned HIVE-22776: --- > Add mssql to test against metastore schema updates > -- > > Key: HIVE-22776 > URL: https://issues.apache.org/jira/browse/HIVE-22776 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > > there is a docker container which runs mssql - we could utilize it to run our > metastore upgrade tests -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22653) Remove commons-lang leftovers
[
https://issues.apache.org/jira/browse/HIVE-22653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023083#comment-17023083
]
David Lavati commented on HIVE-22653:
-
{quote}it might have been diffed against the wrong branch?
{quote}
It's possible yes, I'm keeping the PR in order and up to date though, as
commons-lang-2 keeps crawling back.
> Remove commons-lang leftovers
> -
>
> Key: HIVE-22653
> URL: https://issues.apache.org/jira/browse/HIVE-22653
> Project: Hive
> Issue Type: Bug
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22653.01.patch, HIVE-22653.01.patch,
> HIVE-22653.02.patch, HIVE-22653.03.patch, HIVE-22653.04.patch,
> HIVE-22653.04.patch, HIVE-22653.04.patch, HIVE-22653.05.patch,
> HIVE-22653.06.patch, HIVE-22653.07.patch, HIVE-22653.07.patch,
> HIVE-22653.08.patch, HIVE-22653.08.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> HIVE-7145 removed commons-lang - in favor of commons-lang3 - as a direct
> dependency, however a high number of files still refer to commons-lang, which
> is transitively brought in either way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22775) Use the qt:authorizer option in qtests
[ https://issues.apache.org/jira/browse/HIVE-22775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich reassigned HIVE-22775: --- > Use the qt:authorizer option in qtests > -- > > Key: HIVE-22775 > URL: https://issues.apache.org/jira/browse/HIVE-22775 > Project: Hive > Issue Type: Improvement > Components: Tests >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > > qt:authorizer sets up things for auth tests; [~mgergely] suggested to use it > in existing tests -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22774) Add toggle to disable scheduled query service in HMS
[ https://issues.apache.org/jira/browse/HIVE-22774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich reassigned HIVE-22774: --- > Add toggle to disable scheduled query service in HMS > > > Key: HIVE-22774 > URL: https://issues.apache.org/jira/browse/HIVE-22774 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > > * currently it's enabled by default -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023072#comment-17023072
]
Hive QA commented on HIVE-22538:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991714/HIVE-22538.6.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17947 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20320/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20320/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20320/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991714 - PreCommit-HIVE-Build
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.3.patch,
> HIVE-22538.4.patch, HIVE-22538.5.patch, HIVE-22538.6.patch,
> HIVE-22538.6.patch, HIVE-22538.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22723) SHOW CREATE TABLE omits backtics, results in syntax error
[
https://issues.apache.org/jira/browse/HIVE-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023070#comment-17023070
]
Zoltan Haindrich commented on HIVE-22723:
-
I think we may also want to consider to accept identifiers starting with _
mysql, psql accepts:
{code}
create table _t(_a integer)
{code}
meanwhile hive and oracle does not.
> SHOW CREATE TABLE omits backtics, results in syntax error
> -
>
> Key: HIVE-22723
> URL: https://issues.apache.org/jira/browse/HIVE-22723
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Environment: Run SHOW CREATE TABLE via AWS Athena
>Reporter: Alex Yankelevich
>Assignee: Miklos Gergely
>Priority: Minor
> Attachments: HIVE-22723.01.patch
>
>
> {code}
> CREATE EXTERNAL TABLE `alerts1`(
> `_id` string COMMENT 'from deserializer', `org` string COMMENT 'from
> deserializer', `collector` string COMMENT 'from deserializer', `source`
> string COMMENT 'from deserializer', `entitytype` string COMMENT 'from
> deserializer', `entity` string COMMENT 'from deserializer', `rule` string
> COMMENT 'from deserializer', `findingdescid` string COMMENT 'from
> deserializer',
> `details`
> struct>,findingtype:string,producttype:string,entityname:string,extra:struct,entityparams:struct,version:string,build:string>>
> COMMENT 'from deserializer', `seenat` array>
> COMMENT 'from deserializer', `execution` array>
> COMMENT 'from deserializer', `lastmodified` struct<__numberint:string>
> COMMENT 'from deserializer'
> ) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' WITH SERDEPROPERTIES
> (
> 'paths'='_id,collector,details,entity,entityType,execution,findingDescId,lastModified,org,rule,seenAt,source')
> STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat'
> OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
> LOCATION 's3://redshift-poc-ay/alerts' TBLPROPERTIES (
> 'CrawlerSchemaDeserializerVersion'='1.0',
> 'CrawlerSchemaSerializerVersion'='1.0',
> 'UPDATED_BY_CRAWLER'='redshift-poc-alerts-ay', 'averageRecordSize'='9619',
> 'classification'='json', 'compressionType'='none', 'objectCount'='1',
> 'recordCount'='21028', 'sizeKey'='202270843',
> 'transient_lastDdlTime'='1578946186', 'typeOfData'='file')
> {code}
> __numberint, __numberlong columns are not generated enclosed in backticks
> even though they were explicitly created with them. Therefore, the generated
> DDL is syntactically incorrect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue
[ https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22712: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) pushed to master. Thank you [~Rajkumar Singh]! > ReExec Driver execute submit the query in default queue irrespective of user > defined queue > -- > > Key: HIVE-22712 > URL: https://issues.apache.org/jira/browse/HIVE-22712 > Project: Hive > Issue Type: Bug > Components: Hive, HiveServer2 >Affects Versions: 3.1.2 > Environment: Hive-3 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, > HIVE-22712.03.patch, HIVE-22712.04.patch, HIVE-22712.05.patch, > HIVE-22712.06.patch, HIVE-22712.06.patch, HIVE-22712.07.patch, > HIVE-22712.patch > > > we unset the queue name intentionally in > TezSessionState#startSessionAndContainers, > as a result reexec create a new session in the default queue and create a > problem, its a cumbersome to add reexec.overlay.tez.queue.name at session > level. > I could not find a better way of setting the queue name (I am open for the > suggestion here) since it can create a conflict with the Global queue name > vs user-defined queue that's why setting while initialization of > ReExecutionOverlayPlugin. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22706) Jdbc storage handler incorrectly interprets boolean column value in derby
[ https://issues.apache.org/jira/browse/HIVE-22706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22706: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) pushed to master. Thank you [~mgergely] for reviewing the changes! > Jdbc storage handler incorrectly interprets boolean column value in derby > - > > Key: HIVE-22706 > URL: https://issues.apache.org/jira/browse/HIVE-22706 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22706.01.patch, HIVE-22706.01.patch, > HIVE-22706.01.patch, HIVE-22706.01.patch, HIVE-22706.02.patch, > HIVE-22706.02.patch, HIVE-22706.02.patch, HIVE-22706.02.patch > > > in case the column value is false ; the storage handler interprets it as true -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22761) Scheduled query executor fails to report query state as errored if session initialization fails
[ https://issues.apache.org/jira/browse/HIVE-22761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22761: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) pushed to master. Thank you [~mgergely] for reviewing the changes! > Scheduled query executor fails to report query state as errored if session > initialization fails > --- > > Key: HIVE-22761 > URL: https://issues.apache.org/jira/browse/HIVE-22761 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22761.01.patch > > > right now the info object is only initialized after the sessionstate is > inited - which might get into trouble... -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-10362) Support Type check/conversion in dynamic partition column
[ https://issues.apache.org/jira/browse/HIVE-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-10362: - Status: Open (was: Patch Available) > Support Type check/conversion in dynamic partition column > - > > Key: HIVE-10362 > URL: https://issues.apache.org/jira/browse/HIVE-10362 > Project: Hive > Issue Type: Improvement > Components: Query Processor, Types >Affects Versions: 3.0.0, 1.0.0, 4.0.0 >Reporter: Chaoyu Tang >Assignee: Karen Coppage >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-10362.01.patch, HIVE-10362.02.patch > > > There are quite a lot of issues associated with the non-noramlized or > type-mismatched values for partition column. Hive has many ways to introduce > such problematic data. HIVE-10307 mainly provides the support to type > check/convert/normalize the partition column value in static partition > specification. This JIRA tries to deal with the partition column type in > dynamic partition insert. Currently any data can be inserted as a partition > column value as long as it is quoted as a string. For example, > create table dynparttypechecknum (key int, value string) partitioned by (part > int); > insert into dynparttypechecknum partition (part) select key, value, '1' > from src limit 1; > show partitions dynparttypechecknum; > -- > part=1 > The partition column value is non-normalized int 1. It causes some > unnecessary problems such as integer partition column JDO filter pushdown > (see HIVE-6052) and others like HIVE-10210. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-10362) Support Type check/conversion in dynamic partition column
[ https://issues.apache.org/jira/browse/HIVE-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-10362: - Attachment: HIVE-10362.02.patch Status: Patch Available (was: Open) > Support Type check/conversion in dynamic partition column > - > > Key: HIVE-10362 > URL: https://issues.apache.org/jira/browse/HIVE-10362 > Project: Hive > Issue Type: Improvement > Components: Query Processor, Types >Affects Versions: 3.0.0, 1.0.0, 4.0.0 >Reporter: Chaoyu Tang >Assignee: Karen Coppage >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-10362.01.patch, HIVE-10362.02.patch > > > There are quite a lot of issues associated with the non-noramlized or > type-mismatched values for partition column. Hive has many ways to introduce > such problematic data. HIVE-10307 mainly provides the support to type > check/convert/normalize the partition column value in static partition > specification. This JIRA tries to deal with the partition column type in > dynamic partition insert. Currently any data can be inserted as a partition > column value as long as it is quoted as a string. For example, > create table dynparttypechecknum (key int, value string) partitioned by (part > int); > insert into dynparttypechecknum partition (part) select key, value, '1' > from src limit 1; > show partitions dynparttypechecknum; > -- > part=1 > The partition column value is non-normalized int 1. It causes some > unnecessary problems such as integer partition column JDO filter pushdown > (see HIVE-6052) and others like HIVE-10210. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22653) Remove commons-lang leftovers
[
https://issues.apache.org/jira/browse/HIVE-22653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023058#comment-17023058
]
Zoltan Haindrich commented on HIVE-22653:
-
it's not clear why we have this in the latest patch:
{code}
+HIVE_TXN_READ_LOCKS("hive.txn.read.locks", true,
+"flag to turn off the strict read lock when set to false"),
{code}
it might have been diffed against the wrong branch?
> Remove commons-lang leftovers
> -
>
> Key: HIVE-22653
> URL: https://issues.apache.org/jira/browse/HIVE-22653
> Project: Hive
> Issue Type: Bug
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22653.01.patch, HIVE-22653.01.patch,
> HIVE-22653.02.patch, HIVE-22653.03.patch, HIVE-22653.04.patch,
> HIVE-22653.04.patch, HIVE-22653.04.patch, HIVE-22653.05.patch,
> HIVE-22653.06.patch, HIVE-22653.07.patch, HIVE-22653.07.patch,
> HIVE-22653.08.patch, HIVE-22653.08.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> HIVE-7145 removed commons-lang - in favor of commons-lang3 - as a direct
> dependency, however a high number of files still refer to commons-lang, which
> is transitively brought in either way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22746) Make TypeCheckProcFactory generic
[
https://issues.apache.org/jira/browse/HIVE-22746?focusedWorklogId=376891=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-376891
]
ASF GitHub Bot logged work on HIVE-22746:
-
Author: ASF GitHub Bot
Created on: 24/Jan/20 16:11
Start Date: 24/Jan/20 16:11
Worklog Time Spent: 10m
Work Description: miklosgergely commented on pull request #887: HIVE-22746
URL: https://github.com/apache/hive/pull/887#discussion_r370714925
##
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/type/ExprFactory.java
##
@@ -0,0 +1,361 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.parse.type;
+
+import java.math.BigDecimal;
+import java.time.ZoneId;
+import java.util.List;
+import org.apache.hadoop.hive.ql.exec.ColumnInfo;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import
org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSubquerySemanticException;
+import org.apache.hadoop.hive.ql.parse.ASTNode;
+import org.apache.hadoop.hive.ql.parse.SemanticException;
+import org.apache.hadoop.hive.ql.plan.ExprNodeDesc;
+import org.apache.hadoop.hive.ql.plan.SubqueryType;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDF;
+import org.apache.hadoop.hive.serde2.typeinfo.PrimitiveTypeInfo;
+import org.apache.hadoop.hive.serde2.typeinfo.TypeInfo;
+import org.apache.hive.common.util.DateUtils;
+
+/**
+ * Generic expressions factory. Currently, the only implementation produces
+ * Hive {@link ExprNodeDesc}.
+ */
+public abstract class ExprFactory {
+
+ static final BigDecimal NANOS_PER_SEC_BD =
+ new BigDecimal(DateUtils.NANOS_PER_SEC);
+
+ /**
+ * Returns whether the input is an instance of the expression class.
+ */
+ protected abstract boolean isExprInstance(Object o);
+
+ /**
+ * Generates an expression from the input column. This may not necessarily
+ * be a column expression, e.g., if the column is a constant.
+ */
+ protected abstract T toExpr(ColumnInfo colInfo);
+
+ /* FIELD REFERENCES */
+ /**
+ * Returns whether the input object is a column reference expression.
+ */
+ protected abstract boolean isColumnRefExpr(Object o);
+
+ /**
+ * Creates column expression.
+ */
+ protected abstract T createColumnRefExpr(ColumnInfo colInfo);
+
+ /**
+ * Returns column name referenced by a column expression.
+ */
+ protected abstract String getColumnName(T expr);
+
+ /* CONSTANT EXPRESSIONS */
+ /**
+ * Returns whether the input expression is a constant expression.
+ */
+ protected abstract boolean isConstantExpr(Object o);
+
+ /**
+ * Returns whether all input expressions are constant expressions.
+ */
+ protected boolean isAllConstants(List exprs) {
+for (T expr : exprs) {
+ if (!isConstantExpr(expr)) {
+return false;
+ }
+}
+return true;
+ }
+
+ /**
+ * Returns whether the input expression is a struct of
+ * constant expressions (all of them).
+ */
+ protected abstract boolean isConstantStruct(T expr);
+
+ /**
+ * Creates a null constant expression with void type.
+ */
+ protected abstract T createNullConstantExpr();
+
+ /**
+ * Creates a boolean constant expression from input value.
+ */
+ protected abstract T createBooleanConstantExpr(String value);
+
+ /**
+ * Creates a bigint constant expression from input value.
+ */
+ protected abstract T createBigintConstantExpr(String value);
+
+ /**
+ * Creates a int constant expression from input value.
+ */
+ protected abstract T createIntConstantExpr(String value);
+
+ /**
+ * Creates a smallint constant expression from input value.
+ */
+ protected abstract T createSmallintConstantExpr(String value);
+
+ /**
+ * Creates a tinyint constant expression from input value.
+ */
+ protected abstract T createTinyintConstantExpr(String value);
+
+ /**
+ * Creates a float constant expression from input value.
+ */
+ protected abstract T createFloatConstantExpr(String value);
+
+ /**
+ * Creates a double constant expression from input value.
+ */
+ protected abstract T createDoubleConstantExpr(String
[jira] [Commented] (HIVE-22653) Remove commons-lang leftovers
[ https://issues.apache.org/jira/browse/HIVE-22653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023054#comment-17023054 ] Zoltan Haindrich commented on HIVE-22653: - +1 > Remove commons-lang leftovers > - > > Key: HIVE-22653 > URL: https://issues.apache.org/jira/browse/HIVE-22653 > Project: Hive > Issue Type: Bug >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22653.01.patch, HIVE-22653.01.patch, > HIVE-22653.02.patch, HIVE-22653.03.patch, HIVE-22653.04.patch, > HIVE-22653.04.patch, HIVE-22653.04.patch, HIVE-22653.05.patch, > HIVE-22653.06.patch, HIVE-22653.07.patch, HIVE-22653.07.patch, > HIVE-22653.08.patch, HIVE-22653.08.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > HIVE-7145 removed commons-lang - in favor of commons-lang3 - as a direct > dependency, however a high number of files still refer to commons-lang, which > is transitively brought in either way. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22772) Log opType and session level information for each operation
[ https://issues.apache.org/jira/browse/HIVE-22772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22772: -- Attachment: (was: HIVE-22772.1.patch) > Log opType and session level information for each operation > > > Key: HIVE-22772 > URL: https://issues.apache.org/jira/browse/HIVE-22772 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22772.1.patch > > > if would be useful to have this info available for issue troubleshooting. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22772) Log opType and session level information for each operation
[ https://issues.apache.org/jira/browse/HIVE-22772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22772: -- Attachment: HIVE-22772.1.patch > Log opType and session level information for each operation > > > Key: HIVE-22772 > URL: https://issues.apache.org/jira/browse/HIVE-22772 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22772.1.patch > > > if would be useful to have this info available for issue troubleshooting. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22746) Make TypeCheckProcFactory generic
[
https://issues.apache.org/jira/browse/HIVE-22746?focusedWorklogId=376877=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-376877
]
ASF GitHub Bot logged work on HIVE-22746:
-
Author: ASF GitHub Bot
Created on: 24/Jan/20 15:58
Start Date: 24/Jan/20 15:58
Worklog Time Spent: 10m
Work Description: miklosgergely commented on pull request #887: HIVE-22746
URL: https://github.com/apache/hive/pull/887#discussion_r370708452
##
File path: ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java
##
@@ -1001,17 +1002,10 @@ public static boolean isSame(List first,
List second
// Given an expression this method figures out if the type for the
expression belongs to string group
// e.g. (String, Char, Varchar etc)
public static boolean isStringType(ExprNodeDesc expr) {
-TypeInfo typeInfo = expr.getTypeInfo();
-if (typeInfo.getCategory() == ObjectInspector.Category.PRIMITIVE) {
- PrimitiveObjectInspector.PrimitiveCategory primitiveCategory =
((PrimitiveTypeInfo) typeInfo).getPrimitiveCategory();
- if
(PrimitiveObjectInspectorUtils.getPrimitiveGrouping(primitiveCategory) ==
- PrimitiveObjectInspectorUtils.PrimitiveGrouping.STRING_GROUP) {
-return true;
- }
-}
-return false;
+return TypeCheckProcFactory.isStringType(expr.getTypeInfo());
Review comment:
ExprNodeDescUtils.isStringType is only redirecting to
TypeCheckProcFactory.isStringType, therefore it can be removed, and it's
callers may call TypeCheckProcFactory.isStringType directly.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 376877)
Time Spent: 20m (was: 10m)
> Make TypeCheckProcFactory generic
> -
>
> Key: HIVE-22746
> URL: https://issues.apache.org/jira/browse/HIVE-22746
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22746.01.patch, HIVE-22746.01.patch,
> HIVE-22746.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {{TypeCheckProcFactory}} is responsible for processing AST nodes and
> generating ExprNode objects from them. When we generate the expressions for
> Calcite planning, we go through a {{AST node -> ExprNode -> RexNode}}
> transformation. We would like to avoid the overhead of going through the
> ExprNode, and thus generate directly the RexNode from the AST.
> To do that, the first step is to make {{TypeCheckProcFactory}} generic, so it
> can receive an expression factory and create expressions in different realms.
> For the time being, the only factory implementation is the ExprNode factory.
> Thus, this patch focuses mainly on refactoring {{TypeCheckProcFactory}}
> without breaking anything that is already working.
> In a follow-up patch, we will create a {{RexNode}} factory and use it when we
> parse the query in CalcitePlanner.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22723) SHOW CREATE TABLE omits backtics, results in syntax error
[
https://issues.apache.org/jira/browse/HIVE-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-22723:
Description:
{code}
CREATE EXTERNAL TABLE `alerts1`(
`_id` string COMMENT 'from deserializer', `org` string COMMENT 'from
deserializer', `collector` string COMMENT 'from deserializer', `source` string
COMMENT 'from deserializer', `entitytype` string COMMENT 'from deserializer',
`entity` string COMMENT 'from deserializer', `rule` string COMMENT 'from
deserializer', `findingdescid` string COMMENT 'from deserializer',
`details`
struct>,findingtype:string,producttype:string,entityname:string,extra:struct,entityparams:struct,version:string,build:string>>
COMMENT 'from deserializer', `seenat` array>
COMMENT 'from deserializer', `execution` array>
COMMENT 'from deserializer', `lastmodified` struct<__numberint:string> COMMENT
'from deserializer'
) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' WITH SERDEPROPERTIES (
'paths'='_id,collector,details,entity,entityType,execution,findingDescId,lastModified,org,rule,seenAt,source')
STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' LOCATION
's3://redshift-poc-ay/alerts' TBLPROPERTIES (
'CrawlerSchemaDeserializerVersion'='1.0',
'CrawlerSchemaSerializerVersion'='1.0',
'UPDATED_BY_CRAWLER'='redshift-poc-alerts-ay', 'averageRecordSize'='9619',
'classification'='json', 'compressionType'='none', 'objectCount'='1',
'recordCount'='21028', 'sizeKey'='202270843',
'transient_lastDdlTime'='1578946186', 'typeOfData'='file')
{code}
__numberint, __numberlong columns are not generated enclosed in backticks
even though they were explicitly created with them. Therefore, the generated
DDL is syntactically incorrect.
was:CREATE EXTERNAL TABLE `alerts1`( `_id` string COMMENT 'from
deserializer', `org` string COMMENT 'from deserializer', `collector` string
COMMENT 'from deserializer', `source` string COMMENT 'from deserializer',
`entitytype` string COMMENT 'from deserializer', `entity` string COMMENT 'from
deserializer', `rule` string COMMENT 'from deserializer', `findingdescid`
string COMMENT 'from deserializer', `details`
struct>,findingtype:string,producttype:string,entityname:string,extra:struct,entityparams:struct,version:string,build:string>>
COMMENT 'from deserializer', `seenat` array>
COMMENT 'from deserializer', `execution` array>
COMMENT 'from deserializer', `lastmodified` struct<__numberint:string> COMMENT
'from deserializer') ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' WITH
SERDEPROPERTIES (
'paths'='_id,collector,details,entity,entityType,execution,findingDescId,lastModified,org,rule,seenAt,source')
STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' LOCATION
's3://redshift-poc-ay/alerts' TBLPROPERTIES (
'CrawlerSchemaDeserializerVersion'='1.0',
'CrawlerSchemaSerializerVersion'='1.0',
'UPDATED_BY_CRAWLER'='redshift-poc-alerts-ay', 'averageRecordSize'='9619',
'classification'='json', 'compressionType'='none', 'objectCount'='1',
'recordCount'='21028', 'sizeKey'='202270843',
'transient_lastDdlTime'='1578946186', 'typeOfData'='file') __numberint,
__numberlong columns are not generated enclosed in backticks even though they
were explicitly created with them. Therefore, the generated DDL is
syntactically incorrect.
> SHOW CREATE TABLE omits backtics, results in syntax error
> -
>
> Key: HIVE-22723
> URL: https://issues.apache.org/jira/browse/HIVE-22723
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Environment: Run SHOW CREATE TABLE via AWS Athena
>Reporter: Alex Yankelevich
>Assignee: Miklos Gergely
>Priority: Minor
> Attachments: HIVE-22723.01.patch
>
>
> {code}
> CREATE EXTERNAL TABLE `alerts1`(
> `_id` string COMMENT 'from deserializer', `org` string COMMENT 'from
> deserializer', `collector` string COMMENT 'from deserializer', `source`
> string COMMENT 'from deserializer', `entitytype` string COMMENT 'from
> deserializer', `entity` string COMMENT 'from deserializer', `rule` string
> COMMENT 'from deserializer', `findingdescid` string COMMENT 'from
> deserializer',
> `details`
> struct>,findingtype:string,producttype:string,entityname:string,extra:struct,entityparams:struct,version:string,build:string>>
> COMMENT 'from deserializer', `seenat` array>
> COMMENT 'from deserializer', `execution` array>
> COMMENT 'from deserializer', `lastmodified` struct<__numberint:string>
> COMMENT 'from deserializer'
> ) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' WITH SERDEPROPERTIES
> (
>
[jira] [Commented] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023026#comment-17023026
]
Hive QA commented on HIVE-22538:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
59s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
44s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} ql: The patch generated 0 new + 499 unchanged - 4
fixed = 499 total (was 503) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} itests/hive-unit: The patch generated 0 new + 170
unchanged - 2 fixed = 170 total (was 172) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20320/dev-support/hive-personality.sh
|
| git revision | master / 2d444fa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20320/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.3.patch,
> HIVE-22538.4.patch, HIVE-22538.5.patch, HIVE-22538.6.patch,
> HIVE-22538.6.patch, HIVE-22538.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when
[jira] [Updated] (HIVE-22518) SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
[ https://issues.apache.org/jira/browse/HIVE-22518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22518: Attachment: HIVE-22518.06.patch > SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests > > > Key: HIVE-22518 > URL: https://issues.apache.org/jira/browse/HIVE-22518 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22518.01.patch, HIVE-22518.01.patch, > HIVE-22518.02.patch, HIVE-22518.02.patch, HIVE-22518.03.patch, > HIVE-22518.03.patch, HIVE-22518.04.patch, HIVE-22518.05.patch, > HIVE-22518.05.patch, HIVE-22518.06.patch, HIVE-22518.06.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22723) SHOW CREATE TABLE omits backtics, results in syntax error
[ https://issues.apache.org/jira/browse/HIVE-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22723: -- Attachment: HIVE-22723.01.patch > SHOW CREATE TABLE omits backtics, results in syntax error > - > > Key: HIVE-22723 > URL: https://issues.apache.org/jira/browse/HIVE-22723 > Project: Hive > Issue Type: Bug > Components: Hive > Environment: Run SHOW CREATE TABLE via AWS Athena >Reporter: Alex Yankelevich >Assignee: Miklos Gergely >Priority: Minor > Attachments: HIVE-22723.01.patch > > > CREATE EXTERNAL TABLE `alerts1`( `_id` string COMMENT 'from deserializer', > `org` string COMMENT 'from deserializer', `collector` string COMMENT 'from > deserializer', `source` string COMMENT 'from deserializer', `entitytype` > string COMMENT 'from deserializer', `entity` string COMMENT 'from > deserializer', `rule` string COMMENT 'from deserializer', `findingdescid` > string COMMENT 'from deserializer', `details` > struct>,findingtype:string,producttype:string,entityname:string,extra:struct,entityparams:struct,version:string,build:string>> > COMMENT 'from deserializer', `seenat` array> > COMMENT 'from deserializer', `execution` array> > COMMENT 'from deserializer', `lastmodified` struct<__numberint:string> > COMMENT 'from deserializer') ROW FORMAT SERDE > 'org.openx.data.jsonserde.JsonSerDe' WITH SERDEPROPERTIES ( > 'paths'='_id,collector,details,entity,entityType,execution,findingDescId,lastModified,org,rule,seenAt,source') > STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' > OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' > LOCATION 's3://redshift-poc-ay/alerts' TBLPROPERTIES ( > 'CrawlerSchemaDeserializerVersion'='1.0', > 'CrawlerSchemaSerializerVersion'='1.0', > 'UPDATED_BY_CRAWLER'='redshift-poc-alerts-ay', 'averageRecordSize'='9619', > 'classification'='json', 'compressionType'='none', 'objectCount'='1', > 'recordCount'='21028', 'sizeKey'='202270843', > 'transient_lastDdlTime'='1578946186', 'typeOfData'='file') __numberint, > __numberlong columns are not generated enclosed in backticks even though they > were explicitly created with them. Therefore, the generated DDL is > syntactically incorrect. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22723) SHOW CREATE TABLE omits backtics, results in syntax error
[ https://issues.apache.org/jira/browse/HIVE-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22723: -- Status: Patch Available (was: Open) > SHOW CREATE TABLE omits backtics, results in syntax error > - > > Key: HIVE-22723 > URL: https://issues.apache.org/jira/browse/HIVE-22723 > Project: Hive > Issue Type: Bug > Components: Hive > Environment: Run SHOW CREATE TABLE via AWS Athena >Reporter: Alex Yankelevich >Assignee: Miklos Gergely >Priority: Minor > Attachments: HIVE-22723.01.patch > > > CREATE EXTERNAL TABLE `alerts1`( `_id` string COMMENT 'from deserializer', > `org` string COMMENT 'from deserializer', `collector` string COMMENT 'from > deserializer', `source` string COMMENT 'from deserializer', `entitytype` > string COMMENT 'from deserializer', `entity` string COMMENT 'from > deserializer', `rule` string COMMENT 'from deserializer', `findingdescid` > string COMMENT 'from deserializer', `details` > struct>,findingtype:string,producttype:string,entityname:string,extra:struct,entityparams:struct,version:string,build:string>> > COMMENT 'from deserializer', `seenat` array> > COMMENT 'from deserializer', `execution` array> > COMMENT 'from deserializer', `lastmodified` struct<__numberint:string> > COMMENT 'from deserializer') ROW FORMAT SERDE > 'org.openx.data.jsonserde.JsonSerDe' WITH SERDEPROPERTIES ( > 'paths'='_id,collector,details,entity,entityType,execution,findingDescId,lastModified,org,rule,seenAt,source') > STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' > OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' > LOCATION 's3://redshift-poc-ay/alerts' TBLPROPERTIES ( > 'CrawlerSchemaDeserializerVersion'='1.0', > 'CrawlerSchemaSerializerVersion'='1.0', > 'UPDATED_BY_CRAWLER'='redshift-poc-alerts-ay', 'averageRecordSize'='9619', > 'classification'='json', 'compressionType'='none', 'objectCount'='1', > 'recordCount'='21028', 'sizeKey'='202270843', > 'transient_lastDdlTime'='1578946186', 'typeOfData'='file') __numberint, > __numberlong columns are not generated enclosed in backticks even though they > were explicitly created with them. Therefore, the generated DDL is > syntactically incorrect. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue
[
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023001#comment-17023001
]
Hive QA commented on HIVE-22712:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991701/HIVE-22712.07.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17946 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20319/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20319/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20319/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991701 - PreCommit-HIVE-Build
> ReExec Driver execute submit the query in default queue irrespective of user
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
> Issue Type: Bug
> Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch,
> HIVE-22712.03.patch, HIVE-22712.04.patch, HIVE-22712.05.patch,
> HIVE-22712.06.patch, HIVE-22712.06.patch, HIVE-22712.07.patch,
> HIVE-22712.patch
>
>
> we unset the queue name intentionally in
> TezSessionState#startSessionAndContainers,
> as a result reexec create a new session in the default queue and create a
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session
> level.
> I could not find a better way of setting the queue name (I am open for the
> suggestion here) since it can create a conflict with the Global queue name
> vs user-defined queue that's why setting while initialization of
> ReExecutionOverlayPlugin.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22765) Quote table names in CompactionTxnHandler
[
https://issues.apache.org/jira/browse/HIVE-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Chovan updated HIVE-22765:
-
Status: Patch Available (was: Open)
> Quote table names in CompactionTxnHandler
> -
>
> Key: HIVE-22765
> URL: https://issues.apache.org/jira/browse/HIVE-22765
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Laszlo Pinter
>Assignee: Zoltan Chovan
>Priority: Major
> Attachments: HIVE-22765.patch
>
>
> When I run a metastore backed up by a postgres db, I get the following
> exception:
> {code:java}
> MetaException(message:Unable to connect to transaction database
> org.postgresql.util.PSQLException: ERROR: relation "compaction_queue" does
> not existMetaException(message:Unable to connect to transaction database
> org.postgresql.util.PSQLException: ERROR: relation "compaction_queue" does
> not exist Position: 83 at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2468)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2211)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:309)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:446) at
> org.postgresql.jdbc.PgStatement.execute(PgStatement.java:370) at
> org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:311) at
> org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:297) at
> org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:274) at
> org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:225) at
> org.apache.hive.com.zaxxer.hikari.pool.ProxyStatement.executeQuery(ProxyStatement.java:108)
> at
> org.apache.hive.com.zaxxer.hikari.pool.HikariProxyStatement.executeQuery(HikariProxyStatement.java)
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findNextToCompact(CompactionTxnHandler.java:157)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.find_next_compact(HiveMetaStore.java:8061)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy28.find_next_compact(Unknown Source) at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.findNextCompact(HiveMetaStoreClient.java:4046)
> at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:208)
> at com.sun.proxy.$Proxy29.findNextCompact(Unknown Source) at
> org.apache.hadoop.hive.ql.txn.compactor.Worker.run(Worker.java:102){code}
> This is due to missing quotes around table names.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22772) Log opType and session level information for each operation
[ https://issues.apache.org/jira/browse/HIVE-22772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022972#comment-17022972 ] Peter Vary commented on HIVE-22772: --- +1 pending tests > Log opType and session level information for each operation > > > Key: HIVE-22772 > URL: https://issues.apache.org/jira/browse/HIVE-22772 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22772.1.patch > > > if would be useful to have this info available for issue troubleshooting. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22772) Log opType and session level information for each operation
[ https://issues.apache.org/jira/browse/HIVE-22772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22772: -- Assignee: Denys Kuzmenko Status: Patch Available (was: Open) > Log opType and session level information for each operation > > > Key: HIVE-22772 > URL: https://issues.apache.org/jira/browse/HIVE-22772 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22772.1.patch > > > if would be useful to have this info available for issue troubleshooting. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22772) Log opType and session level information for each operation
[ https://issues.apache.org/jira/browse/HIVE-22772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22772: -- Attachment: HIVE-22772.1.patch > Log opType and session level information for each operation > > > Key: HIVE-22772 > URL: https://issues.apache.org/jira/browse/HIVE-22772 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22772.1.patch > > > if would be useful to have this info available for issue troubleshooting. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22772) Log opType and session level information for each operation
[ https://issues.apache.org/jira/browse/HIVE-22772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22772: -- Description: if would be useful to have this info available for issue troubleshooting. > Log opType and session level information for each operation > > > Key: HIVE-22772 > URL: https://issues.apache.org/jira/browse/HIVE-22772 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Priority: Major > > if would be useful to have this info available for issue troubleshooting. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22765) Quote table names in CompactionTxnHandler
[
https://issues.apache.org/jira/browse/HIVE-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Chovan updated HIVE-22765:
-
Attachment: HIVE-22765.patch
> Quote table names in CompactionTxnHandler
> -
>
> Key: HIVE-22765
> URL: https://issues.apache.org/jira/browse/HIVE-22765
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Laszlo Pinter
>Assignee: Zoltan Chovan
>Priority: Major
> Attachments: HIVE-22765.patch
>
>
> When I run a metastore backed up by a postgres db, I get the following
> exception:
> {code:java}
> MetaException(message:Unable to connect to transaction database
> org.postgresql.util.PSQLException: ERROR: relation "compaction_queue" does
> not existMetaException(message:Unable to connect to transaction database
> org.postgresql.util.PSQLException: ERROR: relation "compaction_queue" does
> not exist Position: 83 at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2468)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2211)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:309)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:446) at
> org.postgresql.jdbc.PgStatement.execute(PgStatement.java:370) at
> org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:311) at
> org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:297) at
> org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:274) at
> org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:225) at
> org.apache.hive.com.zaxxer.hikari.pool.ProxyStatement.executeQuery(ProxyStatement.java:108)
> at
> org.apache.hive.com.zaxxer.hikari.pool.HikariProxyStatement.executeQuery(HikariProxyStatement.java)
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findNextToCompact(CompactionTxnHandler.java:157)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.find_next_compact(HiveMetaStore.java:8061)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy28.find_next_compact(Unknown Source) at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.findNextCompact(HiveMetaStoreClient.java:4046)
> at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:208)
> at com.sun.proxy.$Proxy29.findNextCompact(Unknown Source) at
> org.apache.hadoop.hive.ql.txn.compactor.Worker.run(Worker.java:102){code}
> This is due to missing quotes around table names.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue
[
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022962#comment-17022962
]
Hive QA commented on HIVE-22712:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
4s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20319/dev-support/hive-personality.sh
|
| git revision | master / 2d444fa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20319/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ReExec Driver execute submit the query in default queue irrespective of user
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
> Issue Type: Bug
> Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch,
> HIVE-22712.03.patch, HIVE-22712.04.patch, HIVE-22712.05.patch,
> HIVE-22712.06.patch, HIVE-22712.06.patch, HIVE-22712.07.patch,
> HIVE-22712.patch
>
>
> we unset the queue name intentionally in
> TezSessionState#startSessionAndContainers,
> as a result reexec create a new session in the default queue and create a
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session
> level.
> I could not find a better way of setting the queue name (I am open for the
> suggestion here) since it can create a conflict with the Global queue name
> vs user-defined queue that's why setting while initialization of
> ReExecutionOverlayPlugin.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22393) HiveStreamingConnection: Exception in beginTransaction causes AbstractRecordWriter to throw NPE, covering up real exception
[
https://issues.apache.org/jira/browse/HIVE-22393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022936#comment-17022936
]
Hive QA commented on HIVE-22393:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991694/HIVE-22393.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17936 tests
executed
*Failed tests:*
{noformat}
TestJdbcWithMiniLlapArrow - did not produce a TEST-*.xml file (likely timed
out) (batchId=288)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testNoBucketsDP (batchId=340)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20318/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20318/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20318/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991694 - PreCommit-HIVE-Build
> HiveStreamingConnection: Exception in beginTransaction causes
> AbstractRecordWriter to throw NPE, covering up real exception
> ---
>
> Key: HIVE-22393
> URL: https://issues.apache.org/jira/browse/HIVE-22393
> Project: Hive
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 3.1.2
>Reporter: Alex Parrill
>Assignee: Matt Burgess
>Priority: Major
> Attachments: HIVE-22393.1.patch
>
>
> Getting an error in my code that does some basic stuff with
> {{HiveStreamingConnection}}:
> {code}
> 19/10/23 15:12:44 ERROR yarn_logger.Main: Thread worker-0 uncaught exception
> java.lang.RuntimeException: org.apache.hive.streaming.StreamingException:
> Unable to close
> at com.datto.yarn_logger.WorkerThread.run(WorkerThread.java:51)
> Caused by: org.apache.hive.streaming.StreamingException: Unable to close
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.close(HiveStreamingConnection.java:973)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.markDead(HiveStreamingConnection.java:833)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.(HiveStreamingConnection.java:677)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.(HiveStreamingConnection.java:596)
> at
> org.apache.hive.streaming.HiveStreamingConnection.createNewTransactionBatch(HiveStreamingConnection.java:485)
> at
> org.apache.hive.streaming.HiveStreamingConnection.beginNextTransaction(HiveStreamingConnection.java:466)
> at
> org.apache.hive.streaming.HiveStreamingConnection.beginTransaction(HiveStreamingConnection.java:507)
> at com.datto.yarn_logger.WorkerThread.run(WorkerThread.java:49)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hive.streaming.AbstractRecordWriter.logStats(AbstractRecordWriter.java:547)
> at
> org.apache.hive.streaming.AbstractRecordWriter.close(AbstractRecordWriter.java:352)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.closeImpl(HiveStreamingConnection.java:979)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.close(HiveStreamingConnection.java:970)
> ... 7 more
> {code}
> Digging through the stack trace... {{TransactionBatch}} will try to catch
> exception in its constructor, and calls the close method if an exception is
> thrown (which is definitely happening; that's line 677 in
> {{HiveStreamingConnection.java}}). This eventually calls
> {{AbstractRecordWriter::close}} which, calls {{logStats}}, which tries to use
> {{heapMemoryMonitor}}, which is null because presumably {{init}} had never
> been called on the writer yet.
> Easy fix would to just check in {{logStats}} if {{heapMemoryMonitor}} is
> null, and do the same thing if the method it calls returns null in it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22126) hive-exec packaging should shade guava
[ https://issues.apache.org/jira/browse/HIVE-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022752#comment-17022752 ] Eugene Chung edited comment on HIVE-22126 at 1/24/20 1:23 PM: -- [^HIVE-22126.03.patch] * The calcite, calcite-druid, and calcite avatica modules are included to shade their guava usages. * Some unnecessary guava dependencies are removed. was (Author: euigeun_chung): [^HIVE-22126.03.patch] * The calcite, calcite-druid, calcite avatica modules are included to shade guava of them. * Remove some unnecessary guava dependencies > hive-exec packaging should shade guava > -- > > Key: HIVE-22126 > URL: https://issues.apache.org/jira/browse/HIVE-22126 > Project: Hive > Issue Type: Bug >Reporter: Vihang Karajgaonkar >Assignee: Eugene Chung >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22126.01.patch, HIVE-22126.02.patch, > HIVE-22126.03.patch > > > The ql/pom.xml includes complete guava library into hive-exec.jar > https://github.com/apache/hive/blob/master/ql/pom.xml#L990 This causes a > problems for downstream clients of hive which have hive-exec.jar in their > classpath since they are pinned to the same guava version as that of hive. > We should shade guava classes so that other components which depend on > hive-exec can independently use a different version of guava as needed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22723) SHOW CREATE TABLE omits backtics, results in syntax error
[ https://issues.apache.org/jira/browse/HIVE-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely reassigned HIVE-22723: - Assignee: Miklos Gergely > SHOW CREATE TABLE omits backtics, results in syntax error > - > > Key: HIVE-22723 > URL: https://issues.apache.org/jira/browse/HIVE-22723 > Project: Hive > Issue Type: Bug > Components: Hive > Environment: Run SHOW CREATE TABLE via AWS Athena >Reporter: Alex Yankelevich >Assignee: Miklos Gergely >Priority: Minor > > CREATE EXTERNAL TABLE `alerts1`( `_id` string COMMENT 'from deserializer', > `org` string COMMENT 'from deserializer', `collector` string COMMENT 'from > deserializer', `source` string COMMENT 'from deserializer', `entitytype` > string COMMENT 'from deserializer', `entity` string COMMENT 'from > deserializer', `rule` string COMMENT 'from deserializer', `findingdescid` > string COMMENT 'from deserializer', `details` > struct>,findingtype:string,producttype:string,entityname:string,extra:struct,entityparams:struct,version:string,build:string>> > COMMENT 'from deserializer', `seenat` array> > COMMENT 'from deserializer', `execution` array> > COMMENT 'from deserializer', `lastmodified` struct<__numberint:string> > COMMENT 'from deserializer') ROW FORMAT SERDE > 'org.openx.data.jsonserde.JsonSerDe' WITH SERDEPROPERTIES ( > 'paths'='_id,collector,details,entity,entityType,execution,findingDescId,lastModified,org,rule,seenAt,source') > STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' > OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' > LOCATION 's3://redshift-poc-ay/alerts' TBLPROPERTIES ( > 'CrawlerSchemaDeserializerVersion'='1.0', > 'CrawlerSchemaSerializerVersion'='1.0', > 'UPDATED_BY_CRAWLER'='redshift-poc-alerts-ay', 'averageRecordSize'='9619', > 'classification'='json', 'compressionType'='none', 'objectCount'='1', > 'recordCount'='21028', 'sizeKey'='202270843', > 'transient_lastDdlTime'='1578946186', 'typeOfData'='file') __numberint, > __numberlong columns are not generated enclosed in backticks even though they > were explicitly created with them. Therefore, the generated DDL is > syntactically incorrect. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22393) HiveStreamingConnection: Exception in beginTransaction causes AbstractRecordWriter to throw NPE, covering up real exception
[
https://issues.apache.org/jira/browse/HIVE-22393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022895#comment-17022895
]
Hive QA commented on HIVE-22393:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20318/dev-support/hive-personality.sh
|
| git revision | master / 2d444fa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: streaming U: streaming |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20318/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> HiveStreamingConnection: Exception in beginTransaction causes
> AbstractRecordWriter to throw NPE, covering up real exception
> ---
>
> Key: HIVE-22393
> URL: https://issues.apache.org/jira/browse/HIVE-22393
> Project: Hive
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 3.1.2
>Reporter: Alex Parrill
>Assignee: Matt Burgess
>Priority: Major
> Attachments: HIVE-22393.1.patch
>
>
> Getting an error in my code that does some basic stuff with
> {{HiveStreamingConnection}}:
> {code}
> 19/10/23 15:12:44 ERROR yarn_logger.Main: Thread worker-0 uncaught exception
> java.lang.RuntimeException: org.apache.hive.streaming.StreamingException:
> Unable to close
> at com.datto.yarn_logger.WorkerThread.run(WorkerThread.java:51)
> Caused by: org.apache.hive.streaming.StreamingException: Unable to close
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.close(HiveStreamingConnection.java:973)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.markDead(HiveStreamingConnection.java:833)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.(HiveStreamingConnection.java:677)
> at
> org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.(HiveStreamingConnection.java:596)
> at
> org.apache.hive.streaming.HiveStreamingConnection.createNewTransactionBatch(HiveStreamingConnection.java:485)
> at
>
[jira] [Commented] (HIVE-22756) Drop XAttrs options for distcp when dest is a blobstore
[
https://issues.apache.org/jira/browse/HIVE-22756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022887#comment-17022887
]
Hive QA commented on HIVE-22756:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991690/HIVE-22756.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17946 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[vector_outer_join4]
(batchId=200)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20317/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20317/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20317/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991690 - PreCommit-HIVE-Build
> Drop XAttrs options for distcp when dest is a blobstore
> ---
>
> Key: HIVE-22756
> URL: https://issues.apache.org/jira/browse/HIVE-22756
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-22756.1.patch, HIVE-22756.2.patch
>
>
> Drop XAttrs options for distcp when dest is a blobstore
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22756) Drop XAttrs options for distcp when dest is a blobstore
[
https://issues.apache.org/jira/browse/HIVE-22756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022864#comment-17022864
]
Hive QA commented on HIVE-22756:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
16s{color} | {color:blue} shims/common in master has 6 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
20s{color} | {color:blue} shims/0.23 in master has 7 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
9s{color} | {color:red} shims/common: The patch generated 1 new + 93 unchanged
- 1 fixed = 94 total (was 94) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} shims/0.23: The patch generated 0 new + 63 unchanged
- 2 fixed = 63 total (was 65) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20317/dev-support/hive-personality.sh
|
| git revision | master / 2d444fa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20317/yetus/diff-checkstyle-shims_common.txt
|
| modules | C: shims/common shims/0.23 common U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20317/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Drop XAttrs options for distcp when dest is a blobstore
> ---
>
> Key: HIVE-22756
> URL: https://issues.apache.org/jira/browse/HIVE-22756
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-22756.1.patch, HIVE-22756.2.patch
>
>
> Drop XAttrs options for distcp when dest is a blobstore
--
This
[jira] [Work logged] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?focusedWorklogId=376793=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-376793
]
ASF GitHub Bot logged work on HIVE-22771:
-
Author: ASF GitHub Bot
Created on: 24/Jan/20 10:32
Start Date: 24/Jan/20 10:32
Worklog Time Spent: 10m
Work Description: Shivamohan07 commented on pull request #889:
Correcting regex to de-scratchify final partition location HIVE-22771
URL: https://github.com/apache/hive/pull/889
Correcting regex to de-scratchify final partition location HIVE-22771
Update FileOutputCommitterContainer.java
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 376793)
Remaining Estimate: 0h
Time Spent: 10m
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> +In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And,
> in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#FF}\\d\\.?\\d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-22771:
--
Labels: pull-request-available (was: )
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Priority: Critical
> Labels: pull-request-available
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> +In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And,
> in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#FF}\\d\\.?\\d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22744) TezTask for the vertex with more than one outedge should have proportional sort memory
[
https://issues.apache.org/jira/browse/HIVE-22744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022843#comment-17022843
]
Hive QA commented on HIVE-22744:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991688/HIVE-22744.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17945 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20316/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20316/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20316/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991688 - PreCommit-HIVE-Build
> TezTask for the vertex with more than one outedge should have proportional
> sort memory
> --
>
> Key: HIVE-22744
> URL: https://issues.apache.org/jira/browse/HIVE-22744
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-22744.1.patch, HIVE-22744.2.patch
>
>
> TezTask for the vertex with more than one outedge should have proportional
> sort memory
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22767) beeline doesn't parse semicolons in comments properly
[
https://issues.apache.org/jira/browse/HIVE-22767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Matyus updated HIVE-22767:
-
Attachment: HIVE-22767.02.patch
> beeline doesn't parse semicolons in comments properly
> -
>
> Key: HIVE-22767
> URL: https://issues.apache.org/jira/browse/HIVE-22767
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Zoltan Matyus
>Assignee: Zoltan Matyus
>Priority: Major
> Attachments: HIVE-22767.01.patch, HIVE-22767.02.patch
>
>
> HIVE-12646 fixed the handling of semicolons in quoted strings, but leaves the
> problem of semicolons in comments. E.g. with beeline connected to any
> database...
> this works: {code:sql}select 1; select /* */ 2; select /* */ 3;{code}
> this doesn't work: {code:sql}select 1; select /* ; */ 2; select /* ; */
> 3;{code}
> This has been fixed and reintroduced before (possibly multiple times).
> Ideally, there should be a single utility method somewhere to separate
> comments, strings and commands -- with the proper testing in place (q files).
> However, I'm trying to make this fix back-portable, so a light touch is
> needed. I'm focusing on beeline for now, and only writing (very thorough)
> unit tests, as I cannot exclude any new q files from TestCliDriver (which
> would break, since it's using a different parsing method).
>
> P.S. excerpt of the error message:
> {noformat}
> 0: jdbc:hive2://...> select 1; select /* ; */ 2; select /* ; */ 3;
> INFO : Compiling command(queryId=...): select 1
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0,
> type:int, comment:null)], properties:null)
> INFO : Completed compiling command(queryId=...); Time taken: 0.38 seconds
> INFO : Executing command(queryId=...): select 1
> INFO : Completed executing command(queryId=...); Time taken: 0.004 seconds
> INFO : OK
> +--+
> | _c0 |
> +--+
> | 1|
> +--+
> 1 row selected (2.007 seconds)
> INFO : Compiling command(queryId=...): select /*
> ERROR : FAILED: ParseException line 1:9 cannot recognize input near ''
> '' '' in select clause
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 cannot recognize
> input near '' '' '' in select clause
> at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:233)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:79)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:72)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:598)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1505)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1452)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1447)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at ...
> {noformat}
>
> Similarly, the following query also fails:
> {code:sql}select /* ' */ 1; select /* ' */ 2;{code}
> I suspect line comments are also not handled properly but I cannot reproduce
> this in interactive beeline...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Status: In Progress (was: Patch Available) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Attachment: HIVE-22736.patch Status: Patch Available (was: In Progress) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22769) Incorrect query results and query failure during split generation for compressed text files
[
https://issues.apache.org/jira/browse/HIVE-22769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marta Kuczora reassigned HIVE-22769:
Assignee: Marta Kuczora
> Incorrect query results and query failure during split generation for
> compressed text files
> ---
>
> Key: HIVE-22769
> URL: https://issues.apache.org/jira/browse/HIVE-22769
> Project: Hive
> Issue Type: Bug
> Components: File Formats
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Chiran Ravani
>Assignee: Marta Kuczora
>Priority: Critical
> Attachments: testcase1.csv.bz2, testcase2.csv.bz2
>
>
> Hive Query produces incorrect results when data is in text format and
> compressed and for certain data the query fails during split generation.
> This behavior is seen when skip.header.line.count and skip.footer.line.count
> are set for table.
> Case 1: Select count/aggregate query produces Incorrect row counts/displays
> all rows (when hive.fetch.task.conversion=none)
> Steps to reproduce:
> 1. Create table as below
> {code}
> CREATE EXTERNAL TABLE `testcase1`(id int, name string) ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.OpenCSVSerde' LOCATION '/user/hive/testcase1'
> TBLPROPERTIES ("skip.header.line.count"="1", "skip.footer.line.count"="1");
> {code}
> 2. Upload attached testcase1.csv.bz2 file to /user/hive/testcase1
> 3. Run count(*) on table.
> {code}
> > select * from testcase1;
> INFO : Compiling
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f):
> select * from testcase1
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:testcase1.id, type:string,
> comment:null), FieldSchema(name:testcase1.name, type:string, comment:null)],
> properties:null)
> INFO : Completed compiling
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f);
> Time taken: 0.07 seconds
> INFO : Executing
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f):
> select * from testcase1
> INFO : Completed executing
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f);
> Time taken: 0.007 seconds
> INFO : OK
> +---+-+
> | testcase1.id | testcase1.name |
> +---+-+
> | 2 | 2019-12-31 |
> +---+-+
> 1 row selected (0.111 seconds)
> > select count(*) from testcase1
> INFO : Compiling
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7):
> select count(*) from testcase1
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0,
> type:bigint, comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7);
> Time taken: 0.073 seconds
> INFO : Executing
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7):
> select count(*) from testcase1
> INFO : Query ID = hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in serial mode
> INFO : Subscribed to counters: [] for queryId:
> hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7
> INFO : Session is already open
> INFO : Dag name: select count(*) from testcase1 (Stage-1)
> INFO : Status: Running (Executing on YARN cluster with App id
> application_1579811438512_0046)
> .
> .
> .
> INFO : Completed executing
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7);
> Time taken: 4.228 seconds
> INFO : OK
> +--+
> | _c0 |
> +--+
> | 3|
> +--+
> 1 row selected (4.335 seconds)
> {code}
> Case 2: Select count/aggregate query fails with java.lang.ClassCastException:
> java.io.PushbackInputStream cannot be cast to org.apache.hadoop.fs.Seekable
> The issue is only seen when there is a space in a field (eg:- "3,2019-12-31
> 01" second column has a space)
> Steps to reproduce:
> 1. Create table as below
> {code}
> CREATE EXTERNAL TABLE `testcase2`(id int, name string) ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.OpenCSVSerde' LOCATION '/user/hive/testcase2'
> TBLPROPERTIES ("skip.header.line.count"="1", "skip.footer.line.count"="1");
> {code}
> 2. Upload attached testcase2.csv.bz2 file to /user/hive/testcase2
> 3. Run count(*) on table.
> {code}
> 0: > select * from testcase2;
> INFO : Compiling
> command(queryId=hive_20200124053159_5d8ce56a-183d-4359-a147-bd470d82e134):
> select * from testcase2
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
>
[jira] [Updated] (HIVE-22729) Provide a failure reason for failed compactions
[
https://issues.apache.org/jira/browse/HIVE-22729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Pinter updated HIVE-22729:
-
Attachment: HIVE-22729.09.patch
> Provide a failure reason for failed compactions
> ---
>
> Key: HIVE-22729
> URL: https://issues.apache.org/jira/browse/HIVE-22729
> Project: Hive
> Issue Type: Improvement
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-22729.01.patch, HIVE-22729.02.patch,
> HIVE-22729.03.patch, HIVE-22729.04.patch, HIVE-22729.05.patch,
> HIVE-22729.06.patch, HIVE-22729.07.patch, HIVE-22729.08.patch,
> HIVE-22729.09.patch
>
>
> We should provide a compaction failure reason as easily accessible as
> possible. Like in the result of the {{SHOW COMPACTIONS}} command.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
