[jira] [Updated] (HIVE-19488) enable CM root based on db parameter, identifying a db as source of replication.

[mahesh kumar behera (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mahesh kumar behera updated HIVE-19488:
---
Attachment: HIVE-19488.02.patch

> enable CM root based on db parameter, identifying a db as source of 
> replication.
> 
>
> Key: HIVE-19488
> URL: https://issues.apache.org/jira/browse/HIVE-19488
> Project: Hive
>  Issue Type: Task
>  Components: Hive, HiveServer2, repl
>Affects Versions: 3.1.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-19488.01.patch, HIVE-19488.02.patch
>
>
> * add a parameter at db level to identify if its a source of replication. 
> beacon will set this.
>  * Enable CM root only for databases that are a source of a replication 
> policy, for other db's skip the CM root functionality.
>  * prevent database drop if the parameter indicating its source of a 
> replication, is set.
>  * as an upgrade to this version, beacon should set the property on all 
> existing database policies, in affect.
>  * the parameter should be of the form . –  repl.source.for : List < policy 
> ids >



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19488) enable CM root based on db parameter, identifying a db as source of replication.

[mahesh kumar behera (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mahesh kumar behera updated HIVE-19488:
---
Attachment: (was: HIVE-19488.02.patch)

> enable CM root based on db parameter, identifying a db as source of 
> replication.
> 
>
> Key: HIVE-19488
> URL: https://issues.apache.org/jira/browse/HIVE-19488
> Project: Hive
>  Issue Type: Task
>  Components: Hive, HiveServer2, repl
>Affects Versions: 3.1.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-19488.01.patch
>
>
> * add a parameter at db level to identify if its a source of replication. 
> beacon will set this.
>  * Enable CM root only for databases that are a source of a replication 
> policy, for other db's skip the CM root functionality.
>  * prevent database drop if the parameter indicating its source of a 
> replication, is set.
>  * as an upgrade to this version, beacon should set the property on all 
> existing database policies, in affect.
>  * the parameter should be of the form . –  repl.source.for : List < policy 
> ids >



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19488) enable CM root based on db parameter, identifying a db as source of replication.

[mahesh kumar behera (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mahesh kumar behera updated HIVE-19488:
---
Attachment: HIVE-19488.02.patch

> enable CM root based on db parameter, identifying a db as source of 
> replication.
> 
>
> Key: HIVE-19488
> URL: https://issues.apache.org/jira/browse/HIVE-19488
> Project: Hive
>  Issue Type: Task
>  Components: Hive, HiveServer2, repl
>Affects Versions: 3.1.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-19488.01.patch, HIVE-19488.02.patch
>
>
> * add a parameter at db level to identify if its a source of replication. 
> beacon will set this.
>  * Enable CM root only for databases that are a source of a replication 
> policy, for other db's skip the CM root functionality.
>  * prevent database drop if the parameter indicating its source of a 
> replication, is set.
>  * as an upgrade to this version, beacon should set the property on all 
> existing database policies, in affect.
>  * the parameter should be of the form . –  repl.source.for : List < policy 
> ids >



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19441) Add support for float aggregator and use LLAP test Driver

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473371#comment-16473371
 ] 

Ashutosh Chauhan commented on HIVE-19441:
-

[~bslim] You may need to rebase the patch?

> Add support for float aggregator and use LLAP test Driver
> -
>
> Key: HIVE-19441
> URL: https://issues.apache.org/jira/browse/HIVE-19441
> Project: Hive
>  Issue Type: Improvement
>  Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19441.patch
>
>
> Adding support to the float kind aggregator.
> Use LLAP as test Driver to reduce execution time of tests from about 2 hours 
> to 15 min:
> Although this patches unveiling an issue with timezone, maybe it is fixed by 
> [~jcamachorodriguez] upcoming set of patches.
>  
> Before
> {code}
> [INFO] Executed tasks
> [INFO]
> [INFO] --- maven-compiler-plugin:3.6.1:testCompile (default-testCompile) @ 
> hive-it-qfile ---
> [INFO] Compiling 21 source files to 
> /Users/sbouguerra/Hdev/hive/itests/qtest/target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ hive-it-qfile 
> ---
> [INFO]
> [INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO] Running org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
> 6,654.117 s - in org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 01:51 h
> [INFO] Finished at: 2018-05-04T12:43:19-07:00
> [INFO] 
> 
> {code}
> After
> {code}
> INFO] Executed tasks
> [INFO]
> [INFO] --- maven-compiler-plugin:3.6.1:testCompile (default-testCompile) @ 
> hive-it-qfile ---
> [INFO] Compiling 22 source files to 
> /Users/sbouguerra/Hdev/hive/itests/qtest/target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ hive-it-qfile 
> ---
> [INFO]
> [INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO] Running org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
> 907.167 s - in org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 15:31 min
> [INFO] Finished at: 2018-05-04T13:15:11-07:00
> [INFO] 
> 
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19326) union_fast_stats MiniLlapLocal golden file has incorrect "accurate" stats (incorrect query results possible)

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473370#comment-16473370
 ] 

Hive QA commented on HIVE-19326:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923076/HIVE-19326.01wip01.patch

{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 36 failed/errored test(s), 13570 tests 
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=94)

[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,scriptfile1.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Commented] (HIVE-19506) Test suites timing out

[Vihang Karajgaonkar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473366#comment-16473366
 ] 

Vihang Karajgaonkar commented on HIVE-19506:


+1 btw, you don't need to comment the {{@Ignore}} annotation. Please feel free 
to merge the patch.

> Test suites timing out
> --
>
> Key: HIVE-19506
> URL: https://issues.apache.org/jira/browse/HIVE-19506
> Project: Hive
>  Issue Type: Bug
>Reporter: Vineet Garg
>Priority: Blocker
> Attachments: HIVE-19506.1.patch, HIVE-19506.2.patch
>
>
> Currently a lot of test suites are timing out consistently:
> TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
> out)
> TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
> out)
> TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
> TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed 
> out)
> TestSequenceFileReadWrite - did not produce a TEST-*.xml file (likely timed 
> out)
> TestTxnExIm - did not produce a TEST-*.xml file (likely timed out)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService

[Eric Wohlstadter (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Wohlstadter updated HIVE-19307:

Status: Patch Available  (was: Open)

Fixed some formatting and added more comments

> Support ArrowOutputStream in LlapOutputFormatService
> 
>
> Key: HIVE-19307
> URL: https://issues.apache.org/jira/browse/HIVE-19307
> Project: Hive
>  Issue Type: Task
>  Components: llap
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-19307.3.patch, HIVE-19307.4.patch, 
> HIVE-19307.5.patch, HIVE-19307.6.patch
>
>
> Support pushing arrow batches through 
> org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService

[Eric Wohlstadter (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Wohlstadter updated HIVE-19307:

Attachment: HIVE-19307.6.patch

> Support ArrowOutputStream in LlapOutputFormatService
> 
>
> Key: HIVE-19307
> URL: https://issues.apache.org/jira/browse/HIVE-19307
> Project: Hive
>  Issue Type: Task
>  Components: llap
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-19307.3.patch, HIVE-19307.4.patch, 
> HIVE-19307.5.patch, HIVE-19307.6.patch
>
>
> Support pushing arrow batches through 
> org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService

[Eric Wohlstadter (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Wohlstadter updated HIVE-19307:

Status: Open  (was: Patch Available)

> Support ArrowOutputStream in LlapOutputFormatService
> 
>
> Key: HIVE-19307
> URL: https://issues.apache.org/jira/browse/HIVE-19307
> Project: Hive
>  Issue Type: Task
>  Components: llap
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-19307.3.patch, HIVE-19307.4.patch, 
> HIVE-19307.5.patch
>
>
> Support pushing arrow batches through 
> org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19326) union_fast_stats MiniLlapLocal golden file has incorrect "accurate" stats (incorrect query results possible)

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473355#comment-16473355
 ] 

Hive QA commented on HIVE-19326:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
49s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
39s{color} | {color:green} ql: The patch generated 0 new + 98 unchanged - 16 
fixed = 98 total (was 114) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git 
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply 
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch 12 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m  
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 11s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10866/dev-support/hive-personality.sh
 |
| git revision | master / 8db979f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10866/yetus/whitespace-eol.txt
 |
| whitespace | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10866/yetus/whitespace-tabs.txt
 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10866/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> union_fast_stats MiniLlapLocal golden file has incorrect "accurate" stats 
> (incorrect query results possible)
> 
>
> Key: HIVE-19326
> URL: https://issues.apache.org/jira/browse/HIVE-19326
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Reporter: Sergey Shelukhin
>Assignee: Zoltan Haindrich
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HIVE-19326.01wip01.patch
>
>
> Found when investigating the results change after converting tables to MM, 
> turns out the MM result is correct but the current one is not.
> The test ends like so:
> {noformat}
> desc formatted small_alltypesorc_a;
> ANALYZE TABLE small_alltypesorc_a COMPUTE STATISTICS;
> desc formatted small_alltypesorc_a;
> insert into table small_alltypesorc_a select * from small_alltypesorc1a;
> desc formatted small_alltypesorc_a;
> {noformat}
> The results from the descs in the golden file are:
> {noformat}
>   COLUMN_STATS_ACCURATE   {\"BASIC_STATS\":\"true\"}
>   numFiles1   
>   

[jira] [Commented] (HIVE-19504) Change default value for hive.auto.convert.join.shuffle.max.size property

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473348#comment-16473348
 ] 

Hive QA commented on HIVE-19504:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923068/HIVE-19504.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 28 failed/errored test(s), 13578 tests 
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Commented] (HIVE-19504) Change default value for hive.auto.convert.join.shuffle.max.size property

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473338#comment-16473338
 ] 

Hive QA commented on HIVE-19504:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
16s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
31s{color} | {color:blue} common in master has 62 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 10m 55s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10865/dev-support/hive-personality.sh
 |
| git revision | master / 8db979f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common U: common |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10865/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Change default value for hive.auto.convert.join.shuffle.max.size property
> -
>
> Key: HIVE-19504
> URL: https://issues.apache.org/jira/browse/HIVE-19504
> Project: Hive
>  Issue Type: Bug
>  Components: Physical Optimizer
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19504.patch
>
>
> The property default value is too low by mistake (10MB), it is missing three 
> trailing zeros.
> {code}
> HIVECONVERTJOINMAXSHUFFLESIZE("hive.auto.convert.join.shuffle.max.size", 
> 1000L,
>"If hive.auto.convert.join.noconditionaltask is off, this parameter 
> does not take affect. \n" +
>"However, if it is on, and the predicted size of the larger input for 
> a given join is greater \n" +
>"than this number, the join will not be converted to a dynamically 
> partitioned hash join. \n" +
>"The value \"-1\" means no limit."),
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18652) Print Spark metrics on console

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473337#comment-16473337
 ] 

Hive QA commented on HIVE-18652:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923070/HIVE-18652.3.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 38 failed/errored test(s), 13578 tests 
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Updated] (HIVE-19495) Arrow SerDe itest failure

[Teddy Choi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Teddy Choi updated HIVE-19495:
--
Status: Patch Available  (was: Open)

> Arrow SerDe itest failure
> -
>
> Key: HIVE-19495
> URL: https://issues.apache.org/jira/browse/HIVE-19495
> Project: Hive
>  Issue Type: Sub-task
>  Components: Serializers/Deserializers
>Reporter: Eric Wohlstadter
>Assignee: Teddy Choi
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19495.1.patch
>
>
> "You tried to write a Bit type when you are using a ValueWriter of type 
> NullableMapWriter."



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19495) Arrow SerDe itest failure

[Teddy Choi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Teddy Choi updated HIVE-19495:
--
Attachment: HIVE-19495.1.patch

> Arrow SerDe itest failure
> -
>
> Key: HIVE-19495
> URL: https://issues.apache.org/jira/browse/HIVE-19495
> Project: Hive
>  Issue Type: Sub-task
>  Components: Serializers/Deserializers
>Reporter: Eric Wohlstadter
>Assignee: Teddy Choi
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19495.1.patch
>
>
> "You tried to write a Bit type when you are using a ValueWriter of type 
> NullableMapWriter."



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HIVE-19511) Null value error with complex nested data type in Arrow batch serializer

[Teddy Choi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Teddy Choi resolved HIVE-19511.
---
Resolution: Duplicate

> Null value error with complex nested data type in Arrow batch serializer
> 
>
> Key: HIVE-19511
> URL: https://issues.apache.org/jira/browse/HIVE-19511
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Critical
> Attachments: HIVE-19511.1.patch
>
>
> Arrow batch serializer doesn't handle null values well in complex nested data 
> types.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19511) Null value error with complex nested data type in Arrow batch serializer

[Teddy Choi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Teddy Choi updated HIVE-19511:
--
Issue Type: Sub-task  (was: Bug)
Parent: HIVE-19305

> Null value error with complex nested data type in Arrow batch serializer
> 
>
> Key: HIVE-19511
> URL: https://issues.apache.org/jira/browse/HIVE-19511
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Critical
> Attachments: HIVE-19511.1.patch
>
>
> Arrow batch serializer doesn't handle null values well in complex nested data 
> types.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18652) Print Spark metrics on console

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473331#comment-16473331
 ] 

Hive QA commented on HIVE-18652:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  4m  
1s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
23s{color} | {color:blue} spark-client in master has 15 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m 
45s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
16s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
36s{color} | {color:red} ql: The patch generated 1 new + 7 unchanged - 0 fixed 
= 8 total (was 7) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
10s{color} | {color:red} spark-client: The patch generated 5 new + 15 unchanged 
- 0 fixed = 20 total (was 15) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 27s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10864/dev-support/hive-personality.sh
 |
| git revision | master / 8db979f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10864/yetus/patch-mvninstall-ql.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10864/yetus/diff-checkstyle-ql.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10864/yetus/diff-checkstyle-spark-client.txt
 |
| modules | C: ql spark-client U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10864/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Print Spark metrics on console
> --
>
> Key: HIVE-18652
> URL: https://issues.apache.org/jira/browse/HIVE-18652
> Project: Hive
>  Issue Type: Sub-task
>  Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18652.1.patch, HIVE-18652.2.patch, 
> HIVE-18652.3.patch
>
>
> For Hive-on-MR, each MR job launched prints out some stats about the job:
> {code}
> INFO  : 2018-02-07 17:51:11,218 Stage-1 map = 0%,  reduce = 0%
> INFO  : 2018-02-07 17:51:18,396 Stage-1 map = 100%,  reduce = 0%, Cumulative 
> CPU 

[jira] [Commented] (HIVE-14388) Add number of rows inserted message after insert command in Beeline

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-14388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473327#comment-16473327
 ] 

Hive QA commented on HIVE-14388:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923066/HIVE-14388.10.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 31 failed/errored test(s), 13568 tests 
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=94)

[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,scriptfile1.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Commented] (HIVE-19506) Test suites timing out

[Vineet Garg (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473325#comment-16473325
 ] 

Vineet Garg commented on HIVE-19506:


[~vihangk1] Thanks for suggestion. I uploaded a new patch with @Ignore. Can you 
look into it and do +1 if you think this is ok? Getting this in master should 
speed up ptests and help clearing up the queue.

> Test suites timing out
> --
>
> Key: HIVE-19506
> URL: https://issues.apache.org/jira/browse/HIVE-19506
> Project: Hive
>  Issue Type: Bug
>Reporter: Vineet Garg
>Priority: Blocker
> Attachments: HIVE-19506.1.patch, HIVE-19506.2.patch
>
>
> Currently a lot of test suites are timing out consistently:
> TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
> out)
> TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
> out)
> TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
> TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed 
> out)
> TestSequenceFileReadWrite - did not produce a TEST-*.xml file (likely timed 
> out)
> TestTxnExIm - did not produce a TEST-*.xml file (likely timed out)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19506) Test suites timing out

[Vineet Garg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vineet Garg updated HIVE-19506:
---
Status: Patch Available  (was: Open)

> Test suites timing out
> --
>
> Key: HIVE-19506
> URL: https://issues.apache.org/jira/browse/HIVE-19506
> Project: Hive
>  Issue Type: Bug
>Reporter: Vineet Garg
>Priority: Blocker
> Attachments: HIVE-19506.1.patch, HIVE-19506.2.patch
>
>
> Currently a lot of test suites are timing out consistently:
> TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
> out)
> TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
> out)
> TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
> TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed 
> out)
> TestSequenceFileReadWrite - did not produce a TEST-*.xml file (likely timed 
> out)
> TestTxnExIm - did not produce a TEST-*.xml file (likely timed out)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19506) Test suites timing out

[Vineet Garg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vineet Garg updated HIVE-19506:
---
Status: Open  (was: Patch Available)

> Test suites timing out
> --
>
> Key: HIVE-19506
> URL: https://issues.apache.org/jira/browse/HIVE-19506
> Project: Hive
>  Issue Type: Bug
>Reporter: Vineet Garg
>Priority: Blocker
> Attachments: HIVE-19506.1.patch, HIVE-19506.2.patch
>
>
> Currently a lot of test suites are timing out consistently:
> TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
> out)
> TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
> out)
> TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
> TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed 
> out)
> TestSequenceFileReadWrite - did not produce a TEST-*.xml file (likely timed 
> out)
> TestTxnExIm - did not produce a TEST-*.xml file (likely timed out)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19506) Test suites timing out

[Vineet Garg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vineet Garg updated HIVE-19506:
---
Attachment: HIVE-19506.2.patch

> Test suites timing out
> --
>
> Key: HIVE-19506
> URL: https://issues.apache.org/jira/browse/HIVE-19506
> Project: Hive
>  Issue Type: Bug
>Reporter: Vineet Garg
>Priority: Blocker
> Attachments: HIVE-19506.1.patch, HIVE-19506.2.patch
>
>
> Currently a lot of test suites are timing out consistently:
> TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
> out)
> TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
> out)
> TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
> TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed 
> out)
> TestSequenceFileReadWrite - did not produce a TEST-*.xml file (likely timed 
> out)
> TestTxnExIm - did not produce a TEST-*.xml file (likely timed out)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-14388) Add number of rows inserted message after insert command in Beeline

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-14388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473317#comment-16473317
 ] 

Hive QA commented on HIVE-14388:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
 2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
33s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
30s{color} | {color:blue} beeline in master has 69 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
34s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
26s{color} | {color:blue} jdbc in master has 17 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
52s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
39s{color} | {color:blue} service in master has 50 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  2m  
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
 5s{color} | {color:green} The patch service-rpc passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
11s{color} | {color:green} The patch beeline passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
16s{color} | {color:green} The patch hive-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
11s{color} | {color:green} The patch jdbc passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
39s{color} | {color:green} ql: The patch generated 0 new + 217 unchanged - 13 
fixed = 217 total (was 230) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
14s{color} | {color:green} The patch service passed checkstyle {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git 
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply 
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  6m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  2m  
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 38m 48s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10863/dev-support/hive-personality.sh
 |
| git revision | master / 8db979f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace | 

[jira] [Commented] (HIVE-18394) Materialized view: "Create Materialized View" should default to rewritable ones

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-18394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473307#comment-16473307
 ] 

Hive QA commented on HIVE-18394:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923067/HIVE-18394.patch

{color:green}SUCCESS:{color} +1 due to 24 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 32 failed/errored test(s), 13578 tests 
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Commented] (HIVE-18394) Materialized view: "Create Materialized View" should default to rewritable ones

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-18394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473297#comment-16473297
 ] 

Hive QA commented on HIVE-18394:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
52s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 36s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10862/dev-support/hive-personality.sh
 |
| git revision | master / 8db979f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10862/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Materialized view: "Create Materialized View" should default to rewritable 
> ones
> ---
>
> Key: HIVE-18394
> URL: https://issues.apache.org/jira/browse/HIVE-18394
> Project: Hive
>  Issue Type: Improvement
>  Components: Materialized views
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Blocker
>  Labels: TODOC3.0
> Attachments: HIVE-18394.patch
>
>
> This is a usability ticket, since it is possible to end up creating 
> materialized views and realize that they need an additional flag to be picked 
> up by the optimizer to do rewrites to.
> {code:sql}
> create materialized view ca as select * from customer, customer_address where 
> c_current_addr_sk = ca_address_sk;
> set hive.materializedview.rewriting=true;
> select count(1) from customer, customer_address where c_current_addr_sk = 
> ca_address_sk; -- does not use materialized view
> {code}
> Needs another step
> {code:sql}
> alter materialized view ca enable rewrite;
> {code}
> And then, it kicks in 
> {code:sql}
> select count(1) from customer, customer_address where c_current_addr_sk = 
> ca_address_sk;
> OK
> 1200
> Time taken: 0.494 seconds, Fetched: 1 row(s)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19500) Prevent multiple selectivity estimations for the same variable in conjuctions

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473291#comment-16473291
 ] 

Hive QA commented on HIVE-19500:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923062/HIVE-19500.01.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 45 failed/errored test(s), 13566 tests 
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestJdbcWithDBTokenStoreNoDoAs - did not produce a TEST-*.xml file (likely 
timed out) (batchId=254)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Commented] (HIVE-19500) Prevent multiple selectivity estimations for the same variable in conjuctions

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473281#comment-16473281
 ] 

Hive QA commented on HIVE-19500:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
58s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
37s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch 2 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m  
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 59s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10861/dev-support/hive-personality.sh
 |
| git revision | master / 8db979f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10861/yetus/whitespace-tabs.txt
 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10861/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Prevent multiple selectivity estimations for the same variable in conjuctions
> -
>
> Key: HIVE-19500
> URL: https://issues.apache.org/jira/browse/HIVE-19500
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19500.01.patch
>
>
> see HIVE-19097 for problem description
> for filters like: {{(d_year in (2001,2002) and d_year = 2001)}} the current 
> estimation is around {{(1/NDV)**2}} (iff column stats are available) 
> actually the source of the problem was a small typo in HIVE-17465 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-13745) UDF current_date、current_timestamp、unix_timestamp NPE

[Yongzhi Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-13745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yongzhi Chen reassigned HIVE-13745:
---

Assignee: Yongzhi Chen  (was: Biao Wu)

> UDF current_date、current_timestamp、unix_timestamp NPE
> -
>
> Key: HIVE-13745
> URL: https://issues.apache.org/jira/browse/HIVE-13745
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Biao Wu
>Assignee: Yongzhi Chen
>Priority: Major
> Attachments: HIVE-13745.1.patch, HIVE-13745.2-branch-2.patch, 
> HIVE-13745.patch
>
>
> NullPointerException when current_date is used in mapreduce



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-13745) UDF current_date、current_timestamp、unix_timestamp NPE

[Yongzhi Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-13745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yongzhi Chen updated HIVE-13745:

Attachment: HIVE-13745.2-branch-2.patch

> UDF current_date、current_timestamp、unix_timestamp NPE
> -
>
> Key: HIVE-13745
> URL: https://issues.apache.org/jira/browse/HIVE-13745
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Biao Wu
>Assignee: Yongzhi Chen
>Priority: Major
> Attachments: HIVE-13745.1.patch, HIVE-13745.2-branch-2.patch, 
> HIVE-13745.patch
>
>
> NullPointerException when current_date is used in mapreduce



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-13745) UDF current_date、current_timestamp、unix_timestamp NPE

[Yongzhi Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-13745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yongzhi Chen updated HIVE-13745:

Attachment: (was: HIVE-13745.2-brach-2.patch)

> UDF current_date、current_timestamp、unix_timestamp NPE
> -
>
> Key: HIVE-13745
> URL: https://issues.apache.org/jira/browse/HIVE-13745
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Biao Wu
>Assignee: Biao Wu
>Priority: Major
> Attachments: HIVE-13745.1.patch, HIVE-13745.patch
>
>
> NullPointerException when current_date is used in mapreduce



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19435) Incremental replication cause data loss if a table is dropped followed by create and insert-into with different partition type.

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473272#comment-16473272
 ] 

Hive QA commented on HIVE-19435:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923058/HIVE-19435.01-branch-3.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 33 failed/errored test(s), 13370 tests 
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=253)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=253)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Commented] (HIVE-13745) UDF current_date、current_timestamp、unix_timestamp NPE

[Yongzhi Chen (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-13745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473273#comment-16473273
 ] 

Yongzhi Chen commented on HIVE-13745:
-

[~kgyrtkirk]The major reason to use HiveConf is not calling 
System.currentTimeMillis, it is to use the time value set in SessionState. The 
System.currentTimeMillis is only called when the SessionState has not set time 
value yet which is consistent with original behavior(without the fix). I do not 
see any issue from that. 

> UDF current_date、current_timestamp、unix_timestamp NPE
> -
>
> Key: HIVE-13745
> URL: https://issues.apache.org/jira/browse/HIVE-13745
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Biao Wu
>Assignee: Biao Wu
>Priority: Major
> Attachments: HIVE-13745.1.patch, HIVE-13745.2-brach-2.patch, 
> HIVE-13745.patch
>
>
> NullPointerException when current_date is used in mapreduce



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19435) Incremental replication cause data loss if a table is dropped followed by create and insert-into with different partition type.

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473260#comment-16473260
 ] 

Hive QA commented on HIVE-19435:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  8s{color} 
| {color:red} 
/data/hiveptest/logs/PreCommit-HIVE-Build-10860/patches/PreCommit-HIVE-Build-10860.patch
 does not apply to master. Rebase required? Wrong Branch? See 
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help. 
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10860/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Incremental replication cause data loss if a table is dropped followed by 
> create and insert-into with different partition type.
> ---
>
> Key: HIVE-19435
> URL: https://issues.apache.org/jira/browse/HIVE-19435
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
>  Labels: DR, pull-request-available, replication
> Fix For: 3.0.0, 3.1.0
>
> Attachments: HIVE-19435.01-branch-3.patch, HIVE-19435.01.patch, 
> HIVE-19435.02.patch, HIVE-19435.03.patch
>
>
> If the incremental dump have drop of partitioned table followed by 
> create/insert on non-partitioned table with same name, doesn't replicate the 
> data. Explained below.
> Let's say we have a partitioned table T1 which was already replicated to 
> target.
> DROP_TABLE(T1)->CREATE_TABLE(T1) (Non-partitioned) -> INSERT(T1)(10) 
> After REPL LOAD, T1 doesn't have any data.
> Same is valid for non-partitioned to partitioned and partition spec mismatch 
> case as well.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19486) Discrepancy between the config and the code in Hikari connectionPoolingType

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473257#comment-16473257
 ] 

Hive QA commented on HIVE-19486:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923050/HIVE-19486.1.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 13578 tests 
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Commented] (HIVE-19486) Discrepancy between the config and the code in Hikari connectionPoolingType

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473252#comment-16473252
 ] 

Hive QA commented on HIVE-19486:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
21s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m  
1s{color} | {color:blue} standalone-metastore in master has 215 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m  
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m  6s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10859/dev-support/hive-personality.sh
 |
| git revision | master / 8db979f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore U: standalone-metastore |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10859/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Discrepancy between the config and the code in Hikari connectionPoolingType
> ---
>
> Key: HIVE-19486
> URL: https://issues.apache.org/jira/browse/HIVE-19486
> Project: Hive
>  Issue Type: Bug
>Reporter: Antal Sinkovits
>Assignee: Antal Sinkovits
>Priority: Major
> Attachments: HIVE-19486.1.patch
>
>
> MetaStoreConf contains datanucleus.connectionPoolingType "HikariCP" while the 
> code checks for "Hikari".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19460) Improve stats estimations for NOT IN operator

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473250#comment-16473250
 ] 

Hive QA commented on HIVE-19460:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923021/HIVE-19460.01wip03.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 129 failed/errored test(s), 13580 tests 
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Commented] (HIVE-19487) Precommits generating 20G of logs

[Vihang Karajgaonkar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473247#comment-16473247
 ] 

Vihang Karajgaonkar commented on HIVE-19487:


2018-05-11 20:30:43,168 DEBUG [TestExecutor] ExecutionPhase.execute:96 PBatch: 
UnitTestBatch [name=221_UTBatch_service_8_tests, id=221, moduleName=service, 
batchSize=8, isParallel=true, testList=[TestCustomQueryFilter, TestUserFilter, 
TestUserSearchFilter, TestQuery, TestLdapAtnProviderWithMiniDS, 
TestHiveSQLException, TestCLIServiceConnectionLimits, TestCLIServiceRestore]]

> Precommits generating 20G of logs
> -
>
> Key: HIVE-19487
> URL: https://issues.apache.org/jira/browse/HIVE-19487
> Project: Hive
>  Issue Type: Test
>Reporter: Vihang Karajgaonkar
>Priority: Major
>
> Precommit jobs are generating huge logs. The disk gets full pretty quickly. I 
> am not sure what was the log size but I think it was definitely not a problem 
> previously. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19487) Precommits generating 20G of logs

[Vihang Karajgaonkar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473244#comment-16473244
 ] 

Vihang Karajgaonkar commented on HIVE-19487:


Following for the largest logs 
{noformat}
-rw-r--r-- 1 tomcat hiveptest 1169724848 May 11 21:11 
./succeeded/173-TestMiniLlapLocalCliDriver-vector_windowing_expressions.q-auto_join1.q-ins
ert_values_partitioned.q-and-27-more/logs/hive.log
-rw-r--r-- 1 tomcat hiveptest 1702616342 May 11 20:43 
./succeeded/35-TestCliDriver-constantPropWhen.q-udf_floor.q-join20.q-and-27-more/logs/hive
.log
-rw-r--r-- 1 tomcat hiveptest 4231255074 May 11 21:12 
./succeeded/221_UTBatch_service_8_tests/logs/hive.log
-rw-r--r-- 1 tomcat hiveptest 1062224231 May 11 21:08 
./succeeded/169-TestMiniLlapLocalCliDriver-union_top_level.q-vector_create_struct_table.q-
schema_evol_text_vecrow_part_all_primitive.q-and-27-more/logs/hive.log
{noformat}

> Precommits generating 20G of logs
> -
>
> Key: HIVE-19487
> URL: https://issues.apache.org/jira/browse/HIVE-19487
> Project: Hive
>  Issue Type: Test
>Reporter: Vihang Karajgaonkar
>Priority: Major
>
> Precommit jobs are generating huge logs. The disk gets full pretty quickly. I 
> am not sure what was the log size but I think it was definitely not a problem 
> previously. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-13745) UDF current_date、current_timestamp、unix_timestamp NPE

[Vihang Karajgaonkar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-13745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473242#comment-16473242
 ] 

Vihang Karajgaonkar commented on HIVE-13745:


there is a spelling mistake in the patch name. It should be branch-2. The 
precommit job failed due to that.

> UDF current_date、current_timestamp、unix_timestamp NPE
> -
>
> Key: HIVE-13745
> URL: https://issues.apache.org/jira/browse/HIVE-13745
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Biao Wu
>Assignee: Biao Wu
>Priority: Major
> Attachments: HIVE-13745.1.patch, HIVE-13745.2-brach-2.patch, 
> HIVE-13745.patch
>
>
> NullPointerException when current_date is used in mapreduce



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HIVE-19510) Add performance metric to find the total time spend in rsync

[Vihang Karajgaonkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vihang Karajgaonkar resolved HIVE-19510.

Resolution: Fixed

merged to HIVE-19429 branch.

> Add performance metric to find the total time spend in rsync
> 
>
> Key: HIVE-19510
> URL: https://issues.apache.org/jira/browse/HIVE-19510
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-19510.01.patch
>
>
> I think we are spending a lot of time copying logs from worker nodes to the 
> server. We should add some logging to print aggregate time spent in rsync to 
> confirm.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19510) Add performance metric to find the total time spend in rsync

[Vihang Karajgaonkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vihang Karajgaonkar updated HIVE-19510:
---
Attachment: HIVE-19510.01.patch

> Add performance metric to find the total time spend in rsync
> 
>
> Key: HIVE-19510
> URL: https://issues.apache.org/jira/browse/HIVE-19510
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-19510.01.patch
>
>
> I think we are spending a lot of time copying logs from worker nodes to the 
> server. We should add some logging to print aggregate time spent in rsync to 
> confirm.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19511) Null value error with complex nested data type in Arrow batch serializer

[Teddy Choi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Teddy Choi updated HIVE-19511:
--
Attachment: HIVE-19511.1.patch

> Null value error with complex nested data type in Arrow batch serializer
> 
>
> Key: HIVE-19511
> URL: https://issues.apache.org/jira/browse/HIVE-19511
> Project: Hive
>  Issue Type: Bug
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Critical
> Attachments: HIVE-19511.1.patch
>
>
> Arrow batch serializer doesn't handle null values well in complex nested data 
> types.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19511) Null value error with complex nested data type in Arrow batch serializer

[Teddy Choi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Teddy Choi updated HIVE-19511:
--
Summary: Null value error with complex nested data type in Arrow batch 
serializer  (was: Arrow batch serializer null handling bug)

> Null value error with complex nested data type in Arrow batch serializer
> 
>
> Key: HIVE-19511
> URL: https://issues.apache.org/jira/browse/HIVE-19511
> Project: Hive
>  Issue Type: Bug
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Critical
>
> Arrow batch serializer doesn't handle null values well in complex nested data 
> types.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19511) Arrow batch serializer null handling bug

[Teddy Choi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Teddy Choi updated HIVE-19511:
--
Description: Arrow batch serializer doesn't handle null values well in 
complex nested data types.

> Arrow batch serializer null handling bug
> 
>
> Key: HIVE-19511
> URL: https://issues.apache.org/jira/browse/HIVE-19511
> Project: Hive
>  Issue Type: Bug
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Critical
>
> Arrow batch serializer doesn't handle null values well in complex nested data 
> types.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19460) Improve stats estimations for NOT IN operator

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473230#comment-16473230
 ] 

Hive QA commented on HIVE-19460:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
56s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
29s{color} | {color:blue} common in master has 62 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
55s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
14s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
38s{color} | {color:red} ql: The patch generated 18 new + 107 unchanged - 0 
fixed = 125 total (was 107) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  4m  
3s{color} | {color:red} ql generated 9 new + 2321 unchanged - 0 fixed = 2330 
total (was 2321) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 51s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
|  |  Found reliance on default encoding in 
org.apache.hadoop.hive.ql.optimizer.stats.annotation.HiveMurmur3Adapter.murmur3(Object):in
 
org.apache.hadoop.hive.ql.optimizer.stats.annotation.HiveMurmur3Adapter.murmur3(Object):
 String.getBytes()  At HiveMurmur3Adapter.java:[line 68] |
|  |  Boxing/unboxing to parse a primitive 
org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$FilterStatsRule$RangeOps.intersect(ExprNodeDesc)
  At 
StatsRulesProcFactory.java:org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$FilterStatsRule$RangeOps.intersect(ExprNodeDesc)
  At StatsRulesProcFactory.java:[line 627] |
|  |  Boxing/unboxing to parse a primitive 
org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$FilterStatsRule$RangeOps.intersect(ExprNodeDesc)
  At 
StatsRulesProcFactory.java:org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$FilterStatsRule$RangeOps.intersect(ExprNodeDesc)
  At StatsRulesProcFactory.java:[line 633] |
|  |  
org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$FilterStatsRule$RangeOps.intersect(ExprNodeDesc)
 invokes inefficient new Byte(String) constructor; use Byte.valueOf(String) 
instead  At StatsRulesProcFactory.java:constructor; use Byte.valueOf(String) 
instead  At StatsRulesProcFactory.java:[line 613] |
|  |  
org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$FilterStatsRule$RangeOps.intersect(ExprNodeDesc)
 invokes inefficient new Integer(String) constructor; use 
Integer.valueOf(String) instead  At StatsRulesProcFactory.java:constructor; use 
Integer.valueOf(String) instead  At StatsRulesProcFactory.java:[line 627] |
|  |  

[jira] [Assigned] (HIVE-19511) Arrow batch serializer null handling bug

[Teddy Choi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Teddy Choi reassigned HIVE-19511:
-


> Arrow batch serializer null handling bug
> 
>
> Key: HIVE-19511
> URL: https://issues.apache.org/jira/browse/HIVE-19511
> Project: Hive
>  Issue Type: Bug
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Critical
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19510) Add performance metric to find the total time spend in rsync

[Vihang Karajgaonkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vihang Karajgaonkar reassigned HIVE-19510:
--


> Add performance metric to find the total time spend in rsync
> 
>
> Key: HIVE-19510
> URL: https://issues.apache.org/jira/browse/HIVE-19510
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
>
> I think we are spending a lot of time copying logs from worker nodes to the 
> server. We should add some logging to print aggregate time spent in rsync to 
> confirm.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19505) Increase number of parallel runs from 2 per node to 4

[Vihang Karajgaonkar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473218#comment-16473218
 ] 

Vihang Karajgaonkar commented on HIVE-19505:


Changing from 2 to 3 didn't help much. The execution phase came down from ~1 hr 
30 min to ~ 1hr 20 min.  Ideally should have been close to ~1hr. This shows 
that the bottleneck is may be some where else. I think we are spending a lot of 
time copying the files which are close to 20G. See HIVE-19487

> Increase number of parallel runs from 2 per node to 4
> -
>
> Key: HIVE-19505
> URL: https://issues.apache.org/jira/browse/HIVE-19505
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
>
> The test cluster is under utilized. For the short run we can increase the 
> parallel threads from 2 to 3 (or may be 4) which will speedup parallel test 
> batches.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19454) Test failure : org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails with java.lang.AssertionError

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473209#comment-16473209
 ] 

Hive QA commented on HIVE-19454:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923015/HIVE-19454.2.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 32 failed/errored test(s), 13568 tests 
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=94)

[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,scriptfile1.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Work started] (HIVE-19499) Bootstrap REPL LOAD shall add tasks to create checkpoints for tables/partitions.

[Sankar Hariappan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HIVE-19499 started by Sankar Hariappan.
---
> Bootstrap REPL LOAD shall add tasks to create checkpoints for 
> tables/partitions.
> 
>
> Key: HIVE-19499
> URL: https://issues.apache.org/jira/browse/HIVE-19499
> Project: Hive
>  Issue Type: Sub-task
>  Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
>  Labels: DR, replication
> Fix For: 3.1.0
>
>
> Currently. bootstrap REPL LOAD expect the target database to be empty or not 
> exist to start bootstrap load.
> But, this adds overhead when there is a failure in between bootstrap load and 
> there is no way to resume it from where it fails. So, it is needed to create 
> checkpoints in table/partitions to skip the completely loaded objects.
> Use hash of the fully qualified path of the dump directory as a checkpoint 
> identifier. This should be added to the table / partition properties in hive 
> via a task, as the last task in the DAG for table / partition creation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19454) Test failure : org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails with java.lang.AssertionError

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473186#comment-16473186
 ] 

Hive QA commented on HIVE-19454:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  4m  
6s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m  
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m  9s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10857/dev-support/hive-personality.sh
 |
| git revision | master / 8db979f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10857/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Test failure : 
> org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails 
> with java.lang.AssertionError
> --
>
> Key: HIVE-19454
> URL: https://issues.apache.org/jira/browse/HIVE-19454
> Project: Hive
>  Issue Type: Bug
> Environment: uname -a
> Linux pts00433-vm40 3.10.0-862.el7.ppc64le #1 SMP Wed Mar 21 22:20:19 UTC 
> 2018 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Priority: Major
> Attachments: HIVE-19454.1.patch, HIVE-19454.2.patch
>
>
> org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testNonAcidToAcidConversion1
>  refers to the same test case.
> Maven command used is : mvn -Dtest=TestTxnCommands2 test
> Error:
> {code:java}
> [INFO] Running org.apache.hadoop.hive.ql.TestTxnCommands2
> [ERROR] Tests run: 44, Failures: 1, Errors: 0, Skipped: 3, Time elapsed: 
> 618.215 s <<< FAILURE! - in org.apache.hadoop.hive.ql.TestTxnCommands2
> [ERROR] 
> testNonAcidToAcidConversion1(org.apache.hadoop.hive.ql.TestTxnCommands2)  
> Time elapsed: 17.557 s  <<< FAILURE!
> java.lang.AssertionError
>     at org.junit.Assert.fail(Assert.java:86)
>     at org.junit.Assert.assertTrue(Assert.java:41)
>     at org.junit.Assert.assertTrue(Assert.java:52)
>     at 
> org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1(TestTxnCommands2.java:499)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at 
> 

[jira] [Commented] (HIVE-19166) TestMiniLlapLocalCliDriver sysdb failure

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473163#comment-16473163
 ] 

Hive QA commented on HIVE-19166:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923007/HIVE-19166.06.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 34 failed/errored test(s), 13568 tests 
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=94)

[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,scriptfile1.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Updated] (HIVE-19476) Fix failures in TestReplicationScenariosAcidTables, TestReplicationOnHDFSEncryptedZones and TestCopyUtils

[Sankar Hariappan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sankar Hariappan updated HIVE-19476:

Status: Open  (was: Patch Available)

> Fix failures in TestReplicationScenariosAcidTables, 
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils
> -
>
> Key: HIVE-19476
> URL: https://issues.apache.org/jira/browse/HIVE-19476
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
>  Labels: Tests, pull-request-available
> Fix For: 3.0.0, 3.1.0
>
> Attachments: HIVE-19476.01-branch-3.patch, HIVE-19476.01.patch, 
> HIVE-19476.02.patch
>
>
> TestReplicationScenariosAcidTables broken due to HIVE-17657.
> cc [~sershe]
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils are failing due to 
> auto-stats gathered which thrown exception irrelevant to theses tests.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19476) Fix failures in TestReplicationScenariosAcidTables, TestReplicationOnHDFSEncryptedZones and TestCopyUtils

[Sankar Hariappan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sankar Hariappan updated HIVE-19476:

Attachment: (was: HIVE-19476.01-branch-3.patch)

> Fix failures in TestReplicationScenariosAcidTables, 
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils
> -
>
> Key: HIVE-19476
> URL: https://issues.apache.org/jira/browse/HIVE-19476
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
>  Labels: Tests, pull-request-available
> Fix For: 3.0.0, 3.1.0
>
> Attachments: HIVE-19476.01-branch-3.patch, HIVE-19476.01.patch, 
> HIVE-19476.02.patch
>
>
> TestReplicationScenariosAcidTables broken due to HIVE-17657.
> cc [~sershe]
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils are failing due to 
> auto-stats gathered which thrown exception irrelevant to theses tests.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19476) Fix failures in TestReplicationScenariosAcidTables, TestReplicationOnHDFSEncryptedZones and TestCopyUtils

[Sankar Hariappan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sankar Hariappan updated HIVE-19476:

Attachment: HIVE-19476.01-branch-3.patch

> Fix failures in TestReplicationScenariosAcidTables, 
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils
> -
>
> Key: HIVE-19476
> URL: https://issues.apache.org/jira/browse/HIVE-19476
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
>  Labels: Tests, pull-request-available
> Fix For: 3.0.0, 3.1.0
>
> Attachments: HIVE-19476.01-branch-3.patch, HIVE-19476.01.patch, 
> HIVE-19476.02.patch
>
>
> TestReplicationScenariosAcidTables broken due to HIVE-17657.
> cc [~sershe]
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils are failing due to 
> auto-stats gathered which thrown exception irrelevant to theses tests.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19476) Fix failures in TestReplicationScenariosAcidTables, TestReplicationOnHDFSEncryptedZones and TestCopyUtils

[Sankar Hariappan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sankar Hariappan updated HIVE-19476:

Status: Patch Available  (was: Open)

> Fix failures in TestReplicationScenariosAcidTables, 
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils
> -
>
> Key: HIVE-19476
> URL: https://issues.apache.org/jira/browse/HIVE-19476
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
>  Labels: Tests, pull-request-available
> Fix For: 3.0.0, 3.1.0
>
> Attachments: HIVE-19476.01-branch-3.patch, HIVE-19476.01.patch, 
> HIVE-19476.02.patch
>
>
> TestReplicationScenariosAcidTables broken due to HIVE-17657.
> cc [~sershe]
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils are failing due to 
> auto-stats gathered which thrown exception irrelevant to theses tests.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work started] (HIVE-18666) Materialized view: "create materialized enable rewrite" should fail if rewriting is not possible

[Jesus Camacho Rodriguez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-18666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HIVE-18666 started by Jesus Camacho Rodriguez.
--
> Materialized view: "create materialized enable rewrite" should fail if 
> rewriting is not possible
> 
>
> Key: HIVE-18666
> URL: https://issues.apache.org/jira/browse/HIVE-18666
> Project: Hive
>  Issue Type: Bug
>  Components: Materialized views
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Fix For: 3.0.0
>
>
> {code}
> CREATE MATERIALIZED VIEW TEST_AGG ENABLE REWRITE AS 
> select ... from ext_Table;
> {code}
> works, but then 
> {code}
> alter materialized view TEST_AGG enable rewrite;
> {code}
> fails with {{SemanticException Automatic rewriting for materialized view 
> cannot be enabled if the materialized view uses non-transactional tables}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HIVE-18666) Materialized view: "create materialized enable rewrite" should fail if rewriting is not possible

[Jesus Camacho Rodriguez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-18666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesus Camacho Rodriguez resolved HIVE-18666.

   Resolution: Fixed
Fix Version/s: 3.0.0

This should be fixed now, please reopen if you still see the issue.

> Materialized view: "create materialized enable rewrite" should fail if 
> rewriting is not possible
> 
>
> Key: HIVE-18666
> URL: https://issues.apache.org/jira/browse/HIVE-18666
> Project: Hive
>  Issue Type: Bug
>  Components: Materialized views
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Fix For: 3.0.0
>
>
> {code}
> CREATE MATERIALIZED VIEW TEST_AGG ENABLE REWRITE AS 
> select ... from ext_Table;
> {code}
> works, but then 
> {code}
> alter materialized view TEST_AGG enable rewrite;
> {code}
> fails with {{SemanticException Automatic rewriting for materialized view 
> cannot be enabled if the materialized view uses non-transactional tables}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19509) Disable tests that are failing continuously

[Jesus Camacho Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473135#comment-16473135
 ] 

Jesus Camacho Rodriguez commented on HIVE-19509:


Once we get to the final list, I will include it in the description for 
reference. For the time being, this is the initial list of candidates:

{noformat}
 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fouter_join_ppr]
8.9 sec 9
 org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mm_bhif]   
2.6 sec 9
 
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[special_character_in_tabnames_1]
   0.25 sec9
 
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acidvec_part]
  0.51 sec9
 
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_vec_part_llap_io]
  0.42 sec9
 
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
  35 sec  9
 org.apache.hadoop.hive.metastore.TestStats.partitionedTableDeprecatedCalls 
7.8 sec 9
 org.apache.hadoop.hive.metastore.TestOldSchema.testPartitionOps5.8 sec 
9
 org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 22 ms   
9
 
org.apache.hadoop.hive.ql.parse.TestCopyUtils.testPrivilegedDistCpWithSameUserAsCurrentDoesNotTryToImpersonate
 12 sec  9
 
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAcidTablesBootstrapWithOpenTxnsTimeout
  1 min 24 sec9
 
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAcidTablesBootstrap
 36 sec  9
 org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress  1 min 23 sec
9
 org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel  1 min 
25 sec9
 org.apache.hive.jdbc.TestSSL.testSSLFetchHttp  5.9 sec 9
 org.apache.hive.spark.client.rpc.TestRpc.testServerPort19 ms   9
 org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 16 sec  44
 org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1] 
9.7 sec 86
 
org.apache.hadoop.hive.ql.parse.TestReplicationOnHDFSEncryptedZones.targetAndSourceHaveDifferentEncryptionZoneKeys
 13 sec  87
 org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] 
49 sec  99
 org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion   45 sec  202
 org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion0120 sec  
202
 
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
  9.8 sec 227
{noformat}

> Disable tests that are failing continuously
> ---
>
> Key: HIVE-19509
> URL: https://issues.apache.org/jira/browse/HIVE-19509
> Project: Hive
>  Issue Type: Task
>  Components: Testing Infrastructure, Tests
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19509.patch
>
>
> As per discussion in mailing list. We will bring the failure count to zero 
> and be less tolerant with ptest runs that are not clean.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (HIVE-19509) Disable tests that are failing continuously

[Jesus Camacho Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473135#comment-16473135
 ] 

Jesus Camacho Rodriguez edited comment on HIVE-19509 at 5/12/18 3:11 PM:
-

Once we get to the final list, I will include it in the JIRA description for 
reference. For the time being, this is the initial list of candidates:

{noformat}
 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fouter_join_ppr]
8.9 sec 9
 org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mm_bhif]   
2.6 sec 9
 
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[special_character_in_tabnames_1]
   0.25 sec9
 
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acidvec_part]
  0.51 sec9
 
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_vec_part_llap_io]
  0.42 sec9
 
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
  35 sec  9
 org.apache.hadoop.hive.metastore.TestStats.partitionedTableDeprecatedCalls 
7.8 sec 9
 org.apache.hadoop.hive.metastore.TestOldSchema.testPartitionOps5.8 sec 
9
 org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 22 ms   
9
 
org.apache.hadoop.hive.ql.parse.TestCopyUtils.testPrivilegedDistCpWithSameUserAsCurrentDoesNotTryToImpersonate
 12 sec  9
 
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAcidTablesBootstrapWithOpenTxnsTimeout
  1 min 24 sec9
 
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAcidTablesBootstrap
 36 sec  9
 org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress  1 min 23 sec
9
 org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel  1 min 
25 sec9
 org.apache.hive.jdbc.TestSSL.testSSLFetchHttp  5.9 sec 9
 org.apache.hive.spark.client.rpc.TestRpc.testServerPort19 ms   9
 org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 16 sec  44
 org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1] 
9.7 sec 86
 
org.apache.hadoop.hive.ql.parse.TestReplicationOnHDFSEncryptedZones.targetAndSourceHaveDifferentEncryptionZoneKeys
 13 sec  87
 org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] 
49 sec  99
 org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion   45 sec  202
 org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion0120 sec  
202
 
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
  9.8 sec 227
{noformat}


was (Author: jcamachorodriguez):
Once we get to the final list, I will include it in the description for 
reference. For the time being, this is the initial list of candidates:

{noformat}
 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fouter_join_ppr]
8.9 sec 9
 org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mm_bhif]   
2.6 sec 9
 
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[special_character_in_tabnames_1]
   0.25 sec9
 
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acidvec_part]
  0.51 sec9
 
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_vec_part_llap_io]
  0.42 sec9
 
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
  35 sec  9
 org.apache.hadoop.hive.metastore.TestStats.partitionedTableDeprecatedCalls 
7.8 sec 9
 org.apache.hadoop.hive.metastore.TestOldSchema.testPartitionOps5.8 sec 
9
 org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 22 ms   
9
 
org.apache.hadoop.hive.ql.parse.TestCopyUtils.testPrivilegedDistCpWithSameUserAsCurrentDoesNotTryToImpersonate
 12 sec  9
 
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAcidTablesBootstrapWithOpenTxnsTimeout
  1 min 24 sec9
 
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAcidTablesBootstrap
 36 sec  9
 org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress  1 min 23 sec
9
 org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel  1 min 
25 sec9
 org.apache.hive.jdbc.TestSSL.testSSLFetchHttp  5.9 sec 9
 org.apache.hive.spark.client.rpc.TestRpc.testServerPort19 ms   9
 org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 16 sec  44
 org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1] 
9.7 sec 86
 
org.apache.hadoop.hive.ql.parse.TestReplicationOnHDFSEncryptedZones.targetAndSourceHaveDifferentEncryptionZoneKeys
 13 sec  87
 org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] 
49 sec  99
 org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion   45 sec  202
 

[jira] [Updated] (HIVE-19476) Fix failures in TestReplicationScenariosAcidTables, TestReplicationOnHDFSEncryptedZones and TestCopyUtils

[Sankar Hariappan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sankar Hariappan updated HIVE-19476:

Fix Version/s: 3.0.0
   Status: Patch Available  (was: Open)

> Fix failures in TestReplicationScenariosAcidTables, 
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils
> -
>
> Key: HIVE-19476
> URL: https://issues.apache.org/jira/browse/HIVE-19476
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
>  Labels: Tests, pull-request-available
> Fix For: 3.0.0, 3.1.0
>
> Attachments: HIVE-19476.01-branch-3.patch, HIVE-19476.01.patch, 
> HIVE-19476.02.patch
>
>
> TestReplicationScenariosAcidTables broken due to HIVE-17657.
> cc [~sershe]
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils are failing due to 
> auto-stats gathered which thrown exception irrelevant to theses tests.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19476) Fix failures in TestReplicationScenariosAcidTables, TestReplicationOnHDFSEncryptedZones and TestCopyUtils

[Sankar Hariappan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473134#comment-16473134
 ] 

Sankar Hariappan commented on HIVE-19476:
-

Test failures are irrelevant.

Patch committed to master. Attached branch-3 patch for ptest build.

> Fix failures in TestReplicationScenariosAcidTables, 
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils
> -
>
> Key: HIVE-19476
> URL: https://issues.apache.org/jira/browse/HIVE-19476
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
>  Labels: Tests, pull-request-available
> Fix For: 3.0.0, 3.1.0
>
> Attachments: HIVE-19476.01-branch-3.patch, HIVE-19476.01.patch, 
> HIVE-19476.02.patch
>
>
> TestReplicationScenariosAcidTables broken due to HIVE-17657.
> cc [~sershe]
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils are failing due to 
> auto-stats gathered which thrown exception irrelevant to theses tests.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19476) Fix failures in TestReplicationScenariosAcidTables, TestReplicationOnHDFSEncryptedZones and TestCopyUtils

[Sankar Hariappan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sankar Hariappan updated HIVE-19476:

Status: Open  (was: Patch Available)

> Fix failures in TestReplicationScenariosAcidTables, 
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils
> -
>
> Key: HIVE-19476
> URL: https://issues.apache.org/jira/browse/HIVE-19476
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
>  Labels: Tests, pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-19476.01-branch-3.patch, HIVE-19476.01.patch, 
> HIVE-19476.02.patch
>
>
> TestReplicationScenariosAcidTables broken due to HIVE-17657.
> cc [~sershe]
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils are failing due to 
> auto-stats gathered which thrown exception irrelevant to theses tests.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19476) Fix failures in TestReplicationScenariosAcidTables, TestReplicationOnHDFSEncryptedZones and TestCopyUtils

[Sankar Hariappan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sankar Hariappan updated HIVE-19476:

Attachment: HIVE-19476.01-branch-3.patch

> Fix failures in TestReplicationScenariosAcidTables, 
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils
> -
>
> Key: HIVE-19476
> URL: https://issues.apache.org/jira/browse/HIVE-19476
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
>  Labels: Tests, pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-19476.01-branch-3.patch, HIVE-19476.01.patch, 
> HIVE-19476.02.patch
>
>
> TestReplicationScenariosAcidTables broken due to HIVE-17657.
> cc [~sershe]
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils are failing due to 
> auto-stats gathered which thrown exception irrelevant to theses tests.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19509) Disable tests that are failing continuously

[Jesus Camacho Rodriguez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesus Camacho Rodriguez updated HIVE-19509:
---
Attachment: HIVE-19509.patch

> Disable tests that are failing continuously
> ---
>
> Key: HIVE-19509
> URL: https://issues.apache.org/jira/browse/HIVE-19509
> Project: Hive
>  Issue Type: Task
>  Components: Testing Infrastructure, Tests
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19509.patch
>
>
> As per discussion in mailing list. We will bring the failure count to zero 
> and be less tolerant with ptest runs that are not clean.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19465) Upgrade ORC to 1.5.0

[Jesus Camacho Rodriguez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesus Camacho Rodriguez updated HIVE-19465:
---
Attachment: HIVE-19465.02.patch

> Upgrade ORC to 1.5.0
> 
>
> Key: HIVE-19465
> URL: https://issues.apache.org/jira/browse/HIVE-19465
> Project: Hive
>  Issue Type: Improvement
>  Components: File Formats
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19465.01.patch, HIVE-19465.02.patch, 
> HIVE-19465.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19465) Upgrade ORC to 1.5.0

[Jesus Camacho Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473132#comment-16473132
 ] 

Jesus Camacho Rodriguez commented on HIVE-19465:


Rebased patch.

> Upgrade ORC to 1.5.0
> 
>
> Key: HIVE-19465
> URL: https://issues.apache.org/jira/browse/HIVE-19465
> Project: Hive
>  Issue Type: Improvement
>  Components: File Formats
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19465.01.patch, HIVE-19465.02.patch, 
> HIVE-19465.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19166) TestMiniLlapLocalCliDriver sysdb failure

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473123#comment-16473123
 ] 

Hive QA commented on HIVE-19166:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
16s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}  2m 11s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10855/dev-support/hive-personality.sh
 |
| git revision | master / 615e71e |
| modules | C: . ql U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10855/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> TestMiniLlapLocalCliDriver sysdb failure
> 
>
> Key: HIVE-19166
> URL: https://issues.apache.org/jira/browse/HIVE-19166
> Project: Hive
>  Issue Type: Sub-task
>  Components: Test
>Reporter: Vineet Garg
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19166.04.patch, HIVE-19166.05.patch, 
> HIVE-19166.06.patch, HIVE-19166.1.patch, HIVE-19166.2.patch, 
> HIVE-19166.3.patch
>
>
> Broken by HIVE-18715



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work started] (HIVE-19509) Disable tests that are failing continuously

[Jesus Camacho Rodriguez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HIVE-19509 started by Jesus Camacho Rodriguez.
--
> Disable tests that are failing continuously
> ---
>
> Key: HIVE-19509
> URL: https://issues.apache.org/jira/browse/HIVE-19509
> Project: Hive
>  Issue Type: Task
>  Components: Testing Infrastructure, Tests
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> As per discussion in mailing list. We will bring the failure count to zero 
> and be less tolerant with ptest runs that are not clean.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19509) Disable tests that are failing continuously

[Jesus Camacho Rodriguez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesus Camacho Rodriguez reassigned HIVE-19509:
--


> Disable tests that are failing continuously
> ---
>
> Key: HIVE-19509
> URL: https://issues.apache.org/jira/browse/HIVE-19509
> Project: Hive
>  Issue Type: Task
>  Components: Testing Infrastructure, Tests
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> As per discussion in mailing list. We will bring the failure count to zero 
> and be less tolerant with ptest runs that are not clean.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (HIVE-19497) SessionHiveMetaStoreClient.getTable should respect default catalog

[Dongjoon Hyun (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473119#comment-16473119
 ] 

Dongjoon Hyun edited comment on HIVE-19497 at 5/12/18 2:49 PM:
---

Thank you for review, [~vgarg].

According to the other JIRA issue, the result(35 test failures) looks good. 
Please let me know if there is something for me to do here. [~alangates] and 
[~vgarg].


was (Author: dongjoon):
Thank you for review, [~vgarg].

According to the other JIRA issue, the result looks good. Please let me know if 
there is something for me to do here. [~alangates] and [~vgarg].

> SessionHiveMetaStoreClient.getTable should respect default catalog
> --
>
> Key: HIVE-19497
> URL: https://issues.apache.org/jira/browse/HIVE-19497
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Dongjoon Hyun
>Priority: Blocker
> Attachments: 
> 0001-HIVE-19497-SessionHiveMetaStoreClient.getTable-shoul.patch
>
>
> Currently, SessionHiveMetaStoreClient always use `DEFAULT_CATALOG_NAME` 
> during looking up the underlying tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19497) SessionHiveMetaStoreClient.getTable should respect default catalog

[Dongjoon Hyun (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473119#comment-16473119
 ] 

Dongjoon Hyun commented on HIVE-19497:
--

Thank you for review, [~vgarg].

According to the other JIRA issue, the result looks good. Please let me know if 
there is something for me to do here. [~alangates] and [~vgarg].

> SessionHiveMetaStoreClient.getTable should respect default catalog
> --
>
> Key: HIVE-19497
> URL: https://issues.apache.org/jira/browse/HIVE-19497
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Dongjoon Hyun
>Priority: Blocker
> Attachments: 
> 0001-HIVE-19497-SessionHiveMetaStoreClient.getTable-shoul.patch
>
>
> Currently, SessionHiveMetaStoreClient always use `DEFAULT_CATALOG_NAME` 
> during looking up the underlying tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19101) Apply rule [HiveJoinPushTransitivePredicatesRule] is getting stuck when there are huge number of predicates

[Jesus Camacho Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473118#comment-16473118
 ] 

Jesus Camacho Rodriguez commented on HIVE-19101:


[~vitalii], HIVE-19433 may fix the issue. I agree there is no reason to not use 
the Calcite original methods, we need to check what are the differences between 
the Hive and Calcite version: if there are contributions that have been made to 
Hive but not ported to Calcite, we should push them to Calcite and remove the 
methods from the Hive side when we upgrade to new version. If there are no 
differences, it does not make sense to have the same code in two different 
places.

> Apply rule [HiveJoinPushTransitivePredicatesRule] is getting stuck when there 
> are huge number of predicates 
> 
>
> Key: HIVE-19101
> URL: https://issues.apache.org/jira/browse/HIVE-19101
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Affects Versions: 2.1.1, 2.2.0, 2.3.0, 2.3.1, 2.3.2
>Reporter: Ganesha Shreedhara
>Priority: Major
> Attachments: queries
>
>
> Hive query is getting stuck during the optimisation phase while applying 
> HiveJoinPushTransitivePredicatesRule when there are huge number of predicates.
>  
> *DEBUG Log:*
> {code:java}
> 2018-04-04T11:22:47,991 [user: ganeshas] -1 DEBUG 
> [6f5e8faa-505c-48e3-a2cd-ce7bfced27f0 main] plan.RelOptPlanner: call#10963: 
> Apply rule [ReduceExpressionsRule(Join)] to 
> [rel#881:HiveJoin.HIVE.[](left=HepRelVertex#879,right=HepRelVertex#887,condition==($1,
>  $73),joinType=inner,algorithm=none,cost=not available)]
> 2018-04-04T11:22:48,359 [user: ganeshas] -1 DEBUG 
> [6f5e8faa-505c-48e3-a2cd-ce7bfced27f0 main] plan.RelOptPlanner: call#10964: 
> Apply rule [HiveJoinAddNotNullRule] to 
> [rel#881:HiveJoin.HIVE.[](left=HepRelVertex#879,right=HepRelVertex#887,condition==($1,
>  $73),joinType=inner,algorithm=none,cost=not available)]
> 2018-04-04T11:22:48,360 [user: ganeshas] -1 DEBUG 
> [6f5e8faa-505c-48e3-a2cd-ce7bfced27f0 main] plan.RelOptPlanner: call#10965: 
> Apply rule [HiveJoinPushTransitivePredicatesRule] to 
> [rel#881:HiveJoin.HIVE.[](left=HepRelVertex#879,right=HepRelVertex#887,condition==($1,
>  $73),joinType=inner,algorithm=none,cost=not available)]{code}
>  
> *Thread Status:*
>  
> {code:java}
> "6f5e8faa-505c-48e3-a2cd-ce7bfced27f0 main" prio=5 tid=0x7ff18e006800 
> nid=0x1c03 runnable [0x78176000]
>  java.lang.Thread.State: RUNNABLE
>  at java.util.Arrays.copyOfRange(Arrays.java:2694)
>  at java.lang.String.(String.java:203)
>  at java.lang.StringBuilder.toString(StringBuilder.java:405)
>  at org.apache.calcite.rex.RexCall.computeDigest(RexCall.java:95)
>  at org.apache.calcite.rex.RexCall.toString(RexCall.java:100)
>  at org.apache.calcite.rex.RexCall.computeDigest(RexCall.java:84)
>  at org.apache.calcite.rex.RexCall.toString(RexCall.java:100)
>  at 
> org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdPredicates$JoinConditionBasedPredicateInference.infer(HiveRelMdPredicates.java:516)
>  at 
> org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdPredicates$JoinConditionBasedPredicateInference.inferPredicates(HiveRelMdPredicates.java:426)
>  at 
> org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdPredicates.getPredicates(HiveRelMdPredicates.java:186)
>  at GeneratedMetadataHandler_Predicates.getPredicates_$(Unknown Source)
>  at GeneratedMetadataHandler_Predicates.getPredicates(Unknown Source)
>  at 
> org.apache.calcite.rel.metadata.RelMetadataQuery.getPulledUpPredicates(RelMetadataQuery.java:721)
>  at 
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveJoinPushTransitivePredicatesRule.onMatch(HiveJoinPushTransitivePredicatesRule.java:83)
>  at 
> org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:314)
>  at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:502)
>  at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:381)
>  at 
> org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:275)
>  at 
> org.apache.calcite.plan.hep.HepInstruction$RuleCollection.execute(HepInstruction.java:72)
>  at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:206)
>  at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:193)
>  at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.hepPlan(CalcitePlanner.java:1575)
>  at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.applyPreJoinOrderingTransforms(CalcitePlanner.java:1448)
>  at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1174)
>  at 
> 

[jira] [Commented] (HIVE-19476) Fix failures in TestReplicationScenariosAcidTables, TestReplicationOnHDFSEncryptedZones and TestCopyUtils

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473117#comment-16473117
 ] 

Hive QA commented on HIVE-19476:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922976/HIVE-19476.02.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 32 failed/errored test(s), 13568 tests 
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=94)

[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,scriptfile1.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

[Rui Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473112#comment-16473112
 ] 

Rui Li commented on HIVE-18533:
---

Thanks for the update [~stakiar]. There's some issue with 
TestSparkLauncherSparkClient. I think we can't assert the Future is done after 
we issue the state change -- there can be a race condition and it's not against 
the Future contracts. We can remove those assertions, or we can assert 
Future::get should return in some reasonable amount of time.

> Add option to use InProcessLauncher to submit spark jobs
> 
>
> Key: HIVE-18533
> URL: https://issues.apache.org/jira/browse/HIVE-18533
> Project: Hive
>  Issue Type: Improvement
>  Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18533.1.patch, HIVE-18533.2.patch, 
> HIVE-18533.3.patch, HIVE-18533.4.patch, HIVE-18533.5.patch, 
> HIVE-18533.6.patch, HIVE-18533.7.patch, HIVE-18533.8.patch, 
> HIVE-18533.9.patch, HIVE-18533.91.patch, HIVE-18533.94.patch, 
> HIVE-18831.93.patch
>
>
> See discussion in HIVE-16484 for details.
> I think this will help with reducing the amount of time it takes to open a 
> HoS session + debuggability (no need launch a separate process to run a Spark 
> app).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19101) Apply rule [HiveJoinPushTransitivePredicatesRule] is getting stuck when there are huge number of predicates

[Vitalii Diravka (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473106#comment-16473106
 ] 

Vitalii Diravka commented on HIVE-19101:


[~jcamachorodriguez] I think moving onto Calcite 
JoinPushTransitivePredicatesRule and RelMdPredicates to infer the predicates 
should be considered for HIve CBO.
I assume there is no such issue for Calcite instances.

> Apply rule [HiveJoinPushTransitivePredicatesRule] is getting stuck when there 
> are huge number of predicates 
> 
>
> Key: HIVE-19101
> URL: https://issues.apache.org/jira/browse/HIVE-19101
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Affects Versions: 2.1.1, 2.2.0, 2.3.0, 2.3.1, 2.3.2
>Reporter: Ganesha Shreedhara
>Priority: Major
> Attachments: queries
>
>
> Hive query is getting stuck during the optimisation phase while applying 
> HiveJoinPushTransitivePredicatesRule when there are huge number of predicates.
>  
> *DEBUG Log:*
> {code:java}
> 2018-04-04T11:22:47,991 [user: ganeshas] -1 DEBUG 
> [6f5e8faa-505c-48e3-a2cd-ce7bfced27f0 main] plan.RelOptPlanner: call#10963: 
> Apply rule [ReduceExpressionsRule(Join)] to 
> [rel#881:HiveJoin.HIVE.[](left=HepRelVertex#879,right=HepRelVertex#887,condition==($1,
>  $73),joinType=inner,algorithm=none,cost=not available)]
> 2018-04-04T11:22:48,359 [user: ganeshas] -1 DEBUG 
> [6f5e8faa-505c-48e3-a2cd-ce7bfced27f0 main] plan.RelOptPlanner: call#10964: 
> Apply rule [HiveJoinAddNotNullRule] to 
> [rel#881:HiveJoin.HIVE.[](left=HepRelVertex#879,right=HepRelVertex#887,condition==($1,
>  $73),joinType=inner,algorithm=none,cost=not available)]
> 2018-04-04T11:22:48,360 [user: ganeshas] -1 DEBUG 
> [6f5e8faa-505c-48e3-a2cd-ce7bfced27f0 main] plan.RelOptPlanner: call#10965: 
> Apply rule [HiveJoinPushTransitivePredicatesRule] to 
> [rel#881:HiveJoin.HIVE.[](left=HepRelVertex#879,right=HepRelVertex#887,condition==($1,
>  $73),joinType=inner,algorithm=none,cost=not available)]{code}
>  
> *Thread Status:*
>  
> {code:java}
> "6f5e8faa-505c-48e3-a2cd-ce7bfced27f0 main" prio=5 tid=0x7ff18e006800 
> nid=0x1c03 runnable [0x78176000]
>  java.lang.Thread.State: RUNNABLE
>  at java.util.Arrays.copyOfRange(Arrays.java:2694)
>  at java.lang.String.(String.java:203)
>  at java.lang.StringBuilder.toString(StringBuilder.java:405)
>  at org.apache.calcite.rex.RexCall.computeDigest(RexCall.java:95)
>  at org.apache.calcite.rex.RexCall.toString(RexCall.java:100)
>  at org.apache.calcite.rex.RexCall.computeDigest(RexCall.java:84)
>  at org.apache.calcite.rex.RexCall.toString(RexCall.java:100)
>  at 
> org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdPredicates$JoinConditionBasedPredicateInference.infer(HiveRelMdPredicates.java:516)
>  at 
> org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdPredicates$JoinConditionBasedPredicateInference.inferPredicates(HiveRelMdPredicates.java:426)
>  at 
> org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdPredicates.getPredicates(HiveRelMdPredicates.java:186)
>  at GeneratedMetadataHandler_Predicates.getPredicates_$(Unknown Source)
>  at GeneratedMetadataHandler_Predicates.getPredicates(Unknown Source)
>  at 
> org.apache.calcite.rel.metadata.RelMetadataQuery.getPulledUpPredicates(RelMetadataQuery.java:721)
>  at 
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveJoinPushTransitivePredicatesRule.onMatch(HiveJoinPushTransitivePredicatesRule.java:83)
>  at 
> org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:314)
>  at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:502)
>  at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:381)
>  at 
> org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:275)
>  at 
> org.apache.calcite.plan.hep.HepInstruction$RuleCollection.execute(HepInstruction.java:72)
>  at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:206)
>  at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:193)
>  at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.hepPlan(CalcitePlanner.java:1575)
>  at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.applyPreJoinOrderingTransforms(CalcitePlanner.java:1448)
>  at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1174)
>  at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1096)
>  at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:113)
>  at 
> org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:997)
>  at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:149)
>  at 

[jira] [Commented] (HIVE-19476) Fix failures in TestReplicationScenariosAcidTables, TestReplicationOnHDFSEncryptedZones and TestCopyUtils

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473100#comment-16473100
 ] 

Hive QA commented on HIVE-19476:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
 0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
56s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
34s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
53s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
15s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 14s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10854/dev-support/hive-personality.sh
 |
| git revision | master / 615e71e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: itests/hive-unit ql U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10854/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Fix failures in TestReplicationScenariosAcidTables, 
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils
> -
>
> Key: HIVE-19476
> URL: https://issues.apache.org/jira/browse/HIVE-19476
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
>  Labels: Tests, pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-19476.01.patch, HIVE-19476.02.patch
>
>
> TestReplicationScenariosAcidTables broken due to HIVE-17657.
> cc [~sershe]
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils are failing due to 
> auto-stats gathered which thrown exception irrelevant to theses tests.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19497) SessionHiveMetaStoreClient.getTable should respect default catalog

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473090#comment-16473090
 ] 

Hive QA commented on HIVE-19497:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922943/0001-HIVE-19497-SessionHiveMetaStoreClient.getTable-shoul.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 35 failed/errored test(s), 13578 tests 
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Commented] (HIVE-19497) SessionHiveMetaStoreClient.getTable should respect default catalog

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473079#comment-16473079
 ] 

Hive QA commented on HIVE-19497:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
56s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m  
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 32s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10853/dev-support/hive-personality.sh
 |
| git revision | master / 615e71e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10853/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> SessionHiveMetaStoreClient.getTable should respect default catalog
> --
>
> Key: HIVE-19497
> URL: https://issues.apache.org/jira/browse/HIVE-19497
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Dongjoon Hyun
>Priority: Blocker
> Attachments: 
> 0001-HIVE-19497-SessionHiveMetaStoreClient.getTable-shoul.patch
>
>
> Currently, SessionHiveMetaStoreClient always use `DEFAULT_CATALOG_NAME` 
> during looking up the underlying tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19493) VectorUDFDateDiffColCol copySelected does not handle nulls correctly

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473072#comment-16473072
 ] 

Hive QA commented on HIVE-19493:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922939/HIVE-19493.02.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 42 failed/errored test(s), 13578 tests 
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Commented] (HIVE-19493) VectorUDFDateDiffColCol copySelected does not handle nulls correctly

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473047#comment-16473047
 ] 

Hive QA commented on HIVE-19493:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
58s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
37s{color} | {color:green} ql: The patch generated 0 new + 21 unchanged - 2 
fixed = 21 total (was 23) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m  
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 36s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10852/dev-support/hive-personality.sh
 |
| git revision | master / 615e71e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10852/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> VectorUDFDateDiffColCol copySelected does not handle nulls correctly
> 
>
> Key: HIVE-19493
> URL: https://issues.apache.org/jira/browse/HIVE-19493
> Project: Hive
>  Issue Type: Bug
>  Components: Vectorization
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-19493.01.patch, HIVE-19493.02.patch
>
>
> The {{copySelected}} method in {{VectorUDFDateDiffColCol}} class was missed 
> during HIVE-18622



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473025#comment-16473025
 ] 

Hive QA commented on HIVE-19307:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922930/HIVE-19307.5.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 38 failed/errored test(s), 13578 tests 
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473012#comment-16473012
 ] 

Hive QA commented on HIVE-19307:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
 0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 2s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
30s{color} | {color:blue} common in master has 62 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
47s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
41s{color} | {color:red} ql: The patch generated 14 new + 576 unchanged - 3 
fixed = 590 total (was 579) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch 2 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m  0s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10851/dev-support/hive-personality.sh
 |
| git revision | master / 615e71e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10851/yetus/diff-checkstyle-ql.txt
 |
| whitespace | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10851/yetus/whitespace-tabs.txt
 |
| modules | C: common ql U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10851/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Support ArrowOutputStream in LlapOutputFormatService
> 
>
> Key: HIVE-19307
> URL: https://issues.apache.org/jira/browse/HIVE-19307
> Project: Hive
>  Issue Type: Task
>  Components: llap
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-19307.3.patch, HIVE-19307.4.patch, 
> HIVE-19307.5.patch
>
>
> Support pushing arrow batches through 
> org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19370) Issue: ADD Months function on timestamp datatype fields in hive

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473001#comment-16473001
 ] 

Hive QA commented on HIVE-19370:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922932/HIVE-19370.01.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 35 failed/errored test(s), 13579 tests 
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Updated] (HIVE-12898) Hive should support ORC block skipping on nested fields

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-12898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated HIVE-12898:
--
Labels: pull-request-available  (was: )

> Hive should support ORC block skipping on nested fields
> ---
>
> Key: HIVE-12898
> URL: https://issues.apache.org/jira/browse/HIVE-12898
> Project: Hive
>  Issue Type: Improvement
>  Components: ORC
>Affects Versions: 0.14.0, 1.2.1
>Reporter: Michael Haeusler
>Assignee: Ashish Sharma
>Priority: Major
>  Labels: pull-request-available
>
> Hive supports predicate pushdown (block skipping) for ORC tables only on 
> top-level fields. Hive should also support block skipping on nested fields 
> (within structs).
> Example top-level: the following query selects 0 rows, using a predicate on 
> top-level column foo. We also see 0 INPUT_RECORDS in the summary:
> {code:sql}
> SET hive.tez.exec.print.summary=true;
> CREATE TABLE t_toplevel STORED AS ORC AS SELECT 23 AS foo;
> SELECT * FROM t_toplevel WHERE foo=42 ORDER BY foo;
> [...]
> VERTICES TOTAL_TASKS  FAILED_ATTEMPTS KILLED_TASKS DURATION_SECONDS   
>  CPU_TIME_MILLIS GC_TIME_MILLIS  INPUT_RECORDS   OUTPUT_RECORDS
> Map 1  100 1.22   
>2,640102  00
> {code}
> Example nested: the following query also selects 0 rows, but using a 
> predicate on nested column foo.bar. Unfortunately we see 1 INPUT_RECORDS in 
> the summary:
> {code:sql}
> SET hive.tez.exec.print.summary=true;
> CREATE TABLE t_nested STORED AS ORC AS SELECT NAMED_STRUCT('bar', 23) AS foo;
> SELECT * FROM t_nested WHERE foo.bar=42 ORDER BY foo;
> [...]
> VERTICES TOTAL_TASKS  FAILED_ATTEMPTS KILLED_TASKS DURATION_SECONDS   
>  CPU_TIME_MILLIS GC_TIME_MILLIS  INPUT_RECORDS   OUTPUT_RECORDS
> Map 1  100 3.66   
>5,210 68  10
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-12898) Hive should support ORC block skipping on nested fields

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-12898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472993#comment-16472993
 ] 

ASF GitHub Bot commented on HIVE-12898:
---

GitHub user ashish-kumar-sharma opened a pull request:

https://github.com/apache/hive/pull/346

HIVE-12898: First commit

1. Predicate Pushdown For Nested field

1.1 Objective

In the ORC(Optimized Row Columnar) all the primitive type column consist of 
index. Predicate refer to the column name in where clause and pushdown mean 
skipping rows groups, strips and block while reading by comparing the meta 
store in the strips. Meta consist of max, sum ,min value present in the given 
column. 

Currently predicate pushdown only work for top level column of the schema. 
Extending the Predicate Pushdown for nested structure in hive.  


1.2 Current state - 
 
1.2.1 Schema
struct>
 
1.2.2 Query 
select col3.col5.col6 from table where col3.col5.col6 > 10;
 
1.2.3 Conf 
Hive.io.filter.expr.serialized = “ASdni2enalfkncwjnlsdnfrnqwoglqernmgkqrg”;
Hive.io.filter.text - “where c.e.f > 10”;
 
1.2.4 Pushdown Predicate not supported in Nested field
 
Generate ExprNodeGenericFuncDesc  object which is of type ExprNodeFieldDesc 
which is serialized and stored in Hive.io.filter.expr.serialized.

But while parsing ExprNodeGenericFuncDesc object to generate searchArg in 
function ConvertAstToSearchArg() there is strict checking of  
(ExprNodeGenericFuncDesc instanceof ExprNodeColumnDesc). Due to which it 
completely skip the SearchArgment creation.  


1.2.5 Result - 

builder.literal(SearchArgument.TruthValue.YES_NO_NULL);

1.3 Expected state - 

1.3.1 Schema
struct>
 
1.3.2 Query
select col3.col5.col6 from table where col3.col5.col6 > 10;
 
1.3.3 Conf
Hive.io.filter.expr.serialized = “ASdni2enalfkncwjnlsdnfrnqwoglqernmgkqrg”;
Hive.io.filter.text - “where c.e.f > 10”;
 
1.3.4 Pushdown Predicate support in Nested field
 
Generate ExprNodeGenericFuncDesc  object which is of type ExprNodeFieldDesc 
which is serialized and stored in Hive.io.filter.expr.serialized.

But while parsing ExprNodeGenericFuncDesc object to generate searchArg in 
function ConvertAstToSearchArg() there should also contain an check for 
ExprNodeFieldDesc and separate parsing plan which convert the fieldName to 
ColumnID and generate PredicateLeaf nodes.

1.3.5 Result

leaf-0 = (LESS_THAN c.e.f 10), expr = (not leaf-0)


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Flipkart/hive nestedppd

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/346.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #346


commit f3e46b62b4fab6877f2373c49d933ebe7119ec2f
Author: Aashish Kumar Sharma 
Date:   2018-05-12T08:47:39Z

HIVE-12898: First commit




> Hive should support ORC block skipping on nested fields
> ---
>
> Key: HIVE-12898
> URL: https://issues.apache.org/jira/browse/HIVE-12898
> Project: Hive
>  Issue Type: Improvement
>  Components: ORC
>Affects Versions: 0.14.0, 1.2.1
>Reporter: Michael Haeusler
>Assignee: Ashish Sharma
>Priority: Major
>  Labels: pull-request-available
>
> Hive supports predicate pushdown (block skipping) for ORC tables only on 
> top-level fields. Hive should also support block skipping on nested fields 
> (within structs).
> Example top-level: the following query selects 0 rows, using a predicate on 
> top-level column foo. We also see 0 INPUT_RECORDS in the summary:
> {code:sql}
> SET hive.tez.exec.print.summary=true;
> CREATE TABLE t_toplevel STORED AS ORC AS SELECT 23 AS foo;
> SELECT * FROM t_toplevel WHERE foo=42 ORDER BY foo;
> [...]
> VERTICES TOTAL_TASKS  FAILED_ATTEMPTS KILLED_TASKS DURATION_SECONDS   
>  CPU_TIME_MILLIS GC_TIME_MILLIS  INPUT_RECORDS   OUTPUT_RECORDS
> Map 1  100 1.22   
>2,640102  00
> {code}
> Example nested: the following query also selects 0 rows, but using a 
> predicate on nested column foo.bar. Unfortunately we see 1 INPUT_RECORDS in 
> the summary:
> {code:sql}
> SET hive.tez.exec.print.summary=true;
> CREATE TABLE t_nested STORED AS ORC AS SELECT NAMED_STRUCT('bar', 23) AS foo;
> SELECT * FROM 

[jira] [Commented] (HIVE-19370) Issue: ADD Months function on timestamp datatype fields in hive

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472990#comment-16472990
 ] 

Hive QA commented on HIVE-19370:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
51s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
31s{color} | {color:blue} common in master has 62 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
52s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
15s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
10s{color} | {color:red} common: The patch generated 5 new + 7 unchanged - 0 
fixed = 12 total (was 7) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 59s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10849/dev-support/hive-personality.sh
 |
| git revision | master / 615e71e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10849/yetus/diff-checkstyle-common.txt
 |
| modules | C: common ql U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10849/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Issue: ADD Months function on timestamp datatype fields in hive
> ---
>
> Key: HIVE-19370
> URL: https://issues.apache.org/jira/browse/HIVE-19370
> Project: Hive
>  Issue Type: Bug
>Reporter: Amit Chauhan
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
> Attachments: HIVE-19370.01.patch
>
>
> *Issue:*
> while using ADD_Months function on a timestamp datatype column the output 
> omits the time part[HH:MM:SS] part from output.
> which should not be the case.
> *query:* EMAIL_FAILURE_DTMZ is of datatype timestamp in hive.
> hive> select CUSTOMER_ID,EMAIL_FAILURE_DTMZ,ADD_MONTHS (EMAIL_FAILURE_DTMZ , 
> 1) from TABLE1 where CUSTOMER_ID=125674937;
> OK
> 125674937   2015-12-09 12:25:53 2016-01-09
> *hiver version :*
> hive> !hive --version;
>  Hive 1.2.1000.2.5.6.0-40
>  
> can you please help if somehow I can get below as output:
>  
> 125674937   2015-12-09 12:25:53   2016-01-09 12:25:53



--
This message was 

[jira] [Updated] (HIVE-18866) Semijoin: Implement a Long -> Hash64 vector fast-path

[Gopal V (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-18866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gopal V updated HIVE-18866:
---
Attachment: 0001-hash64-WIP.patch

> Semijoin: Implement a Long -> Hash64 vector fast-path
> -
>
> Key: HIVE-18866
> URL: https://issues.apache.org/jira/browse/HIVE-18866
> Project: Hive
>  Issue Type: Improvement
>  Components: Vectorization
>Reporter: Gopal V
>Priority: Major
>  Labels: performance
> Attachments: 0001-hash64-WIP.patch, perf-hash64-long.png
>
>
> A significant amount of CPU is wasted with JMM restrictions on byte[] arrays.
> To transform from one Long -> another Long, this goes into a byte[] array, 
> which shows up as a hotspot.
> !perf-hash64-long.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-18866) Semijoin: Implement a Long -> Hash64 vector fast-path

[Gopal V (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-18866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gopal V updated HIVE-18866:
---
Attachment: (was: HIVE-18079.9.patch)

> Semijoin: Implement a Long -> Hash64 vector fast-path
> -
>
> Key: HIVE-18866
> URL: https://issues.apache.org/jira/browse/HIVE-18866
> Project: Hive
>  Issue Type: Improvement
>  Components: Vectorization
>Reporter: Gopal V
>Priority: Major
>  Labels: performance
> Attachments: perf-hash64-long.png
>
>
> A significant amount of CPU is wasted with JMM restrictions on byte[] arrays.
> To transform from one Long -> another Long, this goes into a byte[] array, 
> which shows up as a hotspot.
> !perf-hash64-long.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-18866) Semijoin: Implement a Long -> Hash64 vector fast-path

[Gopal V (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-18866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gopal V updated HIVE-18866:
---
Attachment: HIVE-18079.9.patch

> Semijoin: Implement a Long -> Hash64 vector fast-path
> -
>
> Key: HIVE-18866
> URL: https://issues.apache.org/jira/browse/HIVE-18866
> Project: Hive
>  Issue Type: Improvement
>  Components: Vectorization
>Reporter: Gopal V
>Priority: Major
>  Labels: performance
> Attachments: perf-hash64-long.png
>
>
> A significant amount of CPU is wasted with JMM restrictions on byte[] arrays.
> To transform from one Long -> another Long, this goes into a byte[] array, 
> which shows up as a hotspot.
> !perf-hash64-long.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19494) Accept shade prefix during reflective instantiation of output format

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472973#comment-16472973
 ] 

Hive QA commented on HIVE-19494:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922922/HIVE-19494.1.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 36 failed/errored test(s), 13578 tests 
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Updated] (HIVE-18079) Statistics: Allow HyperLogLog to be merged to the lowest-common-denominator bit-size

[Gopal V (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-18079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gopal V updated HIVE-18079:
---
Attachment: HIVE-18079.12.patch

> Statistics: Allow HyperLogLog to be merged to the lowest-common-denominator 
> bit-size
> 
>
> Key: HIVE-18079
> URL: https://issues.apache.org/jira/browse/HIVE-18079
> Project: Hive
>  Issue Type: Improvement
>  Components: Standalone Metastore, Statistics
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18079.1.patch, HIVE-18079.10.patch, 
> HIVE-18079.11.patch, HIVE-18079.12.patch, HIVE-18079.2.patch, 
> HIVE-18079.4.patch, HIVE-18079.5.patch, HIVE-18079.6.patch, 
> HIVE-18079.7.patch, HIVE-18079.8.patch, HIVE-18079.9.patch
>
>
> HyperLogLog can merge a 14 bit HLL into a 10 bit HLL bitset, because of its 
> mathematical hash distribution & construction.
> Allow the squashing of a 14 bit HLL -> 10 bit HLL without needing a second 
> scan over the data-set.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18079) Statistics: Allow HyperLogLog to be merged to the lowest-common-denominator bit-size

[Gopal V (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-18079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472967#comment-16472967
 ] 

Gopal V commented on HIVE-18079:


Fixed the ordering issue in tez_vector_dynpart_hashjoin_1.q

> Statistics: Allow HyperLogLog to be merged to the lowest-common-denominator 
> bit-size
> 
>
> Key: HIVE-18079
> URL: https://issues.apache.org/jira/browse/HIVE-18079
> Project: Hive
>  Issue Type: Improvement
>  Components: Standalone Metastore, Statistics
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18079.1.patch, HIVE-18079.10.patch, 
> HIVE-18079.11.patch, HIVE-18079.12.patch, HIVE-18079.2.patch, 
> HIVE-18079.4.patch, HIVE-18079.5.patch, HIVE-18079.6.patch, 
> HIVE-18079.7.patch, HIVE-18079.8.patch, HIVE-18079.9.patch
>
>
> HyperLogLog can merge a 14 bit HLL into a 10 bit HLL bitset, because of its 
> mathematical hash distribution & construction.
> Allow the squashing of a 14 bit HLL -> 10 bit HLL without needing a second 
> scan over the data-set.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19494) Accept shade prefix during reflective instantiation of output format

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472955#comment-16472955
 ] 

Hive QA commented on HIVE-19494:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
 3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
25s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
29s{color} | {color:blue} common in master has 62 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
24s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m  
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 57s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10847/dev-support/hive-personality.sh
 |
| git revision | master / 615e71e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common streaming U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10847/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Accept shade prefix during reflective instantiation of output format
> 
>
> Key: HIVE-19494
> URL: https://issues.apache.org/jira/browse/HIVE-19494
> Project: Hive
>  Issue Type: Sub-task
>  Components: Streaming
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19494.1.patch
>
>
> Hive Streaming API jars are sometimes shaded with a different prefix when 
> used in environments where another version of hive already exists (spark for 
> example). In most cases, shading is done with rename of classes with some 
> prefix. If an uber/assembly jar is generated with renamed prefix, Hive 
> Streaming API will not work as Hive Streaming API will reflectively 
> instantiate outputformat class using FQCN string provided by metastore table 
> storage descriptor object. 
> For example: 
> RecordWriter will create instance of OutputFormat using string 
> "org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat". When a shaded jar with 
> renamed class references are used, this class will not be 

[jira] [Comment Edited] (HIVE-19501) Fix HyperLogLog to be threadsafe

[Gopal V (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472950#comment-16472950
 ] 

Gopal V edited comment on HIVE-19501 at 5/12/18 7:10 AM:
-

The short and int versions are nearly identical, except they need to account 
for the sign-bit during the cast up from int -> long by doing 

{code}
  public static long hash64(int data) {
long k1 = Integer.reverseBytes(data) & (-1L >>> 32);
int length = Integer.BYTES;
long hash = DEFAULT_SEED;
k1 *= C1;
k1 = Long.rotateLeft(k1, R1);
k1 *= C2;
hash ^= k1;
// finalization
hash ^= length;
hash = fmix64(hash);
return hash;
  }
{code}



was (Author: gopalv):
The short and int versions are nearly identical, except they need to account 
for the sign-bit during the cast up from int -> long by doing 

{code}
long k = Integer.reverseBytes(data) & (-1L >>> 32);
int length = Integer.BYTES;
{code}


> Fix HyperLogLog to be threadsafe
> 
>
> Key: HIVE-19501
> URL: https://issues.apache.org/jira/browse/HIVE-19501
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Priority: Major
>
> not sure if this is an issue in reality or not; but there are 3 static fields 
> in HyperLogLog which are rewritten during working; if there are multiple 
> threads are calculating HLL in the same JVM, there is a theoretical chance 
> that they might overwrite eachothers value...
> static fields:
> https://github.com/apache/hive/blob/8028ce8a4cf5a03e2998c33e032a511fae770b47/standalone-metastore/src/main/java/org/apache/hadoop/hive/common/ndv/hll/HyperLogLog.java#L65
> usage:
> https://github.com/apache/hive/blob/8028ce8a4cf5a03e2998c33e032a511fae770b47/standalone-metastore/src/main/java/org/apache/hadoop/hive/common/ndv/hll/HyperLogLog.java#L216



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19501) Fix HyperLogLog to be threadsafe

[Gopal V (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472950#comment-16472950
 ] 

Gopal V commented on HIVE-19501:


The short and int versions are nearly identical, except they need to account 
for the sign-bit during the cast up from int -> long by doing 

{code}
long k = Integer.reverseBytes(data) & (-1L >>> 32);
int length = Integer.BYTES;
{code}


> Fix HyperLogLog to be threadsafe
> 
>
> Key: HIVE-19501
> URL: https://issues.apache.org/jira/browse/HIVE-19501
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Priority: Major
>
> not sure if this is an issue in reality or not; but there are 3 static fields 
> in HyperLogLog which are rewritten during working; if there are multiple 
> threads are calculating HLL in the same JVM, there is a theoretical chance 
> that they might overwrite eachothers value...
> static fields:
> https://github.com/apache/hive/blob/8028ce8a4cf5a03e2998c33e032a511fae770b47/standalone-metastore/src/main/java/org/apache/hadoop/hive/common/ndv/hll/HyperLogLog.java#L65
> usage:
> https://github.com/apache/hive/blob/8028ce8a4cf5a03e2998c33e032a511fae770b47/standalone-metastore/src/main/java/org/apache/hadoop/hive/common/ndv/hll/HyperLogLog.java#L216



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19496) Check untar folder

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472946#comment-16472946
 ] 

Hive QA commented on HIVE-19496:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922921/HIVE-19496.1.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 37 failed/errored test(s), 13568 tests 
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestJdbcWithDBTokenStoreNoDoAs - did not produce a TEST-*.xml file (likely 
timed out) (batchId=254)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Commented] (HIVE-19501) Fix HyperLogLog to be threadsafe

[Gopal V (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472944#comment-16472944
 ] 

Gopal V commented on HIVE-19501:


FYI, the murmur64 for Long is actually pretty trivial.

{code}
  public static long hash64(long data) {
long hash = DEFAULT_SEED;
long k = Long.reverseBytes(data);
int length = Long.BYTES;
// mix functions
k *= C1;
k = Long.rotateLeft(k, R1);
k *= C2;
hash ^= k;
hash = Long.rotateLeft(hash, R2) * M + N1;
// finalization
hash ^= length;
hash = fmix64(hash);

return hash;
  }
{code}

and does not need a byte[] array.

> Fix HyperLogLog to be threadsafe
> 
>
> Key: HIVE-19501
> URL: https://issues.apache.org/jira/browse/HIVE-19501
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Priority: Major
>
> not sure if this is an issue in reality or not; but there are 3 static fields 
> in HyperLogLog which are rewritten during working; if there are multiple 
> threads are calculating HLL in the same JVM, there is a theoretical chance 
> that they might overwrite eachothers value...
> static fields:
> https://github.com/apache/hive/blob/8028ce8a4cf5a03e2998c33e032a511fae770b47/standalone-metastore/src/main/java/org/apache/hadoop/hive/common/ndv/hll/HyperLogLog.java#L65
> usage:
> https://github.com/apache/hive/blob/8028ce8a4cf5a03e2998c33e032a511fae770b47/standalone-metastore/src/main/java/org/apache/hadoop/hive/common/ndv/hll/HyperLogLog.java#L216



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)