[jira] [Commented] (HIVE-15223) replace org.json usage in EximUtil with some alternative
[
https://issues.apache.org/jira/browse/HIVE-15223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431762#comment-16431762
]
Hive QA commented on HIVE-15223:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12850960/HIVE-15223.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10106/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10106/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10106/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-04-10 05:42:17.007
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-10106/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-04-10 05:42:17.027
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
2e92451..dcd9b59 master -> origin/master
+ git reset --hard HEAD
HEAD is now at 2e92451 HIVE-18857: Store default value text instead of default
value expression in metastore(Vineet Garg, reviewed by Ashutosh Chauhan)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 3 commits, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at dcd9b59 HIVE-19146 : Delete dangling q.out
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-04-10 05:42:29.295
+ rm -rf ../yetus_PreCommit-HIVE-Build-10106
+ mkdir ../yetus_PreCommit-HIVE-Build-10106
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-10106
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-10106/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/EximUtil.java:28
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/parse/EximUtil.java'
with conflicts.
Going to apply patch with: git apply -p0
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/EximUtil.java:28
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/parse/EximUtil.java'
with conflicts.
U ql/src/java/org/apache/hadoop/hive/ql/parse/EximUtil.java
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12850960 - PreCommit-HIVE-Build
> replace org.json usage in EximUtil with some alternative
>
>
> Key: HIVE-15223
> URL: https://issues.apache.org/jira/browse/HIVE-15223
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Teddy Choi
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-15223.1.patch
>
>
> The metadata is stored in json format...which changed lately with the advent
> of replication v2.
> I think jackson would be nice to have here - it could possibly aid to make
> this Metadata reading / writing more resilient against future serialization
> issues.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-15444) tez.queue.name is invalid after tez job running on CLI
[
https://issues.apache.org/jira/browse/HIVE-15444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431759#comment-16431759
]
Hive QA commented on HIVE-15444:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12843757/HIVE-15444.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 38 failed/errored test(s), 13245 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=252)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=252)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[jira] [Commented] (HIVE-18831) Differentiate errors that are thrown by Spark tasks
[ https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431756#comment-16431756 ] Sahil Takiar commented on HIVE-18831: - Thanks [~lirui] addressed your comments and attached updated patch. > Differentiate errors that are thrown by Spark tasks > --- > > Key: HIVE-18831 > URL: https://issues.apache.org/jira/browse/HIVE-18831 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-18831.1.patch, HIVE-18831.2.patch, > HIVE-18831.3.patch, HIVE-18831.4.patch, HIVE-18831.6.patch, > HIVE-18831.7.patch, HIVE-18831.8.WIP.patch, HIVE-18831.9.patch, > HIVE-18831.90.patch, HIVE-18831.91.patch > > > We propagate exceptions from Spark task failures to the client well, but we > don't differentiate between errors from HS2 / RSC vs. errors thrown by > individual tasks. > Main motivation is that when the client sees a propagated Spark exception its > difficult to know what part of the excution threw the exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18831) Differentiate errors that are thrown by Spark tasks
[ https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18831: Attachment: HIVE-18831.91.patch > Differentiate errors that are thrown by Spark tasks > --- > > Key: HIVE-18831 > URL: https://issues.apache.org/jira/browse/HIVE-18831 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-18831.1.patch, HIVE-18831.2.patch, > HIVE-18831.3.patch, HIVE-18831.4.patch, HIVE-18831.6.patch, > HIVE-18831.7.patch, HIVE-18831.8.WIP.patch, HIVE-18831.9.patch, > HIVE-18831.90.patch, HIVE-18831.91.patch > > > We propagate exceptions from Spark task failures to the client well, but we > don't differentiate between errors from HS2 / RSC vs. errors thrown by > individual tasks. > Main motivation is that when the client sees a propagated Spark exception its > difficult to know what part of the excution threw the exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19133) HS2 WebUI phase-wise performance metrics not showing correctly
[ https://issues.apache.org/jira/browse/HIVE-19133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431752#comment-16431752 ] Zoltan Haindrich commented on HIVE-19133: - Thanks for looking into this [~bharos92] ! In the logs earlier "run" provided a structured entry which enclosed the other informations...the separation seems logical to me... I think the original idea was that extracting the xml parts for the same query gathers a valid xml portion...but that seems to be hard to achieve now I'm not sure if the following idea is usefull or not: would it be possible to display some info (the queryid?) to identify to which query the perf entry belongs to? since there could be multiple queries running at the same time this is already an issueI'm filtering for threadids usually...which is not that good; since some parts of it is on a different thread (HIVE-19093) while I was writing all this stuff...I convinced myself that this is way better to have (and it probably also fixes HIVE-19093) +1 > HS2 WebUI phase-wise performance metrics not showing correctly > -- > > Key: HIVE-19133 > URL: https://issues.apache.org/jira/browse/HIVE-19133 > Project: Hive > Issue Type: Bug > Components: HiveServer2, Web UI >Reporter: Bharathkrishna Guruvayoor Murali >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-19133.1.patch, WebUI-compile time query metrics.png > > > The query specific WebUI metrics (go to drilldown -> performance logging) are > not showing up in the correct phase and are often mixed up. > Attaching screenshot. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18910) Migrate to Murmur hash for shuffle and bucketing
[ https://issues.apache.org/jira/browse/HIVE-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-18910: -- Attachment: HIVE-18910.27.patch > Migrate to Murmur hash for shuffle and bucketing > > > Key: HIVE-18910 > URL: https://issues.apache.org/jira/browse/HIVE-18910 > Project: Hive > Issue Type: Task >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-18910.1.patch, HIVE-18910.10.patch, > HIVE-18910.11.patch, HIVE-18910.12.patch, HIVE-18910.13.patch, > HIVE-18910.14.patch, HIVE-18910.15.patch, HIVE-18910.16.patch, > HIVE-18910.17.patch, HIVE-18910.18.patch, HIVE-18910.19.patch, > HIVE-18910.2.patch, HIVE-18910.20.patch, HIVE-18910.21.patch, > HIVE-18910.22.patch, HIVE-18910.23.patch, HIVE-18910.24.patch, > HIVE-18910.25.patch, HIVE-18910.26.patch, HIVE-18910.27.patch, > HIVE-18910.3.patch, HIVE-18910.4.patch, HIVE-18910.5.patch, > HIVE-18910.6.patch, HIVE-18910.7.patch, HIVE-18910.8.patch, HIVE-18910.9.patch > > > Hive uses JAVA hash which is not as good as murmur for better distribution > and efficiency in bucketing a table. > Migrate to murmur hash but still keep backward compatibility for existing > users so that they dont have to reload the existing tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19146) Delete dangling q.out
[ https://issues.apache.org/jira/browse/HIVE-19146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19146: Resolution: Fixed Fix Version/s: 3.1.0 Status: Resolved (was: Patch Available) Pushed to master. > Delete dangling q.out > -- > > Key: HIVE-19146 > URL: https://issues.apache.org/jira/browse/HIVE-19146 > Project: Hive > Issue Type: Test > Components: Test >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19146.patch > > > Fails TestDanglingQOuts -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19146) Delete dangling q.out
[ https://issues.apache.org/jira/browse/HIVE-19146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19146: Attachment: HIVE-19146.patch > Delete dangling q.out > -- > > Key: HIVE-19146 > URL: https://issues.apache.org/jira/browse/HIVE-19146 > Project: Hive > Issue Type: Test > Components: Test >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Attachments: HIVE-19146.patch > > > Fails TestDanglingQOuts -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19146) Delete dangling q.out
[ https://issues.apache.org/jira/browse/HIVE-19146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19146: Status: Patch Available (was: Open) > Delete dangling q.out > -- > > Key: HIVE-19146 > URL: https://issues.apache.org/jira/browse/HIVE-19146 > Project: Hive > Issue Type: Test > Components: Test >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Attachments: HIVE-19146.patch > > > Fails TestDanglingQOuts -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19146) Delete dangling q.out
[ https://issues.apache.org/jira/browse/HIVE-19146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan reassigned HIVE-19146: --- > Delete dangling q.out > -- > > Key: HIVE-19146 > URL: https://issues.apache.org/jira/browse/HIVE-19146 > Project: Hive > Issue Type: Test > Components: Test >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > > Fails TestDanglingQOuts -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19145) Stabilize statsoptimizer.q test
[ https://issues.apache.org/jira/browse/HIVE-19145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19145: Resolution: Fixed Fix Version/s: 3.1.0 Status: Resolved (was: Patch Available) Pushed to master. > Stabilize statsoptimizer.q test > --- > > Key: HIVE-19145 > URL: https://issues.apache.org/jira/browse/HIVE-19145 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19145.1.patch > > > Uses current_date() which is prone to fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19144) TestSparkCliDriver:subquery_scalar - golden file needs to be udpated
[ https://issues.apache.org/jira/browse/HIVE-19144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19144: Resolution: Fixed Fix Version/s: (was: 3.0.0) 3.1.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks, Vineet! > TestSparkCliDriver:subquery_scalar - golden file needs to be udpated > > > Key: HIVE-19144 > URL: https://issues.apache.org/jira/browse/HIVE-19144 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19144.1.patch > > > Looks like HIVE-18979 missed the update -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19145) Stabilize statsoptimizer.q test
[ https://issues.apache.org/jira/browse/HIVE-19145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19145: Attachment: HIVE-19145.1.patch > Stabilize statsoptimizer.q test > --- > > Key: HIVE-19145 > URL: https://issues.apache.org/jira/browse/HIVE-19145 > Project: Hive > Issue Type: Test > Components: Test >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Attachments: HIVE-19145.1.patch > > > Uses current_date() which is prone to fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19145) Stabilize statsoptimizer.q test
[ https://issues.apache.org/jira/browse/HIVE-19145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19145: Status: Patch Available (was: Open) > Stabilize statsoptimizer.q test > --- > > Key: HIVE-19145 > URL: https://issues.apache.org/jira/browse/HIVE-19145 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Attachments: HIVE-19145.1.patch > > > Uses current_date() which is prone to fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19145) Stabilize statsoptimizer.q test
[ https://issues.apache.org/jira/browse/HIVE-19145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19145: --- Issue Type: Sub-task (was: Test) Parent: HIVE-19142 > Stabilize statsoptimizer.q test > --- > > Key: HIVE-19145 > URL: https://issues.apache.org/jira/browse/HIVE-19145 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Attachments: HIVE-19145.1.patch > > > Uses current_date() which is prone to fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-15444) tez.queue.name is invalid after tez job running on CLI
[
https://issues.apache.org/jira/browse/HIVE-15444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431681#comment-16431681
]
Hive QA commented on HIVE-15444:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10105/dev-support/hive-personality.sh
|
| git revision | master / 2e92451 |
| Default Java | 1.8.0_111 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10105/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> tez.queue.name is invalid after tez job running on CLI
> --
>
> Key: HIVE-15444
> URL: https://issues.apache.org/jira/browse/HIVE-15444
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.1, 2.2.0
>Reporter: Fei Hui
>Assignee: Fei Hui
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-15444.1.patch
>
>
> hive> set tez.queue.name;
> tez.queue.name is undefined
> hive> set tez.queue.name=HQ_OLPS;
> hive> set tez.queue.name;
> tez.queue.name=HQ_OLPS
> hive> insert into abc values(2,2);
> Query ID = hadoop_20161216181208_6c382e49-ac4a-4f52-ba1e-3ed962733fc1
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (Executing on YARN cluster with App id
> application_1481877998678_0011)
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. container SUCCEEDED 1 100
> 0 0
> --
> VERTICES: 01/01 [==>>] 100% ELAPSED TIME: 6.57 s
> --
> Loading data to table default.abc
> OK
> Time taken: 19.983 seconds
> hive> set tez.queue.name;
> tez.queue.name is undefined
> hive> set hive.execution.engine;
> hive.execution.engine=tez
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19145) Stabilize statsoptimizer.q test
[ https://issues.apache.org/jira/browse/HIVE-19145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan reassigned HIVE-19145: --- > Stabilize statsoptimizer.q test > --- > > Key: HIVE-19145 > URL: https://issues.apache.org/jira/browse/HIVE-19145 > Project: Hive > Issue Type: Test > Components: Test >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > > Uses current_date() which is prone to fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19144) TestSparkCliDriver:subquery_scalar - golden file needs to be udpated
[ https://issues.apache.org/jira/browse/HIVE-19144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431676#comment-16431676 ] Ashutosh Chauhan commented on HIVE-19144: - +1 > TestSparkCliDriver:subquery_scalar - golden file needs to be udpated > > > Key: HIVE-19144 > URL: https://issues.apache.org/jira/browse/HIVE-19144 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19144.1.patch > > > Looks like HIVE-18979 missed the update -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-16950) Dropping hive database/table which was created explicitly in default database location, deletes all databases data from default database location
[
https://issues.apache.org/jira/browse/HIVE-16950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431674#comment-16431674
]
Hive QA commented on HIVE-16950:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12907564/HIVE-16950.4.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10104/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10104/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10104/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-04-10 03:58:00.851
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-10104/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-04-10 03:58:00.853
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
55fb0a1..2e92451 master -> origin/master
946f619..d1a9358 branch-3 -> origin/branch-3
+ git reset --hard HEAD
HEAD is now at 55fb0a1 HIVE-19143 : Update golden files for negative tests
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 2e92451 HIVE-18857: Store default value text instead of default
value expression in metastore(Vineet Garg, reviewed by Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-04-10 03:58:04.664
+ rm -rf ../yetus_PreCommit-HIVE-Build-10104
+ mkdir ../yetus_PreCommit-HIVE-Build-10104
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-10104
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-10104/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error:
a/hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/TestStreaming.java:
does not exist in index
error:
a/hcatalog/webhcat/java-client/src/test/java/org/apache/hive/hcatalog/api/TestHCatClient.java:
does not exist in index
error:
a/itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/TestDbNotificationListener.java:
does not exist in index
error:
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:
does not exist in index
error:
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/utils/FileUtils.java:
does not exist in index
error: patch failed:
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:916
Falling back to three-way merge...
Applied patch to
'standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java'
with conflicts.
Going to apply patch with: git apply -p1
error: patch failed:
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:916
Falling back to three-way merge...
Applied patch to
'standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java'
with conflicts.
U
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12907564 - PreCommit-HIVE-Build
> Dropping hive database/table which was created explicitly in default database
> location, deletes all databases data from default database location
> -
>
> Key: HIVE-16950
> URL:
[jira] [Commented] (HIVE-18839) Implement incremental rebuild for materialized views (only insert operations in source tables)
[
https://issues.apache.org/jira/browse/HIVE-18839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431671#comment-16431671
]
Hive QA commented on HIVE-18839:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918226/HIVE-18839.04.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 141 failed/errored test(s), 14040 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=253)
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
TestSequenceFileReadWrite - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[statsoptimizer]
(batchId=62)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[mm_all]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[groupby_groupingset_bug]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mergejoin]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[update_access_time_non_current_db]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_dynamic_semijoin_reduction]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=183)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.org.apache.hadoop.hive.cli.TestNegativeCliDriver
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.org.apache.hadoop.hive.cli.TestNegativeCliDriver
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_notnull_constraint_violation]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_into_acid_notnull]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_into_notnull_constraint]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_multi_into_notnull]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_overwrite_notnull_constraint]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[smb_bucketmapjoin]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[smb_mapjoin_14]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[sortmerge_mapjoin_mismatch_1]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_2]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_publisher_error_1]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_publisher_error_2]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_corr_in_agg]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_in_implicit_gby]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_notin_implicit_gby]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[truncate_bucketed_column]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[truncate_column_list_bucketing]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[truncate_column_seqfile]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[truncate_nonexistant_column]
(batchId=95)
[jira] [Commented] (HIVE-18988) Support bootstrap replication of ACID tables
[ https://issues.apache.org/jira/browse/HIVE-18988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431639#comment-16431639 ] Sankar Hariappan commented on HIVE-18988: - [~anishek], [~ekoifman], [~thejas], Can you please review this patch? > Support bootstrap replication of ACID tables > > > Key: HIVE-18988 > URL: https://issues.apache.org/jira/browse/HIVE-18988 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, repl >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: ACID, DR, pull-request-available, replication > Fix For: 3.1.0 > > Attachments: HIVE-18988.01.patch, HIVE-18988.02.patch > > > Bootstrapping of ACID tables, need special handling to replicate a stable > state of data. > - If ACID feature enables, then perform bootstrap dump for ACID tables with > in read txn. > -> Dump table/partition metadata. > -> Get the list of valid data files for a table using same logic as read txn > do. > -> Dump latest ValidWriteIdList as per current read txn. > - Find the valid last replication state such that it points to event ID of > open_txn event of oldest on-going txn. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HIVE-18787) TestSparkCliDriver.testCliDriver[subquery_scalar] is consistently failing
[ https://issues.apache.org/jira/browse/HIVE-18787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg resolved HIVE-18787. Resolution: Duplicate > TestSparkCliDriver.testCliDriver[subquery_scalar] is consistently failing > - > > Key: HIVE-18787 > URL: https://issues.apache.org/jira/browse/HIVE-18787 > Project: Hive > Issue Type: Test > Components: Test >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > > Not sure what caused this to start failing, but its been failing for a while. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18839) Implement incremental rebuild for materialized views (only insert operations in source tables)
[
https://issues.apache.org/jira/browse/HIVE-18839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431633#comment-16431633
]
Hive QA commented on HIVE-18839:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
57s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
53s{color} | {color:red} ql: The patch generated 41 new + 1163 unchanged - 20
fixed = 1204 total (was 1183) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
33s{color} | {color:red} standalone-metastore: The patch generated 10 new +
1603 unchanged - 1 fixed = 1613 total (was 1604) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 28 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10103/dev-support/hive-personality.sh
|
| git revision | master / 55fb0a1 |
| Default Java | 1.8.0_111 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10103/yetus/patch-mvninstall-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10103/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10103/yetus/diff-checkstyle-standalone-metastore.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10103/yetus/whitespace-eol.txt
|
| modules | C: common itests ql standalone-metastore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10103/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Implement incremental rebuild for materialized views (only insert operations
> in source tables)
> --
>
> Key: HIVE-18839
> URL: https://issues.apache.org/jira/browse/HIVE-18839
> Project: Hive
> Issue Type: Improvement
> Components: Materialized views
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: TODOC3.0
> Attachments: HIVE-18839.01.patch, HIVE-18839.02.patch,
> HIVE-18839.03.patch, HIVE-18839.04.patch, HIVE-18839.patch
>
>
> Implementation will follow current code path for full
[jira] [Commented] (HIVE-19144) TestSparkCliDriver:subquery_scalar - golden file needs to be udpated
[ https://issues.apache.org/jira/browse/HIVE-19144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431630#comment-16431630 ] Vineet Garg commented on HIVE-19144: [~jcamachorodriguez] [~ashutoshc] Can you take a look? > TestSparkCliDriver:subquery_scalar - golden file needs to be udpated > > > Key: HIVE-19144 > URL: https://issues.apache.org/jira/browse/HIVE-19144 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19144.1.patch > > > Looks like HIVE-18979 missed the update -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19144) TestSparkCliDriver:subquery_scalar - golden file needs to be udpated
[ https://issues.apache.org/jira/browse/HIVE-19144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19144: --- Status: Patch Available (was: Open) > TestSparkCliDriver:subquery_scalar - golden file needs to be udpated > > > Key: HIVE-19144 > URL: https://issues.apache.org/jira/browse/HIVE-19144 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19144.1.patch > > > Looks like HIVE-18979 missed the update -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19144) TestSparkCliDriver:subquery_scalar - golden file needs to be udpated
[ https://issues.apache.org/jira/browse/HIVE-19144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19144: --- Attachment: HIVE-19144.1.patch > TestSparkCliDriver:subquery_scalar - golden file needs to be udpated > > > Key: HIVE-19144 > URL: https://issues.apache.org/jira/browse/HIVE-19144 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19144.1.patch > > > Looks like HIVE-18979 missed the update -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19144) TestSparkCliDriver:subquery_scalar - golden file needs to be udpated
[ https://issues.apache.org/jira/browse/HIVE-19144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-19144: -- > TestSparkCliDriver:subquery_scalar - golden file needs to be udpated > > > Key: HIVE-19144 > URL: https://issues.apache.org/jira/browse/HIVE-19144 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19144.1.patch > > > Looks like HIVE-18979 missed the update -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18857) Store default value text instead of default value expression in metastore
[ https://issues.apache.org/jira/browse/HIVE-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-18857: --- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to both master and branch-3 > Store default value text instead of default value expression in metastore > - > > Key: HIVE-18857 > URL: https://issues.apache.org/jira/browse/HIVE-18857 > Project: Hive > Issue Type: Improvement > Components: SQL >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18857.1.patch, HIVE-18857.2.patch, > HIVE-18857.3.patch > > > Currently for default value an expression is generated and serialized to > store in metastore. It should be improved to rather serialize the default > value itself instead of expression and store that in metastore. This will > have the following benefits: > * It will make metastore schema upgrade safe. e.g. if a UDF function name is > changed hive wouldn't be able to parse back the expression for this UDF which > was serialized in earlier version. > * It will make metastore schema for default constraint hive agnostic. Other > databases would be able to use the value as it is. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18857) Store default value text instead of default value expression in metastore
[ https://issues.apache.org/jira/browse/HIVE-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-18857: --- Fix Version/s: (was: 3.1.0) 3.0.0 > Store default value text instead of default value expression in metastore > - > > Key: HIVE-18857 > URL: https://issues.apache.org/jira/browse/HIVE-18857 > Project: Hive > Issue Type: Improvement > Components: SQL >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18857.1.patch, HIVE-18857.2.patch, > HIVE-18857.3.patch > > > Currently for default value an expression is generated and serialized to > store in metastore. It should be improved to rather serialize the default > value itself instead of expression and store that in metastore. This will > have the following benefits: > * It will make metastore schema upgrade safe. e.g. if a UDF function name is > changed hive wouldn't be able to parse back the expression for this UDF which > was serialized in earlier version. > * It will make metastore schema for default constraint hive agnostic. Other > databases would be able to use the value as it is. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18857) Store default value text instead of default value expression in metastore
[
https://issues.apache.org/jira/browse/HIVE-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431612#comment-16431612
]
Hive QA commented on HIVE-18857:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918214/HIVE-18857.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 90 failed/errored test(s), 13642 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=252)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=252)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=96)
[jira] [Updated] (HIVE-19143) Update golden files for negative tests
[ https://issues.apache.org/jira/browse/HIVE-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19143: Fix Version/s: (was: 3.1.0) 3.0.0 > Update golden files for negative tests > -- > > Key: HIVE-19143 > URL: https://issues.apache.org/jira/browse/HIVE-19143 > Project: Hive > Issue Type: Test > Components: Test >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19143.1.patch > > > Missed in HIVE-18859 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19143) Update golden files for negative tests
[ https://issues.apache.org/jira/browse/HIVE-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431572#comment-16431572 ] Ashutosh Chauhan commented on HIVE-19143: - Done. > Update golden files for negative tests > -- > > Key: HIVE-19143 > URL: https://issues.apache.org/jira/browse/HIVE-19143 > Project: Hive > Issue Type: Test > Components: Test >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19143.1.patch > > > Missed in HIVE-18859 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18857) Store default value text instead of default value expression in metastore
[
https://issues.apache.org/jira/browse/HIVE-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431567#comment-16431567
]
Hive QA commented on HIVE-18857:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
53s{color} | {color:red} ql: The patch generated 1 new + 427 unchanged - 0
fixed = 428 total (was 427) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10099/dev-support/hive-personality.sh
|
| git revision | master / a103410 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10099/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10099/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Store default value text instead of default value expression in metastore
> -
>
> Key: HIVE-18857
> URL: https://issues.apache.org/jira/browse/HIVE-18857
> Project: Hive
> Issue Type: Improvement
> Components: SQL
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-18857.1.patch, HIVE-18857.2.patch,
> HIVE-18857.3.patch
>
>
> Currently for default value an expression is generated and serialized to
> store in metastore. It should be improved to rather serialize the default
> value itself instead of expression and store that in metastore. This will
> have the following benefits:
> * It will make metastore schema upgrade safe. e.g. if a UDF function name is
> changed hive wouldn't be able to parse back the expression for this UDF which
> was serialized in earlier version.
> * It will make metastore schema for default constraint hive agnostic. Other
> databases would be able to use the value as it is.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19142) Umbrella: branch-3 failing tests
[ https://issues.apache.org/jira/browse/HIVE-19142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431566#comment-16431566 ] Vineet Garg commented on HIVE-19142: authorization tests are fixed by HIVE-19143 > Umbrella: branch-3 failing tests > > > Key: HIVE-19142 > URL: https://issues.apache.org/jira/browse/HIVE-19142 > Project: Hive > Issue Type: Test >Reporter: Vineet Garg >Priority: Major > > This is the list [~alangates] specified on HIVE-19135 which are non-oom test > failures: > *Errors*: > TestAcidOnTez.testGetSplitsLocks > TestJdbcWithLocalClusterSpark.testSparkQuery > TestJdbcWithLocalClusterSpark.testTempTable > TestJdbcWithMiniLlap.testLlapInputFormatEndToEnd > TestMTQueries.testMTQueries1 > TestMultiSessionsHS2WithLocalClusterSpark.testSparkQuery > TestNegativeCliDriver.alter_notnull_constraint_violation > TestNegativeCliDriver.insert_into_acid_notnull > TestNegativeCliDriver.insert_into_notnull_constraint > TestNegativeCliDriver.insert_multi_into_notnull > TestNegativeCliDriver.insert_overwrite_notnull_constraint > TestNegativeCliDriver.update_notnull_constraint > *Failures*: > TestBlobstoreCliDriver.insert_into_dynamic_partitions > TestBlobstoreCliDriver.insert_overwrite_directory > TestBlobstoreCliDriver.insert_overwrite_dynamic_partitions > TestCliDriver.acid_table_stats > TestCliDriver.auto_sortmerge_join_2 > TestCliDriver.avro_alter_table_update_columns > TestCliDriver.avrotblsjoin > TestCliDriver.dbtxnmgr_showlocks > TestCliDriver.orc_merge10 > TestCliDriver.orc_schema_evolution_float > TestCliDriver.parquet_ppd_multifiles > TestCliDriver.schema_evol_par_vec_table_dictionary_encoding > TestCliDriver.schema_evol_par_vec_table_non_dictionary_encoding > TestCliDriver.selectindate > TestCliDriver.statsoptimizer > TestCliDriver.vector_bround > TestCliDriver.vector_case_when_1 > TestCliDriver.vector_coalesce_2 > TestCliDriver.vector_coalesce_3 > TestCliDriver.vector_interval_1 > TestCliDriver.vectorized_parquet_types > TestMetastoreVersion.testMetastoreVersion > TestMetastoreVersion.testVersionMatching > TestMiniDruidCliDriver.druidkafkamini_basic > TestMiniLlapCliDriver.llap_smb > TestMiniLlapCliDriver.unionDistinct_1 > TestMiniTezCliDriver.explainanalyze_5 > -TestNegativeCliDriver.authorization_caseinsensitivity- > -TestNegativeCliDriver.authorization_fail_1- > -TestNegativeCliDriver.authorization_grant_table_dup- > -TestNegativeCliDriver.authorization_role_case- > -TestNegativeCliDriver.authorization_role_grant_nosuchrole- > -TestNegativeCliDriver.authorization_table_grant_nosuchrole- > TestNegativeCliDriver.subquery_subquery_chain > TestSessionState.testCreatePath > TestSessionState.testCreatePath > TestSparkStatistics.testSparkStatistics -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19142) Umbrella: branch-3 failing tests
[
https://issues.apache.org/jira/browse/HIVE-19142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19142:
---
Description:
This is the list [~alangates] specified on HIVE-19135 which are non-oom test
failures:
*Errors*:
TestAcidOnTez.testGetSplitsLocks
TestJdbcWithLocalClusterSpark.testSparkQuery
TestJdbcWithLocalClusterSpark.testTempTable
TestJdbcWithMiniLlap.testLlapInputFormatEndToEnd
TestMTQueries.testMTQueries1
TestMultiSessionsHS2WithLocalClusterSpark.testSparkQuery
TestNegativeCliDriver.alter_notnull_constraint_violation
TestNegativeCliDriver.insert_into_acid_notnull
TestNegativeCliDriver.insert_into_notnull_constraint
TestNegativeCliDriver.insert_multi_into_notnull
TestNegativeCliDriver.insert_overwrite_notnull_constraint
TestNegativeCliDriver.update_notnull_constraint
*Failures*:
TestBlobstoreCliDriver.insert_into_dynamic_partitions
TestBlobstoreCliDriver.insert_overwrite_directory
TestBlobstoreCliDriver.insert_overwrite_dynamic_partitions
TestCliDriver.acid_table_stats
TestCliDriver.auto_sortmerge_join_2
TestCliDriver.avro_alter_table_update_columns
TestCliDriver.avrotblsjoin
TestCliDriver.dbtxnmgr_showlocks
TestCliDriver.orc_merge10
TestCliDriver.orc_schema_evolution_float
TestCliDriver.parquet_ppd_multifiles
TestCliDriver.schema_evol_par_vec_table_dictionary_encoding
TestCliDriver.schema_evol_par_vec_table_non_dictionary_encoding
TestCliDriver.selectindate
TestCliDriver.statsoptimizer
TestCliDriver.vector_bround
TestCliDriver.vector_case_when_1
TestCliDriver.vector_coalesce_2
TestCliDriver.vector_coalesce_3
TestCliDriver.vector_interval_1
TestCliDriver.vectorized_parquet_types
TestMetastoreVersion.testMetastoreVersion
TestMetastoreVersion.testVersionMatching
TestMiniDruidCliDriver.druidkafkamini_basic
TestMiniLlapCliDriver.llap_smb
TestMiniLlapCliDriver.unionDistinct_1
TestMiniTezCliDriver.explainanalyze_5
-TestNegativeCliDriver.authorization_caseinsensitivity-
-TestNegativeCliDriver.authorization_fail_1-
-TestNegativeCliDriver.authorization_grant_table_dup-
-TestNegativeCliDriver.authorization_role_case-
-TestNegativeCliDriver.authorization_role_grant_nosuchrole-
-TestNegativeCliDriver.authorization_table_grant_nosuchrole-
TestNegativeCliDriver.subquery_subquery_chain
TestSessionState.testCreatePath
TestSessionState.testCreatePath
TestSparkStatistics.testSparkStatistics
was:
This is the list [~alangates] specified on HIVE-19135 which are non-oom test
failures:
{noformat}
Errors:
TestAcidOnTez.testGetSplitsLocks
TestJdbcWithLocalClusterSpark.testSparkQuery
TestJdbcWithLocalClusterSpark.testTempTable
TestJdbcWithMiniLlap.testLlapInputFormatEndToEnd
TestMTQueries.testMTQueries1
TestMultiSessionsHS2WithLocalClusterSpark.testSparkQuery
TestNegativeCliDriver.alter_notnull_constraint_violation
TestNegativeCliDriver.insert_into_acid_notnull
TestNegativeCliDriver.insert_into_notnull_constraint
TestNegativeCliDriver.insert_multi_into_notnull
TestNegativeCliDriver.insert_overwrite_notnull_constraint
TestNegativeCliDriver.update_notnull_constraint
Failures:
TestBlobstoreCliDriver.insert_into_dynamic_partitions
TestBlobstoreCliDriver.insert_overwrite_directory
TestBlobstoreCliDriver.insert_overwrite_dynamic_partitions
TestCliDriver.acid_table_stats
TestCliDriver.auto_sortmerge_join_2
TestCliDriver.avro_alter_table_update_columns
TestCliDriver.avrotblsjoin
TestCliDriver.dbtxnmgr_showlocks
TestCliDriver.orc_merge10
TestCliDriver.orc_schema_evolution_float
TestCliDriver.parquet_ppd_multifiles
TestCliDriver.schema_evol_par_vec_table_dictionary_encoding
TestCliDriver.schema_evol_par_vec_table_non_dictionary_encoding
TestCliDriver.selectindate
TestCliDriver.statsoptimizer
TestCliDriver.vector_bround
TestCliDriver.vector_case_when_1
TestCliDriver.vector_coalesce_2
TestCliDriver.vector_coalesce_3
TestCliDriver.vector_interval_1
TestCliDriver.vectorized_parquet_types
TestMetastoreVersion.testMetastoreVersion
TestMetastoreVersion.testVersionMatching
TestMiniDruidCliDriver.druidkafkamini_basic
TestMiniLlapCliDriver.llap_smb
TestMiniLlapCliDriver.unionDistinct_1
TestMiniTezCliDriver.explainanalyze_5
TestNegativeCliDriver.authorization_caseinsensitivity
TestNegativeCliDriver.authorization_fail_1
TestNegativeCliDriver.authorization_grant_table_dup
TestNegativeCliDriver.authorization_role_case
TestNegativeCliDriver.authorization_role_grant_nosuchrole
TestNegativeCliDriver.authorization_table_grant_nosuchrole
TestNegativeCliDriver.subquery_subquery_chain
TestSessionState.testCreatePath
TestSessionState.testCreatePath
TestSparkStatistics.testSparkStatistics
{noformat}
> Umbrella: branch-3 failing tests
>
>
> Key: HIVE-19142
> URL: https://issues.apache.org/jira/browse/HIVE-19142
> Project: Hive
> Issue Type: Test
>Reporter: Vineet Garg
>Priority: Major
>
> This is the list [~alangates] specified on HIVE-19135 which
[jira] [Commented] (HIVE-19143) Update golden files for negative tests
[ https://issues.apache.org/jira/browse/HIVE-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431564#comment-16431564 ] Vineet Garg commented on HIVE-19143: [~ashutoshc] Yes please go ahead and push it to branch-3 > Update golden files for negative tests > -- > > Key: HIVE-19143 > URL: https://issues.apache.org/jira/browse/HIVE-19143 > Project: Hive > Issue Type: Test > Components: Test >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19143.1.patch > > > Missed in HIVE-18859 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19143) Update golden files for negative tests
[ https://issues.apache.org/jira/browse/HIVE-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19143: Resolution: Fixed Fix Version/s: 3.1.0 Status: Resolved (was: Patch Available) Pushed to master. [~vgarg] It will be useful for branch-3 as well > Update golden files for negative tests > -- > > Key: HIVE-19143 > URL: https://issues.apache.org/jira/browse/HIVE-19143 > Project: Hive > Issue Type: Test > Components: Test >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19143.1.patch > > > Missed in HIVE-18859 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19143) Update golden files for negative tests
[ https://issues.apache.org/jira/browse/HIVE-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19143: Attachment: HIVE-19143.1.patch > Update golden files for negative tests > -- > > Key: HIVE-19143 > URL: https://issues.apache.org/jira/browse/HIVE-19143 > Project: Hive > Issue Type: Test > Components: Test >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Attachments: HIVE-19143.1.patch > > > Missed in HIVE-18859 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19143) Update golden files for negative tests
[ https://issues.apache.org/jira/browse/HIVE-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19143: Status: Patch Available (was: Open) > Update golden files for negative tests > -- > > Key: HIVE-19143 > URL: https://issues.apache.org/jira/browse/HIVE-19143 > Project: Hive > Issue Type: Test > Components: Test >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Attachments: HIVE-19143.1.patch > > > Missed in HIVE-18859 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19143) Update golden files for negative tests
[ https://issues.apache.org/jira/browse/HIVE-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan reassigned HIVE-19143: --- > Update golden files for negative tests > -- > > Key: HIVE-19143 > URL: https://issues.apache.org/jira/browse/HIVE-19143 > Project: Hive > Issue Type: Test > Components: Test >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > > Missed in HIVE-18859 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19126) CachedStore: Use memory estimation to limit cache size during prewarm
[ https://issues.apache.org/jira/browse/HIVE-19126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431560#comment-16431560 ] Thejas M Nair commented on HIVE-19126: -- +1 > CachedStore: Use memory estimation to limit cache size during prewarm > - > > Key: HIVE-19126 > URL: https://issues.apache.org/jira/browse/HIVE-19126 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-19126.1.patch, HIVE-19126.2.patch, > HIVE-19126.3.patch, HIVE-19126.4.patch > > > We can rely on > https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/IncrementalObjectSizeEstimator.java > to estimate memory of SharedCache. This jira addresses the size estimation > during prewarm, so that we can stop when we hit the memory limit. In a > follow-up jira, we will work on estimation/eviction after prewarm is > complete, so that we can keep the frequently used tables and their partitions > in cache. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19133) HS2 WebUI phase-wise performance metrics not showing correctly
[
https://issues.apache.org/jira/browse/HIVE-19133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431553#comment-16431553
]
Hive QA commented on HIVE-19133:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918211/HIVE-19133.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 77 failed/errored test(s), 13565 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestBeeLineExceptionHandling - did not produce a TEST-*.xml file (likely timed
out) (batchId=190)
TestBeeLineHistory - did not produce a TEST-*.xml file (likely timed out)
(batchId=190)
TestBeelineArgParsing - did not produce a TEST-*.xml file (likely timed out)
(batchId=190)
TestClientCommandHookFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=190)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=252)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestHiveCli - did not produce a TEST-*.xml file (likely timed out) (batchId=190)
TestHiveSchemaTool - did not produce a TEST-*.xml file (likely timed out)
(batchId=190)
TestIncrementalRows - did not produce a TEST-*.xml file (likely timed out)
(batchId=190)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=252)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=96)
[jira] [Commented] (HIVE-18609) Results cache invalidation based on table updates
[ https://issues.apache.org/jira/browse/HIVE-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431552#comment-16431552 ] Jason Dere commented on HIVE-18609: --- Rebased with master. [~jcamachorodriguez] [~gopalv] can you take a look? > Results cache invalidation based on table updates > - > > Key: HIVE-18609 > URL: https://issues.apache.org/jira/browse/HIVE-18609 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Attachments: HIVE-18609.1.patch, HIVE-18609.2.patch > > > Look into using the materialized view invalidation mechanisms to > automatically invalidate queries in the results cache if the underlying > tables used in the cached queries have been modified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18609) Results cache invalidation based on table updates
[ https://issues.apache.org/jira/browse/HIVE-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-18609: -- Attachment: HIVE-18609.2.patch > Results cache invalidation based on table updates > - > > Key: HIVE-18609 > URL: https://issues.apache.org/jira/browse/HIVE-18609 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Attachments: HIVE-18609.1.patch, HIVE-18609.2.patch > > > Look into using the materialized view invalidation mechanisms to > automatically invalidate queries in the results cache if the underlying > tables used in the cached queries have been modified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19126) CachedStore: Use memory estimation to limit cache size during prewarm
[ https://issues.apache.org/jira/browse/HIVE-19126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-19126: Attachment: HIVE-19126.4.patch > CachedStore: Use memory estimation to limit cache size during prewarm > - > > Key: HIVE-19126 > URL: https://issues.apache.org/jira/browse/HIVE-19126 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-19126.1.patch, HIVE-19126.2.patch, > HIVE-19126.3.patch, HIVE-19126.4.patch > > > We can rely on > https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/IncrementalObjectSizeEstimator.java > to estimate memory of SharedCache. This jira addresses the size estimation > during prewarm, so that we can stop when we hit the memory limit. In a > follow-up jira, we will work on estimation/eviction after prewarm is > complete, so that we can keep the frequently used tables and their partitions > in cache. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19127) Concurrency fixes in QueryResultsCache
[ https://issues.apache.org/jira/browse/HIVE-19127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-19127: -- Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed to master and branch-3 > Concurrency fixes in QueryResultsCache > -- > > Key: HIVE-19127 > URL: https://issues.apache.org/jira/browse/HIVE-19127 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19127.1.patch, HIVE-19127.2.patch > > > A few fixes around concurrent access in the results cache > - Take a lock on the cache entry when in the process of setting the cache > entry from PENDING state to VALID state, so that other threads cannot > invalidate the entry > - The write lock on the cache was not being taken when removing an entry from > the cache. > - synchronize access when iterating through the lru keyset -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19014) utilize YARN-8028 (queue ACL check) in Hive Tez session pool
[ https://issues.apache.org/jira/browse/HIVE-19014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431503#comment-16431503 ] Jason Dere commented on HIVE-19014: --- Cherry-picked to branch-3 as well. > utilize YARN-8028 (queue ACL check) in Hive Tez session pool > > > Key: HIVE-19014 > URL: https://issues.apache.org/jira/browse/HIVE-19014 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19014.01.patch, HIVE-19014.02.patch, > HIVE-19014.03.patch, HIVE-19014.04.patch, HIVE-19014.05.patch, > HIVE-19014.06.patch, HIVE-19014.07.patch, HIVE-19014.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19141) TestNegativeCliDriver insert_into_notnull_constraint, insert_into_acid_notnull failing
[ https://issues.apache.org/jira/browse/HIVE-19141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19141: --- Description: These tests have been consistently failing for a while. I suspect HIVE-18727 has caused these failures. HIVE-18727 changed the code to throw ERROR instead of EXCEPTION if constraints are violated. I guess Negative cli driver doesn't handle errors. Following are full list of related failures: TestNegativeCliDriver.alter_notnull_constraint_violation TestNegativeCliDriver.insert_into_acid_notnull TestNegativeCliDriver.insert_into_notnull_constraint TestNegativeCliDriver.insert_multi_into_notnull TestNegativeCliDriver.insert_overwrite_notnull_constraint TestNegativeCliDriver.update_notnull_constraint was:These tests have been consistently failing for a while. I suspect HIVE-18727 has caused these failures. HIVE-18727 changed the code to throw ERROR instead of EXCEPTION if constraints are violated. I guess Negative cli driver doesn't handle errors. > TestNegativeCliDriver insert_into_notnull_constraint, > insert_into_acid_notnull failing > -- > > Key: HIVE-19141 > URL: https://issues.apache.org/jira/browse/HIVE-19141 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0, 3.1.0 > > > These tests have been consistently failing for a while. I suspect HIVE-18727 > has caused these failures. HIVE-18727 changed the code to throw ERROR instead > of EXCEPTION if constraints are violated. I guess Negative cli driver doesn't > handle errors. > Following are full list of related failures: > TestNegativeCliDriver.alter_notnull_constraint_violation > TestNegativeCliDriver.insert_into_acid_notnull > TestNegativeCliDriver.insert_into_notnull_constraint > TestNegativeCliDriver.insert_multi_into_notnull > TestNegativeCliDriver.insert_overwrite_notnull_constraint > TestNegativeCliDriver.update_notnull_constraint -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19142) Umbrella: branch-3 failing tests
[
https://issues.apache.org/jira/browse/HIVE-19142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19142:
---
Description:
From
This is the list [~alangates] specified on HIVE-19135 which are non-oom test
failures:
{noformat}
Errors:
TestAcidOnTez.testGetSplitsLocks
TestJdbcWithLocalClusterSpark.testSparkQuery
TestJdbcWithLocalClusterSpark.testTempTable
TestJdbcWithMiniLlap.testLlapInputFormatEndToEnd
TestMTQueries.testMTQueries1
TestMultiSessionsHS2WithLocalClusterSpark.testSparkQuery
TestNegativeCliDriver.alter_notnull_constraint_violation
TestNegativeCliDriver.insert_into_acid_notnull
TestNegativeCliDriver.insert_into_notnull_constraint
TestNegativeCliDriver.insert_multi_into_notnull
TestNegativeCliDriver.insert_overwrite_notnull_constraint
TestNegativeCliDriver.update_notnull_constraint
Failures:
TestBlobstoreCliDriver.insert_into_dynamic_partitions
TestBlobstoreCliDriver.insert_overwrite_directory
TestBlobstoreCliDriver.insert_overwrite_dynamic_partitions
TestCliDriver.acid_table_stats
TestCliDriver.auto_sortmerge_join_2
TestCliDriver.avro_alter_table_update_columns
TestCliDriver.avrotblsjoin
TestCliDriver.dbtxnmgr_showlocks
TestCliDriver.orc_merge10
TestCliDriver.orc_schema_evolution_float
TestCliDriver.parquet_ppd_multifiles
TestCliDriver.schema_evol_par_vec_table_dictionary_encoding
TestCliDriver.schema_evol_par_vec_table_non_dictionary_encoding
TestCliDriver.selectindate
TestCliDriver.statsoptimizer
TestCliDriver.vector_bround
TestCliDriver.vector_case_when_1
TestCliDriver.vector_coalesce_2
TestCliDriver.vector_coalesce_3
TestCliDriver.vector_interval_1
TestCliDriver.vectorized_parquet_types
TestMetastoreVersion.testMetastoreVersion
TestMetastoreVersion.testVersionMatching
TestMiniDruidCliDriver.druidkafkamini_basic
TestMiniLlapCliDriver.llap_smb
TestMiniLlapCliDriver.unionDistinct_1
TestMiniTezCliDriver.explainanalyze_5
TestNegativeCliDriver.authorization_caseinsensitivity
TestNegativeCliDriver.authorization_fail_1
TestNegativeCliDriver.authorization_grant_table_dup
TestNegativeCliDriver.authorization_role_case
TestNegativeCliDriver.authorization_role_grant_nosuchrole
TestNegativeCliDriver.authorization_table_grant_nosuchrole
TestNegativeCliDriver.subquery_subquery_chain
TestSessionState.testCreatePath
TestSessionState.testCreatePath
TestSparkStatistics.testSparkStatistics
{noformat}
was:
From
This is the list [~alangates] specified on HIVE-19135 which are non-oom test
failures:
{noformat}
*Errors*:
TestAcidOnTez.testGetSplitsLocks
TestJdbcWithLocalClusterSpark.testSparkQuery
TestJdbcWithLocalClusterSpark.testTempTable
TestJdbcWithMiniLlap.testLlapInputFormatEndToEnd
TestMTQueries.testMTQueries1
TestMultiSessionsHS2WithLocalClusterSpark.testSparkQuery
TestNegativeCliDriver.alter_notnull_constraint_violation
TestNegativeCliDriver.insert_into_acid_notnull
TestNegativeCliDriver.insert_into_notnull_constraint
TestNegativeCliDriver.insert_multi_into_notnull
TestNegativeCliDriver.insert_overwrite_notnull_constraint
TestNegativeCliDriver.update_notnull_constraint
*Failures*:
TestBlobstoreCliDriver.insert_into_dynamic_partitions
TestBlobstoreCliDriver.insert_overwrite_directory
TestBlobstoreCliDriver.insert_overwrite_dynamic_partitions
TestCliDriver.acid_table_stats
TestCliDriver.auto_sortmerge_join_2
TestCliDriver.avro_alter_table_update_columns
TestCliDriver.avrotblsjoin
TestCliDriver.dbtxnmgr_showlocks
TestCliDriver.orc_merge10
TestCliDriver.orc_schema_evolution_float
TestCliDriver.parquet_ppd_multifiles
TestCliDriver.schema_evol_par_vec_table_dictionary_encoding
TestCliDriver.schema_evol_par_vec_table_non_dictionary_encoding
TestCliDriver.selectindate
TestCliDriver.statsoptimizer
TestCliDriver.vector_bround
TestCliDriver.vector_case_when_1
TestCliDriver.vector_coalesce_2
TestCliDriver.vector_coalesce_3
TestCliDriver.vector_interval_1
TestCliDriver.vectorized_parquet_types
TestMetastoreVersion.testMetastoreVersion
TestMetastoreVersion.testVersionMatching
TestMiniDruidCliDriver.druidkafkamini_basic
TestMiniLlapCliDriver.llap_smb
TestMiniLlapCliDriver.unionDistinct_1
TestMiniTezCliDriver.explainanalyze_5
TestNegativeCliDriver.authorization_caseinsensitivity
TestNegativeCliDriver.authorization_fail_1
TestNegativeCliDriver.authorization_grant_table_dup
TestNegativeCliDriver.authorization_role_case
TestNegativeCliDriver.authorization_role_grant_nosuchrole
TestNegativeCliDriver.authorization_table_grant_nosuchrole
TestNegativeCliDriver.subquery_subquery_chain
TestSessionState.testCreatePath
TestSessionState.testCreatePath
TestSparkStatistics.testSparkStatistics
{noformat}
> Umbrella: branch-3 failing tests
>
>
> Key: HIVE-19142
> URL: https://issues.apache.org/jira/browse/HIVE-19142
> Project: Hive
> Issue Type: Test
>Reporter: Vineet Garg
>Priority: Major
>
> From
> This is the list [~alangates]
[jira] [Updated] (HIVE-19142) Umbrella: branch-3 failing tests
[
https://issues.apache.org/jira/browse/HIVE-19142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19142:
---
Description:
From
This is the list [~alangates] specified on HIVE-19135 which are non-oom test
failures:
{noformat}
*Errors*:
TestAcidOnTez.testGetSplitsLocks
TestJdbcWithLocalClusterSpark.testSparkQuery
TestJdbcWithLocalClusterSpark.testTempTable
TestJdbcWithMiniLlap.testLlapInputFormatEndToEnd
TestMTQueries.testMTQueries1
TestMultiSessionsHS2WithLocalClusterSpark.testSparkQuery
TestNegativeCliDriver.alter_notnull_constraint_violation
TestNegativeCliDriver.insert_into_acid_notnull
TestNegativeCliDriver.insert_into_notnull_constraint
TestNegativeCliDriver.insert_multi_into_notnull
TestNegativeCliDriver.insert_overwrite_notnull_constraint
TestNegativeCliDriver.update_notnull_constraint
*Failures*:
TestBlobstoreCliDriver.insert_into_dynamic_partitions
TestBlobstoreCliDriver.insert_overwrite_directory
TestBlobstoreCliDriver.insert_overwrite_dynamic_partitions
TestCliDriver.acid_table_stats
TestCliDriver.auto_sortmerge_join_2
TestCliDriver.avro_alter_table_update_columns
TestCliDriver.avrotblsjoin
TestCliDriver.dbtxnmgr_showlocks
TestCliDriver.orc_merge10
TestCliDriver.orc_schema_evolution_float
TestCliDriver.parquet_ppd_multifiles
TestCliDriver.schema_evol_par_vec_table_dictionary_encoding
TestCliDriver.schema_evol_par_vec_table_non_dictionary_encoding
TestCliDriver.selectindate
TestCliDriver.statsoptimizer
TestCliDriver.vector_bround
TestCliDriver.vector_case_when_1
TestCliDriver.vector_coalesce_2
TestCliDriver.vector_coalesce_3
TestCliDriver.vector_interval_1
TestCliDriver.vectorized_parquet_types
TestMetastoreVersion.testMetastoreVersion
TestMetastoreVersion.testVersionMatching
TestMiniDruidCliDriver.druidkafkamini_basic
TestMiniLlapCliDriver.llap_smb
TestMiniLlapCliDriver.unionDistinct_1
TestMiniTezCliDriver.explainanalyze_5
TestNegativeCliDriver.authorization_caseinsensitivity
TestNegativeCliDriver.authorization_fail_1
TestNegativeCliDriver.authorization_grant_table_dup
TestNegativeCliDriver.authorization_role_case
TestNegativeCliDriver.authorization_role_grant_nosuchrole
TestNegativeCliDriver.authorization_table_grant_nosuchrole
TestNegativeCliDriver.subquery_subquery_chain
TestSessionState.testCreatePath
TestSessionState.testCreatePath
TestSparkStatistics.testSparkStatistics
{noformat}
> Umbrella: branch-3 failing tests
>
>
> Key: HIVE-19142
> URL: https://issues.apache.org/jira/browse/HIVE-19142
> Project: Hive
> Issue Type: Test
>Reporter: Vineet Garg
>Priority: Major
>
> From
> This is the list [~alangates] specified on HIVE-19135 which are non-oom test
> failures:
> {noformat}
> *Errors*:
> TestAcidOnTez.testGetSplitsLocks
> TestJdbcWithLocalClusterSpark.testSparkQuery
> TestJdbcWithLocalClusterSpark.testTempTable
> TestJdbcWithMiniLlap.testLlapInputFormatEndToEnd
> TestMTQueries.testMTQueries1
> TestMultiSessionsHS2WithLocalClusterSpark.testSparkQuery
> TestNegativeCliDriver.alter_notnull_constraint_violation
> TestNegativeCliDriver.insert_into_acid_notnull
> TestNegativeCliDriver.insert_into_notnull_constraint
> TestNegativeCliDriver.insert_multi_into_notnull
> TestNegativeCliDriver.insert_overwrite_notnull_constraint
> TestNegativeCliDriver.update_notnull_constraint
> *Failures*:
> TestBlobstoreCliDriver.insert_into_dynamic_partitions
> TestBlobstoreCliDriver.insert_overwrite_directory
> TestBlobstoreCliDriver.insert_overwrite_dynamic_partitions
> TestCliDriver.acid_table_stats
> TestCliDriver.auto_sortmerge_join_2
> TestCliDriver.avro_alter_table_update_columns
> TestCliDriver.avrotblsjoin
> TestCliDriver.dbtxnmgr_showlocks
> TestCliDriver.orc_merge10
> TestCliDriver.orc_schema_evolution_float
> TestCliDriver.parquet_ppd_multifiles
> TestCliDriver.schema_evol_par_vec_table_dictionary_encoding
> TestCliDriver.schema_evol_par_vec_table_non_dictionary_encoding
> TestCliDriver.selectindate
> TestCliDriver.statsoptimizer
> TestCliDriver.vector_bround
> TestCliDriver.vector_case_when_1
> TestCliDriver.vector_coalesce_2
> TestCliDriver.vector_coalesce_3
> TestCliDriver.vector_interval_1
> TestCliDriver.vectorized_parquet_types
> TestMetastoreVersion.testMetastoreVersion
> TestMetastoreVersion.testVersionMatching
> TestMiniDruidCliDriver.druidkafkamini_basic
> TestMiniLlapCliDriver.llap_smb
> TestMiniLlapCliDriver.unionDistinct_1
> TestMiniTezCliDriver.explainanalyze_5
> TestNegativeCliDriver.authorization_caseinsensitivity
> TestNegativeCliDriver.authorization_fail_1
> TestNegativeCliDriver.authorization_grant_table_dup
> TestNegativeCliDriver.authorization_role_case
> TestNegativeCliDriver.authorization_role_grant_nosuchrole
> TestNegativeCliDriver.authorization_table_grant_nosuchrole
> TestNegativeCliDriver.subquery_subquery_chain
> TestSessionState.testCreatePath
>
[jira] [Updated] (HIVE-19142) Umbrella: branch-3 failing tests
[
https://issues.apache.org/jira/browse/HIVE-19142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19142:
---
Description:
This is the list [~alangates] specified on HIVE-19135 which are non-oom test
failures:
{noformat}
Errors:
TestAcidOnTez.testGetSplitsLocks
TestJdbcWithLocalClusterSpark.testSparkQuery
TestJdbcWithLocalClusterSpark.testTempTable
TestJdbcWithMiniLlap.testLlapInputFormatEndToEnd
TestMTQueries.testMTQueries1
TestMultiSessionsHS2WithLocalClusterSpark.testSparkQuery
TestNegativeCliDriver.alter_notnull_constraint_violation
TestNegativeCliDriver.insert_into_acid_notnull
TestNegativeCliDriver.insert_into_notnull_constraint
TestNegativeCliDriver.insert_multi_into_notnull
TestNegativeCliDriver.insert_overwrite_notnull_constraint
TestNegativeCliDriver.update_notnull_constraint
Failures:
TestBlobstoreCliDriver.insert_into_dynamic_partitions
TestBlobstoreCliDriver.insert_overwrite_directory
TestBlobstoreCliDriver.insert_overwrite_dynamic_partitions
TestCliDriver.acid_table_stats
TestCliDriver.auto_sortmerge_join_2
TestCliDriver.avro_alter_table_update_columns
TestCliDriver.avrotblsjoin
TestCliDriver.dbtxnmgr_showlocks
TestCliDriver.orc_merge10
TestCliDriver.orc_schema_evolution_float
TestCliDriver.parquet_ppd_multifiles
TestCliDriver.schema_evol_par_vec_table_dictionary_encoding
TestCliDriver.schema_evol_par_vec_table_non_dictionary_encoding
TestCliDriver.selectindate
TestCliDriver.statsoptimizer
TestCliDriver.vector_bround
TestCliDriver.vector_case_when_1
TestCliDriver.vector_coalesce_2
TestCliDriver.vector_coalesce_3
TestCliDriver.vector_interval_1
TestCliDriver.vectorized_parquet_types
TestMetastoreVersion.testMetastoreVersion
TestMetastoreVersion.testVersionMatching
TestMiniDruidCliDriver.druidkafkamini_basic
TestMiniLlapCliDriver.llap_smb
TestMiniLlapCliDriver.unionDistinct_1
TestMiniTezCliDriver.explainanalyze_5
TestNegativeCliDriver.authorization_caseinsensitivity
TestNegativeCliDriver.authorization_fail_1
TestNegativeCliDriver.authorization_grant_table_dup
TestNegativeCliDriver.authorization_role_case
TestNegativeCliDriver.authorization_role_grant_nosuchrole
TestNegativeCliDriver.authorization_table_grant_nosuchrole
TestNegativeCliDriver.subquery_subquery_chain
TestSessionState.testCreatePath
TestSessionState.testCreatePath
TestSparkStatistics.testSparkStatistics
{noformat}
was:
From
This is the list [~alangates] specified on HIVE-19135 which are non-oom test
failures:
{noformat}
Errors:
TestAcidOnTez.testGetSplitsLocks
TestJdbcWithLocalClusterSpark.testSparkQuery
TestJdbcWithLocalClusterSpark.testTempTable
TestJdbcWithMiniLlap.testLlapInputFormatEndToEnd
TestMTQueries.testMTQueries1
TestMultiSessionsHS2WithLocalClusterSpark.testSparkQuery
TestNegativeCliDriver.alter_notnull_constraint_violation
TestNegativeCliDriver.insert_into_acid_notnull
TestNegativeCliDriver.insert_into_notnull_constraint
TestNegativeCliDriver.insert_multi_into_notnull
TestNegativeCliDriver.insert_overwrite_notnull_constraint
TestNegativeCliDriver.update_notnull_constraint
Failures:
TestBlobstoreCliDriver.insert_into_dynamic_partitions
TestBlobstoreCliDriver.insert_overwrite_directory
TestBlobstoreCliDriver.insert_overwrite_dynamic_partitions
TestCliDriver.acid_table_stats
TestCliDriver.auto_sortmerge_join_2
TestCliDriver.avro_alter_table_update_columns
TestCliDriver.avrotblsjoin
TestCliDriver.dbtxnmgr_showlocks
TestCliDriver.orc_merge10
TestCliDriver.orc_schema_evolution_float
TestCliDriver.parquet_ppd_multifiles
TestCliDriver.schema_evol_par_vec_table_dictionary_encoding
TestCliDriver.schema_evol_par_vec_table_non_dictionary_encoding
TestCliDriver.selectindate
TestCliDriver.statsoptimizer
TestCliDriver.vector_bround
TestCliDriver.vector_case_when_1
TestCliDriver.vector_coalesce_2
TestCliDriver.vector_coalesce_3
TestCliDriver.vector_interval_1
TestCliDriver.vectorized_parquet_types
TestMetastoreVersion.testMetastoreVersion
TestMetastoreVersion.testVersionMatching
TestMiniDruidCliDriver.druidkafkamini_basic
TestMiniLlapCliDriver.llap_smb
TestMiniLlapCliDriver.unionDistinct_1
TestMiniTezCliDriver.explainanalyze_5
TestNegativeCliDriver.authorization_caseinsensitivity
TestNegativeCliDriver.authorization_fail_1
TestNegativeCliDriver.authorization_grant_table_dup
TestNegativeCliDriver.authorization_role_case
TestNegativeCliDriver.authorization_role_grant_nosuchrole
TestNegativeCliDriver.authorization_table_grant_nosuchrole
TestNegativeCliDriver.subquery_subquery_chain
TestSessionState.testCreatePath
TestSessionState.testCreatePath
TestSparkStatistics.testSparkStatistics
{noformat}
> Umbrella: branch-3 failing tests
>
>
> Key: HIVE-19142
> URL: https://issues.apache.org/jira/browse/HIVE-19142
> Project: Hive
> Issue Type: Test
>Reporter: Vineet Garg
>Priority: Major
>
> This is the list [~alangates] specified on
[jira] [Updated] (HIVE-19128) Update golden files for spark perf tests
[ https://issues.apache.org/jira/browse/HIVE-19128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19128: --- Issue Type: Sub-task (was: Test) Parent: HIVE-19142 > Update golden files for spark perf tests > > > Key: HIVE-19128 > URL: https://issues.apache.org/jira/browse/HIVE-19128 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19128.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19014) utilize YARN-8028 (queue ACL check) in Hive Tez session pool
[ https://issues.apache.org/jira/browse/HIVE-19014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431490#comment-16431490 ] Vineet Garg commented on HIVE-19014: [~jdere] no problem :). We had already cut-off branch for 3.0.0 (branch-3) so this will need to be pushed in to branch-3 if you are targeting 3.0.0. > utilize YARN-8028 (queue ACL check) in Hive Tez session pool > > > Key: HIVE-19014 > URL: https://issues.apache.org/jira/browse/HIVE-19014 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19014.01.patch, HIVE-19014.02.patch, > HIVE-19014.03.patch, HIVE-19014.04.patch, HIVE-19014.05.patch, > HIVE-19014.06.patch, HIVE-19014.07.patch, HIVE-19014.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19141) TestNegativeCliDriver insert_into_notnull_constraint, insert_into_acid_notnull failing
[ https://issues.apache.org/jira/browse/HIVE-19141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19141: --- Issue Type: Sub-task (was: Test) Parent: HIVE-19142 > TestNegativeCliDriver insert_into_notnull_constraint, > insert_into_acid_notnull failing > -- > > Key: HIVE-19141 > URL: https://issues.apache.org/jira/browse/HIVE-19141 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0, 3.1.0 > > > These tests have been consistently failing for a while. I suspect HIVE-18727 > has caused these failures. HIVE-18727 changed the code to throw ERROR instead > of EXCEPTION if constraints are violated. I guess Negative cli driver doesn't > handle errors. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18522) Running all negative qtests seems to leave some threads behind
[ https://issues.apache.org/jira/browse/HIVE-18522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-18522: --- Issue Type: Sub-task (was: Bug) Parent: HIVE-19142 > Running all negative qtests seems to leave some threads behind > -- > > Key: HIVE-18522 > URL: https://issues.apache.org/jira/browse/HIVE-18522 > Project: Hive > Issue Type: Sub-task > Components: Tests >Reporter: Zoltan Haindrich >Priority: Major > > running all the negative queries is seemingly produce some extra threads...it > have gone from 40 to 80 the suspicious live thread names: "HMSHandler #x", > "Hearbeater-#x", "communicationthread" ... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19014) utilize YARN-8028 (queue ACL check) in Hive Tez session pool
[ https://issues.apache.org/jira/browse/HIVE-19014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-19014: -- Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed to master. sorry [~vgarg], didn't get to closing the Jira :) > utilize YARN-8028 (queue ACL check) in Hive Tez session pool > > > Key: HIVE-19014 > URL: https://issues.apache.org/jira/browse/HIVE-19014 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19014.01.patch, HIVE-19014.02.patch, > HIVE-19014.03.patch, HIVE-19014.04.patch, HIVE-19014.05.patch, > HIVE-19014.06.patch, HIVE-19014.07.patch, HIVE-19014.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19135) Need tool to allow admins to create catalogs and move existing dbs to catalog during upgrade
[ https://issues.apache.org/jira/browse/HIVE-19135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431484#comment-16431484 ] Alan Gates commented on HIVE-19135: --- Running tests locally I got: Errors: TestAcidOnTez.testGetSplitsLocks TestJdbcWithLocalClusterSpark.testSparkQuery TestJdbcWithLocalClusterSpark.testTempTable TestJdbcWithMiniLlap.testLlapInputFormatEndToEnd TestMTQueries.testMTQueries1 TestMultiSessionsHS2WithLocalClusterSpark.testSparkQuery TestNegativeCliDriver.alter_notnull_constraint_violation TestNegativeCliDriver.insert_into_acid_notnull TestNegativeCliDriver.insert_into_notnull_constraint TestNegativeCliDriver.insert_multi_into_notnull TestNegativeCliDriver.insert_overwrite_notnull_constraint TestNegativeCliDriver.update_notnull_constraint Failures: TestBlobstoreCliDriver.insert_into_dynamic_partitions TestBlobstoreCliDriver.insert_overwrite_directory TestBlobstoreCliDriver.insert_overwrite_dynamic_partitions TestCliDriver.acid_table_stats TestCliDriver.auto_sortmerge_join_2 TestCliDriver.avro_alter_table_update_columns TestCliDriver.avrotblsjoin TestCliDriver.dbtxnmgr_showlocks TestCliDriver.orc_merge10 TestCliDriver.orc_schema_evolution_float TestCliDriver.parquet_ppd_multifiles TestCliDriver.schema_evol_par_vec_table_dictionary_encoding TestCliDriver.schema_evol_par_vec_table_non_dictionary_encoding TestCliDriver.selectindate TestCliDriver.statsoptimizer TestCliDriver.vector_bround TestCliDriver.vector_case_when_1 TestCliDriver.vector_coalesce_2 TestCliDriver.vector_coalesce_3 TestCliDriver.vector_interval_1 TestCliDriver.vectorized_parquet_types TestMetastoreVersion.testMetastoreVersion TestMetastoreVersion.testVersionMatching TestMiniDruidCliDriver.druidkafkamini_basic TestMiniLlapCliDriver.llap_smb TestMiniLlapCliDriver.unionDistinct_1 TestMiniTezCliDriver.explainanalyze_5 TestNegativeCliDriver.authorization_caseinsensitivity TestNegativeCliDriver.authorization_fail_1 TestNegativeCliDriver.authorization_grant_table_dup TestNegativeCliDriver.authorization_role_case TestNegativeCliDriver.authorization_role_grant_nosuchrole TestNegativeCliDriver.authorization_table_grant_nosuchrole TestNegativeCliDriver.subquery_subquery_chain TestSessionState.testCreatePath TestSessionState.testCreatePath TestSparkStatistics.testSparkStatistics I see the same results when I run the tests against master. > Need tool to allow admins to create catalogs and move existing dbs to catalog > during upgrade > > > Key: HIVE-19135 > URL: https://issues.apache.org/jira/browse/HIVE-19135 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Blocker > Fix For: 3.1.0 > > Attachments: HIVE19135.patch > > > As part of upgrading to Hive 3 admins may wish to create new catalogs and > move some existing databases into those catalogs. We can do this by adding > options to schematool. This guarantees that only admins can do these > operations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19000) Fix TestNegativeCliDriver
[ https://issues.apache.org/jira/browse/HIVE-19000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19000: --- Issue Type: Sub-task (was: Test) Parent: HIVE-19142 > Fix TestNegativeCliDriver > - > > Key: HIVE-19000 > URL: https://issues.apache.org/jira/browse/HIVE-19000 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Priority: Major > > TestNegativeCliDriver is failing since a while. We should investigate and fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19135) Need tool to allow admins to create catalogs and move existing dbs to catalog during upgrade
[ https://issues.apache.org/jira/browse/HIVE-19135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-19135: -- Status: Patch Available (was: Open) > Need tool to allow admins to create catalogs and move existing dbs to catalog > during upgrade > > > Key: HIVE-19135 > URL: https://issues.apache.org/jira/browse/HIVE-19135 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Blocker > Fix For: 3.1.0 > > Attachments: HIVE19135.patch > > > As part of upgrading to Hive 3 admins may wish to create new catalogs and > move some existing databases into those catalogs. We can do this by adding > options to schematool. This guarantees that only admins can do these > operations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19014) utilize YARN-8028 (queue ACL check) in Hive Tez session pool
[ https://issues.apache.org/jira/browse/HIVE-19014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431483#comment-16431483 ] Vineet Garg commented on HIVE-19014: [~jdere] [~sershe] Can you please add fix version? > utilize YARN-8028 (queue ACL check) in Hive Tez session pool > > > Key: HIVE-19014 > URL: https://issues.apache.org/jira/browse/HIVE-19014 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19014.01.patch, HIVE-19014.02.patch, > HIVE-19014.03.patch, HIVE-19014.04.patch, HIVE-19014.05.patch, > HIVE-19014.06.patch, HIVE-19014.07.patch, HIVE-19014.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19135) Need tool to allow admins to create catalogs and move existing dbs to catalog during upgrade
[ https://issues.apache.org/jira/browse/HIVE-19135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-19135: -- Attachment: HIVE19135.patch > Need tool to allow admins to create catalogs and move existing dbs to catalog > during upgrade > > > Key: HIVE-19135 > URL: https://issues.apache.org/jira/browse/HIVE-19135 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Blocker > Fix For: 3.1.0 > > Attachments: HIVE19135.patch > > > As part of upgrading to Hive 3 admins may wish to create new catalogs and > move some existing databases into those catalogs. We can do this by adding > options to schematool. This guarantees that only admins can do these > operations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19133) HS2 WebUI phase-wise performance metrics not showing correctly
[
https://issues.apache.org/jira/browse/HIVE-19133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431482#comment-16431482
]
Hive QA commented on HIVE-19133:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10098/dev-support/hive-personality.sh
|
| git revision | master / 109c594 |
| Default Java | 1.8.0_111 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10098/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> HS2 WebUI phase-wise performance metrics not showing correctly
> --
>
> Key: HIVE-19133
> URL: https://issues.apache.org/jira/browse/HIVE-19133
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Web UI
>Reporter: Bharathkrishna Guruvayoor Murali
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
> Attachments: HIVE-19133.1.patch, WebUI-compile time query metrics.png
>
>
> The query specific WebUI metrics (go to drilldown -> performance logging) are
> not showing up in the correct phase and are often mixed up.
> Attaching screenshot.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18840) CachedStore: Prioritize loading of recently accessed tables during prewarm
[ https://issues.apache.org/jira/browse/HIVE-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-18840: Attachment: HIVE-18840.2.patch > CachedStore: Prioritize loading of recently accessed tables during prewarm > -- > > Key: HIVE-18840 > URL: https://issues.apache.org/jira/browse/HIVE-18840 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-18840.1.patch, HIVE-18840.2.patch, > HIVE-18840.2.patch, HIVE-18840.2.patch, HIVE-18840.2.patch > > > On clusters with large metadata, prewarming the cache can take several hours. > Now that CachedStore does not block on prewarm anymore (after HIVE-18264), we > should prioritize loading of recently accessed tables during prewarm. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18840) CachedStore: Prioritize loading of recently accessed tables during prewarm
[ https://issues.apache.org/jira/browse/HIVE-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431476#comment-16431476 ] Vaibhav Gumashta commented on HIVE-18840: - Another rebase. > CachedStore: Prioritize loading of recently accessed tables during prewarm > -- > > Key: HIVE-18840 > URL: https://issues.apache.org/jira/browse/HIVE-18840 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-18840.1.patch, HIVE-18840.2.patch, > HIVE-18840.2.patch, HIVE-18840.2.patch, HIVE-18840.2.patch > > > On clusters with large metadata, prewarming the cache can take several hours. > Now that CachedStore does not block on prewarm anymore (after HIVE-18264), we > should prioritize loading of recently accessed tables during prewarm. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19141) TestNegativeCliDriver insert_into_notnull_constraint, insert_into_acid_notnull failing
[ https://issues.apache.org/jira/browse/HIVE-19141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-19141: -- > TestNegativeCliDriver insert_into_notnull_constraint, > insert_into_acid_notnull failing > -- > > Key: HIVE-19141 > URL: https://issues.apache.org/jira/browse/HIVE-19141 > Project: Hive > Issue Type: Test >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0, 3.1.0 > > > These tests have been consistently failing for a while. I suspect HIVE-18727 > has caused these failures. HIVE-18727 changed the code to throw ERROR instead > of EXCEPTION if constraints are violated. I guess Negative cli driver doesn't > handle errors. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19127) Concurrency fixes in QueryResultsCache
[
https://issues.apache.org/jira/browse/HIVE-19127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431457#comment-16431457
]
Hive QA commented on HIVE-19127:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918210/HIVE-19127.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 155 failed/errored test(s), 13652 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=252)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=252)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=96)
[jira] [Updated] (HIVE-19126) CachedStore: Use memory estimation to limit cache size during prewarm
[ https://issues.apache.org/jira/browse/HIVE-19126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-19126: Attachment: HIVE-19126.3.patch > CachedStore: Use memory estimation to limit cache size during prewarm > - > > Key: HIVE-19126 > URL: https://issues.apache.org/jira/browse/HIVE-19126 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-19126.1.patch, HIVE-19126.2.patch, > HIVE-19126.3.patch > > > We can rely on > https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/IncrementalObjectSizeEstimator.java > to estimate memory of SharedCache. This jira addresses the size estimation > during prewarm, so that we can stop when we hit the memory limit. In a > follow-up jira, we will work on estimation/eviction after prewarm is > complete, so that we can keep the frequently used tables and their partitions > in cache. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19126) CachedStore: Use memory estimation to limit cache size during prewarm
[ https://issues.apache.org/jira/browse/HIVE-19126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-19126: Status: Patch Available (was: Open) > CachedStore: Use memory estimation to limit cache size during prewarm > - > > Key: HIVE-19126 > URL: https://issues.apache.org/jira/browse/HIVE-19126 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-19126.1.patch, HIVE-19126.2.patch, > HIVE-19126.3.patch > > > We can rely on > https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/IncrementalObjectSizeEstimator.java > to estimate memory of SharedCache. This jira addresses the size estimation > during prewarm, so that we can stop when we hit the memory limit. In a > follow-up jira, we will work on estimation/eviction after prewarm is > complete, so that we can keep the frequently used tables and their partitions > in cache. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18739) Add support for Export from Acid table
[ https://issues.apache.org/jira/browse/HIVE-18739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-18739: -- Status: Open (was: Patch Available) > Add support for Export from Acid table > -- > > Key: HIVE-18739 > URL: https://issues.apache.org/jira/browse/HIVE-18739 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Major > Attachments: HIVE-18739.01.patch, HIVE-18739.04.patch, > HIVE-18739.04.patch, HIVE-18739.06.patch, HIVE-18739.08.patch, > HIVE-18739.09.patch, HIVE-18739.10.patch, HIVE-18739.11.patch, > HIVE-18739.12.patch, HIVE-18739.13.patch, HIVE-18739.14.patch, > HIVE-18739.15.patch, HIVE-18739.16.patch, HIVE-18739.17.patch, > HIVE-18739.19.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18739) Add support for Export from Acid table
[ https://issues.apache.org/jira/browse/HIVE-18739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-18739: -- Attachment: HIVE-18739.19.patch > Add support for Export from Acid table > -- > > Key: HIVE-18739 > URL: https://issues.apache.org/jira/browse/HIVE-18739 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Major > Attachments: HIVE-18739.01.patch, HIVE-18739.04.patch, > HIVE-18739.04.patch, HIVE-18739.06.patch, HIVE-18739.08.patch, > HIVE-18739.09.patch, HIVE-18739.10.patch, HIVE-18739.11.patch, > HIVE-18739.12.patch, HIVE-18739.13.patch, HIVE-18739.14.patch, > HIVE-18739.15.patch, HIVE-18739.16.patch, HIVE-18739.17.patch, > HIVE-18739.19.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19140) Update metastore upgrade scripts to prepare for 3.1.0 development
[ https://issues.apache.org/jira/browse/HIVE-19140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-19140: -- > Update metastore upgrade scripts to prepare for 3.1.0 development > - > > Key: HIVE-19140 > URL: https://issues.apache.org/jira/browse/HIVE-19140 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.1.0 > > > Now that branch for hive 3.0.0 is cut and we have started preparing for hive > 3.1.0 development we need to add metastore upgrade scripts to upgrade from > 3.0.0 to 3.1.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19128) Update golden files for spark perf tests
[ https://issues.apache.org/jira/browse/HIVE-19128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431376#comment-16431376 ] Ashutosh Chauhan commented on HIVE-19128: - Pushed to branch-3 as well. > Update golden files for spark perf tests > > > Key: HIVE-19128 > URL: https://issues.apache.org/jira/browse/HIVE-19128 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19128.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19128) Update golden files for spark perf tests
[ https://issues.apache.org/jira/browse/HIVE-19128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431368#comment-16431368 ] Vineet Garg commented on HIVE-19128: [~ashutoshc] Yes please go ahead and commit it in branch-3. Updating JIRA to add fix version 3.1.0 > Update golden files for spark perf tests > > > Key: HIVE-19128 > URL: https://issues.apache.org/jira/browse/HIVE-19128 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19128.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19128) Update golden files for spark perf tests
[ https://issues.apache.org/jira/browse/HIVE-19128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19128: --- Fix Version/s: 3.1.0 > Update golden files for spark perf tests > > > Key: HIVE-19128 > URL: https://issues.apache.org/jira/browse/HIVE-19128 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19128.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19128) Update golden files for spark perf tests
[ https://issues.apache.org/jira/browse/HIVE-19128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431365#comment-16431365 ] Ashutosh Chauhan commented on HIVE-19128: - [~vgarg] It will be useful in branch-3 as well. Shall I commit there too? > Update golden files for spark perf tests > > > Key: HIVE-19128 > URL: https://issues.apache.org/jira/browse/HIVE-19128 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19128.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19128) Update golden files for spark perf tests
[ https://issues.apache.org/jira/browse/HIVE-19128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19128: Resolution: Fixed Assignee: Ashutosh Chauhan Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pushed updated golden files. > Update golden files for spark perf tests > > > Key: HIVE-19128 > URL: https://issues.apache.org/jira/browse/HIVE-19128 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19128.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19138) Results cache: allow queries waiting on pending cache entries to check cache again if pending query fails
[ https://issues.apache.org/jira/browse/HIVE-19138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431356#comment-16431356 ] Jason Dere commented on HIVE-19138: --- RB at https://reviews.apache.org/r/66516/ [~gopalv] [~djaiswal] can you take a look? > Results cache: allow queries waiting on pending cache entries to check cache > again if pending query fails > - > > Key: HIVE-19138 > URL: https://issues.apache.org/jira/browse/HIVE-19138 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Attachments: HIVE-19138.1.patch > > > HIVE-18846 allows the results cache to refer to currently executing queries > so that another query can wait for these results to become ready in the > results cache. If the pending query fails then Hive will automatically skip > the cache and do the full query compilation. Make a fix here so that if the > pending query fails, Hive will still try to check the cache again in case the > cache has another cached/pending result that can be used to answer the query. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18946) Fix columnstats merge NPE
[
https://issues.apache.org/jira/browse/HIVE-18946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431353#comment-16431353
]
Ashutosh Chauhan commented on HIVE-18946:
-
compareTo() should be flipped for max vs min. Can't reuse same logic for both.
> Fix columnstats merge NPE
> -
>
> Key: HIVE-18946
> URL: https://issues.apache.org/jira/browse/HIVE-18946
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-18946.01.patch, HIVE-18946.02.patch
>
>
> after analyzing an empty table may lead to an NPE when inserting into it...
> {code}
> 2018-03-13T06:54:22,503 ERROR [df3fb505-e0bc-4595-a874-b735dab8dff6 main]
> metastore.RetryingHMSHandler: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.metastore.api.Decimal.compareTo(Decimal.java:318)
> at
> org.apache.hadoop.hive.metastore.columnstats.merge.DecimalColumnStatsMerger.merge(DecimalColumnStatsMerger.java:35)
> at
> org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.mergeColStats(MetaStoreUtils.java:778)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.set_aggr_stats_for(HiveMetaStore.java:6934)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy55.set_aggr_stats_for(Unknown Source)
> [...]
> {code}
> reproduce
> {code}
> set hive.stats.autogather=true;
> set hive.explain.user=true;
> drop table if exists testdeci2;
> create table testdeci2(
> id int,
> amount decimal(10,3),
> sales_tax decimal(10,3),
> item string)
> stored as orc location '/tmp/testdeci2'
> TBLPROPERTIES ("transactional"="false")
> ;
> analyze table testdeci2 compute statistics for columns;
> insert into table testdeci2
> values(1,12.123,12345.123,'desk1'),(2,123.123,1234.123,'desk2');
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19138) Results cache: allow queries waiting on pending cache entries to check cache again if pending query fails
[ https://issues.apache.org/jira/browse/HIVE-19138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-19138: -- Attachment: HIVE-19138.1.patch > Results cache: allow queries waiting on pending cache entries to check cache > again if pending query fails > - > > Key: HIVE-19138 > URL: https://issues.apache.org/jira/browse/HIVE-19138 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Attachments: HIVE-19138.1.patch > > > HIVE-18846 allows the results cache to refer to currently executing queries > so that another query can wait for these results to become ready in the > results cache. If the pending query fails then Hive will automatically skip > the cache and do the full query compilation. Make a fix here so that if the > pending query fails, Hive will still try to check the cache again in case the > cache has another cached/pending result that can be used to answer the query. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19138) Results cache: allow queries waiting on pending cache entries to check cache again if pending query fails
[ https://issues.apache.org/jira/browse/HIVE-19138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-19138: -- Status: Patch Available (was: Open) > Results cache: allow queries waiting on pending cache entries to check cache > again if pending query fails > - > > Key: HIVE-19138 > URL: https://issues.apache.org/jira/browse/HIVE-19138 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Attachments: HIVE-19138.1.patch > > > HIVE-18846 allows the results cache to refer to currently executing queries > so that another query can wait for these results to become ready in the > results cache. If the pending query fails then Hive will automatically skip > the cache and do the full query compilation. Make a fix here so that if the > pending query fails, Hive will still try to check the cache again in case the > cache has another cached/pending result that can be used to answer the query. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19138) Results cache: allow queries waiting on pending cache entries to check cache again if pending query fails
[ https://issues.apache.org/jira/browse/HIVE-19138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere reassigned HIVE-19138: - > Results cache: allow queries waiting on pending cache entries to check cache > again if pending query fails > - > > Key: HIVE-19138 > URL: https://issues.apache.org/jira/browse/HIVE-19138 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > > HIVE-18846 allows the results cache to refer to currently executing queries > so that another query can wait for these results to become ready in the > results cache. If the pending query fails then Hive will automatically skip > the cache and do the full query compilation. Make a fix here so that if the > pending query fails, Hive will still try to check the cache again in case the > cache has another cached/pending result that can be used to answer the query. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18999) Filter operator does not work for List
[
https://issues.apache.org/jira/browse/HIVE-18999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431342#comment-16431342
]
Steve Yeom commented on HIVE-18999:
---
If nothing prevents, we can just move the prepareInSet() changes into
compilation time.
> Filter operator does not work for List
> --
>
> Key: HIVE-18999
> URL: https://issues.apache.org/jira/browse/HIVE-18999
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.0.0
>Reporter: Steve Yeom
>Assignee: Steve Yeom
>Priority: Major
> Attachments: HIVE-18999.01.patch, HIVE-18999.02.patch,
> HIVE-18999.03.patch
>
>
> {code:sql}
> create table table1(col0 int, col1 bigint, col2 string, col3 bigint, col4
> bigint);
> insert into table1 values (1, 1, 'ccl',2014, 11);
> insert into table1 values (1, 1, 'ccl',2015, 11);
> insert into table1 values (1, 1, 'ccl',2014, 11);
> insert into table1 values (1, 1, 'ccl',2013, 11);
> -- INCORRECT
> SELECT COUNT(t1.col0) from table1 t1 where struct(col3, col4) in
> (struct(2014,11));
> -- CORRECT
> SELECT COUNT(t1.col0) from table1 t1 where struct(col3, col4) in
> (struct('2014','11'));
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18991) Drop database cascade doesn't work with materialized views
[ https://issues.apache.org/jira/browse/HIVE-18991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18991: --- Attachment: HIVE-18991.08.patch > Drop database cascade doesn't work with materialized views > -- > > Key: HIVE-18991 > URL: https://issues.apache.org/jira/browse/HIVE-18991 > Project: Hive > Issue Type: Bug > Components: Materialized views, Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-18991.01.patch, HIVE-18991.03.patch, > HIVE-18991.06.patch, HIVE-18991.07.patch, HIVE-18991.08.patch, > HIVE-18991.patch > > > Create a database, add a table and then a materialized view that depends on > the table. Then drop the database with cascade set. Sometimes this will > fail because when HiveMetaStore.drop_database_core goes to drop all of the > tables it may drop the base table before the materialized view, which will > cause an integrity constraint violation in the RDBMS. To resolve this that > method should change to fetch and drop materialized views before tables. > cc [~jcamachorodriguez] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18991) Drop database cascade doesn't work with materialized views
[ https://issues.apache.org/jira/browse/HIVE-18991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431332#comment-16431332 ] Jesus Camacho Rodriguez commented on HIVE-18991: Fixed related tests, triggering another ptests run. > Drop database cascade doesn't work with materialized views > -- > > Key: HIVE-18991 > URL: https://issues.apache.org/jira/browse/HIVE-18991 > Project: Hive > Issue Type: Bug > Components: Materialized views, Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-18991.01.patch, HIVE-18991.03.patch, > HIVE-18991.06.patch, HIVE-18991.07.patch, HIVE-18991.08.patch, > HIVE-18991.patch > > > Create a database, add a table and then a materialized view that depends on > the table. Then drop the database with cascade set. Sometimes this will > fail because when HiveMetaStore.drop_database_core goes to drop all of the > tables it may drop the base table before the materialized view, which will > cause an integrity constraint violation in the RDBMS. To resolve this that > method should change to fetch and drop materialized views before tables. > cc [~jcamachorodriguez] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19134) Update copyright NOTICE and fix rat check failures
[ https://issues.apache.org/jira/browse/HIVE-19134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19134: --- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master and 3.0.0 > Update copyright NOTICE and fix rat check failures > -- > > Key: HIVE-19134 > URL: https://issues.apache.org/jira/browse/HIVE-19134 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19134.1.patch, HIVE-19134.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19127) Concurrency fixes in QueryResultsCache
[
https://issues.apache.org/jira/browse/HIVE-19127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431328#comment-16431328
]
Hive QA commented on HIVE-19127:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
47s{color} | {color:red} ql: The patch generated 1 new + 10 unchanged - 0 fixed
= 11 total (was 10) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10097/dev-support/hive-personality.sh
|
| git revision | master / d589ee7 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10097/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10097/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10097/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Concurrency fixes in QueryResultsCache
> --
>
> Key: HIVE-19127
> URL: https://issues.apache.org/jira/browse/HIVE-19127
> Project: Hive
> Issue Type: Sub-task
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19127.1.patch, HIVE-19127.2.patch
>
>
> A few fixes around concurrent access in the results cache
> - Take a lock on the cache entry when in the process of setting the cache
> entry from PENDING state to VALID state, so that other threads cannot
> invalidate the entry
> - The write lock on the cache was not being taken when removing an entry from
> the cache.
> - synchronize access when iterating through the lru keyset
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18999) Filter operator does not work for List
[
https://issues.apache.org/jira/browse/HIVE-18999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431300#comment-16431300
]
Steve Yeom commented on HIVE-18999:
---
1. The newly added q file for this jira "struct_in_filter.q" runs successfully
with "hive.vectorized.execution.enabled=true".
2. Out of two modified files, "ql/udf/generic/GenericUDFIn.java"
and "ql/udf/generic/GenericUDFUtils.java",
GenericUDFUtils.update() is called during compilation (logical plan
generation/optimization),.
GenericUDFIn.prepareInSet() is called during execution but JUST ONE TIME when
the "constantInSet"
is null.
I am investigating the possibility that "deferred object list" input
arguments to the method may have
cases where their final values are determined ONLY during execution.
> Filter operator does not work for List
> --
>
> Key: HIVE-18999
> URL: https://issues.apache.org/jira/browse/HIVE-18999
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.0.0
>Reporter: Steve Yeom
>Assignee: Steve Yeom
>Priority: Major
> Attachments: HIVE-18999.01.patch, HIVE-18999.02.patch,
> HIVE-18999.03.patch
>
>
> {code:sql}
> create table table1(col0 int, col1 bigint, col2 string, col3 bigint, col4
> bigint);
> insert into table1 values (1, 1, 'ccl',2014, 11);
> insert into table1 values (1, 1, 'ccl',2015, 11);
> insert into table1 values (1, 1, 'ccl',2014, 11);
> insert into table1 values (1, 1, 'ccl',2013, 11);
> -- INCORRECT
> SELECT COUNT(t1.col0) from table1 t1 where struct(col3, col4) in
> (struct(2014,11));
> -- CORRECT
> SELECT COUNT(t1.col0) from table1 t1 where struct(col3, col4) in
> (struct('2014','11'));
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19014) utilize YARN-8028 (queue ACL check) in Hive Tez session pool
[
https://issues.apache.org/jira/browse/HIVE-19014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431293#comment-16431293
]
Hive QA commented on HIVE-19014:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918207/HIVE-19014.07.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 186 failed/errored test(s), 14048 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=252)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=252)
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=216)
TestSequenceFileReadWrite - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[statsoptimizer]
(batchId=62)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez_empty]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[groupby_groupingset_bug]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mergejoin]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[update_access_time_non_current_db]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_dynamic_semijoin_reduction]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.org.apache.hadoop.hive.cli.TestNegativeCliDriver
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.org.apache.hadoop.hive.cli.TestNegativeCliDriver
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_notnull_constraint_violation]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_caseinsensitivity]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_fail_1]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_grant_table_dup]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_role_case]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_role_grant_nosuchrole]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_table_grant_nosuchrole]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_into_acid_notnull]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_into_notnull_constraint]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_multi_into_notnull]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_overwrite_notnull_constraint]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[smb_bucketmapjoin]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[smb_mapjoin_14]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[sortmerge_mapjoin_mismatch_1]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_2]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_publisher_error_1]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_publisher_error_2]
(batchId=95)
[jira] [Commented] (HIVE-19074) Vectorization: Add llap vectorization_div0.q.out Q output file
[ https://issues.apache.org/jira/browse/HIVE-19074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431291#comment-16431291 ] Matt McCline commented on HIVE-19074: - Added missing changes. The whole fix made it into branch-3.0.0 before it branched. It is in master, too. > Vectorization: Add llap vectorization_div0.q.out Q output file > -- > > Key: HIVE-19074 > URL: https://issues.apache.org/jira/browse/HIVE-19074 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-19074.02.patch, HIVE-19074.03.patch, > HIVE-19074.05.patch > > > At some point llap/vectorization_div0.q.out got omitted. > The Q file output is unstable because of missing ORDER BY columns. You must > have ORDER BY on all/critical the columns when there is a LIMIT clause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19074) Vectorization: Add llap vectorization_div0.q.out Q output file
[ https://issues.apache.org/jira/browse/HIVE-19074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19074: Resolution: Fixed Status: Resolved (was: Patch Available) > Vectorization: Add llap vectorization_div0.q.out Q output file > -- > > Key: HIVE-19074 > URL: https://issues.apache.org/jira/browse/HIVE-19074 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-19074.02.patch, HIVE-19074.03.patch, > HIVE-19074.05.patch > > > At some point llap/vectorization_div0.q.out got omitted. > The Q file output is unstable because of missing ORDER BY columns. You must > have ORDER BY on all/critical the columns when there is a LIMIT clause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19074) Vectorization: Add llap vectorization_div0.q.out Q output file
[ https://issues.apache.org/jira/browse/HIVE-19074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19074: Fix Version/s: (was: 3.1.0) 3.0.0 > Vectorization: Add llap vectorization_div0.q.out Q output file > -- > > Key: HIVE-19074 > URL: https://issues.apache.org/jira/browse/HIVE-19074 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-19074.02.patch, HIVE-19074.03.patch, > HIVE-19074.05.patch > > > At some point llap/vectorization_div0.q.out got omitted. > The Q file output is unstable because of missing ORDER BY columns. You must > have ORDER BY on all/critical the columns when there is a LIMIT clause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19137) orcfiledump doesn't print hive.acid.version value
[
https://issues.apache.org/jira/browse/HIVE-19137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-19137:
--
Description:
HIVE-18659 added hive.acid.version in the file footer.
orcfiledump prints something like
{noformat}
User Metadata:
hive.acid.key.index=1,536870912,1;
hive.acid.stats=2,0,0
hive.acid.version=
{noformat}
probably because
public static void setAcidVersionInDataFile(Writer writer) {
//so that we know which version wrote the file
ByteBuffer bf = ByteBuffer.allocate(4).putInt(ORC_ACID_VERSION);
bf.rewind(); //don't ask - some ByteBuffer weridness. w/o this, empty
buffer is written
writer.addUserMetadata(ACID_VERSION_KEY, bf);
}
{noformat}
use
{{UTF8.encode())}} instead
was:
HIVE-18659 added hive.acid.version in the file footer.
orcfiledump prints something like
{noformat}
User Metadata:
hive.acid.key.index=1,536870912,1;
hive.acid.stats=2,0,0
hive.acid.version=
{noformat}
probably because
public static void setAcidVersionInDataFile(Writer writer) {
//so that we know which version wrote the file
ByteBuffer bf = ByteBuffer.allocate(4).putInt(ORC_ACID_VERSION);
bf.rewind(); //don't ask - some ByteBuffer weridness. w/o this, empty
buffer is written
writer.addUserMetadata(ACID_VERSION_KEY, bf);
}
{noformat}
use
{{UTF8.encode()) instead
> orcfiledump doesn't print hive.acid.version value
> -
>
> Key: HIVE-19137
> URL: https://issues.apache.org/jira/browse/HIVE-19137
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
>
> HIVE-18659 added hive.acid.version in the file footer.
> orcfiledump prints something like
> {noformat}
> User Metadata:
> hive.acid.key.index=1,536870912,1;
> hive.acid.stats=2,0,0
> hive.acid.version=
> {noformat}
> probably because
> public static void setAcidVersionInDataFile(Writer writer) {
> //so that we know which version wrote the file
> ByteBuffer bf = ByteBuffer.allocate(4).putInt(ORC_ACID_VERSION);
> bf.rewind(); //don't ask - some ByteBuffer weridness. w/o this, empty
> buffer is written
> writer.addUserMetadata(ACID_VERSION_KEY, bf);
> }
> {noformat}
> use
> {{UTF8.encode())}} instead
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19137) orcfiledump doesn't print hive.acid.version value
[
https://issues.apache.org/jira/browse/HIVE-19137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-19137:
--
Description:
HIVE-18659 added hive.acid.version in the file footer.
orcfiledump prints something like
{noformat}
User Metadata:
hive.acid.key.index=1,536870912,1;
hive.acid.stats=2,0,0
hive.acid.version=
{noformat}
probably because
{noformat}
public static void setAcidVersionInDataFile(Writer writer) {
//so that we know which version wrote the file
ByteBuffer bf = ByteBuffer.allocate(4).putInt(ORC_ACID_VERSION);
bf.rewind(); //don't ask - some ByteBuffer weridness. w/o this, empty
buffer is written
writer.addUserMetadata(ACID_VERSION_KEY, bf);
}
{noformat}
use
{{UTF8.encode())}} instead
was:
HIVE-18659 added hive.acid.version in the file footer.
orcfiledump prints something like
{noformat}
User Metadata:
hive.acid.key.index=1,536870912,1;
hive.acid.stats=2,0,0
hive.acid.version=
{noformat}
probably because
public static void setAcidVersionInDataFile(Writer writer) {
//so that we know which version wrote the file
ByteBuffer bf = ByteBuffer.allocate(4).putInt(ORC_ACID_VERSION);
bf.rewind(); //don't ask - some ByteBuffer weridness. w/o this, empty
buffer is written
writer.addUserMetadata(ACID_VERSION_KEY, bf);
}
{noformat}
use
{{UTF8.encode())}} instead
> orcfiledump doesn't print hive.acid.version value
> -
>
> Key: HIVE-19137
> URL: https://issues.apache.org/jira/browse/HIVE-19137
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
>
> HIVE-18659 added hive.acid.version in the file footer.
> orcfiledump prints something like
> {noformat}
> User Metadata:
> hive.acid.key.index=1,536870912,1;
> hive.acid.stats=2,0,0
> hive.acid.version=
> {noformat}
> probably because
> {noformat}
> public static void setAcidVersionInDataFile(Writer writer) {
> //so that we know which version wrote the file
> ByteBuffer bf = ByteBuffer.allocate(4).putInt(ORC_ACID_VERSION);
> bf.rewind(); //don't ask - some ByteBuffer weridness. w/o this, empty
> buffer is written
> writer.addUserMetadata(ACID_VERSION_KEY, bf);
> }
> {noformat}
> use
> {{UTF8.encode())}} instead
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19137) orcfiledump doesn't print hive.acid.version value
[
https://issues.apache.org/jira/browse/HIVE-19137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman reassigned HIVE-19137:
-
> orcfiledump doesn't print hive.acid.version value
> -
>
> Key: HIVE-19137
> URL: https://issues.apache.org/jira/browse/HIVE-19137
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
>
> HIVE-18659 added hive.acid.version in the file footer.
> orcfiledump prints something like
> {noformat}
> User Metadata:
> hive.acid.key.index=1,536870912,1;
> hive.acid.stats=2,0,0
> hive.acid.version=
> {noformat}
> probably because
> public static void setAcidVersionInDataFile(Writer writer) {
> //so that we know which version wrote the file
> ByteBuffer bf = ByteBuffer.allocate(4).putInt(ORC_ACID_VERSION);
> bf.rewind(); //don't ask - some ByteBuffer weridness. w/o this, empty
> buffer is written
> writer.addUserMetadata(ACID_VERSION_KEY, bf);
> }
> {noformat}
> use
> {{UTF8.encode()) instead
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19133) HS2 WebUI phase-wise performance metrics not showing correctly
[ https://issues.apache.org/jira/browse/HIVE-19133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431280#comment-16431280 ] Bharathkrishna Guruvayoor Murali commented on HIVE-19133: - + Adding [~kgyrtkirk] and [~gopalv] as you have reported similar issue. > HS2 WebUI phase-wise performance metrics not showing correctly > -- > > Key: HIVE-19133 > URL: https://issues.apache.org/jira/browse/HIVE-19133 > Project: Hive > Issue Type: Bug > Components: HiveServer2, Web UI >Reporter: Bharathkrishna Guruvayoor Murali >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-19133.1.patch, WebUI-compile time query metrics.png > > > The query specific WebUI metrics (go to drilldown -> performance logging) are > not showing up in the correct phase and are often mixed up. > Attaching screenshot. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-16346) inheritPerms should be conditional based on the target filesystem
[
https://issues.apache.org/jira/browse/HIVE-16346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16346:
---
Target Version/s: (was: 3.0.0)
> inheritPerms should be conditional based on the target filesystem
> -
>
> Key: HIVE-16346
> URL: https://issues.apache.org/jira/browse/HIVE-16346
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Fix For: 2.4.0
>
> Attachments: HIVE-16346.1-branch-2.patch,
> HIVE-16346.2-branch-2.patch, HIVE-16346.3-branch-2.patch
>
>
> Right now, a lot of the logic in {{Hive.java}} attempts to set permissions of
> different files that have been moved / copied. This is only triggered if
> {{hive.warehouse.subdir.inherit.perms}} is set to true.
> However, on blobstores such as S3, there is no concept of file permissions so
> these calls are unnecessary, which can hurt performance.
> One solution would be to set {{hive.warehouse.subdir.inherit.perms}} to
> false, but this would be a global change that affects an entire HS2 instance.
> So HDFS tables will no longer have permissions inheritance.
> A better solution would be to make the inheritance of permissions conditional
> on the target filesystem.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19133) HS2 WebUI phase-wise performance metrics not showing correctly
[ https://issues.apache.org/jira/browse/HIVE-19133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430967#comment-16430967 ] Bharathkrishna Guruvayoor Murali edited comment on HIVE-19133 at 4/9/18 8:54 PM: - It was noted that the PerfLogger is passed between the background and handler threads which caused the issue. Hence, resetting the perflogger both in compilation and execution phase so that old values present in the maps are cleared. Also, using queryDisplay.getQueryStartTime() because the DRIVER_RUN value of compilation phase is now not available in the execution phase after this change because Perflogger is reset. The main cause of this issue is that perflogger is passed across threads (which I feel is kind of abusing the ThreadLocal nature of perflogger). This happens in [https://github.com/apache/hive/blob/a2394c5bf2e5b257a3a1b3e2bda4a25b92005315/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java#L294] I feel we can remove this part where it sets parentPerfLogger, but not doing it as I don't know if it is done that way for any specific reason. Anyone who has some insights on this please let me know if it can be removed. was (Author: bharos92): It was noted that the PerfLogger is passed between the background and handler threads which caused the issue. Hence, resetting the perflogger both in compilation and execution phase so that old values present in the maps are cleared. Also, using queryDisplay.getQueryStartTime() because the DRIVER_RUN value of compilation phase is now not available in the execution phase after this change because Perflogger is reset. The main cause of this issue is that perflogger is passed across threads (which I feel is kind of abusing the ThreadLocal nature of perflogger). This happens in [https://github.com/apache/hive/blob/a2394c5bf2e5b257a3a1b3e2bda4a25b92005315/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java#L294] I feel we can remove this part where it sets parentPerfLogger, but not doing it as I don't know if it serves any specific purpose. Anyone who has some insights on this please let me know if it can be removed. > HS2 WebUI phase-wise performance metrics not showing correctly > -- > > Key: HIVE-19133 > URL: https://issues.apache.org/jira/browse/HIVE-19133 > Project: Hive > Issue Type: Bug > Components: HiveServer2, Web UI >Reporter: Bharathkrishna Guruvayoor Murali >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-19133.1.patch, WebUI-compile time query metrics.png > > > The query specific WebUI metrics (go to drilldown -> performance logging) are > not showing up in the correct phase and are often mixed up. > Attaching screenshot. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18410) [Performance][Avro] Reading flat Avro tables is very expensive in Hive
[
https://issues.apache.org/jira/browse/HIVE-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18410:
---
Fix Version/s: (was: 3.0.0)
3.1.0
> [Performance][Avro] Reading flat Avro tables is very expensive in Hive
> --
>
> Key: HIVE-18410
> URL: https://issues.apache.org/jira/browse/HIVE-18410
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 1.2.1, 2.1.0, 3.0.0, 2.3.2
>Reporter: Ratandeep Ratti
>Assignee: Ratandeep Ratti
>Priority: Major
> Fix For: 2.3.2, 3.1.0
>
> Attachments: HIVE-18410.patch, HIVE-18410_1.patch,
> HIVE-18410_2.patch, HIVE-18410_3.patch, profiling_with_patch.nps,
> profiling_with_patch.png, profiling_without_patch.nps,
> profiling_without_patch.png
>
>
> There's a performance penalty when reading flat [no nested fields] Avro
> tables. When reading the same flat dataset in Pig, it takes half the time.
> On profiling, a lot of time is spent in
> {{AvroDeserializer.deserializeSingleItemNullableUnion()}}. The bulk of the
> time is spent in GenericData.get().resolveUnion(), which calls
> GenericData.getSchemaName(Object datum), which does a lot of instanceof
> checks. This could be simplified with performance benefits. A approach is
> described in this patch which almost halves the runtime.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16674) Hive metastore JVM dumps core
[
https://issues.apache.org/jira/browse/HIVE-16674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431274#comment-16431274
]
Vineet Garg commented on HIVE-16674:
Deferring this to 3.1.0 since the branch for 3.0.0 has been cut off. Please
update the JIRA if you would like to get your patch in 3.0.0.
> Hive metastore JVM dumps core
> -
>
> Key: HIVE-16674
> URL: https://issues.apache.org/jira/browse/HIVE-16674
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.2.1
> Environment: Hive-1.2.1
> Kerberos enabled cluster
>Reporter: Vlad Gudikov
>Assignee: Vlad Gudikov
>Priority: Blocker
> Fix For: 1.2.1, 3.1.0
>
> Attachments: HIVE-16674.1.patch, HIVE-16674.patch
>
>
> While trying to run a Hive query on 24 partitions executed on an external
> table with large amount of partitions (4K+). I get an error
> {code}
> - org.apache.thrift.transport.TSaslTransport$SaslParticipant.wrap(byte[],
> int, int) @bci=27, line=568 (Compiled frame)
> - org.apache.thrift.transport.TSaslTransport.flush() @bci=52, line=492
> (Compiled frame)
> - org.apache.thrift.transport.TSaslServerTransport.flush() @bci=1, line=41
> (Compiled frame)
> - org.apache.thrift.ProcessFunction.process(int,
> org.apache.thrift.protocol.TProtocol, org.apache.thrift.protocol.TProtocol,
> java.lang.Object) @bci=236, line=55 (Compiled frame)
> -
> org.apache.thrift.TBaseProcessor.process(org.apache.thrift.protocol.TProtocol,
> org.apache.thrift.protocol.TProtocol) @bci=126, line=39 (Compiled frame)
> -
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run()
> @bci=15, line=690 (Compiled frame)
> -
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run()
> @bci=1, line=685 (Compiled frame)
> -
> java.security.AccessController.doPrivileged(java.security.PrivilegedExceptionAction,
> java.security.AccessControlContext) @bci=0 (Compiled frame)
> - javax.security.auth.Subject.doAs(javax.security.auth.Subject,
> java.security.PrivilegedExceptionAction) @bci=42, line=422 (Compiled frame)
> -
> org.apache.hadoop.security.UserGroupInformation.doAs(java.security.PrivilegedExceptionAction)
> @bci=14, line=1595 (Compiled frame)
> -
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(org.apache.thrift.protocol.TProtocol,
> org.apache.thrift.protocol.TProtocol) @bci=273, line=685 (Compiled frame)
> - org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run() @bci=151,
> line=285 (Interpreted frame)
> -
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
> @bci=95, line=1142 (Interpreted frame)
> - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617
> (Interpreted frame)
> - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16674) Hive metastore JVM dumps core
[
https://issues.apache.org/jira/browse/HIVE-16674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16674:
---
Fix Version/s: (was: 3.0.0)
3.1.0
> Hive metastore JVM dumps core
> -
>
> Key: HIVE-16674
> URL: https://issues.apache.org/jira/browse/HIVE-16674
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.2.1
> Environment: Hive-1.2.1
> Kerberos enabled cluster
>Reporter: Vlad Gudikov
>Assignee: Vlad Gudikov
>Priority: Blocker
> Fix For: 1.2.1, 3.1.0
>
> Attachments: HIVE-16674.1.patch, HIVE-16674.patch
>
>
> While trying to run a Hive query on 24 partitions executed on an external
> table with large amount of partitions (4K+). I get an error
> {code}
> - org.apache.thrift.transport.TSaslTransport$SaslParticipant.wrap(byte[],
> int, int) @bci=27, line=568 (Compiled frame)
> - org.apache.thrift.transport.TSaslTransport.flush() @bci=52, line=492
> (Compiled frame)
> - org.apache.thrift.transport.TSaslServerTransport.flush() @bci=1, line=41
> (Compiled frame)
> - org.apache.thrift.ProcessFunction.process(int,
> org.apache.thrift.protocol.TProtocol, org.apache.thrift.protocol.TProtocol,
> java.lang.Object) @bci=236, line=55 (Compiled frame)
> -
> org.apache.thrift.TBaseProcessor.process(org.apache.thrift.protocol.TProtocol,
> org.apache.thrift.protocol.TProtocol) @bci=126, line=39 (Compiled frame)
> -
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run()
> @bci=15, line=690 (Compiled frame)
> -
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run()
> @bci=1, line=685 (Compiled frame)
> -
> java.security.AccessController.doPrivileged(java.security.PrivilegedExceptionAction,
> java.security.AccessControlContext) @bci=0 (Compiled frame)
> - javax.security.auth.Subject.doAs(javax.security.auth.Subject,
> java.security.PrivilegedExceptionAction) @bci=42, line=422 (Compiled frame)
> -
> org.apache.hadoop.security.UserGroupInformation.doAs(java.security.PrivilegedExceptionAction)
> @bci=14, line=1595 (Compiled frame)
> -
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(org.apache.thrift.protocol.TProtocol,
> org.apache.thrift.protocol.TProtocol) @bci=273, line=685 (Compiled frame)
> - org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run() @bci=151,
> line=285 (Interpreted frame)
> -
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
> @bci=95, line=1142 (Interpreted frame)
> - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617
> (Interpreted frame)
> - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
