[jira] [Commented] (HIVE-20208) order by position alias do not work
[
https://issues.apache.org/jira/browse/HIVE-20208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550277#comment-16550277
]
Hive QA commented on HIVE-20208:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
57s{color} | {color:blue} ql in master has 2281 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12718/dev-support/hive-personality.sh
|
| git revision | master / 851c8ab |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12718/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> order by position alias do not work
> ---
>
> Key: HIVE-20208
> URL: https://issues.apache.org/jira/browse/HIVE-20208
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: xulongfetion
>Assignee: Vineet Garg
>Priority: Blocker
> Labels: patch-available
> Attachments: HIVE-orderby-position-alias.patch
>
>
> the order by position alias do not work if there's more than one position
> alias. For query like
> "select a, b, c from table1 order by 1, 2;", hive will report column not
> found error.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17975) "HiveServer2 Start Time" changes when page refreshes
[ https://issues.apache.org/jira/browse/HIVE-17975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550273#comment-16550273 ] Xiaomin Zhang commented on HIVE-17975: -- Thanks [~pvary] I assign this to me and submitted the patch again. > "HiveServer2 Start Time" changes when page refreshes > > > Key: HIVE-17975 > URL: https://issues.apache.org/jira/browse/HIVE-17975 > Project: Hive > Issue Type: Bug > Components: Web UI >Affects Versions: 2.1.0 >Reporter: Xiaomin Zhang >Assignee: Xiaomin Zhang >Priority: Minor > Attachments: HIVE-17975.patch, HIVE-17975.patch > > > This looks affected by HIVE-14207, that a copy of hiveConf is used in the > HttpServer.Builder, while the 'startcode' is only added into the original > hiveConf. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17975) "HiveServer2 Start Time" changes when page refreshes
[ https://issues.apache.org/jira/browse/HIVE-17975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaomin Zhang updated HIVE-17975: - Attachment: HIVE-17975.patch Status: Patch Available (was: In Progress) > "HiveServer2 Start Time" changes when page refreshes > > > Key: HIVE-17975 > URL: https://issues.apache.org/jira/browse/HIVE-17975 > Project: Hive > Issue Type: Bug > Components: Web UI >Affects Versions: 2.1.0 >Reporter: Xiaomin Zhang >Assignee: Xiaomin Zhang >Priority: Minor > Attachments: HIVE-17975.patch, HIVE-17975.patch > > > This looks affected by HIVE-14207, that a copy of hiveConf is used in the > HttpServer.Builder, while the 'startcode' is only added into the original > hiveConf. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-17975) "HiveServer2 Start Time" changes when page refreshes
[ https://issues.apache.org/jira/browse/HIVE-17975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-17975 started by Xiaomin Zhang. > "HiveServer2 Start Time" changes when page refreshes > > > Key: HIVE-17975 > URL: https://issues.apache.org/jira/browse/HIVE-17975 > Project: Hive > Issue Type: Bug > Components: Web UI >Affects Versions: 2.1.0 >Reporter: Xiaomin Zhang >Assignee: Xiaomin Zhang >Priority: Minor > Attachments: HIVE-17975.patch > > > This looks affected by HIVE-14207, that a copy of hiveConf is used in the > HttpServer.Builder, while the 'startcode' is only added into the original > hiveConf. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-17975) "HiveServer2 Start Time" changes when page refreshes
[ https://issues.apache.org/jira/browse/HIVE-17975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaomin Zhang reassigned HIVE-17975: Assignee: Xiaomin Zhang > "HiveServer2 Start Time" changes when page refreshes > > > Key: HIVE-17975 > URL: https://issues.apache.org/jira/browse/HIVE-17975 > Project: Hive > Issue Type: Bug > Components: Web UI >Affects Versions: 2.1.0 >Reporter: Xiaomin Zhang >Assignee: Xiaomin Zhang >Priority: Minor > Attachments: HIVE-17975.patch > > > This looks affected by HIVE-14207, that a copy of hiveConf is used in the > HttpServer.Builder, while the 'startcode' is only added into the original > hiveConf. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19661) switch Hive UDFs to use Re2J regex engine
[
https://issues.apache.org/jira/browse/HIVE-19661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550257#comment-16550257
]
Hive QA commented on HIVE-19661:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12932257/HIVE-19661.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14679 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12717/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12717/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12717/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12932257 - PreCommit-HIVE-Build
> switch Hive UDFs to use Re2J regex engine
> -
>
> Key: HIVE-19661
> URL: https://issues.apache.org/jira/browse/HIVE-19661
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-19661.01.patch, HIVE-19661.02.patch,
> HIVE-19661.patch
>
>
> Java regex engine can be very slow in some cases e.g.
> https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8203458
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19661) switch Hive UDFs to use Re2J regex engine
[
https://issues.apache.org/jira/browse/HIVE-19661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550256#comment-16550256
]
Hive QA commented on HIVE-19661:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} common in master has 64 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 2281 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} common: The patch generated 3 new + 423 unchanged - 0
fixed = 426 total (was 423) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
27s{color} | {color:red} root: The patch generated 3 new + 457 unchanged - 0
fixed = 460 total (was 457) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 3 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 59m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12717/dev-support/hive-personality.sh
|
| git revision | master / 851c8ab |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12717/yetus/diff-checkstyle-common.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12717/yetus/diff-checkstyle-root.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12717/yetus/whitespace-tabs.txt
|
| modules | C: common . ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12717/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> switch Hive UDFs to use Re2J regex engine
> -
>
> Key: HIVE-19661
> URL: https://issues.apache.org/jira/browse/HIVE-19661
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-19661.01.patch, HIVE-19661.02.patch,
> HIVE-19661.patch
>
>
> Java regex engine can be very slow in
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teddy Choi updated HIVE-20150: -- Issue Type: Improvement (was: New Feature) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Teddy Choi >Priority: Major > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teddy Choi updated HIVE-20150: -- Component/s: Physical Optimizer > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: New Feature > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Teddy Choi >Priority: Major > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teddy Choi updated HIVE-20150: -- Affects Version/s: 4.0.0 Target Version/s: 4.0.0, 3.2.0 > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: New Feature > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Teddy Choi >Priority: Major > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20205) Upgrade HBase dependencies off alpha4 release
[
https://issues.apache.org/jira/browse/HIVE-20205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550219#comment-16550219
]
Hive QA commented on HIVE-20205:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12932252/HIVE-20205.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12716/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12716/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12716/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12932252/HIVE-20205.1.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12932252 - PreCommit-HIVE-Build
> Upgrade HBase dependencies off alpha4 release
> -
>
> Key: HIVE-20205
> URL: https://issues.apache.org/jira/browse/HIVE-20205
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Minor
> Attachments: HIVE-20205.1.patch, HIVE-20205.patch, HIVE-20205.patch
>
>
> Appears Hive has dependencies on hbase 2.0.0-alpha4 releases. HBase 2.0.0 and
> 2.0.1 have been released. HBase team recommends 2.0.1 and says there shouldnt
> be any API surprises. (but we never know)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19829) Incremental replication load should create tasks in execution phase rather than semantic phase
[
https://issues.apache.org/jira/browse/HIVE-19829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550217#comment-16550217
]
Hive QA commented on HIVE-19829:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12932025/HIVE-19829.11-branch-3.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 96 failed/errored test(s), 14406 tests
executed
*Failed tests:*
{noformat}
TestAddPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestAddPartitionsFromPartSpec - did not produce a TEST-*.xml file (likely timed
out) (batchId=228)
TestAdminUser - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestAggregateStatsCache - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestAlterPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestAppendPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=271)
TestCachedStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestCatalogCaching - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestCatalogNonDefaultClient - did not produce a TEST-*.xml file (likely timed
out) (batchId=226)
TestCatalogNonDefaultSvr - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestCatalogOldClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestCatalogs - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestCheckConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=236)
TestDatabases - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDeadline - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestDefaultConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDropPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=271)
TestEmbeddedHiveMetaStore - did not produce a TEST-*.xml file (likely timed
out) (batchId=229)
TestExchangePartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestFMSketchSerialization - did not produce a TEST-*.xml file (likely timed
out) (batchId=236)
TestFilterHooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestForeignKey - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestFunctions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestGetPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestGetTableMeta - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestHLLNoBias - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHLLSerialization - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHdfsUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestHiveAlterHandler - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestHiveMetaStoreGetMetaConf - did not produce a TEST-*.xml file (likely timed
out) (batchId=234)
TestHiveMetaStorePartitionSpecs - did not produce a TEST-*.xml file (likely
timed out) (batchId=228)
TestHiveMetaStoreSchemaMethods - did not produce a TEST-*.xml file (likely
timed out) (batchId=234)
TestHiveMetaStoreTimeout - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHiveMetaStoreTxns - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHiveMetaStoreWithEnvironmentContext - did not produce a TEST-*.xml file
(likely timed out) (batchId=231)
TestHiveMetastoreCli - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestHyperLogLog - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHyperLogLogDense - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHyperLogLogMerge - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHyperLogLogSparse - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestJSONMessageDeserializer - did not produce a TEST-*.xml file (likely timed
out) (batchId=234)
TestListPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestLockRequestBuilder - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestMarkPartition - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestMarkPartitionRemote - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestMetaStoreConnectionUrlHook - did not produce a TEST-*.xml file
[jira] [Updated] (HIVE-20208) order by position alias do not work
[ https://issues.apache.org/jira/browse/HIVE-20208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xulongfetion updated HIVE-20208: Attachment: (was: order_by_position_alias.patch) > order by position alias do not work > --- > > Key: HIVE-20208 > URL: https://issues.apache.org/jira/browse/HIVE-20208 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: xulongfetion >Assignee: Vineet Garg >Priority: Blocker > Labels: patch-available > Attachments: HIVE-orderby-position-alias.patch > > > the order by position alias do not work if there's more than one position > alias. For query like > "select a, b, c from table1 order by 1, 2;", hive will report column not > found error. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20208) order by position alias do not work
[ https://issues.apache.org/jira/browse/HIVE-20208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550189#comment-16550189 ] xulongfetion commented on HIVE-20208: - [~vgarg] I have make a new patch following the doc here: [https://cwiki.apache.org/confluence/display/Hive/HowToContribute.] And I'm working on the master, would you like to try again? > order by position alias do not work > --- > > Key: HIVE-20208 > URL: https://issues.apache.org/jira/browse/HIVE-20208 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: xulongfetion >Assignee: Vineet Garg >Priority: Blocker > Labels: patch-available > Attachments: HIVE-orderby-position-alias.patch > > > the order by position alias do not work if there's more than one position > alias. For query like > "select a, b, c from table1 order by 1, 2;", hive will report column not > found error. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20208) order by position alias do not work
[ https://issues.apache.org/jira/browse/HIVE-20208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xulongfetion updated HIVE-20208: Attachment: HIVE-orderby-position-alias.patch > order by position alias do not work > --- > > Key: HIVE-20208 > URL: https://issues.apache.org/jira/browse/HIVE-20208 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: xulongfetion >Assignee: Vineet Garg >Priority: Blocker > Labels: patch-available > Attachments: HIVE-orderby-position-alias.patch > > > the order by position alias do not work if there's more than one position > alias. For query like > "select a, b, c from table1 order by 1, 2;", hive will report column not > found error. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20193) cboInfo is not present in the explain plan json
[ https://issues.apache.org/jira/browse/HIVE-20193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-20193: --- Resolution: Duplicate Status: Resolved (was: Patch Available) > cboInfo is not present in the explain plan json > --- > > Key: HIVE-20193 > URL: https://issues.apache.org/jira/browse/HIVE-20193 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: anishek >Assignee: anishek >Priority: Major > Attachments: HIVE-20193.1.patch, HIVE-20193.2.patch > > > *cboInfo* attribute is not present in the explain plan json that is provided > to the pre exec hook in hive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20210) Simple Fetch optimizer should lead to MapReduce when filter on non-partition column and conversion is minimal
[ https://issues.apache.org/jira/browse/HIVE-20210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey(Xilang) Yan updated HIVE-20210: --- Status: Patch Available (was: In Progress) > Simple Fetch optimizer should lead to MapReduce when filter on non-partition > column and conversion is minimal > - > > Key: HIVE-20210 > URL: https://issues.apache.org/jira/browse/HIVE-20210 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.3.2, 2.3.1, 2.3.0 >Reporter: Jeffrey(Xilang) Yan >Assignee: Jeffrey(Xilang) Yan >Priority: Major > Attachments: HIVE-20210.2.patch, HIVE-20210.patch > > > When conversion is minimal, simple fetch can be used only when filter on > partition column or no filter at all. But it lead to simple fetch even if > filter on non-partition column. Unit test " select * from srcpart where key > > 100 limit 10 " in the nonmr_fetch.q demonstration this issue – the unit test > is not correct indeed(it should be Map Reduce but in test it is Simple > Fetch). > This issue lead to a serious problem when data size is huge. When conversion > is more and filter on both partition column and non-partition column, it will > not chech hive.fetch.task.conversion.threshold, which lead to the query to > takes hours to finish. This issue doesn't exist in 1.2.1, how it works should > be a magic... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20210) Simple Fetch optimizer should lead to MapReduce when filter on non-partition column and conversion is minimal
[ https://issues.apache.org/jira/browse/HIVE-20210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey(Xilang) Yan updated HIVE-20210: --- Attachment: HIVE-20210.2.patch > Simple Fetch optimizer should lead to MapReduce when filter on non-partition > column and conversion is minimal > - > > Key: HIVE-20210 > URL: https://issues.apache.org/jira/browse/HIVE-20210 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.3.0, 2.3.1, 2.3.2 >Reporter: Jeffrey(Xilang) Yan >Assignee: Jeffrey(Xilang) Yan >Priority: Major > Attachments: HIVE-20210.2.patch, HIVE-20210.patch > > > When conversion is minimal, simple fetch can be used only when filter on > partition column or no filter at all. But it lead to simple fetch even if > filter on non-partition column. Unit test " select * from srcpart where key > > 100 limit 10 " in the nonmr_fetch.q demonstration this issue – the unit test > is not correct indeed(it should be Map Reduce but in test it is Simple > Fetch). > This issue lead to a serious problem when data size is huge. When conversion > is more and filter on both partition column and non-partition column, it will > not chech hive.fetch.task.conversion.threshold, which lead to the query to > takes hours to finish. This issue doesn't exist in 1.2.1, how it works should > be a magic... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20210) Simple Fetch optimizer should lead to MapReduce when filter on non-partition column and conversion is minimal
[ https://issues.apache.org/jira/browse/HIVE-20210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey(Xilang) Yan updated HIVE-20210: --- Status: In Progress (was: Patch Available) > Simple Fetch optimizer should lead to MapReduce when filter on non-partition > column and conversion is minimal > - > > Key: HIVE-20210 > URL: https://issues.apache.org/jira/browse/HIVE-20210 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.3.2, 2.3.1, 2.3.0 >Reporter: Jeffrey(Xilang) Yan >Assignee: Jeffrey(Xilang) Yan >Priority: Major > Attachments: HIVE-20210.2.patch, HIVE-20210.patch > > > When conversion is minimal, simple fetch can be used only when filter on > partition column or no filter at all. But it lead to simple fetch even if > filter on non-partition column. Unit test " select * from srcpart where key > > 100 limit 10 " in the nonmr_fetch.q demonstration this issue – the unit test > is not correct indeed(it should be Map Reduce but in test it is Simple > Fetch). > This issue lead to a serious problem when data size is huge. When conversion > is more and filter on both partition column and non-partition column, it will > not chech hive.fetch.task.conversion.threshold, which lead to the query to > takes hours to finish. This issue doesn't exist in 1.2.1, how it works should > be a magic... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20210) Simple Fetch optimizer should lead to MapReduce when filter on non-partition column and conversion is minimal
[ https://issues.apache.org/jira/browse/HIVE-20210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey(Xilang) Yan updated HIVE-20210: --- Attachment: (was: HIVE-20210.2.patch) > Simple Fetch optimizer should lead to MapReduce when filter on non-partition > column and conversion is minimal > - > > Key: HIVE-20210 > URL: https://issues.apache.org/jira/browse/HIVE-20210 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.3.0, 2.3.1, 2.3.2 >Reporter: Jeffrey(Xilang) Yan >Assignee: Jeffrey(Xilang) Yan >Priority: Major > Attachments: HIVE-20210.2.patch, HIVE-20210.patch > > > When conversion is minimal, simple fetch can be used only when filter on > partition column or no filter at all. But it lead to simple fetch even if > filter on non-partition column. Unit test " select * from srcpart where key > > 100 limit 10 " in the nonmr_fetch.q demonstration this issue – the unit test > is not correct indeed(it should be Map Reduce but in test it is Simple > Fetch). > This issue lead to a serious problem when data size is huge. When conversion > is more and filter on both partition column and non-partition column, it will > not chech hive.fetch.task.conversion.threshold, which lead to the query to > takes hours to finish. This issue doesn't exist in 1.2.1, how it works should > be a magic... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20210) Simple Fetch optimizer should lead to MapReduce when filter on non-partition column and conversion is minimal
[ https://issues.apache.org/jira/browse/HIVE-20210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey(Xilang) Yan updated HIVE-20210: --- Attachment: HIVE-20210.2.patch > Simple Fetch optimizer should lead to MapReduce when filter on non-partition > column and conversion is minimal > - > > Key: HIVE-20210 > URL: https://issues.apache.org/jira/browse/HIVE-20210 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.3.0, 2.3.1, 2.3.2 >Reporter: Jeffrey(Xilang) Yan >Assignee: Jeffrey(Xilang) Yan >Priority: Major > Attachments: HIVE-20210.2.patch, HIVE-20210.patch > > > When conversion is minimal, simple fetch can be used only when filter on > partition column or no filter at all. But it lead to simple fetch even if > filter on non-partition column. Unit test " select * from srcpart where key > > 100 limit 10 " in the nonmr_fetch.q demonstration this issue – the unit test > is not correct indeed(it should be Map Reduce but in test it is Simple > Fetch). > This issue lead to a serious problem when data size is huge. When conversion > is more and filter on both partition column and non-partition column, it will > not chech hive.fetch.task.conversion.threshold, which lead to the query to > takes hours to finish. This issue doesn't exist in 1.2.1, how it works should > be a magic... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20056) SparkPartitionPruner shouldn't be triggered by Spark tasks
[
https://issues.apache.org/jira/browse/HIVE-20056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550173#comment-16550173
]
Rui Li commented on HIVE-20056:
---
Thanks [~stakiar], nice catch. Patch looks good to me.
Nit:
{code}
+if (work instanceof MapWork) {
+ if (HiveConf.isSparkDPPAny(jobConf)) {
{code}
=> {{if (work instanceof MapWork && HiveConf.isSparkDPPAny(jobConf))}}
{code}
if (spec == null) {
-throw new AssertionException("No partition spec found in dynamic
pruning");
+throw new IllegalStateException("No partition spec found in dynamic
pruning");
}
{code}
Just use Preconditions?
> SparkPartitionPruner shouldn't be triggered by Spark tasks
> --
>
> Key: HIVE-20056
> URL: https://issues.apache.org/jira/browse/HIVE-20056
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-20056.1.patch
>
>
> It looks like {{SparkDynamicPartitionPruner}} is being called by every Spark
> task because it gets created whenever {{getRecordReader}} is called on the
> associated {{InputFormat}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19829) Incremental replication load should create tasks in execution phase rather than semantic phase
[
https://issues.apache.org/jira/browse/HIVE-19829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550162#comment-16550162
]
Hive QA commented on HIVE-19829:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 15s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-12715/patches/PreCommit-HIVE-Build-12715.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12715/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Incremental replication load should create tasks in execution phase rather
> than semantic phase
> --
>
> Key: HIVE-19829
> URL: https://issues.apache.org/jira/browse/HIVE-19829
> Project: Hive
> Issue Type: Task
> Components: repl
>Affects Versions: 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19829.01.patch, HIVE-19829.02.patch,
> HIVE-19829.03.patch, HIVE-19829.04.patch, HIVE-19829.06.patch,
> HIVE-19829.07.patch, HIVE-19829.07.patch, HIVE-19829.08-branch-3.patch,
> HIVE-19829.08.patch, HIVE-19829.09.patch, HIVE-19829.10-branch-3.patch,
> HIVE-19829.10.patch, HIVE-19829.11-branch-3.patch
>
>
> Split the incremental load into multiple iterations. In each iteration create
> number of tasks equal to the configured value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20205) Upgrade HBase dependencies off alpha4 release
[
https://issues.apache.org/jira/browse/HIVE-20205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550152#comment-16550152
]
Hive QA commented on HIVE-20205:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12932252/HIVE-20205.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 14648 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestHBaseCliDriver.org.apache.hadoop.hive.cli.TestHBaseCliDriver
(batchId=100)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.org.apache.hadoop.hive.cli.TestHBaseCliDriver
(batchId=101)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.org.apache.hadoop.hive.cli.TestHBaseCliDriver
(batchId=102)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.org.apache.hadoop.hive.cli.TestHBaseCliDriver
(batchId=103)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.org.apache.hadoop.hive.cli.TestHBaseCliDriver
(batchId=104)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.org.apache.hadoop.hive.cli.TestHBaseCliDriver
(batchId=105)
org.apache.hadoop.hive.cli.TestHBaseNegativeCliDriver.org.apache.hadoop.hive.cli.TestHBaseNegativeCliDriver
(batchId=260)
org.apache.hadoop.hive.ql.exec.tez.TestDynamicPartitionPruner.testSingleSourceMultipleFiltersOrdering1
(batchId=304)
org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler
(batchId=256)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighShuffleBytes
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryExecutionTime
(batchId=250)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12714/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12714/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12714/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12932252 - PreCommit-HIVE-Build
> Upgrade HBase dependencies off alpha4 release
> -
>
> Key: HIVE-20205
> URL: https://issues.apache.org/jira/browse/HIVE-20205
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Minor
> Attachments: HIVE-20205.1.patch, HIVE-20205.patch, HIVE-20205.patch
>
>
> Appears Hive has dependencies on hbase 2.0.0-alpha4 releases. HBase 2.0.0 and
> 2.0.1 have been released. HBase team recommends 2.0.1 and says there shouldnt
> be any API surprises. (but we never know)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19915) master-txnstats branch - support CachedStore
[ https://issues.apache.org/jira/browse/HIVE-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19915: Status: Patch Available (was: Open) Added support by changing ObjectStore to return the final object/params after modifications. The get methods then verify stats. get_aggr is not supported, it makes calls to both rawStore and cached store on the default path which makes it impossible to make consistent. > master-txnstats branch - support CachedStore > > > Key: HIVE-19915 > URL: https://issues.apache.org/jira/browse/HIVE-19915 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19915.patch, branch-19915.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19915) master-txnstats branch - support CachedStore
[ https://issues.apache.org/jira/browse/HIVE-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550150#comment-16550150 ] Sergey Shelukhin commented on HIVE-19915: - cc [~gopalv] > master-txnstats branch - support CachedStore > > > Key: HIVE-19915 > URL: https://issues.apache.org/jira/browse/HIVE-19915 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19915.patch, branch-19915.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20131) SQL Script changes for creating txn write notification in 3.2.0 files
[ https://issues.apache.org/jira/browse/HIVE-20131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550146#comment-16550146 ] Vineet Garg commented on HIVE-20131: bq. So for branch-3 we will have a file for 3.2 schema and a upgrade file for 3.1to3.2 ? That is correct. > SQL Script changes for creating txn write notification in 3.2.0 files > --- > > Key: HIVE-20131 > URL: https://issues.apache.org/jira/browse/HIVE-20131 > Project: Hive > Issue Type: Sub-task > Components: repl, Transactions >Affects Versions: 3.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: ACID, DR, pull-request-available, replication > Fix For: 4.0.0 > > Attachments: HIVE-20131.01.patch > > > 1. Change partition name size from 1024 to 767 . (mySQL 5.6 and before that > supports max 767 length keys) > 2. Remove the create txn_write_notification_log table creation from 3.1.0 > scripts and add a new scripts for 3.2.0 > 3. Remove the file 3.1.0-to-4.0.0 and instead add file for 3.2.0-to-4.0.0 and > 3.1.0-to-3.2.0 > 4. Change in metastore init schema xml file to take 4.0.0 instead of 3.1.0 > as current version. > h1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19915) master-txnstats branch - support CachedStore
[ https://issues.apache.org/jira/browse/HIVE-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19915: Attachment: HIVE-19915.patch > master-txnstats branch - support CachedStore > > > Key: HIVE-19915 > URL: https://issues.apache.org/jira/browse/HIVE-19915 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19915.patch, branch-19915.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19915) master-txnstats branch - support CachedStore
[ https://issues.apache.org/jira/browse/HIVE-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19915: Attachment: branch-19915.patch > master-txnstats branch - support CachedStore > > > Key: HIVE-19915 > URL: https://issues.apache.org/jira/browse/HIVE-19915 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: branch-19915.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20205) Upgrade HBase dependencies off alpha4 release
[
https://issues.apache.org/jira/browse/HIVE-20205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550140#comment-16550140
]
Hive QA commented on HIVE-20205:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 41m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12714/dev-support/hive-personality.sh
|
| git revision | master / 851c8ab |
| Default Java | 1.8.0_111 |
| modules | C: . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12714/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Upgrade HBase dependencies off alpha4 release
> -
>
> Key: HIVE-20205
> URL: https://issues.apache.org/jira/browse/HIVE-20205
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Minor
> Attachments: HIVE-20205.1.patch, HIVE-20205.patch, HIVE-20205.patch
>
>
> Appears Hive has dependencies on hbase 2.0.0-alpha4 releases. HBase 2.0.0 and
> 2.0.1 have been released. HBase team recommends 2.0.1 and says there shouldnt
> be any API surprises. (but we never know)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20131) SQL Script changes for creating txn write notification in 3.2.0 files
[ https://issues.apache.org/jira/browse/HIVE-20131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550138#comment-16550138 ] mahesh kumar behera commented on HIVE-20131: Thanks [~vgarg] for review ..for branch-3 i will make sure the sql script changes in HIVE-19267 are going in to appropriate files. So for branch-3 we will have a file for 3.2 schema and a upgrade file for 3.1to3.2 ? [~sankarh] . can you please commit this patch > SQL Script changes for creating txn write notification in 3.2.0 files > --- > > Key: HIVE-20131 > URL: https://issues.apache.org/jira/browse/HIVE-20131 > Project: Hive > Issue Type: Sub-task > Components: repl, Transactions >Affects Versions: 3.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: ACID, DR, pull-request-available, replication > Fix For: 4.0.0 > > Attachments: HIVE-20131.01.patch > > > 1. Change partition name size from 1024 to 767 . (mySQL 5.6 and before that > supports max 767 length keys) > 2. Remove the create txn_write_notification_log table creation from 3.1.0 > scripts and add a new scripts for 3.2.0 > 3. Remove the file 3.1.0-to-4.0.0 and instead add file for 3.2.0-to-4.0.0 and > 3.1.0-to-3.2.0 > 4. Change in metastore init schema xml file to take 4.0.0 instead of 3.1.0 > as current version. > h1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20131) SQL Script changes for creating txn write notification in 3.2.0 files
[ https://issues.apache.org/jira/browse/HIVE-20131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550112#comment-16550112 ] Vineet Garg commented on HIVE-20131: Thanks [~maheshk114]. +1. Can you upload a patch for branch-3 as well? > SQL Script changes for creating txn write notification in 3.2.0 files > --- > > Key: HIVE-20131 > URL: https://issues.apache.org/jira/browse/HIVE-20131 > Project: Hive > Issue Type: Sub-task > Components: repl, Transactions >Affects Versions: 3.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: ACID, DR, pull-request-available, replication > Fix For: 4.0.0 > > Attachments: HIVE-20131.01.patch > > > 1. Change partition name size from 1024 to 767 . (mySQL 5.6 and before that > supports max 767 length keys) > 2. Remove the create txn_write_notification_log table creation from 3.1.0 > scripts and add a new scripts for 3.2.0 > 3. Remove the file 3.1.0-to-4.0.0 and instead add file for 3.2.0-to-4.0.0 and > 3.1.0-to-3.2.0 > 4. Change in metastore init schema xml file to take 4.0.0 instead of 3.1.0 > as current version. > h1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[
https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550105#comment-16550105
]
Hive QA commented on HIVE-19924:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12932219/HIVE-19924.04.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12713/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12713/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12713/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12932219/HIVE-19924.04.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12932219 - PreCommit-HIVE-Build
> Tag distcp jobs run by Repl Load
>
>
> Key: HIVE-19924
> URL: https://issues.apache.org/jira/browse/HIVE-19924
> Project: Hive
> Issue Type: Task
> Components: repl
>Affects Versions: 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: DR, replication
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch,
> HIVE-19924.03.patch, HIVE-19924.04.patch
>
>
> Add tags in jobconf for distcp related jobs started by replication. This will
> allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon
> issues a kill command.
> * one of the tags should definitely be the query_id that starts the job :
> With this flow beacon before retrying the bootstrap load, will issue a kill
> command to hs2 with the query id of the previous issued command. hs2 will
> then kill an running jobs on yarn tagged with the Query_id.
> * To get around the additional failure point as mentioned above. The jobs
> can be tagged with an additional unique tag_id provided by Beacon in the WITH
> clause in repl load command to be used to tag distcp jobs ). Enhance the kill
> api to take the tag as input and kill jobs associated with that tag. Problem
> here is how do we validate the association of the tag with a hive query id to
> make sure this api is not used to kill jobs run by other components, however
> we can provide this capability to only admins and should be ok in that case.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[
https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550103#comment-16550103
]
Hive QA commented on HIVE-19924:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12932219/HIVE-19924.04.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 14648 tests
executed
*Failed tests:*
{noformat}
TestJdbcWithMiniHS2 - did not produce a TEST-*.xml file (likely timed out)
(batchId=250)
TestJdbcWithMiniHS2ErasureCoding - did not produce a TEST-*.xml file (likely
timed out) (batchId=250)
TestNoSaslAuth - did not produce a TEST-*.xml file (likely timed out)
(batchId=250)
org.apache.hive.jdbc.TestJdbcDriver2.testGetQueryId (batchId=249)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveAndKill
(batchId=250)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveBackKill
(batchId=250)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill
(batchId=250)
org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerDAGTotalTasks
(batchId=247)
org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerSlowQueryExecutionTime
(batchId=247)
org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerTotalLaunchedTasks
(batchId=247)
org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerVertexTotalTasks
(batchId=247)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testMultipleTriggers1
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testMultipleTriggers2
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedFiles
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomReadOps
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDagRawInputSplitsKill
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDagTotalTasks
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDefaultRawInputSplits
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighBytesRead
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighBytesWrite
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighShuffleBytes
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerShortQueryElapsedTime
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryElapsedTime
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryExecutionTime
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerTotalTasks
(batchId=250)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerVertexRawInputSplitsKill
(batchId=250)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12712/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12712/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12712/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 29 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12932219 - PreCommit-HIVE-Build
> Tag distcp jobs run by Repl Load
>
>
> Key: HIVE-19924
> URL: https://issues.apache.org/jira/browse/HIVE-19924
> Project: Hive
> Issue Type: Task
> Components: repl
>Affects Versions: 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: DR, replication
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch,
> HIVE-19924.03.patch, HIVE-19924.04.patch
>
>
> Add tags in jobconf for distcp related jobs started by replication. This will
> allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon
> issues a kill command.
> * one of the tags should definitely be the query_id that starts the job :
> With this flow beacon before retrying the
[jira] [Commented] (HIVE-18513) Query results caching
[ https://issues.apache.org/jira/browse/HIVE-18513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550096#comment-16550096 ] Thai Bui commented on HIVE-18513: - [~lars_francke] I have not work on disabling this limitation on cache and external table, but it's still on my agenda for one of the most important features I wanted to implement for Hive. Ideally, I really want to blur the boundary of managed vs. external table. I think most of the features that only work with managed tables should work with external tables as well. By default, they can be turned off but with an option to be turned on. [~jdere] Should we consider opening up the cache for external table? If yes, I can create a ticket and find sometime to work on it. Since you originally design this feature, we would like to hear your opinions on this as well. Would enabling this feature badly break a lot of assumptions in Hive that it's untenable to be implemented in Hive? > Query results caching > - > > Key: HIVE-18513 > URL: https://issues.apache.org/jira/browse/HIVE-18513 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18513.1.patch, HIVE-18513.2.patch, > HIVE-18513.3.patch, HIVE-18513.4.patch, HIVE-18513.5.patch, HIVE-18513.6.patch > > > Add a query results cache that can save the results of an executed Hive query > for reuse on subsequent queries. This may be useful in cases where the same > query is issued many times, since Hive can return back the results of a > cached query rather than having to execute the full query on the cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[
https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550085#comment-16550085
]
Hive QA commented on HIVE-19924:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
5s{color} | {color:blue} ql in master has 2281 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} service in master has 48 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
42s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 1 new + 55 unchanged - 12
fixed = 56 total (was 67) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} service: The patch generated 2 new + 123 unchanged - 0
fixed = 125 total (was 123) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
49s{color} | {color:red} service generated 1 new + 48 unchanged - 0 fixed = 49
total (was 48) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 9s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:service |
| | Redundant nullcheck of operation, which is known to be non-null in
org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationHandle)
Redundant null check at OperationManager.java:is known to be non-null in
org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationHandle)
Redundant null check at OperationManager.java:[line 307] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12712/dev-support/hive-personality.sh
|
| git revision | master / 851c8ab |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12712/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12712/yetus/diff-checkstyle-service.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12712/yetus/new-findbugs-service.html
|
| modules | C: itests/hive-unit ql service U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12712/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Tag distcp jobs run by
[jira] [Commented] (HIVE-19441) Use LLAP test Driver for Druid test suites
[
https://issues.apache.org/jira/browse/HIVE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550067#comment-16550067
]
Hive QA commented on HIVE-19441:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12932286/HIVE-19441.13.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 14679 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=190)
[druid_basic2.q,druidmini_mv.q,druid_timestamptz2.q,druidmini_extractTime.q,druidmini_floorTime.q]
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=191)
[druidmini_dynamic_partition.q,druidmini_expressions.q,druidmini_test_alter.q,druidmini_test1.q,druidmini_test_insert.q]
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=192)
[druidmini_masking.q,druidmini_joins.q,druid_timestamptz.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12711/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12711/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12711/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12932286 - PreCommit-HIVE-Build
> Use LLAP test Driver for Druid test suites
> --
>
> Key: HIVE-19441
> URL: https://issues.apache.org/jira/browse/HIVE-19441
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19441.10.patch, HIVE-19441.11.patch,
> HIVE-19441.12.patch, HIVE-19441.13.patch, HIVE-19441.2.patch,
> HIVE-19441.3.patch, HIVE-19441.4.patch, HIVE-19441.5.patch,
> HIVE-19441.6.patch, HIVE-19441.7.patch, HIVE-19441.8.patch,
> HIVE-19441.8.patch, HIVE-19441.9.patch, HIVE-19441.patch
>
>
> Adding support to the float kind aggregator.
> Use LLAP as test Driver to reduce execution time of tests from about 2 hours
> to 15 min:
> Although this patches unveiling an issue with timezone, maybe it is fixed by
> [~jcamachorodriguez] upcoming set of patches.
>
> Before
> {code}
> [INFO] Executed tasks
> [INFO]
> [INFO] --- maven-compiler-plugin:3.6.1:testCompile (default-testCompile) @
> hive-it-qfile ---
> [INFO] Compiling 21 source files to
> /Users/sbouguerra/Hdev/hive/itests/qtest/target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ hive-it-qfile
> ---
> [INFO]
> [INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO] Running org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 6,654.117 s - in org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
> [INFO] Total time: 01:51 h
> [INFO] Finished at: 2018-05-04T12:43:19-07:00
> [INFO]
>
> {code}
> After
> {code}
> INFO] Executed tasks
> [INFO]
> [INFO] --- maven-compiler-plugin:3.6.1:testCompile (default-testCompile) @
> hive-it-qfile ---
> [INFO] Compiling 22 source files to
> /Users/sbouguerra/Hdev/hive/itests/qtest/target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ hive-it-qfile
> ---
> [INFO]
> [INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO] Running org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 907.167 s - in org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
[jira] [Assigned] (HIVE-19915) master-txnstats branch - support CachedStore
[ https://issues.apache.org/jira/browse/HIVE-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-19915: --- Assignee: Sergey Shelukhin (was: Steve Yeom) > master-txnstats branch - support CachedStore > > > Key: HIVE-19915 > URL: https://issues.apache.org/jira/browse/HIVE-19915 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20208) order by position alias do not work
[ https://issues.apache.org/jira/browse/HIVE-20208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550053#comment-16550053 ] Vineet Garg commented on HIVE-20208: [~xulongfetion] I am unable to reproduce this on master. Can you provide full reproducer? What version are you using? > order by position alias do not work > --- > > Key: HIVE-20208 > URL: https://issues.apache.org/jira/browse/HIVE-20208 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: xulongfetion >Assignee: Vineet Garg >Priority: Blocker > Labels: patch-available > Attachments: order_by_position_alias.patch > > > the order by position alias do not work if there's more than one position > alias. For query like > "select a, b, c from table1 order by 1, 2;", hive will report column not > found error. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19441) Use LLAP test Driver for Druid test suites
[
https://issues.apache.org/jira/browse/HIVE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550043#comment-16550043
]
Hive QA commented on HIVE-19441:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
51s{color} | {color:blue} itests/util in master has 52 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
16s{color} | {color:blue} ql in master has 2281 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
22s{color} | {color:blue} testutils/ptest2 in master has 24 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
33s{color} | {color:red} qtest in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} itests/qtest: The patch generated 5 new + 3 unchanged
- 0 fixed = 8 total (was 3) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} itests/util: The patch generated 2 new + 90 unchanged
- 0 fixed = 92 total (was 90) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} ql: The patch generated 0 new + 11 unchanged - 1
fixed = 11 total (was 12) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
7s{color} | {color:green} The patch ptest2 passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12711/dev-support/hive-personality.sh
|
| git revision | master / 851c8ab |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12711/yetus/patch-mvninstall-itests_qtest.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12711/yetus/diff-checkstyle-itests_qtest.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12711/yetus/diff-checkstyle-itests_util.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12711/yetus/patch-asflicense-problems.txt
|
| modules | C: itests itests/qtest itests/util ql testutils/ptest2 U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12711/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Use
[jira] [Updated] (HIVE-20215) Hive unable to plan/compile query containing subquery with multiple same name columns
[
https://issues.apache.org/jira/browse/HIVE-20215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-20215:
---
Summary: Hive unable to plan/compile query containing subquery with
multiple same name columns (was: Hive unable to plan/compile query)
> Hive unable to plan/compile query containing subquery with multiple same name
> columns
> -
>
> Key: HIVE-20215
> URL: https://issues.apache.org/jira/browse/HIVE-20215
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
>
> *Reproducer*
> ==
> {code:sql}
> >create table t1(c1 int)
> >explain select count(*) from (select c1, c1 from t1) subq
> {code}
> {noformat}
> FAILED: SemanticException [Error 10007]: Ambiguous column reference c1 in subq
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20046) [phase 2] remove NUM_FILES check or add a negative test
[
https://issues.apache.org/jira/browse/HIVE-20046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-20046:
Status: Patch Available (was: Open)
> [phase 2] remove NUM_FILES check or add a negative test
> ---
>
> Key: HIVE-20046
> URL: https://issues.apache.org/jira/browse/HIVE-20046
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-20046.patch
>
>
> {noformat}
> // Since newly initialized empty table has 0 for the parameter.
> if (Long.parseLong(statsParams.get(StatsSetupConst.NUM_FILES)) == 0) {
> return true;
> }
> {noformat}
> This doesn't look safe; # of files could be set to 0 by an invalid update, or
> potentially a parallel update that we cannot see (not sure if this is
> possible; there's some code in metastore that updates basic stats outside of
> the scope of the query).
> It would be better to remove this, and see if it breaks some tests. If we do
> need this, there should be a negative test at some point
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20046) [phase 2] remove NUM_FILES check or add a negative test
[
https://issues.apache.org/jira/browse/HIVE-20046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550017#comment-16550017
]
Sergey Shelukhin commented on HIVE-20046:
-
Removed the check. Some local tests I've run locally pass just fine.
> [phase 2] remove NUM_FILES check or add a negative test
> ---
>
> Key: HIVE-20046
> URL: https://issues.apache.org/jira/browse/HIVE-20046
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-20046.patch
>
>
> {noformat}
> // Since newly initialized empty table has 0 for the parameter.
> if (Long.parseLong(statsParams.get(StatsSetupConst.NUM_FILES)) == 0) {
> return true;
> }
> {noformat}
> This doesn't look safe; # of files could be set to 0 by an invalid update, or
> potentially a parallel update that we cannot see (not sure if this is
> possible; there's some code in metastore that updates basic stats outside of
> the scope of the query).
> It would be better to remove this, and see if it breaks some tests. If we do
> need this, there should be a negative test at some point
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20046) [phase 2] remove NUM_FILES check or add a negative test
[
https://issues.apache.org/jira/browse/HIVE-20046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-20046:
Attachment: HIVE-20046.patch
> [phase 2] remove NUM_FILES check or add a negative test
> ---
>
> Key: HIVE-20046
> URL: https://issues.apache.org/jira/browse/HIVE-20046
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-20046.patch
>
>
> {noformat}
> // Since newly initialized empty table has 0 for the parameter.
> if (Long.parseLong(statsParams.get(StatsSetupConst.NUM_FILES)) == 0) {
> return true;
> }
> {noformat}
> This doesn't look safe; # of files could be set to 0 by an invalid update, or
> potentially a parallel update that we cannot see (not sure if this is
> possible; there's some code in metastore that updates basic stats outside of
> the scope of the query).
> It would be better to remove this, and see if it breaks some tests. If we do
> need this, there should be a negative test at some point
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20215) Hive unable to plan/compile query
[
https://issues.apache.org/jira/browse/HIVE-20215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550009#comment-16550009
]
Vineet Garg commented on HIVE-20215:
Also query such as {code:sql}explain select t.c1 from (select t11.c1, t12.c1
from t1 as t11 inner join t1 as t12 on t11.c1=t12.c1) as t;{code}
compile successfully but it should throw an error since reference to t.c1 is
ambiguous.
> Hive unable to plan/compile query
> -
>
> Key: HIVE-20215
> URL: https://issues.apache.org/jira/browse/HIVE-20215
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
>
> *Reproducer*
> ==
> {code:sql}
> >create table t1(c1 int)
> >explain select count(*) from (select c1, c1 from t1) subq
> {code}
> {noformat}
> FAILED: SemanticException [Error 10007]: Ambiguous column reference c1 in subq
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20210) Simple Fetch optimizer should lead to MapReduce when filter on non-partition column and conversion is minimal
[
https://issues.apache.org/jira/browse/HIVE-20210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550003#comment-16550003
]
Hive QA commented on HIVE-20210:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12932207/HIVE-20210.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12710/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12710/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12710/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-07-19 23:20:45.071
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-12710/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-07-19 23:20:45.073
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
651e795..cdb32a7 master-txnstats -> origin/master-txnstats
+ git reset --hard HEAD
HEAD is now at 851c8ab HIVE-17896: TopNKey: Create a standalone vectorizable
TopNKey operator (Teddy Choi, reviewed by Jesus Camacho Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 851c8ab HIVE-17896: TopNKey: Create a standalone vectorizable
TopNKey operator (Teddy Choi, reviewed by Jesus Camacho Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-07-19 23:20:46.684
+ rm -rf ../yetus_PreCommit-HIVE-Build-12710
+ mkdir ../yetus_PreCommit-HIVE-Build-12710
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-12710
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-12710/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed: ql/src/test/results/clientpositive/nonmr_fetch.q.out:126
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/nonmr_fetch.q.out' with
conflicts.
Going to apply patch with: git apply -p0
error: patch failed: ql/src/test/results/clientpositive/nonmr_fetch.q.out:126
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/nonmr_fetch.q.out' with
conflicts.
U ql/src/test/results/clientpositive/nonmr_fetch.q.out
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-12710
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12932207 - PreCommit-HIVE-Build
> Simple Fetch optimizer should lead to MapReduce when filter on non-partition
> column and conversion is minimal
> -
>
> Key: HIVE-20210
> URL: https://issues.apache.org/jira/browse/HIVE-20210
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 2.3.0, 2.3.1, 2.3.2
>Reporter: Jeffrey(Xilang) Yan
>Assignee: Jeffrey(Xilang) Yan
>Priority: Major
> Attachments: HIVE-20210.patch
>
>
> When conversion is minimal, simple fetch can be used only when filter on
> partition column or no filter at all. But it lead to simple fetch even if
> filter on non-partition column. Unit test " select * from srcpart where key >
> 100 limit 10 " in the nonmr_fetch.q demonstration this issue – the unit test
> is not correct indeed(it should be Map Reduce but in test it is Simple
> Fetch).
> This issue lead to a serious problem when data size is huge. When conversion
> is more and
[jira] [Assigned] (HIVE-20215) Hive unable to plan/compile query
[
https://issues.apache.org/jira/browse/HIVE-20215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reassigned HIVE-20215:
--
> Hive unable to plan/compile query
> -
>
> Key: HIVE-20215
> URL: https://issues.apache.org/jira/browse/HIVE-20215
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
>
> *Reproducer*
> ==
> {code:sql}
> >create table t1(c1 int)
> >explain select count(*) from (select c1, c1 from t1) subq
> {code}
> {noformat}
> FAILED: SemanticException [Error 10007]: Ambiguous column reference c1 in subq
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20046) [phase 2] remove NUM_FILES check or add a negative test
[
https://issues.apache.org/jira/browse/HIVE-20046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin reassigned HIVE-20046:
---
Assignee: Sergey Shelukhin
> [phase 2] remove NUM_FILES check or add a negative test
> ---
>
> Key: HIVE-20046
> URL: https://issues.apache.org/jira/browse/HIVE-20046
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
>
> {noformat}
> // Since newly initialized empty table has 0 for the parameter.
> if (Long.parseLong(statsParams.get(StatsSetupConst.NUM_FILES)) == 0) {
> return true;
> }
> {noformat}
> This doesn't look safe; # of files could be set to 0 by an invalid update, or
> potentially a parallel update that we cannot see (not sure if this is
> possible; there's some code in metastore that updates basic stats outside of
> the scope of the query).
> It would be better to remove this, and see if it breaks some tests. If we do
> need this, there should be a negative test at some point
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20202) Add profiler endpoint to HS2 and LLAP
[
https://issues.apache.org/jira/browse/HIVE-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550002#comment-16550002
]
Hive QA commented on HIVE-20202:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12932302/HIVE-20202.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14679 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12708/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12708/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12708/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12932302 - PreCommit-HIVE-Build
> Add profiler endpoint to HS2 and LLAP
> -
>
> Key: HIVE-20202
> URL: https://issues.apache.org/jira/browse/HIVE-20202
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20202.1.patch, HIVE-20202.2.patch,
> HIVE-20202.3.patch, HIVE-20202.4.patch, HIVE-20202.5.patch
>
>
> Add a web endpoint for profiling based on async-profiler. This servlet should
> be added to httpserver so that HS2 and LLAP daemons can output flamegraphs
> when their /prof endpoint is hit. Since this will be based on
> [https://github.com/jvm-profiling-tools/async-profiler] heap allocation, lock
> contentions, HW counters etc. will also be supported in addition to cpu
> profiling. In most cases the profiling overhead is pretty low and is safe to
> run on production. More analysis on CPU and memory overhead here
> [https://github.com/jvm-profiling-tools/async-profiler/issues/14] and
> [https://github.com/jvm-profiling-tools/async-profiler/issues/131]
>
> For the impatient, here is the usage doc and the sample output
> [https://github.com/prasanthj/nightswatch/blob/master/README.md]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19770) Support for CBO for queries with multiple same columns in select
[
https://issues.apache.org/jira/browse/HIVE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19770:
---
Status: Patch Available (was: Open)
> Support for CBO for queries with multiple same columns in select
>
>
> Key: HIVE-19770
> URL: https://issues.apache.org/jira/browse/HIVE-19770
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-19770.1.patch, HIVE-19770.2.patch,
> HIVE-19770.3.patch, HIVE-19770.4.patch, HIVE-19770.5.patch
>
>
> Currently queries such as {code:sql} select a,a from t1 where b > 10 {code}
> are not supported for CBO.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19770) Support for CBO for queries with multiple same columns in select
[
https://issues.apache.org/jira/browse/HIVE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19770:
---
Attachment: HIVE-19770.5.patch
> Support for CBO for queries with multiple same columns in select
>
>
> Key: HIVE-19770
> URL: https://issues.apache.org/jira/browse/HIVE-19770
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-19770.1.patch, HIVE-19770.2.patch,
> HIVE-19770.3.patch, HIVE-19770.4.patch, HIVE-19770.5.patch
>
>
> Currently queries such as {code:sql} select a,a from t1 where b > 10 {code}
> are not supported for CBO.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19770) Support for CBO for queries with multiple same columns in select
[
https://issues.apache.org/jira/browse/HIVE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19770:
---
Status: Open (was: Patch Available)
> Support for CBO for queries with multiple same columns in select
>
>
> Key: HIVE-19770
> URL: https://issues.apache.org/jira/browse/HIVE-19770
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-19770.1.patch, HIVE-19770.2.patch,
> HIVE-19770.3.patch, HIVE-19770.4.patch
>
>
> Currently queries such as {code:sql} select a,a from t1 where b > 10 {code}
> are not supported for CBO.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20169) Print Final Rows Processed in MapOperator
[
https://issues.apache.org/jira/browse/HIVE-20169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549980#comment-16549980
]
Bharathkrishna Guruvayoor Murali commented on HIVE-20169:
-
I can take a look at this as I am working on HIVE-19766 which is related to
counters.
Assigning to myself.
> Print Final Rows Processed in MapOperator
> -
>
> Key: HIVE-20169
> URL: https://issues.apache.org/jira/browse/HIVE-20169
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Priority: Minor
> Labels: newbie, noob
>
> https://github.com/apache/hive/blob/ac6b2a3fb195916e22b2e5f465add2ffbcdc7430/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java#L573-L582
> This class emits a log message every time it a certain number of records are
> processed, but it does not print a final count.
> Overload the {{MapOperator}} class's {{closeOp}} method to print a final log
> message providing the total number of rows read by this mapper.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20047) remove txnID argument for txn stats methods
[ https://issues.apache.org/jira/browse/HIVE-20047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-20047: Resolution: Fixed Fix Version/s: txnstats Status: Resolved (was: Patch Available) Rebased on top of recent bug fixes, fixed one test that was indeed broken by this patch, and committed to branch. > remove txnID argument for txn stats methods > --- > > Key: HIVE-20047 > URL: https://issues.apache.org/jira/browse/HIVE-20047 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: txnstats > > Attachments: HIVE-20047.patch > > > Followup from HIVE-19975. > W.r.t. write IDs and txn IDs, stats validity check currently verifies one of > two things - that stats write ID is valid for query write ID list, or that > stats txn ID (derived from write ID) is the same as the query txn ID. > I'm not sure the latter check is needed; removing it would allow us to make a > bunch of APIs a little bit simpler. > [~ekoifman] do you have any feedback? Can any stats reader (e.g. compile) > observe stats written by the same txn; but in such manner that it doesn't > have the write ID of the same-txn stats writer, in its valid write ID list? > I'm assuming it's not possible, e.g. in multi statement txn each query would > have the previous same-txn writer for the same table in its valid write ID > list? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20169) Print Final Rows Processed in MapOperator
[
https://issues.apache.org/jira/browse/HIVE-20169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharathkrishna Guruvayoor Murali reassigned HIVE-20169:
---
Assignee: Bharathkrishna Guruvayoor Murali
> Print Final Rows Processed in MapOperator
> -
>
> Key: HIVE-20169
> URL: https://issues.apache.org/jira/browse/HIVE-20169
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Minor
> Labels: newbie, noob
>
> https://github.com/apache/hive/blob/ac6b2a3fb195916e22b2e5f465add2ffbcdc7430/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java#L573-L582
> This class emits a log message every time it a certain number of records are
> processed, but it does not print a final count.
> Overload the {{MapOperator}} class's {{closeOp}} method to print a final log
> message providing the total number of rows read by this mapper.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19766) Show the number of rows inserted when execution engine is Spark
[ https://issues.apache.org/jira/browse/HIVE-19766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-19766: Status: Patch Available (was: Open) > Show the number of rows inserted when execution engine is Spark > --- > > Key: HIVE-19766 > URL: https://issues.apache.org/jira/browse/HIVE-19766 > Project: Hive > Issue Type: Sub-task >Reporter: Bharathkrishna Guruvayoor Murali >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-19766.1.patch > > > Currently when insert query is run, the beeline output shows No rows affected. > The logic to show the number of rows inserted is now present when execution > engine is MR. > This Jira is to make this work with Spark. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19766) Show the number of rows inserted when execution engine is Spark
[ https://issues.apache.org/jira/browse/HIVE-19766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-19766: Attachment: HIVE-19766.1.patch > Show the number of rows inserted when execution engine is Spark > --- > > Key: HIVE-19766 > URL: https://issues.apache.org/jira/browse/HIVE-19766 > Project: Hive > Issue Type: Sub-task >Reporter: Bharathkrishna Guruvayoor Murali >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-19766.1.patch > > > Currently when insert query is run, the beeline output shows No rows affected. > The logic to show the number of rows inserted is now present when execution > engine is MR. > This Jira is to make this work with Spark. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20047) remove txnID argument for txn stats methods
[ https://issues.apache.org/jira/browse/HIVE-20047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-20047: Summary: remove txnID argument for txn stats methods (was: [phase 1.5] consider removing txnID argument for txn stats methods) > remove txnID argument for txn stats methods > --- > > Key: HIVE-20047 > URL: https://issues.apache.org/jira/browse/HIVE-20047 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20047.patch > > > Followup from HIVE-19975. > W.r.t. write IDs and txn IDs, stats validity check currently verifies one of > two things - that stats write ID is valid for query write ID list, or that > stats txn ID (derived from write ID) is the same as the query txn ID. > I'm not sure the latter check is needed; removing it would allow us to make a > bunch of APIs a little bit simpler. > [~ekoifman] do you have any feedback? Can any stats reader (e.g. compile) > observe stats written by the same txn; but in such manner that it doesn't > have the write ID of the same-txn stats writer, in its valid write ID list? > I'm assuming it's not possible, e.g. in multi statement txn each query would > have the previous same-txn writer for the same table in its valid write ID > list? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20202) Add profiler endpoint to HS2 and LLAP
[
https://issues.apache.org/jira/browse/HIVE-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549968#comment-16549968
]
Hive QA commented on HIVE-20202:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} common in master has 64 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} common: The patch generated 10 new + 19 unchanged - 0
fixed = 29 total (was 19) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
25s{color} | {color:red} root: The patch generated 10 new + 19 unchanged - 0
fixed = 29 total (was 19) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
43s{color} | {color:red} common generated 1 new + 64 unchanged - 0 fixed = 65
total (was 64) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 29s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:common |
| | Class org.apache.hive.http.ProfileServlet defines non-transient
non-serializable instance field profProcess In ProfileServlet.java:instance
field profProcess In ProfileServlet.java |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12708/dev-support/hive-personality.sh
|
| git revision | master / 851c8ab |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12708/yetus/diff-checkstyle-common.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12708/yetus/diff-checkstyle-root.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12708/yetus/new-findbugs-common.html
|
| modules | C: common . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12708/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add profiler endpoint to HS2 and LLAP
> -
>
> Key: HIVE-20202
> URL: https://issues.apache.org/jira/browse/HIVE-20202
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20202.1.patch, HIVE-20202.2.patch,
> HIVE-20202.3.patch, HIVE-20202.4.patch, HIVE-20202.5.patch
>
>
> Add a web endpoint for profiling
[jira] [Commented] (HIVE-20192) HS2 with embedded metastore is leaking JDOPersistenceManager objects.
[
https://issues.apache.org/jira/browse/HIVE-20192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549956#comment-16549956
]
Vihang Karajgaonkar commented on HIVE-20192:
I took a look at the patch it is unclear to me why the leak is happening. I see
that in the initializeHelper method if there is a exception you are issuing a
shutdown on the ObjectStore to clean up persistenceManager but shouldn't this
uncaught exception cause the thread to be closed in the first place and thereby
cleaning up the threadlocal rawstore anyways? Is it better to issue a shutdown
on the threadlocal rawstore from {{ThriftBinaryCLIService#deleteContext}}
method instead? Based on my understanding it looks like we are trying to keep
track of the threadlocal rawstore using custom implementation of Thread in a
map and depend on finalize method to do cleanup. This in theory means that
cleanup is only happening when the threads are GCed instead of doing it as soon
as when sessions are closed. Also, if a thrift thread is reused there would
already be an entry in the {{threadRawStoreMap}} and
{{cacheThreadLocalRawStore}} will overwrite that entry which can also cause a
leak. This can potentially be verified by keeping the min threads and max
threads as equal (so no thread is ever GCed) you keep opening and closing
connections to HMS, eventually these threadLocalRawstore should pile up.
> HS2 with embedded metastore is leaking JDOPersistenceManager objects.
> -
>
> Key: HIVE-20192
> URL: https://issues.apache.org/jira/browse/HIVE-20192
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.0.0, 3.1.0, 4.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: HiveServer2, pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-20192.01.patch
>
>
> Hiveserver2 instances where crashing every 3-4 days and observed HS2 in on
> unresponsive state. Also, observed that the FGC collection happening regularly
> From JXray report it is seen that pmCache(List of JDOPersistenceManager
> objects) is occupying 84% of the heap and there are around 16,000 references
> of UDFClassLoader.
> {code:java}
> 10,759,230K (84.7%) Object tree for GC root(s) Java Static
> org.apache.hadoop.hive.metastore.ObjectStore.pmf
> - org.datanucleus.api.jdo.JDOPersistenceManagerFactory.pmCache ↘ 10,744,419K
> (84.6%), 1 reference(s)
> - j.u.Collections$SetFromMap.m ↘ 10,744,419K (84.6%), 1 reference(s)
> - {java.util.concurrent.ConcurrentHashMap}.keys ↘ 10,743,764K (84.5%),
> 16,872 reference(s)
> - org.datanucleus.api.jdo.JDOPersistenceManager.ec ↘ 10,738,831K
> (84.5%), 16,872 reference(s)
> ... 3 more references together retaining 4,933K (< 0.1%)
> - java.util.concurrent.ConcurrentHashMap self 655K (< 0.1%), 1 object(s)
> ... 2 more references together retaining 48b (< 0.1%)
> - org.datanucleus.api.jdo.JDOPersistenceManagerFactory.nucleusContext ↘
> 14,810K (0.1%), 1 reference(s)
> ... 3 more references together retaining 96b (< 0.1%){code}
> When the RawStore object is re-created, it is not allowed to be updated into
> the ThreadWithGarbageCleanup.threadRawStoreMap which leads to the new
> RawStore never gets cleaned-up when the thread exit.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20213) Upgrade Calcite to 1.17.0
[ https://issues.apache.org/jira/browse/HIVE-20213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20213: --- Attachment: HIVE-20213.patch > Upgrade Calcite to 1.17.0 > - > > Key: HIVE-20213 > URL: https://issues.apache.org/jira/browse/HIVE-20213 > Project: Hive > Issue Type: Improvement > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20213.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20044) Arrow Serde should pad char values and handle empty strings correctly
[ https://issues.apache.org/jira/browse/HIVE-20044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549934#comment-16549934 ] Eric Wohlstadter commented on HIVE-20044: - [~teddy.choi] Looks like you should kick this patch again to get a green build. /cc [~ashutoshc] I think TestMiniDruidCliDriver is flaky. > Arrow Serde should pad char values and handle empty strings correctly > - > > Key: HIVE-20044 > URL: https://issues.apache.org/jira/browse/HIVE-20044 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Reporter: Teddy Choi >Assignee: Teddy Choi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20044.1.branch-3.patch, HIVE-20044.1.patch, > HIVE-20044.patch > > > When Arrow Serde serializes char values, it loses padding. Also when it > counts empty strings, sometimes it makes a smaller number. It should pad char > values and handle empty strings correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20213) Upgrade Calcite to 1.17.0
[ https://issues.apache.org/jira/browse/HIVE-20213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20213: --- Assignee: Jesus Camacho Rodriguez Status: Patch Available (was: Open) > Upgrade Calcite to 1.17.0 > - > > Key: HIVE-20213 > URL: https://issues.apache.org/jira/browse/HIVE-20213 > Project: Hive > Issue Type: Improvement > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20213.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20117) schema changes for txn stats
[ https://issues.apache.org/jira/browse/HIVE-20117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-20117: Attachment: HIVE-20117.03.patch > schema changes for txn stats > > > Key: HIVE-20117 > URL: https://issues.apache.org/jira/browse/HIVE-20117 > Project: Hive > Issue Type: Bug > Components: Statistics, Transactions >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20117.01.patch, HIVE-20117.02.patch, > HIVE-20117.03.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19166) TestMiniLlapLocalCliDriver sysdb failure
[
https://issues.apache.org/jira/browse/HIVE-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549925#comment-16549925
]
Hive QA commented on HIVE-19166:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12932204/HIVE-19166.17.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14674 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=165)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12707/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12707/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12707/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12932204 - PreCommit-HIVE-Build
> TestMiniLlapLocalCliDriver sysdb failure
>
>
> Key: HIVE-19166
> URL: https://issues.apache.org/jira/browse/HIVE-19166
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-19166.04.patch, HIVE-19166.05.patch,
> HIVE-19166.06.patch, HIVE-19166.09.patch, HIVE-19166.1.patch,
> HIVE-19166.10.patch, HIVE-19166.11.patch, HIVE-19166.12.patch,
> HIVE-19166.13.patch, HIVE-19166.14.patch, HIVE-19166.15.patch,
> HIVE-19166.16.patch, HIVE-19166.17.patch, HIVE-19166.2.patch,
> HIVE-19166.3.patch
>
>
> Broken by HIVE-18715
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19532) fix tests for master-txnstats branch
[ https://issues.apache.org/jira/browse/HIVE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19532: Attachment: HIVE-19532.20.patch > fix tests for master-txnstats branch > > > Key: HIVE-19532 > URL: https://issues.apache.org/jira/browse/HIVE-19532 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-19532.01.patch, HIVE-19532.01.prepatch, > HIVE-19532.02.patch, HIVE-19532.02.prepatch, HIVE-19532.03.patch, > HIVE-19532.04.patch, HIVE-19532.05.patch, HIVE-19532.07.patch, > HIVE-19532.08.patch, HIVE-19532.09.patch, HIVE-19532.10.patch, > HIVE-19532.11.patch, HIVE-19532.12.patch, HIVE-19532.13.patch, > HIVE-19532.14.patch, HIVE-19532.15.patch, HIVE-19532.16.patch, > HIVE-19532.19.patch, HIVE-19532.20.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19532) fix tests for master-txnstats branch
[ https://issues.apache.org/jira/browse/HIVE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19532: Attachment: (was: HIVE-19532.18.patch) > fix tests for master-txnstats branch > > > Key: HIVE-19532 > URL: https://issues.apache.org/jira/browse/HIVE-19532 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-19532.01.patch, HIVE-19532.01.prepatch, > HIVE-19532.02.patch, HIVE-19532.02.prepatch, HIVE-19532.03.patch, > HIVE-19532.04.patch, HIVE-19532.05.patch, HIVE-19532.07.patch, > HIVE-19532.08.patch, HIVE-19532.09.patch, HIVE-19532.10.patch, > HIVE-19532.11.patch, HIVE-19532.12.patch, HIVE-19532.13.patch, > HIVE-19532.14.patch, HIVE-19532.15.patch, HIVE-19532.16.patch, > HIVE-19532.19.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19532) fix tests for master-txnstats branch
[ https://issues.apache.org/jira/browse/HIVE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19532: Attachment: (was: HIVE-19532.17.patch) > fix tests for master-txnstats branch > > > Key: HIVE-19532 > URL: https://issues.apache.org/jira/browse/HIVE-19532 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-19532.01.patch, HIVE-19532.01.prepatch, > HIVE-19532.02.patch, HIVE-19532.02.prepatch, HIVE-19532.03.patch, > HIVE-19532.04.patch, HIVE-19532.05.patch, HIVE-19532.07.patch, > HIVE-19532.08.patch, HIVE-19532.09.patch, HIVE-19532.10.patch, > HIVE-19532.11.patch, HIVE-19532.12.patch, HIVE-19532.13.patch, > HIVE-19532.14.patch, HIVE-19532.15.patch, HIVE-19532.16.patch, > HIVE-19532.19.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20213) Upgrade Calcite to 1.17.0
[ https://issues.apache.org/jira/browse/HIVE-20213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20213: --- Target Version/s: 4.0.0, 3.2.0 > Upgrade Calcite to 1.17.0 > - > > Key: HIVE-20213 > URL: https://issues.apache.org/jira/browse/HIVE-20213 > Project: Hive > Issue Type: Improvement > Components: CBO >Reporter: Jesus Camacho Rodriguez >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20213) Upgrade Calcite to 1.17.0
[ https://issues.apache.org/jira/browse/HIVE-20213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-20213: -- Assignee: (was: Jesus Camacho Rodriguez) > Upgrade Calcite to 1.17.0 > - > > Key: HIVE-20213 > URL: https://issues.apache.org/jira/browse/HIVE-20213 > Project: Hive > Issue Type: Improvement > Components: CBO >Reporter: Jesus Camacho Rodriguez >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19166) TestMiniLlapLocalCliDriver sysdb failure
[
https://issues.apache.org/jira/browse/HIVE-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549896#comment-16549896
]
Hive QA commented on HIVE-19166:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
47s{color} | {color:blue} itests/util in master has 52 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
16s{color} | {color:blue} ql in master has 2273 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12707/dev-support/hive-personality.sh
|
| git revision | master / e867d1c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: itests/hive-unit itests/util metastore ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12707/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TestMiniLlapLocalCliDriver sysdb failure
>
>
> Key: HIVE-19166
> URL: https://issues.apache.org/jira/browse/HIVE-19166
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-19166.04.patch, HIVE-19166.05.patch,
> HIVE-19166.06.patch, HIVE-19166.09.patch, HIVE-19166.1.patch,
> HIVE-19166.10.patch, HIVE-19166.11.patch, HIVE-19166.12.patch,
> HIVE-19166.13.patch, HIVE-19166.14.patch, HIVE-19166.15.patch,
> HIVE-19166.16.patch, HIVE-19166.17.patch, HIVE-19166.2.patch,
> HIVE-19166.3.patch
>
>
> Broken by HIVE-18715
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20213) Upgrade Calcite to 1.17.0
[ https://issues.apache.org/jira/browse/HIVE-20213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-20213: -- > Upgrade Calcite to 1.17.0 > - > > Key: HIVE-20213 > URL: https://issues.apache.org/jira/browse/HIVE-20213 > Project: Hive > Issue Type: Improvement > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20196) Separate MetastoreConf into common and server parts
[ https://issues.apache.org/jira/browse/HIVE-20196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-20196: -- Attachment: HIVE-20196.02.patch > Separate MetastoreConf into common and server parts > --- > > Key: HIVE-20196 > URL: https://issues.apache.org/jira/browse/HIVE-20196 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Affects Versions: 4.0.0, 3.2.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-20196.01.patch, HIVE-20196.02.patch > > > MetastoreConf has knowledge about some server-specific classes. We need to > separate these into a separate server-specific class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20212) Hiveserver2 in http mode emitting metric default.General.open_connections incorrectly
[ https://issues.apache.org/jira/browse/HIVE-20212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20212: --- Attachment: HIVE-20212.patch > Hiveserver2 in http mode emitting metric default.General.open_connections > incorrectly > - > > Key: HIVE-20212 > URL: https://issues.apache.org/jira/browse/HIVE-20212 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Minor > Attachments: HIVE-20212.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20212) Hiveserver2 in http mode emitting metric default.General.open_connections incorrectly
[ https://issues.apache.org/jira/browse/HIVE-20212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20212: --- Status: Patch Available (was: In Progress) > Hiveserver2 in http mode emitting metric default.General.open_connections > incorrectly > - > > Key: HIVE-20212 > URL: https://issues.apache.org/jira/browse/HIVE-20212 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Minor > Attachments: HIVE-20212.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-20212) Hiveserver2 in http mode emitting metric default.General.open_connections incorrectly
[ https://issues.apache.org/jira/browse/HIVE-20212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-20212 started by Jesus Camacho Rodriguez. -- > Hiveserver2 in http mode emitting metric default.General.open_connections > incorrectly > - > > Key: HIVE-20212 > URL: https://issues.apache.org/jira/browse/HIVE-20212 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Minor > Attachments: HIVE-20212.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17896) TopNKey: Create a standalone vectorizable TopNKey operator
[ https://issues.apache.org/jira/browse/HIVE-17896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-17896: --- Resolution: Fixed Fix Version/s: 3.2.0 4.0.0 Status: Resolved (was: Patch Available) Pushed to master, branch-3. Thanks [~teddy.choi]! > TopNKey: Create a standalone vectorizable TopNKey operator > -- > > Key: HIVE-17896 > URL: https://issues.apache.org/jira/browse/HIVE-17896 > Project: Hive > Issue Type: New Feature > Components: Operators >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Teddy Choi >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-17896.1.patch, HIVE-17896.10.patch, > HIVE-17896.11.patch, HIVE-17896.12.patch, HIVE-17896.13.patch, > HIVE-17896.3.patch, HIVE-17896.4.patch, HIVE-17896.5.patch, > HIVE-17896.6.patch, HIVE-17896.7.patch, HIVE-17896.8.patch, HIVE-17896.9.patch > > > For TPC-DS Query27, the TopN operation is delayed by the group-by - the > group-by operator buffers up all the rows before discarding the 99% of the > rows in the TopN Hash within the ReduceSink Operator. > The RS TopN operator is very restrictive as it only supports doing the > filtering on the shuffle keys, but it is better to do this before breaking > the vectors into rows and losing the isRepeating properties. > Adding a TopN Key operator in the physical operator tree allows the following > to happen. > GBY->RS(Top=1) > can become > TNK(1)->GBY->RS(Top=1) > So that, the TopNKey can remove rows before they are buffered into the GBY > and consume memory. > Here's the equivalent implementation in Presto > https://github.com/prestodb/presto/blob/master/presto-main/src/main/java/com/facebook/presto/operator/TopNOperator.java#L35 > Adding this as a sub-feature of GroupBy prevents further optimizations if the > GBY is on keys "a,b,c" and the TopNKey is on just "a". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-4367) enhance TRUNCATE syntax to drop data of external table
[ https://issues.apache.org/jira/browse/HIVE-4367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-4367: - Attachment: HIVE-4367.7.patch > enhance TRUNCATE syntax to drop data of external table > -- > > Key: HIVE-4367 > URL: https://issues.apache.org/jira/browse/HIVE-4367 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: caofangkun >Assignee: caofangkun >Priority: Minor > Attachments: HIVE-4367-1.patch, HIVE-4367.2.patch.txt, > HIVE-4367.3.patch, HIVE-4367.4.patch, HIVE-4367.5.patch, HIVE-4367.6.patch, > HIVE-4367.7.patch > > > In my use case , > sometimes I have to remove data of external tables to free up storage space > of the cluster . > So it's necessary for to enhance the syntax like > "TRUNCATE TABLE srcpart_truncate PARTITION (dt='201130412') FORCE;" > to remove data from EXTERNAL table. > And I add a configuration property to enable remove data to Trash > > hive.truncate.skiptrash > false > > if true will remove data to trash, else false drop data immediately > > > For example : > hive (default)> TRUNCATE TABLE external1 partition (ds='11'); > FAILED: Error in semantic analysis: Cannot truncate non-managed table > external1 > hive (default)> TRUNCATE TABLE external1 partition (ds='11') FORCE; > [2013-04-16 17:15:52]: Compile Start > [2013-04-16 17:15:52]: Compile End > [2013-04-16 17:15:52]: OK > [2013-04-16 17:15:52]: Time taken: 0.413 seconds > hive (default)> set hive.truncate.skiptrash; > hive.truncate.skiptrash=false > hive (default)> set hive.truncate.skiptrash=true; > hive (default)> TRUNCATE TABLE external1 partition (ds='12') FORCE; > [2013-04-16 17:16:21]: Compile Start > [2013-04-16 17:16:21]: Compile End > [2013-04-16 17:16:21]: OK > [2013-04-16 17:16:21]: Time taken: 0.143 seconds > hive (default)> dfs -ls /user/test/.Trash/Current/; > Found 1 items > drwxr-xr-x -test supergroup 0 2013-04-16 17:06 /user/test/.Trash/Current/ds=11 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19532) fix tests for master-txnstats branch
[ https://issues.apache.org/jira/browse/HIVE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549874#comment-16549874 ] Sergey Shelukhin edited comment on HIVE-19532 at 7/19/18 9:13 PM: -- All the q test failures are trivial updates, except acid_stats4 - rename partition doesn't actually work, not sure how I didn't notice it before. The non q test failure is a test setup issue. I will fix and commit all the test fixes/updates together. was (Author: sershe): All the failures are trivial, except acid_stats4 - rename partition doesn't actually work, not sure how I didn't notice it before. I will fix and commit all the test fixes/updates together > fix tests for master-txnstats branch > > > Key: HIVE-19532 > URL: https://issues.apache.org/jira/browse/HIVE-19532 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-19532.01.patch, HIVE-19532.01.prepatch, > HIVE-19532.02.patch, HIVE-19532.02.prepatch, HIVE-19532.03.patch, > HIVE-19532.04.patch, HIVE-19532.05.patch, HIVE-19532.07.patch, > HIVE-19532.08.patch, HIVE-19532.09.patch, HIVE-19532.10.patch, > HIVE-19532.11.patch, HIVE-19532.12.patch, HIVE-19532.13.patch, > HIVE-19532.14.patch, HIVE-19532.15.patch, HIVE-19532.16.patch, > HIVE-19532.17.patch, HIVE-19532.18.patch, HIVE-19532.19.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19532) fix tests for master-txnstats branch
[ https://issues.apache.org/jira/browse/HIVE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549874#comment-16549874 ] Sergey Shelukhin commented on HIVE-19532: - All the failures are trivial, except acid_stats4 - rename partition doesn't actually work, not sure how I didn't notice it before. I will fix and commit all the test fixes/updates together > fix tests for master-txnstats branch > > > Key: HIVE-19532 > URL: https://issues.apache.org/jira/browse/HIVE-19532 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-19532.01.patch, HIVE-19532.01.prepatch, > HIVE-19532.02.patch, HIVE-19532.02.prepatch, HIVE-19532.03.patch, > HIVE-19532.04.patch, HIVE-19532.05.patch, HIVE-19532.07.patch, > HIVE-19532.08.patch, HIVE-19532.09.patch, HIVE-19532.10.patch, > HIVE-19532.11.patch, HIVE-19532.12.patch, HIVE-19532.13.patch, > HIVE-19532.14.patch, HIVE-19532.15.patch, HIVE-19532.16.patch, > HIVE-19532.17.patch, HIVE-19532.18.patch, HIVE-19532.19.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20203) Arrow SerDe leaks a DirectByteBuffer
[ https://issues.apache.org/jira/browse/HIVE-20203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter updated HIVE-20203: Status: Patch Available (was: Open) > Arrow SerDe leaks a DirectByteBuffer > > > Key: HIVE-20203 > URL: https://issues.apache.org/jira/browse/HIVE-20203 > Project: Hive > Issue Type: Bug >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Blocker > Attachments: HIVE-20203.1.patch, HIVE-20203.2.patch, > HIVE-20203.3.patch > > > ArrowColumnarBatchSerDe allocates an arrow NullableMapVector for each task > that uses the serde. > The vector is a DirectByteBuffer allocated from Arrow's off-heap buffer pool. > This buffer is never closed and leaks about 1K of physical memory for each > task. > This patch does three things: > # Ensure the buffer is closed when the RecordWriter for the task is closed. > # Adds per-task memory accounting by assigning a ChildAllocator to each task > from the RootAllocator. > # Enforces that the ChildAllocator for a task has released all memory > assigned to it, when the task is completed. > The patch assumes that close() is always called on the RecordWriter when a > task is finished (even if there is a failure during task execution). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20203) Arrow SerDe leaks a DirectByteBuffer
[ https://issues.apache.org/jira/browse/HIVE-20203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter updated HIVE-20203: Attachment: HIVE-20203.3.patch > Arrow SerDe leaks a DirectByteBuffer > > > Key: HIVE-20203 > URL: https://issues.apache.org/jira/browse/HIVE-20203 > Project: Hive > Issue Type: Bug >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Blocker > Attachments: HIVE-20203.1.patch, HIVE-20203.2.patch, > HIVE-20203.3.patch > > > ArrowColumnarBatchSerDe allocates an arrow NullableMapVector for each task > that uses the serde. > The vector is a DirectByteBuffer allocated from Arrow's off-heap buffer pool. > This buffer is never closed and leaks about 1K of physical memory for each > task. > This patch does three things: > # Ensure the buffer is closed when the RecordWriter for the task is closed. > # Adds per-task memory accounting by assigning a ChildAllocator to each task > from the RootAllocator. > # Enforces that the ChildAllocator for a task has released all memory > assigned to it, when the task is completed. > The patch assumes that close() is always called on the RecordWriter when a > task is finished (even if there is a failure during task execution). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20203) Arrow SerDe leaks a DirectByteBuffer
[ https://issues.apache.org/jira/browse/HIVE-20203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter updated HIVE-20203: Status: Open (was: Patch Available) > Arrow SerDe leaks a DirectByteBuffer > > > Key: HIVE-20203 > URL: https://issues.apache.org/jira/browse/HIVE-20203 > Project: Hive > Issue Type: Bug >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Blocker > Attachments: HIVE-20203.1.patch, HIVE-20203.2.patch > > > ArrowColumnarBatchSerDe allocates an arrow NullableMapVector for each task > that uses the serde. > The vector is a DirectByteBuffer allocated from Arrow's off-heap buffer pool. > This buffer is never closed and leaks about 1K of physical memory for each > task. > This patch does three things: > # Ensure the buffer is closed when the RecordWriter for the task is closed. > # Adds per-task memory accounting by assigning a ChildAllocator to each task > from the RootAllocator. > # Enforces that the ChildAllocator for a task has released all memory > assigned to it, when the task is completed. > The patch assumes that close() is always called on the RecordWriter when a > task is finished (even if there is a failure during task execution). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20193) cboInfo is not present in the explain plan json
[
https://issues.apache.org/jira/browse/HIVE-20193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549849#comment-16549849
]
Hive QA commented on HIVE-20193:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12932201/HIVE-20193.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12706/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12706/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12706/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-07-19 20:47:54.217
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-12706/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-07-19 20:47:54.219
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at e867d1c HIVE-20149: TestHiveCli failing/timing out(Vineet Garg,
reviewed by Zoltan Haindrich)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at e867d1c HIVE-20149: TestHiveCli failing/timing out(Vineet Garg,
reviewed by Zoltan Haindrich)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-07-19 20:47:54.873
+ rm -rf ../yetus_PreCommit-HIVE-Build-12706
+ mkdir ../yetus_PreCommit-HIVE-Build-12706
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-12706
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-12706/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: a/ql/src/java/org/apache/hadoop/hive/ql/Driver.java: does not exist in
index
error: a/ql/src/java/org/apache/hadoop/hive/ql/QueryPlan.java: does not exist
in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainTask.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/hooks/HiveProtoLoggingHook.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java:
does not exist in index
error: a/ql/src/test/org/apache/hadoop/hive/ql/exec/TestExplainTask.java: does
not exist in index
error: a/ql/src/test/results/clientpositive/input4.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/join0.q.out: does not exist in index
error: a/ql/src/test/results/clientpositive/parallel_join0.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/plan_json.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/vector_outer_join3.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/vector_outer_join4.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/vector_outer_join6.q.out: does not
exist in index
error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/Driver.java:1007
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/Driver.java' with
conflicts.
error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/QueryPlan.java:71
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/QueryPlan.java' cleanly.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainTask.java:217
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainTask.java'
with conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/hooks/HiveProtoLoggingHook.java:479
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/hooks/HiveProtoLoggingHook.java'
[jira] [Commented] (HIVE-20203) Arrow SerDe leaks a DirectByteBuffer
[
https://issues.apache.org/jira/browse/HIVE-20203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549845#comment-16549845
]
Hive QA commented on HIVE-20203:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12932198/HIVE-20203.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12705/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12705/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12705/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-07-19 20:46:43.071
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-12705/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-07-19 20:46:43.073
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
6d15ce4..e867d1c master -> origin/master
+ git reset --hard HEAD
HEAD is now at 6d15ce4 HIVE-19360: CBO: Add an "optimizedSQL" to QueryPlan
object (Gopal V, reviewed by Jesus Camacho Rodriguez)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 2 commits, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at e867d1c HIVE-20149: TestHiveCli failing/timing out(Vineet Garg,
reviewed by Zoltan Haindrich)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-07-19 20:46:44.433
+ rm -rf ../yetus_PreCommit-HIVE-Build-12705
+ mkdir ../yetus_PreCommit-HIVE-Build-12705
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-12705
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-12705/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/io/arrow/Serializer.java:96
Falling back to three-way merge...
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/io/arrow/Serializer.java:96
error: ql/src/java/org/apache/hadoop/hive/ql/io/arrow/Serializer.java: patch
does not apply
error: src/java/org/apache/hadoop/hive/conf/HiveConf.java: does not exist in
index
error: src/java/org/apache/hadoop/hive/llap/LlapArrowRecordWriter.java: does
not exist in index
error: src/java/org/apache/hadoop/hive/ql/io/arrow/ArrowWrapperWritable.java:
does not exist in index
error: src/java/org/apache/hadoop/hive/ql/io/arrow/Serializer.java: does not
exist in index
error: java/org/apache/hadoop/hive/conf/HiveConf.java: does not exist in index
error: java/org/apache/hadoop/hive/llap/LlapArrowRecordWriter.java: does not
exist in index
error: java/org/apache/hadoop/hive/ql/io/arrow/ArrowWrapperWritable.java: does
not exist in index
error: java/org/apache/hadoop/hive/ql/io/arrow/Serializer.java: does not exist
in index
The patch does not appear to apply with p0, p1, or p2
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-12705
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12932198 - PreCommit-HIVE-Build
> Arrow SerDe leaks a DirectByteBuffer
>
>
> Key: HIVE-20203
> URL: https://issues.apache.org/jira/browse/HIVE-20203
> Project: Hive
> Issue Type: Bug
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Blocker
> Attachments: HIVE-20203.1.patch, HIVE-20203.2.patch
>
>
> ArrowColumnarBatchSerDe allocates an arrow NullableMapVector for each task
> that uses the serde.
> The vector is a DirectByteBuffer allocated
[jira] [Commented] (HIVE-17896) TopNKey: Create a standalone vectorizable TopNKey operator
[
https://issues.apache.org/jira/browse/HIVE-17896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549839#comment-16549839
]
Hive QA commented on HIVE-17896:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12932197/HIVE-17896.13.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14678 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12704/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12704/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12704/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12932197 - PreCommit-HIVE-Build
> TopNKey: Create a standalone vectorizable TopNKey operator
> --
>
> Key: HIVE-17896
> URL: https://issues.apache.org/jira/browse/HIVE-17896
> Project: Hive
> Issue Type: New Feature
> Components: Operators
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Teddy Choi
>Priority: Major
> Attachments: HIVE-17896.1.patch, HIVE-17896.10.patch,
> HIVE-17896.11.patch, HIVE-17896.12.patch, HIVE-17896.13.patch,
> HIVE-17896.3.patch, HIVE-17896.4.patch, HIVE-17896.5.patch,
> HIVE-17896.6.patch, HIVE-17896.7.patch, HIVE-17896.8.patch, HIVE-17896.9.patch
>
>
> For TPC-DS Query27, the TopN operation is delayed by the group-by - the
> group-by operator buffers up all the rows before discarding the 99% of the
> rows in the TopN Hash within the ReduceSink Operator.
> The RS TopN operator is very restrictive as it only supports doing the
> filtering on the shuffle keys, but it is better to do this before breaking
> the vectors into rows and losing the isRepeating properties.
> Adding a TopN Key operator in the physical operator tree allows the following
> to happen.
> GBY->RS(Top=1)
> can become
> TNK(1)->GBY->RS(Top=1)
> So that, the TopNKey can remove rows before they are buffered into the GBY
> and consume memory.
> Here's the equivalent implementation in Presto
> https://github.com/prestodb/presto/blob/master/presto-main/src/main/java/com/facebook/presto/operator/TopNOperator.java#L35
> Adding this as a sub-feature of GroupBy prevents further optimizations if the
> GBY is on keys "a,b,c" and the TopNKey is on just "a".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20097) Convert standalone-metastore to a submodule
[ https://issues.apache.org/jira/browse/HIVE-20097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-20097: -- Status: Patch Available (was: Reopened) Submitting branch-3 based patch > Convert standalone-metastore to a submodule > --- > > Key: HIVE-20097 > URL: https://issues.apache.org/jira/browse/HIVE-20097 > Project: Hive > Issue Type: Sub-task > Components: Hive, Metastore, Standalone Metastore >Affects Versions: 3.1.0, 4.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20097.01.patch, HIVE-20097.02.patch, > HIVE-20097.03.patch, HIVE-20097.04.patch, HIVE-20097.05.patch, > HIVE-20097.06.patch, HIVE-20097.07-branch-3.patch, HIVE-20097.07.patch > > > This is a subtask to stage HIVE-17751 changes into several smaller phases. > The first part is moving existing code in hive-standalone-metastore to a > sub-module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20097) Convert standalone-metastore to a submodule
[ https://issues.apache.org/jira/browse/HIVE-20097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-20097: -- Attachment: HIVE-20097.07-branch-3.patch > Convert standalone-metastore to a submodule > --- > > Key: HIVE-20097 > URL: https://issues.apache.org/jira/browse/HIVE-20097 > Project: Hive > Issue Type: Sub-task > Components: Hive, Metastore, Standalone Metastore >Affects Versions: 3.1.0, 4.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20097.01.patch, HIVE-20097.02.patch, > HIVE-20097.03.patch, HIVE-20097.04.patch, HIVE-20097.05.patch, > HIVE-20097.06.patch, HIVE-20097.07-branch-3.patch, HIVE-20097.07.patch > > > This is a subtask to stage HIVE-17751 changes into several smaller phases. > The first part is moving existing code in hive-standalone-metastore to a > sub-module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HIVE-20097) Convert standalone-metastore to a submodule
[ https://issues.apache.org/jira/browse/HIVE-20097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov reopened HIVE-20097: --- Reopening for branch-3 backport > Convert standalone-metastore to a submodule > --- > > Key: HIVE-20097 > URL: https://issues.apache.org/jira/browse/HIVE-20097 > Project: Hive > Issue Type: Sub-task > Components: Hive, Metastore, Standalone Metastore >Affects Versions: 3.1.0, 4.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20097.01.patch, HIVE-20097.02.patch, > HIVE-20097.03.patch, HIVE-20097.04.patch, HIVE-20097.05.patch, > HIVE-20097.06.patch, HIVE-20097.07-branch-3.patch, HIVE-20097.07.patch > > > This is a subtask to stage HIVE-17751 changes into several smaller phases. > The first part is moving existing code in hive-standalone-metastore to a > sub-module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20212) Hiveserver2 in http mode emitting metric default.General.open_connections incorrectly
[ https://issues.apache.org/jira/browse/HIVE-20212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20212: --- Target Version/s: 4.0.0, 3.2.0 > Hiveserver2 in http mode emitting metric default.General.open_connections > incorrectly > - > > Key: HIVE-20212 > URL: https://issues.apache.org/jira/browse/HIVE-20212 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20212) Hiveserver2 in http mode emitting metric default.General.open_connections incorrectly
[ https://issues.apache.org/jira/browse/HIVE-20212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20212: --- Reporter: Jesus Camacho Rodriguez (was: Dinesh Chitlangia) > Hiveserver2 in http mode emitting metric default.General.open_connections > incorrectly > - > > Key: HIVE-20212 > URL: https://issues.apache.org/jira/browse/HIVE-20212 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20212) Hiveserver2 in http mode emitting metric default.General.open_connections incorrectly
[ https://issues.apache.org/jira/browse/HIVE-20212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20212: --- Fix Version/s: (was: 3.0.0) > Hiveserver2 in http mode emitting metric default.General.open_connections > incorrectly > - > > Key: HIVE-20212 > URL: https://issues.apache.org/jira/browse/HIVE-20212 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20212) Hiveserver2 in http mode emitting metric default.General.open_connections incorrectly
[ https://issues.apache.org/jira/browse/HIVE-20212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20212: --- Summary: Hiveserver2 in http mode emitting metric default.General.open_connections incorrectly (was: Hiveserver2 in http mode not emitting metric default.General.open_connections) > Hiveserver2 in http mode emitting metric default.General.open_connections > incorrectly > - > > Key: HIVE-20212 > URL: https://issues.apache.org/jira/browse/HIVE-20212 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Dinesh Chitlangia >Assignee: Jesus Camacho Rodriguez >Priority: Minor > Fix For: 3.0.0 > > > Instances in binary mode are emitting the metric > _default.General.open_connections_ but the instances operating in http mode > are not emitting this metric. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20212) Hiveserver2 in http mode not emitting metric default.General.open_connections
[ https://issues.apache.org/jira/browse/HIVE-20212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-20212: -- > Hiveserver2 in http mode not emitting metric default.General.open_connections > - > > Key: HIVE-20212 > URL: https://issues.apache.org/jira/browse/HIVE-20212 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Dinesh Chitlangia >Assignee: Jesus Camacho Rodriguez >Priority: Minor > Fix For: 3.0.0 > > > Instances in binary mode are emitting the metric > _default.General.open_connections_ but the instances operating in http mode > are not emitting this metric. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17896) TopNKey: Create a standalone vectorizable TopNKey operator
[
https://issues.apache.org/jira/browse/HIVE-17896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549795#comment-16549795
]
Hive QA commented on HIVE-17896:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
49s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 64 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} serde in master has 194 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
1s{color} | {color:blue} ql in master has 2273 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
45s{color} | {color:red} ql: The patch generated 32 new + 430 unchanged - 0
fixed = 462 total (was 430) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
53s{color} | {color:red} serde generated 1 new + 194 unchanged - 0 fixed = 195
total (was 194) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
27s{color} | {color:red} ql generated 8 new + 2273 unchanged - 0 fixed = 2281
total (was 2273) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 58s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:serde |
| |
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.compare(Object[],
ObjectInspector[], Object[], ObjectInspector[], boolean[]) negates the return
value of
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.compare(Object,
ObjectInspector, Object, ObjectInspector) At
ObjectInspectorUtils.java:negates the return value of
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.compare(Object,
ObjectInspector, Object, ObjectInspector) At ObjectInspectorUtils.java:[line
956] |
| FindBugs | module:ql |
| | new
org.apache.hadoop.hive.ql.exec.TopNKeyOperator$KeyWrapperComparator(ObjectInspector[],
ObjectInspector[], boolean[]) may expose internal representation by storing an
externally mutable object into
TopNKeyOperator$KeyWrapperComparator.columnSortOrderIsDesc At
TopNKeyOperator.java:expose internal representation by storing an externally
mutable object into TopNKeyOperator$KeyWrapperComparator.columnSortOrderIsDesc
At TopNKeyOperator.java:[line 83] |
| | new
org.apache.hadoop.hive.ql.exec.TopNKeyOperator$KeyWrapperComparator(ObjectInspector[],
ObjectInspector[], boolean[]) may expose internal representation by storing an
externally mutable object into
TopNKeyOperator$KeyWrapperComparator.objectInspectors1 At
TopNKeyOperator.java:expose internal representation by storing an externally
mutable object into
[jira] [Updated] (HIVE-20212) Hiveserver2 in http mode emitting metric default.General.open_connections incorrectly
[ https://issues.apache.org/jira/browse/HIVE-20212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20212: --- Description: (was: Instances in binary mode are emitting the metric _default.General.open_connections_ but the instances operating in http mode are not emitting this metric.) > Hiveserver2 in http mode emitting metric default.General.open_connections > incorrectly > - > > Key: HIVE-20212 > URL: https://issues.apache.org/jira/browse/HIVE-20212 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Dinesh Chitlangia >Assignee: Jesus Camacho Rodriguez >Priority: Minor > Fix For: 3.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20204) Type conversion during IN () comparisons is using different rules from other comparison operations
[
https://issues.apache.org/jira/browse/HIVE-20204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-20204:
--
Attachment: HIVE-20204.2.patch
> Type conversion during IN () comparisons is using different rules from other
> comparison operations
> --
>
> Key: HIVE-20204
> URL: https://issues.apache.org/jira/browse/HIVE-20204
> Project: Hive
> Issue Type: Bug
> Components: Types
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-20204.1.patch, HIVE-20204.2.patch
>
>
> Noticed this while looking at HIVE-20082.
> The type conversion done during GenericUDFIn (via
> ReturnObjectInspectorResolver) uses FunctionRegistry.getCommonClass(),
> whereas the other comparison operators (=, <, >, <=, >=) use
> FunctionRegistry.getCommonClassForComparison(). As a result, dec_column IN
> ('1.1', '2.2') compares the values as strings, whereas dec_column = '1.1'
> would compare the values as doubles. This makes a difference for HIVE-20082
> since it is related to changing the 0-padding during decimal-to-string
> conversions.
> cc [~ashutoshc]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20115) Acid tables should not use footer scan for analyze
[ https://issues.apache.org/jira/browse/HIVE-20115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-20115: Attachment: HIVE-20115.01.patch > Acid tables should not use footer scan for analyze > -- > > Key: HIVE-20115 > URL: https://issues.apache.org/jira/browse/HIVE-20115 > Project: Hive > Issue Type: Sub-task > Components: Statistics, Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20115.01.patch, HIVE-20115.01.patch, > HIVE-20115.patch > > > Discovered via incorrect stats in acid_no_buckets test on master-txnstats > branch -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20202) Add profiler endpoint to HS2 and LLAP
[ https://issues.apache.org/jira/browse/HIVE-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-20202: - Attachment: HIVE-20202.5.patch > Add profiler endpoint to HS2 and LLAP > - > > Key: HIVE-20202 > URL: https://issues.apache.org/jira/browse/HIVE-20202 > Project: Hive > Issue Type: New Feature >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20202.1.patch, HIVE-20202.2.patch, > HIVE-20202.3.patch, HIVE-20202.4.patch, HIVE-20202.5.patch > > > Add a web endpoint for profiling based on async-profiler. This servlet should > be added to httpserver so that HS2 and LLAP daemons can output flamegraphs > when their /prof endpoint is hit. Since this will be based on > [https://github.com/jvm-profiling-tools/async-profiler] heap allocation, lock > contentions, HW counters etc. will also be supported in addition to cpu > profiling. In most cases the profiling overhead is pretty low and is safe to > run on production. More analysis on CPU and memory overhead here > [https://github.com/jvm-profiling-tools/async-profiler/issues/14] and > [https://github.com/jvm-profiling-tools/async-profiler/issues/131] > > For the impatient, here is the usage doc and the sample output > [https://github.com/prasanthj/nightswatch/blob/master/README.md] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19990) Query with interval literal in join condition fails
[
https://issues.apache.org/jira/browse/HIVE-19990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19990:
---
Status: Patch Available (was: Open)
> Query with interval literal in join condition fails
> ---
>
> Key: HIVE-19990
> URL: https://issues.apache.org/jira/browse/HIVE-19990
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-19990.1.patch, HIVE-19990.2.patch,
> HIVE-19990.3.patch, HIVE-19990.4.patch
>
>
> *Reproducer*
> {code:sql}
> > create table date_dim_d1(
> d_week_seqint,
> d_datestring);
> > SELECT
>d1.d_week_seq
> FROM
>date_dim_d1 d1
>JOIN date_dim_d1 d3
> WHERE
>Cast(d3.d_date AS date) > Cast(d1.d_date AS date) + INTERVAL '5' day ;
> {code}
> *Exception*
> {code}
> org.apache.hadoop.hive.ql.parse.SemanticException: '5 00:00:00.0'
> encountered with 0 children
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.parseJoinCondPopulateAlias(SemanticAnalyzer.java:2780)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.parseJoinCondPopulateAlias(SemanticAnalyzer.java:2775)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.parseJoinCondition(SemanticAnalyzer.java:3060)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.parseJoinCondition(SemanticAnalyzer.java:2959)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genJoinTree(SemanticAnalyzer.java:9633)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11380)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11285)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:12071)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:593)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12150)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:330)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:658)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1829)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1776)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1771)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:214)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:832)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:770)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:694)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
