[jira] [Commented] (HIVE-20839) "Cannot find field" error during dynamically partitioned hash join
[
https://issues.apache.org/jira/browse/HIVE-20839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674309#comment-16674309
]
Hive QA commented on HIVE-20839:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
39s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14743/dev-support/hive-personality.sh
|
| git revision | master / 670e49f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14743/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> "Cannot find field" error during dynamically partitioned hash join
> --
>
> Key: HIVE-20839
> URL: https://issues.apache.org/jira/browse/HIVE-20839
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-20839.1.patch, HIVE-20839.2.patch,
> HIVE-20839.3.patch, HIVE-20839.4.patch
>
>
> Occurs in some cases in the non-CBO optimized queries, either if CBO is
> disabled or has failed due to error.
> {noformat}
> 2018-10-11T04:40:22,724 ERROR [TezTR-85144_8944_1085_28_996_2
> (1539092085144_8944_1085_28_000996_2)] tez.ReduceRecordProcessor: Hit error
> while closing operators - failing tree
> 2018-10-11T04:40:22,724 ERROR [TezTR-85144_8944_1085_28_996_2
> (1539092085144_8944_1085_28_000996_2)] tez.TezProcessor:
> java.lang.RuntimeException: cannot find field _col304 from [0:_col0, 1:_col1,
> 2:_col2, 3:_col3, 4:_col4, 5:_col5, 6:_col6, 7:_col7, 8:_col8, 9:_col9,
> 10:_col10, 11:_col11, 12:_col12, 13:_col13, 14:_col15, 15:_col16, 16:_col17,
> 17:_col18, 18:_col19, 19:_col20, 20:_col21, 21:_col22, 22:_col23, 23:_col24,
> 24:_col25, 25:_col26, 26:_col27, 27:_col28, 28:_col29, 29:_col30, 30:_col31,
> 31:_col32, 32:_col33, 33:_col34, 34:_col35, 35:_col36, 36:_col37, 37:_col38,
> 38:_col39, 39:_col40, 40:_col41, 41:_col42, 42:_col43, 43:_col44, 44:_col45,
> 45:_col46, 46:_col47, 47:_col48, 48:_col49, 49:_col50, 50:_col51, 51:_col52,
> 52:_col53, 53:_col54, 54:_col55, 55:_col56, 56:_col57, 57:_col58, 58:_col59,
> 59:_col60, 60:_col61,
[jira] [Commented] (HIVE-20853) Expose ShuffleHandler.registerDag in the llap daemon API
[
https://issues.apache.org/jira/browse/HIVE-20853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674301#comment-16674301
]
Hive QA commented on HIVE-20853:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946800/HIVE-20853.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14742/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14742/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14742/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12946800/HIVE-20853.2.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946800 - PreCommit-HIVE-Build
> Expose ShuffleHandler.registerDag in the llap daemon API
>
>
> Key: HIVE-20853
> URL: https://issues.apache.org/jira/browse/HIVE-20853
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Affects Versions: 3.1.0
>Reporter: Jaume M
>Assignee: Jaume M
>Priority: Critical
> Attachments: HIVE-20853.1.patch, HIVE-20853.2.patch
>
>
> Currently DAGs are only registered when a submitWork is called for that DAG.
> At this point the crendentials are added to the ShuffleHandler and it can
> start serving.
> However Tez might (and will) schedule tasks to fetch from the ShuffleHandler
> before anything of this happens and all this tasks will fail which may
> results in the query failing.
> This happens in the scenario in which a LlapDaemon just comes up and tez
> fetchers try to open a connection before a DAG has been registered.
> Adding this API will allow to register the DAG against the Daemon when the AM
> notices that a new Daemon is up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20853) Expose ShuffleHandler.registerDag in the llap daemon API
[
https://issues.apache.org/jira/browse/HIVE-20853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674300#comment-16674300
]
Hive QA commented on HIVE-20853:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946800/HIVE-20853.2.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 44 failed/errored test(s), 14677 tests
executed
*Failed tests:*
{noformat}
TestLlapDaemonUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=332)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=194)
[kafka_storage_handler.q,druidmini_mv.q,druid_timestamptz2.q,druidmini_extractTime.q,druidmini_floorTime.q]
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
(batchId=196)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=183)
[jira] [Commented] (HIVE-20853) Expose ShuffleHandler.registerDag in the llap daemon API
[
https://issues.apache.org/jira/browse/HIVE-20853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674296#comment-16674296
]
Hive QA commented on HIVE-20853:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} llap-common in master has 71 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
22s{color} | {color:blue} llap-client in master has 26 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
21s{color} | {color:blue} llap-tez in master has 17 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} llap-server in master has 83 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
16s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
16s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 16s{color}
| {color:red} llap-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
9s{color} | {color:red} llap-client: The patch generated 1 new + 6 unchanged -
0 fixed = 7 total (was 6) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} llap-tez: The patch generated 5 new + 113 unchanged -
7 fixed = 118 total (was 120) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} llap-server: The patch generated 34 new + 157
unchanged - 0 fixed = 191 total (was 157) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 5 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
36s{color} | {color:red} llap-common generated 5 new + 71 unchanged - 0 fixed =
76 total (was 71) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
16s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 37s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:llap-common |
| |
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$RegisterDagRequestProto.PARSER
isn't final but should be At LlapDaemonProtocolProtos.java:be At
LlapDaemonProtocolProtos.java:[line 11608] |
| | Class
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$RegisterDagRequestProto
defines non-transient non-serializable instance field unknownFields In
LlapDaemonProtocolProtos.java:instance field unknownFields In
LlapDaemonProtocolProtos.java |
| |
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$RegisterDagResponseProto.PARSER
isn't final but should be At LlapDaemonProtocolProtos.java:be At
LlapDaemonProtocolProtos.java:[line 12356] |
| | Class
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$RegisterDagResponseProto
[jira] [Commented] (HIVE-20835) Interaction between constraints and MV rewriting may create loop in Calcite planner
[
https://issues.apache.org/jira/browse/HIVE-20835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674291#comment-16674291
]
Hive QA commented on HIVE-20835:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946796/HIVE-20835.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 41 failed/errored test(s), 15525 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestTxnCommands2.testACIDwithSchemaEvolutionAndCompaction
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testAcidOrcWritePreservesFieldNames
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testAcidWithSchemaEvolution
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testAlterTable (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testBucketCodec (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testBucketizedInputFormat
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testCleanerForTxnToWriteId
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testCompactWithDelete (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testDeleteIn (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testDynamicPartitionsMerge
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testDynamicPartitionsMerge2
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testETLSplitStrategyForACID
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testEmptyInTblproperties
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testFailHeartbeater (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testFailureOnAlteringTransactionalProperties
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testFileSystemUnCaching (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testInitiatorWithMultipleFailedCompactions
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testInsertOverwrite1 (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testInsertOverwrite2 (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testInsertOverwriteWithSelfJoin
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMerge (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMerge2 (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMerge3 (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMergeWithPredicate (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMmTableCompaction (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMultiInsert (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMultiInsertStatement
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNoHistory (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidInsert (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion02
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion2
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion3
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testOpenTxnsCounter (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testOrcNoPPD (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testOrcPPD (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testOriginalFileReaderWhenNonAcidConvertedToAcid
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testUpdateMixedCase (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.testValidTxnsBookkeeping
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.updateDeletePartitioned (batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.writeBetweenWorkerAndCleaner
(batchId=307)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14740/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14740/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14740/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 41 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946796 - PreCommit-HIVE-Build
> Interaction between constraints and MV rewriting may create loop in Calcite
> planner
> ---
>
> Key: HIVE-20835
> URL: https://issues.apache.org/jira/browse/HIVE-20835
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Jesus
[jira] [Commented] (HIVE-20835) Interaction between constraints and MV rewriting may create loop in Calcite planner
[
https://issues.apache.org/jira/browse/HIVE-20835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674274#comment-16674274
]
Hive QA commented on HIVE-20835:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} itests/util in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
41s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 3 new + 170 unchanged - 2
fixed = 173 total (was 172) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14740/dev-support/hive-personality.sh
|
| git revision | master / 670e49f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14740/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14740/yetus/whitespace-eol.txt
|
| modules | C: itests itests/util ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14740/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Interaction between constraints and MV rewriting may create loop in Calcite
> planner
> ---
>
> Key: HIVE-20835
> URL: https://issues.apache.org/jira/browse/HIVE-20835
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20835.01.patch, HIVE-20835.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20817) Reading Timestamp datatype via HiveServer2 gives errors
[
https://issues.apache.org/jira/browse/HIVE-20817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674272#comment-16674272
]
Hive QA commented on HIVE-20817:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946784/HIVE-20817.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15524 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14739/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14739/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14739/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946784 - PreCommit-HIVE-Build
> Reading Timestamp datatype via HiveServer2 gives errors
> ---
>
> Key: HIVE-20817
> URL: https://issues.apache.org/jira/browse/HIVE-20817
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-20817.01.patch, HIVE-20817.02.patch
>
>
> CREATE TABLE JdbcBasicRead ( empno int, desg string,empname string,doj
> timestamp,Salary float,mgrid smallint, deptno tinyint ) ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ',';
> LOAD DATA LOCAL INPATH '/tmp/art_jdbc/hive/input/input_7columns.txt'
> OVERWRITE INTO TABLE JdbcBasicRead;
> Sample Data.
> —
> 7369,M,SMITH,1980-12-17 17:07:29.234234,5000.00,7902,20
> 7499,X,ALLEN,1981-02-20 17:07:29.234234,1250.00,7698,30
> 7521,X,WARD,1981-02-22 17:07:29.234234,01600.57,7698,40
> 7566,M,JONES,1981-04-02 17:07:29.234234,02975.65,7839,10
> 7654,X,MARTIN,1981-09-28 17:07:29.234234,01250.00,7698,20
> 7698,M,BLAKE,1981-05-01 17:07:29.234234,2850.98,7839,30
> 7782,M,CLARK,1981-06-09 17:07:29.234234,02450.00,7839,20
> —
> Select statement: SELECT empno, desg, empname, doj, salary, mgrid, deptno
> FROM JdbcBasicWrite
> {code}
> 2018-09-25T07:11:03,222 WARN [HiveServer2-Handler-Pool: Thread-83]:
> thrift.ThriftCLIService (:()) - Error fetching results:
> org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException:
> org.apache.hadoop.hive.common.type.Timestamp cannot be cast to
> java.sql.Timestamp
> at
> org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:469)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:328)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:910)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source) ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_112]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
> at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> ~[hadoop-common-3.1.1.3.0.1.0-187.jar:?]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at com.sun.proxy.$Proxy46.fetchResults(Unknown Source) ~[?:?]
> at org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:564)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:786)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1837)
>
[jira] [Commented] (HIVE-20817) Reading Timestamp datatype via HiveServer2 gives errors
[
https://issues.apache.org/jira/browse/HIVE-20817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674262#comment-16674262
]
Hive QA commented on HIVE-20817:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} service in master has 48 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14739/dev-support/hive-personality.sh
|
| git revision | master / 670e49f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: service U: service |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14739/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Reading Timestamp datatype via HiveServer2 gives errors
> ---
>
> Key: HIVE-20817
> URL: https://issues.apache.org/jira/browse/HIVE-20817
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-20817.01.patch, HIVE-20817.02.patch
>
>
> CREATE TABLE JdbcBasicRead ( empno int, desg string,empname string,doj
> timestamp,Salary float,mgrid smallint, deptno tinyint ) ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ',';
> LOAD DATA LOCAL INPATH '/tmp/art_jdbc/hive/input/input_7columns.txt'
> OVERWRITE INTO TABLE JdbcBasicRead;
> Sample Data.
> —
> 7369,M,SMITH,1980-12-17 17:07:29.234234,5000.00,7902,20
> 7499,X,ALLEN,1981-02-20 17:07:29.234234,1250.00,7698,30
> 7521,X,WARD,1981-02-22 17:07:29.234234,01600.57,7698,40
> 7566,M,JONES,1981-04-02 17:07:29.234234,02975.65,7839,10
> 7654,X,MARTIN,1981-09-28 17:07:29.234234,01250.00,7698,20
> 7698,M,BLAKE,1981-05-01 17:07:29.234234,2850.98,7839,30
> 7782,M,CLARK,1981-06-09 17:07:29.234234,02450.00,7839,20
> —
> Select statement: SELECT empno, desg, empname, doj, salary, mgrid, deptno
> FROM JdbcBasicWrite
> {code}
> 2018-09-25T07:11:03,222 WARN [HiveServer2-Handler-Pool: Thread-83]:
> thrift.ThriftCLIService (:()) - Error fetching results:
> org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException:
> org.apache.hadoop.hive.common.type.Timestamp cannot be cast to
>
[jira] [Commented] (HIVE-16839) Unbalanced calls to openTransaction/commitTransaction when alter the same partition concurrently
[
https://issues.apache.org/jira/browse/HIVE-16839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674261#comment-16674261
]
Hive QA commented on HIVE-16839:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946774/HIVE-16839.01.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14738/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14738/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14738/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12946774/HIVE-16839.01.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946774 - PreCommit-HIVE-Build
> Unbalanced calls to openTransaction/commitTransaction when alter the same
> partition concurrently
>
>
> Key: HIVE-16839
> URL: https://issues.apache.org/jira/browse/HIVE-16839
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.13.1, 1.1.0
>Reporter: Nemon Lou
>Assignee: Guang Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-16839.01.patch
>
>
> SQL to reproduce:
> prepare:
> {noformat}
> hdfs dfs -mkdir -p
> /hzsrc/external/writing_dc/ltgsm/16e7a9b2-21a1-3f4f-8061-bc3395281627
> 1,create external table tb_ltgsm_external (id int) PARTITIONED by (cp
> string,ld string);
> {noformat}
> open one beeline run these two sql many times
> {noformat} 2,ALTER TABLE tb_ltgsm_external ADD IF NOT EXISTS PARTITION
> (cp=2017060513,ld=2017060610);
> 3,ALTER TABLE tb_ltgsm_external PARTITION (cp=2017060513,ld=2017060610) SET
> LOCATION
> 'hdfs://hacluster/hzsrc/external/writing_dc/ltgsm/16e7a9b2-21a1-3f4f-8061-bc3395281627';
> {noformat}
> open another beeline to run this sql many times at the same time.
> {noformat}
> 4,ALTER TABLE tb_ltgsm_external DROP PARTITION (cp=2017060513,ld=2017060610);
> {noformat}
> MetaStore logs:
> {noformat}
> 2017-06-06 21:58:34,213 | ERROR | pool-6-thread-197 | Retrying HMSHandler
> after 2000 ms (attempt 1 of 10) with error:
> javax.jdo.JDOObjectNotFoundException: No such database row
> FailedObject:49[OID]org.apache.hadoop.hive.metastore.model.MStorageDescriptor
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:475)
> at
> org.datanucleus.api.jdo.JDOAdapter.getApiExceptionForNucleusException(JDOAdapter.java:1158)
> at
> org.datanucleus.state.JDOStateManager.isLoaded(JDOStateManager.java:3231)
> at
> org.apache.hadoop.hive.metastore.model.MStorageDescriptor.jdoGetcd(MStorageDescriptor.java)
> at
> org.apache.hadoop.hive.metastore.model.MStorageDescriptor.getCD(MStorageDescriptor.java:184)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.convertToStorageDescriptor(ObjectStore.java:1282)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.convertToStorageDescriptor(ObjectStore.java:1299)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.convertToPart(ObjectStore.java:1680)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getPartition(ObjectStore.java:1586)
> at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:98)
> at com.sun.proxy.$Proxy0.getPartition(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveAlterHandler.alterPartitions(HiveAlterHandler.java:538)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions(HiveMetaStore.java:3317)
> at sun.reflect.GeneratedMethodAccessor37.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:102)
> at com.sun.proxy.$Proxy12.alter_partitions(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions.getResult(ThriftHiveMetastore.java:9963)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partitions.getResult(ThriftHiveMetastore.java:9947)
> at
[jira] [Commented] (HIVE-20823) Make Compactor run in a transaction
[
https://issues.apache.org/jira/browse/HIVE-20823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674260#comment-16674260
]
Hive QA commented on HIVE-20823:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946771/HIVE-20823.03.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 95 failed/errored test(s), 15513 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=195)
[druidmini_dynamic_partition.q,druidmini_test_ts.q,druidmini_expressions.q,druidmini_test_alter.q,druidmini_test_insert.q]
org.apache.hadoop.hive.ql.TestTxnCommands2.testCleanerForTxnToWriteId
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2.writeBetweenWorkerAndCleaner
(batchId=307)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testCleanerForTxnToWriteId
(batchId=319)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.writeBetweenWorkerAndCleaner
(batchId=319)
org.apache.hadoop.hive.ql.TestTxnCommands3.testCleaner2 (batchId=320)
org.apache.hadoop.hive.ql.TestTxnCommandsForMmTable.testSnapshotIsolationWithAbortedTxnOnMmTable
(batchId=283)
org.apache.hadoop.hive.ql.TestTxnCommandsForOrcMmTable.testSnapshotIsolationWithAbortedTxnOnMmTable
(batchId=305)
org.apache.hadoop.hive.ql.io.TestAcidUtils.deleteDeltasWithOpenTxnInRead
(batchId=294)
org.apache.hadoop.hive.ql.io.TestAcidUtils.deltasAndDeleteDeltasWithOpenTxnsNotInCompact
(batchId=294)
org.apache.hadoop.hive.ql.io.TestAcidUtils.deltasWithOpenTxnInRead (batchId=294)
org.apache.hadoop.hive.ql.io.TestAcidUtils.deltasWithOpenTxnInRead2
(batchId=294)
org.apache.hadoop.hive.ql.io.TestAcidUtils.deltasWithOpenTxnsNotInCompact
(batchId=294)
org.apache.hadoop.hive.ql.io.TestAcidUtils.deltasWithOpenTxnsNotInCompact2
(batchId=294)
org.apache.hadoop.hive.ql.io.TestAcidUtils.testBaseDeltas (batchId=294)
org.apache.hadoop.hive.ql.io.TestAcidUtils.testBaseWithDeleteDeltas
(batchId=294)
org.apache.hadoop.hive.ql.io.TestAcidUtils.testMinorCompactedDeltaMakesInBetweenDelteDeltaObsolete
(batchId=294)
org.apache.hadoop.hive.ql.io.TestAcidUtils.testObsoleteOriginals (batchId=294)
org.apache.hadoop.hive.ql.io.TestAcidUtils.testOriginalDeltas (batchId=294)
org.apache.hadoop.hive.ql.io.TestAcidUtils.testOverlapingDelta (batchId=294)
org.apache.hadoop.hive.ql.io.TestAcidUtils.testOverlapingDelta2 (batchId=294)
org.apache.hadoop.hive.ql.io.TestAcidUtils.testOverlapingDeltaAndDeleteDelta
(batchId=294)
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testACIDReaderFooterSerializeWithDeltas
(batchId=297)
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testACIDReaderNoFooterSerializeWithDeltas
(batchId=297)
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testACIDSplitStrategy
(batchId=297)
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testACIDSplitStrategyForSplitUpdate
(batchId=297)
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testCombinationInputFormatWithAcid
(batchId=297)
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testEtlCombinedStrategy
(batchId=297)
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testFSCallsVectorizedOrcAcidRowBatchReader
(batchId=297)
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testVectorizationWithAcid
(batchId=297)
org.apache.hadoop.hive.ql.io.orc.TestOrcRawRecordMerger.testEmpty (batchId=298)
org.apache.hadoop.hive.ql.io.orc.TestOrcRawRecordMerger.testNewBaseAndDelta
(batchId=298)
org.apache.hadoop.hive.ql.io.orc.TestOrcRawRecordMerger.testRecordReaderDelta
(batchId=298)
org.apache.hadoop.hive.ql.io.orc.TestOrcRawRecordMerger.testRecordReaderIncompleteDelta
(batchId=298)
org.apache.hadoop.hive.ql.io.orc.TestOrcRawRecordMerger.testRecordReaderNewBaseAndDelta
(batchId=298)
org.apache.hadoop.hive.ql.io.orc.TestOrcRawRecordMerger.testRecordReaderOldBaseAndDelta
(batchId=298)
org.apache.hadoop.hive.ql.io.orc.TestVectorizedOrcAcidRowBatchReader.testDeleteEventFilteringOff
(batchId=297)
org.apache.hadoop.hive.ql.io.orc.TestVectorizedOrcAcidRowBatchReader.testDeleteEventFilteringOff2
(batchId=297)
org.apache.hadoop.hive.ql.io.orc.TestVectorizedOrcAcidRowBatchReader.testDeleteEventFilteringOn
(batchId=297)
org.apache.hadoop.hive.ql.io.orc.TestVectorizedOrcAcidRowBatchReader.testDeleteEventFilteringOn2
(batchId=297)
org.apache.hadoop.hive.ql.io.orc.TestVectorizedOrcAcidRowBatchReader.testDeleteEventOriginalFilteringOff
(batchId=297)
org.apache.hadoop.hive.ql.io.orc.TestVectorizedOrcAcidRowBatchReader.testDeleteEventOriginalFilteringOn
(batchId=297)
org.apache.hadoop.hive.ql.io.orc.TestVectorizedOrcAcidRowBatchReader.testVectorizedOrcAcidRowBatchReader
(batchId=297)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testAcidBootstrapReplLoadRetryAfterFailure
[jira] [Commented] (HIVE-20823) Make Compactor run in a transaction
[
https://issues.apache.org/jira/browse/HIVE-20823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674256#comment-16674256
]
Hive QA commented on HIVE-20823:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
23s{color} | {color:blue} storage-api in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
16s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
36s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
2s{color} | {color:blue} standalone-metastore/metastore-server in master has
185 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
49s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} storage-api: The patch generated 0 new + 16
unchanged - 2 fixed = 16 total (was 18) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
6s{color} | {color:green} The patch metastore-common passed checkstyle {color}
|
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
49s{color} | {color:red} ql: The patch generated 28 new + 1274 unchanged - 12
fixed = 1302 total (was 1286) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
6s{color} | {color:green} The patch metastore-server passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
55s{color} | {color:red} ql generated 1 new + 2314 unchanged - 1 fixed = 2315
total (was 2315) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
10s{color} | {color:red} standalone-metastore/metastore-server generated 1 new
+ 184 unchanged - 1 fixed = 185 total (was 185) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 15s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Exception is caught when Exception is not thrown in
org.apache.hadoop.hive.ql.txn.compactor.Cleaner.clean(CompactionInfo, long) At
Cleaner.java:is not thrown in
org.apache.hadoop.hive.ql.txn.compactor.Cleaner.clean(CompactionInfo, long) At
Cleaner.java:[line 203] |
| FindBugs | module:standalone-metastore/metastore-server |
| |
org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findMinOpenTxnGLB(Statement)
may fail to clean up java.sql.ResultSet Obligation to clean up resource
created at CompactionTxnHandler.java:up java.sql.ResultSet Obligation to clean
up resource created at CompactionTxnHandler.java:[line 356] is not
[jira] [Commented] (HIVE-20862) QueryId no longer shows up in the logs
[
https://issues.apache.org/jira/browse/HIVE-20862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674250#comment-16674250
]
Hive QA commented on HIVE-20862:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946769/HIVE-20862.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15522 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=196)
[druidmini_masking.q,druidmini_test1.q,druidkafkamini_basic.q,druidmini_joins.q,druid_timestamptz.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14736/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14736/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14736/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946769 - PreCommit-HIVE-Build
> QueryId no longer shows up in the logs
> --
>
> Key: HIVE-20862
> URL: https://issues.apache.org/jira/browse/HIVE-20862
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Affects Versions: 4.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-20862.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20839) "Cannot find field" error during dynamically partitioned hash join
[
https://issues.apache.org/jira/browse/HIVE-20839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-20839:
--
Attachment: HIVE-20839.4.patch
> "Cannot find field" error during dynamically partitioned hash join
> --
>
> Key: HIVE-20839
> URL: https://issues.apache.org/jira/browse/HIVE-20839
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-20839.1.patch, HIVE-20839.2.patch,
> HIVE-20839.3.patch, HIVE-20839.4.patch
>
>
> Occurs in some cases in the non-CBO optimized queries, either if CBO is
> disabled or has failed due to error.
> {noformat}
> 2018-10-11T04:40:22,724 ERROR [TezTR-85144_8944_1085_28_996_2
> (1539092085144_8944_1085_28_000996_2)] tez.ReduceRecordProcessor: Hit error
> while closing operators - failing tree
> 2018-10-11T04:40:22,724 ERROR [TezTR-85144_8944_1085_28_996_2
> (1539092085144_8944_1085_28_000996_2)] tez.TezProcessor:
> java.lang.RuntimeException: cannot find field _col304 from [0:_col0, 1:_col1,
> 2:_col2, 3:_col3, 4:_col4, 5:_col5, 6:_col6, 7:_col7, 8:_col8, 9:_col9,
> 10:_col10, 11:_col11, 12:_col12, 13:_col13, 14:_col15, 15:_col16, 16:_col17,
> 17:_col18, 18:_col19, 19:_col20, 20:_col21, 21:_col22, 22:_col23, 23:_col24,
> 24:_col25, 25:_col26, 26:_col27, 27:_col28, 28:_col29, 29:_col30, 30:_col31,
> 31:_col32, 32:_col33, 33:_col34, 34:_col35, 35:_col36, 36:_col37, 37:_col38,
> 38:_col39, 39:_col40, 40:_col41, 41:_col42, 42:_col43, 43:_col44, 44:_col45,
> 45:_col46, 46:_col47, 47:_col48, 48:_col49, 49:_col50, 50:_col51, 51:_col52,
> 52:_col53, 53:_col54, 54:_col55, 55:_col56, 56:_col57, 57:_col58, 58:_col59,
> 59:_col60, 60:_col61, 61:_col62, 62:_col63, 63:_col64, 64:_col65, 65:_col66,
> 66:_col67, 67:_col68, 68:_col70, 69:_col72, 70:_col73, 71:_col74, 72:_col75,
> 73:_col76, 74:_col77, 75:_col78, 76:_col79, 77:_col80, 78:_col81, 79:_col82,
> 80:_col83, 81:_col84, 82:_col85, 83:_col86, 84:_col87, 85:_col88, 86:_col89,
> 87:_col90, 88:_col91, 89:_col92, 90:_col93, 91:_col94, 92:_col95, 93:_col96,
> 94:_col97, 95:_col98, 96:_col99, 97:_col100, 98:_col101, 99:_col102,
> 100:_col103, 101:_col104, 102:_col105, 103:_col106, 104:_col107, 105:_col108,
> 106:_col109, 107:_col110, 108:_col111, 109:_col112, 110:_col113, 111:_col114,
> 112:_col115, 113:_col116, 114:_col117, 115:_col118, 116:_col119, 117:_col120,
> 118:_col121, 119:_col122, 120:_col123, 121:_col124, 122:_col125, 123:_col126,
> 124:_col127, 125:_col128, 126:_col129, 127:_col130, 128:_col131, 129:_col132,
> 130:_col133, 131:_col134, 132:_col135, 133:_col136, 134:_col137, 135:_col138,
> 136:_col139, 137:_col140, 138:_col141, 139:_col142, 140:_col143, 141:_col144,
> 142:_col145, 143:_col146, 144:_col147, 145:_col148, 146:_col149, 147:_col150,
> 148:_col151, 149:_col152, 150:_col153, 151:_col154, 152:_col155, 153:_col156,
> 154:_col157, 155:_col158, 156:_col159, 157:_col160, 158:_col161, 159:_col162,
> 160:_col163, 161:_col164, 162:_col165, 163:_col166, 164:_col167, 165:_col168,
> 166:_col169, 167:_col170, 168:_col171, 169:_col318]
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:485)
> at
> org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:153)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:80)
> at
> org.apache.hadoop.hive.ql.exec.JoinUtil.getObjectInspectorsFromEvaluators(JoinUtil.java:91)
> at
> org.apache.hadoop.hive.ql.exec.AbstractMapJoinOperator.initializeOp(AbstractMapJoinOperator.java:74)
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.initializeOp(MapJoinOperator.java:144)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:374)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:195)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:188)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:172)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20834) Hive QueryResultCache entries keeping reference to SemanticAnalyzer from cached query
[ https://issues.apache.org/jira/browse/HIVE-20834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-20834: -- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) I've committed this to master > Hive QueryResultCache entries keeping reference to SemanticAnalyzer from > cached query > - > > Key: HIVE-20834 > URL: https://issues.apache.org/jira/browse/HIVE-20834 > Project: Hive > Issue Type: Bug >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20834.1.patch, HIVE-20834.2.patch, > HIVE-20834.3.patch, dominator_tree.png > > > QueryResultCache.LookupInfo ends up keeping a reference to the > SemanticAnalyzer from the cached query, for as long as the cached entry is in > the cache. We should not be keeping the SemanticAnalyzer around after the > query is done executing since they can hold on to quite a bit of memory. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20853) Expose ShuffleHandler.registerDag in the llap daemon API
[ https://issues.apache.org/jira/browse/HIVE-20853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated HIVE-20853: --- Attachment: HIVE-20853.2.patch Status: Patch Available (was: Open) > Expose ShuffleHandler.registerDag in the llap daemon API > > > Key: HIVE-20853 > URL: https://issues.apache.org/jira/browse/HIVE-20853 > Project: Hive > Issue Type: Improvement > Components: llap >Affects Versions: 3.1.0 >Reporter: Jaume M >Assignee: Jaume M >Priority: Critical > Attachments: HIVE-20853.1.patch, HIVE-20853.2.patch > > > Currently DAGs are only registered when a submitWork is called for that DAG. > At this point the crendentials are added to the ShuffleHandler and it can > start serving. > However Tez might (and will) schedule tasks to fetch from the ShuffleHandler > before anything of this happens and all this tasks will fail which may > results in the query failing. > This happens in the scenario in which a LlapDaemon just comes up and tez > fetchers try to open a connection before a DAG has been registered. > Adding this API will allow to register the DAG against the Daemon when the AM > notices that a new Daemon is up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20853) Expose ShuffleHandler.registerDag in the llap daemon API
[ https://issues.apache.org/jira/browse/HIVE-20853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated HIVE-20853: --- Status: Open (was: Patch Available) > Expose ShuffleHandler.registerDag in the llap daemon API > > > Key: HIVE-20853 > URL: https://issues.apache.org/jira/browse/HIVE-20853 > Project: Hive > Issue Type: Improvement > Components: llap >Affects Versions: 3.1.0 >Reporter: Jaume M >Assignee: Jaume M >Priority: Critical > Attachments: HIVE-20853.1.patch > > > Currently DAGs are only registered when a submitWork is called for that DAG. > At this point the crendentials are added to the ShuffleHandler and it can > start serving. > However Tez might (and will) schedule tasks to fetch from the ShuffleHandler > before anything of this happens and all this tasks will fail which may > results in the query failing. > This happens in the scenario in which a LlapDaemon just comes up and tez > fetchers try to open a connection before a DAG has been registered. > Adding this API will allow to register the DAG against the Daemon when the AM > notices that a new Daemon is up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20853) Expose ShuffleHandler.registerDag in the llap daemon API
[ https://issues.apache.org/jira/browse/HIVE-20853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated HIVE-20853: --- Attachment: (was: HIVE-20853.2.patch) > Expose ShuffleHandler.registerDag in the llap daemon API > > > Key: HIVE-20853 > URL: https://issues.apache.org/jira/browse/HIVE-20853 > Project: Hive > Issue Type: Improvement > Components: llap >Affects Versions: 3.1.0 >Reporter: Jaume M >Assignee: Jaume M >Priority: Critical > Attachments: HIVE-20853.1.patch > > > Currently DAGs are only registered when a submitWork is called for that DAG. > At this point the crendentials are added to the ShuffleHandler and it can > start serving. > However Tez might (and will) schedule tasks to fetch from the ShuffleHandler > before anything of this happens and all this tasks will fail which may > results in the query failing. > This happens in the scenario in which a LlapDaemon just comes up and tez > fetchers try to open a connection before a DAG has been registered. > Adding this API will allow to register the DAG against the Daemon when the AM > notices that a new Daemon is up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20862) QueryId no longer shows up in the logs
[
https://issues.apache.org/jira/browse/HIVE-20862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674245#comment-16674245
]
Hive QA commented on HIVE-20862:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
18s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
48s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
40s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 1 new + 205 unchanged - 0
fixed = 206 total (was 205) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 50s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14736/dev-support/hive-personality.sh
|
| git revision | master / bb77751 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14736/yetus/diff-checkstyle-ql.txt
|
| modules | C: standalone-metastore/metastore-common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14736/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> QueryId no longer shows up in the logs
> --
>
> Key: HIVE-20862
> URL: https://issues.apache.org/jira/browse/HIVE-20862
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Affects Versions: 4.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-20862.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20863) remove dead code
[
https://issues.apache.org/jira/browse/HIVE-20863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674238#comment-16674238
]
Hive QA commented on HIVE-20863:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946770/HIVE-20863.01.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14735/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14735/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14735/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-11-03 23:22:38.891
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-14735/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-11-03 23:22:38.894
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at bb77751 HIVE-20815: JdbcRecordReader.next shall not eat
exception (Daniel Dai, reviewed by Thejas Nair)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at bb77751 HIVE-20815: JdbcRecordReader.next shall not eat
exception (Daniel Dai, reviewed by Thejas Nair)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-11-03 23:22:39.993
+ rm -rf ../yetus_PreCommit-HIVE-Build-14735
+ mkdir ../yetus_PreCommit-HIVE-Build-14735
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-14735
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-14735/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: git apply -p0
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc8990296713408484449.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc8990296713408484449.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
protoc-jar: executing: [/tmp/protoc6732864523836172093.exe, --version]
libprotoc 2.5.0
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/parser/Filter.g
org/apache/hadoop/hive/metastore/parser/Filter.g
log4j:WARN No appenders could be found for logger (DataNucleus.Persistence).
log4j:WARN Please initialize the log4j system properly.
DataNucleus Enhancer (version 4.1.17) for API "JDO"
DataNucleus Enhancer completed with success for 41 classes.
[ERROR] COMPILATION ERROR :
[ERROR]
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStoreTxns.java:[193,18]
cannot find symbol
symbol: method isTxnRangeValid(long,long)
location: variable validTxns of type
org.apache.hadoop.hive.common.ValidTxnList
[ERROR]
[jira] [Commented] (HIVE-20842) Fix logic introduced in HIVE-20660 to estimate statistics for group by
[
https://issues.apache.org/jira/browse/HIVE-20842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674237#comment-16674237
]
Hive QA commented on HIVE-20842:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946765/HIVE-20842.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14734/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14734/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14734/
Messages:
{noformat}
This message was trimmed, see log for full details
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-11-03 23:21:26.758
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-14734/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-11-03 23:21:26.762
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
ae1eb15..bb77751 master -> origin/master
+ git reset --hard HEAD
HEAD is now at ae1eb15 HIVE-20778: Join reordering may not be triggered if all
joins in plan are created by decorrelation logic (Vineet Garg via Jesus Camacho
Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at bb77751 HIVE-20815: JdbcRecordReader.next shall not eat
exception (Daniel Dai, reviewed by Thejas Nair)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-11-03 23:21:27.781
+ rm -rf ../yetus_PreCommit-HIVE-Build-14734
+ mkdir ../yetus_PreCommit-HIVE-Build-14734
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-14734
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-14734/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error:
a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java:
does not exist in index
error: a/ql/src/test/results/clientpositive/annotate_stats_groupby.q.out: does
not exist in index
error:
a/ql/src/test/results/clientpositive/cbo_rp_annotate_stats_groupby.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/bucket_groupby.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/bucket_map_join_tez2.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/check_constraint.q.out: does
not exist in index
error:
a/ql/src/test/results/clientpositive/llap/constraints_optimization.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/correlationoptimizer1.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/correlationoptimizer6.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/llap/dynpart_sort_optimization2.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/llap/enforce_constraint_notnull.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/except_distinct.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/explainanalyze_2.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/explainuser_1.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/explainuser_2.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/explainuser_4.q.out: does
[jira] [Commented] (HIVE-20853) Expose ShuffleHandler.registerDag in the llap daemon API
[
https://issues.apache.org/jira/browse/HIVE-20853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674236#comment-16674236
]
Hive QA commented on HIVE-20853:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946763/HIVE-20853.1.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 75 failed/errored test(s), 14676 tests
executed
*Failed tests:*
{noformat}
TestLlapDaemonUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=332)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
(batchId=194)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
(batchId=196)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
(batchId=183)
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testChangeGuaranteedTotal
(batchId=278)
[jira] [Updated] (HIVE-20853) Expose ShuffleHandler.registerDag in the llap daemon API
[ https://issues.apache.org/jira/browse/HIVE-20853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated HIVE-20853: --- Attachment: HIVE-20853.2.patch Status: Patch Available (was: Open) > Expose ShuffleHandler.registerDag in the llap daemon API > > > Key: HIVE-20853 > URL: https://issues.apache.org/jira/browse/HIVE-20853 > Project: Hive > Issue Type: Improvement > Components: llap >Affects Versions: 3.1.0 >Reporter: Jaume M >Assignee: Jaume M >Priority: Critical > Attachments: HIVE-20853.1.patch, HIVE-20853.2.patch > > > Currently DAGs are only registered when a submitWork is called for that DAG. > At this point the crendentials are added to the ShuffleHandler and it can > start serving. > However Tez might (and will) schedule tasks to fetch from the ShuffleHandler > before anything of this happens and all this tasks will fail which may > results in the query failing. > This happens in the scenario in which a LlapDaemon just comes up and tez > fetchers try to open a connection before a DAG has been registered. > Adding this API will allow to register the DAG against the Daemon when the AM > notices that a new Daemon is up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20853) Expose ShuffleHandler.registerDag in the llap daemon API
[ https://issues.apache.org/jira/browse/HIVE-20853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated HIVE-20853: --- Status: Open (was: Patch Available) > Expose ShuffleHandler.registerDag in the llap daemon API > > > Key: HIVE-20853 > URL: https://issues.apache.org/jira/browse/HIVE-20853 > Project: Hive > Issue Type: Improvement > Components: llap >Affects Versions: 3.1.0 >Reporter: Jaume M >Assignee: Jaume M >Priority: Critical > Attachments: HIVE-20853.1.patch, HIVE-20853.2.patch > > > Currently DAGs are only registered when a submitWork is called for that DAG. > At this point the crendentials are added to the ShuffleHandler and it can > start serving. > However Tez might (and will) schedule tasks to fetch from the ShuffleHandler > before anything of this happens and all this tasks will fail which may > results in the query failing. > This happens in the scenario in which a LlapDaemon just comes up and tez > fetchers try to open a connection before a DAG has been registered. > Adding this API will allow to register the DAG against the Daemon when the AM > notices that a new Daemon is up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20853) Expose ShuffleHandler.registerDag in the llap daemon API
[
https://issues.apache.org/jira/browse/HIVE-20853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674233#comment-16674233
]
Hive QA commented on HIVE-20853:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} llap-common in master has 71 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
22s{color} | {color:blue} llap-client in master has 26 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
23s{color} | {color:blue} llap-tez in master has 17 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} llap-server in master has 83 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
18s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
15s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 15s{color}
| {color:red} llap-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
9s{color} | {color:red} llap-client: The patch generated 1 new + 6 unchanged -
0 fixed = 7 total (was 6) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
9s{color} | {color:red} llap-tez: The patch generated 5 new + 113 unchanged - 7
fixed = 118 total (was 120) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} llap-server: The patch generated 34 new + 157
unchanged - 0 fixed = 191 total (was 157) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 5 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
34s{color} | {color:red} llap-common generated 5 new + 71 unchanged - 0 fixed =
76 total (was 71) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
17s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 36s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:llap-common |
| |
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$RegisterDagRequestProto.PARSER
isn't final but should be At LlapDaemonProtocolProtos.java:be At
LlapDaemonProtocolProtos.java:[line 11608] |
| | Class
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$RegisterDagRequestProto
defines non-transient non-serializable instance field unknownFields In
LlapDaemonProtocolProtos.java:instance field unknownFields In
LlapDaemonProtocolProtos.java |
| |
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$RegisterDagResponseProto.PARSER
isn't final but should be At LlapDaemonProtocolProtos.java:be At
LlapDaemonProtocolProtos.java:[line 12356] |
| | Class
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$RegisterDagResponseProto
[jira] [Updated] (HIVE-20815) JdbcRecordReader.next shall not eat exception
[ https://issues.apache.org/jira/browse/HIVE-20815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-20815: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Patch pushed to master. Thanks Thejas for review! > JdbcRecordReader.next shall not eat exception > - > > Key: HIVE-20815 > URL: https://issues.apache.org/jira/browse/HIVE-20815 > Project: Hive > Issue Type: Bug > Components: StorageHandler >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20815.1.patch, HIVE-20815.2.patch, > HIVE-20815.3.patch, HIVE-20815.4.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-16839) Unbalanced calls to openTransaction/commitTransaction when alter the same partition concurrently
[
https://issues.apache.org/jira/browse/HIVE-16839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674230#comment-16674230
]
Hive QA commented on HIVE-16839:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946774/HIVE-16839.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15524 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestObjectStore.testConcurrentDropPartitions
(batchId=231)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14732/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14732/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14732/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946774 - PreCommit-HIVE-Build
> Unbalanced calls to openTransaction/commitTransaction when alter the same
> partition concurrently
>
>
> Key: HIVE-16839
> URL: https://issues.apache.org/jira/browse/HIVE-16839
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.13.1, 1.1.0
>Reporter: Nemon Lou
>Assignee: Guang Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-16839.01.patch
>
>
> SQL to reproduce:
> prepare:
> {noformat}
> hdfs dfs -mkdir -p
> /hzsrc/external/writing_dc/ltgsm/16e7a9b2-21a1-3f4f-8061-bc3395281627
> 1,create external table tb_ltgsm_external (id int) PARTITIONED by (cp
> string,ld string);
> {noformat}
> open one beeline run these two sql many times
> {noformat} 2,ALTER TABLE tb_ltgsm_external ADD IF NOT EXISTS PARTITION
> (cp=2017060513,ld=2017060610);
> 3,ALTER TABLE tb_ltgsm_external PARTITION (cp=2017060513,ld=2017060610) SET
> LOCATION
> 'hdfs://hacluster/hzsrc/external/writing_dc/ltgsm/16e7a9b2-21a1-3f4f-8061-bc3395281627';
> {noformat}
> open another beeline to run this sql many times at the same time.
> {noformat}
> 4,ALTER TABLE tb_ltgsm_external DROP PARTITION (cp=2017060513,ld=2017060610);
> {noformat}
> MetaStore logs:
> {noformat}
> 2017-06-06 21:58:34,213 | ERROR | pool-6-thread-197 | Retrying HMSHandler
> after 2000 ms (attempt 1 of 10) with error:
> javax.jdo.JDOObjectNotFoundException: No such database row
> FailedObject:49[OID]org.apache.hadoop.hive.metastore.model.MStorageDescriptor
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:475)
> at
> org.datanucleus.api.jdo.JDOAdapter.getApiExceptionForNucleusException(JDOAdapter.java:1158)
> at
> org.datanucleus.state.JDOStateManager.isLoaded(JDOStateManager.java:3231)
> at
> org.apache.hadoop.hive.metastore.model.MStorageDescriptor.jdoGetcd(MStorageDescriptor.java)
> at
> org.apache.hadoop.hive.metastore.model.MStorageDescriptor.getCD(MStorageDescriptor.java:184)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.convertToStorageDescriptor(ObjectStore.java:1282)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.convertToStorageDescriptor(ObjectStore.java:1299)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.convertToPart(ObjectStore.java:1680)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getPartition(ObjectStore.java:1586)
> at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:98)
> at com.sun.proxy.$Proxy0.getPartition(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveAlterHandler.alterPartitions(HiveAlterHandler.java:538)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions(HiveMetaStore.java:3317)
> at sun.reflect.GeneratedMethodAccessor37.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:102)
> at com.sun.proxy.$Proxy12.alter_partitions(Unknown Source)
> at
>
[jira] [Commented] (HIVE-14269) Performance optimizations for data on S3
[ https://issues.apache.org/jira/browse/HIVE-14269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674223#comment-16674223 ] RACHIT CHAUHAN commented on HIVE-14269: --- Hi [~spena]: Since so many inter-related issues have been created around this thing, it's becoming difficult for new members looking at it. A summarized version of what this Jira issue has been able to achieve till date will help a lot. Can you kindly summarize about how we can optimize S3 writes for HIVE tables ? > Performance optimizations for data on S3 > > > Key: HIVE-14269 > URL: https://issues.apache.org/jira/browse/HIVE-14269 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.1.0 >Reporter: Sergio Peña >Assignee: Sergio Peña >Priority: Major > > Working with tables that resides on Amazon S3 (or any other object store) > have several performance impact when reading or writing data, and also > consistency issues. > This JIRA is an umbrella task to monitor all the performance improvements > that can be done in Hive to work better with S3 data. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-16839) Unbalanced calls to openTransaction/commitTransaction when alter the same partition concurrently
[
https://issues.apache.org/jira/browse/HIVE-16839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674214#comment-16674214
]
Hive QA commented on HIVE-16839:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
6s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
5s{color} | {color:blue} standalone-metastore/metastore-server in master has
185 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14732/dev-support/hive-personality.sh
|
| git revision | master / ae1eb15 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14732/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Unbalanced calls to openTransaction/commitTransaction when alter the same
> partition concurrently
>
>
> Key: HIVE-16839
> URL: https://issues.apache.org/jira/browse/HIVE-16839
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.13.1, 1.1.0
>Reporter: Nemon Lou
>Assignee: Guang Yang
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-16839.01.patch
>
>
> SQL to reproduce:
> prepare:
> {noformat}
> hdfs dfs -mkdir -p
> /hzsrc/external/writing_dc/ltgsm/16e7a9b2-21a1-3f4f-8061-bc3395281627
> 1,create external table tb_ltgsm_external (id int) PARTITIONED by (cp
> string,ld string);
> {noformat}
> open one beeline run these two sql many times
> {noformat} 2,ALTER TABLE tb_ltgsm_external ADD IF NOT EXISTS PARTITION
> (cp=2017060513,ld=2017060610);
> 3,ALTER TABLE tb_ltgsm_external PARTITION (cp=2017060513,ld=2017060610) SET
> LOCATION
> 'hdfs://hacluster/hzsrc/external/writing_dc/ltgsm/16e7a9b2-21a1-3f4f-8061-bc3395281627';
> {noformat}
> open another beeline to run this sql many times at the same time.
> {noformat}
> 4,ALTER TABLE tb_ltgsm_external DROP PARTITION (cp=2017060513,ld=2017060610);
> {noformat}
> MetaStore logs:
> {noformat}
> 2017-06-06 21:58:34,213 | ERROR | pool-6-thread-197 | Retrying HMSHandler
> after 2000 ms (attempt 1 of 10) with error:
> javax.jdo.JDOObjectNotFoundException: No such database row
>
[jira] [Commented] (HIVE-20850) Push case conditional from projections to dimension tables if possible
[
https://issues.apache.org/jira/browse/HIVE-20850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674206#comment-16674206
]
Hive QA commented on HIVE-20850:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946748/HIVE-20850.01.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14731/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14731/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14731/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-11-03 21:00:26.489
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-14731/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-11-03 21:00:26.492
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at ae1eb15 HIVE-20778: Join reordering may not be triggered if all
joins in plan are created by decorrelation logic (Vineet Garg via Jesus Camacho
Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at ae1eb15 HIVE-20778: Join reordering may not be triggered if all
joins in plan are created by decorrelation logic (Vineet Garg via Jesus Camacho
Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-11-03 21:00:27.167
+ rm -rf ../yetus_PreCommit-HIVE-Build-14731
+ mkdir ../yetus_PreCommit-HIVE-Build-14731
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-14731
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-14731/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java:2065
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java' cleanly.
error: patch failed:
ql/src/test/results/clientpositive/perf/tez/cbo_query15.q.out:49
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/perf/tez/cbo_query15.q.out' with conflicts.
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:219: trailing whitespace.
group by ss_store_sk,
/data/hiveptest/working/scratch/build.patch:226: trailing whitespace.
from (select ss_item_sk item_sk,avg(ss_net_profit) rank_col
/data/hiveptest/working/scratch/build.patch:250: trailing whitespace.
where asceding.rnk = descending.rnk
/data/hiveptest/working/scratch/build.patch:5119: trailing whitespace.
group by ss_store_sk,
/data/hiveptest/working/scratch/build.patch:5130: trailing whitespace.
group by ss_store_sk,
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java:2065
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java' cleanly.
error: patch failed:
ql/src/test/results/clientpositive/perf/tez/cbo_query15.q.out:49
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/perf/tez/cbo_query15.q.out' with conflicts.
U ql/src/test/results/clientpositive/perf/tez/cbo_query15.q.out
warning: squelched 5 whitespace errors
warning: 10 lines add whitespace errors.
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-14731
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946748 - PreCommit-HIVE-Build
> Push case conditional from projections to dimension tables if
[jira] [Commented] (HIVE-20740) Remove global lock in ObjectStore.setConf method
[
https://issues.apache.org/jira/browse/HIVE-20740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674205#comment-16674205
]
Hive QA commented on HIVE-20740:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946744/HIVE-20740.09.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15524 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestDDLWithRemoteMetastoreSecondNamenode.testAlterPartitionSetLocationNonDefaultNameNode
(batchId=244)
org.apache.hadoop.hive.ql.TestDDLWithRemoteMetastoreSecondNamenode.testCreateDatabaseWithTableNonDefaultNameNode
(batchId=244)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14730/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14730/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14730/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946744 - PreCommit-HIVE-Build
> Remove global lock in ObjectStore.setConf method
>
>
> Key: HIVE-20740
> URL: https://issues.apache.org/jira/browse/HIVE-20740
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-20740.01.patch, HIVE-20740.02.patch,
> HIVE-20740.04.patch, HIVE-20740.05.patch, HIVE-20740.06.patch,
> HIVE-20740.08.patch, HIVE-20740.09.patch
>
>
> The ObjectStore#setConf method has a global lock which can block other
> clients in concurrent workloads.
> {code}
> @Override
> @SuppressWarnings("nls")
> public void setConf(Configuration conf) {
> // Although an instance of ObjectStore is accessed by one thread, there
> may
> // be many threads with ObjectStore instances. So the static variables
> // pmf and prop need to be protected with locks.
> pmfPropLock.lock();
> try {
> isInitialized = false;
> this.conf = conf;
> this.areTxnStatsSupported = MetastoreConf.getBoolVar(conf,
> ConfVars.HIVE_TXN_STATS_ENABLED);
> configureSSL(conf);
> Properties propsFromConf = getDataSourceProps(conf);
> boolean propsChanged = !propsFromConf.equals(prop);
> if (propsChanged) {
> if (pmf != null){
> clearOutPmfClassLoaderCache(pmf);
> if (!forTwoMetastoreTesting) {
> // close the underlying connection pool to avoid leaks
> pmf.close();
> }
> }
> pmf = null;
> prop = null;
> }
> assert(!isActiveTransaction());
> shutdown();
> // Always want to re-create pm as we don't know if it were created by
> the
> // most recent instance of the pmf
> pm = null;
> directSql = null;
> expressionProxy = null;
> openTrasactionCalls = 0;
> currentTransaction = null;
> transactionStatus = TXN_STATUS.NO_STATE;
> initialize(propsFromConf);
> String partitionValidationRegex =
> MetastoreConf.getVar(this.conf,
> ConfVars.PARTITION_NAME_WHITELIST_PATTERN);
> if (partitionValidationRegex != null &&
> !partitionValidationRegex.isEmpty()) {
> partitionValidationPattern =
> Pattern.compile(partitionValidationRegex);
> } else {
> partitionValidationPattern = null;
> }
> // Note, if metrics have not been initialized this will return null,
> which means we aren't
> // using metrics. Thus we should always check whether this is non-null
> before using.
> MetricRegistry registry = Metrics.getRegistry();
> if (registry != null) {
> directSqlErrors =
> Metrics.getOrCreateCounter(MetricsConstants.DIRECTSQL_ERRORS);
> }
> this.batchSize = MetastoreConf.getIntVar(conf,
> ConfVars.RAWSTORE_PARTITION_BATCH_SIZE);
> if (!isInitialized) {
> throw new RuntimeException(
> "Unable to create persistence manager. Check dss.log for details");
> } else {
> LOG.debug("Initialized ObjectStore");
> }
> } finally {
> pmfPropLock.unlock();
> }
> }
> {code}
> The {{pmfPropLock}} is a static object and it disallows any other new
> connection to HMS which is trying to instantiate ObjectStore. We should
> either remove the lock or reduce the scope of the lock so that it is held for
> a very small amount of
[jira] [Commented] (HIVE-20740) Remove global lock in ObjectStore.setConf method
[
https://issues.apache.org/jira/browse/HIVE-20740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674188#comment-16674188
]
Hive QA commented on HIVE-20740:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
45s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
4s{color} | {color:blue} standalone-metastore/metastore-server in master has
185 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
34s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
31s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} itests/hive-unit: The patch generated 1 new + 608
unchanged - 0 fixed = 609 total (was 608) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
8s{color} | {color:red} standalone-metastore/metastore-server generated 2 new +
183 unchanged - 2 fixed = 185 total (was 185) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 50s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
| | Dead store to thislimit in
org.apache.hadoop.hive.metastore.ObjectStore.getCallStack() At
ObjectStore.java:org.apache.hadoop.hive.metastore.ObjectStore.getCallStack()
At ObjectStore.java:[line 9522] |
| |
org.apache.hadoop.hive.metastore.PersistenceManagerProvider.updatePmfProperties(Configuration)
does not release lock on all paths At PersistenceManagerProvider.java:on all
paths At PersistenceManagerProvider.java:[line 151] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14730/dev-support/hive-personality.sh
|
| git revision | master / ae1eb15 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14730/yetus/patch-mvninstall-itests_hive-unit.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14730/yetus/patch-mvninstall-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14730/yetus/diff-checkstyle-itests_hive-unit.txt
|
| findbugs |
[jira] [Commented] (HIVE-20855) Clean up checkstyle & other warnings for CachedStore
[
https://issues.apache.org/jira/browse/HIVE-20855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674181#comment-16674181
]
Hive QA commented on HIVE-20855:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946721/HIVE-20855.2.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15523 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.security.TestLlapSignerImpl.testSigning
(batchId=332)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14728/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14728/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14728/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946721 - PreCommit-HIVE-Build
> Clean up checkstyle & other warnings for CachedStore
>
>
> Key: HIVE-20855
> URL: https://issues.apache.org/jira/browse/HIVE-20855
> Project: Hive
> Issue Type: Improvement
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
> Attachments: HIVE-20855.1.patch, HIVE-20855.2.patch
>
>
> While documenting the CachedStore in the Wiki I found a bunch of warnings,
> inconsistencies etc. in CachedStore, RawStore and a few other related classes.
> This is a patch that cleans them up. It's almost whitespace-only.
> It does not contain semantic code changes, it does contain two or three
> deprecations in favor of renamed methods or fields.
> This is going to be boring to review so according to the bylaws I'll ask for
> feedback on the mailing list and will commit after a week or so.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20855) Clean up checkstyle & other warnings for CachedStore
[
https://issues.apache.org/jira/browse/HIVE-20855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674160#comment-16674160
]
Hive QA commented on HIVE-20855:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
6s{color} | {color:blue} standalone-metastore/metastore-server in master has
185 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
18s{color} | {color:red} hcatalog-unit in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
9s{color} | {color:red} standalone-metastore/metastore-server generated 2 new +
183 unchanged - 2 fixed = 185 total (was 185) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} hcatalog-unit in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} standalone-metastore_metastore-server generated 0
new + 33 unchanged - 15 fixed = 33 total (was 48) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 41s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
| | Null pointer dereference of partNames in
org.apache.hadoop.hive.metastore.ObjectStore.getAggrStatsFor(String, String,
String, List, List, String) Dereferenced at ObjectStore.java:in
org.apache.hadoop.hive.metastore.ObjectStore.getAggrStatsFor(String, String,
String, List, List, String) Dereferenced at ObjectStore.java:[line 9067] |
| | Load of known null value in
org.apache.hadoop.hive.metastore.ObjectStore.getAggrStatsFor(String, String,
String, List, List, String) At ObjectStore.java:in
org.apache.hadoop.hive.metastore.ObjectStore.getAggrStatsFor(String, String,
String, List, List, String) At ObjectStore.java:[line 9067] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14728/dev-support/hive-personality.sh
|
| git revision | master / ae1eb15 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14728/yetus/patch-mvninstall-itests_hcatalog-unit.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14728/yetus/new-findbugs-standalone-metastore_metastore-server.html
|
| modules | C: itests/hcatalog-unit standalone-metastore/metastore-server U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14728/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Clean up checkstyle &
[jira] [Commented] (HIVE-20815) JdbcRecordReader.next shall not eat exception
[
https://issues.apache.org/jira/browse/HIVE-20815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674150#comment-16674150
]
Hive QA commented on HIVE-20815:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946719/HIVE-20815.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15524 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14727/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14727/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14727/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946719 - PreCommit-HIVE-Build
> JdbcRecordReader.next shall not eat exception
> -
>
> Key: HIVE-20815
> URL: https://issues.apache.org/jira/browse/HIVE-20815
> Project: Hive
> Issue Type: Bug
> Components: StorageHandler
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-20815.1.patch, HIVE-20815.2.patch,
> HIVE-20815.3.patch, HIVE-20815.4.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20835) Interaction between constraints and MV rewriting may create loop in Calcite planner
[ https://issues.apache.org/jira/browse/HIVE-20835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674148#comment-16674148 ] Jesus Camacho Rodriguez commented on HIVE-20835: [~kgyrtkirk], [~ashutoshc], could you take a look? https://reviews.apache.org/r/69244/ It seems we may miss some opportunities to push down expressions when there are Project operators already in the Join inputs, however this fixes the issue with the loop in the planner. Probably we can tackle the extension in a follow-up. > Interaction between constraints and MV rewriting may create loop in Calcite > planner > --- > > Key: HIVE-20835 > URL: https://issues.apache.org/jira/browse/HIVE-20835 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20835.01.patch, HIVE-20835.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20835) Interaction between constraints and MV rewriting may create loop in Calcite planner
[ https://issues.apache.org/jira/browse/HIVE-20835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20835: --- Attachment: HIVE-20835.01.patch > Interaction between constraints and MV rewriting may create loop in Calcite > planner > --- > > Key: HIVE-20835 > URL: https://issues.apache.org/jira/browse/HIVE-20835 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20835.01.patch, HIVE-20835.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20815) JdbcRecordReader.next shall not eat exception
[
https://issues.apache.org/jira/browse/HIVE-20815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674139#comment-16674139
]
Hive QA commented on HIVE-20815:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
21s{color} | {color:blue} jdbc-handler in master has 12 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
48s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14727/dev-support/hive-personality.sh
|
| git revision | master / ae1eb15 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: jdbc-handler ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14727/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> JdbcRecordReader.next shall not eat exception
> -
>
> Key: HIVE-20815
> URL: https://issues.apache.org/jira/browse/HIVE-20815
> Project: Hive
> Issue Type: Bug
> Components: StorageHandler
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-20815.1.patch, HIVE-20815.2.patch,
> HIVE-20815.3.patch, HIVE-20815.4.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20833) package.jdo needs to be updated to conform with HIVE-20221 changes
[
https://issues.apache.org/jira/browse/HIVE-20833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674126#comment-16674126
]
Hive QA commented on HIVE-20833:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946717/HIVE-20833.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15520 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge_diff_fs]
(batchId=189)
org.apache.hive.jdbc.TestJdbcDriver2.testSelectExecAsync2 (batchId=259)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14726/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14726/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14726/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946717 - PreCommit-HIVE-Build
> package.jdo needs to be updated to conform with HIVE-20221 changes
> --
>
> Key: HIVE-20833
> URL: https://issues.apache.org/jira/browse/HIVE-20833
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20833.1.patch, HIVE-20833.2.patch,
> HIVE-20833.3.patch, HIVE-20833.4.patch
>
>
> Following test if run with TestMiniLlapLocalCliDriver will fail:
> {code:sql}
> CREATE TABLE `alterPartTbl`(
>`po_header_id` bigint,
>`vendor_num` string,
>`requester_name` string,
>`approver_name` string,
>`buyer_name` string,
>`preparer_name` string,
>`po_requisition_number` string,
>`po_requisition_id` bigint,
>`po_requisition_desc` string,
>`rate_type` string,
>`rate_date` date,
>`rate` double,
>`blanket_total_amount` double,
>`authorization_status` string,
>`revision_num` bigint,
>`revised_date` date,
>`approved_flag` string,
>`approved_date` timestamp,
>`amount_limit` double,
>`note_to_authorizer` string,
>`note_to_vendor` string,
>`note_to_receiver` string,
>`vendor_order_num` string,
>`comments` string,
>`acceptance_required_flag` string,
>`acceptance_due_date` date,
>`closed_date` timestamp,
>`user_hold_flag` string,
>`approval_required_flag` string,
>`cancel_flag` string,
>`firm_status_lookup_code` string,
>`firm_date` date,
>`frozen_flag` string,
>`closed_code` string,
>`org_id` bigint,
>`reference_num` string,
>`wf_item_type` string,
>`wf_item_key` string,
>`submit_date` date,
>`sap_company_code` string,
>`sap_fiscal_year` bigint,
>`po_number` string,
>`sap_line_item` bigint,
>`closed_status_flag` string,
>`balancing_segment` string,
>`cost_center_segment` string,
>`base_amount_limit` double,
>`base_blanket_total_amount` double,
>`base_open_amount` double,
>`base_ordered_amount` double,
>`cancel_date` timestamp,
>`cbc_accounting_date` date,
>`change_requested_by` string,
>`change_summary` string,
>`confirming_order_flag` string,
>`document_creation_method` string,
>`edi_processed_flag` string,
>`edi_processed_status` string,
>`enabled_flag` string,
>`encumbrance_required_flag` string,
>`end_date` date,
>`end_date_active` date,
>`from_header_id` bigint,
>`from_type_lookup_code` string,
>`global_agreement_flag` string,
>`government_context` string,
>`interface_source_code` string,
>`ledger_currency_code` string,
>`open_amount` double,
>`ordered_amount` double,
>`pay_on_code` string,
>`payment_term_name` string,
>`pending_signature_flag` string,
>`po_revision_num` double,
>`preparer_id` bigint,
>`price_update_tolerance` double,
>`print_count` double,
>`printed_date` date,
>`reply_date` date,
>`reply_method_lookup_code` string,
>`rfq_close_date` date,
>`segment2` string,
>`segment3` string,
>`segment4` string,
>`segment5` string,
>`shipping_control` string,
>`start_date` date,
>`start_date_active` date,
>`summary_flag` string,
>`supply_agreement_flag` string,
>`usd_amount_limit` double,
>`usd_blanket_total_amount` double,
>`usd_exchange_rate` double,
>`usd_open_amount` double,
>`usd_order_amount`
[jira] [Commented] (HIVE-20833) package.jdo needs to be updated to conform with HIVE-20221 changes
[
https://issues.apache.org/jira/browse/HIVE-20833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674110#comment-16674110
]
Hive QA commented on HIVE-20833:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
44s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
4s{color} | {color:blue} standalone-metastore/metastore-server in master has
185 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14726/dev-support/hive-personality.sh
|
| git revision | master / f3fab45 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: itests ql standalone-metastore/metastore-server U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14726/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> package.jdo needs to be updated to conform with HIVE-20221 changes
> --
>
> Key: HIVE-20833
> URL: https://issues.apache.org/jira/browse/HIVE-20833
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20833.1.patch, HIVE-20833.2.patch,
> HIVE-20833.3.patch, HIVE-20833.4.patch
>
>
> Following test if run with TestMiniLlapLocalCliDriver will fail:
> {code:sql}
> CREATE TABLE `alterPartTbl`(
>`po_header_id` bigint,
>`vendor_num` string,
>`requester_name` string,
>`approver_name` string,
>`buyer_name` string,
>`preparer_name` string,
>`po_requisition_number` string,
>`po_requisition_id` bigint,
>`po_requisition_desc` string,
>`rate_type` string,
>`rate_date` date,
>`rate` double,
>`blanket_total_amount` double,
>`authorization_status` string,
>`revision_num` bigint,
>`revised_date` date,
>`approved_flag` string,
>`approved_date` timestamp,
>
[jira] [Updated] (HIVE-20778) Join reordering may not be triggered if all joins in plan are created by decorrelation logic
[ https://issues.apache.org/jira/browse/HIVE-20778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20778: --- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Pushed to master, thanks [~vgarg]! > Join reordering may not be triggered if all joins in plan are created by > decorrelation logic > > > Key: HIVE-20778 > URL: https://issues.apache.org/jira/browse/HIVE-20778 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Vineet Garg >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20778.1.patch, HIVE-20778.2.patch, > HIVE-20778.3.patch, HIVE-20778.4.patch, HIVE-20778.5.patch, HIVE-20778.6.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20617) Fix type of constants in IN expressions to have correct type
[
https://issues.apache.org/jira/browse/HIVE-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-20617:
---
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
Pushed to master, thanks [~kgyrtkirk]!
> Fix type of constants in IN expressions to have correct type
>
>
> Key: HIVE-20617
> URL: https://issues.apache.org/jira/browse/HIVE-20617
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20617.01.patch, HIVE-20617.02.patch,
> HIVE-20617.03.patch, HIVE-20617.05.patch, HIVE-20617.06.patch,
> HIVE-20617.07.patch, HIVE-20617.08.patch, HIVE-20617.08.patch,
> HIVE-20617.08.patch, HIVE-20617.08.patch, HIVE-20617.08.patch,
> HIVE-20617.08.patch, HIVE-20617.08.patch, HIVE-20617.08.patch,
> HIVE-20617.09.patch, HIVE-20617.10.patch, HIVE-20617.10.patch,
> HIVE-20617.11.patch, HIVE-20617.11.patch, HIVE-20617.12.patch,
> HIVE-20617.13.patch, HIVE-20617.14.patch, HIVE-20617.15.patch,
> HIVE-20617.16.patch, HIVE-20617.16.patch, HIVE-20617.17.patch,
> HIVE-20617.18.patch
>
>
> In statements like {{struct(a,b) IN (const struct('x','y'), ... )}} the
> comparision in UDFIn may fail because if a or b is of char/varchar type the
> constants will retain string type - especially after PointlookupOptimizer
> compaction.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20821) Rewrite SUM0 into SUM + COALESCE combination
[ https://issues.apache.org/jira/browse/HIVE-20821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20821: --- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Pushed to master. > Rewrite SUM0 into SUM + COALESCE combination > > > Key: HIVE-20821 > URL: https://issues.apache.org/jira/browse/HIVE-20821 > Project: Hive > Issue Type: Improvement > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20821.01.patch, HIVE-20821.patch > > > Since SUM0 is not vectorized, but SUM + COALESCE are. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20617) Fix type of constants in IN expressions to have correct type
[
https://issues.apache.org/jira/browse/HIVE-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674089#comment-16674089
]
Hive QA commented on HIVE-20617:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946735/HIVE-20617.18.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15523 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14725/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14725/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14725/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946735 - PreCommit-HIVE-Build
> Fix type of constants in IN expressions to have correct type
>
>
> Key: HIVE-20617
> URL: https://issues.apache.org/jira/browse/HIVE-20617
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-20617.01.patch, HIVE-20617.02.patch,
> HIVE-20617.03.patch, HIVE-20617.05.patch, HIVE-20617.06.patch,
> HIVE-20617.07.patch, HIVE-20617.08.patch, HIVE-20617.08.patch,
> HIVE-20617.08.patch, HIVE-20617.08.patch, HIVE-20617.08.patch,
> HIVE-20617.08.patch, HIVE-20617.08.patch, HIVE-20617.08.patch,
> HIVE-20617.09.patch, HIVE-20617.10.patch, HIVE-20617.10.patch,
> HIVE-20617.11.patch, HIVE-20617.11.patch, HIVE-20617.12.patch,
> HIVE-20617.13.patch, HIVE-20617.14.patch, HIVE-20617.15.patch,
> HIVE-20617.16.patch, HIVE-20617.16.patch, HIVE-20617.17.patch,
> HIVE-20617.18.patch
>
>
> In statements like {{struct(a,b) IN (const struct('x','y'), ... )}} the
> comparision in UDFIn may fail because if a or b is of char/varchar type the
> constants will retain string type - especially after PointlookupOptimizer
> compaction.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20617) Fix type of constants in IN expressions to have correct type
[
https://issues.apache.org/jira/browse/HIVE-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674077#comment-16674077
]
Hive QA commented on HIVE-20617:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 8 new + 498 unchanged - 17
fixed = 506 total (was 515) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
59s{color} | {color:red} ql generated 5 new + 2310 unchanged - 5 fixed = 2315
total (was 2315) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 48s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | cg must be non-null but is marked as nullable At
HivePointLookupOptimizerRule.java:is marked as nullable At
HivePointLookupOptimizerRule.java:[line 329] |
| |
org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.interpretConstantAsPrimitive(PrimitiveTypeInfo,
Object, TypeInfo) invokes inefficient new Byte(String) constructor; use
Byte.valueOf(String) instead At TypeCheckProcFactory.java:new Byte(String)
constructor; use Byte.valueOf(String) instead At
TypeCheckProcFactory.java:[line 1385] |
| |
org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.interpretConstantAsPrimitive(PrimitiveTypeInfo,
Object, TypeInfo) invokes inefficient new Integer(String) constructor; use
Integer.valueOf(String) instead At TypeCheckProcFactory.java:new
Integer(String) constructor; use Integer.valueOf(String) instead At
TypeCheckProcFactory.java:[line 1377] |
| |
org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.interpretConstantAsPrimitive(PrimitiveTypeInfo,
Object, TypeInfo) invokes inefficient new Long(String) constructor; use
Long.valueOf(String) instead At TypeCheckProcFactory.java:new Long(String)
constructor; use Long.valueOf(String) instead At
TypeCheckProcFactory.java:[line 1379] |
| |
org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.interpretConstantAsPrimitive(PrimitiveTypeInfo,
Object, TypeInfo) invokes inefficient new Short(String) constructor; use
Short.valueOf(String) instead At TypeCheckProcFactory.java:new Short(String)
[jira] [Commented] (HIVE-20778) Join reordering may not be triggered if all joins in plan are created by decorrelation logic
[
https://issues.apache.org/jira/browse/HIVE-20778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674071#comment-16674071
]
Hive QA commented on HIVE-20778:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946708/HIVE-20778.6.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14724/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14724/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14724/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12946708/HIVE-20778.6.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946708 - PreCommit-HIVE-Build
> Join reordering may not be triggered if all joins in plan are created by
> decorrelation logic
>
>
> Key: HIVE-20778
> URL: https://issues.apache.org/jira/browse/HIVE-20778
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20778.1.patch, HIVE-20778.2.patch,
> HIVE-20778.3.patch, HIVE-20778.4.patch, HIVE-20778.5.patch, HIVE-20778.6.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20778) Join reordering may not be triggered if all joins in plan are created by decorrelation logic
[
https://issues.apache.org/jira/browse/HIVE-20778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674070#comment-16674070
]
Hive QA commented on HIVE-20778:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946708/HIVE-20778.6.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15520 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14723/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14723/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14723/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946708 - PreCommit-HIVE-Build
> Join reordering may not be triggered if all joins in plan are created by
> decorrelation logic
>
>
> Key: HIVE-20778
> URL: https://issues.apache.org/jira/browse/HIVE-20778
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20778.1.patch, HIVE-20778.2.patch,
> HIVE-20778.3.patch, HIVE-20778.4.patch, HIVE-20778.5.patch, HIVE-20778.6.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20778) Join reordering may not be triggered if all joins in plan are created by decorrelation logic
[
https://issues.apache.org/jira/browse/HIVE-20778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674054#comment-16674054
]
Hive QA commented on HIVE-20778:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
41s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14723/dev-support/hive-personality.sh
|
| git revision | master / f3fab45 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14723/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Join reordering may not be triggered if all joins in plan are created by
> decorrelation logic
>
>
> Key: HIVE-20778
> URL: https://issues.apache.org/jira/browse/HIVE-20778
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20778.1.patch, HIVE-20778.2.patch,
> HIVE-20778.3.patch, HIVE-20778.4.patch, HIVE-20778.5.patch, HIVE-20778.6.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20834) Hive QueryResultCache entries keeping reference to SemanticAnalyzer from cached query
[
https://issues.apache.org/jira/browse/HIVE-20834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674049#comment-16674049
]
Hive QA commented on HIVE-20834:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946700/HIVE-20834.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15520 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14722/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14722/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14722/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946700 - PreCommit-HIVE-Build
> Hive QueryResultCache entries keeping reference to SemanticAnalyzer from
> cached query
> -
>
> Key: HIVE-20834
> URL: https://issues.apache.org/jira/browse/HIVE-20834
> Project: Hive
> Issue Type: Bug
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-20834.1.patch, HIVE-20834.2.patch,
> HIVE-20834.3.patch, dominator_tree.png
>
>
> QueryResultCache.LookupInfo ends up keeping a reference to the
> SemanticAnalyzer from the cached query, for as long as the cached entry is in
> the cache. We should not be keeping the SemanticAnalyzer around after the
> query is done executing since they can hold on to quite a bit of memory.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20834) Hive QueryResultCache entries keeping reference to SemanticAnalyzer from cached query
[
https://issues.apache.org/jira/browse/HIVE-20834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674038#comment-16674038
]
Hive QA commented on HIVE-20834:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
39s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 2 new + 466 unchanged - 1
fixed = 468 total (was 467) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14722/dev-support/hive-personality.sh
|
| git revision | master / f3fab45 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14722/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14722/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Hive QueryResultCache entries keeping reference to SemanticAnalyzer from
> cached query
> -
>
> Key: HIVE-20834
> URL: https://issues.apache.org/jira/browse/HIVE-20834
> Project: Hive
> Issue Type: Bug
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-20834.1.patch, HIVE-20834.2.patch,
> HIVE-20834.3.patch, dominator_tree.png
>
>
> QueryResultCache.LookupInfo ends up keeping a reference to the
> SemanticAnalyzer from the cached query, for as long as the cached entry is in
> the cache. We should not be keeping the SemanticAnalyzer around after the
> query is done executing since they can hold on to quite a bit of memory.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20817) Reading Timestamp datatype via HiveServer2 gives errors
[
https://issues.apache.org/jira/browse/HIVE-20817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mahesh kumar behera updated HIVE-20817:
---
Attachment: HIVE-20817.02.patch
> Reading Timestamp datatype via HiveServer2 gives errors
> ---
>
> Key: HIVE-20817
> URL: https://issues.apache.org/jira/browse/HIVE-20817
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-20817.01.patch, HIVE-20817.02.patch
>
>
> CREATE TABLE JdbcBasicRead ( empno int, desg string,empname string,doj
> timestamp,Salary float,mgrid smallint, deptno tinyint ) ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ',';
> LOAD DATA LOCAL INPATH '/tmp/art_jdbc/hive/input/input_7columns.txt'
> OVERWRITE INTO TABLE JdbcBasicRead;
> Sample Data.
> —
> 7369,M,SMITH,1980-12-17 17:07:29.234234,5000.00,7902,20
> 7499,X,ALLEN,1981-02-20 17:07:29.234234,1250.00,7698,30
> 7521,X,WARD,1981-02-22 17:07:29.234234,01600.57,7698,40
> 7566,M,JONES,1981-04-02 17:07:29.234234,02975.65,7839,10
> 7654,X,MARTIN,1981-09-28 17:07:29.234234,01250.00,7698,20
> 7698,M,BLAKE,1981-05-01 17:07:29.234234,2850.98,7839,30
> 7782,M,CLARK,1981-06-09 17:07:29.234234,02450.00,7839,20
> —
> Select statement: SELECT empno, desg, empname, doj, salary, mgrid, deptno
> FROM JdbcBasicWrite
> {code}
> 2018-09-25T07:11:03,222 WARN [HiveServer2-Handler-Pool: Thread-83]:
> thrift.ThriftCLIService (:()) - Error fetching results:
> org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException:
> org.apache.hadoop.hive.common.type.Timestamp cannot be cast to
> java.sql.Timestamp
> at
> org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:469)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:328)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:910)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source) ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_112]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
> at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> ~[hadoop-common-3.1.1.3.0.1.0-187.jar:?]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at com.sun.proxy.$Proxy46.fetchResults(Unknown Source) ~[?:?]
> at org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:564)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:786)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1837)
> ~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1822)
> ~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> ~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> ~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> ~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[?:1.8.0_112]
> at
>
[jira] [Updated] (HIVE-20817) Reading Timestamp datatype via HiveServer2 gives errors
[
https://issues.apache.org/jira/browse/HIVE-20817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mahesh kumar behera updated HIVE-20817:
---
Status: Patch Available (was: Open)
> Reading Timestamp datatype via HiveServer2 gives errors
> ---
>
> Key: HIVE-20817
> URL: https://issues.apache.org/jira/browse/HIVE-20817
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-20817.01.patch, HIVE-20817.02.patch
>
>
> CREATE TABLE JdbcBasicRead ( empno int, desg string,empname string,doj
> timestamp,Salary float,mgrid smallint, deptno tinyint ) ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ',';
> LOAD DATA LOCAL INPATH '/tmp/art_jdbc/hive/input/input_7columns.txt'
> OVERWRITE INTO TABLE JdbcBasicRead;
> Sample Data.
> —
> 7369,M,SMITH,1980-12-17 17:07:29.234234,5000.00,7902,20
> 7499,X,ALLEN,1981-02-20 17:07:29.234234,1250.00,7698,30
> 7521,X,WARD,1981-02-22 17:07:29.234234,01600.57,7698,40
> 7566,M,JONES,1981-04-02 17:07:29.234234,02975.65,7839,10
> 7654,X,MARTIN,1981-09-28 17:07:29.234234,01250.00,7698,20
> 7698,M,BLAKE,1981-05-01 17:07:29.234234,2850.98,7839,30
> 7782,M,CLARK,1981-06-09 17:07:29.234234,02450.00,7839,20
> —
> Select statement: SELECT empno, desg, empname, doj, salary, mgrid, deptno
> FROM JdbcBasicWrite
> {code}
> 2018-09-25T07:11:03,222 WARN [HiveServer2-Handler-Pool: Thread-83]:
> thrift.ThriftCLIService (:()) - Error fetching results:
> org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException:
> org.apache.hadoop.hive.common.type.Timestamp cannot be cast to
> java.sql.Timestamp
> at
> org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:469)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:328)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:910)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source) ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_112]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
> at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> ~[hadoop-common-3.1.1.3.0.1.0-187.jar:?]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at com.sun.proxy.$Proxy46.fetchResults(Unknown Source) ~[?:?]
> at org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:564)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:786)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1837)
> ~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1822)
> ~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> ~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> ~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> ~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[?:1.8.0_112]
> at
>
[jira] [Updated] (HIVE-20817) Reading Timestamp datatype via HiveServer2 gives errors
[
https://issues.apache.org/jira/browse/HIVE-20817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mahesh kumar behera updated HIVE-20817:
---
Status: Open (was: Patch Available)
> Reading Timestamp datatype via HiveServer2 gives errors
> ---
>
> Key: HIVE-20817
> URL: https://issues.apache.org/jira/browse/HIVE-20817
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-20817.01.patch
>
>
> CREATE TABLE JdbcBasicRead ( empno int, desg string,empname string,doj
> timestamp,Salary float,mgrid smallint, deptno tinyint ) ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ',';
> LOAD DATA LOCAL INPATH '/tmp/art_jdbc/hive/input/input_7columns.txt'
> OVERWRITE INTO TABLE JdbcBasicRead;
> Sample Data.
> —
> 7369,M,SMITH,1980-12-17 17:07:29.234234,5000.00,7902,20
> 7499,X,ALLEN,1981-02-20 17:07:29.234234,1250.00,7698,30
> 7521,X,WARD,1981-02-22 17:07:29.234234,01600.57,7698,40
> 7566,M,JONES,1981-04-02 17:07:29.234234,02975.65,7839,10
> 7654,X,MARTIN,1981-09-28 17:07:29.234234,01250.00,7698,20
> 7698,M,BLAKE,1981-05-01 17:07:29.234234,2850.98,7839,30
> 7782,M,CLARK,1981-06-09 17:07:29.234234,02450.00,7839,20
> —
> Select statement: SELECT empno, desg, empname, doj, salary, mgrid, deptno
> FROM JdbcBasicWrite
> {code}
> 2018-09-25T07:11:03,222 WARN [HiveServer2-Handler-Pool: Thread-83]:
> thrift.ThriftCLIService (:()) - Error fetching results:
> org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException:
> org.apache.hadoop.hive.common.type.Timestamp cannot be cast to
> java.sql.Timestamp
> at
> org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:469)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:328)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:910)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source) ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_112]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
> at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> ~[hadoop-common-3.1.1.3.0.1.0-187.jar:?]
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at com.sun.proxy.$Proxy46.fetchResults(Unknown Source) ~[?:?]
> at org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:564)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:786)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1837)
> ~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1822)
> ~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> ~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> ~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
> ~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> ~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[?:1.8.0_112]
> at
>
[jira] [Commented] (HIVE-20838) Timestamps with timezone are set to null when using the streaming API
[
https://issues.apache.org/jira/browse/HIVE-20838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674030#comment-16674030
]
Hive QA commented on HIVE-20838:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946692/HIVE-20838.5.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 15521 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_nonvec_part_all_primitive]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_nonvec_part_all_primitive_llap_io]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_vec_part_all_primitive]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_vec_part_all_primitive_llap_io]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_nonvec_part_all_primitive]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_nonvec_part_all_primitive_llap_io]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part_all_primitive]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part_all_primitive_llap_io]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vecrow_part_all_primitive]
(batchId=175)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=258)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14721/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14721/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14721/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946692 - PreCommit-HIVE-Build
> Timestamps with timezone are set to null when using the streaming API
> -
>
> Key: HIVE-20838
> URL: https://issues.apache.org/jira/browse/HIVE-20838
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Jaume M
>Assignee: Jaume M
>Priority: Major
> Attachments: HIVE-20838.1.patch, HIVE-20838.2.patch,
> HIVE-20838.3.patch, HIVE-20838.4.patch, HIVE-20838.5.patch
>
>
> For example:
> {code}
> beeline> create table default.timest (a TIMESTAMP) stored as orc " +
> "TBLPROPERTIES('transactional'='true')
> # And then:
> connection.write("2018-10-19 10:35:00 America/Los_Angeles".getBytes());
> {code}
> inserts NULL.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20838) Timestamps with timezone are set to null when using the streaming API
[
https://issues.apache.org/jira/browse/HIVE-20838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674022#comment-16674022
]
Hive QA commented on HIVE-20838:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
30s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} serde in master has 198 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} streaming: The patch generated 2 new + 456 unchanged -
0 fixed = 458 total (was 456) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14721/dev-support/hive-personality.sh
|
| git revision | master / f3fab45 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14721/yetus/diff-checkstyle-streaming.txt
|
| modules | C: common serde streaming U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14721/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Timestamps with timezone are set to null when using the streaming API
> -
>
> Key: HIVE-20838
> URL: https://issues.apache.org/jira/browse/HIVE-20838
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Jaume M
>Assignee: Jaume M
>Priority: Major
> Attachments: HIVE-20838.1.patch, HIVE-20838.2.patch,
> HIVE-20838.3.patch, HIVE-20838.4.patch, HIVE-20838.5.patch
>
>
> For example:
> {code}
> beeline> create table default.timest (a TIMESTAMP) stored as orc " +
> "TBLPROPERTIES('transactional'='true')
> # And then:
> connection.write("2018-10-19 10:35:00 America/Los_Angeles".getBytes());
> {code}
> inserts NULL.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20805) Hive does not copy source data when importing as non-hive user
[
https://issues.apache.org/jira/browse/HIVE-20805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674008#comment-16674008
]
Hive QA commented on HIVE-20805:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946642/HIVE-20805.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15520 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14720/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14720/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14720/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946642 - PreCommit-HIVE-Build
> Hive does not copy source data when importing as non-hive user
> ---
>
> Key: HIVE-20805
> URL: https://issues.apache.org/jira/browse/HIVE-20805
> Project: Hive
> Issue Type: Bug
> Components: Hive, HiveServer2
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-20805.03.patch, HIVE-20805.1.patch,
> HIVE-20805.2.patch
>
>
> while loading data to a managed table from user given path, Hive uses move
> operation to copy data from user location to table location. In case move can
> not be used due to permission issue or mismatched encryption zone etc, hive
> uses copy and then deletes the files from source location to keep to behavior
> same. But in case the user does not have write access to the source location,
> delete will fail with file permission exception and load operation will fail.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20805) Hive does not copy source data when importing as non-hive user
[
https://issues.apache.org/jira/browse/HIVE-20805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674000#comment-16674000
]
Hive QA commented on HIVE-20805:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
30s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
42s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14720/dev-support/hive-personality.sh
|
| git revision | master / f3fab45 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14720/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Hive does not copy source data when importing as non-hive user
> ---
>
> Key: HIVE-20805
> URL: https://issues.apache.org/jira/browse/HIVE-20805
> Project: Hive
> Issue Type: Bug
> Components: Hive, HiveServer2
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-20805.03.patch, HIVE-20805.1.patch,
> HIVE-20805.2.patch
>
>
> while loading data to a managed table from user given path, Hive uses move
> operation to copy data from user location to table location. In case move can
> not be used due to permission issue or mismatched encryption zone etc, hive
> uses copy and then deletes the files from source location to keep to behavior
> same. But in case the user does not have write access to the source location,
> delete will fail with file permission exception and load operation will fail.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20613) CachedStore: Add more UT coverage (outside of .q files)
[
https://issues.apache.org/jira/browse/HIVE-20613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673976#comment-16673976
]
Hive QA commented on HIVE-20613:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946631/HIVE-20613.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15526 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=196)
[druidmini_masking.q,druidmini_test1.q,druidkafkamini_basic.q,druidmini_joins.q,druid_timestamptz.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14719/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14719/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14719/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946631 - PreCommit-HIVE-Build
> CachedStore: Add more UT coverage (outside of .q files)
> ---
>
> Key: HIVE-20613
> URL: https://issues.apache.org/jira/browse/HIVE-20613
> Project: Hive
> Issue Type: Sub-task
> Components: Metastore
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-20613.1.patch, HIVE-20613.2.patch
>
>
> 1. Add tests which will use the background thread for updating the cached
> data (database, table, partition, table stats, partition stats)
> 2. Add more tests for existing APIs: stats aggregation, listing partitions
> when partial specs are provided, testing the storage descriptor
> caching/deduplication (specially when tables/ptns are dropped/added), table
> col stats, partition col stats
> 3. Test 1. in a multithreaded scenario
> 4. Test whitelist/blacklist
> 5. Test prewarm memory limit estimation code
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20613) CachedStore: Add more UT coverage (outside of .q files)
[
https://issues.apache.org/jira/browse/HIVE-20613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673961#comment-16673961
]
Hive QA commented on HIVE-20613:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
6s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
6s{color} | {color:blue} standalone-metastore/metastore-server in master has
185 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 14 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
11s{color} | {color:green} standalone-metastore/metastore-server generated 0
new + 184 unchanged - 1 fixed = 184 total (was 185) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14719/dev-support/hive-personality.sh
|
| git revision | master / f3fab45 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14719/yetus/whitespace-eol.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14719/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CachedStore: Add more UT coverage (outside of .q files)
> ---
>
> Key: HIVE-20613
> URL: https://issues.apache.org/jira/browse/HIVE-20613
> Project: Hive
> Issue Type: Sub-task
> Components: Metastore
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-20613.1.patch, HIVE-20613.2.patch
>
>
> 1. Add tests which will use the background thread for updating the cached
> data (database, table, partition, table stats, partition stats)
> 2. Add more tests for existing APIs: stats aggregation, listing partitions
> when partial specs are provided, testing the storage descriptor
> caching/deduplication (specially when tables/ptns are dropped/added), table
> col stats, partition col stats
> 3. Test 1. in a multithreaded scenario
> 4. Test whitelist/blacklist
> 5. Test prewarm memory limit estimation code
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20858) Serializer is not correctly initialized with configuration in Utilities.createEmptyBuckets()
[
https://issues.apache.org/jira/browse/HIVE-20858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673958#comment-16673958
]
Hive QA commented on HIVE-20858:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946623/HIVE-20858.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15520 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14718/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14718/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14718/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946623 - PreCommit-HIVE-Build
> Serializer is not correctly initialized with configuration in
> Utilities.createEmptyBuckets()
>
>
> Key: HIVE-20858
> URL: https://issues.apache.org/jira/browse/HIVE-20858
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Wei Zheng
>Assignee: Wei Zheng
>Priority: Major
> Attachments: HIVE-20858.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20858) Serializer is not correctly initialized with configuration in Utilities.createEmptyBuckets()
[
https://issues.apache.org/jira/browse/HIVE-20858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673950#comment-16673950
]
Hive QA commented on HIVE-20858:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
42s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14718/dev-support/hive-personality.sh
|
| git revision | master / f3fab45 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14718/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Serializer is not correctly initialized with configuration in
> Utilities.createEmptyBuckets()
>
>
> Key: HIVE-20858
> URL: https://issues.apache.org/jira/browse/HIVE-20858
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Wei Zheng
>Assignee: Wei Zheng
>Priority: Major
> Attachments: HIVE-20858.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20854) Sensible Defaults: Hive's Zookeeper heartbeat interval is 20 minutes, change to 2
[
https://issues.apache.org/jira/browse/HIVE-20854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673945#comment-16673945
]
Hive QA commented on HIVE-20854:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946616/HIVE-20854.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 15502 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=250)
TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out)
(batchId=250)
TestLocationQueries - did not produce a TEST-*.xml file (likely timed out)
(batchId=250)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=196)
[druidmini_masking.q,druidmini_test1.q,druidkafkamini_basic.q,druidmini_joins.q,druid_timestamptz.q]
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=250)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=250)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[show_conf] (batchId=27)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_rp_limit]
(batchId=171)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14717/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14717/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14717/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946616 - PreCommit-HIVE-Build
> Sensible Defaults: Hive's Zookeeper heartbeat interval is 20 minutes, change
> to 2
> -
>
> Key: HIVE-20854
> URL: https://issues.apache.org/jira/browse/HIVE-20854
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-20854.1.patch
>
>
> {code}
> HIVE_ZOOKEEPER_SESSION_TIMEOUT("hive.zookeeper.session.timeout",
> "120ms",
> new TimeValidator(TimeUnit.MILLISECONDS),
> "ZooKeeper client's session timeout (in milliseconds). The client is
> disconnected, and as a result, all locks released, \n" +
> "if a heartbeat is not sent in the timeout."),
> {code}
> That's 1,200,000ms which is too long for all practical purposes - a 20 minute
> outage in case a node has a failure is too long.
> That is too long for the JDBC load-balancing, LLAP failure tolerance and the
> lock manager expiry.
> Change to 2 minutes, as a sensible default
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
