[jira] [Commented] (HIVE-17084) Turn on hive.stats.fetch.column.stats configuration flag
[ https://issues.apache.org/jira/browse/HIVE-17084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751679#comment-16751679 ] Ashutosh Chauhan commented on HIVE-17084: - Sorry I missed it initially but I noticed that plans for load_dyn_part* tests have changed. These tests are no longer doing doing SPDO optimization of extra sort. That decision is based on stats. We shall restore those tests to do original plan especially since [~vgarg] is making active changes in that area. It won't be necessary to unset stats config, we can perhaps lower the stats threshold to make SPDO run on those queries. > Turn on hive.stats.fetch.column.stats configuration flag > > > Key: HIVE-17084 > URL: https://issues.apache.org/jira/browse/HIVE-17084 > Project: Hive > Issue Type: Task > Components: Statistics >Reporter: Vineet Garg >Assignee: Zoltan Haindrich >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-17084.08.patch, HIVE-17084.09.patch, > HIVE-17084.1.patch, HIVE-17084.10.patch, HIVE-17084.11.patch, > HIVE-17084.12.patch, HIVE-17084.13.patch, HIVE-17084.14.patch, > HIVE-17084.15.patch, HIVE-17084.16.patch, HIVE-17084.17.patch, > HIVE-17084.18.patch, HIVE-17084.19.patch, HIVE-17084.20.patch, > HIVE-17084.21.patch, HIVE-17084.22.patch, HIVE-17084.23.patch, > HIVE-17084.24.patch, HIVE-17084.25.patch, HIVE-17084.26.patch, > HIVE-17084.27.patch, HIVE-17084.28.patch, HIVE-17084.29.patch, > HIVE-17084.30.patch, HIVE-17084.31.patch, HIVE-17084.31.patch, > HIVE-17084.31.patch, HIVE-170884.4.patch, HIVE-170884.5.patch, > HIVE-170884.7.patch > > > This flag is off by default and could result in bad plans due to missing > column statistics. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20776) Run HMS filterHooks on server-side in addition to client-side
[
https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751644#comment-16751644
]
Hive QA commented on HIVE-20776:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956202/HIVE-20776.017-branch-3.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 130 failed/errored test(s), 14480 tests
executed
*Failed tests:*
{noformat}
TestAddPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestAddPartitionsFromPartSpec - did not produce a TEST-*.xml file (likely timed
out) (batchId=230)
TestAdminUser - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
TestAggregateStatsCache - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
TestAlterPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestAppendPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=274)
TestCachedStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestCatalogCaching - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestCatalogNonDefaultClient - did not produce a TEST-*.xml file (likely timed
out) (batchId=228)
TestCatalogNonDefaultSvr - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
TestCatalogOldClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestCatalogs - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestCheckConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=238)
TestDatabases - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDeadline - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestDefaultConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDropPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=274)
TestEmbeddedHiveMetaStore - did not produce a TEST-*.xml file (likely timed
out) (batchId=231)
TestExchangePartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestFMSketchSerialization - did not produce a TEST-*.xml file (likely timed
out) (batchId=239)
TestFilterHooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestForeignKey - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestFunctions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestGetPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestGetPartitionsUsingProjectionAndFilterSpecs - did not produce a TEST-*.xml
file (likely timed out) (batchId=230)
TestGetTableMeta - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHLLNoBias - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHLLSerialization - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHdfsUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
TestHiveAlterHandler - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHiveMetaStoreGetMetaConf - did not produce a TEST-*.xml file (likely timed
out) (batchId=238)
TestHiveMetaStorePartitionSpecs - did not produce a TEST-*.xml file (likely
timed out) (batchId=230)
TestHiveMetaStoreSchemaMethods - did not produce a TEST-*.xml file (likely
timed out) (batchId=235)
TestHiveMetaStoreTimeout - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHiveMetaStoreTxns - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHiveMetaStoreWithEnvironmentContext - did not produce a TEST-*.xml file
(likely timed out) (batchId=233)
TestHiveMetaToolCommandLine - did not produce a TEST-*.xml file (likely timed
out) (batchId=235)
TestHiveMetastoreCli - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHyperLogLog - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHyperLogLogDense - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHyperLogLogMerge - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHyperLogLogSparse - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestJSONMessageDeserializer - did not produce a TEST-*.xml file (likely timed
out) (batchId=235)
TestListPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestLockRequestBuilder - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestMarkPartition - did not produce
[jira] [Updated] (HIVE-20776) Run HMS filterHooks on server-side in addition to client-side
[ https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated HIVE-20776: - Attachment: HIVE-20776.017-branch-3.patch > Run HMS filterHooks on server-side in addition to client-side > - > > Key: HIVE-20776 > URL: https://issues.apache.org/jira/browse/HIVE-20776 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Karthik Manamcheri >Assignee: Na Li >Priority: Major > Attachments: HIVE-20776.001.patch, HIVE-20776.003.patch, > HIVE-20776.004.patch, HIVE-20776.005.patch, HIVE-20776.006.patch, > HIVE-20776.007.patch, HIVE-20776.007.patch, HIVE-20776.008.patch, > HIVE-20776.009.patch, HIVE-20776.009.patch, HIVE-20776.010.patch, > HIVE-20776.011.patch, HIVE-20776.011.patch, HIVE-20776.012.patch, > HIVE-20776.013.patch, HIVE-20776.014.patch, HIVE-20776.015-branch-3.patch, > HIVE-20776.015.branch-3.patch, HIVE-20776.015.patch, > HIVE-20776.015_a.branch-3.patch, HIVE-20776.016-branch-3.patch, > HIVE-20776.017-branch-3.patch > > > In HMS, I noticed that all the filter hooks are applied on the client side > (in HiveMetaStoreClient.java). Is there any reason why we can't apply the > filters on the server-side? > Motivation: Some newer apache projects such as Kudu use HMS for metadata > storage. Kudu is not completely Java-based and there are interaction points > where they have C++ clients. In such cases, it would be ideal to have > consistent behavior from HMS side as far as filters, etc are concerned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-11708) Logical operators raises ClassCastExceptions with NULL
[
https://issues.apache.org/jira/browse/HIVE-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751649#comment-16751649
]
Ashutosh Chauhan commented on HIVE-11708:
-
Patch looks good. Can you please also add test for it?
> Logical operators raises ClassCastExceptions with NULL
> --
>
> Key: HIVE-11708
> URL: https://issues.apache.org/jira/browse/HIVE-11708
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.13.0, 1.2.1
>Reporter: Satoshi Tagomori
>Assignee: Ryu Kobayashi
>Priority: Major
> Attachments: HIVE-11708.01.patch
>
>
> According to Language Manual UDF, logical operators returns NULL if one of
> arguments is NULL.
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-LogicalOperators
> But query below fails with ClassCastException.
> {code}
> SELECT COUNT(*) AS c
> FROM tbl
> WHERE 1=1 AND NULL
> {code}
> Exception (on 0.13):
> {noformat}
> 15/08/27 08:56:23 ERROR ql.Driver: FAILED: ClassCastException
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
> java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd.initialize(GenericUDFOPAnd.java:52)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:116)
> at
> org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:231)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:934)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:1128)
> at
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:78)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:132)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:109)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:184)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:9716)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:9672)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3208)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3005)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:8228)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8183)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9015)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9281)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:427)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:323)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:980)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1045)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906)
> {noformat}
> I confirmed that Hive 1.2.1 of HDP2.3 Sandbox also raises this exception.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21045) Add HMS total api count stats and connection pool stats to metrics
[
https://issues.apache.org/jira/browse/HIVE-21045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751493#comment-16751493
]
Hive QA commented on HIVE-21045:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
7s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
2s{color} | {color:blue} standalone-metastore/metastore-server in master has
184 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15782/dev-support/hive-personality.sh
|
| git revision | master / 1327d47 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15782/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add HMS total api count stats and connection pool stats to metrics
> --
>
> Key: HIVE-21045
> URL: https://issues.apache.org/jira/browse/HIVE-21045
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Karthik Manamcheri
>Assignee: Karthik Manamcheri
>Priority: Minor
> Attachments: HIVE-21045.1.patch, HIVE-21045.2.patch,
> HIVE-21045.3.patch, HIVE-21045.4.patch, HIVE-21045.5.patch, HIVE-21045.6.patch
>
>
> There are two key metrics which I think we lack and which would be really
> great to help with scaling visibility in HMS.
> *Total API calls duration stats*
> We already compute and log the duration of API calls in the {{PerfLogger}}.
> We don't have any gauge or timer on what the average duration of an API call
> is for the past some bucket of time. This will give us an insight into if
> there is load on the server which is increasing the average API response time.
>
> *Connection Pool stats*
> We can use different connection pooling libraries such as bonecp or hikaricp.
> These pool managers expose statistics such as average time waiting to get a
> connection, number of connections active, etc. We should expose this as a
> metric so that we can track if the the connection pool size configured is too
> small and we are saturating!
> These metrics would help catch problems with HMS resource contention before
> they actually have jobs failing.
--
This message was sent by
[jira] [Updated] (HIVE-21083) Remove the requirement to specify the truststore location when TLS to the database is turned on
[
https://issues.apache.org/jira/browse/HIVE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Morio Ramdenbourg updated HIVE-21083:
-
Attachment: HIVE-21083.patch
Status: Patch Available (was: In Progress)
> Remove the requirement to specify the truststore location when TLS to the
> database is turned on
> ---
>
> Key: HIVE-21083
> URL: https://issues.apache.org/jira/browse/HIVE-21083
> Project: Hive
> Issue Type: Improvement
> Components: Metastore, Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Morio Ramdenbourg
>Assignee: Morio Ramdenbourg
>Priority: Major
> Attachments: HIVE-21083.patch
>
>
> In the current implementation,
> [ObjectStore.configureSSL|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L349-L382]
> throws an exception if TLS to the database is turned on
> (_metastore.dbaccess.ssl.use.SSL_) but a truststore file location
> (_metastore.dbaccess.ssl.truststore.path_) is not specified.
> However, according to the [JSSE (Java 8)
> documentation|https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#InstallationAndCustomization],
> the Java truststore file location system property
> (_javax.net.ssl.trustStore_) defaults to using the "_jssecacerts_, if it
> exists. Otherwise, _cacerts_" files. These are the default truststores that
> come with the Java installation and contain a list of well-known certificate
> authorities.
> It was identified that one valid way of configuring TLS is by adding to these
> default files. In that case, no changes to the truststore properties are
> necessary. We should support this case by changing the following logic to
> remove the requirement for the truststore file location config property:
> {code:java}
> String trustStorePath = MetastoreConf.getVar(conf,
> ConfVars.DBACCESS_SSL_TRUSTSTORE_PATH).trim();
> if (trustStorePath.isEmpty()) {
> throw new IllegalArgumentException("SSL to the database store has
> been enabled but " +
> ConfVars.DBACCESS_SSL_TRUSTSTORE_PATH.toString() + " is empty. "
> + "Set this property to enable SSL.");
> }
> {code}
> We should also loosen the requirement on the truststore password if the user
> decides to use the Java defaults
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21132) Semi join edge is not being removed despite max bloomfilter entries set to 1
[
https://issues.apache.org/jira/browse/HIVE-21132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751671#comment-16751671
]
Hive QA commented on HIVE-21132:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
5s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
45s{color} | {color:blue} ql in master has 2304 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 54 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15784/dev-support/hive-personality.sh
|
| git revision | master / 1327d47 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15784/yetus/whitespace-eol.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15784/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Semi join edge is not being removed despite max bloomfilter entries set to 1
>
>
> Key: HIVE-21132
> URL: https://issues.apache.org/jira/browse/HIVE-21132
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-21132.1.patch, HIVE-21132.2.patch,
> HIVE-21132.3.patch
>
>
> * Reproducer
> {code:sql}
> --! qt:dataset:lineitem
> --! qt:dataset:part
> --! qt:dataset:src
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> --set hive.compute.query.using.stats=false;
> set hive.mapred.mode=nonstrict;
> set hive.explain.user=false;
> set hive.optimize.ppd=true;
> set hive.ppd.remove.duplicatefilters=true;
> set hive.tez.dynamic.partition.pruning=true;
> set hive.tez.dynamic.semijoin.reduction=true;
> set hive.optimize.metadataonly=false;
> set hive.optimize.index.filter=true;
> set hive.stats.autogather=true;
> set hive.tez.bigtable.minsize.semijoin.reduction=1;
> set
[jira] [Commented] (HIVE-21134) Hive Build Version as UDF
[ https://issues.apache.org/jira/browse/HIVE-21134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751461#comment-16751461 ] Mani M commented on HIVE-21134: --- Hi [~pvary] Thanks for +1. Can you accept the pull request. > Hive Build Version as UDF > - > > Key: HIVE-21134 > URL: https://issues.apache.org/jira/browse/HIVE-21134 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Mani M >Assignee: Mani M >Priority: Minor > Labels: UDF, pull-request-available > Attachments: HIVE.21134.02.PATCH, HIVE.21134.03.PATCH, > HIVE.21134.04.PATCH, HIVE.21134.05.PATCH, HIVE.21134.06.PATCH, > HIVE.21134.07.PATCH, HIVE.21134.PATCH > > > This Jira is to get the Hive Build Version as UDF. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21045) Add HMS total api count stats and connection pool stats to metrics
[
https://issues.apache.org/jira/browse/HIVE-21045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751582#comment-16751582
]
Hive QA commented on HIVE-21045:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956169/HIVE-21045.6.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 15706 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testUseSSLProperty
(batchId=230)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.org.apache.hadoop.hive.ql.TestWarehouseExternalDir
(batchId=243)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testExternalDefaultPaths
(batchId=243)
org.apache.hive.jdbc.TestActivePassiveHA.testActivePassiveHA (batchId=261)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15782/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15782/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15782/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12956169 - PreCommit-HIVE-Build
> Add HMS total api count stats and connection pool stats to metrics
> --
>
> Key: HIVE-21045
> URL: https://issues.apache.org/jira/browse/HIVE-21045
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Karthik Manamcheri
>Assignee: Karthik Manamcheri
>Priority: Minor
> Attachments: HIVE-21045.1.patch, HIVE-21045.2.patch,
> HIVE-21045.3.patch, HIVE-21045.4.patch, HIVE-21045.5.patch, HIVE-21045.6.patch
>
>
> There are two key metrics which I think we lack and which would be really
> great to help with scaling visibility in HMS.
> *Total API calls duration stats*
> We already compute and log the duration of API calls in the {{PerfLogger}}.
> We don't have any gauge or timer on what the average duration of an API call
> is for the past some bucket of time. This will give us an insight into if
> there is load on the server which is increasing the average API response time.
>
> *Connection Pool stats*
> We can use different connection pooling libraries such as bonecp or hikaricp.
> These pool managers expose statistics such as average time waiting to get a
> connection, number of connections active, etc. We should expose this as a
> metric so that we can track if the the connection pool size configured is too
> small and we are saturating!
> These metrics would help catch problems with HMS resource contention before
> they actually have jobs failing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21157) Slf4j warning again from 2.X.X
[
https://issues.apache.org/jira/browse/HIVE-21157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751643#comment-16751643
]
Ashutosh Chauhan commented on HIVE-21157:
-
This will exclude log4j-slf4j-impl from Hive's lib and instead will make
hadoop's version of that jar to be picked up. I am not sure that what we wants.
Infact hadoop's version is already behind, so dont think we want that. Instead,
hive's version should be picked up.
> Slf4j warning again from 2.X.X
> --
>
> Key: HIVE-21157
> URL: https://issues.apache.org/jira/browse/HIVE-21157
> Project: Hive
> Issue Type: Bug
> Components: CLI
>Affects Versions: 2.0.0
>Reporter: Ryu Kobayashi
>Assignee: Ryu Kobayashi
>Priority: Minor
> Attachments: HIVE-21157.01.patch
>
>
> The warning of Slf4j has occurred again since 2.0.0:
> {code}
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21156) SharedWorkOptimizer may preserve filter in TS incorrectly
[
https://issues.apache.org/jira/browse/HIVE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751472#comment-16751472
]
Hive QA commented on HIVE-21156:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956167/HIVE-21156.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15710 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_delimited]
(batchId=275)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sharedwork]
(batchId=178)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15781/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15781/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15781/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12956167 - PreCommit-HIVE-Build
> SharedWorkOptimizer may preserve filter in TS incorrectly
> -
>
> Key: HIVE-21156
> URL: https://issues.apache.org/jira/browse/HIVE-21156
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21156.01.patch, HIVE-21156.patch
>
>
> For some scan reutilizations, we may end up with a Filter expression
> associated with the scan that should be removed by the optimizer. This can
> lead to incorrect results. Repro case is part of the patch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21156) SharedWorkOptimizer may preserve filter in TS incorrectly
[ https://issues.apache.org/jira/browse/HIVE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21156: --- Attachment: HIVE-21156.02.patch > SharedWorkOptimizer may preserve filter in TS incorrectly > - > > Key: HIVE-21156 > URL: https://issues.apache.org/jira/browse/HIVE-21156 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21156.01.patch, HIVE-21156.02.patch, > HIVE-21156.patch > > > For some scan reutilizations, we may end up with a Filter expression > associated with the scan that should be removed by the optimizer. This can > lead to incorrect results. Repro case is part of the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20776) Run HMS filterHooks on server-side in addition to client-side
[
https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751590#comment-16751590
]
Hive QA commented on HIVE-20776:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 10s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-15783/patches/PreCommit-HIVE-Build-15783.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15783/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Run HMS filterHooks on server-side in addition to client-side
> -
>
> Key: HIVE-20776
> URL: https://issues.apache.org/jira/browse/HIVE-20776
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Karthik Manamcheri
>Assignee: Na Li
>Priority: Major
> Attachments: HIVE-20776.001.patch, HIVE-20776.003.patch,
> HIVE-20776.004.patch, HIVE-20776.005.patch, HIVE-20776.006.patch,
> HIVE-20776.007.patch, HIVE-20776.007.patch, HIVE-20776.008.patch,
> HIVE-20776.009.patch, HIVE-20776.009.patch, HIVE-20776.010.patch,
> HIVE-20776.011.patch, HIVE-20776.011.patch, HIVE-20776.012.patch,
> HIVE-20776.013.patch, HIVE-20776.014.patch, HIVE-20776.015-branch-3.patch,
> HIVE-20776.015.branch-3.patch, HIVE-20776.015.patch,
> HIVE-20776.015_a.branch-3.patch, HIVE-20776.016-branch-3.patch,
> HIVE-20776.017-branch-3.patch
>
>
> In HMS, I noticed that all the filter hooks are applied on the client side
> (in HiveMetaStoreClient.java). Is there any reason why we can't apply the
> filters on the server-side?
> Motivation: Some newer apache projects such as Kudu use HMS for metadata
> storage. Kudu is not completely Java-based and there are interaction points
> where they have C++ clients. In such cases, it would be ideal to have
> consistent behavior from HMS side as far as filters, etc are concerned.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21156) SharedWorkOptimizer may preserve filter in TS incorrectly
[
https://issues.apache.org/jira/browse/HIVE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751762#comment-16751762
]
Hive QA commented on HIVE-21156:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956203/HIVE-21156.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15710 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_groupby_reduce]
(batchId=61)
org.apache.hadoop.hive.common.metrics.metrics2.TestCodahaleMetrics.testFileReporting
(batchId=285)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15785/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15785/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15785/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12956203 - PreCommit-HIVE-Build
> SharedWorkOptimizer may preserve filter in TS incorrectly
> -
>
> Key: HIVE-21156
> URL: https://issues.apache.org/jira/browse/HIVE-21156
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21156.01.patch, HIVE-21156.02.patch,
> HIVE-21156.02.patch, HIVE-21156.patch
>
>
> For some scan reutilizations, we may end up with a Filter expression
> associated with the scan that should be removed by the optimizer. This can
> lead to incorrect results. Repro case is part of the patch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21083) Remove the requirement to specify the truststore location when TLS to the database is turned on
[
https://issues.apache.org/jira/browse/HIVE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Morio Ramdenbourg updated HIVE-21083:
-
Status: In Progress (was: Patch Available)
> Remove the requirement to specify the truststore location when TLS to the
> database is turned on
> ---
>
> Key: HIVE-21083
> URL: https://issues.apache.org/jira/browse/HIVE-21083
> Project: Hive
> Issue Type: Improvement
> Components: Metastore, Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Morio Ramdenbourg
>Assignee: Morio Ramdenbourg
>Priority: Major
> Attachments: HIVE-21083.patch
>
>
> In the current implementation,
> [ObjectStore.configureSSL|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L349-L382]
> throws an exception if TLS to the database is turned on
> (_metastore.dbaccess.ssl.use.SSL_) but a truststore file location
> (_metastore.dbaccess.ssl.truststore.path_) is not specified.
> However, according to the [JSSE (Java 8)
> documentation|https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#InstallationAndCustomization],
> the Java truststore file location system property
> (_javax.net.ssl.trustStore_) defaults to using the "_jssecacerts_, if it
> exists. Otherwise, _cacerts_" files. These are the default truststores that
> come with the Java installation and contain a list of well-known certificate
> authorities.
> It was identified that one valid way of configuring TLS is by adding to these
> default files. In that case, no changes to the truststore properties are
> necessary. We should support this case by changing the following logic to
> remove the requirement for the truststore file location config property:
> {code:java}
> String trustStorePath = MetastoreConf.getVar(conf,
> ConfVars.DBACCESS_SSL_TRUSTSTORE_PATH).trim();
> if (trustStorePath.isEmpty()) {
> throw new IllegalArgumentException("SSL to the database store has
> been enabled but " +
> ConfVars.DBACCESS_SSL_TRUSTSTORE_PATH.toString() + " is empty. "
> + "Set this property to enable SSL.");
> }
> {code}
> We should also loosen the requirement on the truststore password if the user
> decides to use the Java defaults
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21083) Remove the requirement to specify the truststore location when TLS to the database is turned on
[
https://issues.apache.org/jira/browse/HIVE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Morio Ramdenbourg updated HIVE-21083:
-
Attachment: HIVE-21083.2.patch
Status: Patch Available (was: In Progress)
> Remove the requirement to specify the truststore location when TLS to the
> database is turned on
> ---
>
> Key: HIVE-21083
> URL: https://issues.apache.org/jira/browse/HIVE-21083
> Project: Hive
> Issue Type: Improvement
> Components: Metastore, Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Morio Ramdenbourg
>Assignee: Morio Ramdenbourg
>Priority: Major
> Attachments: HIVE-21083.2.patch, HIVE-21083.patch
>
>
> In the current implementation,
> [ObjectStore.configureSSL|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L349-L382]
> throws an exception if TLS to the database is turned on
> (_metastore.dbaccess.ssl.use.SSL_) but a truststore file location
> (_metastore.dbaccess.ssl.truststore.path_) is not specified.
> However, according to the [JSSE (Java 8)
> documentation|https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#InstallationAndCustomization],
> the Java truststore file location system property
> (_javax.net.ssl.trustStore_) defaults to using the "_jssecacerts_, if it
> exists. Otherwise, _cacerts_" files. These are the default truststores that
> come with the Java installation and contain a list of well-known certificate
> authorities.
> It was identified that one valid way of configuring TLS is by adding to these
> default files. In that case, no changes to the truststore properties are
> necessary. We should support this case by changing the following logic to
> remove the requirement for the truststore file location config property:
> {code:java}
> String trustStorePath = MetastoreConf.getVar(conf,
> ConfVars.DBACCESS_SSL_TRUSTSTORE_PATH).trim();
> if (trustStorePath.isEmpty()) {
> throw new IllegalArgumentException("SSL to the database store has
> been enabled but " +
> ConfVars.DBACCESS_SSL_TRUSTSTORE_PATH.toString() + " is empty. "
> + "Set this property to enable SSL.");
> }
> {code}
> We should also loosen the requirement on the truststore password if the user
> decides to use the Java defaults
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21156) SharedWorkOptimizer may preserve filter in TS incorrectly
[
https://issues.apache.org/jira/browse/HIVE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751833#comment-16751833
]
Hive QA commented on HIVE-21156:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
39s{color} | {color:blue} ql in master has 2304 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15788/dev-support/hive-personality.sh
|
| git revision | master / 1327d47 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15788/yetus/whitespace-eol.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15788/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> SharedWorkOptimizer may preserve filter in TS incorrectly
> -
>
> Key: HIVE-21156
> URL: https://issues.apache.org/jira/browse/HIVE-21156
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21156.01.patch, HIVE-21156.02.patch,
> HIVE-21156.02.patch, HIVE-21156.patch
>
>
> For some scan reutilizations, we may end up with a Filter expression
> associated with the scan that should be removed by the optimizer. This can
> lead to incorrect results. Repro case is part of the patch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21156) SharedWorkOptimizer may preserve filter in TS incorrectly
[
https://issues.apache.org/jira/browse/HIVE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751853#comment-16751853
]
Hive QA commented on HIVE-21156:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956224/HIVE-21156.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15710 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15788/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15788/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15788/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12956224 - PreCommit-HIVE-Build
> SharedWorkOptimizer may preserve filter in TS incorrectly
> -
>
> Key: HIVE-21156
> URL: https://issues.apache.org/jira/browse/HIVE-21156
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21156.01.patch, HIVE-21156.02.patch,
> HIVE-21156.02.patch, HIVE-21156.patch
>
>
> For some scan reutilizations, we may end up with a Filter expression
> associated with the scan that should be removed by the optimizer. This can
> lead to incorrect results. Repro case is part of the patch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-11708) Logical operators raises ClassCastExceptions with NULL
[
https://issues.apache.org/jira/browse/HIVE-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751987#comment-16751987
]
Hive QA commented on HIVE-11708:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956262/HIVE-11708.02.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15721 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15791/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15791/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15791/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12956262 - PreCommit-HIVE-Build
> Logical operators raises ClassCastExceptions with NULL
> --
>
> Key: HIVE-11708
> URL: https://issues.apache.org/jira/browse/HIVE-11708
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.13.0, 1.2.1
>Reporter: Satoshi Tagomori
>Assignee: Ryu Kobayashi
>Priority: Major
> Attachments: HIVE-11708.01.patch, HIVE-11708.02.patch
>
>
> According to Language Manual UDF, logical operators returns NULL if one of
> arguments is NULL.
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-LogicalOperators
> But query below fails with ClassCastException.
> {code}
> SELECT COUNT(*) AS c
> FROM tbl
> WHERE 1=1 AND NULL
> {code}
> Exception (on 0.13):
> {noformat}
> 15/08/27 08:56:23 ERROR ql.Driver: FAILED: ClassCastException
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
> java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd.initialize(GenericUDFOPAnd.java:52)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:116)
> at
> org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:231)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:934)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:1128)
> at
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:78)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:132)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:109)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:184)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:9716)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:9672)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3208)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3005)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:8228)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8183)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9015)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9281)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:427)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:323)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:980)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1045)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916)
>
[jira] [Updated] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated HIVE-21052: --- Status: Open (was: Patch Available) > Make sure transactions get cleaned if they are aborted before addPartitions > is called > - > > Key: HIVE-21052 > URL: https://issues.apache.org/jira/browse/HIVE-21052 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Jaume M >Assignee: Jaume M >Priority: Critical > Attachments: Aborted Txn w_Direct Write.pdf, HIVE-21052.1.patch, > HIVE-21052.2.patch, HIVE-21052.3.patch, HIVE-21052.4.patch, > HIVE-21052.5.patch, HIVE-21052.6.patch, HIVE-21052.7.patch, > HIVE-21052.8.patch, HIVE-21052.9.patch > > > If the transaction is aborted between openTxn and addPartitions and data has > been written on the table the transaction manager will think it's an empty > transaction and no cleaning will be done. > This is currently an issue in the streaming API and in micromanaged tables. > As proposed by [~ekoifman] this can be solved by: > * Writing an entry with a special marker to TXN_COMPONENTS at openTxn and > when addPartitions is called remove this entry from TXN_COMPONENTS and add > the corresponding partition entry to TXN_COMPONENTS. > * If the cleaner finds and entry with a special marker in TXN_COMPONENTS that > specifies that a transaction was opened and it was aborted it must generate > jobs for the worker for every possible partition available. > cc [~ewohlstadter] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21164) ACID: explore how we can avoid a move step during compaction
[ https://issues.apache.org/jira/browse/HIVE-21164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta reassigned HIVE-21164: --- Assignee: Vaibhav Gumashta > ACID: explore how we can avoid a move step during compaction > > > Key: HIVE-21164 > URL: https://issues.apache.org/jira/browse/HIVE-21164 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > > Currently, we write compacted data to a temporary location and then move the > files to a final location, which is an expensive operation on some cloud file > systems. Since HIVE-20823 is already in, it can control the visibility of > compacted data for the readers. Therefore, we can perhaps avoid writing data > to a temporary location and directly write compacted data to the intended > final path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21083) Remove the requirement to specify the truststore location when TLS to the database is turned on
[
https://issues.apache.org/jira/browse/HIVE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751814#comment-16751814
]
Hive QA commented on HIVE-21083:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956215/HIVE-21083.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15708 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestSSL.testMetastoreWithSSL (batchId=260)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15787/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15787/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15787/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12956215 - PreCommit-HIVE-Build
> Remove the requirement to specify the truststore location when TLS to the
> database is turned on
> ---
>
> Key: HIVE-21083
> URL: https://issues.apache.org/jira/browse/HIVE-21083
> Project: Hive
> Issue Type: Improvement
> Components: Metastore, Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Morio Ramdenbourg
>Assignee: Morio Ramdenbourg
>Priority: Major
> Attachments: HIVE-21083.2.patch, HIVE-21083.patch
>
>
> In the current implementation,
> [ObjectStore.configureSSL|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L349-L382]
> throws an exception if TLS to the database is turned on
> (_metastore.dbaccess.ssl.use.SSL_) but a truststore file location
> (_metastore.dbaccess.ssl.truststore.path_) is not specified.
> However, according to the [JSSE (Java 8)
> documentation|https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#InstallationAndCustomization],
> the Java truststore file location system property
> (_javax.net.ssl.trustStore_) defaults to using the "_jssecacerts_, if it
> exists. Otherwise, _cacerts_" files. These are the default truststores that
> come with the Java installation and contain a list of well-known certificate
> authorities.
> It was identified that one valid way of configuring TLS is by adding to these
> default files. In that case, no changes to the truststore properties are
> necessary. We should support this case by changing the following logic to
> remove the requirement for the truststore file location config property:
> {code:java}
> String trustStorePath = MetastoreConf.getVar(conf,
> ConfVars.DBACCESS_SSL_TRUSTSTORE_PATH).trim();
> if (trustStorePath.isEmpty()) {
> throw new IllegalArgumentException("SSL to the database store has
> been enabled but " +
> ConfVars.DBACCESS_SSL_TRUSTSTORE_PATH.toString() + " is empty. "
> + "Set this property to enable SSL.");
> }
> {code}
> We should also loosen the requirement on the truststore password if the user
> decides to use the Java defaults
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21157) Slf4j warning again from 2.X.X
[
https://issues.apache.org/jira/browse/HIVE-21157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryu Kobayashi updated HIVE-21157:
-
Resolution: Won't Fix
Status: Resolved (was: Patch Available)
> Slf4j warning again from 2.X.X
> --
>
> Key: HIVE-21157
> URL: https://issues.apache.org/jira/browse/HIVE-21157
> Project: Hive
> Issue Type: Bug
> Components: CLI
>Affects Versions: 2.0.0
>Reporter: Ryu Kobayashi
>Assignee: Ryu Kobayashi
>Priority: Minor
> Attachments: HIVE-21157.01.patch
>
>
> The warning of Slf4j has occurred again since 2.0.0:
> {code}
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21157) Slf4j warning again from 2.X.X
[
https://issues.apache.org/jira/browse/HIVE-21157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751828#comment-16751828
]
Ryu Kobayashi commented on HIVE-21157:
--
[~ashutoshc] I see. I wanted to exclude this slf4j's warning but I will
invalidate this ticket.
> Slf4j warning again from 2.X.X
> --
>
> Key: HIVE-21157
> URL: https://issues.apache.org/jira/browse/HIVE-21157
> Project: Hive
> Issue Type: Bug
> Components: CLI
>Affects Versions: 2.0.0
>Reporter: Ryu Kobayashi
>Assignee: Ryu Kobayashi
>Priority: Minor
> Attachments: HIVE-21157.01.patch
>
>
> The warning of Slf4j has occurred again since 2.0.0:
> {code}
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21163) ParseUtils.parseQueryAndGetSchema fails on views with global limit
[
https://issues.apache.org/jira/browse/HIVE-21163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Teddy Choi reassigned HIVE-21163:
-
Assignee: Teddy Choi
> ParseUtils.parseQueryAndGetSchema fails on views with global limit
> --
>
> Key: HIVE-21163
> URL: https://issues.apache.org/jira/browse/HIVE-21163
> Project: Hive
> Issue Type: Bug
>Reporter: Eric Wohlstadter
>Assignee: Teddy Choi
>Priority: Major
>
> {code:java}
> hive> USE tpcds_bin_partitioned_orc_1000;
> hive> CREATE VIEW profit_view AS SELECT ss_net_profit, d_date FROM
> store_sales, date_dim WHERE d_date = ss_sold_date LIMIT 100;
> hive> SELECT get_splits("SELECT * from profit_view", 0);
> Error: java.io.IOException:
> org.apache.hadoop.hive.ql.parse.SemanticException: View profit_view is
> corresponding to HiveSortLimit#3447, rather than a HiveProject.
> (state=,code=0)
> {code}
> This works fine if the view doesn't have a global limit.
> It also works fine if you define a view without a global limit, and then
> apply a limit on top of the view.
> {{Calcite.genLogicalPlan}} is expecting a {{HiveProject}} root but when going
> through {{ParseUtils.parseQueryAndGetSchema}} the {{HiveSortLimit}} appears
> at the root. Perhaps it is simply missing a step to wrap the limit with a
> projection?
> {code}
> Caused by: org.apache.hadoop.hive.ql.parse.SemanticException: View
> profit_view is corresponding to HiveSortLimit#2275, rather than a HiveProject.
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4931)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1741)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1689)
> at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:118)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:1043)
> at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:154)
> at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:111)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1448)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genLogicalPlan(CalcitePlanner.java:395)
> at
> org.apache.hadoop.hive.ql.parse.ParseUtils.parseQueryAndGetSchema(ParseUtils.java:561)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDTFGetSplits.createPlanFragment(GenericUDTFGetSplits.java:254)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called
[
https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751895#comment-16751895
]
Hive QA commented on HIVE-21052:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956234/HIVE-21052.9.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15714 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15789/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15789/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15789/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12956234 - PreCommit-HIVE-Build
> Make sure transactions get cleaned if they are aborted before addPartitions
> is called
> -
>
> Key: HIVE-21052
> URL: https://issues.apache.org/jira/browse/HIVE-21052
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Jaume M
>Assignee: Jaume M
>Priority: Critical
> Attachments: Aborted Txn w_Direct Write.pdf, HIVE-21052.1.patch,
> HIVE-21052.2.patch, HIVE-21052.3.patch, HIVE-21052.4.patch,
> HIVE-21052.5.patch, HIVE-21052.6.patch, HIVE-21052.7.patch,
> HIVE-21052.8.patch, HIVE-21052.9.patch
>
>
> If the transaction is aborted between openTxn and addPartitions and data has
> been written on the table the transaction manager will think it's an empty
> transaction and no cleaning will be done.
> This is currently an issue in the streaming API and in micromanaged tables.
> As proposed by [~ekoifman] this can be solved by:
> * Writing an entry with a special marker to TXN_COMPONENTS at openTxn and
> when addPartitions is called remove this entry from TXN_COMPONENTS and add
> the corresponding partition entry to TXN_COMPONENTS.
> * If the cleaner finds and entry with a special marker in TXN_COMPONENTS that
> specifies that a transaction was opened and it was aborted it must generate
> jobs for the worker for every possible partition available.
> cc [~ewohlstadter]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-11708) Logical operators raises ClassCastExceptions with NULL
[
https://issues.apache.org/jira/browse/HIVE-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751941#comment-16751941
]
Hive QA commented on HIVE-11708:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
32s{color} | {color:blue} ql in master has 2304 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 2 new + 5 unchanged - 0 fixed
= 7 total (was 5) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 55s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15791/dev-support/hive-personality.sh
|
| git revision | master / 1327d47 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15791/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15791/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Logical operators raises ClassCastExceptions with NULL
> --
>
> Key: HIVE-11708
> URL: https://issues.apache.org/jira/browse/HIVE-11708
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.13.0, 1.2.1
>Reporter: Satoshi Tagomori
>Assignee: Ryu Kobayashi
>Priority: Major
> Attachments: HIVE-11708.01.patch, HIVE-11708.02.patch
>
>
> According to Language Manual UDF, logical operators returns NULL if one of
> arguments is NULL.
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-LogicalOperators
> But query below fails with ClassCastException.
> {code}
> SELECT COUNT(*) AS c
> FROM tbl
> WHERE 1=1 AND NULL
> {code}
> Exception (on 0.13):
> {noformat}
> 15/08/27 08:56:23 ERROR ql.Driver: FAILED: ClassCastException
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
> java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd.initialize(GenericUDFOPAnd.java:52)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:116)
> at
>
[jira] [Commented] (HIVE-21132) Semi join edge is not being removed despite max bloomfilter entries set to 1
[
https://issues.apache.org/jira/browse/HIVE-21132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751699#comment-16751699
]
Hive QA commented on HIVE-21132:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956191/HIVE-21132.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15710 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15784/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15784/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15784/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12956191 - PreCommit-HIVE-Build
> Semi join edge is not being removed despite max bloomfilter entries set to 1
>
>
> Key: HIVE-21132
> URL: https://issues.apache.org/jira/browse/HIVE-21132
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-21132.1.patch, HIVE-21132.2.patch,
> HIVE-21132.3.patch
>
>
> * Reproducer
> {code:sql}
> --! qt:dataset:lineitem
> --! qt:dataset:part
> --! qt:dataset:src
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> --set hive.compute.query.using.stats=false;
> set hive.mapred.mode=nonstrict;
> set hive.explain.user=false;
> set hive.optimize.ppd=true;
> set hive.ppd.remove.duplicatefilters=true;
> set hive.tez.dynamic.partition.pruning=true;
> set hive.tez.dynamic.semijoin.reduction=true;
> set hive.optimize.metadataonly=false;
> set hive.optimize.index.filter=true;
> set hive.stats.autogather=true;
> set hive.tez.bigtable.minsize.semijoin.reduction=1;
> set hive.tez.min.bloom.filter.entries=1;
> set hive.stats.fetch.column.stats=true;
> set hive.tez.bloom.filter.factor=1.0f;
> set hive.auto.convert.join=false;
> set hive.optimize.shared.work=false;
> create database tpch_test;
> use tpch_test;
> CREATE TABLE `customer`(
> `c_custkey` bigint,
> `c_name` string,
> `c_address` string,
> `c_nationkey` bigint,
> `c_phone` string,
> `c_acctbal` double,
> `c_mktsegment` string,
> `c_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543026723');
> CREATE TABLE `lineitem`(
> `l_orderkey` bigint,
> `l_partkey` bigint,
> `l_suppkey` bigint,
> `l_linenumber` int,
> `l_quantity` double,
> `l_extendedprice` double,
> `l_discount` double,
> `l_tax` double,
> `l_returnflag` string,
> `l_linestatus` string,
> `l_shipdate` string,
> `l_commitdate` string,
> `l_receiptdate` string,
> `l_shipinstruct` string,
> `l_shipmode` string,
> `l_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543027179');
> CREATE TABLE `orders`(
> `o_orderkey` bigint,
> `o_custkey` bigint,
> `o_orderstatus` string,
> `o_totalprice` double,
> `o_orderdate` string,
> `o_orderpriority` string,
> `o_clerk` string,
> `o_shippriority` int,
> `o_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543026824');
> alter table customer update statistics
> set('numRows'='15000','rawDataSize'='8633707142');
> alter table lineitem update statistics
> set('numRows'='589709','rawDataSize'='184245066955');
> alter table orders update statistics
> set('numRows'='15','rawDataSize'='46741318253');
>
[jira] [Updated] (HIVE-11708) Logical operators raises ClassCastExceptions with NULL
[
https://issues.apache.org/jira/browse/HIVE-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryu Kobayashi updated HIVE-11708:
-
Attachment: HIVE-11708.02.patch
> Logical operators raises ClassCastExceptions with NULL
> --
>
> Key: HIVE-11708
> URL: https://issues.apache.org/jira/browse/HIVE-11708
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.13.0, 1.2.1
>Reporter: Satoshi Tagomori
>Assignee: Ryu Kobayashi
>Priority: Major
> Attachments: HIVE-11708.01.patch, HIVE-11708.02.patch
>
>
> According to Language Manual UDF, logical operators returns NULL if one of
> arguments is NULL.
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-LogicalOperators
> But query below fails with ClassCastException.
> {code}
> SELECT COUNT(*) AS c
> FROM tbl
> WHERE 1=1 AND NULL
> {code}
> Exception (on 0.13):
> {noformat}
> 15/08/27 08:56:23 ERROR ql.Driver: FAILED: ClassCastException
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
> java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd.initialize(GenericUDFOPAnd.java:52)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:116)
> at
> org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:231)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:934)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:1128)
> at
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:78)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:132)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:109)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:184)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:9716)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:9672)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3208)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3005)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:8228)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8183)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9015)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9281)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:427)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:323)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:980)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1045)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906)
> {noformat}
> I confirmed that Hive 1.2.1 of HDP2.3 Sandbox also raises this exception.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21126) Allow session level queries in LlapBaseInputFormat#getSplits() before actual get_splits() call
[
https://issues.apache.org/jira/browse/HIVE-21126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Teddy Choi updated HIVE-21126:
--
Fix Version/s: 3.2.0
4.0.0
> Allow session level queries in LlapBaseInputFormat#getSplits() before actual
> get_splits() call
> --
>
> Key: HIVE-21126
> URL: https://issues.apache.org/jira/browse/HIVE-21126
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Affects Versions: 3.1.1
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-21126.1.patch, HIVE-21126.2.patch,
> HIVE-21126.3.patch
>
>
> Facilitate execution of session level queries before \{{select get_splits()}}
> call. This will allow us to set params like \{{tez.grouping.split-count}}
> which can be taken into consideration while splits calculation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21126) Allow session level queries in LlapBaseInputFormat#getSplits() before actual get_splits() call
[
https://issues.apache.org/jira/browse/HIVE-21126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Teddy Choi updated HIVE-21126:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Allow session level queries in LlapBaseInputFormat#getSplits() before actual
> get_splits() call
> --
>
> Key: HIVE-21126
> URL: https://issues.apache.org/jira/browse/HIVE-21126
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Affects Versions: 3.1.1
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-21126.1.patch, HIVE-21126.2.patch,
> HIVE-21126.3.patch
>
>
> Facilitate execution of session level queries before \{{select get_splits()}}
> call. This will allow us to set params like \{{tez.grouping.split-count}}
> which can be taken into consideration while splits calculation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated HIVE-21052: --- Attachment: HIVE-21052.9.patch Status: Patch Available (was: Open) > Make sure transactions get cleaned if they are aborted before addPartitions > is called > - > > Key: HIVE-21052 > URL: https://issues.apache.org/jira/browse/HIVE-21052 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Jaume M >Assignee: Jaume M >Priority: Critical > Attachments: Aborted Txn w_Direct Write.pdf, HIVE-21052.1.patch, > HIVE-21052.2.patch, HIVE-21052.3.patch, HIVE-21052.4.patch, > HIVE-21052.5.patch, HIVE-21052.6.patch, HIVE-21052.7.patch, > HIVE-21052.8.patch, HIVE-21052.9.patch > > > If the transaction is aborted between openTxn and addPartitions and data has > been written on the table the transaction manager will think it's an empty > transaction and no cleaning will be done. > This is currently an issue in the streaming API and in micromanaged tables. > As proposed by [~ekoifman] this can be solved by: > * Writing an entry with a special marker to TXN_COMPONENTS at openTxn and > when addPartitions is called remove this entry from TXN_COMPONENTS and add > the corresponding partition entry to TXN_COMPONENTS. > * If the cleaner finds and entry with a special marker in TXN_COMPONENTS that > specifies that a transaction was opened and it was aborted it must generate > jobs for the worker for every possible partition available. > cc [~ewohlstadter] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20776) Run HMS filterHooks on server-side in addition to client-side
[
https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751764#comment-16751764
]
Hive QA commented on HIVE-20776:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956202/HIVE-20776.017-branch-3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15786/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15786/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15786/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12956202/HIVE-20776.017-branch-3.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12956202 - PreCommit-HIVE-Build
> Run HMS filterHooks on server-side in addition to client-side
> -
>
> Key: HIVE-20776
> URL: https://issues.apache.org/jira/browse/HIVE-20776
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Karthik Manamcheri
>Assignee: Na Li
>Priority: Major
> Attachments: HIVE-20776.001.patch, HIVE-20776.003.patch,
> HIVE-20776.004.patch, HIVE-20776.005.patch, HIVE-20776.006.patch,
> HIVE-20776.007.patch, HIVE-20776.007.patch, HIVE-20776.008.patch,
> HIVE-20776.009.patch, HIVE-20776.009.patch, HIVE-20776.010.patch,
> HIVE-20776.011.patch, HIVE-20776.011.patch, HIVE-20776.012.patch,
> HIVE-20776.013.patch, HIVE-20776.014.patch, HIVE-20776.015-branch-3.patch,
> HIVE-20776.015.branch-3.patch, HIVE-20776.015.patch,
> HIVE-20776.015_a.branch-3.patch, HIVE-20776.016-branch-3.patch,
> HIVE-20776.017-branch-3.patch
>
>
> In HMS, I noticed that all the filter hooks are applied on the client side
> (in HiveMetaStoreClient.java). Is there any reason why we can't apply the
> filters on the server-side?
> Motivation: Some newer apache projects such as Kudu use HMS for metadata
> storage. Kudu is not completely Java-based and there are interaction points
> where they have C++ clients. In such cases, it would be ideal to have
> consistent behavior from HMS side as far as filters, etc are concerned.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called
[
https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751888#comment-16751888
]
Hive QA commented on HIVE-21052:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
16s{color} | {color:blue} shims/common in master has 6 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
18s{color} | {color:blue} shims/0.23 in master has 7 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
12s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
2s{color} | {color:blue} standalone-metastore/metastore-server in master has
184 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
37s{color} | {color:blue} ql in master has 2304 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
42s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
9s{color} | {color:red} shims/common: The patch generated 1 new + 95 unchanged
- 0 fixed = 96 total (was 95) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
9s{color} | {color:red} shims/0.23: The patch generated 5 new + 69 unchanged -
0 fixed = 74 total (was 69) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 20 new + 571 unchanged - 11
fixed = 591 total (was 582) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} itests/hive-unit: The patch generated 10 new + 149
unchanged - 0 fixed = 159 total (was 149) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
13s{color} | {color:red} standalone-metastore/metastore-server generated 4 new
+ 184 unchanged - 0 fixed = 188 total (was 184) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
45s{color} | {color:red} ql generated 1 new + 2303 unchanged - 1 fixed = 2304
total (was 2304) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 42s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
| |
org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findPotentialCompactions(int)
may fail to clean up java.sql.ResultSet Obligation to clean up
[jira] [Commented] (HIVE-21083) Remove the requirement to specify the truststore location when TLS to the database is turned on
[
https://issues.apache.org/jira/browse/HIVE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751902#comment-16751902
]
Hive QA commented on HIVE-21083:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
14s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
5s{color} | {color:blue} standalone-metastore/metastore-server in master has
184 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15790/dev-support/hive-personality.sh
|
| git revision | master / 1327d47 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-common
standalone-metastore/metastore-server U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15790/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove the requirement to specify the truststore location when TLS to the
> database is turned on
> ---
>
> Key: HIVE-21083
> URL: https://issues.apache.org/jira/browse/HIVE-21083
> Project: Hive
> Issue Type: Improvement
> Components: Metastore, Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Morio Ramdenbourg
>Assignee: Morio Ramdenbourg
>Priority: Major
> Attachments: HIVE-21083.2.patch, HIVE-21083.patch
>
>
> In the current implementation,
> [ObjectStore.configureSSL|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L349-L382]
> throws an exception if TLS to the database is turned on
> (_metastore.dbaccess.ssl.use.SSL_) but a truststore file location
> (_metastore.dbaccess.ssl.truststore.path_) is not specified.
> However, according to the [JSSE (Java 8)
>
[jira] [Commented] (HIVE-21083) Remove the requirement to specify the truststore location when TLS to the database is turned on
[
https://issues.apache.org/jira/browse/HIVE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751782#comment-16751782
]
Hive QA commented on HIVE-21083:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
12s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
5s{color} | {color:blue} standalone-metastore/metastore-server in master has
184 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
7s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15787/dev-support/hive-personality.sh
|
| git revision | master / 1327d47 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-common
standalone-metastore/metastore-server U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15787/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove the requirement to specify the truststore location when TLS to the
> database is turned on
> ---
>
> Key: HIVE-21083
> URL: https://issues.apache.org/jira/browse/HIVE-21083
> Project: Hive
> Issue Type: Improvement
> Components: Metastore, Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Morio Ramdenbourg
>Assignee: Morio Ramdenbourg
>Priority: Major
> Attachments: HIVE-21083.patch
>
>
> In the current implementation,
> [ObjectStore.configureSSL|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L349-L382]
> throws an exception if TLS to the database is turned on
> (_metastore.dbaccess.ssl.use.SSL_) but a truststore file location
> (_metastore.dbaccess.ssl.truststore.path_) is not specified.
> However, according to the [JSSE (Java 8)
>
[jira] [Commented] (HIVE-21156) SharedWorkOptimizer may preserve filter in TS incorrectly
[
https://issues.apache.org/jira/browse/HIVE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751723#comment-16751723
]
Hive QA commented on HIVE-21156:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 2304 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15785/dev-support/hive-personality.sh
|
| git revision | master / 1327d47 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15785/yetus/whitespace-eol.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15785/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> SharedWorkOptimizer may preserve filter in TS incorrectly
> -
>
> Key: HIVE-21156
> URL: https://issues.apache.org/jira/browse/HIVE-21156
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21156.01.patch, HIVE-21156.02.patch,
> HIVE-21156.02.patch, HIVE-21156.patch
>
>
> For some scan reutilizations, we may end up with a Filter expression
> associated with the scan that should be removed by the optimizer. This can
> lead to incorrect results. Repro case is part of the patch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21156) SharedWorkOptimizer may preserve filter in TS incorrectly
[ https://issues.apache.org/jira/browse/HIVE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21156: --- Attachment: HIVE-21156.02.patch > SharedWorkOptimizer may preserve filter in TS incorrectly > - > > Key: HIVE-21156 > URL: https://issues.apache.org/jira/browse/HIVE-21156 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21156.01.patch, HIVE-21156.02.patch, > HIVE-21156.02.patch, HIVE-21156.patch > > > For some scan reutilizations, we may end up with a Filter expression > associated with the scan that should be removed by the optimizer. This can > lead to incorrect results. Repro case is part of the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21153) task run in parallel cause ArrayIndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/HIVE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yx91490 updated HIVE-21153: --- Description: It happens occasionally when we run sql in parallel and the job is blocked.the reason is HiveConf class is not thread safe, and SessionState.createSessionDirs() is not Synchronized. the error message: java.lang.ArrayIndexOutOfBoundsException: 22 at java.util.ArrayList.add(ArrayList.java:459) at org.apache.hadoop.hive.conf.HiveConf.setupRestrictList(HiveConf.java:3216) at org.apache.hadoop.hive.conf.HiveConf.addToRestrictList(HiveConf.java:3191) at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:651) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:547) at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:78) was: It happens occasionally when we run sql in parallel.the reason is HiveConf class is not thread safe, and SessionState.createSessionDirs() is not Synchronized. the error message: java.lang.ArrayIndexOutOfBoundsException: 22 at java.util.ArrayList.add(ArrayList.java:459) at org.apache.hadoop.hive.conf.HiveConf.setupRestrictList(HiveConf.java:3216) at org.apache.hadoop.hive.conf.HiveConf.addToRestrictList(HiveConf.java:3191) at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:651) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:547) at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:78) > task run in parallel cause ArrayIndexOutOfBoundsException > - > > Key: HIVE-21153 > URL: https://issues.apache.org/jira/browse/HIVE-21153 > Project: Hive > Issue Type: Bug >Reporter: yx91490 >Priority: Major > > It happens occasionally when we run sql in parallel and the job is > blocked.the reason is HiveConf class is not thread safe, and > SessionState.createSessionDirs() is not Synchronized. > the error message: > java.lang.ArrayIndexOutOfBoundsException: 22 > at java.util.ArrayList.add(ArrayList.java:459) > at org.apache.hadoop.hive.conf.HiveConf.setupRestrictList(HiveConf.java:3216) > at org.apache.hadoop.hive.conf.HiveConf.addToRestrictList(HiveConf.java:3191) > at > org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:651) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:547) > at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:78) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21083) Remove the requirement to specify the truststore location when TLS to the database is turned on
[
https://issues.apache.org/jira/browse/HIVE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751920#comment-16751920
]
Hive QA commented on HIVE-21083:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956251/HIVE-21083.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15708 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15790/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15790/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15790/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12956251 - PreCommit-HIVE-Build
> Remove the requirement to specify the truststore location when TLS to the
> database is turned on
> ---
>
> Key: HIVE-21083
> URL: https://issues.apache.org/jira/browse/HIVE-21083
> Project: Hive
> Issue Type: Improvement
> Components: Metastore, Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Morio Ramdenbourg
>Assignee: Morio Ramdenbourg
>Priority: Major
> Attachments: HIVE-21083.2.patch, HIVE-21083.patch
>
>
> In the current implementation,
> [ObjectStore.configureSSL|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L349-L382]
> throws an exception if TLS to the database is turned on
> (_metastore.dbaccess.ssl.use.SSL_) but a truststore file location
> (_metastore.dbaccess.ssl.truststore.path_) is not specified.
> However, according to the [JSSE (Java 8)
> documentation|https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#InstallationAndCustomization],
> the Java truststore file location system property
> (_javax.net.ssl.trustStore_) defaults to using the "_jssecacerts_, if it
> exists. Otherwise, _cacerts_" files. These are the default truststores that
> come with the Java installation and contain a list of well-known certificate
> authorities.
> It was identified that one valid way of configuring TLS is by adding to these
> default files. In that case, no changes to the truststore properties are
> necessary. We should support this case by changing the following logic to
> remove the requirement for the truststore file location config property:
> {code:java}
> String trustStorePath = MetastoreConf.getVar(conf,
> ConfVars.DBACCESS_SSL_TRUSTSTORE_PATH).trim();
> if (trustStorePath.isEmpty()) {
> throw new IllegalArgumentException("SSL to the database store has
> been enabled but " +
> ConfVars.DBACCESS_SSL_TRUSTSTORE_PATH.toString() + " is empty. "
> + "Set this property to enable SSL.");
> }
> {code}
> We should also loosen the requirement on the truststore password if the user
> decides to use the Java defaults
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21156) SharedWorkOptimizer may preserve filter in TS incorrectly
[ https://issues.apache.org/jira/browse/HIVE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21156: --- Resolution: Fixed Fix Version/s: 3.1.2 3.2.0 4.0.0 Status: Resolved (was: Patch Available) Pushed to master, branch-3, branch-3.1. Thanks for reviewing [~vgarg] > SharedWorkOptimizer may preserve filter in TS incorrectly > - > > Key: HIVE-21156 > URL: https://issues.apache.org/jira/browse/HIVE-21156 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0, 3.2.0, 3.1.2 > > Attachments: HIVE-21156.01.patch, HIVE-21156.02.patch, > HIVE-21156.02.patch, HIVE-21156.patch > > > For some scan reutilizations, we may end up with a Filter expression > associated with the scan that should be removed by the optimizer. This can > lead to incorrect results. Repro case is part of the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-11708) Logical operators raises ClassCastExceptions with NULL
[
https://issues.apache.org/jira/browse/HIVE-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751826#comment-16751826
]
Ryu Kobayashi commented on HIVE-11708:
--
[~ashutoshc] Thanks! I added test.
> Logical operators raises ClassCastExceptions with NULL
> --
>
> Key: HIVE-11708
> URL: https://issues.apache.org/jira/browse/HIVE-11708
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.13.0, 1.2.1
>Reporter: Satoshi Tagomori
>Assignee: Ryu Kobayashi
>Priority: Major
> Attachments: HIVE-11708.01.patch, HIVE-11708.02.patch
>
>
> According to Language Manual UDF, logical operators returns NULL if one of
> arguments is NULL.
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-LogicalOperators
> But query below fails with ClassCastException.
> {code}
> SELECT COUNT(*) AS c
> FROM tbl
> WHERE 1=1 AND NULL
> {code}
> Exception (on 0.13):
> {noformat}
> 15/08/27 08:56:23 ERROR ql.Driver: FAILED: ClassCastException
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
> java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd.initialize(GenericUDFOPAnd.java:52)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:116)
> at
> org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:231)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:934)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:1128)
> at
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:78)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:132)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:109)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:184)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:9716)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:9672)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3208)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3005)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:8228)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8183)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9015)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9281)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:427)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:323)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:980)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1045)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906)
> {noformat}
> I confirmed that Hive 1.2.1 of HDP2.3 Sandbox also raises this exception.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-11708) Logical operators raises ClassCastExceptions with NULL
[
https://issues.apache.org/jira/browse/HIVE-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751883#comment-16751883
]
Ashutosh Chauhan commented on HIVE-11708:
-
+1 pending tests
> Logical operators raises ClassCastExceptions with NULL
> --
>
> Key: HIVE-11708
> URL: https://issues.apache.org/jira/browse/HIVE-11708
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.13.0, 1.2.1
>Reporter: Satoshi Tagomori
>Assignee: Ryu Kobayashi
>Priority: Major
> Attachments: HIVE-11708.01.patch, HIVE-11708.02.patch
>
>
> According to Language Manual UDF, logical operators returns NULL if one of
> arguments is NULL.
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-LogicalOperators
> But query below fails with ClassCastException.
> {code}
> SELECT COUNT(*) AS c
> FROM tbl
> WHERE 1=1 AND NULL
> {code}
> Exception (on 0.13):
> {noformat}
> 15/08/27 08:56:23 ERROR ql.Driver: FAILED: ClassCastException
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
> java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd.initialize(GenericUDFOPAnd.java:52)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:116)
> at
> org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:231)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:934)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:1128)
> at
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:78)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:132)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:109)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:184)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:9716)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:9672)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3208)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3005)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:8228)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8183)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9015)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9281)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:427)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:323)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:980)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1045)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906)
> {noformat}
> I confirmed that Hive 1.2.1 of HDP2.3 Sandbox also raises this exception.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-10773) MapJoinOperator times out on loading HashTable
[ https://issues.apache.org/jira/browse/HIVE-10773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751954#comment-16751954 ] Zhihua Deng commented on HIVE-10773: We met the same issue running job on mapreduce. In one of our cases, the 99% stored in dumped hashtable are one-to-one kv mappings. Even though the file is not larger than 10m, the mapper tasks more than half an hour to load the table with about 200,000 keys. > MapJoinOperator times out on loading HashTable > -- > > Key: HIVE-10773 > URL: https://issues.apache.org/jira/browse/HIVE-10773 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.14.0 >Reporter: frank luo >Priority: Major > > When running a map join, depends on data, it might timeout with last two > lines in the log as below. And when I do "set > mapreduce.task.timeout=60;", which is defaulted to 30, the query can > go through fine. The size of hashtable file is roughly 400M. > 2015-05-20 13:27:03,237 INFO [main] > org.apache.hadoop.hive.ql.exec.MapJoinOperator: *** Load from HashTable > for input file: > hdfs://nameservice1/tmp/hive/jluo/2ee8914d-1cef-4af4-aac6-51f64d630346/hive_2015-05-20_13-13-35_335_1565066409090716856-1/-mr-10007/00_0 > 2015-05-20 13:27:03,237 INFO [main] > org.apache.hadoop.hive.ql.exec.MapJoinOperator: Load back 1 hashtable file > from tmp file > uri:file:/data/12/hadoop/yarn/local/usercache/xxy/appcache/application_1430337284339_2087 > /container_1430337284339_2087_01_03/Stage-3.tar.gz/MapJoin-mapfile31--.hashtable > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HIVE-21125) Hive does not check for dependent materialized views when issuing a DROP TABLE command
[
https://issues.apache.org/jira/browse/HIVE-21125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Taylor Cox resolved HIVE-21125.
---
Resolution: Invalid
Looks like this fix is checked into Hive 3.2. Backporting is a separate
discussion.
> Hive does not check for dependent materialized views when issuing a DROP
> TABLE command
> --
>
> Key: HIVE-21125
> URL: https://issues.apache.org/jira/browse/HIVE-21125
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Affects Versions: 3.1.0
>Reporter: Taylor Cox
>Assignee: Taylor Cox
>Priority: Major
>
> Dropping a table leads to undefined behavior when that table is the source of
> an existing materialized view. The following behavior is observed:
>
> * Table still appears in 'show tables' despite not being in metastore
> * Actions on table hang and then display a "could not fetch table" error
> * Rebuilding any dependent materialized view has same error
> It seems that the root cause is the fact that users are allowed to issue a
> DROP TABLE command against a table even if there is a materialized view using
> this table at the time. This is not something I have seen other query
> languages permit.
> Repro steps: Launch these commands from any Hive 3 client:
> {code:java}
> create table footable (id int); insert into footable values (1), (2), (3);
> create materialized view mv_footable as select count(*) from footable;
> drop table footable;
> --These lines have unexpected behavior
> show tables;
> select * from footable;
> alter materialized view mv_footable rebuild;{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21125) Hive does not check for dependent materialized views when issuing a DROP TABLE command
[
https://issues.apache.org/jira/browse/HIVE-21125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751739#comment-16751739
]
Taylor Cox commented on HIVE-21125:
---
https://jira.apache.org/jira/browse/HIVE-18620
> Hive does not check for dependent materialized views when issuing a DROP
> TABLE command
> --
>
> Key: HIVE-21125
> URL: https://issues.apache.org/jira/browse/HIVE-21125
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Affects Versions: 3.1.0
>Reporter: Taylor Cox
>Assignee: Taylor Cox
>Priority: Major
>
> Dropping a table leads to undefined behavior when that table is the source of
> an existing materialized view. The following behavior is observed:
>
> * Table still appears in 'show tables' despite not being in metastore
> * Actions on table hang and then display a "could not fetch table" error
> * Rebuilding any dependent materialized view has same error
> It seems that the root cause is the fact that users are allowed to issue a
> DROP TABLE command against a table even if there is a materialized view using
> this table at the time. This is not something I have seen other query
> languages permit.
> Repro steps: Launch these commands from any Hive 3 client:
> {code:java}
> create table footable (id int); insert into footable values (1), (2), (3);
> create materialized view mv_footable as select count(*) from footable;
> drop table footable;
> --These lines have unexpected behavior
> show tables;
> select * from footable;
> alter materialized view mv_footable rebuild;{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21165) ACID: pass query hint to the writers to write hive.acid.key.index
[ https://issues.apache.org/jira/browse/HIVE-21165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta reassigned HIVE-21165: --- Assignee: Vaibhav Gumashta > ACID: pass query hint to the writers to write hive.acid.key.index > - > > Key: HIVE-21165 > URL: https://issues.apache.org/jira/browse/HIVE-21165 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > > For the query based compactor from HIVE-20699, the compaction runs as a sql > query. However, this mechanism skips over writing hive.acid.key.index for > each stripe, which is used to skip over stripes that are not supposed to be > read. We need a way to pass a query hint to the writer so that it can write > this index data, when invoked from a sql query. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21079) Replicate column statistics for partitions of partitioned Hive table.
[ https://issues.apache.org/jira/browse/HIVE-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750850#comment-16750850 ] ASF GitHub Bot commented on HIVE-21079: --- Github user ashutosh-bapat closed the pull request at: https://github.com/apache/hive/pull/522 > Replicate column statistics for partitions of partitioned Hive table. > - > > Key: HIVE-21079 > URL: https://issues.apache.org/jira/browse/HIVE-21079 > Project: Hive > Issue Type: Sub-task >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21079.01.patch, HIVE-21079.02.patch > > > This task is for replicating statistics for partitions of a partitioned Hive > table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-11708) Logical operators raises ClassCastExceptions with NULL
[
https://issues.apache.org/jira/browse/HIVE-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750896#comment-16750896
]
Hive QA commented on HIVE-11708:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956071/HIVE-11708.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15710 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.org.apache.hadoop.hive.ql.TestWarehouseExternalDir
(batchId=243)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testExternalDefaultPaths
(batchId=243)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15773/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15773/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15773/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12956071 - PreCommit-HIVE-Build
> Logical operators raises ClassCastExceptions with NULL
> --
>
> Key: HIVE-11708
> URL: https://issues.apache.org/jira/browse/HIVE-11708
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.13.0, 1.2.1
>Reporter: Satoshi Tagomori
>Assignee: Ryu Kobayashi
>Priority: Major
> Attachments: HIVE-11708.01.patch
>
>
> According to Language Manual UDF, logical operators returns NULL if one of
> arguments is NULL.
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-LogicalOperators
> But query below fails with ClassCastException.
> {code}
> SELECT COUNT(*) AS c
> FROM tbl
> WHERE 1=1 AND NULL
> {code}
> Exception (on 0.13):
> {noformat}
> 15/08/27 08:56:23 ERROR ql.Driver: FAILED: ClassCastException
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
> java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd.initialize(GenericUDFOPAnd.java:52)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:116)
> at
> org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:231)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:934)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:1128)
> at
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:78)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:132)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:109)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:184)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:9716)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:9672)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3208)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3005)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:8228)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8183)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9015)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9281)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
> at
[jira] [Commented] (HIVE-21159) Modify Merge statement logic to perform Update split early
[
https://issues.apache.org/jira/browse/HIVE-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750946#comment-16750946
]
Hive QA commented on HIVE-21159:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
37s{color} | {color:blue} ql in master has 2304 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
45s{color} | {color:red} ql: The patch generated 38 new + 1021 unchanged - 1
fixed = 1059 total (was 1022) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15774/dev-support/hive-personality.sh
|
| git revision | master / a7e704c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15774/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15774/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Modify Merge statement logic to perform Update split early
> --
>
> Key: HIVE-21159
> URL: https://issues.apache.org/jira/browse/HIVE-21159
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-21159.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21078) Replicate column and table level statistics for unpartitioned Hive tables
[ https://issues.apache.org/jira/browse/HIVE-21078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750855#comment-16750855 ] ASF GitHub Bot commented on HIVE-21078: --- Github user ashutosh-bapat closed the pull request at: https://github.com/apache/hive/pull/511 > Replicate column and table level statistics for unpartitioned Hive tables > - > > Key: HIVE-21078 > URL: https://issues.apache.org/jira/browse/HIVE-21078 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 4.0.0 >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21078.01.patch, HIVE-21078.02.patch, > HIVE-21078.03.patch, HIVE-21078.04.patch, HIVE-21078.05.patch, > HIVE-21078.06.patch, HIVE-21078.07.patch, HIVE-21078.08.patch, > HIVE-21078.09.patch, HIVE-21078.10.patch, HIVE-21078.11.patch, > HIVE-21078.sameas.05.patch > > > This task is for replicating column and table level statistics for > unpartitioned tables. The same for partitioned tables will be worked upon in > a separate sub-task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21079) Replicate column statistics for partitions of partitioned Hive table.
[ https://issues.apache.org/jira/browse/HIVE-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750853#comment-16750853 ] ASF GitHub Bot commented on HIVE-21079: --- GitHub user ashutosh-bapat reopened a pull request: https://github.com/apache/hive/pull/522 HIVE-21079: Stats replication for partitioned table The first commit is for stats replication for partitioned table. The other two commits are fixing bugs in existing code, AFAIU. @sankarh can you please review? You can merge this pull request into a Git repository by running: $ git pull https://github.com/ashutosh-bapat/hive hive21079 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/522.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #522 commit a8b729ab1f120cd50c0ad0e096bb0f724a178838 Author: Ashutosh Bapat Date: 2019-01-15T12:06:29Z HIVE-21079: Replicate statistics for partitioned, non-transactional tables. Ashutosh Bapat commit c8aea7b85a06ab53873ce60eb08fcf0514806787 Author: Ashutosh Bapat Date: 2019-01-18T05:31:44Z HIVE-21079: ALTER PARTITION events not applied during incremental replication In AlterPartitionHandler, we set withinContext.replicationSpec.setIsMetadataOnly(true); In ImportSemanticAnalyzer.createReplImportTasks(), per code around line 1197, we do not add new PartitionSpecs and corresponding tasks. This means that we never apply an ALTER_PARTITION event during incremental load. That looks like a serious bug. Either we should check PartitionDescs irrespective of replicationSpec.setIsMetadataOnly() OR we shouldn’t set replicationSpec.setIsMetadataOnly() to true while dumping an ALTER_PARTITION event. We set replicationSpec.setIsMetadataOnly(true) for ALTER TABLE events as well, so doing that for ALTER PARTITION event looks fine. Ashutosh Bapat. commit 536492395cd5c280738c2ec1038c39036b477209 Author: Ashutosh Bapat Date: 2019-01-18T06:07:37Z HIVE-21079: Do not dump partition related events during a metadata only dump. During bootstrap metadata-only dump we do not dump partitions (See TableExport.getPartitions(). For bootstrap dump we always pass TableSpec with TABLE_ONLY set.). So don't dump partition related events for a metadata-only dump. Ashutosh Bapat. > Replicate column statistics for partitions of partitioned Hive table. > - > > Key: HIVE-21079 > URL: https://issues.apache.org/jira/browse/HIVE-21079 > Project: Hive > Issue Type: Sub-task >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21079.01.patch, HIVE-21079.02.patch > > > This task is for replicating statistics for partitions of a partitioned Hive > table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21079) Replicate column statistics for partitions of partitioned Hive table.
[ https://issues.apache.org/jira/browse/HIVE-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-21079: -- Labels: pull-request-available (was: ) > Replicate column statistics for partitions of partitioned Hive table. > - > > Key: HIVE-21079 > URL: https://issues.apache.org/jira/browse/HIVE-21079 > Project: Hive > Issue Type: Sub-task >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21079.01.patch, HIVE-21079.02.patch > > > This task is for replicating statistics for partitions of a partitioned Hive > table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-11708) Logical operators raises ClassCastExceptions with NULL
[
https://issues.apache.org/jira/browse/HIVE-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750860#comment-16750860
]
Hive QA commented on HIVE-11708:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
36s{color} | {color:blue} ql in master has 2304 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15773/dev-support/hive-personality.sh
|
| git revision | master / a7e704c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15773/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Logical operators raises ClassCastExceptions with NULL
> --
>
> Key: HIVE-11708
> URL: https://issues.apache.org/jira/browse/HIVE-11708
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.13.0, 1.2.1
>Reporter: Satoshi Tagomori
>Assignee: Ryu Kobayashi
>Priority: Major
> Attachments: HIVE-11708.01.patch
>
>
> According to Language Manual UDF, logical operators returns NULL if one of
> arguments is NULL.
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-LogicalOperators
> But query below fails with ClassCastException.
> {code}
> SELECT COUNT(*) AS c
> FROM tbl
> WHERE 1=1 AND NULL
> {code}
> Exception (on 0.13):
> {noformat}
> 15/08/27 08:56:23 ERROR ql.Driver: FAILED: ClassCastException
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
> java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableVoidObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd.initialize(GenericUDFOPAnd.java:52)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:116)
> at
> org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:231)
> at
>
[jira] [Commented] (HIVE-21079) Replicate column statistics for partitions of partitioned Hive table.
[
https://issues.apache.org/jira/browse/HIVE-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751023#comment-16751023
]
Hive QA commented on HIVE-21079:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
33s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
10s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
59s{color} | {color:blue} standalone-metastore/metastore-server in master has
184 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
37s{color} | {color:blue} ql in master has 2304 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} hcatalog/server-extensions in master has 3 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
40s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 5 new + 363 unchanged - 5
fixed = 368 total (was 368) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} itests/hive-unit: The patch generated 1 new + 565
unchanged - 3 fixed = 566 total (was 568) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 8m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 44m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15775/dev-support/hive-personality.sh
|
| git revision | master / a7e704c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15775/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15775/yetus/diff-checkstyle-itests_hive-unit.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15775/yetus/whitespace-eol.txt
|
| modules | C: standalone-metastore/metastore-common
standalone-metastore/metastore-server ql hcatalog/server-extensions
standalone-metastore/metastore-tools/tools-common itests/hive-unit U: . |
| Console output |
[jira] [Commented] (HIVE-21135) Grouping sets size can not be greater than 64 compile error
[ https://issues.apache.org/jira/browse/HIVE-21135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751030#comment-16751030 ] David Lavati commented on HIVE-21135: - Hi [~naga...@gmail.com], this has recently been addressed in HIVE-21018, but it wasn't yet a part of HDP 3.1. > Grouping sets size can not be greater than 64 compile error > --- > > Key: HIVE-21135 > URL: https://issues.apache.org/jira/browse/HIVE-21135 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.0 > Environment: HDP 3.1 ,Ubuntu environment ,mysql metastore >Reporter: Nagaraju Anthati >Priority: Blocker > > I am trying to get distinct records from an alias dataset , then the hive is > throwing compile error .The alias dataset has 8k columns , could not proceed > further , not sure if it is config related. > query : select distinct data.* from ( . some join+8kcolumns in > the result set...)data > > > error :compile error : Grouping sets size can not be grater than 64:27:x > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21079) Replicate column statistics for partitions of partitioned Hive table.
[
https://issues.apache.org/jira/browse/HIVE-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751036#comment-16751036
]
Hive QA commented on HIVE-21079:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956089/HIVE-21079.02.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15709 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15775/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15775/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15775/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12956089 - PreCommit-HIVE-Build
> Replicate column statistics for partitions of partitioned Hive table.
> -
>
> Key: HIVE-21079
> URL: https://issues.apache.org/jira/browse/HIVE-21079
> Project: Hive
> Issue Type: Sub-task
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21079.01.patch, HIVE-21079.02.patch
>
>
> This task is for replicating statistics for partitions of a partitioned Hive
> table.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21159) Modify Merge statement logic to perform Update split early
[
https://issues.apache.org/jira/browse/HIVE-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750988#comment-16750988
]
Hive QA commented on HIVE-21159:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956074/HIVE-21159.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 15709 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_subquery]
(batchId=42)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_transactional_full_acid]
(batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explain_locks]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_acid_no_masking]
(batchId=25)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_no_buckets]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[check_constraint]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_3]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[enforce_constraint_notnull]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_into_default_keyword]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[semijoin_hint]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sqlmerge]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sqlmerge_stats]
(batchId=180)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[merge_constraint_notnull]
(batchId=100)
org.apache.hadoop.hive.ql.TestTxnCommands.testMergeOnTezEdges (batchId=327)
org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testMergeOnTezEdges
(batchId=309)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15774/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15774/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15774/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12956074 - PreCommit-HIVE-Build
> Modify Merge statement logic to perform Update split early
> --
>
> Key: HIVE-21159
> URL: https://issues.apache.org/jira/browse/HIVE-21159
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-21159.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21134) Hive Build Version as UDF
[ https://issues.apache.org/jira/browse/HIVE-21134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751021#comment-16751021 ] Mani M commented on HIVE-21134: --- HI [~pvary] Pull request is available in the below url Kindly review and accept it [https://github.com/apache/hive/pull/521] > Hive Build Version as UDF > - > > Key: HIVE-21134 > URL: https://issues.apache.org/jira/browse/HIVE-21134 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Mani M >Assignee: Mani M >Priority: Minor > Labels: UDF, pull-request-available > Attachments: HIVE.21134.02.PATCH, HIVE.21134.03.PATCH, > HIVE.21134.04.PATCH, HIVE.21134.05.PATCH, HIVE.21134.06.PATCH, > HIVE.21134.07.PATCH, HIVE.21134.PATCH > > > This Jira is to get the Hive Build Version as UDF. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21143) Add rewrite rules to open/close Between operators
[ https://issues.apache.org/jira/browse/HIVE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21143: Attachment: HIVE-21143.07.patch > Add rewrite rules to open/close Between operators > - > > Key: HIVE-21143 > URL: https://issues.apache.org/jira/browse/HIVE-21143 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21143.01.patch, HIVE-21143.02.patch, > HIVE-21143.03.patch, HIVE-21143.03.patch, HIVE-21143.04.patch, > HIVE-21143.05.patch, HIVE-21143.06.patch, HIVE-21143.07.patch > > > During query compilation it's better to have BETWEEN statements in open form, > as Calcite current not considering them during simplification. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21135) Grouping sets size can not be greater than 64 compile error
[ https://issues.apache.org/jira/browse/HIVE-21135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751117#comment-16751117 ] Nagaraju Anthati commented on HIVE-21135: - Hi David , Thanks alot for the update just wondering if any there is any work around for this issue or do we need to wait until this is effective in HDP next release. Thanks Nagaraju > Grouping sets size can not be greater than 64 compile error > --- > > Key: HIVE-21135 > URL: https://issues.apache.org/jira/browse/HIVE-21135 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.0 > Environment: HDP 3.1 ,Ubuntu environment ,mysql metastore >Reporter: Nagaraju Anthati >Priority: Blocker > > I am trying to get distinct records from an alias dataset , then the hive is > throwing compile error .The alias dataset has 8k columns , could not proceed > further , not sure if it is config related. > query : select distinct data.* from ( . some join+8kcolumns in > the result set...)data > > > error :compile error : Grouping sets size can not be grater than 64:27:x > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21134) Hive Build Version as UDF
[ https://issues.apache.org/jira/browse/HIVE-21134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751146#comment-16751146 ] Peter Vary commented on HIVE-21134: --- +1 > Hive Build Version as UDF > - > > Key: HIVE-21134 > URL: https://issues.apache.org/jira/browse/HIVE-21134 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Mani M >Assignee: Mani M >Priority: Minor > Labels: UDF, pull-request-available > Attachments: HIVE.21134.02.PATCH, HIVE.21134.03.PATCH, > HIVE.21134.04.PATCH, HIVE.21134.05.PATCH, HIVE.21134.06.PATCH, > HIVE.21134.07.PATCH, HIVE.21134.PATCH > > > This Jira is to get the Hive Build Version as UDF. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21143) Add rewrite rules to open/close Between operators
[
https://issues.apache.org/jira/browse/HIVE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751120#comment-16751120
]
Hive QA commented on HIVE-21143:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
31s{color} | {color:blue} ql in master has 2304 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 4 new + 119 unchanged - 0
fixed = 123 total (was 119) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 14 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15776/dev-support/hive-personality.sh
|
| git revision | master / a7e704c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15776/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15776/yetus/whitespace-eol.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15776/yetus/whitespace-tabs.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15776/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add rewrite rules to open/close Between operators
> -
>
> Key: HIVE-21143
> URL: https://issues.apache.org/jira/browse/HIVE-21143
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21143.01.patch, HIVE-21143.02.patch,
> HIVE-21143.03.patch, HIVE-21143.03.patch, HIVE-21143.04.patch,
> HIVE-21143.05.patch, HIVE-21143.06.patch
>
>
> During query compilation it's better to have BETWEEN statements in open form,
> as Calcite current not considering them during simplification.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21143) Add rewrite rules to open/close Between operators
[
https://issues.apache.org/jira/browse/HIVE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751156#comment-16751156
]
Hive QA commented on HIVE-21143:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956103/HIVE-21143.06.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15679 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=159)
[external_jdbc_table_typeconversion.q,vector_udf_octet_length.q,schema_evol_orc_acidvec_table_update.q,materialized_view_rewrite_part_2.q,vector_decimal_5.q,vector_case_when_conversion.q,escape1.q,schema_evol_orc_acid_table_update_llap_io.q,cte_mat_5.q,acid_meta_columns_decode.q,vector_string_decimal.q,results_cache_lifetime.q,cross_prod_3.q,join46.q,dynpart_sort_optimization2.q,tez_bmj_schema_evolution.q,insert_into_default_keyword.q,bucketmapjoin4.q,vector_orc_null_check.q,semijoin7.q,uber_reduce.q,schema_evol_orc_nonvec_part_all_complex.q,is_distinct_from.q,schema_evol_text_vec_part_all_complex_llap_io.q,auto_sortmerge_join_3.q,vectorization_9.q,materialized_view_create_rewrite.q,merge2.q,join_nulls.q,bucketmapjoin2.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_wise_fileformat2]
(batchId=94)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15776/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15776/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15776/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12956103 - PreCommit-HIVE-Build
> Add rewrite rules to open/close Between operators
> -
>
> Key: HIVE-21143
> URL: https://issues.apache.org/jira/browse/HIVE-21143
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21143.01.patch, HIVE-21143.02.patch,
> HIVE-21143.03.patch, HIVE-21143.03.patch, HIVE-21143.04.patch,
> HIVE-21143.05.patch, HIVE-21143.06.patch
>
>
> During query compilation it's better to have BETWEEN statements in open form,
> as Calcite current not considering them during simplification.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21143) Add rewrite rules to open/close Between operators
[
https://issues.apache.org/jira/browse/HIVE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751208#comment-16751208
]
Hive QA commented on HIVE-21143:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
32s{color} | {color:blue} ql in master has 2304 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 4 new + 119 unchanged - 0
fixed = 123 total (was 119) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch 14 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15777/dev-support/hive-personality.sh
|
| git revision | master / a7e704c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15777/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15777/yetus/whitespace-eol.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15777/yetus/whitespace-tabs.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15777/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add rewrite rules to open/close Between operators
> -
>
> Key: HIVE-21143
> URL: https://issues.apache.org/jira/browse/HIVE-21143
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21143.01.patch, HIVE-21143.02.patch,
> HIVE-21143.03.patch, HIVE-21143.03.patch, HIVE-21143.04.patch,
> HIVE-21143.05.patch, HIVE-21143.06.patch, HIVE-21143.07.patch
>
>
> During query compilation it's better to have BETWEEN statements in open form,
> as Calcite current not considering them during simplification.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20776) Run HMS filterHooks on server-side in addition to client-side
[ https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated HIVE-20776: - Attachment: HIVE-20776.015_a.branch-3.patch > Run HMS filterHooks on server-side in addition to client-side > - > > Key: HIVE-20776 > URL: https://issues.apache.org/jira/browse/HIVE-20776 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Karthik Manamcheri >Assignee: Na Li >Priority: Major > Attachments: HIVE-20776.001.patch, HIVE-20776.003.patch, > HIVE-20776.004.patch, HIVE-20776.005.patch, HIVE-20776.006.patch, > HIVE-20776.007.patch, HIVE-20776.007.patch, HIVE-20776.008.patch, > HIVE-20776.009.patch, HIVE-20776.009.patch, HIVE-20776.010.patch, > HIVE-20776.011.patch, HIVE-20776.011.patch, HIVE-20776.012.patch, > HIVE-20776.013.patch, HIVE-20776.014.patch, HIVE-20776.015.branch-3.patch, > HIVE-20776.015.patch, HIVE-20776.015_a.branch-3.patch > > > In HMS, I noticed that all the filter hooks are applied on the client side > (in HiveMetaStoreClient.java). Is there any reason why we can't apply the > filters on the server-side? > Motivation: Some newer apache projects such as Kudu use HMS for metadata > storage. Kudu is not completely Java-based and there are interaction points > where they have C++ clients. In such cases, it would be ideal to have > consistent behavior from HMS side as far as filters, etc are concerned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20847) Review of NullScan Code
[
https://issues.apache.org/jira/browse/HIVE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20847:
---
Status: Open (was: Patch Available)
> Review of NullScan Code
> ---
>
> Key: HIVE-20847
> URL: https://issues.apache.org/jira/browse/HIVE-20847
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Affects Versions: 4.0.0, 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20847.1.patch, HIVE-20847.1.patch,
> HIVE-20847.1.patch, HIVE-20847.2.patch, HIVE-20847.3.patch, HIVE-20847.4.patch
>
>
> What got me looking at this class was the verboseness of some of the logging.
> I would like to request that we DEBUG the logging since this level of detail
> means nothing to a cluster admin.
> Also... this {{contains}} call would be better applied onto a {{HashSet}}
> instead of an {{ArrayList}}.
> {code:java|title=NullScanTaskDispatcher.java}
> private void processAlias(MapWork work, Path path, ArrayList
> aliasesAffected, ArrayList aliases) {
> // the aliases that are allowed to map to a null scan.
> ArrayList allowed = new ArrayList();
> for (String alias : aliasesAffected) {
> if (aliases.contains(alias)) {
> allowed.add(alias);
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20847) Review of NullScan Code
[
https://issues.apache.org/jira/browse/HIVE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20847:
---
Status: Patch Available (was: Open)
> Review of NullScan Code
> ---
>
> Key: HIVE-20847
> URL: https://issues.apache.org/jira/browse/HIVE-20847
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Affects Versions: 4.0.0, 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20847.1.patch, HIVE-20847.1.patch,
> HIVE-20847.1.patch, HIVE-20847.2.patch, HIVE-20847.3.patch, HIVE-20847.4.patch
>
>
> What got me looking at this class was the verboseness of some of the logging.
> I would like to request that we DEBUG the logging since this level of detail
> means nothing to a cluster admin.
> Also... this {{contains}} call would be better applied onto a {{HashSet}}
> instead of an {{ArrayList}}.
> {code:java|title=NullScanTaskDispatcher.java}
> private void processAlias(MapWork work, Path path, ArrayList
> aliasesAffected, ArrayList aliases) {
> // the aliases that are allowed to map to a null scan.
> ArrayList allowed = new ArrayList();
> for (String alias : aliasesAffected) {
> if (aliases.contains(alias)) {
> allowed.add(alias);
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21156) SharedWorkOptimizer may preserve filter in TS incorrectly
[ https://issues.apache.org/jira/browse/HIVE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21156: --- Attachment: HIVE-21156.01.patch > SharedWorkOptimizer may preserve filter in TS incorrectly > - > > Key: HIVE-21156 > URL: https://issues.apache.org/jira/browse/HIVE-21156 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21156.01.patch, HIVE-21156.patch > > > For some scan reutilizations, we may end up with a Filter expression > associated with the scan that should be removed by the optimizer. This can > lead to incorrect results. Repro case is part of the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20233) Style improvements to Operator.java
[
https://issues.apache.org/jira/browse/HIVE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-20233:
---
Summary: Style improvements to Operator.java (was: Review Operator.java)
> Style improvements to Operator.java
> ---
>
> Key: HIVE-20233
> URL: https://issues.apache.org/jira/browse/HIVE-20233
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20233.1.patch, HIVE-20233.2.patch,
> HIVE-20233.3.patch, HIVE-20233.4.patch, HIVE-20233.4.patch,
> HIVE-20233.5.patch, HIVE-20233.6.patch, HIVE-20233.7.patch, HIVE-20233.8.patch
>
>
> Various improvements to {{Operator.java}}
>
> * Code formatting
> * Remove superfluous logging guards
> * Add logging message parameters
> * Minor performance improvements
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20233) Code formatting improvements to Operator.java
[
https://issues.apache.org/jira/browse/HIVE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751290#comment-16751290
]
Jesus Camacho Rodriguez commented on HIVE-20233:
Pushed to master, thanks [~belugabehr]!
> Code formatting improvements to Operator.java
> -
>
> Key: HIVE-20233
> URL: https://issues.apache.org/jira/browse/HIVE-20233
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-20233.1.patch, HIVE-20233.2.patch,
> HIVE-20233.3.patch, HIVE-20233.4.patch, HIVE-20233.4.patch,
> HIVE-20233.5.patch, HIVE-20233.6.patch, HIVE-20233.7.patch, HIVE-20233.8.patch
>
>
> Various improvements to {{Operator.java}}
>
> * Code formatting
> * Remove superfluous logging guards
> * Add logging message parameters
> * Minor performance improvements
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20233) Code formatting improvements to Operator.java
[
https://issues.apache.org/jira/browse/HIVE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-20233:
---
Summary: Code formatting improvements to Operator.java (was: Style
improvements to Operator.java)
> Code formatting improvements to Operator.java
> -
>
> Key: HIVE-20233
> URL: https://issues.apache.org/jira/browse/HIVE-20233
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20233.1.patch, HIVE-20233.2.patch,
> HIVE-20233.3.patch, HIVE-20233.4.patch, HIVE-20233.4.patch,
> HIVE-20233.5.patch, HIVE-20233.6.patch, HIVE-20233.7.patch, HIVE-20233.8.patch
>
>
> Various improvements to {{Operator.java}}
>
> * Code formatting
> * Remove superfluous logging guards
> * Add logging message parameters
> * Minor performance improvements
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20776) Run HMS filterHooks on server-side in addition to client-side
[ https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751231#comment-16751231 ] Na Li commented on HIVE-20776: -- It is cherry-picked from master patch > Run HMS filterHooks on server-side in addition to client-side > - > > Key: HIVE-20776 > URL: https://issues.apache.org/jira/browse/HIVE-20776 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Karthik Manamcheri >Assignee: Na Li >Priority: Major > Attachments: HIVE-20776.001.patch, HIVE-20776.003.patch, > HIVE-20776.004.patch, HIVE-20776.005.patch, HIVE-20776.006.patch, > HIVE-20776.007.patch, HIVE-20776.007.patch, HIVE-20776.008.patch, > HIVE-20776.009.patch, HIVE-20776.009.patch, HIVE-20776.010.patch, > HIVE-20776.011.patch, HIVE-20776.011.patch, HIVE-20776.012.patch, > HIVE-20776.013.patch, HIVE-20776.014.patch, HIVE-20776.015.branch-3.patch, > HIVE-20776.015.patch, HIVE-20776.015_a.branch-3.patch > > > In HMS, I noticed that all the filter hooks are applied on the client side > (in HiveMetaStoreClient.java). Is there any reason why we can't apply the > filters on the server-side? > Motivation: Some newer apache projects such as Kudu use HMS for metadata > storage. Kudu is not completely Java-based and there are interaction points > where they have C++ clients. In such cases, it would be ideal to have > consistent behavior from HMS side as far as filters, etc are concerned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20776) Run HMS filterHooks on server-side in addition to client-side
[
https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751251#comment-16751251
]
Hive QA commented on HIVE-20776:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956157/HIVE-20776.015_a.branch-3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15778/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15778/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15778/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-01-24 15:47:12.279
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-15778/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-01-24 15:47:12.282
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at a7e704c HIVE-21148: Remove Use StandardCharsets Where Possible
(BELUGA BEHR via Slim Bouguerra)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at a7e704c HIVE-21148: Remove Use StandardCharsets Where Possible
(BELUGA BEHR via Slim Bouguerra)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-01-24 15:47:12.969
+ rm -rf ../yetus_PreCommit-HIVE-Build-15778
+ mkdir ../yetus_PreCommit-HIVE-Build-15778
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-15778
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-15778/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error:
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:
does not exist in index
error:
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java:
does not exist in index
error:
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java:
does not exist in index
error:
a/standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/TestFilterHooks.java:
does not exist in index
error:
a/standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/client/TestListPartitions.java:
does not exist in index
error:
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:
does not exist in index
error:
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java:
does not exist in index
error:
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java:
does not exist in index
error:
standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/TestFilterHooks.java:
does not exist in index
error:
standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/client/TestListPartitions.java:
does not exist in index
error: src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java: does
not exist in index
error: src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java:
does not exist in index
error: src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java:
does not exist in index
error: src/test/java/org/apache/hadoop/hive/metastore/TestFilterHooks.java:
does not exist in index
error:
src/test/java/org/apache/hadoop/hive/metastore/client/TestListPartitions.java:
does not exist in index
The patch does not appear to apply with p0, p1, or p2
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-15778
+ exit 1
'
{noformat}
This message is
[jira] [Commented] (HIVE-20233) Review Operator.java
[
https://issues.apache.org/jira/browse/HIVE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751277#comment-16751277
]
Jesus Camacho Rodriguez commented on HIVE-20233:
+1
> Review Operator.java
>
>
> Key: HIVE-20233
> URL: https://issues.apache.org/jira/browse/HIVE-20233
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20233.1.patch, HIVE-20233.2.patch,
> HIVE-20233.3.patch, HIVE-20233.4.patch, HIVE-20233.4.patch,
> HIVE-20233.5.patch, HIVE-20233.6.patch, HIVE-20233.7.patch, HIVE-20233.8.patch
>
>
> Various improvements to {{Operator.java}}
>
> * Code formatting
> * Remove superfluous logging guards
> * Add logging message parameters
> * Minor performance improvements
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20776) Run HMS filterHooks on server-side in addition to client-side
[ https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated HIVE-20776: - Attachment: HIVE-20776.015-branch-3.patch > Run HMS filterHooks on server-side in addition to client-side > - > > Key: HIVE-20776 > URL: https://issues.apache.org/jira/browse/HIVE-20776 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Karthik Manamcheri >Assignee: Na Li >Priority: Major > Attachments: HIVE-20776.001.patch, HIVE-20776.003.patch, > HIVE-20776.004.patch, HIVE-20776.005.patch, HIVE-20776.006.patch, > HIVE-20776.007.patch, HIVE-20776.007.patch, HIVE-20776.008.patch, > HIVE-20776.009.patch, HIVE-20776.009.patch, HIVE-20776.010.patch, > HIVE-20776.011.patch, HIVE-20776.011.patch, HIVE-20776.012.patch, > HIVE-20776.013.patch, HIVE-20776.014.patch, HIVE-20776.015-branch-3.patch, > HIVE-20776.015.branch-3.patch, HIVE-20776.015.patch, > HIVE-20776.015_a.branch-3.patch > > > In HMS, I noticed that all the filter hooks are applied on the client side > (in HiveMetaStoreClient.java). Is there any reason why we can't apply the > filters on the server-side? > Motivation: Some newer apache projects such as Kudu use HMS for metadata > storage. Kudu is not completely Java-based and there are interaction points > where they have C++ clients. In such cases, it would be ideal to have > consistent behavior from HMS side as far as filters, etc are concerned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21156) SharedWorkOptimizer may preserve filter in TS incorrectly
[ https://issues.apache.org/jira/browse/HIVE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751284#comment-16751284 ] Jesus Camacho Rodriguez commented on HIVE-21156: [~vgarg], can you review? Thanks > SharedWorkOptimizer may preserve filter in TS incorrectly > - > > Key: HIVE-21156 > URL: https://issues.apache.org/jira/browse/HIVE-21156 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21156.01.patch, HIVE-21156.patch > > > For some scan reutilizations, we may end up with a Filter expression > associated with the scan that should be removed by the optimizer. This can > lead to incorrect results. Repro case is part of the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20847) Review of NullScan Code
[
https://issues.apache.org/jira/browse/HIVE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751279#comment-16751279
]
Hive QA commented on HIVE-20847:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
32s{color} | {color:blue} ql in master has 2304 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} ql: The patch generated 0 new + 0 unchanged - 19
fixed = 0 total (was 19) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15779/dev-support/hive-personality.sh
|
| git revision | master / a7e704c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15779/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Review of NullScan Code
> ---
>
> Key: HIVE-20847
> URL: https://issues.apache.org/jira/browse/HIVE-20847
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Affects Versions: 4.0.0, 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20847.1.patch, HIVE-20847.1.patch,
> HIVE-20847.1.patch, HIVE-20847.2.patch, HIVE-20847.3.patch, HIVE-20847.4.patch
>
>
> What got me looking at this class was the verboseness of some of the logging.
> I would like to request that we DEBUG the logging since this level of detail
> means nothing to a cluster admin.
> Also... this {{contains}} call would be better applied onto a {{HashSet}}
> instead of an {{ArrayList}}.
> {code:java|title=NullScanTaskDispatcher.java}
> private void processAlias(MapWork work, Path path, ArrayList
> aliasesAffected, ArrayList aliases) {
> // the aliases that are allowed to map to a null scan.
> ArrayList allowed = new ArrayList();
> for (String alias : aliasesAffected) {
> if (aliases.contains(alias)) {
> allowed.add(alias);
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21045) Add HMS total api count stats and connection pool stats to metrics
[
https://issues.apache.org/jira/browse/HIVE-21045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Manamcheri updated HIVE-21045:
--
Attachment: HIVE-21045.6.patch
> Add HMS total api count stats and connection pool stats to metrics
> --
>
> Key: HIVE-21045
> URL: https://issues.apache.org/jira/browse/HIVE-21045
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Karthik Manamcheri
>Assignee: Karthik Manamcheri
>Priority: Minor
> Attachments: HIVE-21045.1.patch, HIVE-21045.2.patch,
> HIVE-21045.3.patch, HIVE-21045.4.patch, HIVE-21045.5.patch, HIVE-21045.6.patch
>
>
> There are two key metrics which I think we lack and which would be really
> great to help with scaling visibility in HMS.
> *Total API calls duration stats*
> We already compute and log the duration of API calls in the {{PerfLogger}}.
> We don't have any gauge or timer on what the average duration of an API call
> is for the past some bucket of time. This will give us an insight into if
> there is load on the server which is increasing the average API response time.
>
> *Connection Pool stats*
> We can use different connection pooling libraries such as bonecp or hikaricp.
> These pool managers expose statistics such as average time waiting to get a
> connection, number of connections active, etc. We should expose this as a
> metric so that we can track if the the connection pool size configured is too
> small and we are saturating!
> These metrics would help catch problems with HMS resource contention before
> they actually have jobs failing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20847) Review of NullScan Code
[
https://issues.apache.org/jira/browse/HIVE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20847:
---
Attachment: HIVE-20847.4.patch
> Review of NullScan Code
> ---
>
> Key: HIVE-20847
> URL: https://issues.apache.org/jira/browse/HIVE-20847
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Affects Versions: 4.0.0, 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20847.1.patch, HIVE-20847.1.patch,
> HIVE-20847.1.patch, HIVE-20847.2.patch, HIVE-20847.3.patch, HIVE-20847.4.patch
>
>
> What got me looking at this class was the verboseness of some of the logging.
> I would like to request that we DEBUG the logging since this level of detail
> means nothing to a cluster admin.
> Also... this {{contains}} call would be better applied onto a {{HashSet}}
> instead of an {{ArrayList}}.
> {code:java|title=NullScanTaskDispatcher.java}
> private void processAlias(MapWork work, Path path, ArrayList
> aliasesAffected, ArrayList aliases) {
> // the aliases that are allowed to map to a null scan.
> ArrayList allowed = new ArrayList();
> for (String alias : aliasesAffected) {
> if (aliases.contains(alias)) {
> allowed.add(alias);
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21143) Add rewrite rules to open/close Between operators
[
https://issues.apache.org/jira/browse/HIVE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751248#comment-16751248
]
Hive QA commented on HIVE-21143:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956152/HIVE-21143.07.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15706 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestSSL.testMetastoreWithSSL (batchId=260)
org.apache.hive.service.cli.thrift.TestThriftCLIServiceWithBinary.org.apache.hive.service.cli.thrift.TestThriftCLIServiceWithBinary
(batchId=259)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15777/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15777/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15777/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12956152 - PreCommit-HIVE-Build
> Add rewrite rules to open/close Between operators
> -
>
> Key: HIVE-21143
> URL: https://issues.apache.org/jira/browse/HIVE-21143
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21143.01.patch, HIVE-21143.02.patch,
> HIVE-21143.03.patch, HIVE-21143.03.patch, HIVE-21143.04.patch,
> HIVE-21143.05.patch, HIVE-21143.06.patch, HIVE-21143.07.patch
>
>
> During query compilation it's better to have BETWEEN statements in open form,
> as Calcite current not considering them during simplification.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20233) Code formatting improvements to Operator.java
[
https://issues.apache.org/jira/browse/HIVE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-20233:
---
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
> Code formatting improvements to Operator.java
> -
>
> Key: HIVE-20233
> URL: https://issues.apache.org/jira/browse/HIVE-20233
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-20233.1.patch, HIVE-20233.2.patch,
> HIVE-20233.3.patch, HIVE-20233.4.patch, HIVE-20233.4.patch,
> HIVE-20233.5.patch, HIVE-20233.6.patch, HIVE-20233.7.patch, HIVE-20233.8.patch
>
>
> Various improvements to {{Operator.java}}
>
> * Code formatting
> * Remove superfluous logging guards
> * Add logging message parameters
> * Minor performance improvements
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20776) Run HMS filterHooks on server-side in addition to client-side
[
https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751337#comment-16751337
]
Hive QA commented on HIVE-20776:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 9s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-15780/patches/PreCommit-HIVE-Build-15780.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15780/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Run HMS filterHooks on server-side in addition to client-side
> -
>
> Key: HIVE-20776
> URL: https://issues.apache.org/jira/browse/HIVE-20776
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Karthik Manamcheri
>Assignee: Na Li
>Priority: Major
> Attachments: HIVE-20776.001.patch, HIVE-20776.003.patch,
> HIVE-20776.004.patch, HIVE-20776.005.patch, HIVE-20776.006.patch,
> HIVE-20776.007.patch, HIVE-20776.007.patch, HIVE-20776.008.patch,
> HIVE-20776.009.patch, HIVE-20776.009.patch, HIVE-20776.010.patch,
> HIVE-20776.011.patch, HIVE-20776.011.patch, HIVE-20776.012.patch,
> HIVE-20776.013.patch, HIVE-20776.014.patch, HIVE-20776.015-branch-3.patch,
> HIVE-20776.015.branch-3.patch, HIVE-20776.015.patch,
> HIVE-20776.015_a.branch-3.patch
>
>
> In HMS, I noticed that all the filter hooks are applied on the client side
> (in HiveMetaStoreClient.java). Is there any reason why we can't apply the
> filters on the server-side?
> Motivation: Some newer apache projects such as Kudu use HMS for metadata
> storage. Kudu is not completely Java-based and there are interaction points
> where they have C++ clients. In such cases, it would be ideal to have
> consistent behavior from HMS side as far as filters, etc are concerned.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20776) Run HMS filterHooks on server-side in addition to client-side
[ https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated HIVE-20776: - Attachment: HIVE-20776.016-branch-3.patch > Run HMS filterHooks on server-side in addition to client-side > - > > Key: HIVE-20776 > URL: https://issues.apache.org/jira/browse/HIVE-20776 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Karthik Manamcheri >Assignee: Na Li >Priority: Major > Attachments: HIVE-20776.001.patch, HIVE-20776.003.patch, > HIVE-20776.004.patch, HIVE-20776.005.patch, HIVE-20776.006.patch, > HIVE-20776.007.patch, HIVE-20776.007.patch, HIVE-20776.008.patch, > HIVE-20776.009.patch, HIVE-20776.009.patch, HIVE-20776.010.patch, > HIVE-20776.011.patch, HIVE-20776.011.patch, HIVE-20776.012.patch, > HIVE-20776.013.patch, HIVE-20776.014.patch, HIVE-20776.015-branch-3.patch, > HIVE-20776.015.branch-3.patch, HIVE-20776.015.patch, > HIVE-20776.015_a.branch-3.patch, HIVE-20776.016-branch-3.patch > > > In HMS, I noticed that all the filter hooks are applied on the client side > (in HiveMetaStoreClient.java). Is there any reason why we can't apply the > filters on the server-side? > Motivation: Some newer apache projects such as Kudu use HMS for metadata > storage. Kudu is not completely Java-based and there are interaction points > where they have C++ clients. In such cases, it would be ideal to have > consistent behavior from HMS side as far as filters, etc are concerned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20847) Review of NullScan Code
[
https://issues.apache.org/jira/browse/HIVE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751321#comment-16751321
]
Hive QA commented on HIVE-20847:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956161/HIVE-20847.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15709 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15779/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15779/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15779/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12956161 - PreCommit-HIVE-Build
> Review of NullScan Code
> ---
>
> Key: HIVE-20847
> URL: https://issues.apache.org/jira/browse/HIVE-20847
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Affects Versions: 4.0.0, 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20847.1.patch, HIVE-20847.1.patch,
> HIVE-20847.1.patch, HIVE-20847.2.patch, HIVE-20847.3.patch, HIVE-20847.4.patch
>
>
> What got me looking at this class was the verboseness of some of the logging.
> I would like to request that we DEBUG the logging since this level of detail
> means nothing to a cluster admin.
> Also... this {{contains}} call would be better applied onto a {{HashSet}}
> instead of an {{ArrayList}}.
> {code:java|title=NullScanTaskDispatcher.java}
> private void processAlias(MapWork work, Path path, ArrayList
> aliasesAffected, ArrayList aliases) {
> // the aliases that are allowed to map to a null scan.
> ArrayList allowed = new ArrayList();
> for (String alias : aliasesAffected) {
> if (aliases.contains(alias)) {
> allowed.add(alias);
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21083) Remove the requirement to specify the truststore file location when TLS to the database is turned on
[
https://issues.apache.org/jira/browse/HIVE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Morio Ramdenbourg updated HIVE-21083:
-
Description:
In the current implementation,
[ObjectStore.configureSSL|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L349-L382]
throws an exception if TLS to the database is turned on
(_metastore.dbaccess.ssl.use.SSL_) but a truststore file location
(_metastore.dbaccess.ssl.truststore.path_) is not specified.
However, according to the [JSSE (Java 8)
documentation|https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#InstallationAndCustomization],
the Java truststore file location system property (_javax.net.ssl.trustStore_)
defaults to using the "_jssecacerts_, if it exists. Otherwise, _cacerts_"
files. These are the default truststores that come with the Java installation
and contain a list of well-known certificate authorities.
It was identified that one valid way of configuring TLS is by adding to these
default files. In that case, no changes to the truststore properties are
necessary. We should support this case by changing the following logic to
remove the requirement for the truststore file location config property:
{code:java}
String trustStorePath = MetastoreConf.getVar(conf,
ConfVars.DBACCESS_SSL_TRUSTSTORE_PATH).trim();
if (trustStorePath.isEmpty()) {
throw new IllegalArgumentException("SSL to the database store has been
enabled but " +
ConfVars.DBACCESS_SSL_TRUSTSTORE_PATH.toString() + " is empty. "
+ "Set this property to enable SSL.");
}
{code}
We should also loosen the requirement on the truststore password if the user
decides to use the Java defaults
was:
[ObjectStore.configureSSL|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L349-L382]
throws an exception if TLS to the database is turned on
(_metastore.dbaccess.ssl.use.SSL_) but a truststore file location
(_metastore.dbaccess.ssl.truststore.path_) is not specified.
However, according to the [JSSE (Java 8)
documentation|https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#InstallationAndCustomization],
the Java truststore file location system property (_javax.net.ssl.trustStore_)
defaults to using the "_jssecacerts_, if it exists. Otherwise, _cacerts_"
files. These are the default truststores that come with the Java installation
and contain a list of well-known certificate authorities.
One valid way of supplying certificates to trust is by appending to these
default files. In that case, no config changes (other than turning SSL on) are
necessary. We should support this case by changing the following logic to
remove the requirement for the truststore file location config property:
{code:java}
String trustStorePath = MetastoreConf.getVar(conf,
ConfVars.DBACCESS_SSL_TRUSTSTORE_PATH).trim();
if (trustStorePath.isEmpty()) {
throw new IllegalArgumentException("SSL to the database store has been
enabled but " +
ConfVars.DBACCESS_SSL_TRUSTSTORE_PATH.toString() + " is empty. "
+ "Set this property to enable SSL.");
}
{code}
Additionally, we should also not set the truststore file location and password
values if they are empty. Instead, we should just ignore them.
> Remove the requirement to specify the truststore file location when TLS to
> the database is turned on
>
>
> Key: HIVE-21083
> URL: https://issues.apache.org/jira/browse/HIVE-21083
> Project: Hive
> Issue Type: Improvement
> Components: Metastore, Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Morio Ramdenbourg
>Assignee: Morio Ramdenbourg
>Priority: Major
>
> In the current implementation,
> [ObjectStore.configureSSL|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L349-L382]
> throws an exception if TLS to the database is turned on
> (_metastore.dbaccess.ssl.use.SSL_) but a truststore file location
> (_metastore.dbaccess.ssl.truststore.path_) is not specified.
> However, according to the [JSSE (Java 8)
> documentation|https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#InstallationAndCustomization],
> the Java truststore file location system property
> (_javax.net.ssl.trustStore_) defaults to using the "_jssecacerts_, if it
> exists. Otherwise, _cacerts_" files. These are the default truststores that
> come with the Java installation and contain a list of well-known
[jira] [Updated] (HIVE-21083) Remove the requirement to specify the truststore location when TLS to the database is turned on
[
https://issues.apache.org/jira/browse/HIVE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Morio Ramdenbourg updated HIVE-21083:
-
Summary: Remove the requirement to specify the truststore location when TLS
to the database is turned on (was: Remove the requirement to specify the
truststore file location when TLS to the database is turned on)
> Remove the requirement to specify the truststore location when TLS to the
> database is turned on
> ---
>
> Key: HIVE-21083
> URL: https://issues.apache.org/jira/browse/HIVE-21083
> Project: Hive
> Issue Type: Improvement
> Components: Metastore, Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Morio Ramdenbourg
>Assignee: Morio Ramdenbourg
>Priority: Major
>
> In the current implementation,
> [ObjectStore.configureSSL|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L349-L382]
> throws an exception if TLS to the database is turned on
> (_metastore.dbaccess.ssl.use.SSL_) but a truststore file location
> (_metastore.dbaccess.ssl.truststore.path_) is not specified.
> However, according to the [JSSE (Java 8)
> documentation|https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#InstallationAndCustomization],
> the Java truststore file location system property
> (_javax.net.ssl.trustStore_) defaults to using the "_jssecacerts_, if it
> exists. Otherwise, _cacerts_" files. These are the default truststores that
> come with the Java installation and contain a list of well-known certificate
> authorities.
> It was identified that one valid way of configuring TLS is by adding to these
> default files. In that case, no changes to the truststore properties are
> necessary. We should support this case by changing the following logic to
> remove the requirement for the truststore file location config property:
> {code:java}
> String trustStorePath = MetastoreConf.getVar(conf,
> ConfVars.DBACCESS_SSL_TRUSTSTORE_PATH).trim();
> if (trustStorePath.isEmpty()) {
> throw new IllegalArgumentException("SSL to the database store has
> been enabled but " +
> ConfVars.DBACCESS_SSL_TRUSTSTORE_PATH.toString() + " is empty. "
> + "Set this property to enable SSL.");
> }
> {code}
> We should also loosen the requirement on the truststore password if the user
> decides to use the Java defaults
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21009) LDAP - Specify binddn for ldap-search
[
https://issues.apache.org/jira/browse/HIVE-21009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David McGinnis reassigned HIVE-21009:
-
Assignee: David McGinnis
> LDAP - Specify binddn for ldap-search
> -
>
> Key: HIVE-21009
> URL: https://issues.apache.org/jira/browse/HIVE-21009
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 2.1.0, 2.1.1, 2.2.0, 2.3.0, 2.3.1, 2.3.2
>Reporter: Thomas Uhren
>Assignee: David McGinnis
>Priority: Major
>
> When user accounts cannot do an LDAP search, there is currently no way of
> specifying a custom binddn to use for the ldap-search.
> So I'm missing something like that:
> {code}
> hive.server2.authentication.ldap.bindn=cn=ldapuser,ou=user,dc=example
> hive.server2.authentication.ldap.bindnpw=password
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21009) LDAP - Specify binddn for ldap-search
[
https://issues.apache.org/jira/browse/HIVE-21009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751407#comment-16751407
]
David McGinnis commented on HIVE-21009:
---
[~tuhren]: I had to make a patch recently for a client to do this exact thing
in the LDAPAuthenticationProviderImpl class, so I'm open to spruce it up and
contribute it back to the project. Will assign this to myself and clean up the
patch to get it ready for contribution.
> LDAP - Specify binddn for ldap-search
> -
>
> Key: HIVE-21009
> URL: https://issues.apache.org/jira/browse/HIVE-21009
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 2.1.0, 2.1.1, 2.2.0, 2.3.0, 2.3.1, 2.3.2
>Reporter: Thomas Uhren
>Priority: Major
>
> When user accounts cannot do an LDAP search, there is currently no way of
> specifying a custom binddn to use for the ldap-search.
> So I'm missing something like that:
> {code}
> hive.server2.authentication.ldap.bindn=cn=ldapuser,ou=user,dc=example
> hive.server2.authentication.ldap.bindnpw=password
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HIVE-21083) Remove the requirement to specify the truststore location when TLS to the database is turned on
[
https://issues.apache.org/jira/browse/HIVE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-21083 started by Morio Ramdenbourg.
> Remove the requirement to specify the truststore location when TLS to the
> database is turned on
> ---
>
> Key: HIVE-21083
> URL: https://issues.apache.org/jira/browse/HIVE-21083
> Project: Hive
> Issue Type: Improvement
> Components: Metastore, Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Morio Ramdenbourg
>Assignee: Morio Ramdenbourg
>Priority: Major
>
> In the current implementation,
> [ObjectStore.configureSSL|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L349-L382]
> throws an exception if TLS to the database is turned on
> (_metastore.dbaccess.ssl.use.SSL_) but a truststore file location
> (_metastore.dbaccess.ssl.truststore.path_) is not specified.
> However, according to the [JSSE (Java 8)
> documentation|https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#InstallationAndCustomization],
> the Java truststore file location system property
> (_javax.net.ssl.trustStore_) defaults to using the "_jssecacerts_, if it
> exists. Otherwise, _cacerts_" files. These are the default truststores that
> come with the Java installation and contain a list of well-known certificate
> authorities.
> It was identified that one valid way of configuring TLS is by adding to these
> default files. In that case, no changes to the truststore properties are
> necessary. We should support this case by changing the following logic to
> remove the requirement for the truststore file location config property:
> {code:java}
> String trustStorePath = MetastoreConf.getVar(conf,
> ConfVars.DBACCESS_SSL_TRUSTSTORE_PATH).trim();
> if (trustStorePath.isEmpty()) {
> throw new IllegalArgumentException("SSL to the database store has
> been enabled but " +
> ConfVars.DBACCESS_SSL_TRUSTSTORE_PATH.toString() + " is empty. "
> + "Set this property to enable SSL.");
> }
> {code}
> We should also loosen the requirement on the truststore password if the user
> decides to use the Java defaults
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20776) Run HMS filterHooks on server-side in addition to client-side
[
https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751410#comment-16751410
]
Hive QA commented on HIVE-20776:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956166/HIVE-20776.015-branch-3.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 130 failed/errored test(s), 14480 tests
executed
*Failed tests:*
{noformat}
TestAddPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestAddPartitionsFromPartSpec - did not produce a TEST-*.xml file (likely timed
out) (batchId=230)
TestAdminUser - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
TestAggregateStatsCache - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
TestAlterPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestAppendPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=274)
TestCachedStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestCatalogCaching - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestCatalogNonDefaultClient - did not produce a TEST-*.xml file (likely timed
out) (batchId=228)
TestCatalogNonDefaultSvr - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
TestCatalogOldClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestCatalogs - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestCheckConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=238)
TestDatabases - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDeadline - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestDefaultConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDropPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=274)
TestEmbeddedHiveMetaStore - did not produce a TEST-*.xml file (likely timed
out) (batchId=231)
TestExchangePartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestFMSketchSerialization - did not produce a TEST-*.xml file (likely timed
out) (batchId=239)
TestFilterHooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestForeignKey - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestFunctions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestGetPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestGetPartitionsUsingProjectionAndFilterSpecs - did not produce a TEST-*.xml
file (likely timed out) (batchId=230)
TestGetTableMeta - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHLLNoBias - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHLLSerialization - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHdfsUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
TestHiveAlterHandler - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHiveMetaStoreGetMetaConf - did not produce a TEST-*.xml file (likely timed
out) (batchId=238)
TestHiveMetaStorePartitionSpecs - did not produce a TEST-*.xml file (likely
timed out) (batchId=230)
TestHiveMetaStoreSchemaMethods - did not produce a TEST-*.xml file (likely
timed out) (batchId=235)
TestHiveMetaStoreTimeout - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHiveMetaStoreTxns - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHiveMetaStoreWithEnvironmentContext - did not produce a TEST-*.xml file
(likely timed out) (batchId=233)
TestHiveMetaToolCommandLine - did not produce a TEST-*.xml file (likely timed
out) (batchId=235)
TestHiveMetastoreCli - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHyperLogLog - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHyperLogLogDense - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHyperLogLogMerge - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHyperLogLogSparse - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestJSONMessageDeserializer - did not produce a TEST-*.xml file (likely timed
out) (batchId=235)
TestListPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestLockRequestBuilder - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestMarkPartition - did not produce
[jira] [Commented] (HIVE-21156) SharedWorkOptimizer may preserve filter in TS incorrectly
[
https://issues.apache.org/jira/browse/HIVE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751430#comment-16751430
]
Hive QA commented on HIVE-21156:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
59s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
31s{color} | {color:blue} ql in master has 2304 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15781/dev-support/hive-personality.sh
|
| git revision | master / 1327d47 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15781/yetus/whitespace-eol.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15781/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> SharedWorkOptimizer may preserve filter in TS incorrectly
> -
>
> Key: HIVE-21156
> URL: https://issues.apache.org/jira/browse/HIVE-21156
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21156.01.patch, HIVE-21156.patch
>
>
> For some scan reutilizations, we may end up with a Filter expression
> associated with the scan that should be removed by the optimizer. This can
> lead to incorrect results. Repro case is part of the patch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21156) SharedWorkOptimizer may preserve filter in TS incorrectly
[ https://issues.apache.org/jira/browse/HIVE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751432#comment-16751432 ] Vineet Garg commented on HIVE-21156: LGTM +1 > SharedWorkOptimizer may preserve filter in TS incorrectly > - > > Key: HIVE-21156 > URL: https://issues.apache.org/jira/browse/HIVE-21156 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21156.01.patch, HIVE-21156.patch > > > For some scan reutilizations, we may end up with a Filter expression > associated with the scan that should be removed by the optimizer. This can > lead to incorrect results. Repro case is part of the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21132) Semi join edge is not being removed despite max bloomfilter entries set to 1
[
https://issues.apache.org/jira/browse/HIVE-21132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-21132:
---
Status: Open (was: Patch Available)
> Semi join edge is not being removed despite max bloomfilter entries set to 1
>
>
> Key: HIVE-21132
> URL: https://issues.apache.org/jira/browse/HIVE-21132
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-21132.1.patch, HIVE-21132.2.patch,
> HIVE-21132.3.patch
>
>
> * Reproducer
> {code:sql}
> --! qt:dataset:lineitem
> --! qt:dataset:part
> --! qt:dataset:src
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> --set hive.compute.query.using.stats=false;
> set hive.mapred.mode=nonstrict;
> set hive.explain.user=false;
> set hive.optimize.ppd=true;
> set hive.ppd.remove.duplicatefilters=true;
> set hive.tez.dynamic.partition.pruning=true;
> set hive.tez.dynamic.semijoin.reduction=true;
> set hive.optimize.metadataonly=false;
> set hive.optimize.index.filter=true;
> set hive.stats.autogather=true;
> set hive.tez.bigtable.minsize.semijoin.reduction=1;
> set hive.tez.min.bloom.filter.entries=1;
> set hive.stats.fetch.column.stats=true;
> set hive.tez.bloom.filter.factor=1.0f;
> set hive.auto.convert.join=false;
> set hive.optimize.shared.work=false;
> create database tpch_test;
> use tpch_test;
> CREATE TABLE `customer`(
> `c_custkey` bigint,
> `c_name` string,
> `c_address` string,
> `c_nationkey` bigint,
> `c_phone` string,
> `c_acctbal` double,
> `c_mktsegment` string,
> `c_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543026723');
> CREATE TABLE `lineitem`(
> `l_orderkey` bigint,
> `l_partkey` bigint,
> `l_suppkey` bigint,
> `l_linenumber` int,
> `l_quantity` double,
> `l_extendedprice` double,
> `l_discount` double,
> `l_tax` double,
> `l_returnflag` string,
> `l_linestatus` string,
> `l_shipdate` string,
> `l_commitdate` string,
> `l_receiptdate` string,
> `l_shipinstruct` string,
> `l_shipmode` string,
> `l_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543027179');
> CREATE TABLE `orders`(
> `o_orderkey` bigint,
> `o_custkey` bigint,
> `o_orderstatus` string,
> `o_totalprice` double,
> `o_orderdate` string,
> `o_orderpriority` string,
> `o_clerk` string,
> `o_shippriority` int,
> `o_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543026824');
> alter table customer update statistics
> set('numRows'='15000','rawDataSize'='8633707142');
> alter table lineitem update statistics
> set('numRows'='589709','rawDataSize'='184245066955');
> alter table orders update statistics
> set('numRows'='15','rawDataSize'='46741318253');
> create view q18_tmp_cached as
> select l_orderkey, sum(l_quantity) as t_sum_quantity
> from lineitem
> where l_orderkey is not null
> group by l_orderkey;
> -- Set bloom filter size to huge number so we get any possible semijoin
> reductions
> set hive.tez.min.bloom.filter.entries=0;
> set hive.tez.max.bloom.filter.entries=1;
> create table q18_large_volume_customer_cached stored as orc tblproperties
> ('transactional'='true', 'transactional_properties'='default') as
> select c_name, c_custkey, o_orderkey, o_orderdate, o_totalprice,
> sum(l_quantity)
> from customer, orders, q18_tmp_cached t, lineitem l
> where
> c_custkey = o_custkey and o_orderkey = t.l_orderkey
> and o_orderkey is not null and t.t_sum_quantity > 300
> and o_orderkey = l.l_orderkey and l.l_orderkey is not null
> group by c_name, c_custkey, o_orderkey, o_orderdate, o_totalprice
> order by o_totalprice
[jira] [Updated] (HIVE-21132) Semi join edge is not being removed despite max bloomfilter entries set to 1
[
https://issues.apache.org/jira/browse/HIVE-21132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-21132:
---
Status: Patch Available (was: Open)
> Semi join edge is not being removed despite max bloomfilter entries set to 1
>
>
> Key: HIVE-21132
> URL: https://issues.apache.org/jira/browse/HIVE-21132
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-21132.1.patch, HIVE-21132.2.patch,
> HIVE-21132.3.patch
>
>
> * Reproducer
> {code:sql}
> --! qt:dataset:lineitem
> --! qt:dataset:part
> --! qt:dataset:src
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> --set hive.compute.query.using.stats=false;
> set hive.mapred.mode=nonstrict;
> set hive.explain.user=false;
> set hive.optimize.ppd=true;
> set hive.ppd.remove.duplicatefilters=true;
> set hive.tez.dynamic.partition.pruning=true;
> set hive.tez.dynamic.semijoin.reduction=true;
> set hive.optimize.metadataonly=false;
> set hive.optimize.index.filter=true;
> set hive.stats.autogather=true;
> set hive.tez.bigtable.minsize.semijoin.reduction=1;
> set hive.tez.min.bloom.filter.entries=1;
> set hive.stats.fetch.column.stats=true;
> set hive.tez.bloom.filter.factor=1.0f;
> set hive.auto.convert.join=false;
> set hive.optimize.shared.work=false;
> create database tpch_test;
> use tpch_test;
> CREATE TABLE `customer`(
> `c_custkey` bigint,
> `c_name` string,
> `c_address` string,
> `c_nationkey` bigint,
> `c_phone` string,
> `c_acctbal` double,
> `c_mktsegment` string,
> `c_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543026723');
> CREATE TABLE `lineitem`(
> `l_orderkey` bigint,
> `l_partkey` bigint,
> `l_suppkey` bigint,
> `l_linenumber` int,
> `l_quantity` double,
> `l_extendedprice` double,
> `l_discount` double,
> `l_tax` double,
> `l_returnflag` string,
> `l_linestatus` string,
> `l_shipdate` string,
> `l_commitdate` string,
> `l_receiptdate` string,
> `l_shipinstruct` string,
> `l_shipmode` string,
> `l_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543027179');
> CREATE TABLE `orders`(
> `o_orderkey` bigint,
> `o_custkey` bigint,
> `o_orderstatus` string,
> `o_totalprice` double,
> `o_orderdate` string,
> `o_orderpriority` string,
> `o_clerk` string,
> `o_shippriority` int,
> `o_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543026824');
> alter table customer update statistics
> set('numRows'='15000','rawDataSize'='8633707142');
> alter table lineitem update statistics
> set('numRows'='589709','rawDataSize'='184245066955');
> alter table orders update statistics
> set('numRows'='15','rawDataSize'='46741318253');
> create view q18_tmp_cached as
> select l_orderkey, sum(l_quantity) as t_sum_quantity
> from lineitem
> where l_orderkey is not null
> group by l_orderkey;
> -- Set bloom filter size to huge number so we get any possible semijoin
> reductions
> set hive.tez.min.bloom.filter.entries=0;
> set hive.tez.max.bloom.filter.entries=1;
> create table q18_large_volume_customer_cached stored as orc tblproperties
> ('transactional'='true', 'transactional_properties'='default') as
> select c_name, c_custkey, o_orderkey, o_orderdate, o_totalprice,
> sum(l_quantity)
> from customer, orders, q18_tmp_cached t, lineitem l
> where
> c_custkey = o_custkey and o_orderkey = t.l_orderkey
> and o_orderkey is not null and t.t_sum_quantity > 300
> and o_orderkey = l.l_orderkey and l.l_orderkey is not null
> group by c_name, c_custkey, o_orderkey, o_orderdate, o_totalprice
> order by o_totalprice
[jira] [Updated] (HIVE-21132) Semi join edge is not being removed despite max bloomfilter entries set to 1
[
https://issues.apache.org/jira/browse/HIVE-21132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-21132:
---
Attachment: HIVE-21132.3.patch
> Semi join edge is not being removed despite max bloomfilter entries set to 1
>
>
> Key: HIVE-21132
> URL: https://issues.apache.org/jira/browse/HIVE-21132
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-21132.1.patch, HIVE-21132.2.patch,
> HIVE-21132.3.patch
>
>
> * Reproducer
> {code:sql}
> --! qt:dataset:lineitem
> --! qt:dataset:part
> --! qt:dataset:src
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> --set hive.compute.query.using.stats=false;
> set hive.mapred.mode=nonstrict;
> set hive.explain.user=false;
> set hive.optimize.ppd=true;
> set hive.ppd.remove.duplicatefilters=true;
> set hive.tez.dynamic.partition.pruning=true;
> set hive.tez.dynamic.semijoin.reduction=true;
> set hive.optimize.metadataonly=false;
> set hive.optimize.index.filter=true;
> set hive.stats.autogather=true;
> set hive.tez.bigtable.minsize.semijoin.reduction=1;
> set hive.tez.min.bloom.filter.entries=1;
> set hive.stats.fetch.column.stats=true;
> set hive.tez.bloom.filter.factor=1.0f;
> set hive.auto.convert.join=false;
> set hive.optimize.shared.work=false;
> create database tpch_test;
> use tpch_test;
> CREATE TABLE `customer`(
> `c_custkey` bigint,
> `c_name` string,
> `c_address` string,
> `c_nationkey` bigint,
> `c_phone` string,
> `c_acctbal` double,
> `c_mktsegment` string,
> `c_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543026723');
> CREATE TABLE `lineitem`(
> `l_orderkey` bigint,
> `l_partkey` bigint,
> `l_suppkey` bigint,
> `l_linenumber` int,
> `l_quantity` double,
> `l_extendedprice` double,
> `l_discount` double,
> `l_tax` double,
> `l_returnflag` string,
> `l_linestatus` string,
> `l_shipdate` string,
> `l_commitdate` string,
> `l_receiptdate` string,
> `l_shipinstruct` string,
> `l_shipmode` string,
> `l_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543027179');
> CREATE TABLE `orders`(
> `o_orderkey` bigint,
> `o_custkey` bigint,
> `o_orderstatus` string,
> `o_totalprice` double,
> `o_orderdate` string,
> `o_orderpriority` string,
> `o_clerk` string,
> `o_shippriority` int,
> `o_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543026824');
> alter table customer update statistics
> set('numRows'='15000','rawDataSize'='8633707142');
> alter table lineitem update statistics
> set('numRows'='589709','rawDataSize'='184245066955');
> alter table orders update statistics
> set('numRows'='15','rawDataSize'='46741318253');
> create view q18_tmp_cached as
> select l_orderkey, sum(l_quantity) as t_sum_quantity
> from lineitem
> where l_orderkey is not null
> group by l_orderkey;
> -- Set bloom filter size to huge number so we get any possible semijoin
> reductions
> set hive.tez.min.bloom.filter.entries=0;
> set hive.tez.max.bloom.filter.entries=1;
> create table q18_large_volume_customer_cached stored as orc tblproperties
> ('transactional'='true', 'transactional_properties'='default') as
> select c_name, c_custkey, o_orderkey, o_orderdate, o_totalprice,
> sum(l_quantity)
> from customer, orders, q18_tmp_cached t, lineitem l
> where
> c_custkey = o_custkey and o_orderkey = t.l_orderkey
> and o_orderkey is not null and t.t_sum_quantity > 300
> and o_orderkey = l.l_orderkey and l.l_orderkey is not null
> group by c_name, c_custkey, o_orderkey, o_orderdate, o_totalprice
> order by o_totalprice desc,
