[jira] [Commented] (HIVE-16340) Allow Kerberos + SSL connections to HMS
[ https://issues.apache.org/jira/browse/HIVE-16340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963864#comment-15963864 ] Lefty Leverenz commented on HIVE-16340: --- Should this be documented in the wiki? > Allow Kerberos + SSL connections to HMS > --- > > Key: HIVE-16340 > URL: https://issues.apache.org/jira/browse/HIVE-16340 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Sahil Takiar >Assignee: Sahil Takiar > Fix For: 3.0.0 > > Attachments: HIVE-16340.1.patch, HIVE-16340.2.patch, > HIVE-16340.3.patch > > > It should be possible to connect to HMS with Kerberos authentication and SSL > enabled, at the same time. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16368) Unexpected java.lang.ArrayIndexOutOfBoundsException from query with LaterView Operation for hive on MR.
[
https://issues.apache.org/jira/browse/HIVE-16368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963862#comment-15963862
]
zhihai xu commented on HIVE-16368:
--
Thanks [~ashutoshc] for reviewing and committing the patch!
> Unexpected java.lang.ArrayIndexOutOfBoundsException from query with LaterView
> Operation for hive on MR.
> ---
>
> Key: HIVE-16368
> URL: https://issues.apache.org/jira/browse/HIVE-16368
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: zhihai xu
>Assignee: zhihai xu
> Fix For: 3.0.0
>
> Attachments: HIVE-16368.000.patch, HIVE-16368.001.patch,
> HIVE-16368.002.patch
>
>
> Unexpected java.lang.ArrayIndexOutOfBoundsException from query. It happened
> in LaterView Operation. It happened for hive-on-mr. The reason is because the
> column prune change the column order in LaterView operation, for back-back
> reducesink operators using MR engine, FileSinkOperator and TableScanOperator
> are added before the second ReduceSink operator, The serialization column
> order used by FileSinkOperator in LazyBinarySerDe of previous reducer is
> different from deserialization column order from table desc used by
> MapOperator/TableScanOperator in LazyBinarySerDe of current failed mapper.
> The serialization is decided by the outputObjInspector from
> LateralViewJoinOperator,
> {code}
> ArrayList fieldNames = conf.getOutputInternalColNames();
> outputObjInspector = ObjectInspectorFactory
> .getStandardStructObjectInspector(fieldNames, ois);
> {code}
> So the column order for serialization is decided by getOutputInternalColNames
> in LateralViewJoinOperator.
> The deserialization is decided by TableScanOperator which is created at
> GenMapRedUtils.splitTasks.
> {code}
> TableDesc tt_desc = PlanUtils.getIntermediateFileTableDesc(PlanUtils
> .getFieldSchemasFromRowSchema(parent.getSchema(), "temporarycol"));
> // Create the temporary file, its corresponding FileSinkOperaotr, and
> // its corresponding TableScanOperator.
> TableScanOperator tableScanOp =
> createTemporaryFile(parent, op, taskTmpDir, tt_desc, parseCtx);
> {code}
> The column order for deserialization is decided by rowSchema of
> LateralViewJoinOperator.
> But ColumnPrunerLateralViewJoinProc changed the order of
> outputInternalColNames but still keep the original order of rowSchema,
> Which cause the mismatch between serialization and deserialization for two
> back-to-back MR jobs.
> Similar issue for ColumnPrunerLateralViewForwardProc which change the column
> order of its child selector colList but not rowSchema.
> The exception is
> {code}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 875968094
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.byteArrayToLong(LazyBinaryUtils.java:78)
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryDouble.init(LazyBinaryDouble.java:43)
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.uncheckedGetField(LazyBinaryStruct.java:264)
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.getField(LazyBinaryStruct.java:201)
> at
> org.apache.hadoop.hive.serde2.lazybinary.objectinspector.LazyBinaryStructObjectInspector.getStructFieldData(LazyBinaryStructObjectInspector.java:64)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator._evaluate(ExprNodeColumnEvaluator.java:94)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65)
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.makeValueWritable(ReduceSinkOperator.java:554)
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:381)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-11418) Dropping a database in an encryption zone with CASCADE and trash enabled fails
[
https://issues.apache.org/jira/browse/HIVE-11418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-11418:
Attachment: HIVE-11418.2.patch
> Dropping a database in an encryption zone with CASCADE and trash enabled fails
> --
>
> Key: HIVE-11418
> URL: https://issues.apache.org/jira/browse/HIVE-11418
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 1.2.0
>Reporter: Sergio Peña
>Assignee: Sahil Takiar
> Attachments: HIVE-11418.1.patch, HIVE-11418.2.patch
>
>
> Here's the query that fails:
> {noformat}
> hive> CREATE DATABASE db;
> hive> USE db;
> hive> CREATE TABLE a(id int);
> hive> SET fs.trash.interval=1;
> hive> DROP DATABASE db CASCADE;
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Unable to drop
> db.a because it is in an encryption zone and trash
> is enabled. Use PURGE option to skip trash.)
> {noformat}
> DROP DATABASE does not support PURGE, so we have to remove the tables one by
> one, and then drop the database.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15630) add operation handle before operation.run instead of after operation.run
[
https://issues.apache.org/jira/browse/HIVE-15630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963856#comment-15963856
]
Lefty Leverenz commented on HIVE-15630:
---
Thanks for the version fix, Zhihai Xu.
> add operation handle before operation.run instead of after operation.run
>
>
> Key: HIVE-15630
> URL: https://issues.apache.org/jira/browse/HIVE-15630
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.2.0
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HIVE-15630.000.patch, HIVE-15630.001.patch
>
>
> Add operation handle before operation.run instead of after operation.run. So
> when session is closed, all the running operations from {{operation.run}} can
> also be closed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-15630) add operation handle before operation.run instead of after operation.run
[
https://issues.apache.org/jira/browse/HIVE-15630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhihai xu updated HIVE-15630:
-
Fix Version/s: (was: 2.2.0)
3.0.0
> add operation handle before operation.run instead of after operation.run
>
>
> Key: HIVE-15630
> URL: https://issues.apache.org/jira/browse/HIVE-15630
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.2.0
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HIVE-15630.000.patch, HIVE-15630.001.patch
>
>
> Add operation handle before operation.run instead of after operation.run. So
> when session is closed, all the running operations from {{operation.run}} can
> also be closed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16383) Switch to HikariCP as default connection pooling
[ https://issues.apache.org/jira/browse/HIVE-16383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963849#comment-15963849 ] Lefty Leverenz commented on HIVE-16383: --- Doc note: This changes the default value and list of possible values for *datanucleus.connectionPoolingType* and adds *datanucleus.connectionPool.maxPoolSize* to HiveConf.java, so the wiki will need to be updated for release 3.0.0. * [Configuration Properties -- MetaStore | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-MetaStore] * [Configuration Properties -- MetaStore -- datanucleus.connectionPoolingType | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-datanucleus.connectionPoolingType] Added a TODOC3.0 label. Formatting note: The description of *datanucleus.connectionPool.maxPoolSize* is 6 lines long but will appear on a single line in the generated template file because \n wasn't used for line breaks. Can this be fixed? * [AdminManual Configuration -- hive-site.xml and hive-default.xml.template | https://cwiki.apache.org/confluence/display/Hive/AdminManual+Configuration#AdminManualConfiguration-hive-site.xmlandhive-default.xml.template] > Switch to HikariCP as default connection pooling > > > Key: HIVE-16383 > URL: https://issues.apache.org/jira/browse/HIVE-16383 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Labels: TODOC3.0 > Fix For: 3.0.0 > > Attachments: HIVE-16383.1.patch, HIVE-16383.2.patch > > > Since 3.0 is planned to move to JDK8, we can now switch to HikariCP as > default connection pooling for DN because of its improved performance over > others. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16383) Switch to HikariCP as default connection pooling
[ https://issues.apache.org/jira/browse/HIVE-16383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-16383: -- Labels: TODOC3.0 (was: ) > Switch to HikariCP as default connection pooling > > > Key: HIVE-16383 > URL: https://issues.apache.org/jira/browse/HIVE-16383 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Labels: TODOC3.0 > Fix For: 3.0.0 > > Attachments: HIVE-16383.1.patch, HIVE-16383.2.patch > > > Since 3.0 is planned to move to JDK8, we can now switch to HikariCP as > default connection pooling for DN because of its improved performance over > others. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-12614) RESET command does not close spark session
[ https://issues.apache.org/jira/browse/HIVE-12614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963840#comment-15963840 ] Sahil Takiar commented on HIVE-12614: - Thanks [~nemon]! Looks like current tests pass, will work on adding a new test. > RESET command does not close spark session > -- > > Key: HIVE-12614 > URL: https://issues.apache.org/jira/browse/HIVE-12614 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 1.3.0, 2.1.0 >Reporter: Nemon Lou >Assignee: Sahil Takiar >Priority: Minor > Attachments: HIVE-12614.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16385) StatsNoJobTask could exit early before all partitions have been processed
[
https://issues.apache.org/jira/browse/HIVE-16385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963838#comment-15963838
]
Lefty Leverenz commented on HIVE-16385:
---
[~csun], do you intend to commit this to branch-2.3, branch-2.2, branch-2.1,
and branch-2.0? If not, please change the fix version to 3.0.0 (master).
Thanks.
> StatsNoJobTask could exit early before all partitions have been processed
> -
>
> Key: HIVE-16385
> URL: https://issues.apache.org/jira/browse/HIVE-16385
> Project: Hive
> Issue Type: Bug
> Components: Statistics
>Reporter: Chao Sun
>Assignee: Chao Sun
> Fix For: 2.0.2, 2.1.2, 2.2.0, 2.3.0
>
> Attachments: HIVE-16385.1.patch
>
>
> For a partitioned table, the class {{StatsNoJobTask}} is supposed to launch
> threads for all partitions and compute their stats. However, it could exit
> early after at most 100 seconds:
> {code}
> private void shutdownAndAwaitTermination(ExecutorService threadPool) {
> // Disable new tasks from being submitted
> threadPool.shutdown();
> try {
> // Wait a while for existing tasks to terminate
> if (!threadPool.awaitTermination(100, TimeUnit.SECONDS)) {
> // Cancel currently executing tasks
> threadPool.shutdownNow();
> // Wait a while for tasks to respond to being cancelled
> if (!threadPool.awaitTermination(100, TimeUnit.SECONDS)) {
> LOG.debug("Stats collection thread pool did not terminate");
> }
> }
> } catch (InterruptedException ie) {
> // Cancel again if current thread also interrupted
> threadPool.shutdownNow();
> // Preserve interrupt status
> Thread.currentThread().interrupt();
> }
> }
> {code}
> The {{shutdown}} call does not wait for all submitted tasks to complete, and
> the {{awaitTermination}} call waits at most 100 seconds.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-10865) Beeline needs to support DELIMITER command
[ https://issues.apache.org/jira/browse/HIVE-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963834#comment-15963834 ] Sahil Takiar commented on HIVE-10865: - [~vihangk1] could you take a look? Test failures are flaky. > Beeline needs to support DELIMITER command > -- > > Key: HIVE-10865 > URL: https://issues.apache.org/jira/browse/HIVE-10865 > Project: Hive > Issue Type: Bug > Components: Beeline >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Sahil Takiar > Attachments: HIVE-10865.1.patch, HIVE-10865.2.patch, > HIVE-10865.3.patch > > > MySQL Client provides a DELIMITER command to set statement delimiter. > Beeline needs to support a similar command to allow commands having > semi-colon as non-statement delimiter (as with MySQL stored procedures). This > is a follow-up jira for HIVE-10659 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16267) Enable bootstrap function metadata to be loaded in repl load
[ https://issues.apache.org/jira/browse/HIVE-16267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963827#comment-15963827 ] anishek commented on HIVE-16267: [~thejas] https://github.com/apache/hive/pull/168 > Enable bootstrap function metadata to be loaded in repl load > > > Key: HIVE-16267 > URL: https://issues.apache.org/jira/browse/HIVE-16267 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2 >Reporter: anishek >Assignee: anishek > Fix For: 3.0.0 > > Attachments: HIVE-16267.1.patch, HIVE-16267.2.patch > > Original Estimate: 72h > Remaining Estimate: 72h > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16209) Vectorization: Add support for complex types to VectorExtractRow and VectorAssignRow
[
https://issues.apache.org/jira/browse/HIVE-16209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963824#comment-15963824
]
Hive QA commented on HIVE-16209:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12862776/HIVE-16209.7.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4637/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4637/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4637/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-04-11 04:39:02.680
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-4637/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-04-11 04:39:02.683
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at bf5c9e7 HIVE-16380 : removing global test dependency of
jsonassert (Anishek Agarwal via Thejas Nair)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at bf5c9e7 HIVE-16380 : removing global test dependency of
jsonassert (Anishek Agarwal via Thejas Nair)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-04-11 04:39:04.624
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: patch -p0
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorAssignRow.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorExtractRow.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatchCtx.java
patching file
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorRowObject.java
patching file
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/VectorRandomRowSource.java
patching file
serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java
patching file
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ArrayListObjectInspector.java
patching file
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ArrayMapObjectInspector.java
patching file
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ArrayStructObjectInspector.java
patching file
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ArrayUnionObjectInspector.java
patching file
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java
patching file
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/SettableUnionObjectInspector.java
patching file
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/StandardUnionObjectInspector.java
patching file
serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils.java
patching file
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java
patching file
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/metastore/target/generated-sources/antlr3/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/metastore/src/java/org/apache/hadoop/hive/metastore/parser/Filter.g
org/apache/hadoop/hive/metastore/parser/Filter.g
DataNucleus Enhancer (version 4.1.17) for API "JDO"
[jira] [Commented] (HIVE-12614) RESET command does not close spark session
[
https://issues.apache.org/jira/browse/HIVE-12614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963819#comment-15963819
]
Hive QA commented on HIVE-12614:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12862770/HIVE-12614.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 10570 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData (batchId=221)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4636/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4636/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4636/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12862770 - PreCommit-HIVE-Build
> RESET command does not close spark session
> --
>
> Key: HIVE-12614
> URL: https://issues.apache.org/jira/browse/HIVE-12614
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 1.3.0, 2.1.0
>Reporter: Nemon Lou
>Assignee: Sahil Takiar
>Priority: Minor
> Attachments: HIVE-12614.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-16209) Vectorization: Add support for complex types to VectorExtractRow and VectorAssignRow
[ https://issues.apache.org/jira/browse/HIVE-16209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teddy Choi updated HIVE-16209: -- Attachment: HIVE-16209.7.patch HIVE-16209.7.patch added detailed comments and renamed variables for readability. For example, flatIndex was replaced with flattenedId, because there already is OrcFileMetadata.getFlattenedColumnCount() method. > Vectorization: Add support for complex types to VectorExtractRow and > VectorAssignRow > > > Key: HIVE-16209 > URL: https://issues.apache.org/jira/browse/HIVE-16209 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Matt McCline >Assignee: Teddy Choi >Priority: Critical > Attachments: HIVE-16209.1.patch, HIVE-16209.3.patch, > HIVE-16209.4.patch, HIVE-16209.5.patch, HIVE-16209.6.patch, HIVE-16209.7.patch > > > Supports complex types in non-native VectorReduceSink, row mode Text > Vectorization, and some cases of Vectorized Schema Evolution. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16394) HoS does not support queue name change in middle of session
[ https://issues.apache.org/jira/browse/HIVE-16394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963810#comment-15963810 ] Lefty Leverenz commented on HIVE-16394: --- Should this change be documented in the wiki? * [Hive on Spark: Getting Started | https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started] > HoS does not support queue name change in middle of session > --- > > Key: HIVE-16394 > URL: https://issues.apache.org/jira/browse/HIVE-16394 > Project: Hive > Issue Type: Bug >Reporter: Chaoyu Tang >Assignee: Chaoyu Tang > Fix For: 3.0.0 > > Attachments: HIVE-16394.patch > > > The mapreduce.job.queuename only effects when HoS executes its query first > time. After that, changing mapreduce.job.queuename won't change the query > yarn scheduler queue name. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15630) add operation handle before operation.run instead of after operation.run
[
https://issues.apache.org/jira/browse/HIVE-15630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963806#comment-15963806
]
Lefty Leverenz commented on HIVE-15630:
---
[~jxiang], pushing to master means this will be in release 3.0.0, not 2.2.0.
If you want the patch to be in release 2.2.0, you should commit it to
branch-2.2. And for release 2.3.0, commit it to branch-2.3.
Please correct the fix version. Thank you.
> add operation handle before operation.run instead of after operation.run
>
>
> Key: HIVE-15630
> URL: https://issues.apache.org/jira/browse/HIVE-15630
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.2.0
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Minor
> Fix For: 2.2.0
>
> Attachments: HIVE-15630.000.patch, HIVE-15630.001.patch
>
>
> Add operation handle before operation.run instead of after operation.run. So
> when session is closed, all the running operations from {{operation.run}} can
> also be closed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15982) Support the width_bucket function
[ https://issues.apache.org/jira/browse/HIVE-15982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963804#comment-15963804 ] Sahil Takiar commented on HIVE-15982: - [~cartershanklin] if no one is working on this, I would like to take this up. If thats ok with you, I will assign this to myself. > Support the width_bucket function > - > > Key: HIVE-15982 > URL: https://issues.apache.org/jira/browse/HIVE-15982 > Project: Hive > Issue Type: Sub-task > Components: SQL >Reporter: Carter Shanklin > > Support the width_bucket(wbo, wbb1, wbb2, wbc) which returns an integer > between 0 and wbc+1 by mapping wbo into the ith equally sized bucket made by > dividing wbb1 and wbb2 into equally sized regions. If wbo < wbb1, return 1, > if wbo > wbb2 return wbc+1. Reference: SQL standard section 4.4. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-11133) Support hive.explain.user for Spark [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-11133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963794#comment-15963794
]
Hive QA commented on HIVE-11133:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12862766/HIVE-11133.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 10571 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=144)
org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData (batchId=222)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4635/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4635/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4635/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12862766 - PreCommit-HIVE-Build
> Support hive.explain.user for Spark [Spark Branch]
> --
>
> Key: HIVE-11133
> URL: https://issues.apache.org/jira/browse/HIVE-11133
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Mohit Sabharwal
>Assignee: Sahil Takiar
> Attachments: HIVE-11133.1.patch
>
>
> User friendly explain output ({{set hive.explain.user=true}}) should support
> Spark as well.
> Once supported, we should also enable related q-tests like {{explainuser_1.q}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-11418) Dropping a database in an encryption zone with CASCADE and trash enabled fails
[
https://issues.apache.org/jira/browse/HIVE-11418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963762#comment-15963762
]
Hive QA commented on HIVE-11418:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12862765/HIVE-11418.1.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10557 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[encryption_drop_table_in_encrypted_db]
(batchId=15)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=143)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=101)
org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData (batchId=221)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4634/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4634/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4634/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12862765 - PreCommit-HIVE-Build
> Dropping a database in an encryption zone with CASCADE and trash enabled fails
> --
>
> Key: HIVE-11418
> URL: https://issues.apache.org/jira/browse/HIVE-11418
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 1.2.0
>Reporter: Sergio Peña
>Assignee: Sahil Takiar
> Attachments: HIVE-11418.1.patch
>
>
> Here's the query that fails:
> {noformat}
> hive> CREATE DATABASE db;
> hive> USE db;
> hive> CREATE TABLE a(id int);
> hive> SET fs.trash.interval=1;
> hive> DROP DATABASE db CASCADE;
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Unable to drop
> db.a because it is in an encryption zone and trash
> is enabled. Use PURGE option to skip trash.)
> {noformat}
> DROP DATABASE does not support PURGE, so we have to remove the tables one by
> one, and then drop the database.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16239) remove useless hiveserver
[
https://issues.apache.org/jira/browse/HIVE-16239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963758#comment-15963758
]
Fei Hui commented on HIVE-16239:
ping [~Ferd]
> remove useless hiveserver
> -
>
> Key: HIVE-16239
> URL: https://issues.apache.org/jira/browse/HIVE-16239
> Project: Hive
> Issue Type: Bug
> Components: CLI
>Affects Versions: 2.0.1, 2.1.1
>Reporter: Fei Hui
>Assignee: Fei Hui
> Attachments: HIVE-16239.1-branch-2.0.patch,
> HIVE-16239.1-branch-2.1.patch, HIVE-16239.2-branch-2.0.patch,
> HIVE-16239.2-branch-2.1.patch
>
>
> {quote}
> [hadoop@header hive]$ hive --service hiveserver
> Starting Hive Thrift Server
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/opt/apps/apache-hive-2.0.1-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/apps/spark-1.6.2-bin-hadoop2.7/lib/spark-assembly-1.6.2-hadoop2.7.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/apps/hadoop-2.7.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hive.service.HiveServer
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:214)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {quote}
> hiveserver does not exist, we should remove hiveserver from cli on branch-2.0
> After removing it, we get useful message
> {quote}
> Service hiveserver not found
> Available Services: beeline cli hbaseimport hbaseschematool help
> hiveburninclient hiveserver2 hplsql hwi jar lineage llap metastore metatool
> orcfiledump rcfilecat schemaTool version
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-12614) RESET command does not close spark session
[ https://issues.apache.org/jira/browse/HIVE-12614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963745#comment-15963745 ] Nemon Lou commented on HIVE-12614: -- [~stakiar] Thanks for taking it over. :) > RESET command does not close spark session > -- > > Key: HIVE-12614 > URL: https://issues.apache.org/jira/browse/HIVE-12614 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 1.3.0, 2.1.0 >Reporter: Nemon Lou >Assignee: Sahil Takiar >Priority: Minor > Attachments: HIVE-12614.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-12614) RESET command does not close spark session
[ https://issues.apache.org/jira/browse/HIVE-12614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-12614: Status: Patch Available (was: Open) > RESET command does not close spark session > -- > > Key: HIVE-12614 > URL: https://issues.apache.org/jira/browse/HIVE-12614 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 2.1.0, 1.3.0 >Reporter: Nemon Lou >Assignee: Sahil Takiar >Priority: Minor > Attachments: HIVE-12614.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (HIVE-12614) RESET command does not close spark session
[ https://issues.apache.org/jira/browse/HIVE-12614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar reassigned HIVE-12614: --- Assignee: Sahil Takiar (was: Nemon Lou) > RESET command does not close spark session > -- > > Key: HIVE-12614 > URL: https://issues.apache.org/jira/browse/HIVE-12614 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 1.3.0, 2.1.0 >Reporter: Nemon Lou >Assignee: Sahil Takiar >Priority: Minor > Attachments: HIVE-12614.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-12614) RESET command does not close spark session
[ https://issues.apache.org/jira/browse/HIVE-12614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-12614: Attachment: HIVE-12614.1.patch > RESET command does not close spark session > -- > > Key: HIVE-12614 > URL: https://issues.apache.org/jira/browse/HIVE-12614 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 1.3.0, 2.1.0 >Reporter: Nemon Lou >Assignee: Sahil Takiar >Priority: Minor > Attachments: HIVE-12614.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-12614) RESET command does not close spark session
[ https://issues.apache.org/jira/browse/HIVE-12614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963743#comment-15963743 ] Sahil Takiar commented on HIVE-12614: - [~nemon] do you have plans to work on this? If not do you mind if I assign it to myself? > RESET command does not close spark session > -- > > Key: HIVE-12614 > URL: https://issues.apache.org/jira/browse/HIVE-12614 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 1.3.0, 2.1.0 >Reporter: Nemon Lou >Assignee: Nemon Lou >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-10865) Beeline needs to support DELIMITER command
[
https://issues.apache.org/jira/browse/HIVE-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963737#comment-15963737
]
Hive QA commented on HIVE-10865:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12862756/HIVE-10865.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 10574 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData (batchId=221)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4633/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4633/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4633/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12862756 - PreCommit-HIVE-Build
> Beeline needs to support DELIMITER command
> --
>
> Key: HIVE-10865
> URL: https://issues.apache.org/jira/browse/HIVE-10865
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Sahil Takiar
> Attachments: HIVE-10865.1.patch, HIVE-10865.2.patch,
> HIVE-10865.3.patch
>
>
> MySQL Client provides a DELIMITER command to set statement delimiter.
> Beeline needs to support a similar command to allow commands having
> semi-colon as non-statement delimiter (as with MySQL stored procedures). This
> is a follow-up jira for HIVE-10659
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16267) Enable bootstrap function metadata to be loaded in repl load
[ https://issues.apache.org/jira/browse/HIVE-16267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963732#comment-15963732 ] Thejas M Nair commented on HIVE-16267: -- [~anishek] Can you please add a git pull or review board link ? > Enable bootstrap function metadata to be loaded in repl load > > > Key: HIVE-16267 > URL: https://issues.apache.org/jira/browse/HIVE-16267 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2 >Reporter: anishek >Assignee: anishek > Fix For: 3.0.0 > > Attachments: HIVE-16267.1.patch, HIVE-16267.2.patch > > Original Estimate: 72h > Remaining Estimate: 72h > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-11133) Support hive.explain.user for Spark [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-11133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-11133:
Status: Patch Available (was: Open)
> Support hive.explain.user for Spark [Spark Branch]
> --
>
> Key: HIVE-11133
> URL: https://issues.apache.org/jira/browse/HIVE-11133
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Mohit Sabharwal
>Assignee: Sahil Takiar
> Attachments: HIVE-11133.1.patch
>
>
> User friendly explain output ({{set hive.explain.user=true}}) should support
> Spark as well.
> Once supported, we should also enable related q-tests like {{explainuser_1.q}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-11133) Support hive.explain.user for Spark [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-11133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-11133:
Attachment: HIVE-11133.1.patch
Initial patch: enabled {{hive.explain.user}} for Spark, added
{{explainuser_1.q}} for Spark, had to disable the correlation optimizer as it
doesn't seem to work for Spark.
Attaching patch to see what test failures are present. Still need to
investigate to see what extra info is in the Spark plan.
> Support hive.explain.user for Spark [Spark Branch]
> --
>
> Key: HIVE-11133
> URL: https://issues.apache.org/jira/browse/HIVE-11133
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Mohit Sabharwal
>Assignee: Sahil Takiar
> Attachments: HIVE-11133.1.patch
>
>
> User friendly explain output ({{set hive.explain.user=true}}) should support
> Spark as well.
> Once supported, we should also enable related q-tests like {{explainuser_1.q}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (HIVE-11133) Support hive.explain.user for Spark [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-11133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar reassigned HIVE-11133:
---
Assignee: Sahil Takiar
> Support hive.explain.user for Spark [Spark Branch]
> --
>
> Key: HIVE-11133
> URL: https://issues.apache.org/jira/browse/HIVE-11133
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Mohit Sabharwal
>Assignee: Sahil Takiar
>
> User friendly explain output ({{set hive.explain.user=true}}) should support
> Spark as well.
> Once supported, we should also enable related q-tests like {{explainuser_1.q}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-11418) Dropping a database in an encryption zone with CASCADE and trash enabled fails
[
https://issues.apache.org/jira/browse/HIVE-11418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-11418:
Status: Patch Available (was: Open)
> Dropping a database in an encryption zone with CASCADE and trash enabled fails
> --
>
> Key: HIVE-11418
> URL: https://issues.apache.org/jira/browse/HIVE-11418
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 1.2.0
>Reporter: Sergio Peña
>Assignee: Sahil Takiar
> Attachments: HIVE-11418.1.patch
>
>
> Here's the query that fails:
> {noformat}
> hive> CREATE DATABASE db;
> hive> USE db;
> hive> CREATE TABLE a(id int);
> hive> SET fs.trash.interval=1;
> hive> DROP DATABASE db CASCADE;
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Unable to drop
> db.a because it is in an encryption zone and trash
> is enabled. Use PURGE option to skip trash.)
> {noformat}
> DROP DATABASE does not support PURGE, so we have to remove the tables one by
> one, and then drop the database.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-11418) Dropping a database in an encryption zone with CASCADE and trash enabled fails
[
https://issues.apache.org/jira/browse/HIVE-11418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-11418:
Attachment: HIVE-11418.1.patch
RB: https://reviews.apache.org/r/58330/
Removed the check in {{HiveMetaStore#checkTrashPurgeCombination}} - updated the
q test, added a new one called {{encryption_drop_table_in_encrypted_db.q}} that
specifically tests encrypting a folder, and then moving a sub-folder into the
Trash.
> Dropping a database in an encryption zone with CASCADE and trash enabled fails
> --
>
> Key: HIVE-11418
> URL: https://issues.apache.org/jira/browse/HIVE-11418
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 1.2.0
>Reporter: Sergio Peña
>Assignee: Sahil Takiar
> Attachments: HIVE-11418.1.patch
>
>
> Here's the query that fails:
> {noformat}
> hive> CREATE DATABASE db;
> hive> USE db;
> hive> CREATE TABLE a(id int);
> hive> SET fs.trash.interval=1;
> hive> DROP DATABASE db CASCADE;
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Unable to drop
> db.a because it is in an encryption zone and trash
> is enabled. Use PURGE option to skip trash.)
> {noformat}
> DROP DATABASE does not support PURGE, so we have to remove the tables one by
> one, and then drop the database.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16328) HoS: more aggressive mapjoin optimization when hive.spark.use.file.size.for.mapjoin is true
[
https://issues.apache.org/jira/browse/HIVE-16328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963701#comment-15963701
]
Hive QA commented on HIVE-16328:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12862755/HIVE-16328.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 10570 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=143)
org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData (batchId=221)
org.apache.hive.jdbc.TestMultiSessionsHS2WithLocalClusterSpark.testSparkQuery
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4632/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4632/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4632/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12862755 - PreCommit-HIVE-Build
> HoS: more aggressive mapjoin optimization when
> hive.spark.use.file.size.for.mapjoin is true
> ---
>
> Key: HIVE-16328
> URL: https://issues.apache.org/jira/browse/HIVE-16328
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Reporter: Chao Sun
>Assignee: Chao Sun
> Attachments: HIVE-16328.1.patch, HIVE-16328.2.patch
>
>
> In HIVE-15489, when {{hive.spark.use.file.size.for.mapjoin}} is set to true,
> and if the JOIN op has any upstream RS operator, then we will stop converting
> the JOIN op to MAPJOIN op.
> However, this is overly conservative. A better solution is to treat the
> branch that has upstream RS as the big table and check if all other branches
> are map-only AND can fit in hash table size.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15708) Upgrade calcite version to 1.12
[ https://issues.apache.org/jira/browse/HIVE-15708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963692#comment-15963692 ] Ashutosh Chauhan commented on HIVE-15708: - Can you please update RB with latest patch? > Upgrade calcite version to 1.12 > --- > > Key: HIVE-15708 > URL: https://issues.apache.org/jira/browse/HIVE-15708 > Project: Hive > Issue Type: Task > Components: CBO, Logical Optimizer >Affects Versions: 2.2.0 >Reporter: Ashutosh Chauhan >Assignee: Remus Rusanu > Attachments: HIVE-15708.01.patch, HIVE-15708.02.patch, > HIVE-15708.03.patch, HIVE-15708.04.patch, HIVE-15708.05.patch, > HIVE-15708.06.patch, HIVE-15708.07.patch, HIVE-15708.08.patch, > HIVE-15708.09.patch, HIVE-15708.10.patch, HIVE-15708.11.patch, > HIVE-15708.12.patch, HIVE-15708.13.patch, HIVE-15708.14.patch, > HIVE-15708.15.patch, HIVE-15708.15.patch, HIVE-15708.16.patch, > HIVE-15708.17.patch, HIVE-15708.18.patch, HIVE-15708.19.patch, > HIVe-15708.20.patch, HIVE-15708.21.patch > > > Currently we are on 1.10 Need to upgrade calcite version to 1.11 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-10865) Beeline needs to support DELIMITER command
[
https://issues.apache.org/jira/browse/HIVE-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963660#comment-15963660
]
Hive QA commented on HIVE-10865:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12862756/HIVE-10865.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 10574 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData (batchId=221)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4631/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4631/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4631/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12862756 - PreCommit-HIVE-Build
> Beeline needs to support DELIMITER command
> --
>
> Key: HIVE-10865
> URL: https://issues.apache.org/jira/browse/HIVE-10865
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Sahil Takiar
> Attachments: HIVE-10865.1.patch, HIVE-10865.2.patch,
> HIVE-10865.3.patch
>
>
> MySQL Client provides a DELIMITER command to set statement delimiter.
> Beeline needs to support a similar command to allow commands having
> semi-colon as non-statement delimiter (as with MySQL stored procedures). This
> is a follow-up jira for HIVE-10659
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-16380) removing global test dependency of jsonassert
[ https://issues.apache.org/jira/browse/HIVE-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-16380: - Resolution: Fixed Fix Version/s: 2.3.0 2.4.0 Status: Resolved (was: Patch Available) Patch committed to master, branch-2, branch-2.3 Thanks Anishek! > removing global test dependency of jsonassert > - > > Key: HIVE-16380 > URL: https://issues.apache.org/jira/browse/HIVE-16380 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: anishek >Assignee: anishek >Priority: Minor > Fix For: 2.4.0, 2.3.0, 3.0.0 > > Attachments: HIVE-16380.1.patch, HIVE-16380.2.patch > > > as part of commit done for HIVE-16219, there seems to additional changes in > the root level pom.xml, they should not be required. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16416) Service: move constants out from HiveAuthFactory
[
https://issues.apache.org/jira/browse/HIVE-16416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963635#comment-15963635
]
Hive QA commented on HIVE-16416:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12862744/HIVE-16416.1.patch
{color:green}SUCCESS:{color} +1 due to 10 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 10570 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_display_colstats_tbllvl]
(batchId=73)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=143)
org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData (batchId=221)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4630/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4630/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4630/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12862744 - PreCommit-HIVE-Build
> Service: move constants out from HiveAuthFactory
>
>
> Key: HIVE-16416
> URL: https://issues.apache.org/jira/browse/HIVE-16416
> Project: Hive
> Issue Type: Sub-task
> Components: Server Infrastructure
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-16416.1.patch
>
>
> It took me a while to notice that there are only some constants which are
> keep pulling in this class :)
> it contains a tricky dependency to the whole ql module; but in client mode
> that part is totally unused - moving the constants out from it, enables the
> client to operate without the factory.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-10865) Beeline needs to support DELIMITER command
[ https://issues.apache.org/jira/browse/HIVE-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-10865: Attachment: HIVE-10865.3.patch > Beeline needs to support DELIMITER command > -- > > Key: HIVE-10865 > URL: https://issues.apache.org/jira/browse/HIVE-10865 > Project: Hive > Issue Type: Bug > Components: Beeline >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Sahil Takiar > Attachments: HIVE-10865.1.patch, HIVE-10865.2.patch, > HIVE-10865.3.patch > > > MySQL Client provides a DELIMITER command to set statement delimiter. > Beeline needs to support a similar command to allow commands having > semi-colon as non-statement delimiter (as with MySQL stored procedures). This > is a follow-up jira for HIVE-10659 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16328) HoS: more aggressive mapjoin optimization when hive.spark.use.file.size.for.mapjoin is true
[
https://issues.apache.org/jira/browse/HIVE-16328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-16328:
Attachment: HIVE-16328.2.patch
Discussed with Xuefu & Jimmy offline. I think instead of rejecting branches
with RS, we can reject branches that contain either JOIN or LATERAL_VIEW, which
could potentially increase output data size. We mark those as big table
branches, and, if we have more than one such branch, the JOIN op cannot be
turned to a MAPJOIN op. This opens up more opportunities for optimizations.
Attaching patch v2 which realizes the above idea.
> HoS: more aggressive mapjoin optimization when
> hive.spark.use.file.size.for.mapjoin is true
> ---
>
> Key: HIVE-16328
> URL: https://issues.apache.org/jira/browse/HIVE-16328
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Reporter: Chao Sun
>Assignee: Chao Sun
> Attachments: HIVE-16328.1.patch, HIVE-16328.2.patch
>
>
> In HIVE-15489, when {{hive.spark.use.file.size.for.mapjoin}} is set to true,
> and if the JOIN op has any upstream RS operator, then we will stop converting
> the JOIN op to MAPJOIN op.
> However, this is overly conservative. A better solution is to treat the
> branch that has upstream RS as the big table and check if all other branches
> are map-only AND can fit in hash table size.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-10865) Beeline needs to support DELIMITER command
[
https://issues.apache.org/jira/browse/HIVE-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-10865:
Attachment: HIVE-10865.2.patch
Updated patch, adds supports for {{!delimiter [delimiter]}} command in Beeline.
Some other bug fixes. Created an RB: https://reviews.apache.org/r/58318
> Beeline needs to support DELIMITER command
> --
>
> Key: HIVE-10865
> URL: https://issues.apache.org/jira/browse/HIVE-10865
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Sahil Takiar
> Attachments: HIVE-10865.1.patch, HIVE-10865.2.patch
>
>
> MySQL Client provides a DELIMITER command to set statement delimiter.
> Beeline needs to support a similar command to allow commands having
> semi-colon as non-statement delimiter (as with MySQL stored procedures). This
> is a follow-up jira for HIVE-10659
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-16417) Introduce Service-client module
[
https://issues.apache.org/jira/browse/HIVE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-16417:
Attachment: HIVE-16417.1.patch
patch #1) move files around...and update pom.xml-s
> Introduce Service-client module
> ---
>
> Key: HIVE-16417
> URL: https://issues.apache.org/jira/browse/HIVE-16417
> Project: Hive
> Issue Type: Sub-task
> Components: Metastore, Server Infrastructure
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-16417.1.patch
>
>
> Moving the relevant classes out from service, enables the jdbc driver to
> relax its dependencies to only use {{service-client}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (HIVE-16417) Introduce Service-client module
[
https://issues.apache.org/jira/browse/HIVE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich reassigned HIVE-16417:
---
> Introduce Service-client module
> ---
>
> Key: HIVE-16417
> URL: https://issues.apache.org/jira/browse/HIVE-16417
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>
> Moving the relevant classes out from service, enables the jdbc driver to
> relax its dependencies to only use {{service-client}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-16416) Service: move constants out from HiveAuthFactory
[ https://issues.apache.org/jira/browse/HIVE-16416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-16416: Status: Patch Available (was: Open) > Service: move constants out from HiveAuthFactory > > > Key: HIVE-16416 > URL: https://issues.apache.org/jira/browse/HIVE-16416 > Project: Hive > Issue Type: Sub-task > Components: Server Infrastructure >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-16416.1.patch > > > It took me a while to notice that there are only some constants which are > keep pulling in this class :) > it contains a tricky dependency to the whole ql module; but in client mode > that part is totally unused - moving the constants out from it, enables the > client to operate without the factory. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16416) Service: move constants out from HiveAuthFactory
[
https://issues.apache.org/jira/browse/HIVE-16416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-16416:
Attachment: HIVE-16416.1.patch
patch#1) move constants to {{HiveAuthConstants}} class
> Service: move constants out from HiveAuthFactory
>
>
> Key: HIVE-16416
> URL: https://issues.apache.org/jira/browse/HIVE-16416
> Project: Hive
> Issue Type: Sub-task
> Components: Server Infrastructure
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-16416.1.patch
>
>
> It took me a while to notice that there are only some constants which are
> keep pulling in this class :)
> it contains a tricky dependency to the whole ql module; but in client mode
> that part is totally unused - moving the constants out from it, enables the
> client to operate without the factory.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (HIVE-16416) Service: move constants out from HiveAuthFactory
[ https://issues.apache.org/jira/browse/HIVE-16416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich reassigned HIVE-16416: --- > Service: move constants out from HiveAuthFactory > > > Key: HIVE-16416 > URL: https://issues.apache.org/jira/browse/HIVE-16416 > Project: Hive > Issue Type: Sub-task > Components: Server Infrastructure >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > > It took me a while to notice that there are only some constants which are > keep pulling in this class :) > it contains a tricky dependency to the whole ql module; but in client mode > that part is totally unused - moving the constants out from it, enables the > client to operate without the factory. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16415) Add blobstore tests for insertion of zero rows
[
https://issues.apache.org/jira/browse/HIVE-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963359#comment-15963359
]
Hive QA commented on HIVE-16415:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12862730/HIVE-16415.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 10572 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData (batchId=221)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4629/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4629/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4629/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12862730 - PreCommit-HIVE-Build
> Add blobstore tests for insertion of zero rows
> --
>
> Key: HIVE-16415
> URL: https://issues.apache.org/jira/browse/HIVE-16415
> Project: Hive
> Issue Type: Test
> Components: Tests
>Affects Versions: 2.1.1
>Reporter: Thomas Poepping
>Assignee: Thomas Poepping
> Attachments: HIVE-16415.patch
>
>
> This patch introduces two regression tests into the hive-blobstore qtest
> module: zero_rows_hdfs.q and zero_rows_blobstore.q. These test doing INSERT
> commands with a WHERE clause where the condition of the WHERE clause causes
> zero rows to be considered.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-16415) Add blobstore tests for insertion of zero rows
[ https://issues.apache.org/jira/browse/HIVE-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Poepping updated HIVE-16415: --- Status: Patch Available (was: Open) > Add blobstore tests for insertion of zero rows > -- > > Key: HIVE-16415 > URL: https://issues.apache.org/jira/browse/HIVE-16415 > Project: Hive > Issue Type: Test > Components: Tests >Affects Versions: 2.1.1 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Attachments: HIVE-16415.patch > > > This patch introduces two regression tests into the hive-blobstore qtest > module: zero_rows_hdfs.q and zero_rows_blobstore.q. These test doing INSERT > commands with a WHERE clause where the condition of the WHERE clause causes > zero rows to be considered. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16415) Add blobstore tests for insertion of zero rows
[ https://issues.apache.org/jira/browse/HIVE-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Poepping updated HIVE-16415: --- Attachment: HIVE-16415.patch Attached patch > Add blobstore tests for insertion of zero rows > -- > > Key: HIVE-16415 > URL: https://issues.apache.org/jira/browse/HIVE-16415 > Project: Hive > Issue Type: Test > Components: Tests >Affects Versions: 2.1.1 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Attachments: HIVE-16415.patch > > > This patch introduces two regression tests into the hive-blobstore qtest > module: zero_rows_hdfs.q and zero_rows_blobstore.q. These test doing INSERT > commands with a WHERE clause where the condition of the WHERE clause causes > zero rows to be considered. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (HIVE-16415) Add blobstore tests for insertion of zero rows
[ https://issues.apache.org/jira/browse/HIVE-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Poepping reassigned HIVE-16415: -- > Add blobstore tests for insertion of zero rows > -- > > Key: HIVE-16415 > URL: https://issues.apache.org/jira/browse/HIVE-16415 > Project: Hive > Issue Type: Test > Components: Tests >Affects Versions: 2.1.1 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > > This patch introduces two regression tests into the hive-blobstore qtest > module: zero_rows_hdfs.q and zero_rows_blobstore.q. These test doing INSERT > commands with a WHERE clause where the condition of the WHERE clause causes > zero rows to be considered. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15708) Upgrade calcite version to 1.12
[
https://issues.apache.org/jira/browse/HIVE-15708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963119#comment-15963119
]
Remus Rusanu commented on HIVE-15708:
-
for {{optimize_filter_literal}} I've updated the golden file. {{varchar_join1}}
and {{explainanalyze_1}} I could not repro and they have no diff in lab run,
while {{TestJdbcDriver2}} failure is failing other builds too.
> Upgrade calcite version to 1.12
> ---
>
> Key: HIVE-15708
> URL: https://issues.apache.org/jira/browse/HIVE-15708
> Project: Hive
> Issue Type: Task
> Components: CBO, Logical Optimizer
>Affects Versions: 2.2.0
>Reporter: Ashutosh Chauhan
>Assignee: Remus Rusanu
> Attachments: HIVE-15708.01.patch, HIVE-15708.02.patch,
> HIVE-15708.03.patch, HIVE-15708.04.patch, HIVE-15708.05.patch,
> HIVE-15708.06.patch, HIVE-15708.07.patch, HIVE-15708.08.patch,
> HIVE-15708.09.patch, HIVE-15708.10.patch, HIVE-15708.11.patch,
> HIVE-15708.12.patch, HIVE-15708.13.patch, HIVE-15708.14.patch,
> HIVE-15708.15.patch, HIVE-15708.15.patch, HIVE-15708.16.patch,
> HIVE-15708.17.patch, HIVE-15708.18.patch, HIVE-15708.19.patch,
> HIVe-15708.20.patch, HIVE-15708.21.patch
>
>
> Currently we are on 1.10 Need to upgrade calcite version to 1.11
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16316) Prepare master branch for 3.0.0 development.
[ https://issues.apache.org/jira/browse/HIVE-16316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963087#comment-15963087 ] Naveen Gangam commented on HIVE-16316: -- The 2 scenarios to consider are 1) Hive 3.0 is released before Hive 2.3. Correct if if I am wrong, I think we are covered in this case because the upgrade path would be 2.2.0 --> 3.0.0. We have all the scripts in place for this upgrade. 2) Hive 3.0 is release after Hive 2.3. Where or not there are real schema changes between 2.3 and 3.0, there will always be a change to update the version in the VERSION table. So when 2.3 is released, we will need to update the master to include scripts to upgrade from 2.3 to 3.0. The reason for deferring this change is 2-fold. a) there may never be a 2.3 release. or b) Hive 2.4 or 2.5 might even be released before Hive 3.0 in which case we have to have scripts for upgrade from the latest release. Does this make sense? Thanks > Prepare master branch for 3.0.0 development. > > > Key: HIVE-16316 > URL: https://issues.apache.org/jira/browse/HIVE-16316 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam > Fix For: 3.0.0 > > Attachments: HIVE-16316.patch > > > master branch is now being used for 3.0.0 development. The build files will > need to reflect this change. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16316) Prepare master branch for 3.0.0 development.
[ https://issues.apache.org/jira/browse/HIVE-16316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963071#comment-15963071 ] Aihua Xu commented on HIVE-16316: - I didn't notice that during the code review. Not sure how typically we handle branch version upstream. 2.3 and later 2.x technically should be considered as minor release from 2.2 and shouldn't involve schema change? > Prepare master branch for 3.0.0 development. > > > Key: HIVE-16316 > URL: https://issues.apache.org/jira/browse/HIVE-16316 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam > Fix For: 3.0.0 > > Attachments: HIVE-16316.patch > > > master branch is now being used for 3.0.0 development. The build files will > need to reflect this change. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15708) Upgrade calcite version to 1.12
[
https://issues.apache.org/jira/browse/HIVE-15708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963061#comment-15963061
]
Hive QA commented on HIVE-15708:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12862712/HIVE-15708.21.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10570 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[optimize_filter_literal]
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[varchar_join1]
(batchId=5)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_1]
(batchId=96)
org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData (batchId=221)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4628/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4628/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4628/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12862712 - PreCommit-HIVE-Build
> Upgrade calcite version to 1.12
> ---
>
> Key: HIVE-15708
> URL: https://issues.apache.org/jira/browse/HIVE-15708
> Project: Hive
> Issue Type: Task
> Components: CBO, Logical Optimizer
>Affects Versions: 2.2.0
>Reporter: Ashutosh Chauhan
>Assignee: Remus Rusanu
> Attachments: HIVE-15708.01.patch, HIVE-15708.02.patch,
> HIVE-15708.03.patch, HIVE-15708.04.patch, HIVE-15708.05.patch,
> HIVE-15708.06.patch, HIVE-15708.07.patch, HIVE-15708.08.patch,
> HIVE-15708.09.patch, HIVE-15708.10.patch, HIVE-15708.11.patch,
> HIVE-15708.12.patch, HIVE-15708.13.patch, HIVE-15708.14.patch,
> HIVE-15708.15.patch, HIVE-15708.15.patch, HIVE-15708.16.patch,
> HIVE-15708.17.patch, HIVE-15708.18.patch, HIVE-15708.19.patch,
> HIVe-15708.20.patch, HIVE-15708.21.patch
>
>
> Currently we are on 1.10 Need to upgrade calcite version to 1.11
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16287) Alter table partition rename with location - moves partition back to hive warehouse
[
https://issues.apache.org/jira/browse/HIVE-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963047#comment-15963047
]
Sergio Peña commented on HIVE-16287:
[~vihangk1] The patch looks good. I just wonder if we should wrap the code in a
new method, something like:
{noformat}
Path getPartitionPath(String dbname, Table table, Partition partition);
{noformat}
> Alter table partition rename with location - moves partition back to hive
> warehouse
> ---
>
> Key: HIVE-16287
> URL: https://issues.apache.org/jira/browse/HIVE-16287
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.1.0
> Environment: RHEL 6.8
>Reporter: Ying Chen
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Attachments: HIVE-16287.01.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> I was renaming my partition in a table that I've created using the location
> clause, and noticed that when after rename is completed, my partition is
> moved to the hive warehouse (hive.metastore.warehouse.dir).
> {quote}
> create table test_local_part (col1 int) partitioned by (col2 int) location
> '/tmp/testtable/test_local_part';
> insert into test_local_part partition (col2=1) values (1),(3);
> insert into test_local_part partition (col2=2) values (3);
> alter table test_local_part partition (col2='1') rename to partition
> (col2='4');
> {quote}
> Running:
>describe formatted test_local_part partition (col2='2')
> # Detailed Partition Information
> Partition Value: [2]
> Database: default
> Table:test_local_part
> CreateTime: Mon Mar 20 13:25:28 PDT 2017
> LastAccessTime: UNKNOWN
> Protect Mode: None
> Location:
> *hdfs://my.server.com:8020/tmp/testtable/test_local_part/col2=2*
> Running:
>describe formatted test_local_part partition (col2='4')
> # Detailed Partition Information
> Partition Value: [4]
> Database: default
> Table:test_local_part
> CreateTime: Mon Mar 20 13:24:53 PDT 2017
> LastAccessTime: UNKNOWN
> Protect Mode: None
> Location:
> *hdfs://my.server.com:8020/apps/hive/warehouse/test_local_part/col2=4*
> ---
> Per Sergio's comment - "The rename should create the new partition name in
> the same location of the table. "
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16409) TestEventHandlerFactory has lacked the ASF header
[ https://issues.apache.org/jira/browse/HIVE-16409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963012#comment-15963012 ] Aihua Xu commented on HIVE-16409: - Looks good. +1. > TestEventHandlerFactory has lacked the ASF header > -- > > Key: HIVE-16409 > URL: https://issues.apache.org/jira/browse/HIVE-16409 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Saijin Huang >Assignee: Saijin Huang >Priority: Minor > Attachments: HIVE-16409.1.patch > > > when i test the issue HIVE-16061, i found TestEventHandlerFactory.java lack > the ASF header. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16345) BeeLineDriver should be able to run qtest files which are using default database tables
[ https://issues.apache.org/jira/browse/HIVE-16345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongzhi Chen updated HIVE-16345: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Push the changes into master. Thanks [~pvary] for your contribution. > BeeLineDriver should be able to run qtest files which are using default > database tables > --- > > Key: HIVE-16345 > URL: https://issues.apache.org/jira/browse/HIVE-16345 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Reporter: Peter Vary >Assignee: Peter Vary > Fix For: 3.0.0 > > Attachments: HIVE-16345.2.patch, HIVE-16345.3.patch, HIVE-16345.patch > > > It would be good to be able to run the default clientpositive tests. > Currently we can not do that, since we start with a specific database. We > should filter the query input and replace the table references -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15708) Upgrade calcite version to 1.12
[ https://issues.apache.org/jira/browse/HIVE-15708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu updated HIVE-15708: Attachment: HIVE-15708.21.patch .21.patch rebased to current master and updated several .out results that changed in master > Upgrade calcite version to 1.12 > --- > > Key: HIVE-15708 > URL: https://issues.apache.org/jira/browse/HIVE-15708 > Project: Hive > Issue Type: Task > Components: CBO, Logical Optimizer >Affects Versions: 2.2.0 >Reporter: Ashutosh Chauhan >Assignee: Remus Rusanu > Attachments: HIVE-15708.01.patch, HIVE-15708.02.patch, > HIVE-15708.03.patch, HIVE-15708.04.patch, HIVE-15708.05.patch, > HIVE-15708.06.patch, HIVE-15708.07.patch, HIVE-15708.08.patch, > HIVE-15708.09.patch, HIVE-15708.10.patch, HIVE-15708.11.patch, > HIVE-15708.12.patch, HIVE-15708.13.patch, HIVE-15708.14.patch, > HIVE-15708.15.patch, HIVE-15708.15.patch, HIVE-15708.16.patch, > HIVE-15708.17.patch, HIVE-15708.18.patch, HIVE-15708.19.patch, > HIVe-15708.20.patch, HIVE-15708.21.patch > > > Currently we are on 1.10 Need to upgrade calcite version to 1.11 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15826) Add 'serialization.encoding' To All SerDes
[ https://issues.apache.org/jira/browse/HIVE-15826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962937#comment-15962937 ] David commented on HIVE-15826: -- Could you add column coments in CSV SerDe just like in RegexSerDe? > Add 'serialization.encoding' To All SerDes > -- > > Key: HIVE-15826 > URL: https://issues.apache.org/jira/browse/HIVE-15826 > Project: Hive > Issue Type: Improvement > Components: Serializers/Deserializers >Reporter: BELUGA BEHR >Assignee: Teddy Choi > Attachments: HIVE-15826.1.patch > > > The feature of 'serialization.encoding' has been implemented for the > following SerDes: > MultiDelimitSerDe, LazySimpleSerDe > Please extend this feature to: > JSON, CSV, and RegexSerDe -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16391) Publish proper Hive 1.2 jars (without including all dependencies in uber jar)
[ https://issues.apache.org/jira/browse/HIVE-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962887#comment-15962887 ] Edward Capriolo commented on HIVE-16391: It is good to have someone submit a clean patch, against trunk and also one back ported to your target version 1.2.2. I roughly get the use case but you might want to state this more specifically which submodules you are depending on and what are the problematic dependencies. Potentially include the output of : mvn dependency tree > Publish proper Hive 1.2 jars (without including all dependencies in uber jar) > - > > Key: HIVE-16391 > URL: https://issues.apache.org/jira/browse/HIVE-16391 > Project: Hive > Issue Type: Task > Components: Build Infrastructure >Reporter: Reynold Xin > > Apache Spark currently depends on a forked version of Apache Hive. AFAIK, the > only change in the fork is to work around the issue that Hive publishes only > two sets of jars: one set with no dependency declared, and another with all > the dependencies included in the published uber jar. That is to say, Hive > doesn't publish a set of jars with the proper dependencies declared. > There is general consensus on both sides that we should remove the forked > Hive. > The change in the forked version is recorded here > https://github.com/JoshRosen/hive/tree/release-1.2.1-spark2 > Note that the fork in the past included other fixes but those have all become > unnecessary. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16414) [Hive on Tez] Hive Union queries resource efficiency less on Tez than Mapreduce
[ https://issues.apache.org/jira/browse/HIVE-16414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Teja Chilukuri updated HIVE-16414: --- Summary: [Hive on Tez] Hive Union queries resource efficiency less on Tez than Mapreduce (was: [Hive on Tez] Union queries resources efficiency less on Tez than Mapreduce) > [Hive on Tez] Hive Union queries resource efficiency less on Tez than > Mapreduce > --- > > Key: HIVE-16414 > URL: https://issues.apache.org/jira/browse/HIVE-16414 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 2.1.0 >Reporter: Ravi Teja Chilukuri > > When a hive union query with the sub queries reading the same table is run in > Mapreduce and tez, Mapreduce reads the table only once, no matter how many > reads on the same table are present, > but tez reads the same table multiple times in the form of multiple vertices. > If a table is to be read by X mappers, > Tez runs with kX map tasks where k is the number of sub queries reading from > the same table and > Mapreduce runs with X mappers no matter how many sub queries are present. > For such union queries, we need to fall back to MR instead of TEZ. > *Query:* > http://pastebin.com/t6n91u6a > *Tez explain plan:* > http://pastebin.com/aWwVxhii > *MR explain plan:* > http://pastebin.com/iDbWwtKR -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16267) Enable bootstrap function metadata to be loaded in repl load
[ https://issues.apache.org/jira/browse/HIVE-16267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962637#comment-15962637 ] anishek commented on HIVE-16267: [~sushanth]/[~thejas] please review the patch. > Enable bootstrap function metadata to be loaded in repl load > > > Key: HIVE-16267 > URL: https://issues.apache.org/jira/browse/HIVE-16267 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2 >Reporter: anishek >Assignee: anishek > Fix For: 3.0.0 > > Attachments: HIVE-16267.1.patch, HIVE-16267.2.patch > > Original Estimate: 72h > Remaining Estimate: 72h > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16413) Create table as select does not check ownership of the location
[
https://issues.apache.org/jira/browse/HIVE-16413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962619#comment-15962619
]
Hive QA commented on HIVE-16413:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12862650/HIVE-16413.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10561 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[drop_with_concurrency]
(batchId=234)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[escape_comments]
(batchId=234)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=143)
org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData (batchId=221)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4627/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4627/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4627/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12862650 - PreCommit-HIVE-Build
> Create table as select does not check ownership of the location
> ---
>
> Key: HIVE-16413
> URL: https://issues.apache.org/jira/browse/HIVE-16413
> Project: Hive
> Issue Type: Bug
> Components: Authorization, SQLStandardAuthorization
>Affects Versions: 1.3.0, 1.2.2, 2.1.1
> Environment: hive-1.2.2, with following conf:
> hive.security.authorization.enabled: true
> hive.security.authorization.manager:
> org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactory
> hive.security.authenticator.manager:
> org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator
>Reporter: Niklaus Xiao
>Assignee: Niklaus Xiao
> Fix For: 2.2.0
>
> Attachments: HIVE-16413.patch
>
>
> 1. following statement failed:
> {code}
> create table foo(id int) location 'hdfs:///tmp/foo';
> Error: Error while compiling statement: FAILED: HiveAccessControlException
> Permission denied: Principal [name=userx, type=USER] does not have following
> privileges for operation CREATETABLE [[OBJECT OWNERSHIP] on Object
> [type=DFS_URI, name=hdfs://hacluster/tmp/foo]] (state=42000,code=4)
> {code}
> 2. but when use create table as select, it successed:
> {code}
> 0: jdbc:hive2://189.39.151.44:21066/> create table foo location
> 'hdfs:///tmp/foo' as select * from xxx2;
> INFO : Number of reduce tasks is set to 0 since there's no reduce operator
> INFO : number of splits:1
> INFO : Submitting tokens for job: job_1491449632882_0094
> INFO : Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:hacluster
> INFO : The url to track the job:
> https://189-39-151-44:26001/proxy/application_1491449632882_0094/
> INFO : Starting Job = job_1491449632882_0094, Tracking URL =
> https://189-39-151-44:26001/proxy/application_1491449632882_0094/
> INFO : Kill Command = /opt/hive-1.3.0/bin/..//../hadoop/bin/hadoop job
> -kill job_1491449632882_0094
> INFO : Hadoop job information for Stage-1: number of mappers: 1; number of
> reducers: 0
> INFO : 2017-04-10 09:44:49,185 Stage-1 map = 0%, reduce = 0%
> INFO : 2017-04-10 09:44:57,202 Stage-1 map = 100%, reduce = 0%, Cumulative
> CPU 1.98 sec
> INFO : MapReduce Total cumulative CPU time: 1 seconds 980 msec
> INFO : Ended Job = job_1491449632882_0094
> INFO : Stage-3 is selected by condition resolver.
> INFO : Stage-2 is filtered out by condition resolver.
> INFO : Stage-4 is filtered out by condition resolver.
> INFO : Moving data to directory
> hdfs://hacluster/user/hive/warehouse/.hive-staging_hive_2017-04-10_09-44-32_462_4902211653847168915-1/-ext-10001
> from
> hdfs://hacluster/user/hive/warehouse/.hive-staging_hive_2017-04-10_09-44-32_462_4902211653847168915-1/-ext-10003
> INFO : Moving data to directory hdfs:/tmp/foo from
> hdfs://hacluster/user/hive/warehouse/.hive-staging_hive_2017-04-10_09-44-32_462_4902211653847168915-1/-ext-10001
> No rows affected (26.969 seconds)
> {code}
> 3. and the table location is hdfs://hacluster/tmp/foo :
> {code}
> 0: jdbc:hive2://189.39.151.44:21066/> desc formatted foo;
> +---+---+---+--+
> | col_name| data_type
> |comment|
>
[jira] [Updated] (HIVE-16409) TestEventHandlerFactory has lacked the ASF header
[ https://issues.apache.org/jira/browse/HIVE-16409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saijin Huang updated HIVE-16409: Description: when i test the issue HIVE-16061, i found TestEventHandlerFactory.java lack the ASF header. (was: when i test the isssue HIVE-16061, i found TestEventHandlerFactory.java lack the ASF header.) > TestEventHandlerFactory has lacked the ASF header > -- > > Key: HIVE-16409 > URL: https://issues.apache.org/jira/browse/HIVE-16409 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Saijin Huang >Assignee: Saijin Huang >Priority: Minor > Attachments: HIVE-16409.1.patch > > > when i test the issue HIVE-16061, i found TestEventHandlerFactory.java lack > the ASF header. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16267) Enable bootstrap function metadata to be loaded in repl load
[
https://issues.apache.org/jira/browse/HIVE-16267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962569#comment-15962569
]
Hive QA commented on HIVE-16267:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12862651/HIVE-16267.2.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10562 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[drop_with_concurrency]
(batchId=234)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[escape_comments]
(batchId=234)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[update_all_non_partitioned]
(batchId=7)
org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData (batchId=221)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4626/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4626/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4626/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12862651 - PreCommit-HIVE-Build
> Enable bootstrap function metadata to be loaded in repl load
>
>
> Key: HIVE-16267
> URL: https://issues.apache.org/jira/browse/HIVE-16267
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2
>Reporter: anishek
>Assignee: anishek
> Fix For: 3.0.0
>
> Attachments: HIVE-16267.1.patch, HIVE-16267.2.patch
>
> Original Estimate: 72h
> Remaining Estimate: 72h
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-16267) Enable bootstrap function metadata to be loaded in repl load
[ https://issues.apache.org/jira/browse/HIVE-16267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-16267: --- Status: Patch Available (was: In Progress) modified to additionally include the ability to get last replication id just before getting the object(table/function/database) to be dumped from metastore. The has before relationship to get the id is captured in the test so that reordering does not happen as part of other people working on the code. > Enable bootstrap function metadata to be loaded in repl load > > > Key: HIVE-16267 > URL: https://issues.apache.org/jira/browse/HIVE-16267 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2 >Reporter: anishek >Assignee: anishek > Fix For: 3.0.0 > > Attachments: HIVE-16267.1.patch, HIVE-16267.2.patch > > Original Estimate: 72h > Remaining Estimate: 72h > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16267) Enable bootstrap function metadata to be loaded in repl load
[ https://issues.apache.org/jira/browse/HIVE-16267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-16267: --- Attachment: HIVE-16267.2.patch > Enable bootstrap function metadata to be loaded in repl load > > > Key: HIVE-16267 > URL: https://issues.apache.org/jira/browse/HIVE-16267 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2 >Reporter: anishek >Assignee: anishek > Fix For: 3.0.0 > > Attachments: HIVE-16267.1.patch, HIVE-16267.2.patch > > Original Estimate: 72h > Remaining Estimate: 72h > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16413) Create table as select does not check ownership of the location
[
https://issues.apache.org/jira/browse/HIVE-16413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niklaus Xiao updated HIVE-16413:
Attachment: HIVE-16413.patch
> Create table as select does not check ownership of the location
> ---
>
> Key: HIVE-16413
> URL: https://issues.apache.org/jira/browse/HIVE-16413
> Project: Hive
> Issue Type: Bug
> Components: Authorization, SQLStandardAuthorization
>Affects Versions: 1.3.0, 1.2.2, 2.1.1
> Environment: hive-1.2.2, with following conf:
> hive.security.authorization.enabled: true
> hive.security.authorization.manager:
> org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactory
> hive.security.authenticator.manager:
> org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator
>Reporter: Niklaus Xiao
>Assignee: Niklaus Xiao
> Fix For: 2.2.0
>
> Attachments: HIVE-16413.patch
>
>
> 1. following statement failed:
> {code}
> create table foo(id int) location 'hdfs:///tmp/foo';
> Error: Error while compiling statement: FAILED: HiveAccessControlException
> Permission denied: Principal [name=userx, type=USER] does not have following
> privileges for operation CREATETABLE [[OBJECT OWNERSHIP] on Object
> [type=DFS_URI, name=hdfs://hacluster/tmp/foo]] (state=42000,code=4)
> {code}
> 2. but when use create table as select, it successed:
> {code}
> 0: jdbc:hive2://189.39.151.44:21066/> create table foo location
> 'hdfs:///tmp/foo' as select * from xxx2;
> INFO : Number of reduce tasks is set to 0 since there's no reduce operator
> INFO : number of splits:1
> INFO : Submitting tokens for job: job_1491449632882_0094
> INFO : Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:hacluster
> INFO : The url to track the job:
> https://189-39-151-44:26001/proxy/application_1491449632882_0094/
> INFO : Starting Job = job_1491449632882_0094, Tracking URL =
> https://189-39-151-44:26001/proxy/application_1491449632882_0094/
> INFO : Kill Command = /opt/hive-1.3.0/bin/..//../hadoop/bin/hadoop job
> -kill job_1491449632882_0094
> INFO : Hadoop job information for Stage-1: number of mappers: 1; number of
> reducers: 0
> INFO : 2017-04-10 09:44:49,185 Stage-1 map = 0%, reduce = 0%
> INFO : 2017-04-10 09:44:57,202 Stage-1 map = 100%, reduce = 0%, Cumulative
> CPU 1.98 sec
> INFO : MapReduce Total cumulative CPU time: 1 seconds 980 msec
> INFO : Ended Job = job_1491449632882_0094
> INFO : Stage-3 is selected by condition resolver.
> INFO : Stage-2 is filtered out by condition resolver.
> INFO : Stage-4 is filtered out by condition resolver.
> INFO : Moving data to directory
> hdfs://hacluster/user/hive/warehouse/.hive-staging_hive_2017-04-10_09-44-32_462_4902211653847168915-1/-ext-10001
> from
> hdfs://hacluster/user/hive/warehouse/.hive-staging_hive_2017-04-10_09-44-32_462_4902211653847168915-1/-ext-10003
> INFO : Moving data to directory hdfs:/tmp/foo from
> hdfs://hacluster/user/hive/warehouse/.hive-staging_hive_2017-04-10_09-44-32_462_4902211653847168915-1/-ext-10001
> No rows affected (26.969 seconds)
> {code}
> 3. and the table location is hdfs://hacluster/tmp/foo :
> {code}
> 0: jdbc:hive2://189.39.151.44:21066/> desc formatted foo;
> +---+---+---+--+
> | col_name| data_type
> |comment|
> +---+---+---+--+
> | # col_name| data_type
> | comment |
> | | NULL
> | NULL |
> | id| int
> | |
> | | NULL
> | NULL |
> | # Detailed Table Information | NULL
> | NULL |
> | Database: | default
> | NULL |
> | Owner:| userx
> | NULL |
> | CreateTime: | Mon Apr 10 09:44:59 CST 2017
> | NULL |
> | LastAccessTime: | UNKNOWN
> | NULL |
> | Protect Mode: | None
> | NULL |
> | Retention:
[jira] [Updated] (HIVE-16413) Create table as select does not check ownership of the location
[
https://issues.apache.org/jira/browse/HIVE-16413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niklaus Xiao updated HIVE-16413:
Fix Version/s: 2.2.0
Status: Patch Available (was: Open)
> Create table as select does not check ownership of the location
> ---
>
> Key: HIVE-16413
> URL: https://issues.apache.org/jira/browse/HIVE-16413
> Project: Hive
> Issue Type: Bug
> Components: Authorization, SQLStandardAuthorization
>Affects Versions: 2.1.1, 1.2.2, 1.3.0
> Environment: hive-1.2.2, with following conf:
> hive.security.authorization.enabled: true
> hive.security.authorization.manager:
> org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactory
> hive.security.authenticator.manager:
> org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator
>Reporter: Niklaus Xiao
>Assignee: Niklaus Xiao
> Fix For: 2.2.0
>
> Attachments: HIVE-16413.patch
>
>
> 1. following statement failed:
> {code}
> create table foo(id int) location 'hdfs:///tmp/foo';
> Error: Error while compiling statement: FAILED: HiveAccessControlException
> Permission denied: Principal [name=userx, type=USER] does not have following
> privileges for operation CREATETABLE [[OBJECT OWNERSHIP] on Object
> [type=DFS_URI, name=hdfs://hacluster/tmp/foo]] (state=42000,code=4)
> {code}
> 2. but when use create table as select, it successed:
> {code}
> 0: jdbc:hive2://189.39.151.44:21066/> create table foo location
> 'hdfs:///tmp/foo' as select * from xxx2;
> INFO : Number of reduce tasks is set to 0 since there's no reduce operator
> INFO : number of splits:1
> INFO : Submitting tokens for job: job_1491449632882_0094
> INFO : Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:hacluster
> INFO : The url to track the job:
> https://189-39-151-44:26001/proxy/application_1491449632882_0094/
> INFO : Starting Job = job_1491449632882_0094, Tracking URL =
> https://189-39-151-44:26001/proxy/application_1491449632882_0094/
> INFO : Kill Command = /opt/hive-1.3.0/bin/..//../hadoop/bin/hadoop job
> -kill job_1491449632882_0094
> INFO : Hadoop job information for Stage-1: number of mappers: 1; number of
> reducers: 0
> INFO : 2017-04-10 09:44:49,185 Stage-1 map = 0%, reduce = 0%
> INFO : 2017-04-10 09:44:57,202 Stage-1 map = 100%, reduce = 0%, Cumulative
> CPU 1.98 sec
> INFO : MapReduce Total cumulative CPU time: 1 seconds 980 msec
> INFO : Ended Job = job_1491449632882_0094
> INFO : Stage-3 is selected by condition resolver.
> INFO : Stage-2 is filtered out by condition resolver.
> INFO : Stage-4 is filtered out by condition resolver.
> INFO : Moving data to directory
> hdfs://hacluster/user/hive/warehouse/.hive-staging_hive_2017-04-10_09-44-32_462_4902211653847168915-1/-ext-10001
> from
> hdfs://hacluster/user/hive/warehouse/.hive-staging_hive_2017-04-10_09-44-32_462_4902211653847168915-1/-ext-10003
> INFO : Moving data to directory hdfs:/tmp/foo from
> hdfs://hacluster/user/hive/warehouse/.hive-staging_hive_2017-04-10_09-44-32_462_4902211653847168915-1/-ext-10001
> No rows affected (26.969 seconds)
> {code}
> 3. and the table location is hdfs://hacluster/tmp/foo :
> {code}
> 0: jdbc:hive2://189.39.151.44:21066/> desc formatted foo;
> +---+---+---+--+
> | col_name| data_type
> |comment|
> +---+---+---+--+
> | # col_name| data_type
> | comment |
> | | NULL
> | NULL |
> | id| int
> | |
> | | NULL
> | NULL |
> | # Detailed Table Information | NULL
> | NULL |
> | Database: | default
> | NULL |
> | Owner:| userx
> | NULL |
> | CreateTime: | Mon Apr 10 09:44:59 CST 2017
> | NULL |
> | LastAccessTime: | UNKNOWN
> | NULL |
> | Protect Mode: | None
> | NULL
[jira] [Updated] (HIVE-16413) Create table as select does not check ownership of the location
[
https://issues.apache.org/jira/browse/HIVE-16413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niklaus Xiao updated HIVE-16413:
Description:
1. following statement failed:
{code}
create table foo(id int) location 'hdfs:///tmp/foo';
Error: Error while compiling statement: FAILED: HiveAccessControlException
Permission denied: Principal [name=userx, type=USER] does not have following
privileges for operation CREATETABLE [[OBJECT OWNERSHIP] on Object
[type=DFS_URI, name=hdfs://hacluster/tmp/foo]] (state=42000,code=4)
{code}
2. but when use create table as select, it successed:
{code}
0: jdbc:hive2://189.39.151.44:21066/> create table foo location
'hdfs:///tmp/foo' as select * from xxx2;
INFO : Number of reduce tasks is set to 0 since there's no reduce operator
INFO : number of splits:1
INFO : Submitting tokens for job: job_1491449632882_0094
INFO : Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:hacluster
INFO : The url to track the job:
https://189-39-151-44:26001/proxy/application_1491449632882_0094/
INFO : Starting Job = job_1491449632882_0094, Tracking URL =
https://189-39-151-44:26001/proxy/application_1491449632882_0094/
INFO : Kill Command = /opt/hive-1.3.0/bin/..//../hadoop/bin/hadoop job -kill
job_1491449632882_0094
INFO : Hadoop job information for Stage-1: number of mappers: 1; number of
reducers: 0
INFO : 2017-04-10 09:44:49,185 Stage-1 map = 0%, reduce = 0%
INFO : 2017-04-10 09:44:57,202 Stage-1 map = 100%, reduce = 0%, Cumulative
CPU 1.98 sec
INFO : MapReduce Total cumulative CPU time: 1 seconds 980 msec
INFO : Ended Job = job_1491449632882_0094
INFO : Stage-3 is selected by condition resolver.
INFO : Stage-2 is filtered out by condition resolver.
INFO : Stage-4 is filtered out by condition resolver.
INFO : Moving data to directory
hdfs://hacluster/user/hive/warehouse/.hive-staging_hive_2017-04-10_09-44-32_462_4902211653847168915-1/-ext-10001
from
hdfs://hacluster/user/hive/warehouse/.hive-staging_hive_2017-04-10_09-44-32_462_4902211653847168915-1/-ext-10003
INFO : Moving data to directory hdfs:/tmp/foo from
hdfs://hacluster/user/hive/warehouse/.hive-staging_hive_2017-04-10_09-44-32_462_4902211653847168915-1/-ext-10001
No rows affected (26.969 seconds)
{code}
3. and the table location is hdfs://hacluster/tmp/foo :
{code}
0: jdbc:hive2://189.39.151.44:21066/> desc formatted foo;
+---+---+---+--+
| col_name| data_type
|comment|
+---+---+---+--+
| # col_name| data_type
| comment |
| | NULL
| NULL |
| id| int
| |
| | NULL
| NULL |
| # Detailed Table Information | NULL
| NULL |
| Database: | default
| NULL |
| Owner:| userx
| NULL |
| CreateTime: | Mon Apr 10 09:44:59 CST 2017
| NULL |
| LastAccessTime: | UNKNOWN
| NULL |
| Protect Mode: | None
| NULL |
| Retention:| 0
| NULL |
| Location: | hdfs://hacluster/tmp/foo
| NULL |
| Table Type: | MANAGED_TABLE
| NULL |
| Table Parameters: | NULL
| NULL |
| | COLUMN_STATS_ACCURATE
| false |
| | numFiles
| 1 |
| | numRows
| -1|
| | rawDataSize
| -1|
| | totalSize
| 56|
|
[jira] [Assigned] (HIVE-16413) Create table as select does not check ownership of the location
[
https://issues.apache.org/jira/browse/HIVE-16413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niklaus Xiao reassigned HIVE-16413:
---
> Create table as select does not check ownership of the location
> ---
>
> Key: HIVE-16413
> URL: https://issues.apache.org/jira/browse/HIVE-16413
> Project: Hive
> Issue Type: Bug
> Components: Authorization, SQLStandardAuthorization
>Affects Versions: 2.1.1, 1.2.2, 1.3.0
> Environment: hive-1.2.2, with following conf:
> hive.security.authorization.enabled: true
> hive.security.authorization.manager:
> org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactory
> hive.security.authenticator.manager:
> org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator
>Reporter: Niklaus Xiao
>Assignee: Niklaus Xiao
>
> 1. following statement failed:
> {code}
> create table foo(id int) location 'hdfs:///tmp/foo';
> Error: Error while compiling statement: FAILED: HiveAccessControlException
> Permission denied: Principal [name=userx, type=USER] does not have following
> privileges for operation CREATETABLE [[OBJECT OWNERSHIP] on Object
> [type=DFS_URI, name=hdfs://hacluster/tmp/foo]] (state=42000,code=4)
> {code}
> 2. but when use create table as select, it successed:
> {code}
> 0: jdbc:hive2://189.39.151.44:21066/> create table foo location
> 'hdfs:///tmp/foo' as select * from xxx2;
> INFO : Number of reduce tasks is set to 0 since there's no reduce operator
> INFO : number of splits:1
> INFO : Submitting tokens for job: job_1491449632882_0094
> INFO : Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:hacluster
> INFO : The url to track the job:
> https://189-39-151-44:26001/proxy/application_1491449632882_0094/
> INFO : Starting Job = job_1491449632882_0094, Tracking URL =
> https://189-39-151-44:26001/proxy/application_1491449632882_0094/
> INFO : Kill Command =
> /opt/huawei/Bigdata/FusionInsight_V100R002C60U20/FusionInsight-Hive-1.3.0/hive-1.3.0/bin/..//../hadoop/bin/hadoop
> job -kill job_1491449632882_0094
> INFO : Hadoop job information for Stage-1: number of mappers: 1; number of
> reducers: 0
> INFO : 2017-04-10 09:44:49,185 Stage-1 map = 0%, reduce = 0%
> INFO : 2017-04-10 09:44:57,202 Stage-1 map = 100%, reduce = 0%, Cumulative
> CPU 1.98 sec
> INFO : MapReduce Total cumulative CPU time: 1 seconds 980 msec
> INFO : Ended Job = job_1491449632882_0094
> INFO : Stage-3 is selected by condition resolver.
> INFO : Stage-2 is filtered out by condition resolver.
> INFO : Stage-4 is filtered out by condition resolver.
> INFO : Moving data to directory
> hdfs://hacluster/user/hive/warehouse/.hive-staging_hive_2017-04-10_09-44-32_462_4902211653847168915-1/-ext-10001
> from
> hdfs://hacluster/user/hive/warehouse/.hive-staging_hive_2017-04-10_09-44-32_462_4902211653847168915-1/-ext-10003
> INFO : Moving data to directory hdfs:/tmp/foo from
> hdfs://hacluster/user/hive/warehouse/.hive-staging_hive_2017-04-10_09-44-32_462_4902211653847168915-1/-ext-10001
> No rows affected (26.969 seconds)
> {code}
> 3. and the table location is hdfs://hacluster/tmp/foo :
> {code}
> 0: jdbc:hive2://189.39.151.44:21066/> desc formatted foo;
> +---+---+---+--+
> | col_name| data_type
> |comment|
> +---+---+---+--+
> | # col_name| data_type
> | comment |
> | | NULL
> | NULL |
> | id| int
> | |
> | | NULL
> | NULL |
> | # Detailed Table Information | NULL
> | NULL |
> | Database: | default
> | NULL |
> | Owner:| userx
> | NULL |
> | CreateTime: | Mon Apr 10 09:44:59 CST 2017
> | NULL |
> | LastAccessTime: | UNKNOWN
> | NULL |
> | Protect Mode: | None
> | NULL |
> | Retention:| 0
