[jira] [Created] (HIVE-11546) Projected columns read size should be scaled to split size for ORC Splits
Prasanth Jayachandran created HIVE-11546: Summary: Projected columns read size should be scaled to split size for ORC Splits Key: HIVE-11546 URL: https://issues.apache.org/jira/browse/HIVE-11546 Project: Hive Issue Type: Bug Affects Versions: 1.3.0, 2.0.0 Reporter: Prasanth Jayachandran Assignee: Prasanth Jayachandran HIVE-10114 added projected columns data size information to OrcSplit which Tez Split Grouper can take advantage off (HIVE-7428). The annotated data size was for entire file and was not scaled to the split size. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Build failed in Jenkins: HIVE-TRUNK-JAVA8 #101
See http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/HIVE-TRUNK-JAVA8/101/changes Changes: [jcamacho] HIVE-11437: CBO: Calcite Operator To Hive Operator (Calcite Return Path) : dealing with insert into (Pengcheng Xiong, reviewed by Jesus Camacho Rodriguez) [amareshwari] HIVE-11376 : Removes legacy code wrt skipping files with codec to HiveInputFormat from CombineHiveInputFormat (Rajat Khandelwal, reviewed by Amareshwari) [gopalv] HIVE-11457: Vectorization: Improve GenVectorCode string equals intrinsic (Gopal V, reviewed by Matt McCline) [harisankar] HIVE-11387: CBO: Calcite Operator To Hive Operator (Calcite Return Path) : fix reduce_deduplicate optimization (Pengcheng Xiong, reviewed by Jesus Camacho Rodriguez, Hari Subramaniyan) [gunther] HIVE-11398: Parse wide OR and wide AND trees to flat OR/AND trees (Jesus Camacho Rodriguez via Gopal V) [dong1.chen] HIVE-11498: HIVE Authorization v2 should not check permission for dummy entity (Dapeng Sun via Dong Chen) [amareshwari] HIVE-11278 : Fix Partition.setOutputFormatClass to set class name properly (Rajat Khandelwal, reviewed by Amareshwari) [Alan Gates] HIVE-11511 Output the message of orcfiledump when ORC files are not specified (Shinichi Yamashita via gates) [Chao Sun] HIVE-11340 - Create ORC based table using like clause doesn't copy compression property (Yongzhi Chen, reviewed by Chao Sun) [pxiong] HIVE-11416: CBO: Calcite Operator To Hive Operator (Calcite Return Path): Groupby Optimizer assumes the schema can match after removing RS and GBY (reviewed by Jesus Camacho Rodriguez) [jdere] HIVE-11449: Capacity must be a power of two error when HybridHashTableContainer memory threshold is too low (Jason Dere, reviewed by Sergey Shelukhin) [pxiong] HIVE-11348: Support START TRANSACTION/COMMIT/ROLLBACK commands: support SQL2011 reserved keywords (Pengcheng Xiong reviewed by Eugene Koifman) [daijy] HIVE-11442: Remove commons-configuration.jar from Hive distribution [pxiong] HIVE-11480: CBO: Calcite Operator To Hive Operator (Calcite Return Path): char/varchar as input to GenericUDAF (Pengcheng Xiong, reviewed by Jesus Camacho Rodriguez) [gopalv] HIVE-11462: Constant fold struct() UDF (Gopal V, reviewed by Hari Sankar Sivarama Subramaniyan) -- [...truncated 325 lines...] A ptest2/src/main/java/org/apache/hive/ptest/execution/context/CloudComputeService.java Aptest2/src/main/java/org/apache/hive/ptest/execution/conf Aptest2/src/main/java/org/apache/hive/ptest/execution/conf/Context.java A ptest2/src/main/java/org/apache/hive/ptest/execution/conf/TestConfiguration.java Aptest2/src/main/java/org/apache/hive/ptest/execution/conf/TestParser.java Aptest2/src/main/java/org/apache/hive/ptest/execution/conf/Host.java A ptest2/src/main/java/org/apache/hive/ptest/execution/conf/ExecutionContextConfiguration.java Aptest2/src/main/java/org/apache/hive/ptest/execution/conf/TestBatch.java A ptest2/src/main/java/org/apache/hive/ptest/execution/conf/UnitTestBatch.java A ptest2/src/main/java/org/apache/hive/ptest/execution/conf/QFileTestBatch.java Aptest2/src/main/java/org/apache/hive/ptest/execution/JIRAService.java Aptest2/src/main/java/org/apache/hive/ptest/execution/Drone.java Aptest2/src/main/java/org/apache/hive/ptest/api Aptest2/src/main/java/org/apache/hive/ptest/api/client Aptest2/src/main/java/org/apache/hive/ptest/api/client/PTestClient.java Aptest2/src/main/java/org/apache/hive/ptest/api/response A ptest2/src/main/java/org/apache/hive/ptest/api/response/TestLogResponse.java A ptest2/src/main/java/org/apache/hive/ptest/api/response/TestStatusResponse.java Aptest2/src/main/java/org/apache/hive/ptest/api/response/TestStatus.java A ptest2/src/main/java/org/apache/hive/ptest/api/response/TestStopResponse.java A ptest2/src/main/java/org/apache/hive/ptest/api/response/TestListResponse.java A ptest2/src/main/java/org/apache/hive/ptest/api/response/GenericResponse.java A ptest2/src/main/java/org/apache/hive/ptest/api/response/TestStartResponse.java Aptest2/src/main/java/org/apache/hive/ptest/api/Status.java Aptest2/src/main/java/org/apache/hive/ptest/api/server A ptest2/src/main/java/org/apache/hive/ptest/api/server/ExecutionController.java Aptest2/src/main/java/org/apache/hive/ptest/api/server/TestExecutor.java Aptest2/src/main/java/org/apache/hive/ptest/api/server/Test.java Aptest2/src/main/java/org/apache/hive/ptest/api/server/TestLogger.java Aptest2/src/main/java/org/apache/hive/ptest/api/request Aptest2/src/main/java/org/apache/hive/ptest/api/request/TestListRequest.java A ptest2/src/main/java/org/apache/hive/ptest/api/request/TestStartRequest.java Aptest2/src/main/java/org/apache/hive/ptest/api/request/TestLogRequest.java A ptest2/src/main/java/org/apache/hive/ptest/api/request/TestStatusRequest.java A
Re: [DISCUSS] Hive and HBase dependency
On the hbase-metastore branch I've actually already moved to HBase 1.1. I'm +1 for moving to 1.1 or 1.0 on master and staying at 0.98 on branch-1. Alan. kulkarni.swar...@gmail.com mailto:kulkarni.swar...@gmail.com August 12, 2015 at 8:43 Hi all, It seems like our current dependency on HBase is a little fuzzy to say the least. And with increased features relying on HBase(HBase integration, HBase metastore etc), I think it would be worth giving a thought into how we want to manage this dependency. I have also seen regressions[1][2] come up recently due to this dependency not managed properly. Plus we need to think about moving to HBase 1.0 soon as well to take advantage of the backwards compatibility guarantees that HBase is providing. Our current HBase dependency is 0.98.9. Also with out current bifurcation of branches to create a 1.x branch for stability and 2.x for bleeding edge, I propose that we still keep the version to 0.98.9 on the 1.x branch and move to HBase 1.0 in our 2.0 branch. In that way we can start taking advantage of the latest updates to the HBase API in our 2.x branch and still keep 1.x backwards compatible by avoiding a direct jump to HBase 1.0. If we decide to go this route, we might need to revert back some of the compatibility breaking changes[2] that sneaked into 1.x and move them over to 2.x. Thoughts? Thanks, Swarnim [1] https://issues.apache.org/jira/browse/HIVE-10990 [2] https://issues.apache.org/jira/browse/HIVE-8898
Hive-0.14 - Build # 1042 - Still Failing
Changes for Build #1021 Changes for Build #1022 Changes for Build #1023 Changes for Build #1024 Changes for Build #1025 Changes for Build #1026 Changes for Build #1027 Changes for Build #1028 Changes for Build #1029 Changes for Build #1030 Changes for Build #1031 Changes for Build #1032 Changes for Build #1033 Changes for Build #1034 Changes for Build #1035 Changes for Build #1036 Changes for Build #1037 Changes for Build #1038 Changes for Build #1039 Changes for Build #1040 Changes for Build #1041 Changes for Build #1042 No tests ran. The Apache Jenkins build system has built Hive-0.14 (build #1042) Status: Still Failing Check console output at https://builds.apache.org/job/Hive-0.14/1042/ to view the results.
[jira] [Created] (HIVE-11548) HCatLoader should support predicate pushdown.
Mithun Radhakrishnan created HIVE-11548:
---
Summary: HCatLoader should support predicate pushdown.
Key: HIVE-11548
URL: https://issues.apache.org/jira/browse/HIVE-11548
Project: Hive
Issue Type: New Feature
Components: HCatalog
Reporter: Mithun Radhakrishnan
Assignee: Mithun Radhakrishnan
When one uses {{HCatInputFormat}}/{{HCatLoader}} to read from file-formats that
support predicate pushdown (such as ORC, with
{{hive.optimize.index.filter=true}}), one sees that the predicates aren't
actually pushed down into the storage layer.
The forthcoming patch should allow for filter-pushdown, if any of the
partitions being scanned with {{HCatLoader}} support the functionality. The
patch should technically allow the same for users of {{HCatInputFormat}}, but I
don't currently have a neat interface to build a compound predicate-expression.
Will add this separately, if required.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
too many 1.*.* unreleased versions on the JIRA
On the JIRA, we currently have 1.1.0 marked as unreleased even though 1.2.0 is released (and 1.1.1 is also present); then, we have both 1.0.1 and 1.0.2, plus 1.2.1 and 1.2.2 showing in unreleased. I poked around and cannot see where this can be changed. Release managers for respective releases should probably clean this up, anyway :)
[jira] [Created] (HIVE-11552) implement basic methods for getting/putting file metadata
Sergey Shelukhin created HIVE-11552: --- Summary: implement basic methods for getting/putting file metadata Key: HIVE-11552 URL: https://issues.apache.org/jira/browse/HIVE-11552 Project: Hive Issue Type: Bug Affects Versions: hbase-metastore-branch Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-11554) Exchange partition outputs missing from post execute hooks
Paul Yang created HIVE-11554:
Summary: Exchange partition outputs missing from post execute hooks
Key: HIVE-11554
URL: https://issues.apache.org/jira/browse/HIVE-11554
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 1.2.0, 1.0.0, 0.14.0, 0.13.0, 0.12.0
Reporter: Paul Yang
The pre/post execute hook interface has fields that indicate which Hive objects
were read / written to as a result of running the query. For the exchange
partition operation, these fields (ReadEntity and WriteEntity) are empty.
This is an important issue as the hook interface may be configured to perform
critical warehouse operations.
See
{noformat}
ql/src/test/results/clientpositive/exchange_partition3.q.out
{noformat}
{noformat}
POSTHOOK: query: -- This will exchange both partitions hr=1 and hr=2
ALTER TABLE exchange_part_test1 EXCHANGE PARTITION (ds='2013-04-05') WITH TABLE
exchange_part_test2
POSTHOOK: type: null
{noformat}
The post hook should not say null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-11553) use basic file metadata cache in ETLSplitStrategy-related paths
Sergey Shelukhin created HIVE-11553: --- Summary: use basic file metadata cache in ETLSplitStrategy-related paths Key: HIVE-11553 URL: https://issues.apache.org/jira/browse/HIVE-11553 Project: Hive Issue Type: Bug Affects Versions: hbase-metastore-branch Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-11551) Branch 1.0 : test cases regression after cherry-pick HIVE-9397
Pengcheng Xiong created HIVE-11551: -- Summary: Branch 1.0 : test cases regression after cherry-pick HIVE-9397 Key: HIVE-11551 URL: https://issues.apache.org/jira/browse/HIVE-11551 Project: Hive Issue Type: Bug Reporter: Pengcheng Xiong Assignee: Pengcheng Xiong insert_into1.q,stats_only_null.q,metadata_only_queries_with_filters.q,insert_into2.q,metadata_only_queries.q -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-11549) Hide Hive configuration from spark driver launching process
Xuefu Zhang created HIVE-11549: -- Summary: Hide Hive configuration from spark driver launching process Key: HIVE-11549 URL: https://issues.apache.org/jira/browse/HIVE-11549 Project: Hive Issue Type: Bug Components: Spark Affects Versions: 1.2.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang Hive uses Spark application submission script, spark-submit, to launch remote spark driver. Starting from Spark 1.4, this script also does a lot of things that Hive doesn't need, for instance, accessing metastore for delegation tokens. Hive on Spark doesn't need this, and one way to do this is hide Hive configuration from being visible by that script. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-11550) ACID queries pollute HiveConf
Eugene Koifman created HIVE-11550: - Summary: ACID queries pollute HiveConf Key: HIVE-11550 URL: https://issues.apache.org/jira/browse/HIVE-11550 Project: Hive Issue Type: Bug Components: Transactions Affects Versions: 1.0.0 Reporter: Eugene Koifman Assignee: Eugene Koifman HiveConf is a SessionState level object. Some ACID related logic makes changes to it (which are meant to be per query) but become permanent. See SemanticAnalyzer.checkAcidConstraints() [~alangates], do you know of other cases or ideas on how to deal with this differently? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-11547) beeline does not continue running the script after an error occurs while beeline --force=true is already set.
Wei Huang created HIVE-11547: Summary: beeline does not continue running the script after an error occurs while beeline --force=true is already set. Key: HIVE-11547 URL: https://issues.apache.org/jira/browse/HIVE-11547 Project: Hive Issue Type: Bug Components: Beeline Affects Versions: 1.2.0 Environment: HDP 2.3 on Virtual box Reporter: Wei Huang If you execute beeline to run a SQL script file, using the following command beeline -f query file name the beeline exists after the first error. i.e. when a test query fails beeline quits to the CLI. The beeline --force=true seems to have a bug and it does not continue running the script after an error occurs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
