Repository: hive Updated Branches: refs/heads/branch-1.2 915171b4b -> d69dcadd6
http://git-wip-us.apache.org/repos/asf/hive/blob/d69dcadd/RELEASE_NOTES.txt ---------------------------------------------------------------------- diff --git a/RELEASE_NOTES.txt b/RELEASE_NOTES.txt index feff103..f8ce137 100644 --- a/RELEASE_NOTES.txt +++ b/RELEASE_NOTES.txt @@ -1,5573 +1,62 @@ - -Release Notes - Hive - Version 1.2.1 - -** Sub-task - * [HIVE-10629] - Dropping table in an encrypted zone does not drop warehouse directory - * [HIVE-10630] - Renaming tables across encryption zones renames table even though the operation throws error - * [HIVE-10658] - Insert with values clause may expose data that should be encrypted - * [HIVE-10747] - Enable the cleanup of side effect for the Encryption related qfile test - * [HIVE-10910] - Alter table drop partition queries in encrypted zone failing to remove data from HDFS - - -** Bug - * [HIVE-4577] - hive CLI can't handle hadoop dfs command with space and quotes. - * [HIVE-8931] - Test TestAccumuloCliDriver is not completing - * [HIVE-9069] - Simplify filter predicates for CBO - * [HIVE-9828] - Semantic analyzer does not capture view parent entity for tables referred in view with union all - * [HIVE-9842] - Enable session/operation timeout by default in HiveServer2 - * [HIVE-10107] - Union All : Vertex missing stats resulting in OOM and in-efficient plans - * [HIVE-10244] - Vectorization : TPC-DS Q80 fails with java.lang.ClassCastException when hive.vectorized.execution.reduce.enabled is enabled - * [HIVE-10453] - HS2 leaking open file descriptors when using UDFs - * [HIVE-10528] - Hiveserver2 in HTTP mode is not applying auth_to_local rules - * [HIVE-10563] - MiniTezCliDriver tests ordering issues - * [HIVE-10605] - Make hive version number update automatically in webhcat-default.xml during hive tar generation - * [HIVE-10606] - Divide by zero error in HybridHashTableContainer - * [HIVE-10627] - Queries fail with Failed to breakup Windowing invocations into Groups - * [HIVE-10628] - Incorrect result when vectorized native mapjoin is enabled using null safe operators <=> - * [HIVE-10659] - Beeline command which contains semi-colon as a non-command terminator will fail - * [HIVE-10664] - Unit tests run fail in windows because of illegal escape character in file path - * [HIVE-10672] - Analyze command on a table using row format serde JsonSerDe fails with NoClassDefFoundError - * [HIVE-10674] - jars should not be checked in to the source control repo - * [HIVE-10675] - Provide option to skip Accumulo related Hive tests in itests directory - * [HIVE-10677] - hive.exec.parallel=true has problem when it is used for analyze table column stats - * [HIVE-10678] - update sql standard authorization configuration whitelist - more optimization flags - * [HIVE-10679] - JsonSerde ignores varchar and char size limit specified during table creation - * [HIVE-10684] - Fix the unit test failures for HIVE-7553 after HIVE-10674 removed the binary jar files - * [HIVE-10685] - Alter table concatenate oparetor will cause duplicate data - * [HIVE-10686] - java.lang.IndexOutOfBoundsException for query with rank() over(partition ...) - * [HIVE-10688] - constant folding is broken for case-when udf - * [HIVE-10689] - HS2 metadata api calls should use HiveAuthorizer interface for authorization - * [HIVE-10690] - ArrayIndexOutOfBounds exception in MetaStoreDirectSql.aggrColStatsForPartitions() - * [HIVE-10696] - TestAddResource tests are non-portable - * [HIVE-10704] - Errors in Tez HashTableLoader when estimated table size is 0 - * [HIVE-10711] - Tez HashTableLoader attempts to allocate more memory than available when HIVECONVERTJOINNOCONDITIONALTASKTHRESHOLD exceeds process max mem - * [HIVE-10719] - Hive metastore failure when alter table rename is attempted. - * [HIVE-10724] - WebHCat e2e test TestStreaming_5 fails on Windows - * [HIVE-10727] - Import throws error message "org.apache.thrift.protocol.TProtocolException: Required field 'filesAdded' is unset!" - * [HIVE-10735] - Cached plan race condition - VectorMapJoinCommonOperator has no closeOp() - * [HIVE-10736] - HiveServer2 shutdown of cached tez app-masters is not clean - * [HIVE-10741] - count distinct rewrite is not firing - * [HIVE-10745] - Better null handling by Vectorizer - * [HIVE-10746] - Hive 1.2.0+Tez produces 1-byte FileSplits from mapred.TextInputFormat - * [HIVE-10753] - hs2 jdbc url - wrong connection string cause error on beeline/jdbc/odbc client, misleading message - * [HIVE-10760] - Templeton: HCatalog Get Column for Non-existent column returns Server Error (500) rather than Not Found(404) - * [HIVE-10768] - In QTestGenTask.execute() we should not throw an exception right away if we are unable to clean any old files - * [HIVE-10776] - Schema on insert for bucketed tables throwing NullPointerException - * [HIVE-10789] - union distinct query with NULL constant on both the sides throws "Unsuported vector output type: void" error - * [HIVE-10792] - PPD leads to wrong answer when mapper scans the same table with multiple aliases - * [HIVE-10801] - 'drop view' fails throwing java.lang.NullPointerException - * [HIVE-10805] - OOM in vectorized reduce - * [HIVE-10811] - RelFieldTrimmer throws NoSuchElementException in some cases - * [HIVE-10813] - Fix current test failures after HIVE-8769 - * [HIVE-10819] - SearchArgumentImpl for Timestamp is broken by HIVE-10286 - * [HIVE-10828] - Insert with schema and dynamic partitions NullPointerException - * [HIVE-10829] - ATS hook fails for explainTask - * [HIVE-10839] - TestHCatLoaderEncryption.* tests fail in windows because of path related issues - * [HIVE-10840] - NumberFormatException while running analyze table partition compute statics query - * [HIVE-10841] - [WHERE col is not null] does not work sometimes for queries with many JOIN statements - * [HIVE-10843] - desc database and show tables commands don't pass db to HiveAuthorizer check - * [HIVE-10845] - TezJobMonitor uses killedTaskCount instead of killedTaskAttemptCount - * [HIVE-10857] - Accumulo storage handler fail throwing java.lang.IllegalArgumentException: Cannot determine SASL mechanism for token class: class org.apache.accumulo.core.client.security.tokens.PasswordToken - * [HIVE-10858] - WebHCat specific resources should be added to HADOOP_CLASSPATH first - * [HIVE-10862] - TestHiveAuthorizerShowFilters tests fail when run in sequence - * [HIVE-10874] - Fail in TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx_cbo_2.q due to duplicate column name - * [HIVE-10875] - Select query with view in subquery adds underlying table as direct input - * [HIVE-10877] - TestUtil class name confuses ptest2 - * [HIVE-10885] - with vectorization enabled join operation involving interval_day_time fails - * [HIVE-10887] - TestCliDriver tests ordering issues with Mac and CentOS - * [HIVE-10892] - TestHCatClient should not accept external metastore param from -Dhive.metastore.uris - * [HIVE-10907] - Hive on Tez: Classcast exception in some cases with SMB joins - * [HIVE-10919] - Windows: create table with JsonSerDe failed via beeline unless you add hcatalog core jar to classpath - * [HIVE-10922] - In HS2 doAs=false mode, file system related errors in one query causes other failures - * [HIVE-10925] - Non-static threadlocals in metastore code can potentially cause memory leak - * [HIVE-10929] - In Tez mode,dynamic partitioning query with union all fails at moveTask,Invalid partition key & values - * [HIVE-10934] - Restore support for DROP PARTITION PURGE - * [HIVE-10941] - Provide option to disable spark tests outside itests - * [HIVE-10949] - Disable hive-minikdc tests in Windows - * [HIVE-10957] - QueryPlan's start time is incorrect in certain cases - * [HIVE-10958] - Centos: TestMiniTezCliDriver.testCliDriver_mergejoin fails - * [HIVE-10959] - webhcat launcher job should reconnect to the running child job on task retry - * [HIVE-10965] - direct SQL for stats fails in 0-column case - * [HIVE-10966] - direct SQL for stats has a cast exception on some databases - * [HIVE-10967] - add mapreduce.job.tags to sql std authorization config whitelist - * [HIVE-10968] - Windows: analyze json table via beeline failed throwing Class org.apache.hive.hcatalog.data.JsonSerDe not found - * [HIVE-10971] - count(*) with count(distinct) gives wrong results when hive.groupby.skewindata=true - * [HIVE-10992] - WebHCat should not create delegation tokens when Kerberos is not enabled - * [HIVE-10994] - Hive.moveFile should not fail on a no-op move - * [HIVE-11001] - HS2 http cookie mode does not honor doAs url parameter - * [HIVE-11006] - improve logging wrt ACID module - * [HIVE-11010] - Accumulo storage handler queries via HS2 fail - * [HIVE-11023] - Disable directSQL if datanucleus.identifierFactory = datanucleus2 - * [HIVE-11027] - Hive on tez: Bucket map joins fail when hashcode goes negative - * [HIVE-11031] - ORC concatenation of old files can fail while merging column statistics - * [HIVE-11033] - BloomFilter index is not honored by ORC reader - * [HIVE-11035] - PPD: Orc Split elimination fails because filterColumns=[-1] - * [HIVE-11040] - Change Derby dependency version to 10.10.2.0 - * [HIVE-11041] - Update tests for HIVE-9302 after removing binaries - * [HIVE-11047] - Update versions of branch-1.2 to 1.2.1 - - -** Improvement - * [HIVE-8769] - Physical optimizer : Incorrect CE results in a shuffle join instead of a Map join (PK/FK pattern not detected) - * [HIVE-10812] - Scaling PK/FK's selectivity for stats annotation - * [HIVE-10900] - Fix the indeterministic stats for some hive queries - - -** New Feature - * [HIVE-10636] - CASE comparison operator rotation optimization - * [HIVE-10716] - Fold case/when udf for expression involving nulls in filter operator. - - -** Task - * [HIVE-10327] - Remove ExprNodeNullDesc - * [HIVE-11018] - Turn on cbo in more q files - - -** Test - * [HIVE-10869] - fold_case.q failing on trunk - * [HIVE-10909] - Make TestFilterHooks robust - * [HIVE-10939] - Make TestFileDump robust - * [HIVE-10955] - CliDriver leaves tables behind at end of test run - * [HIVE-11026] - Make vector_outer_join* test more robust - - - -Release Notes - Hive - Version 1.2.0 - -** Sub-task - * [HIVE-8119] - Implement Date in ParquetSerde - * [HIVE-8164] - Adding in a ReplicationTask that converts a Notification Event to actionable tasks - * [HIVE-8165] - Annotation changes for replication - * [HIVE-8379] - NanoTimeUtils performs some work needlessly - * [HIVE-8696] - HCatClientHMSImpl doesn't use a Retrying-HiveMetastoreClient. - * [HIVE-8817] - Create unit test where we insert into an encrypted table and then read from it with pig - * [HIVE-8818] - Create unit test where we insert into an encrypted table and then read from it with hcatalog mapreduce - * [HIVE-9009] - order by (limit) meaning for the last subquery of union in Hive is different from other main stream RDBMS - * [HIVE-9253] - MetaStore server should support timeout for long running requests - * [HIVE-9271] - Add ability for client to request metastore to fire an event - * [HIVE-9273] - Add option to fire metastore event on insert - * [HIVE-9327] - CBO (Calcite Return Path): Removing Row Resolvers from ParseContext - * [HIVE-9333] - Move parquet serialize implementation to DataWritableWriter to improve write speeds - * [HIVE-9432] - CBO (Calcite Return Path): Removing QB from ParseContext - * [HIVE-9501] - DbNotificationListener doesn't include dbname in create database notification and does not include tablename in create table notification - * [HIVE-9508] - MetaStore client socket connection should have a lifetime - * [HIVE-9550] - ObjectStore.getNextNotification() can return events inside NotificationEventResponse as null which conflicts with its thrift "required" tag - * [HIVE-9558] - [Parquet] support HiveDecimalWritable, HiveCharWritable, HiveVarcharWritable in vectorized mode - * [HIVE-9563] - CBO(Calcite Return Path): Translate GB to Hive OP [CBO branch] - * [HIVE-9571] - CBO (Calcite Return Path): Generate FileSink Op [CBO branch] - * [HIVE-9582] - HCatalog should use IMetaStoreClient interface - * [HIVE-9585] - AlterPartitionMessage should return getKeyValues instead of getValues - * [HIVE-9657] - Use new parquet Types API builder to construct data types - * [HIVE-9666] - Improve some qtests - * [HIVE-9690] - Refactoring for non-numeric arithmetic operations - * [HIVE-9750] - avoid log locks in operators - * [HIVE-9792] - Support interval type in expressions/predicates - * [HIVE-9810] - prep object registry for multi threading - * [HIVE-9819] - Add timeout check inside the HMS server - * [HIVE-9824] - LLAP: Native Vectorization of Map Join - * [HIVE-9894] - Use new parquet Types API builder to construct DATE data type - * [HIVE-9906] - Add timeout mechanism in RawStoreProxy - * [HIVE-9937] - LLAP: Vectorized Field-By-Field Serialize / Deserialize to support new Vectorized Map Join - * [HIVE-9982] - CBO (Calcite Return Path): Prune TS Relnode schema - * [HIVE-9998] - Vectorization support for interval types - * [HIVE-10037] - JDBC support for interval expressions - * [HIVE-10044] - Allow interval params for year/month/day/hour/minute/second functions - * [HIVE-10053] - Override new init API fom ReadSupport instead of the deprecated one - * [HIVE-10071] - CBO (Calcite Return Path): Join to MultiJoin rule - * [HIVE-10076] - Bump up parquet-hadoop-bundle and parquet-column to the version of 1.6.0rc6 - * [HIVE-10131] - LLAP: BytesBytesMultiHashMap and mapjoin container should reuse refs - * [HIVE-10227] - Concrete implementation of Export/Import based ReplicationTaskFactory - * [HIVE-10228] - Changes to Hive Export/Import/DropTable/DropPartition to support replication semantics - * [HIVE-10243] - CBO (Calcite Return Path): Introduce JoinAlgorithm Interface - * [HIVE-10252] - Make PPD work for Parquet in row group level - * [HIVE-10262] - CBO (Calcite Return Path): Temporarily disable Aggregate check input for bucketing - * [HIVE-10263] - CBO (Calcite Return Path): Aggregate checking input for bucketing should be conditional - * [HIVE-10326] - CBO (Calcite Return Path): Invoke Hive's Cumulative Cost - * [HIVE-10329] - Hadoop reflectionutils has issues - * [HIVE-10343] - CBO (Calcite Return Path): Parameterize algorithm cost model - * [HIVE-10347] - Merge spark to trunk 4/15/2015 - * [HIVE-10350] - CBO: Use total size instead of bucket count to determine number of splits & parallelism - * [HIVE-10369] - CBO: Don't use HiveDefaultCostModel when With Tez and hive.cbo.costmodel.extended enabled - * [HIVE-10375] - CBO (Calcite Return Path): disable the identity project remover for some union operators - * [HIVE-10386] - CBO (Calcite Return Path): Disable Trivial Project Removal on ret path - * [HIVE-10391] - CBO (Calcite Return Path): HiveOpConverter always assumes that HiveFilter does not include a partition column - * [HIVE-10400] - CBO (Calcite Return Path): Exception when column name contains dot or colon characters - * [HIVE-10413] - [CBO] Return path assumes distinct column cant be same as grouping column - * [HIVE-10416] - CBO (Calcite Return Path): Fix return columns if Sort operator is on top of plan returned by Calcite - * [HIVE-10426] - Rework/simplify ReplicationTaskFactory instantiation - * [HIVE-10455] - CBO (Calcite Return Path): Different data types at Reducer before JoinOp - * [HIVE-10462] - CBO (Calcite Return Path): MapJoin and SMBJoin conversion not triggered - * [HIVE-10493] - Merge multiple joins when join keys are the same - * [HIVE-10506] - CBO (Calcite Return Path): Disallow return path to be enable if CBO is off - * [HIVE-10512] - CBO (Calcite Return Path): SMBJoin conversion throws ClassCastException - * [HIVE-10520] - LLAP: Must reset small table result columns for Native Vectorization of Map Join - * [HIVE-10522] - CBO (Calcite Return Path): fix the wrong needed column names when TS is created - * [HIVE-10526] - CBO (Calcite Return Path): HiveCost epsilon comparison should take row count in to account - * [HIVE-10547] - CBO (Calcite Return Path) : genFileSinkPlan uses wrong partition col to create FS - * [HIVE-10549] - CBO (Calcite Return Path): Enable NonBlockingOpDeDupProc - - -** Bug - * [HIVE-3454] - Problem with CAST(BIGINT as TIMESTAMP) - * [HIVE-4625] - HS2 should not attempt to get delegation token from metastore if using embedded metastore - * [HIVE-5545] - HCatRecord getInteger method returns String when used on Partition columns of type INT - * [HIVE-5672] - Insert with custom separator not supported for non-local directory - * [HIVE-6069] - Improve error message in GenericUDFRound - * [HIVE-6099] - Multi insert does not work properly with distinct count - * [HIVE-6950] - Parsing Error in GROUPING SETS - * [HIVE-7351] - ANALYZE TABLE statement fails on postgres metastore - * [HIVE-8524] - When table is renamed stats are lost as changes are not propagated to metastore tables TAB_COL_STATS and PART_COL_STATS - * [HIVE-8626] - Extend HDFS super-user checks to dropPartitions - * [HIVE-8746] - ORC timestamp columns are sensitive to daylight savings time - * [HIVE-8890] - HiveServer2 dynamic service discovery: use persistent ephemeral nodes curator recipe - * [HIVE-8915] - Log file explosion due to non-existence of COMPACTION_QUEUE table - * [HIVE-9002] - union all does not generate correct result for order by and limit - * [HIVE-9023] - HiveHistoryImpl relies on removed counters to print num rows - * [HIVE-9073] - NPE when using custom windowing UDAFs - * [HIVE-9083] - New metastore API to support to purge partition-data directly in dropPartitions(). - * [HIVE-9086] - Add language support to PURGE data while dropping partitions. - * [HIVE-9115] - Hive build failure on hadoop-2.7 due to HADOOP-11356 - * [HIVE-9118] - Support auto-purge for tables, when dropping tables/partitions. - * [HIVE-9151] - Checking s against null in TezJobMonitor#getNameWithProgress() should be done earlier - * [HIVE-9228] - Problem with subquery using windowing functions - * [HIVE-9303] - Parquet files are written with incorrect definition levels - * [HIVE-9322] - Make null-checks consistent for MapObjectInspector subclasses. - * [HIVE-9350] - Add ability for HiveAuthorizer implementations to filter out results of 'show tables', 'show databases' - * [HIVE-9397] - SELECT max(bar) FROM foo is broken after ANALYZE ... FOR COLUMNS - * [HIVE-9430] - NullPointerException on ALTER TABLE ADD PARTITION if no value given - * [HIVE-9438] - The standalone-jdbc jar missing some jars - * [HIVE-9456] - Make Hive support unicode with MSSQL as Metastore backend - * [HIVE-9468] - Test groupby3_map_skew.q fails due to decimal precision difference - * [HIVE-9471] - Bad seek in uncompressed ORC, at row-group boundary. - * [HIVE-9472] - Implement 7 simple UDFs added to Hive - * [HIVE-9474] - truncate table changes permissions on the target - * [HIVE-9481] - allow column list specification in INSERT statement - * [HIVE-9482] - Hive parquet timestamp compatibility - * [HIVE-9484] - ThriftCLIService#getDelegationToken does case sensitive comparison - * [HIVE-9486] - Use session classloader instead of application loader - * [HIVE-9489] - add javadoc for UDFType annotation - * [HIVE-9496] - Slf4j warning in hive command - * [HIVE-9507] - Make "LATERAL VIEW inline(expression) mytable" tolerant to nulls - * [HIVE-9509] - Restore partition spec validation removed by HIVE-9445 - * [HIVE-9512] - HIVE-9327 causing regression in stats annotation - * [HIVE-9513] - NULL POINTER EXCEPTION - * [HIVE-9526] - ClassCastException thrown by HiveStatement - * [HIVE-9529] - "alter table .. concatenate" under Tez mode should create TezTask - * [HIVE-9539] - Wrong check of version format in TestWebHCatE2e.getHiveVersion() - * [HIVE-9553] - Fix log-line in Partition Pruner - * [HIVE-9555] - assorted ORC refactorings for LLAP on trunk - * [HIVE-9560] - When hive.stats.collect.rawdatasize=true, 'rawDataSize' for an ORC table will result in value '0' after running 'analyze table TABLE_NAME compute statistics;' - * [HIVE-9565] - Minor cleanup in TestMetastoreExpr. - * [HIVE-9567] - JSON SerDe not escaping special chars when writing char/varchar data - * [HIVE-9580] - Server returns incorrect result from JOIN ON VARCHAR columns - * [HIVE-9587] - UDF decode should accept STRING_GROUP types for the second parameter - * [HIVE-9588] - Reimplement HCatClientHMSImpl.dropPartitions() with HMSC.dropPartitions() - * [HIVE-9592] - fix ArrayIndexOutOfBoundsException in date_add and date_sub initialize - * [HIVE-9609] - AddPartitionMessage.getPartitions() can return null - * [HIVE-9612] - Turn off DEBUG logging for Lazy Objects for tests - * [HIVE-9613] - Left join query plan outputs wrong column when using subquery - * [HIVE-9617] - UDF from_utc_timestamp throws NPE if the second argument is null - * [HIVE-9619] - Uninitialized read of numBitVectors in NumDistinctValueEstimator - * [HIVE-9620] - Cannot retrieve column statistics using HMS API if column name contains uppercase characters - * [HIVE-9622] - Getting NPE when trying to restart HS2 when metastore is configured to use org.apache.hadoop.hive.thrift.DBTokenStore - * [HIVE-9623] - NullPointerException in MapJoinOperator.processOp(MapJoinOperator.java:253) for TPC-DS Q75 against un-partitioned schema - * [HIVE-9624] - NullPointerException in MapJoinOperator.processOp(MapJoinOperator.java:253) for TPC-DS Q75 against un-partitioned schema - * [HIVE-9628] - HiveMetaStoreClient.dropPartitions(...List<ObjectPair<Integer,byte[]>>...) doesn't take (boolean needResult) - * [HIVE-9633] - Add HCatClient.dropPartitions() overload to skip deletion of partition-directories. - * [HIVE-9644] - Fold case & when udfs - * [HIVE-9645] - Constant folding case NULL equality - * [HIVE-9647] - Discrepancy in cardinality estimates between partitioned and un-partitioned tables - * [HIVE-9648] - Null check key provider before doing set - * [HIVE-9652] - Tez in place updates should detect redirection of STDERR - * [HIVE-9655] - Dynamic partition table insertion error - * [HIVE-9665] - Parallel move task optimization causes race condition - * [HIVE-9667] - Disable ORC bloom filters for ORC v11 output-format - * [HIVE-9674] - *DropPartitionEvent should handle partition-sets. - * [HIVE-9679] - Remove redundant null-checks from DbNotificationListener. - * [HIVE-9680] - GlobalLimitOptimizer is not checking filters correctly - * [HIVE-9681] - Extend HiveAuthorizationProvider to support partition-sets. - * [HIVE-9684] - Incorrect disk range computation in ORC because of optional stream kind - * [HIVE-9706] - HBase handler support for snapshots should confirm properties before use - * [HIVE-9711] - ORC Vectorization DoubleColumnVector.isRepeating=false if all entries are NaN - * [HIVE-9716] - Map job fails when table's LOCATION does not have scheme - * [HIVE-9717] - The max/min function used by AggrStats for decimal type is not what we expected - * [HIVE-9720] - Metastore does not properly migrate column stats when renaming a table across databases. - * [HIVE-9721] - Hadoop23Shims.setFullFileStatus should check for null - * [HIVE-9727] - GroupingID translation from Calcite - * [HIVE-9731] - WebHCat MapReduce Streaming Job does not allow StreamXmlRecordReader to be specified - * [HIVE-9734] - Correlating expression cannot contain unqualified column references - * [HIVE-9735] - aggregate ( smalllint ) fails when ORC file used ava.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.Short - * [HIVE-9743] - Incorrect result set for vectorized left outer join - * [HIVE-9749] - ObjectStore schema verification logic is incorrect - * [HIVE-9754] - rename GenericUDFLevenstein to GenericUDFLevenshtein - * [HIVE-9755] - Hive built-in "ngram" UDAF fails when a mapper has no matches. - * [HIVE-9767] - Fixes in Hive UDF to be usable in Pig - * [HIVE-9770] - Beeline ignores --showHeader for non-tablular output formats i.e csv,tsv,dsv - * [HIVE-9772] - Hive parquet timestamp conversion doesn't work with new Parquet - * [HIVE-9779] - ATSHook does not log the end user if doAs=false (it logs the hs2 server user) - * [HIVE-9791] - insert into table throws NPE - * [HIVE-9797] - Need update some spark tests for java 8 - * [HIVE-9813] - Hive JDBC - DatabaseMetaData.getColumns method cannot find classes added with "add jar" command - * [HIVE-9817] - fix DateFormat pattern in hive-exec - * [HIVE-9826] - Firing insert event fails on temporary table - * [HIVE-9831] - HiveServer2 should use ConcurrentHashMap in ThreadFactory - * [HIVE-9832] - Merge join followed by union and a map join in hive on tez fails. - * [HIVE-9834] - VectorGroupByOperator logs too much - * [HIVE-9836] - Hive on tez: fails when virtual columns are present in the join conditions (for e.g. partition columns) - * [HIVE-9839] - HiveServer2 leaks OperationHandle on async queries which fail at compile phase - * [HIVE-9841] - IOException thrown by ORC should include the path of processing file - * [HIVE-9845] - HCatSplit repeats information making input split data size huge - * [HIVE-9848] - readlink -f is GNU coreutils only (used in bin/hive) - * [HIVE-9851] - org.apache.hadoop.hive.serde2.avro.AvroSerializer should use org.apache.avro.generic.GenericData.Array when serializing a list - * [HIVE-9855] - Runtime skew join doesn't work when skewed data only exists in big table - * [HIVE-9860] - MapredLocalTask/SecureCmdDoAs leaks local files - * [HIVE-9866] - Changing a column's type doesn't change column stats type in metastore - * [HIVE-9869] - Trunk doesn't build with hadoop-1 - * [HIVE-9873] - Hive on MR throws DeprecatedParquetHiveInput exception - * [HIVE-9877] - Beeline cannot run multiple statements in the same row - * [HIVE-9886] - Hive on tez: NPE when converting join to SMB in sub-query - * [HIVE-9892] - various MSSQL upgrade scripts don't work - * [HIVE-9908] - vectorization error binary type not supported, group by with binary columns - * [HIVE-9915] - Allow specifying file format for managed tables - * [HIVE-9919] - upgrade scripts don't work on some auto-created DBs due to absence of tables - * [HIVE-9920] - DROP DATABASE IF EXISTS throws exception if database does not exist - * [HIVE-9923] - No clear message when "from" is missing - * [HIVE-9929] - StatsUtil#getAvailableMemory could return negative value - * [HIVE-9930] - fix QueryPlan.makeQueryId time format - * [HIVE-9932] - DDLTask.conf hides base class Task.conf - * [HIVE-9934] - Vulnerability in LdapAuthenticationProviderImpl enables HiveServer2 client to degrade the authentication mechanism to "none", allowing authentication without password - * [HIVE-9936] - fix potential NPE in DefaultUDAFEvaluatorResolver - * [HIVE-9944] - Convert array[] to string properly in log messages - * [HIVE-9945] - FunctionTask.conf hides Task.conf field - * [HIVE-9947] - ScriptOperator replaceAll uses unescaped dot and result is not assigned - * [HIVE-9948] - SparkUtilities.getFileName passes File.separator to String.split() method - * [HIVE-9950] - fix rehash in CuckooSetBytes and CuckooSetLong - * [HIVE-9951] - VectorizedRCFileRecordReader creates Exception but does not throw it - * [HIVE-9952] - fix NPE in CorrelationUtilities - * [HIVE-9953] - fix NPE in WindowingTableFunction - * [HIVE-9954] - UDFJson uses the == operator to compare Strings - * [HIVE-9955] - TestVectorizedRowBatchCtx compares byte[] using equals() method - * [HIVE-9956] - use BigDecimal.valueOf instead of new in TestFileDump - * [HIVE-9957] - Hive 1.1.0 not compatible with Hadoop 2.4.0 - * [HIVE-9961] - HookContext for view should return a table type of VIRTUAL_VIEW - * [HIVE-9971] - Clean up operator class - * [HIVE-9975] - Renaming a nonexisting partition should not throw out NullPointerException - * [HIVE-9976] - Possible race condition in DynamicPartitionPruner for <200ms tasks - * [HIVE-9977] - Compactor not running on partitions after dynamic partitioned insert - * [HIVE-9984] - JoinReorder's getOutputSize is exponential - * [HIVE-9991] - Cannot do a SELECT on external tables that are on S3 due to Encryption error - * [HIVE-9994] - Hive query plan returns sensitive data to external applications - * [HIVE-9997] - minor tweaks for bytes mapjoin hash table - * [HIVE-10001] - SMB join in reduce side - * [HIVE-10032] - Remove HCatalog broken java file from source code - * [HIVE-10042] - clean up TreeReaders - ORC refactoring for LLAP on trunk - * [HIVE-10050] - Support overriding memory configuration for AM launched for TempletonControllerJob - * [HIVE-10059] - Make udaf_percentile_approx_23.q test more stable - * [HIVE-10062] - HiveOnTez: Union followed by Multi-GB followed by Multi-insert loses data - * [HIVE-10066] - Hive on Tez job submission through WebHCat doesn't ship Tez artifacts - * [HIVE-10074] - Ability to run HCat Client Unit tests in a system test setting - * [HIVE-10078] - Optionally allow logging of records processed in fixed intervals - * [HIVE-10083] - SMBJoin fails in case one table is uninitialized - * [HIVE-10085] - Lateral view on top of a view throws RuntimeException - * [HIVE-10086] - Hive throws error when accessing Parquet file schema using field name match - * [HIVE-10087] - Beeline's --silent option should suppress query from being echoed when running with -f option - * [HIVE-10093] - Unnecessary HMSHandler initialization for default MemoryTokenStore on HS2 - * [HIVE-10095] - format_number udf throws NPE - * [HIVE-10098] - HS2 local task for map join fails in KMS encrypted cluster - * [HIVE-10106] - Regression : Dynamic partition pruning not working after HIVE-9976 - * [HIVE-10108] - Index#getIndexTableName() returns db.index_table_name - * [HIVE-10122] - Hive metastore filter-by-expression is broken for non-partition expressions - * [HIVE-10123] - Hybrid grace Hash join : Use estimate key count from stats to initialize BytesBytesMultiHashMap - * [HIVE-10128] - BytesBytesMultiHashMap does not allow concurrent read-only access - * [HIVE-10136] - BaseWork.vectorMode hides AbstractOperatorDesc.vectorMode - * [HIVE-10145] - set Tez ACLs appropriately in hive - * [HIVE-10148] - update of bucketing column should not be allowed - * [HIVE-10150] - delete from acidTbl where a in(select a from nonAcidOrcTbl) fails - * [HIVE-10151] - insert into A select from B is broken when both A and B are Acid tables and bucketed the same way - * [HIVE-10152] - ErrorMsg.formatToErrorMsgMap has bad regex - * [HIVE-10167] - HS2 logs the server started only before the server is shut down - * [HIVE-10172] - Fix performance regression caused by HIVE-8122 for ORC - * [HIVE-10178] - DateWritable incorrectly calculates daysSinceEpoch for negative Unix time - * [HIVE-10186] - Hive does not log Tez diagnostics on errors - * [HIVE-10197] - Add lockId to all ACID log statements - * [HIVE-10202] - Beeline outputs prompt+query on standard output when used in non-interactive mode - * [HIVE-10208] - templeton.hive.extra.files should be commented out in webhcat-default.xml - * [HIVE-10213] - MapReduce jobs using dynamic-partitioning fail on commit. - * [HIVE-10225] - CLI JLine does not flush history on quit/Ctrl-C - * [HIVE-10226] - Column stats for Date columns not supported - * [HIVE-10229] - Set conf and processor context in the constructor instead of init - * [HIVE-10231] - Compute partition column stats fails if partition col type is date - * [HIVE-10242] - ACID: insert overwrite prevents create table command - * [HIVE-10265] - Hive CLI crashes on != inequality - * [HIVE-10267] - HIVE-9664 makes hive depend on ivysettings.xml : trivial breakage fix - * [HIVE-10269] - HiveMetaStore.java:[6089,29] cannot find symbol class JvmPauseMonitor - * [HIVE-10270] - Cannot use Decimal constants less than 0.1BD - * [HIVE-10271] - remove hive.server2.thrift.http.min/max.worker.threads properties - * [HIVE-10272] - Some HCat tests fail under windows - * [HIVE-10273] - Union with partition tables which have no data fails with NPE - * [HIVE-10274] - Send context and description to tez via dag info - * [HIVE-10275] - GenericUDF getTimestampValue should return Timestamp instead of Date - * [HIVE-10284] - enable container reuse for grace hash join - * [HIVE-10285] - Incorrect endFunction call in HiveMetaStore - * [HIVE-10286] - SARGs: Type Safety via PredicateLeaf.type - * [HIVE-10288] - Cannot call permanent UDFs - * [HIVE-10290] - Add negative test case to modify a non-existent config value when hive security authorization is enabled. - * [HIVE-10303] - HIVE-9471 broke forward compatibility of ORC files - * [HIVE-10305] - TestOrcFile has a mistake that makes metadata test ineffective - * [HIVE-10306] - We need to print tez summary when hive.server2.logging.level >= PERFORMANCE. - * [HIVE-10309] - TestJdbcWithMiniHS2.java broken because of the removal of hive.server2.thrift.http.max.worker.threads - * [HIVE-10312] - SASL.QOP in JDBC URL is ignored for Delegation token Authentication - * [HIVE-10313] - Literal Decimal ExprNodeConstantDesc should contain value of HiveDecimal instead of String - * [HIVE-10318] - The HMS upgrade test does not test patches that affect the upgrade test scripts - * [HIVE-10323] - Tez merge join operator does not honor hive.join.emit.interval - * [HIVE-10324] - Hive metatool should take table_param_key to allow for changes to avro serde's schema url key - * [HIVE-10331] - ORC : Is null SARG filters out all row groups written in old ORC format - * [HIVE-10340] - Enable ORC test for timezone reading from old format - * [HIVE-10346] - Tez on HBase has problems with settings again - * [HIVE-10364] - The HMS upgrade script test does not publish results when prepare.sh fails. - * [HIVE-10367] - org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.* fails in Windows - * [HIVE-10370] - Hive does not compile with -Phadoop-1 option - * [HIVE-10372] - Update parquet version to 1.6.0 - * [HIVE-10374] - Revert HIVE-10304 deprecation message to HiveCLI - * [HIVE-10384] - RetryingMetaStoreClient does not retry wrapped TTransportExceptions - * [HIVE-10407] - separate out the timestamp ranges for testing purposes - * [HIVE-10409] - Webhcat tests need to be updated, to accomodate HADOOP-10193 - * [HIVE-10421] - DROP TABLE with qualified table name ignores database name when checking partitions - * [HIVE-10428] - NPE in RegexSerDe using HCat - * [HIVE-10431] - HIVE-9555 broke hadoop-1 build - * [HIVE-10437] - NullPointerException on queries where map/reduce is not involved on tables with partitions - * [HIVE-10441] - Fix confusing log statement in SessionState about hive.execution.engine setting - * [HIVE-10442] - HIVE-10098 broke hadoop-1 build - * [HIVE-10443] - HIVE-9870 broke hadoop-1 build - * [HIVE-10444] - HIVE-10223 breaks hadoop-1 build - * [HIVE-10446] - Hybrid Hybrid Grace Hash Join : java.lang.IllegalArgumentException in Kryo while spilling big table - * [HIVE-10447] - Beeline JDBC Driver to support 2 way SSL - * [HIVE-10450] - More than one TableScan in MapWork not supported in Vectorization -- causes query to fail during vectorization - * [HIVE-10451] - PTF deserializer fails if values are not used in reducer - * [HIVE-10456] - Grace Hash Join should not load spilled partitions on abort - * [HIVE-10465] - whitelist restrictions don't get initialized in new copy of HiveConf - * [HIVE-10472] - Jenkins HMS upgrade test is not publishing results because JIRAService class is not found. - * [HIVE-10477] - Provide option to disable Spark tests - * [HIVE-10481] - ACID table update finishes but values not really updated if column names are not all lower case - * [HIVE-10483] - insert overwrite partition deadlocks on itself with DbTxnManager - * [HIVE-10484] - Vectorization : RuntimeException "Big Table Retained Mapping duplicate column" - * [HIVE-10490] - HBase Snapshot IF fails at run time with missing dependency of MetricsRegistry - * [HIVE-10499] - Ensure Session/ZooKeeperClient instances are closed - * [HIVE-10500] - Repeated deadlocks in underlying RDBMS cause transaction or lock failure - * [HIVE-10507] - Expose RetryingMetastoreClient to other external users of metastore client like Flume and Storm. - * [HIVE-10508] - Strip out password information from config passed to Tez/MR in cases where password encryption is not used - * [HIVE-10510] - Change 1.2.0-SNAPSHOT to 1.2.0 in branch-1.2 - * [HIVE-10513] - [CBO] return path : Fix create_func1.q for return path - * [HIVE-10514] - Fix MiniCliDriver tests failure - * [HIVE-10517] - HCatPartition should not be created with "" as location in tests - * [HIVE-10521] - TxnHandler.timeOutTxns only times out some of the expired transactions - * [HIVE-10529] - Remove references to tez task context before storing operator plan in object cache - * [HIVE-10530] - Aggregate stats cache: bug fixes for RDBMS path - * [HIVE-10538] - Fix NPE in FileSinkOperator from hashcode mismatch - * [HIVE-10539] - set default value of hive.repl.task.factory - * [HIVE-10542] - Full outer joins in tez produce incorrect results in certain cases - * [HIVE-10543] - improve error message in MetaStoreAuthzAPIAuthorizerEmbedOnly - * [HIVE-10544] - Beeline/Hive JDBC Driver fails in HTTP mode on Windows with java.lang.NoSuchFieldError: INSTANCE - * [HIVE-10548] - Remove dependency to s3 repository in root pom - * [HIVE-10564] - webhcat should use webhcat-site.xml properties for controller job submission - * [HIVE-10572] - Improve Hive service test to check empty string - * [HIVE-10576] - add jar command does not work with Windows OS - * [HIVE-10578] - update sql standard authorization configuration whitelist - * [HIVE-10579] - Fix -Phadoop-1 build - * [HIVE-10595] - Dropping a table can cause NPEs in the compactor - * [HIVE-10603] - increase default permgen space for HS2 on windows - * [HIVE-10604] - update webhcat-default.xml with 1.2 version numbers - * [HIVE-10607] - Combination of ReducesinkDedup + TopN optimization yields incorrect result if there are multiple GBY in reducer - * [HIVE-10609] - Vectorization : Q64 fails with ClassCastException - * [HIVE-10610] - hive command fails to get hadoop version - * [HIVE-10612] - HIVE-10578 broke TestSQLStdHiveAccessControllerHS2 tests - * [HIVE-10614] - schemaTool upgrade from 0.14.0 to 1.3.0 causes failure - * [HIVE-10638] - HIVE-9736 introduces issues with Hadoop23Shims.checkFileAccess - * [HIVE-10715] - RAT failures - many files do not have ASF licenses - - -** Improvement - * [HIVE-1575] - get_json_object does not support JSON array at the root level - * [HIVE-2573] - Create per-session function registry - * [HIVE-3185] - allow union set operation without ALL keyword (align to ISO-SQL 2011 specification) - * [HIVE-5472] - support a simple scalar which returns the current timestamp - * [HIVE-7175] - Provide password file option to beeline - * [HIVE-8385] - UNION Operator in Hive - * [HIVE-9138] - Add some explain to PTF operator - * [HIVE-9143] - select user(), current_user() - * [HIVE-9298] - Support reading alternate timestamp formats - * [HIVE-9470] - Use a generic writable object to run ColumnaStorageBench write/read tests - * [HIVE-9480] - Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY - * [HIVE-9500] - Support nested structs over 24 levels. - * [HIVE-9518] - Implement MONTHS_BETWEEN aligned with Oracle one - * [HIVE-9521] - Drop support for Java6 - * [HIVE-9556] - create UDF to calculate the Levenshtein distance between two strings - * [HIVE-9564] - Extend HIVE-9298 for JsonSerDe - * [HIVE-9590] - add qtests for ADD_MONTHS UDF - * [HIVE-9594] - Add qtests for LAST_DAY udf - * [HIVE-9596] - move standard getDisplayString impl to GenericUDF - * [HIVE-9607] - Remove unnecessary attach-jdbc-driver execution from package/pom.xml - * [HIVE-9618] - Deduplicate RS keys for ptf/windowing - * [HIVE-9664] - Hive "add jar" command should be able to download and add jars from a repository - * [HIVE-9673] - Set operationhandle in ATS entities for lookups - * [HIVE-9699] - Extend PTFs to provide referenced columns for CP - * [HIVE-9709] - Hive should support replaying cookie from JDBC driver for beeline - * [HIVE-9710] - HiveServer2 should support cookie based authentication, when using HTTP transport. - * [HIVE-9738] - create SOUNDEX udf - * [HIVE-9744] - Move common arguments validation and value extraction code to GenericUDF - * [HIVE-9766] - Add JavaConstantXXXObjectInspector - * [HIVE-9788] - Make double quote optional in tsv/csv/dsv output - * [HIVE-9793] - Remove hard coded paths from cli driver tests - * [HIVE-9800] - Create scripts to do metastore upgrade tests on Jenkins - * [HIVE-9857] - Create Factorial UDF - * [HIVE-9858] - Create cbrt (cube root) UDF - * [HIVE-9859] - Create bitwise left/right shift UDFs - * [HIVE-9870] - Add JvmPauseMonitor threads to HMS and HS2 daemons - * [HIVE-9909] - Specify hive branch to use on jenkins hms tests - * [HIVE-9914] - Post success comments on Jira from Jenkins metastore upgrades scripts - * [HIVE-9917] - After HIVE-3454 is done, make int to timestamp conversion configurable - * [HIVE-9981] - Avoid throwing many exceptions when attempting to create new hdfs encryption shim - * [HIVE-10007] - Support qualified table name in analyze table compute statistics for columns - * [HIVE-10027] - Use descriptions from Avro schema files in column comments - * [HIVE-10072] - Add vectorization support for Hybrid Grace Hash Join - * [HIVE-10114] - Split strategies for ORC - * [HIVE-10119] - Allow Log verbosity to be set in hiveserver2 session - * [HIVE-10120] - Disallow create table with dot/colon in column name - * [HIVE-10146] - Not count session as idle if query is running - * [HIVE-10160] - Give a warning when grouping or ordering by a constant column - * [HIVE-10177] - Enable constant folding for char & varchar - * [HIVE-10206] - Improve Alter Table to not initialize Serde unnecessarily - * [HIVE-10214] - log metastore call timing information aggregated at query level - * [HIVE-10216] - log hive cli classpath at debug level - * [HIVE-10223] - Consolidate several redundant FileSystem API calls. - * [HIVE-10276] - Implement date_format(timestamp, fmt) UDF - * [HIVE-10307] - Support to use number literals in partition column - * [HIVE-10339] - Allow JDBC Driver to pass HTTP header Key/Value pairs - * [HIVE-10382] - Aggregate stats cache for RDBMS based metastore codepath - * [HIVE-10403] - Add n-way join support for Hybrid Grace Hash Join - * [HIVE-10467] - Switch to GIT repository on Jenkins precommit tests - * [HIVE-10546] - genFileSinkPlan should use the generated SEL's RR for the partition col of FS - * [HIVE-10568] - Select count(distinct()) can have more optimal execution plan - - -** New Feature - * [HIVE-7998] - Enhance JDBC Driver to not require class specification - * [HIVE-9039] - Support Union Distinct - * [HIVE-9188] - BloomFilter support in ORC - * [HIVE-9277] - Hybrid Hybrid Grace Hash Join - * [HIVE-9302] - Beeline add commands to register local jdbc driver names and jars - * [HIVE-9780] - Add another level of explain for RDBMS audience - * [HIVE-10038] - Add Calcite's ProjectMergeRule. - * [HIVE-10099] - Enable constant folding for Decimal - * [HIVE-10591] - Support limited integer type promotion in ORC - - -** Task - * [HIVE-6617] - Reduce ambiguity in grammar - * [HIVE-8898] - Remove HIVE-8874 once HBASE-12493 is fixed - * [HIVE-9015] - Constant Folding optimizer doesn't handle expressions involving null - * [HIVE-9411] - Improve error messages in TestMultiOutputFormat - * [HIVE-9416] - Get rid of Extract Operator - * [HIVE-9485] - Update trunk to 1.2.0-SNAPSHOT - * [HIVE-9498] - Update golden files of join38 & subquery_in on trunk due to 9327 - * [HIVE-9520] - Create NEXT_DAY UDF - * [HIVE-9572] - Merge from Spark branch to trunk 02/03/2015 - * [HIVE-9703] - Merge from Spark branch to trunk 02/16/2015 - * [HIVE-9867] - Migrate usage of deprecated Calcite methods - * [HIVE-9903] - Update calcite version - * [HIVE-9966] - Get rid of customBucketMapJoin field from MapJoinDesc - * [HIVE-10019] - Configure jenkins precommit jobs to run HMS upgrade tests - * [HIVE-10222] - Upgrade Calcite dependency to newest version - * [HIVE-10268] - Merge cbo branch into trunk - * [HIVE-10676] - Update Hive's README to mention spark, and to remove jdk1.6 - - -** Test - * [HIVE-7948] - Add an E2E test to verify fix for HIVE-7155 - * [HIVE-8949] - Test for PARQUET-26 - * [HIVE-9399] - ppd_multi_insert.q generate same output in different order, when mapred.reduce.tasks is set to larger than 1 - * [HIVE-9525] - Enable constant propagation optimization in few existing tests where it was disabled. - * [HIVE-10135] - Add qtest to access struct<> data type with parquet format after parquet column index access enabled - * [HIVE-10168] - make groupby3_map.q more stable - * [HIVE-10396] - decimal_precision2.q test is failing on trunk - - - -Release Notes - Hive - Version 1.1.0 - -** Sub-task - * [HIVE-7073] - Implement Binary in ParquetSerDe - * [HIVE-8121] - Create micro-benchmarks for ParquetSerde and evaluate performance - * [HIVE-8122] - Make use of SearchArgument classes for Parquet SERDE - * [HIVE-8130] - Support Date in Avro - * [HIVE-8131] - Support timestamp in Avro - * [HIVE-8362] - Investigate flaky test parallel.q - * [HIVE-8651] - CBO: sort column changed in infer_bucket_sort test - * [HIVE-8707] - Fix ordering differences due to Java 8 HashMap function - * [HIVE-8718] - Refactoring: move mapLocalWork field from MapWork to BaseWork - * [HIVE-8773] - Fix TestWebHCatE2e#getStatus for Java8 - * [HIVE-8862] - Fix ordering diferences on TestParse tests due to Java8 - * [HIVE-8922] - CBO: assorted date and timestamp issues - * [HIVE-8923] - HIVE-8512 needs to be fixed also for CBO - * [HIVE-8936] - Add SORT_QUERY_RESULTS for join tests that do not guarantee order - * [HIVE-8962] - Add SORT_QUERY_RESULTS for join tests that do not guarantee order #2 - * [HIVE-9030] - CBO: Plans with comparison of values with different types - * [HIVE-9033] - Fix ordering differences due to Java8 (part 2) - * [HIVE-9034] - CBO: type change in literal_ints.q - * [HIVE-9035] - CBO: Disable PPD when functions are non-deterministic (ppd_random.q - non-deterministic udf rand() pushed above join) - * [HIVE-9043] - HiveException: Conflict on row inspector for {table} - * [HIVE-9066] - temporarily disable CBO for non-deterministic functions - * [HIVE-9104] - windowing.q failed when mapred.reduce.tasks is set to larger than one - * [HIVE-9109] - Add support for Java 8 specific q-test out files - * [HIVE-9127] - Improve CombineHiveInputFormat.getSplit performance - * [HIVE-9133] - CBO (Calcite Return Path): Refactor Semantic Analyzer to Move CBO code out - * [HIVE-9153] - Perf enhancement on CombineHiveInputFormat and HiveInputFormat - * [HIVE-9161] - Fix ordering differences on UDF functions due to Java8 - * [HIVE-9181] - Fix SkewJoinOptimizer related Java 8 ordering differences - * [HIVE-9193] - Fix ordering differences due to Java 8 (Part 3) - * [HIVE-9194] - Support select distinct * - * [HIVE-9200] - CBO (Calcite Return Path): Inline Join, Properties - * [HIVE-9206] - Fix Desc Formatted related Java 8 ordering differences - * [HIVE-9222] - Fix ordering differences due to Java 8 (Part 4) - * [HIVE-9224] - CBO (Calcite Return Path): Inline Table, Properties - * [HIVE-9239] - Fix ordering differences due to Java 8 (Part 5) - * [HIVE-9241] - Fix TestCliDriver.testCliDriver_subquery_multiinsert - * [HIVE-9257] - Merge from spark to trunk January 2015 - * [HIVE-9259] - Fix ClassCastException when CBO is enabled for HOS [Spark Branch] - * [HIVE-9264] - Merge encryption branch to trunk - * [HIVE-9292] - CBO (Calcite Return Path): Inline GroupBy, Properties - * [HIVE-9315] - CBO (Calcite Return Path): Inline FileSinkOperator, Properties - * [HIVE-9352] - Merge from spark to trunk (follow-up of HIVE-9257) - * [HIVE-9409] - Avoid ser/de loggers as logging framework can be incompatible on driver and workers - * [HIVE-9410] - ClassNotFoundException occurs during hive query case execution with UDF defined [Spark Branch] - * [HIVE-9428] - LocalSparkJobStatus may return failed job as successful [Spark Branch] - * [HIVE-9431] - CBO (Calcite Return Path): Removing AST from ParseContext - * [HIVE-9434] - Shim the method Path.getPathWithoutSchemeAndAuthority - * [HIVE-9444] - CBO (Calcite Return Path): Rewrite GlobalLimitOptimizer - * [HIVE-9449] - Push YARN configuration to Spark while deply Spark on YARN[Spark Branch] - * [HIVE-9450] - [Parquet] Check all data types work for Parquet in Group By operator - * [HIVE-9477] - No error thrown when global limit optimization failed to find enough number of rows [Spark Branch] - * [HIVE-9487] - Make Remote Spark Context secure [Spark Branch] - * [HIVE-9493] - Failed job may not throw exceptions [Spark Branch] - -** Bug - * [HIVE-1344] - error in select disinct - * [HIVE-1654] - select distinct should allow column name regex - * [HIVE-1869] - TestMTQueries failing on jenkins - * [HIVE-3781] - Index related events should be delivered to metastore event listener - * [HIVE-4009] - CLI Tests fail randomly due to MapReduce LocalJobRunner race condition - * [HIVE-5536] - Incorrect Operation Name is passed to hookcontext - * [HIVE-5631] - Index creation on a skew table fails - * [HIVE-5664] - Drop cascade database fails when the db has any tables with indexes - * [HIVE-5865] - AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8' - * [HIVE-6165] - Unify HivePreparedStatement from jdbc:hive and jdbc:hive2 - * [HIVE-6198] - ORC file and struct column names are case sensitive - * [HIVE-6308] - COLUMNS_V2 Metastore table not populated for tables created without an explicit column list. - * [HIVE-6421] - abs() should preserve precision/scale of decimal input - * [HIVE-6468] - HS2 & Metastore using SASL out of memory error when curl sends a get request - * [HIVE-6623] - Add "owner" tag to ptest2 created instances - * [HIVE-6679] - HiveServer2 should support configurable the server side socket timeout and keepalive for various transports types where applicable - * [HIVE-6683] - Beeline does not accept comments at end of line - * [HIVE-6914] - parquet-hive cannot write nested map (map value is map) - * [HIVE-7024] - Escape control characters for explain result - * [HIVE-7069] - Zookeeper connection leak - * [HIVE-7270] - SerDe Properties are not considered by show create table Command - * [HIVE-7932] - It may cause NP exception when add accessed columns to ReadEntity - * [HIVE-7951] - InputFormats implementing (Job)Configurable should not be cached - * [HIVE-7997] - Potential null pointer reference in ObjectInspectorUtils#compareTypes() - * [HIVE-8099] - IN operator for partition column fails when the partition column type is DATE - * [HIVE-8182] - beeline fails when executing multiple-line queries with trailing spaces - * [HIVE-8257] - Accumulo introduces old hadoop-client dependency - * [HIVE-8266] - create function using <resource> statement compilation should include resource URI entity - * [HIVE-8284] - Equality comparison is done between two floating point variables in HiveRelMdUniqueKeys#getUniqueKeys() - * [HIVE-8295] - Add batch retrieve partition objects for metastore direct sql - * [HIVE-8308] - Acid related table properties should be defined in one place and should be case insensitive - * [HIVE-8317] - WebHCat pom should explicitly depend on jersey-core - * [HIVE-8326] - Using DbTxnManager with concurrency off results in run time error - * [HIVE-8330] - HiveResultSet.findColumn() parameters are case sensitive - * [HIVE-8338] - Add ip and command to semantic analyzer hook context - * [HIVE-8345] - q-test for Avro date support - * [HIVE-8359] - Map containing null values are not correctly written in Parquet files - * [HIVE-8374] - schematool fails on Postgres versions < 9.2 - * [HIVE-8381] - Update hive version on trunk to 0.15 - * [HIVE-8387] - add retry logic to ZooKeeperStorage in WebHCat - * [HIVE-8448] - Union All might not work due to the type conversion issue - * [HIVE-8450] - Create table like does not copy over table properties - * [HIVE-8485] - HMS on Oracle incompatibility - * [HIVE-8491] - Fix build name in ptest pre-commit message - * [HIVE-8500] - beeline does not need to set hive.aux.jars.path - * [HIVE-8512] - queries with star and gby produce incorrect results - * [HIVE-8518] - Compile time skew join optimization returns duplicated results - * [HIVE-8523] - Potential null dereference in DDLSemanticAnalyzer#addInputsOutputsAlterTable() - * [HIVE-8532] - return code of "source xxx" clause is missing - * [HIVE-8556] - introduce overflow control and sanity check to BytesBytesMapJoin - * [HIVE-8564] - DROP TABLE IF EXISTS throws exception if the table does not exist. - * [HIVE-8565] - beeline may go into an infinite loop when using EOF - * [HIVE-8576] - Guaranteed NPE in StatsRulesProcFactory - * [HIVE-8594] - Wrong condition in SettableConfigUpdater#setHiveConfWhiteList() - * [HIVE-8600] - Add option to log explain output for query - * [HIVE-8610] - Compile time skew join optimization doesn't work with auto map join - * [HIVE-8611] - grant/revoke syntax should support additional objects for authorization plugins - * [HIVE-8612] - Support metadata result filter hooks - * [HIVE-8613] - percentile_approx raise a comparator error - * [HIVE-8627] - Compute stats on a table from impala caused the table to be corrupted - * [HIVE-8634] - HiveServer2 fair scheduler queue mapping doesn't handle the secondary groups rules correctly - * [HIVE-8636] - CBO: split cbo_correctness test - * [HIVE-8666] - hive.metastore.server.max.threads default is too high - * [HIVE-8680] - Set Max Message for Binary Thrift endpoints - * [HIVE-8693] - Separate out fair scheduler dependency from hadoop 0.23 shim - * [HIVE-8706] - Table statistic collection on counter failed due to table name character case. - * [HIVE-8708] - Add query id to explain log option - * [HIVE-8715] - Hive 14 upgrade scripts can fail for statistics if database was created using auto-create - * [HIVE-8720] - Update orc_merge tests to make it consistent across OS'es - * [HIVE-8728] - Fix ptf.q determinism - * [HIVE-8730] - schemaTool failure when date partition has non-date value - * [HIVE-8736] - add ordering to cbo_correctness to make result consistent - * [HIVE-8739] - handle Derby and Oracle errors with joins and filters in Direct SQL in a invalid-DB-specific path - * [HIVE-8757] - YARN dep in scheduler shim should be optional - * [HIVE-8762] - HiveMetaStore.BooleanPointer should be replaced with an AtomicBoolean - * [HIVE-8784] - Querying partition does not work with JDO enabled against PostgreSQL - * [HIVE-8791] - Hive permission inheritance throws exception S3 - * [HIVE-8796] - TestCliDriver acid tests with decimal needs benchmark to be updated - * [HIVE-8797] - Simultaneous dynamic inserts can result in "partition already exists" error - * [HIVE-8803] - DESC SCHEMA <DATABASE-NAME> is not working - * [HIVE-8805] - CBO skipped due to SemanticException: Line 0:-1 Both left and right aliases encountered in JOIN 'avg_cs_ext_discount_amt' - * [HIVE-8808] - HiveInputFormat caching cannot work with all input formats - * [HIVE-8811] - Dynamic partition pruning can result in NPE during query compilation - * [HIVE-8812] - TestMinimrCliDriver failure if run in the same command as TestHBaseNegativeCliDriver - * [HIVE-8825] - SQLCompletor catches Throwable and ignores it - * [HIVE-8827] - Remove SSLv2Hello from list of disabled protocols - * [HIVE-8830] - hcatalog process don't exit because of non daemon thread - * [HIVE-8845] - Switch to Tez 0.5.2 - * [HIVE-8847] - Fix bugs in jenkins scripts - * [HIVE-8848] - data loading from text files or text file processing doesn't handle nulls correctly - * [HIVE-8850] - ObjectStore:: rollbackTransaction() needs to be looked into further. - * [HIVE-8863] - Cannot drop table with uppercase name after "compute statistics for columns" - * [HIVE-8866] - Vectorization on partitioned table throws ArrayIndexOutOfBoundsException when partitions are not of same #of columns - * [HIVE-8869] - RowSchema not updated for some ops when columns are pruned - * [HIVE-8870] - errors when selecting a struct field within an array from ORC based tables - * [HIVE-8872] - Hive view of HBase range scan intermittently returns incorrect data. - * [HIVE-8873] - Switch to calcite 0.9.2 - * [HIVE-8874] - Error Accessing HBase from Hive via Oozie on Kerberos 5.0.1 cluster - * [HIVE-8875] - hive.optimize.sort.dynamic.partition should be turned off for ACID - * [HIVE-8876] - incorrect upgrade script for Oracle (13->14) - * [HIVE-8877] - improve context logging during job submission via WebHCat - * [HIVE-8879] - Upgrade derby version to address race candition - * [HIVE-8880] - non-synchronized access to split list in OrcInputFormat - * [HIVE-8881] - Receiving json "{"error":"Could not find job job_1415748506143_0002"}" when web client tries to fetch all jobs from webhcat where HDFS does not have the data. - * [HIVE-8886] - Some Vectorized String CONCAT expressions result in runtime error Vectorization: Unsuported vector output type: StringGroup - * [HIVE-8888] - Mapjoin with LateralViewJoin generates wrong plan in Tez - * [HIVE-8889] - JDBC Driver ResultSet.getXXXXXX(String columnLabel) methods Broken - * [HIVE-8890] - HiveServer2 dynamic service discovery: use persistent ephemeral nodes curator recipe - * [HIVE-8891] - Another possible cause to NucleusObjectNotFoundException from drops/rollback - * [HIVE-8893] - Implement whitelist for builtin UDFs to avoid untrused code execution in multiuser mode - * [HIVE-8901] - increase retry attempt, interval on metastore database errors - * [HIVE-8909] - Hive doesn't correctly read Parquet nested types - * [HIVE-8914] - HDFSCleanup thread holds reference to FileSystem - * [HIVE-8916] - Handle user@domain username under LDAP authentication - * [HIVE-8917] - HIVE-5679 adds two thread safety problems - * [HIVE-8926] - Projections that only swap input columns are identified incorrectly as identity projections - * [HIVE-8938] - Compiler should save the transform URI as input entity - * [HIVE-8944] - TestCompactor fails with IncompatibleClassChangeError - * [HIVE-8947] - HIVE-8876 also affects Postgres < 9.2 - * [HIVE-8948] - TestStreaming is flaky - * [HIVE-8964] - Some TestMiniTezCliDriver tests taking two hours - * [HIVE-8965] - Enhance PTest to kill all processes between tests and to report when a TEST*.xml file is not generated - * [HIVE-8967] - Fix bucketmapjoin7.q determinism - * [HIVE-8975] - Possible performance regression on bucket_map_join_tez2.q - * [HIVE-8978] - Fix test determinism issue for qfile: smb_mapjoin_1.q etc - * [HIVE-8990] - mapjoin_mapjoin.q is failing on Tez (missed golden file update) - * [HIVE-9001] - Ship with log4j.properties file that has a reliable time based rolling policy - * [HIVE-9003] - Vectorized IF expr broken for the scalar and scalar case - * [HIVE-9006] - hiveserver thrift api version is still 6 - * [HIVE-9011] - Fix parquet_join.q determinism - * [HIVE-9024] - NullPointerException when starting webhcat server if templeton.hive.properties is not set - * [HIVE-9025] - join38.q (without map join) produces incorrect result when testing with multiple reducers - * [HIVE-9032] - Help for orcfiledump script does not reflect new options - * [HIVE-9048] - Hive build failed on hadoop-1 after HIVE-8828. - * [HIVE-9051] - TezJobMonitor in-place updates logs too often to logfile - * [HIVE-9053] - select constant in union all followed by group by gives wrong result - * [HIVE-9055] - Tez: union all followed by group by followed by another union all gives error - * [HIVE-9060] - Fix child operator references after NonBlockingOpDeDupProc - * [HIVE-9067] - OrcFileMergeOperator may create merge file that does not match properties of input files - * [HIVE-9077] - Set completer in CliDriver is not working - * [HIVE-9090] - Rename "Tez File Merge Work" to smaller name - * [HIVE-9096] - GenericUDF may be left unclosed in PartitionPrune#visitCall() - * [HIVE-9108] - Fix for HIVE-8735 is incorrect (stats with long paths) - * [HIVE-9111] - Potential NPE in OrcStruct for list and map types - * [HIVE-9112] - Query may generate different results depending on the number of reducers - * [HIVE-9113] - Explain on query failed with NPE - * [HIVE-9114] - union all query in cbo test has undefined ordering - * [HIVE-9120] - Hive Query log does not work when hive.exec.parallel is true - * [HIVE-9122] - Need to remove additional references to hive-shims-common-secure, hive-shims-0.20 - * [HIVE-9126] - Backport HIVE-8827 (Remove SSLv2Hello from list of disabled protocols) to 0.14 branch - * [HIVE-9129] - Migrate to newer Calcite snapshot, where ByteString is now in org.apache.calcite.avatica.util - * [HIVE-9130] - vector_partition_diff_num_cols result is not updated after CBO upgrade - * [HIVE-9131] - MiniTez optimize_nullscan test is unstable - * [HIVE-9141] - HiveOnTez: mix of union all, distinct, group by generates error - * [HIVE-9149] - Add unit test to test implicit conversion during dynamic partitioning/distribute by - * [HIVE-9150] - Unrelated types are compared in GenTezWork#getFollowingWorkIndex() - * [HIVE-9154] - Cache pathToPartitionInfo in context aware record reader - * [HIVE-9155] - HIVE_LOCKS uses int instead of bigint hive-txn-schema-0.14.0.mssql.sql - * [HIVE-9162] - stats19 test is environment-dependant - * [HIVE-9166] - Place an upper bound for SARG CNF conversion - * [HIVE-9168] - Vectorized Coalesce for strings is broken - * [HIVE-9177] - Fix child operator references after NonBlockingOpDeDupProc (II) - * [HIVE-9195] - CBO changes constant to column type - * [HIVE-9197] - fix lvj_mapjoin.q diff in trunk - * [HIVE-9199] - Excessive exclusive lock used in some DDLs with DummyTxnManager - * [HIVE-9203] - CREATE TEMPORARY FUNCTION hangs trying to acquire lock - * [HIVE-9205] - Change default tez install directory to use /tmp instead of /user and create the directory if it does not exist - * [HIVE-9215] - Some mapjoin queries broken with IdentityProjectRemover with PPD - * [HIVE-9221] - Remove deprecation warning for hive.metastore.local - * [HIVE-9234] - HiveServer2 leaks FileSystem objects in FileSystem.CACHE - * [HIVE-9242] - Many places in CBO code eat exceptions - * [HIVE-9243] - Static Map in IOContext is not thread safe - * [HIVE-9249] - java.lang.ClassCastException: org.apache.hadoop.hive.serde2.io.HiveVarcharWritable cannot be cast to org.apache.hadoop.hive.common.type.HiveVarchar when joining tables - * [HIVE-9255] - Fastpath for limited fetches from unpartitioned tables - * [HIVE-9296] - Need to add schema upgrade changes for queueing events in the database - * [HIVE-9299] - Reuse Configuration in AvroSerdeUtils - * [HIVE-9300] - Make TCompactProtocol configurable - * [HIVE-9301] - Potential null dereference in MoveTask#createTargetPath() - * [HIVE-9309] - schematool fails on Postgres 8.1 - * [HIVE-9310] - CLI JLine does not flush history back to ~/.hivehistory - * [HIVE-9316] - TestSqoop tests in WebHCat testsuite hardcode libdir path to hdfs - * [HIVE-9317] - move Microsoft copyright to NOTICE file - * [HIVE-9321] - Notification message size can be arbitrarily long, DbNotificationListener limits to 1024 - * [HIVE-9330] - DummyTxnManager will throw NPE if WriteEntity writeType has not been set - * [HIVE-9331] - get rid of pre-optimized-hashtable memory optimizations - * [HIVE-9344] - Fix flaky test optimize_nullscan - * [HIVE-9347] - Bug with max() together with rank() and grouping sets - * [HIVE-9351] - Running Hive Jobs with Tez cause templeton to never report percent complete - * [HIVE-9353] - make TABLE keyword optional in INSERT INTO TABLE foo... - * [HIVE-9359] - Export of a large table causes OOM in Metastore and Client - * [HIVE-9361] - Intermittent NPE in SessionHiveMetaStoreClient.alterTempTable - * [HIVE-9366] - wrong date in description annotation in date_add() and date_sub() udf - * [HIVE-9369] - fix arguments length checking in Upper and Lower UDF - * [HIVE-9377] - UDF in_file() in WHERE predicate causes NPE. - * [HIVE-9381] - HCatalog hardcodes maximum append limit to 1000. - * [HIVE-9382] - Query got rerun with Global Limit optimization on and Fetch optimization off - * [HIVE-9386] - FileNotFoundException when using in_file() - * [HIVE-9390] - Enhance retry logic wrt DB access in TxnHandler - * [HIVE-9393] - reduce noisy log level of ColumnarSerDe.java:116 from INFO to DEBUG - * [HIVE-9396] - date_add()/date_sub() should allow tinyint/smallint/bigint arguments in addition to int - * [HIVE-9401] - Backport: Fastpath for limited fetches from unpartitioned tables - * [HIVE-9404] - NPE in org.apache.hadoop.hive.metastore.txn.TxnHandler.determineDatabaseProduct() - * [HIVE-9414] - Fixup post HIVE-9264 - Merge encryption branch to trunk - * [HIVE-9437] - Beeline does not add any existing HADOOP_CLASSPATH - * [HIVE-9440] - Folders may not be pruned for Hadoop 2 - * [HIVE-9441] - Remove call to deprecated Calcite method - * [HIVE-9443] - ORC PPD - fix fuzzy case evaluation of IS_NULL - * [HIVE-9445] - Revert HIVE-5700 - enforce single date format for partition column storage - * [HIVE-9446] - JDBC DatabaseMetadata.getColumns() does not work for temporary tables - * [HIVE-9448] - Merge spark to trunk 1/23/15 - * [HIVE-9454] - Test failures due to new Calcite version - * [HIVE-9462] - HIVE-8577 - breaks type evolution - * [HIVE-9473] - sql std auth should disallow built-in udfs that allow any java methods to be called - * [HIVE-9475] - HiveMetastoreClient.tableExists does not work - * [HIVE-9476] - Beeline fails to start on trunk - * [HIVE-9502] - Parquet cannot read Map types from files written with Hive <= 0.12 - * [HIVE-9514] - schematool is broken in hive 1.0.0 - * [HIVE-9519] - Bump up spark client connection timeout - * [HIVE-9593] - ORC Reader should ignore unknown metadata streams - * [HIVE-9610] - Continuation of HIVE-9438 - The standalone-jdbc jar missing some classes - * [HIVE-9611] - Allow SPARK_HOME as well as spark.home to define sparks location - * [HIVE-9621] - HiveServer2 http mode - embedded jetty should use SynchronousQueue - * [HIVE-9646] - Beeline doesn't show Spark job progress info [Spark Branch] - * [HIVE-9651] - FileSinkOperator does not pass in conf to serde.initialize - -** Improvement - * [HIVE-2828] - make timestamp accessible in the hbase KeyValue - * [HIVE-3187] - support ISO-2012 timestamp literals - * [HIVE-3280] - Make HiveMetaStoreClient a public API - * [HIVE-4639] - Add has null flag to ORC internal index - * [HIVE-4766] - Support HS2 client login timeout when the thrift thread max# is reached - * [HIVE-4809] - ReduceSinkOperator of PTFOperator can have redundant key columns - * [HIVE-5718] - Support direct fetch for lateral views, sub queries, etc. - * [HIVE-6148] - Support arbitrary structs stored in HBase - * [HIVE-7032] - Remove Triple Negative in Error 10129 - * [HIVE-7313] - Allow in-memory/ssd session-level temp-tables - * [HIVE-7408] - HCatPartition needs getPartCols method - * [HIVE-7550] - Extend cached evaluation to multiple expressions - * [HIVE-7605] - add more javadoc to HivePrivilegeObject - * [HIVE-7685] - Parquet memory manager - * [HIVE-7858] - Parquet compression should be configurable via table property - * [HIVE-7868] - AvroSerDe error handling could be improved - * [HIVE-7896] - orcfiledump should be able to dump data - * [HIVE-7977] - Avoid creating serde for partitions if possible in FetchTask - * [HIVE-8094] - add LIKE keyword support for SHOW FUNCTIONS - * [HIVE-8155] - In select statement after * any random characters are allowed in hive but in RDBMS its not allowed - * [HIVE-8337] - Change default of hive.warehouse.subdir.inherit.perms to true - * [HIVE-8347] - Use base-64 encoding instead of custom encoding for serialized objects - * [HIVE-8357] - Path type entities should use qualified path rather than string - * [HIVE-8395] - CBO: enable by default - * [HIVE-8424] - Support fair scheduler user queue mapping in non-impersonation mode - * [HIVE-8449] - webhcat startup does not create the log directory - * [HIVE-8469] - Add parquet.compression as a Serde Property - * [HIVE-8552] - Remove hard code of offline and make addHost api public - * [HIVE-8642] - Hive stack() UDTF Doesn't Support NULL Insert Values - * [HIVE-8661] - JDBC MinimizeJAR should be configurable in pom.xml - * [HIVE-8710] - Add more tests for transactional inserts - * [HIVE-8813] - Allow tests to be excluded based on pattern/regex - * [HIVE-8823] - Add additional serde properties for parquet - * [HIVE-8829] - Upgrade to Thrift 0.9.2 - * [HIVE-8839] - Support "alter table .. add/replace columns cascade" - * [HIVE-8910] - Refactoring of PassThroughOutputFormat - * [HIVE-8933] - Check release builds for SNAPSHOT dependencies - * [HIVE-8971] - HIVE-8965 exposed some classes which start with Test but are not tests - * [HIVE-8976] - Make nine additional tests deterministic - * [HIVE-8977] - TestParquetDirect should be abstract - * [HIVE-8983] - PTest Backup spark.log in addition to hive.log - * [HIVE-8988] - Support advanced aggregation in Hive to Calcite path - * [HIVE-8996] - Rename getUGIForConf - * [HIVE-9037] - Improve explain plan to show joining keys for shuffle join - * [HIVE-9049] - Metastore should use TCompactProtocol as opposed to TBinaryProtocol - * [HIVE-9062] - Explain plan doesn't print join keys for Tez shuffle join - * [HIVE-9106] - improve the performance of null scan optimizer when several table scans share a physical path - * [HIVE-9119] - ZooKeeperHiveLockManager does not use zookeeper in the proper way - * [HIVE-9140] - Add Calcite's ReduceExpressionRules to Hive - * [HIVE-9158] - Multiple LDAP server URLs in hive.server2.authentication.ldap.url - * [HIVE-9176] - Delegation token interval should be configurable in HadoopThriftAuthBridge - * [HIVE-9189] - Add ProjectRemove rule on CBO path - * [HIVE-9226] - Beeline interweaves the query result and query log sometimes - * [HIVE-9244] - Upgrade 0.23 hadoop-shims to latest stable hadoop-2.6.0 - * [HIVE-9314] - Writable object inspector should use Writable classes from Hadoop for get - * [HIVE-9318] - Add UnionMerge rule on cbo path - * [HIVE-9341] - Apply ColumnPrunning for noop PTFs - * [HIVE-9357] - Create ADD_MONTHS UDF - * [HIVE-9358] - Create LAST_DAY UDF - * [HIVE-9367] - CombineFileInputFormatShim#getDirIndices is expensive - * [HIVE-9372] - Parallel checking non-combinable paths in CombineHiveInputFormat - * [HIVE-9383] - Improve schema verification error message - * [HIVE-9538] - Exclude thirdparty directory from tarballs - * [HIVE-9549] - Include missing directories in source tarball - * [HIVE-9586] - Too verbose log can hurt performance, we should always check log level first - * [HIVE-9608] - Define SPARK_HOME if not defined automagically - -** New Feature - * [HIVE-3405] - UDF initcap to obtain a string with the first letter of each word in uppercase other letters in lowercase - * [HIVE-7122] - Storage format for create like table - * [HIVE-8435] - Add identity project remover optimization - * [HIVE-9174] - Enable queuing of HCatalog notification events in metastore DB - * [HIVE-9175] - Add alters to list of events handled by NotificationListener - * [HIVE-9184] - Modify HCatClient to support new notification methods in HiveMetaStoreClient - -** Task - * [HIVE-6977] - Delete Hiveserver1 - * [HIVE-7111] - Extend join transitivity PPD to non-column expressions - * [HIVE-8351] - Allow testing of both hadoop-1 and hadoop-2 for precommits - * [HIVE-8370] - Fix tabs in DefaultHBaseValueFactory - * [HIVE-8465] - Fix some minor test fails on trunk - * [HIVE-8608] - Move jenkins scripts to source control - * [HIVE-8609] - Move beeline to jline2 - * [HIVE-8826] - Remove jdbm from top level license file - * [HIVE-8828] - Remove hadoop 20 shims - * [HIVE-8894] - Move calcite.version to root pom - * [HIVE-8935] - Add debug logging around token stores - * [HIVE-8974] - Upgrade to Calcite 1.0.0-SNAPSHOT (with lots of renames) - * [HIVE-8979] - Merge shims/common-secure into shims/common - * [HIVE-8989] - Make groupby_multi_single_reducer.q and smb_mapjoin_3.q deterministic - * [HIVE-9183] - Compilation against HBase 1.0.0 fails due to missing dependency on high_scale_lib.Counter - * [HIVE-9304] - [Refactor] remove unused method in SemAly - * [HIVE-9345] - Add encryption branch to pre-commit script - * [HIVE-9402] - Create GREATEST and LEAST udf - * [HIVE-9407] - Fix spacing in pom - * [HIVE-9408] - Add hook interface so queries can be redacted before being placed in job.xml - * [HIVE-9527] - Include dot files in tarball - * [HIVE-9554] - Rename 0.15 upgrade scripts to 1.1 - * [HIVE-9672] - Update RELEASE_NOTES on trunk to reflect the 1.0.0 release - -** Test - * [HIVE-7924] - auto_sortmerge_join_8 sometimes fails with OOM - * [HIVE-8256] - Add SORT_QUERY_RESULTS for test that doesn't guarantee order #2 - * [HIVE-8327] - mvn site -Pfindbugs - * [HIVE-8573] - Fix some non-deterministic vectorization tests - * [HIVE-8601] - Fix auto_sortmerge_join_16 determinism - * [HIVE-8602] - Add SORT_QUERY_RESULTS for skewjoinopt2 - * [HIVE-8753] - TestMiniTezCliDriver.testCliDriver_vector_mapjoin_reduce failing on trunk - * [HIVE-8801] - Make orc_merge_incompat1.q deterministic across platforms - * [HIVE-8929] - incorrect error message for cbo path - * [HIVE-9091] - Add additional unit tests for HiveSQLException - * [HIVE-9173] - Fix auto_join29.q, auto_join_without_localtask.q determinism - * [HIVE-9290] - Make some test results deterministic - * [HIVE-9360] - TestSparkClient throws Timeoutexception - * [HIVE-9403] - File tests determinism with multiple reducers - * [HIVE-9417] - Fix failing test groupby_grouping_window.q on trunk - * [HIVE-9435] - Fix auto_join21.q for Tez - - - -Release Notes - Hive - Version 1.0.0 - -** Bug - * [HIVE-5631] - Index creation on a skew table fails - * [HIVE-5664] - Drop cascade database fails when the db has any tables with indexes - * [HIVE-6198] - ORC file and struct column names are case sensitive - * [HIVE-6468] - HS2 & Metastore using SASL out of memory error when curl sends a get request - * [HIVE-7270] - SerDe Properties are not considered by show create table Command - * [HIVE-8099] - IN operator for partition column fails when the partition column type is DATE - * [HIVE-8295] - Add batch retrieve partition objects for metastore direct sql - * [HIVE-8374] - schematool fails on Postgres versions < 9.2 - * [HIVE-8485] - HMS on Oracle incompatibility - * [HIVE-8706] - Table statistic collection on counter failed due to table name character case. - * [HIVE-8715] - Hive 14 upgrade scripts can fail for statistics if database was created using auto-create - * [HIVE-8739] - handle Derby and Oracle errors with joins and filters in Direct SQL in a invalid-DB-specific path - * [HIVE-8784] - Querying partition does not work with JDO enabled against PostgreSQL - * [HIVE-8805] - CBO skipped due to SemanticException: Line 0:-1 Both left and right aliases encountered in JOIN 'avg_cs_ext_discount_amt' - * [HIVE-8807] - Obsolete default values in webhcat-default.xml - * [HIVE-8811] - Dynamic partition pruning can result in NPE during query compilation - * [HIVE-8827] - Remove SSLv2Hello from list of disabled protocols - * [HIVE-8830] - hcatalog process don't exit because of non daemon thread - * [HIVE-8845] - Switch to Tez 0.5.2 - * [HIVE-8866] - Vectorization on partitioned table throws ArrayIndexOutOfBoundsException when partitions are not of same #of columns - * [HIVE-8870] - errors when selecting a struct field within an array from ORC based tables - * [HIVE-8873] - Switch to calcite 0.9.2 - * [HIVE-8876] - incorrect upgrade script for Oracle (13->14) - * [HIVE-8880] - non-synchronized access to split list in OrcInputFormat - * [HIVE-8886] - Some Vectorized String CONCAT expressions result in runtime error Vectorization: Unsuported vector output type: StringGroup - * [HIVE-8888] - Mapjoin with LateralViewJoin generates wrong plan in Tez - * [HIVE-8947] - HIVE-8876 also affects Postgres < 9.2 - * [HIVE-8966] - Delta files created by hive hcatalog streaming cannot be compacted - * [HIVE-9003] - Vectorized IF expr broken for the scalar and scalar case - * [HIVE-9025] - join38.q (without map join) produces incorrect result when testing with multiple reducers - * [HIVE-9038] - Join tests fail on Tez - * [HIVE-9051] - TezJobMonitor in-place updates logs too often to logfile - * [HIVE-9053] - select constant in union all followed by group by gives wrong result - * [HIVE-9067] - OrcFileMergeOperator may create merge file that does not match properties of input files - * [HIVE-9090] - Rename "Tez File Merge Work" to smaller name - * [HIVE-9108] - Fix for HIVE-8735 is incorrect (stats with long paths) - * [HIVE-9111] - Potential NPE in OrcStruct for list and map types - * [HIVE-9112] - Query may generate different results depending on the number of reducers - * [HIVE-9114] - union all query in cbo test has undefined ordering - * [HIVE-9126] - Backport HIVE-8827 (Remove SSLv2Hello from list of disabled protocols) to 0.14 branch - * [HIVE-9141] - HiveOnTez: mix of union all, distinct, group by generates error - * [HIVE-9155] - HIVE_LOCKS uses int instead of bigint hive-txn-schema-0.14.0.mssql.sql - * [HIVE-9162] - stats19 test is environment-dependant - * [HIVE-9166] - Place an upper bound for SARG CNF conversion - * [HIVE-9168] - Vectorized Coalesce for strings is broken - * [HIVE-9205] - Change default tez install directory to use /tmp instead of /user and create the directory if it does not exist - * [HIVE-9234] - HiveServer2 leaks FileSystem objects in FileSystem.CACHE - * [HIVE-9249] - java.lang.ClassCastException: org.apache.hadoop.hive.serde2.io.HiveVarcharWritable cannot be cast to org.apache.hadoop.hive.common.type.HiveVarchar when joining tables - * [HIVE-9278] - Cached expression feature broken in one case - * [HIVE-9317] - move Microsoft copyright to NOTICE file - * [HIVE-9359] - Export of a large table causes OOM in Metastore and Client - * [HIVE-9361] - Intermittent NPE in SessionHiveMetaStoreClient.alterTempTable - * [HIVE-9390] - Enhance retry logic wrt DB access in TxnHandler - * [HIVE-9401] - Backport: Fastpath for limited fetches from unpartitioned tables - * [HIVE-9404] - NPE in org.apache.hadoop.hive.metastore.txn.TxnHandler.determineDatabaseProduct() - * [HIVE-9436] - RetryingMetaStoreClient does not retry JDOExceptions - * [HIVE-9473] - sql std auth should disallow built-in udfs that allow any java methods to be called - * [HIVE-9514] - schematool is broken in hive 1.0.0 - - -** Improvement - * [HIVE-3280] - Make HiveMetaStoreClient a public API - * [HIVE-8933] - Check release builds for SNAPSHOT dependencies - - -** Task - * [HIVE-6977] - Delete Hiveserver1 - - - - -Release Notes - Hive - Version 0.14.0 - -** Sub-task - * [HIVE-4629] - HS2 should support an API to retrieve query logs - * [HIVE-5176] - Wincompat : Changes for allowing various path compatibilities with Windows - * [HIVE-5179] - Wincompat : change script tests from bash to sh - * [HIVE-5579] - Update statistics rules for different types of joins - * [HIVE-5760] - Add vectorized support for CHAR/VARCHAR data types - * [HIVE-5804] - Support column statistics for expressions in GBY attributes, JOIN condition etc. when annotating operator tree with statistics - * [HIVE-5805] - Support for operators like PTF, Script, Extract etc. in statistics annotation. - * [HIVE-5868] - Add statistics rule for Union operator - * [HIVE-5949] - In statistics annotation add flag to say if statistics is estimated or accurate - * [HIVE-5998] - Add vectorized reader for Parquet files - * [HIVE-6031] - explain subquery rewrite for where clause predicates - * [HIVE-6123] - Implement checkstyle in maven - * [HIVE-6252] - sql std auth - support 'with admin option' in revoke role metastore api - * [HIVE-6290] - Add support for hbase filters for composite keys - * [HIVE-6367] - Implement Decimal in ParquetSerde - * [HIVE-6394] - Implement Timestmap in ParquetSerde - * [HIVE-6445] - Add qop support for kerberos over http in HiveServer2 - * [HIVE-6626] - Hive does not expand the DOWNLOADED_RESOURCES_DIR path - * [HIVE-6627] - HiveServer2 should handle scratch dir permissions / errors in a better way - * [HIVE-6714] - Fix getMapSize() of LazyMap - * [HIVE-6735] - Make scalable dynamic partitioning work in vectorized mode - * [HIVE-6760] - Scalable dynamic partitioning should bail out properly for list bucketing - * [HIVE-6761] - Hashcode computation does not use maximum parallelism for scalable dynamic partitioning - * [HIVE-6798] - Update column stats based on filter expression in stats annotation - * [HIVE-6815] - Version of the HIVE-6374 for Hive 0.13 - * [HIVE-6982] - Export all .sh equivalent for windows (.cmd files) in bin, bin/ext - * [HIVE-6993] - Update hive for Tez VertexLocationHint and getAVailableResource API changes - * [HIVE-7029] - Vectorize ReduceWork - * [HIVE-7078] - Need file sink operators that work with ACID - * [HIVE-7094] - Separate out static/dynamic partitioning code in FileRecordWriterContainer - * [HIVE-7156] - Group-By operator stat-annotation only uses distinct approx to generate rollups - * [HIVE-7184] - TestHadoop20SAuthBridge no longer compiles after HADOOP-10448 - * [HIVE-7204] - Use NULL vertex location hint for Prewarm DAG vertices - * [HIVE-7262] - Partitioned Table Function (PTF) query fails on ORC table when attempting to vectorize - * [HIVE-7286] - Parameterize HCatMapReduceTest for testing against all Hive storage formats - * [HIVE-7291] - Refactor TestParser to understand test-property file - * [HIVE-7350] - Changes related to TEZ-692, TEZ-1169, TEZ-1234 - * [HIVE-7357] - Add vectorized support for BINARY data type - * [HIVE-7398] - Parent GBY of MUX is removed even it's not for semijoin - * [HIVE-7404] - Revoke privilege should support revoking of grant option - * [HIVE-7405] - Vectorize GROUP BY on the Reduce-Side (Part 1 â Basic) - * [HIVE-7420] - Parameterize tests for HCatalog Pig interfaces for testing against all storage formats - * [HIVE-7427] - Changes for EdgeConfigurations - * [HIVE-7457] - Minor HCatalog Pig Adapter test clean up - * [HIVE-7491] - Stats annotation fails to evaluate constant expressions in filter operator - * [HIVE-7513] - Add ROW__ID VirtualColumn - * [HIVE-7535] - Make use of number of nulls column statistics in filter rule - * [HIVE-7536] - Make use of decimal column statistics in statistics annotation - * [HIVE-7544] - Changes related to TEZ-1288 (FastTezSerialization) - * [HIVE-7548] - Precondition checks should not fail the merge task in case of automatic trigger - * [HIVE-7571] - RecordUpdater should read virtual columns from row - * [HIVE-7589] - Some fixes and improvements to statistics annotation rules - * [HIVE-7601] - Bring up tez-branch upto the API changes from TEZ-1058, TEZ-1303, TEZ-1346, TEZ-1041 - * [HIVE-7639] - Bring tez-branch upto api changes in TEZ-1379, TEZ-1057, TEZ-1382 - * [HIVE-7646] - Modify parser to support new grammar for Insert,Update,Delete - * [HIVE-7655] - CBO: Reading of partitioned table stats slows down explain - * [HIVE-7656] - Bring tez-branch up-to the API changes made by TEZ-1372 - * [HIVE-7663] - OrcRecordUpdater needs to implement getStats - * [HIVE-7679] - JOIN operator should update the column stats when number of rows changes - * [HIVE-7734] - Join stats annotation rule is not updating columns statistics correctly - * [HIVE-7735] - Implement Char, Varchar in ParquetSerDe - * [HIVE-7788] - Generate plans for insert, update, and delete - * [HIVE-7790] - Update privileges to check for update and delete - * [HIVE-7808] - Changes to work against Tez-0.5 RC - * [HIVE-7809] - Fix ObjectRegistry to work with Tez 0.5 - * [HIVE-7820] - union_null.q is not deterministic - * [HIVE-7825] - Bring tez-branch up-to the API changes made by TEZ-1472, TEZ-1469 - * [HIVE-7836] - Ease-out denominator for multi-attribute join case in statistics annotation - * [HIVE-7864] - [CBO] Query fails if it refers only partitioning column - * [HIVE-7869] - Build long running HS2 test framework - * [HIVE-7904] - Missing null check cause NPE when updating join column stats in statistics annotation - * [HIVE-7905] - CBO: more cost model changes - * [HIVE-7907] - Bring up tez branch to changes in TEZ-1038, TEZ-1500 - * [HIVE-7935] - Support dynamic service discovery for HiveServer2 - * [HIVE-7979] - Fix testconfiguration.property file in Tez branch - * [HIVE-7990] - With fetch column stats disabled number of elements in grouping set is not taken into account - * [HIVE-7991] - Incorrect calculation of number of rows in JoinStatsRule.process results in overflow - * [HIVE-7992] - StatsRulesProcFactory should gracefully handle overflows - * [HIVE-7994] - BMJ test fails on tez - * [HIVE-7995] - Column statistics from expression does not handle fields within complex types - * [HIVE-8006] - CBO Trunk Merge: Test fail that includes Table Sample, rows(), query hints - * [HIVE-8016] - CBO: PPD to honor hive Join Cond, Casting fixes, Add annotations for IF, Code cleanup - * [HIVE-8021] - CBO: support CTAS and insert ... select - * [HIVE-8046] - CBO: fix issues with Windowing queries - * [HIVE-8069] - CBO: RowResolver after SubQuery predicate handling should be reset to outer query block RR - * [HIVE-8111] - CBO trunk merge: duplicated casts for arithmetic expressions in Hive and CBO - * [HIVE-8125] - CBO Trunk Merge: On Failure Fall Back to Non CBO - * [HIVE-8144] - CBO: HiveProjectRel factory should create RelSubSets - * [HIVE-8145] - CBO: bail from Optiq planning if a Select list contains multiple references to the same name - * [HIVE-8159] - CBO: bail from Optiq planning if a Select list contains multiple references to the same name - * [HIVE-8168] - With dynamic partition enabled fact table selectivity is not taken into account when generating the physical plan (Use CBO cardinality using physical plan generation) - * [HIVE-8172] - HiveServer2 dynamic service discovery should let the JDBC client use default ZooKeeper namespace - * [HIVE-81 <TRUNCATED>
