http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/3be0f122/docs/topics/impala_fixed_issues.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_fixed_issues.xml b/docs/topics/impala_fixed_issues.xml new file mode 100644 index 0000000..f55d95c --- /dev/null +++ b/docs/topics/impala_fixed_issues.xml @@ -0,0 +1,7092 @@ +<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE concept PUBLIC "-//OASIS//DTD DITA Concept//EN" "concept.dtd"> +<concept rev="ver" id="fixed_issues"> + + <title><ph audience="standalone">Fixed Issues in Apache Impala (incubating)</ph><ph audience="integrated">Fixed Issues in Apache Impala (incubating)</ph></title> + <prolog> + <metadata> + <data name="Category" value="Impala"/> + <data name="Category" value="Release Notes"/> + <data name="Category" value="Fixed Issues"/> + <data name="Category" value="Troubleshooting"/> + <data name="Category" value="Upgrading"/> + <data name="Category" value="Data Analysts"/> + <data name="Category" value="Developers"/> + <data name="Category" value="Data Analysts"/> + </metadata> + </prolog> + + <conbody> + + <p> + The following sections describe the major issues fixed in each Impala release. + </p> + + <p> + For known issues that are currently unresolved, see <xref href="impala_known_issues.xml#known_issues"/>. + </p> + + <p outputclass="toc inpage"/> + </conbody> + +<!-- All 2.7.x / 5.9.x subsections go under here --> + + <concept rev="2.7.0" id="fixed_issues_270"> + + <title>Issues Fixed in Impala for CDH 5.9.0</title> + + <conbody> + + <p> +<!-- + The following list contains the most critical fixed issues + (<codeph>priority='Blocker'</codeph>) from the JIRA system. +--> + For the full list of Impala fixed issues in Impala 2.7.0, see + <xref href="https://issues.cloudera.org/issues/?jql=%20type%20%3D%20bug%20and%20project%20%3D%20IMPALA%20AND%20resolution%20%3D%20fixed%20AND%20affectedVersion%20!%3D%20%22Impala%202.7.0%22%20AND%20fixVersion%20%3D%20%22Impala%202.7.0%22%20and%20not%20labels%20%3D%20broken-build%20order%20by%20priority%20desc"; scope="external" format="html">this report in the Impala JIRA tracker</xref>. + </p> + + <p> For the full list of fixed issues for all CDH components in CDH 5.9.0, see <xref + audience="standalone" + href="http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_fixed_in_59.html"; + format="html" scope="external">the CDH 5.9.x release notes</xref><xref + audience="integrated" href="cdh_rn_fixed_in_59.xml" + />. </p> + <ul audience="integrated"> + <li conref="cdh_rn_fixed_in_59.xml#ui_fixed_in_59/impala_start" + conrefend="cdh_rn_fixed_in_59.xml#ui_fixed_in_59/impala_end" /> + </ul> + + </conbody> + </concept> + +<!-- All 2.6.x / 5.8.x subsections go under here --> + + <concept rev="2.6.3" id="fixed_issues_263"> + <title>Issues Fixed in Impala for CDH 5.8.3</title> + <conbody> + <p> For the full list of fixed issues for all CDH components, see <xref + audience="standalone" + href="http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_fixed_in_58.html"; + format="html" scope="external">the CDH 5.8.x release notes</xref><xref + audience="integrated" href="cdh_rn_fixed_in_58.xml#ui_fixed_in_583" + />. </p> + <ul audience="integrated"> + <li conref="cdh_rn_fixed_in_58.xml#ui_fixed_in_583/impala_start" + conrefend="cdh_rn_fixed_in_58.xml#ui_fixed_in_583/impala_end" /> + </ul> + </conbody> + </concept> + + <concept rev="2.6.2" id="fixed_issues_262"> + <title>Issues Fixed in Impala for CDH 5.8.2</title> + <conbody> + <p> For the full list of fixed issues for all CDH components, see <xref + audience="standalone" + href="http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_fixed_in_58.html"; + format="html" scope="external">the CDH 5.8.x release notes</xref><xref + audience="integrated" href="cdh_rn_fixed_in_58.xml#upstream582" + />. </p> + <ul audience="integrated"> + <li conref="cdh_rn_fixed_in_58.xml#upstream582/impala_start" + conrefend="cdh_rn_fixed_in_58.xml#upstream582/impala_end" /> + </ul> + </conbody> + </concept> + + <concept rev="2.6.0" id="fixed_issues_260"> + + <title>Issues Fixed in Impala for CDH 5.8.0</title> + + <conbody> + + <p> + The following list contains the most critical fixed issues + (<codeph>priority='Blocker'</codeph>) from the JIRA system. + For the full list of fixed issues in CDH 5.8.0 / Impala 2.6.0, see + <xref href="https://issues.cloudera.org/issues/?jql=%20type%20%3D%20bug%20and%20project%20%3D%20IMPALA%20AND%20resolution%20%3D%20fixed%20AND%20affectedVersion%20!%3D%20%22Impala%202.6.0%22%20AND%20fixVersion%20%3D%20%22Impala%202.6.0%22%20and%20not%20labels%20%3D%20broken-build%20order%20by%20priority%20desc"; scope="external" format="html">this report in the Impala JIRA tracker</xref>. + </p> + + </conbody> + + <concept id="IMPALA-3385"> + <title>RuntimeState::error_log_ crashes</title> + <conbody> + <p> + A crash could occur, with stack trace pointing to <codeph>impala::RuntimeState::ErrorLog</codeph>. + </p> + <p><b>Bug:</b> <xref scope="external" format="html" href="https://issues.cloudera.org/browse/IMPALA-3385";>IMPALA-3385</xref></p> + <p><b>Severity:</b> High</p> + </conbody> + </concept> + + <concept id="IMPALA-3378"> + <title>HiveUdfCall::Open() produces unsynchronized access to JniUtil::global_refs_ vector</title> + <conbody> + <p> + A crash could occur because of contention between multiple calls to Java UDFs. + </p> + <p><b>Bug:</b> <xref scope="external" format="html" href="https://issues.cloudera.org/browse/IMPALA-3378";>IMPALA-3378</xref></p> + <p><b>Severity:</b> High</p> + </conbody> + </concept> + + <concept id="IMPALA-3379"> + <title>HBaseTableWriter::CreatePutList() produces unsynchronized access to JniUtil::global_refs_ vector</title> + <conbody> + <p> + A crash could occur because of contention between multiple concurrent statements writing to HBase. + </p> + <p><b>Bug:</b> <xref scope="external" format="html" href="https://issues.cloudera.org/browse/IMPALA-3379";>IMPALA-3379</xref></p> + <p><b>Severity:</b> High</p> + </conbody> + </concept> + + <concept id="IMPALA-3317"> + <title>Stress test failure: sorter.cc:745] Check failed: i == 0 (1 vs. 0) </title> + <conbody> + <p> + A crash or wrong results could occur if the spill-to-disk mechanism encountered a zero-length string at + the very end of a data block. + </p> + <p><b>Bug:</b> <xref scope="external" format="html" href="https://issues.cloudera.org/browse/IMPALA-3317";>IMPALA-3317</xref></p> + <p><b>Severity:</b> High</p> + </conbody> + </concept> + + <concept id="IMPALA-3311"> + <title>String data coming out of agg can be corrupted by blocking operators</title> + <conbody> + <p> + If a query plan contains an aggregation node producing string values anywhere within a subplan + (that is,if in the SQL statement, the aggregate function appears within an inline view over a collection column), + the results of the aggregation may be incorrect. + </p> + <p><b>Bug:</b> <xref scope="external" format="html" href="https://issues.cloudera.org/browse/IMPALA-3311";>IMPALA-3311</xref></p> + <p><b>Severity:</b> High</p> + </conbody> + </concept> + + <concept id="IMPALA-3269"> + <title>CTAS with subquery throws AuthzException</title> + <conbody> + <p> + A <codeph>CREATE TABLE AS SELECT</codeph> operation could fail with an authorization error, + due to a slight difference in the privilege checking for the CTAS operation. + </p> + <p><b>Bug:</b> <xref scope="external" format="html" href="https://issues.cloudera.org/browse/IMPALA-3269";>IMPALA-3269</xref></p> + <p><b>Severity:</b> High</p> + </conbody> + </concept> + + <concept id="IMPALA-3237"> + <title>Crash on inserting into table with binary and parquet</title> + <conbody> + <p> + Impala incorrectly allowed <codeph>BINARY</codeph> to be specified as a column type, + resulting in a crash during a write to a Parquet table with a column of that type. + </p> + <p><b>Bug:</b> <xref scope="external" format="html" href="https://issues.cloudera.org/browse/IMPALA-3237";>IMPALA-3237</xref></p> + <p><b>Severity:</b> High</p> + </conbody> + </concept> + + <concept id="IMPALA-3105"> + <title>RowBatch::MaxTupleBufferSize() calculation incorrect, may lead to memory corruption</title> + <conbody> + <p> + A crash could occur while querying tables with very large rows, for example wide tables with many + columns or very large string values. This problem was identified in Impala 2.3, but had low + reproducibility in subsequent releases. The fix ensures the memory allocation size is correct. + </p> + <p><b>Bug:</b> <xref scope="external" format="html" href="https://issues.cloudera.org/browse/IMPALA-3105";>IMPALA-3105</xref></p> + <p><b>Severity:</b> High</p> + </conbody> + </concept> + + <concept id="IMPALA-3494"> + <title>Thrift buffer overflows when serialize more than 3355443200 bytes in impala</title> + <conbody> + <p> + A very large memory allocation within the <cmdname>catalogd</cmdname> daemon could exceed an internal Thrift limit, + causing a crash. + </p> + <p><b>Bug:</b> <xref scope="external" format="html" href="https://issues.cloudera.org/browse/IMPALA-3494";>IMPALA-3494</xref></p> + <p><b>Severity:</b> High</p> + </conbody> + </concept> + + <concept id="IMPALA-3314"> + <title>Altering table partition's storage format is not working and crashing the daemon</title> + <conbody> + <p> + If a partitioned table used a file format other than Avro, and the file format of an individual partition + was changed to Avro, subsequent queries could encounter a crash. + </p> + <p><b>Bug:</b> <xref scope="external" format="html" href="https://issues.cloudera.org/browse/IMPALA-3314";>IMPALA-3314</xref></p> + <p><b>Severity:</b> High</p> + </conbody> + </concept> + + <concept id="IMPALA-3798"> + <title>Race condition may cause scanners to spin with runtime filters on Avro or Sequence files</title> + <conbody> + <p> + A timing problem during runtime filter processing could cause queries against Avro or SequenceFile tables + to hang. + </p> + <p><b>Bug:</b> <xref scope="external" format="html" href="https://issues.cloudera.org/browse/IMPALA-3798";>IMPALA-3798</xref></p> + <p><b>Severity:</b> High</p> + </conbody> + </concept> + + </concept> +<!-- All 2.5.x / 5.7.x subsections go under here --> + + <concept rev="2.5.4" id="fixed_issues_254"> + <title>Issues Fixed in Impala for CDH 5.7.4</title> + <conbody> + <p> For the full list of fixed issues for all CDH components, see <xref + audience="standalone" + href="http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_fixed_in_57.html"; + format="html" scope="external">the CDH 5.7.x release notes</xref><xref + audience="integrated" href="cdh_rn_fixed_in_57.xml#ui_fixed_in_574" + />. </p> + <ul audience="integrated"> + <li conref="cdh_rn_fixed_in_57.xml#ui_fixed_in_574/impala_start" + conrefend="cdh_rn_fixed_in_57.xml#ui_fixed_in_574/impala_end" /> + </ul> + </conbody> + </concept> + +<!-- + <concept rev="2.5.3" id="fixed_issues_253" audience="pre-release"> + + <title>Issues Fixed in Impala for CDH 5.7.3</title> + + <conbody> + + <p> + For the full list of fixed issues for all CDH components, see + <xref audience="standalone" + href="http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_fixed_in_57.html#ui_fixed_in_573"; + format="html" scope="external">the CDH 5.7.x release notes</xref><xref + audience="integrated" href="cdh_rn_fixed_in_57.xml#ui_fixed_in_573"/>. + </p> + + <ul audience="integrated"> + <li conref="cdh_rn_fixed_in_57.xml#ui_fixed_in_573/impala_start" conrefend="cdh_rn_fixed_in_57.xml#ui_fixed_in_573/impala_end"/> + </ul> + + </conbody> + + </concept> +--> + + <concept rev="2.5.2" id="fixed_issues_252"> + <title>Issues Fixed in Impala for CDH 5.7.2</title> + <conbody> + <p> For the full list of fixed issues for all CDH components, see <xref + audience="standalone" + href="http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_fixed_in_57.html#ui_fixed_in_572"; + format="html" scope="external">the CDH 5.7.x release notes</xref><xref + audience="integrated" href="cdh_rn_fixed_in_57.xml#ui_fixed_in_572" + />. </p> + <ul audience="integrated"> + <li conref="cdh_rn_fixed_in_57.xml#ui_fixed_in_572/impala_start" + conrefend="cdh_rn_fixed_in_57.xml#ui_fixed_in_572/impala_end" /> + </ul> + </conbody> + </concept> + + <concept rev="2.5.1" id="fixed_issues_251"> + + <title>Issues Fixed in Impala for CDH 5.7.1</title> + + <conbody> + + <p> + For the full list of fixed issues for all CDH components, see + <xref audience="standalone" + href="http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_fixed_in_57.html#ui_fixed_in_571"; + format="html" scope="external">the CDH 5.7.x release notes</xref><xref + audience="integrated" href="cdh_rn_fixed_in_57.xml#ui_fixed_in_571"/>. + </p> + + <ul audience="integrated"> + <li conref="cdh_rn_fixed_in_57.xml#ui_fixed_in_571/impala_start" conrefend="cdh_rn_fixed_in_57.xml#ui_fixed_in_571/impala_end"/> + </ul> + + </conbody> + + </concept> + + <concept rev="2.5.0" id="fixed_issues_250"> + + <title>Issues Fixed in Impala for CDH 5.7.0</title> + + <conbody> + + <p> + The following list contains the most critical issues (<codeph>priority='Blocker'</codeph>) from the JIRA system. + For the full list of fixed issues in CDH 5.7.0 / Impala 2.5.0, see + <xref href="https://issues.cloudera.org/issues/?jql=%20type%20%3D%20bug%20and%20project%20%3D%20IMPALA%20AND%20resolution%20%3D%20fixed%20AND%20affectedVersion%20!%3D%20%22Impala%202.5.0%22%20AND%20fixVersion%20%3D%20%22Impala%202.5.0%22%20and%20not%20labels%20%3D%20broken-build%20order%20by%20priority%20desc"; scope="external" format="html">this report in the Impala JIRA tracker</xref>. + </p> + + </conbody> + + <concept id="IMPALA-2683"> + <title>Stress test hit assert in LLVM: external function could not be resolved</title> + <conbody> +<p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2683"; scope="external" format="html">IMPALA-2683</xref></p> +<p>The stress test was running a build with the TPC-H, TPC-DS, and TPC-H nested queries with scale factor 3.</p> + </conbody> + </concept> + + <concept id="IMPALA-2365"> + <title>Impalad is crashing if udf jar is not available in hdfs location for first time</title> + <conbody> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2365"; scope="external" format="html">IMPALA-2365</xref></p> + <p> + If a UDF JAR was not available in the HDFS location specified in the <codeph>CREATE FUNCTION</codeph> statement, + the <cmdname>impalad</cmdname> daemon could crash. + </p> + </conbody> + </concept> + + <concept id="IMPALA-2535-570"> + <title>PAGG hits mem_limit when switching to I/O buffers</title> + <conbody> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2535"; scope="external" format="html">IMPALA-2535</xref></p> + <p> + A join query could fail with an out-of-memory error despite the apparent presence of sufficient memory. + The cause was the internal ordering of operations that could cause a later phase of the query to + allocate memory required by an earlier phase of the query. The workaround was to either increase + or decrease the <codeph>MEM_LIMIT</codeph> query option, because the issue would only occur for a specific + combination of memory limit and data volume. + </p> + </conbody> + </concept> + + <concept id="IMPALA-2643-570"> + <title>Prevent migrating incorrectly inferred identity predicates into inline views</title> + <conbody> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2643"; scope="external" format="html">IMPALA-2643</xref></p> + <p> + Referring to the same column twice in a view definition could cause the view to omit + rows where that column contained a <codeph>NULL</codeph> value. This could cause + incorrect results due to an inaccurate <codeph>COUNT(*)</codeph> value or rows missing + from the result set. + </p> + </conbody> + </concept> + + <concept id="IMPALA-1459-570"> + <title>Fix migration/assignment of On-clause predicates inside inline views</title> + <conbody> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-1459"; scope="external" format="html">IMPALA-1459</xref></p> + <p> + Some combinations of <codeph>ON</codeph> clauses in join queries could result in comparisons + being applied at the wrong stage of query processing, leading to incorrect results. + Wrong predicate assignment could happen under the following conditions: + </p> + <ul> + <li> + The query includes an inline view that contains an outer join. + </li> + <li> + That inline view is joined with another table in the enclosing query block. + </li> + <li> + That join has an <codeph>ON</codeph> clause containing a predicate that + only references columns originating from the outer-joined tables inside the inline view. + </li> + </ul> + </conbody> + </concept> + + <concept id="IMPALA-2093"> + <title>Wrong plan of NOT IN aggregate subquery when a constant is used in subquery predicate</title> + <conbody> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2093"; scope="external" format="html">IMPALA-2093</xref></p> + <p> + <codeph>IN</codeph> subqueries might return wrong results if the left-hand side of the <codeph>IN</codeph> is a constant. + For example: + </p> +<codeblock> +select * from alltypestiny t1 + where 10 not in (select sum(int_col) from alltypestiny); +</codeblock> + </conbody> + </concept> + + <concept id="IMPALA-2940"> + <title>Parquet DictDecoders accumulate throughout query</title> + <conbody> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2940"; scope="external" format="html">IMPALA-2940</xref></p> + <p> + Parquet dictionary decoders can accumulate throughout query execution, leading to excessive memory usage. One decoder is created per-column per-split. + </p> + </conbody> + </concept> + + <concept id="IMPALA-3056"> + <title>Planner doesn't set the has_local_target field correctly</title> + <conbody> +<p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-3056"; scope="external" format="html">IMPALA-3056</xref></p> + + </conbody> + </concept> + + <concept id="IMPALA-2742"> + <title>MemPool allocation growth behavior</title> + <conbody> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2742"; scope="external" format="html">IMPALA-2742</xref></p> + <p> + Currently, the MemPool would always double the size of the last allocation. + This can lead to bad behavior if the MemPool transferred the ownership of all its data + except the last chunk. In the next allocation, the next allocated chunk would double + the size of this large chunk, which can be undesirable. + </p> + </conbody> + </concept> + + <concept id="IMPALA-3035"> + <title>Drop partition operations don't follow the catalog's locking protocol</title> + <conbody> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-3035"; scope="external" format="html">IMPALA-3035</xref></p> + <p> + The <codeph>CatalogOpExecutor.alterTableDropPartition()</codeph> function violates + the locking protocol used in the catalog that requires <codeph>catalogLock_</codeph> + to be acquired before any table-level lock. That may cause deadlocks when <codeph>ALTER TABLE DROP PARTITION</codeph> + is executed concurrently with other DDL operations. + </p> + </conbody> + </concept> + + <concept id="IMPALA-2215"> + <title>HAVING clause without aggregation not applied properly</title> + <conbody> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2215"; scope="external" format="html">IMPALA-2215</xref></p> + <p> + A query with a <codeph>HAVING</codeph> clause but no <codeph>GROUP BY</codeph> clause was not being rejected, + despite being invalid syntax. For example: + </p> + +<codeblock> +select case when 1=1 then 'didit' end as c1 from (select 1 as one) a having 1!=1; +</codeblock> + </conbody> + </concept> + + <concept id="IMPALA-2914"> + <title>Hit DCHECK Check failed: HasDateOrTime()</title> + <conbody> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2914"; scope="external" format="html">IMPALA-2914</xref></p> + <p> + <codeph>TimestampValue::ToTimestampVal()</codeph> requires a valid <codeph>TimestampValue</codeph> as input. + This requirement was not enforced in some places, leading to serious errors. + </p> + </conbody> + </concept> + + <concept id="IMPALA-2986"> + <title>Aggregation spill loop gives up too early leading to mem limit exceeded errors</title> + <conbody> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2986"; scope="external" format="html">IMPALA-2986</xref></p> + <p> + An aggregation query could fail with an out-of-memory error, despite sufficient memory being reported as available. + </p> + </conbody> + </concept> + + <concept id="IMPALA-2592"> + <title>DataStreamSender::Channel::CloseInternal() does not close the channel on an error.</title> + <conbody> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2592"; scope="external" format="html">IMPALA-2592</xref></p> + <p> + Some queries do not close an internal communication channel on an error. + This will cause the node on the other side of the channel to wait indefinitely, causing the query to hang. + For example, this issue could happen on a Kerberos-enabled system if the credential cache was outdated. + Although the affected query hangs, the <cmdname>impalad</cmdname> daemons continue processing other queries. + </p> + </conbody> + </concept> + + <concept id="IMPALA-2184"> + <title>Codegen does not catch exceptions in FROM_UNIXTIME()</title> + <conbody> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2184"; scope="external" format="html">IMPALA-2184</xref></p> + <p> + Querying for the min or max value of a timestamp cast from a bigint via <codeph>from_unixtime()</codeph> + fails silently and crashes instances of <cmdname>impalad</cmdname> when the input includes a value outside of the valid range. + </p> + + <p><b>Workaround:</b> Disable native code generation with:</p> +<codeblock> +SET disable_codegen=true; +</codeblock> + </conbody> + </concept> + + <concept id="IMPALA-2788"> + <title>Impala returns wrong result for function 'conv(bigint, from_base, to_base)'</title> + <conbody> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2788"; scope="external" format="html">IMPALA-2788</xref></p> + <p> + Impala returns wrong result for function <codeph>conv()</codeph>. + Function <codeph>conv(bigint, from_base, to_base)</codeph> returns an correct result, + while <codeph>conv(string, from_base, to_base)</codeph> returns the correct value. + For example: + </p> + +<codeblock> +<![CDATA[ +select 2061013007, conv(2061013007, 16, 10), conv('2061013007', 16, 10); ++------------+--------------------------+----------------------------+ +| 2061013007 | conv(2061013007, 16, 10) | conv('2061013007', 16, 10) | ++------------+--------------------------+----------------------------+ +| 2061013007 | 1627467783 | 139066421255 | ++------------+--------------------------+----------------------------+ +Fetched 1 row(s) in 0.65s + +select 2061013007, conv(cast(2061013007 as bigint), 16, 10), conv('2061013007', 16, 10); ++------------+------------------------------------------+----------------------------+ +| 2061013007 | conv(cast(2061013007 as bigint), 16, 10) | conv('2061013007', 16, 10) | ++------------+------------------------------------------+----------------------------+ +| 2061013007 | 1627467783 | 139066421255 | ++------------+------------------------------------------+----------------------------+ + +select 2061013007, conv(cast(2061013007 as string), 16, 10), conv('2061013007', 16, 10); ++------------+------------------------------------------+----------------------------+ +| 2061013007 | conv(cast(2061013007 as string), 16, 10) | conv('2061013007', 16, 10) | ++------------+------------------------------------------+----------------------------+ +| 2061013007 | 139066421255 | 139066421255 | ++------------+------------------------------------------+----------------------------+ + +select 2061013007, conv(cast(cast(2061013007 as decimal(20,0)) as bigint), 16, 10), conv('2061013007', 16, 10); ++------------+-----------------------------------------------------------------+----------------------------+ +| 2061013007 | conv(cast(cast(2061013007 as decimal(20,0)) as bigint), 16, 10) | conv('2061013007', 16, 10) | ++------------+-----------------------------------------------------------------+----------------------------+ +| 2061013007 | 1627467783 | 139066421255 | ++------------+-----------------------------------------------------------------+----------------------------+ +]]> +</codeblock> + + <p><b>Workaround:</b> + Cast the value to string and use <codeph>conv(string, from_base, to_base)</codeph> for conversion. + </p> + </conbody> + </concept> + +<!-- I think this issue is marked incorrectly in JIRA. It arose during the 5.7 development cycle, it's not a + customer-facing issue. + <concept id="IMPALA-3039"> + <title>Restrict the number of runtime filters generated</title> + <conbody> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-3039"; scope="external" format="html">IMPALA-3039</xref></p> + <p> + Large queries with many runtime filters might fail for various reasons, such as memory for Thrift data structures + not being accounted for. + Having to disable all runtime filters to make a query run is not desirable. + The new <codeph>MAX_NUM_RUNTIME_FILTERS</codeph> query option specifies an upper bound + on the number of runtime filters generated for a query. + </p> + </conbody> + </concept> +--> + + </concept> + +<!-- All 2.4.x / 5.6.x subsections go under here --> + + <concept rev="2.4.1" id="fixed_issues_241"> + + <title>Issues Fixed in Impala for CDH 5.6.1</title> + + <conbody> + + <p> + For the full list of fixed issues for all CDH components, see + <xref audience="standalone" + href="http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_fixed_in_56.html#ui_fixed_in_561"; + format="html" scope="external">the CDH 5.6.x release notes</xref><xref + audience="integrated" href="cdh_rn_fixed_in_56.xml#ui_fixed_in_561"/>. + </p> + + <!-- Maybe time to start dropping the 'CDH 5 only' notes... wasn't included for 5.6.0, so why start for 5.6.1... + <note conref="../shared/impala_common.xml#common/only_cdh5_24x"/> --> + + <ul audience="integrated"> + <li conref="cdh_rn_fixed_in_56.xml#ui_fixed_in_561/impala_start" conrefend="cdh_rn_fixed_in_56.xml#ui_fixed_in_561/impala_end"/> + </ul> + + </conbody> + + </concept> + + <concept rev="2.4.0" id="fixed_issues_240"> + + <title>Issues Fixed in Impala for CDH 5.6.0</title> + + <conbody> + + <p> The set of fixes for Impala in CDH 5.6.0 is the same as in CDH 5.5.2. + See <xref href="impala_fixed_issues.xml#fixed_issues_232" + audience="integrated"/> for details. </p> + +<!-- + <p> + This section lists the most serious or frequently encountered customer issues fixed in CDH 5.6.0 / Impala 2.4.0. + For the full list of fixed Impala issues, see + <xref audience="standalone" href="http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_fixed_in_560.html"; format="html" scope="external">the CDH 5.6.x release notes</xref><xref audience="integrated" href="cdh_rn_fixed_in_560.xml"/>. + </p> +--> + + </conbody> + + </concept> + +<!-- All 2.3.x / 5.5.x subsections go under here --> + + <concept rev="2.3.4" id="fixed_issues_234"> + + <title>Issues Fixed in Impala for CDH 5.5.4</title> + + <conbody> + + <p> + For the full list of fixed issues for all CDH components, see + <xref audience="standalone" href="http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_fixed_in_55.html#fixed_issues_554"; format="html" scope="external">the CDH 5.5.x release notes</xref><xref audience="integrated" href="cdh_rn_fixed_in_55.xml#fixed_issues_554"/>. + </p> + + <note conref="../shared/impala_common.xml#common/only_cdh5_23x"/> + + <ul audience="integrated"> + <li conref="cdh_rn_fixed_in_55.xml#ui_fixed_in_554/impala_start" conrefend="cdh_rn_fixed_in_55.xml#ui_fixed_in_554/impala_end"/> + </ul> + + </conbody> + + </concept> + + <concept rev="2.3.2" id="fixed_issues_232"> + + <title>Issues Fixed in Impala for CDH 5.5.2</title> + + <conbody> + + <p> + This section lists the most serious or frequently encountered customer + issues fixed in CDH 5.5.2 / Impala 2.3.2. For the full list of fixed Impala issues, + see <xref audience="standalone" href="http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_fixed_in_552.html"; format="html" scope="external">the CDH 5.5.x release notes</xref><xref href="cdh_rn_fixed_in_55.xml#fixed_issues_552" audience="integrated"/>. + </p> + + </conbody> + +<!-- +Fixed issues in 5.5.2 from Juan Yu spreadsheet (https://docs.google.com/spreadsheets/d/1Ia06xOLKQInKfsJzQfQ6Sf1fO3nYLOpZ-0OWfFNgNSc/edit#gid=2094272728): + +IMPALA-2747: Thrift-client cleans openSSL state before using it in the case of the catalog +IMPALA-2829: SEGV in AnalyticEvalNode touching NULL input_stream_ +IMPALA-2722: Free local allocations per row batch in non-partitioned AGG and HJ +IMPALA-2612: Free local allocations once for every row batch when building hash tables +IMPALA-2643: Prevent migrating incorrectly inferred identity predicates into inline views +IMPALA-2695: Fix GRANTs on URIs with uppercase letters +IMPALA-2664: Avoid sending large partition stats objects over thrift +IMPALA-2648: Ditto +IMPALA-2226: Throw AnalysisError if table properties are too large (for the Hive metastore) +IMPALA-2273: Make MAX_PAGE_HEADER_SIZE configurable +IMPALA-2473: reduce scanner memory usage +IMPALA-2113: Handle error when distinct and aggregates are used with a having clause +IMPALA-2225: Handle error when star based select item and aggregate are incorrectly used +IMPALA-2731: Refactor MemPool usage in HBase scan node +IMPALA-1459: Fix migration/assignment of On-clause predicates inside inline views +IMPALA-2558: DCHECK in parquet scanner after block read error +IMPALA-2535: PAGG hits mem_limit when switching to I/O buffers +IMPALA-2559: Fix check failed: sorter_runs_.back()->is_pinned_ +IMPALA-2614: Don't ignore Status returned by DataStreamRecvr::CreateMerger() +IMPALA-2591: DataStreamSender::Send() does not return an error status if SendBatch() failed +IMPALA-2598: Re-enable SSL and Kerberos on server-server + +These are changes to test cases and so ignored for release note purposes: + +IMPALA-2103: Fix flaky test_impersonation test +IMPALA-2776: Remove escapechartesttable and associated tests. +IMPALA-2812: Remove additional test referencing escapecharstesttable. +IMPALA-2630: skip TestParquet.test_continue_on_error when using old aggs/joins +IMPALA-2624: Increase fs.trash.interval to 24 hours for test suite +--> + + <concept id="IMPALA-2829"> + <title>SEGV in AnalyticEvalNode touching NULL input_stream_</title> + <conbody> + <p> + A query involving an analytic function could encounter a serious error. + This issue was encountered infrequently, depending upon specific combinations + of queries and data. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2829"; scope="external" format="html">IMPALA-2829</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2722"> + <title>Free local allocations per row batch in non-partitioned AGG and HJ</title> + <conbody> + <p> + An outer join query could fail unexpectedly with an out-of-memory error + when the <q>spill to disk</q> mechanism was turned off. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2722"; scope="external" format="html">IMPALA-2722</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2612"> + <!-- In this case, the title from the spreadsheet is clearer than the original JIRA title. --> + <title>Free local allocations once for every row batch when building hash tables</title> + <conbody> + <p> + A join query could encounter a serious error due to an internal failure to allocate memory, which + resulted in dereferencing a <codeph>NULL</codeph> pointer. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2612"; scope="external" format="html">IMPALA-2612</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2643"> + <title>Prevent migrating incorrectly inferred identity predicates into inline views</title> + <conbody> + <p> + Referring to the same column twice in a view definition could cause the view to omit + rows where that column contained a <codeph>NULL</codeph> value. This could cause + incorrect results due to an inaccurate <codeph>COUNT(*)</codeph> value or rows missing + from the result set. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2643"; scope="external" format="html">IMPALA-2643</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2695"> + <title>Fix GRANTs on URIs with uppercase letters</title> + <conbody> + <p> + A <codeph>GRANT</codeph> statement for a URI could be ineffective if the URI + contained uppercase letters, for example in an uppercase directory name. + Subsequent statements, such as <codeph>CREATE EXTERNAL TABLE</codeph> + with a <codeph>LOCATION</codeph> clause, could fail with an authorization exception. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2695"; scope="external" format="html">IMPALA-2695</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2664-552"> + <title id="IMPALA-2648-552">Avoid sending large partition stats objects over thrift</title> + <conbody> + <p> + The <cmdname>catalogd</cmdname> daemon could encounter a serious error + when loading the incremental statistics metadata for tables with large + numbers of partitions and columns. The problem occurred when the + internal representation of metadata for the table exceeded 2 + GB, for example in a table with 20K partitions and 77 columns. The fix causes a + <codeph>COMPUTE INCREMENTAL STATS</codeph> operation to fail if it + would produce metadata that exceeded the maximum size. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2664"; scope="external" format="html">IMPALA-2664</xref>, + <xref href="https://issues.cloudera.org/browse/IMPALA-2648"; scope="external" format="html">IMPALA-2648</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2226"> + <title>Throw AnalysisError if table properties are too large (for the Hive metastore)</title> + <conbody> + <p> + <codeph>CREATE TABLE</codeph> or <codeph>ALTER TABLE</codeph> statements could fail with + metastore database errors due to length limits on the <codeph>SERDEPROPERTIES</codeph> and <codeph>TBLPROPERTIES</codeph> clauses. + (The limit on key size is 256, while the limit on value size is 4000.) The fix makes Impala handle these error conditions + more cleanly, by detecting too-long values rather than passing them to the metastore database. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2226"; scope="external" format="html">IMPALA-2226</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2273-552"> + <title>Make MAX_PAGE_HEADER_SIZE configurable</title> + <conbody> + <p> + Impala could fail to access Parquet data files with page headers larger than 8 MB, which could + occur, for example, if the minimum or maximum values for a column were long strings. The + fix adds a configuration setting <codeph>--max_page_header_size</codeph>, which you can use to + increase the Impala size limit to a value higher than 8 MB. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2273"; scope="external" format="html">IMPALA-2273</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2473"> + <title>reduce scanner memory usage</title> + <conbody> + <p> + Queries on Parquet tables could consume excessive memory (potentially multiple gigabytes) due to producing + large intermediate data values while evaluating groups of rows. The workaround was to reduce the size of + the <codeph>NUM_SCANNER_THREADS</codeph> query option, the <codeph>BATCH_SIZE</codeph> query option, + or both. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2473"; scope="external" format="html">IMPALA-2473</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2113"> + <title>Handle error when distinct and aggregates are used with a having clause</title> + <conbody> + <p> + A query that included a <codeph>DISTINCT</codeph> operator and a <codeph>HAVING</codeph> clause, but no + aggregate functions or <codeph>GROUP BY</codeph>, would fail with an uninformative error message. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2113"; scope="external" format="html">IMPALA-2113</xref></p> + <!-- Don't think this is really a 'major' issue. Left a comment in the IMPALA JIRA. --> + </conbody> + </concept> + + <concept id="IMPALA-2225"> + <title>Handle error when star based select item and aggregate are incorrectly used</title> + <conbody> + <p> + A query that included <codeph>*</codeph> in the <codeph>SELECT</codeph> list, in addition to an + aggregate function call, would fail with an uninformative message if the query had no + <codeph>GROUP BY</codeph> clause. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2225"; scope="external" format="html">IMPALA-2225</xref></p> + <!-- Don't think this is really a 'major' issue. Left a comment in the IMPALA JIRA. --> + </conbody> + </concept> + + <concept id="IMPALA-2731-552"> + <title>Refactor MemPool usage in HBase scan node</title> + <conbody> + <p> + Queries involving HBase tables used substantially more memory than in earlier Impala versions. + The problem occurred starting in Impala 2.2.8, as a result of the changes for IMPALA-2284. + The fix for this issue involves removing a separate memory work area for HBase queries + and reusing other memory that was already allocated. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2731"; scope="external" format="html">IMPALA-2731</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-1459-552"> + <title>Fix migration/assignment of On-clause predicates inside inline views</title> + <conbody> + <p> + Some combinations of <codeph>ON</codeph> clauses in join queries could result in comparisons + being applied at the wrong stage of query processing, leading to incorrect results. + Wrong predicate assignment could happen under the following conditions: + </p> + <ul> + <li> + The query includes an inline view that contains an outer join. + </li> + <li> + That inline view is joined with another table in the enclosing query block. + </li> + <li> + That join has an <codeph>ON</codeph> clause containing a predicate that + only references columns originating from the outer-joined tables inside the inline view. + </li> + </ul> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-1459"; scope="external" format="html">IMPALA-1459</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2558"> + <title>DCHECK in parquet scanner after block read error</title> + <conbody> + <p> + A debug build of Impala could encounter a serious error after encountering some kinds of I/O + errors for Parquet files. This issue only occurred in debug builds, not release builds. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2558"; scope="external" format="html">IMPALA-2558</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2535"> + <title>PAGG hits mem_limit when switching to I/O buffers</title> + <conbody> + <p> + A join query could fail with an out-of-memory error despite the apparent presence of sufficient memory. + The cause was the internal ordering of operations that could cause a later phase of the query to + allocate memory required by an earlier phase of the query. The workaround was to either increase + or decrease the <codeph>MEM_LIMIT</codeph> query option, because the issue would only occur for a specific + combination of memory limit and data volume. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2535"; scope="external" format="html">IMPALA-2535</xref></p> + </conbody> + </concept> + + <!-- This is a pretty incomprehensible JIRA title but there isn't any clearer title in the spreadsheet. --> + <!-- Merged with the following JIRA entry (which has a more understandable title) because it's part of the same root cause. --> + <concept id="IMPALA-2559"> + <title>Fix check failed: sorter_runs_.back()->is_pinned_</title> + <conbody> + <p> + A query could fail with an internal error while calculating the memory limit. + This was an infrequent condition uncovered during stress testing. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2559"; scope="external" format="html">IMPALA-2559</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2614"> + <title>Don't ignore Status returned by DataStreamRecvr::CreateMerger()</title> + <conbody> + <p> + A query could fail with an internal error while calculating the memory limit. + This was an infrequent condition uncovered during stress testing. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2614"; scope="external" format="html">IMPALA-2614</xref>, + <xref href="https://issues.cloudera.org/browse/IMPALA-2559"; scope="external" format="html">IMPALA-2559</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2591"> + <title>DataStreamSender::Send() does not return an error status if SendBatch() failed</title> + <conbody> + <!-- Symptoms listed in the JIRA are very vague. Left a note asking Sailesh for more detail. --> + <p> + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2591"; scope="external" format="html">IMPALA-2591</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2598"> + <title>Re-enable SSL and Kerberos on server-server</title> + <conbody> + <p> + These fixes lift the restriction on using SSL encryption and Kerberos authentication together + for internal communication between Impala components. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2598"; scope="external" format="html">IMPALA-2598</xref>, + <xref href="https://issues.cloudera.org/browse/IMPALA-2747"; scope="external" format="html">IMPALA-2747</xref></p> + </conbody> + </concept> + +<!-- + <concept id="IMPALA-2747"> + <title>Thrift-client cleans openSSL state before using it in the case of the catalog</title> + <conbody> + <p></p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2747"; scope="external" format="html">IMPALA-2747</xref></p> + <p><b>Severity:</b> High</p> + </conbody> + </concept> +--> + + </concept> + + <concept rev="2.3.1" id="fixed_issues_231"> + + <title>Issues Fixed in Impala for CDH 5.5.1</title> + + <conbody> + + <p conref="../shared/impala_common.xml#common/cdh551_noop"/> + + </conbody> + + </concept> + + <concept rev="2.3.0" id="fixed_issues_230"> + + <title>Issues Fixed in Impala for CDH 5.5.0</title> + + <conbody> + <p> This section lists the most serious or frequently encountered customer + issues fixed in CDH 5.5.0 / Impala 2.3.0. Any issues already fixed in + CDH 5.4 maintenance releases (up through CDH 5.4.8) are also included. + Those issues are listed under the respective CDH 5.4 sections and are + not repeated here. For the full list of fixed Impala issues, see <xref + audience="standalone" + href="http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_fixed_in_550.html"; + format="html" scope="external">the CDH 5.5.x release notes</xref><xref + href="cdh_rn_fixed_in_55.xml#concept_hjb_1pf_x5" audience="integrated" + />. </p> + +<!-- Raw "Fixed Issue" info from Silvius' working doc: https://docs.google.com/document/d/1PCboHo2Em-fySAHr3YF1ZZxMEfK8_vFFpP1rkWzyRZM/edit# + +Bug +IMPALA-2168 +SEGV in BufferedTupleStream::num_rows() in a query with very large, spilling ROJ +crash, resource-management + +Bug +IMPALA-2213 +Parquet read can fail if file metadata is stale + +Bug +IMPALA-2192 +Wrong results on TPCH-Q11 during nightly data load +broken-build, correctness + +Bug +IMPALA-2330 +SwitchToIoBuffers may leave stream to wrong state +resource-management + +Bug +IMPALA-2440 +Full outer join using non-partitioned HJ can incorrectly produce extra row of nulls +correctness + +Bug +IMPALA-2435 +Sorter::AddBatch does not check the Status of unsorted_run_->Init() + +Bug +IMPALA-2378 +Impalad exceeded its mem limit +crash, resource-management + +Bug +IMPALA-2369 +Crash: impala::Sorter::Run::Init +crash, nested_types, query_generator + +Bug +IMPALA-2366 +DiskIoMgr error handling with fread() + +Bug +IMPALA-2357 +Crash: impala::BufferedBlockMgr::Block::BytesRemaining +crash, nested_types, query_generator + +Bug +IMPALA-2348 +The Catalog does not close the connection to the HMS after table invalidation +catalog-server, hive + +Bug +IMPALA-2319 +Crash with nested loop join + limit + +Bug +IMPALA-2314 +Crash in PHJ::PrepareNextPartition() in call to LargestSpilledPartition() + +Bug +IMPALA-2165 +Avoid estimating cardinality 0 in SCAN node +performance, planner, resource-management + +Improvement +IMPALA-2143 +Avoid impala-shell sending password in plain text with ssl enabled + +Bug +IMPALA-2133 +HBase filter doesn't unescape string values correctly + +Bug +IMPALA-2090 +Incorrect handling of leap years when adding 1 month interval to date +correctness, query_generator + +Bug +IMPALA-2086 +Incorrect handling of leap years when adding 1 year interval to date +correctness, query_generator + +New Feature +IMPALA-2084 +SPLIT_PART and REGEXP_LIKE functions for Tableau pushdown + +New Feature +IMPALA-2081 +Add PERCENT_RANK, NTILE, CUME_DIST analytic window functions +planner, ramp-up-planner + +Bug +IMPALA-2016 +Cancelling query with group_concat causes crash +crash, query-lifecycle + +Improvement +IMPALA-1975 +Automatically reconnect to Impala in the shell if the connection is lost +ramp-up-task, shell + +Improvement - perf +IMPALA-1968 +Bad plan choices due to incorrect number of estimated hosts. +performance + +Bug +IMPALA-1947 +Avro cols may load incorrectly if schema inconsistent with StorageDescriptor +catalog-server, correctness + +Bug +IMPALA-1917 +Query return empty result if it contains NullLiteral in inlineview +correctness, regression + +(Possible new feature entry?) +Improvement - perf +IMPALA-1881 +Maximize data locality when scanning Parquet files with multiple row groups. +parquet, performance + +(Possible new feature entry?) +Improvement +IMPALA-1870 +Support users containing commas in authorized_proxy_user_config +ramp-up-introductory, ramp-up-task + +Bug +IMPALA-1859 +INSERT OVERWRITE with empty result set leaves existing records in partitioned table +impala, insert, overwrite, partitions + +Bug +IMPALA-1813 +CREATE TABLE LIKE ... STORED AS AVRO does not work. +usability + +Improvement +IMPALA-1136 +Impala is unable to read hive tables created with the "STORED AS AVRO" clause +ramp-up-planner, ramp-up-task, usability + +(Possible new feature entry?) +Improvement - perf +IMPALA-1588 +Cache HDFS file handle to avoid repeated hdfs fopen call + +Bug +IMPALA-1675 +Timestamp: Adding/subtracting very large time intervals to timestamps produces incorrect result + +Improvement +IMPALA-1982 +parquet.hive.serde.ParquetHiveSerDe is deprecated +Skye Wanderman-Milne +Critical +supportability + +Bug +IMPALA-1906 +PARQUET_FILE_SIZE query option not always honored due to an internal miscalculation. +Vlad Berindei +Critical +parquet, ramp-up-introductory, ramp-up-task + +--> + + </conbody> + + <concept id="serious_230"> + <title>Fixes for Serious Errors</title> + <conbody> + <p> + A number of issues were resolved that could result in serious errors + when encountered. The most critical or commonly encountered are + listed here. + </p> + <p><b>Bugs:</b> + +<!-- Bugs marked as 'crash'. +IMPALA-2168 +IMPALA-2378 +IMPALA-2369 +IMPALA-2357 +IMPALA-2319 +IMPALA-2314 +IMPALA-2016 +--> + + <xref href="https://issues.cloudera.org/browse/IMPALA-2168"; scope="external" format="html">IMPALA-2168</xref>, + <xref href="https://issues.cloudera.org/browse/IMPALA-2378"; scope="external" format="html">IMPALA-2378</xref>, + <xref href="https://issues.cloudera.org/browse/IMPALA-2369"; scope="external" format="html">IMPALA-2369</xref>, + <xref href="https://issues.cloudera.org/browse/IMPALA-2357"; scope="external" format="html">IMPALA-2357</xref>, + <xref href="https://issues.cloudera.org/browse/IMPALA-2319"; scope="external" format="html">IMPALA-2319</xref>, + <xref href="https://issues.cloudera.org/browse/IMPALA-2314"; scope="external" format="html">IMPALA-2314</xref>, + <xref href="https://issues.cloudera.org/browse/IMPALA-2016"; scope="external" format="html">IMPALA-2016</xref> + </p> + </conbody> + </concept> + + <concept id="correctness_230"> + <title>Fixes for Correctness Errors</title> + <conbody> + <p> + A number of issues were resolved that could result in wrong results + when encountered. The most critical or commonly encountered are + listed here. + </p> + <p><b>Bugs:</b> + +<!-- Bugs marked as 'correctness'. +IMPALA-2192 +IMPALA-2440 +IMPALA-2090 +IMPALA-2086 +IMPALA-1947 +IMPALA-1917 +--> + + <xref href="https://issues.cloudera.org/browse/IMPALA-2192"; scope="external" format="html">IMPALA-2192</xref>, + <xref href="https://issues.cloudera.org/browse/IMPALA-2440"; scope="external" format="html">IMPALA-2440</xref>, + <xref href="https://issues.cloudera.org/browse/IMPALA-2090"; scope="external" format="html">IMPALA-2090</xref>, + <xref href="https://issues.cloudera.org/browse/IMPALA-2086"; scope="external" format="html">IMPALA-2086</xref>, + <xref href="https://issues.cloudera.org/browse/IMPALA-1947"; scope="external" format="html">IMPALA-1947</xref>, + <xref href="https://issues.cloudera.org/browse/IMPALA-1917"; scope="external" format="html">IMPALA-1917</xref> + </p> + </conbody> + </concept> + + </concept> + +<!-- All 2.2.x / 5.4.x subsections go under here --> + + <concept rev="2.2.10" id="fixed_issues_2210"> + + <title>Issues Fixed in Impala for CDH 5.4.10</title> + + <conbody> + +<!-- + <p> + This section lists the most frequently encountered customer issues fixed in Impala for CDH 5.4.10. + </p> +--> + + <p> + For the full list of fixed issues for all CDH components, see + <xref audience="standalone" + href="http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_fixed_in_54.html#fixed_issues_5410"; + format="html" scope="external">the CDH 5.4.x release notes</xref><xref + audience="integrated" href="cdh_rn_fixed_in_54.xml#fixed_issues_5410"/>. + </p> + + <note conref="../shared/impala_common.xml#common/only_cdh5_22x"/> + + <ul audience="integrated"> + <li conref="cdh_rn_fixed_in_54.xml#ui_fixed_in_5410/impala_start" conrefend="cdh_rn_fixed_in_54.xml#ui_fixed_in_5410/impala_end"/> + </ul> + + </conbody> + + </concept> + + <concept rev="2.2.9" id="fixed_issues_229"> + + <title>Issues Fixed in Impala for CDH 5.4.9</title> + + <conbody> + + <p> + This section lists the most frequently encountered customer issues fixed in Impala for CDH 5.4.9. + </p> + + <note conref="../shared/impala_common.xml#common/only_cdh5_22x"/> + + <p> + For the full list of fixed issues, see + <xref audience="standalone" + href="http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_fixed_in_549.html"; + format="html" scope="external">the CDH 5.4.x release notes</xref><xref + audience="integrated" href="cdh_rn_fixed_in_54.xml#fixed_issues_549"/>. + </p> + + </conbody> + +<!-- List of fixed issues from Juan Yu's spreadsheet: https://docs.google.com/spreadsheets/d/1Ia06xOLKQInKfsJzQfQ6Sf1fO3nYLOpZ-0OWfFNgNSc/edit#gid=1566007471 + +Asterisks indicate the ones that are considered critical and so get a writeup here. + +* 5ff1062 IMPALA-1459: Fix migration/assignment of On-clause predicates inside inline views. +* bcea8cf IMPALA-2446: Fix wrong predicate assignment in outer joins +* 8e010af IMPALA-2648/IMPALA-2664: Avoid sending large partition stats objects over thrift +7e3a789 IMPALA-2260: Adding a large hour interval caused an interval overflow +* 9a86c94 IMPALA-1675: Avoid overflow when adding large intervals to TIMESTAMPs +1a83dfc IMPALA-2086, IMPALA-2090: Avoid boost year/month interval logic +* 1d0b017 IMPALA-1949: Analysis exception when a binary operator contain an IN operator with +0ae066f IMPALA-2141: UnionNode::GetNext() doesn't check for query errors +* 31824ad IMPALA-2273: Make MAX_PAGE_HEADER_SIZE configurable +75497ca IMPALA-2559: Fix check failed: sorter_runs_.back()->is_pinned_ +ddfe66f IMPALA-2265: Sorter was not checking the returned Status of PrepareRead +* a39a2a5 IMPALA-2357: Fix spilling sorts with var-len slots that are NULL or empty. +04bcfb9 IMPALA-0000: Fix reporting of custom OOM error messages. +d9d2a10 IMPALA-0000: Log backtrace on all MEM_LIMIT_EXCEEDED errors +* 4210021 IMPALA-0000: Work-around IMPALA-2344: Fail query with OOM in case block->Pin() fails +29d7b42 IMPALA-2286: Fix race between ~BufferedBlockMgr() and BufferedBlockMgr::Creat +* d87a306 IMPALA-2252: Crash (likely race) tearing down BufferedBlockMgr on query failu +* a1bb4a6 IMPALA-1746: QueryExecState doesn't check for query cancellation or errors +* e9ff989 IMPALA-2533: Impala throws IllegalStateException when inserting data into a p + +* c22683a IMPALA-1917: Do not register aux equivalence predicates with NULL on either side. +* c0a7393 IMPALA-2731: Refactor MemPool usage in HBase scan node. +--> + + <concept id="IMPALA-1917"> + <!-- Title in Juan's spreadsheet. Actual JIRA title more useful for readers IMO. <title>Do not register aux equivalence predicates with NULL on either side.</title> --> + <title>Query return empty result if it contains NullLiteral in inlineview</title> + <conbody> + <p> + If an inline view in a <codeph>FROM</codeph> clause contained a <codeph>NULL</codeph> literal, + the result set was empty. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-1917"; scope="external" format="html">IMPALA-1917</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2731"> + <!-- Title in Juan's spreadsheet. Actual JIRA title more useful for readers IMO. <title>Refactor MemPool usage in HBase scan node.</title> --> + <title>HBase scan node uses 2-4x memory after upgrade to Impala 2.2.8</title> + <conbody> + <p> + Queries involving HBase tables used substantially more memory than in earlier Impala versions. + The problem occurred starting in Impala 2.2.8, as a result of the changes for IMPALA-2284. + The fix for this issue involves removing a separate memory work area for HBase queries + and reusing other memory that was already allocated. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2731"; scope="external" format="html">IMPALA-2731</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-1459"> + <title>Fix migration/assignment of On-clause predicates inside inline views</title> + <conbody> + <!-- Explanation comes from IMPALA-2665. --> + <p> + Some combinations of <codeph>ON</codeph> clauses in join queries could result in comparisons + being applied at the wrong stage of query processing, leading to incorrect results. + Wrong predicate assignment could happen under the following conditions: + </p> + <ul> + <li> + The query includes an inline view that contains an outer join. + </li> + <li> + That inline view is joined with another table in the enclosing query block. + </li> + <li> + That join has an <codeph>ON</codeph> clause containing a predicate that + only references columns originating from the outer-joined tables inside the inline view. + </li> + </ul> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-1459"; scope="external" format="html">IMPALA-1459</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2446"> + <title>Fix wrong predicate assignment in outer joins</title> + <conbody> + <p> + The join predicate for an <codeph>OUTER JOIN</codeph> clause could be applied at the wrong stage + of query processing, leading to incorrect results. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2446"; scope="external" format="html">IMPALA-2446</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2648"><!-- Also IMPALA-2664 --> + <title>Avoid sending large partition stats objects over thrift</title> + <conbody> + <p> The <cmdname>catalogd</cmdname> daemon could encounter a serious error when loading the + incremental statistics metadata for tables with large numbers of partitions and columns. + The problem occurred when the internal representation of metadata for the table exceeded 2 + GB, for example in a table with 20K partitions and 77 columns. The fix causes a + <codeph>COMPUTE INCREMENTAL STATS</codeph> operation to fail if it would produce + metadata that exceeded the maximum size. </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2648"; scope="external" format="html">IMPALA-2648</xref>, + <xref href="https://issues.cloudera.org/browse/IMPALA-2664"; scope="external" format="html">IMPALA-2664</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-1675"> + <title>Avoid overflow when adding large intervals to TIMESTAMPs</title> + <conbody> + <p> Adding or subtracting a large <codeph>INTERVAL</codeph> value to a + <codeph>TIMESTAMP</codeph> value could produce an incorrect result, with the value + wrapping instead of returning an out-of-range error. </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-1675"; scope="external" format="html">IMPALA-1675</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-1949"> + <title>Analysis exception when a binary operator contains an IN operator with values</title> + <conbody> + <p> + An <codeph>IN</codeph> operator with literal values could cause a statement to fail if used + as the argument to a binary operator, such as an equality test for a <codeph>BOOLEAN</codeph> value. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-1949"; scope="external" format="html">IMPALA-1949</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2273"> + <!-- To do: this detail would be useful to capture on the Impala + Parquet page too. --> + <title>Make MAX_PAGE_HEADER_SIZE configurable</title> + <conbody> + <p> Impala could fail to access Parquet data files with page headers larger than 8 MB, which + could occur, for example, if the minimum or maximum values for a column were long strings. + The fix adds a configuration setting <codeph>--max_page_header_size</codeph>, which you + can use to increase the Impala size limit to a value higher than 8 MB. </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2273"; scope="external" format="html">IMPALA-2273</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2357"> + <title>Fix spilling sorts with var-len slots that are NULL or empty.</title> + <conbody> + <p> + A query that activated the spill-to-disk mechanism could fail if it contained a sort expression + involving certain combinations of fixed-length or variable-length types. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2357"; scope="external" format="html">IMPALA-2357</xref></p> + </conbody> + </concept> + + <concept id="block_pin_oom"> + <title>Work-around IMPALA-2344: Fail query with OOM in case block->Pin() fails</title> + <conbody> + <p> + Some queries that activated the spill-to-disk mechanism could produce a serious error + if there was insufficient memory to set up internal work areas. Now those queries + produce normal out-of-memory errors instead. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2344"; scope="external" format="html">IMPALA-2344</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2252"> + <title>Crash (likely race) tearing down BufferedBlockMgr on query failure</title> + <conbody> + <p> + A serious error could occur under rare circumstances, due to a race condition while freeing memory during heavily concurrent workloads. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2252"; scope="external" format="html">IMPALA-2252</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-1746"> + <title>QueryExecState doesn't check for query cancellation or errors</title> + <conbody> + <p> + A call to <codeph>SetError()</codeph> in a user-defined function (UDF) would not cause the query to fail as expected. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-1746"; scope="external" format="html">IMPALA-1746</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2533"> + <title>Impala throws IllegalStateException when inserting data into a partition while select + subquery group by partition columns</title> + <conbody> + <p> + An <codeph>INSERT ... SELECT</codeph> operation into a partitioned table could fail if the <codeph>SELECT</codeph> query + included a <codeph>GROUP BY</codeph> clause referring to the partition key columns. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2533"; scope="external" format="html">IMPALA-2533</xref></p> + </conbody> + </concept> + + </concept> + + <concept rev="2.2.8" id="fixed_issues_228"> + + <title>Issues Fixed in Impala for CDH 5.4.8</title> + + <conbody> + + <p> + This section lists the most frequently encountered customer issues fixed in Impala for CDH 5.4.8. + </p> + + <note conref="../shared/impala_common.xml#common/only_cdh5_22x"/> + + <p> + For the full list of fixed issues, see + <xref audience="standalone" + href="http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_fixed_in_548.html"; format="html" scope="external">the CDH 5.4.x release notes</xref><xref audience="integrated" href="cdh_rn_fixed_in_54.xml#fixed_issues_548"/>. + </p> + + </conbody> + +<!-- +3e0fee5 IMPALA-2369, IMPALA-2435: Impala crashes when the sorter hits an OOM error +ef498c2 IMPALA-1136, IMPALA-2161: Skip \u0000 characters when dealing Avro schemas +8f97d4b IMPALA-2168: Do not try to access streams of repartitioned spilled partition in right-joins +* 294ac14 Additional checks to catch IMPALA-2168 earlier +b9b10ec IMPALA-2514: DCHECK on destroying an ExprContext +96206af IMPALA-2440: Fix old HJ full outer join with no rows +5e1a79a IMPALA-2366: check fread return code correctly +02f3e36 IMPALA-2477: Parquet metadata randomly 'appears stale' +76c1548 IMPALA-2213: make Parquet scanner fail query if the file size metadata is stale +10ba8ed IMPALA-2249: Avoid allocating StringBuffer > 1GB in ScannerContext::Stream::GetBytesInternal() +a87eaa0 IMPALA-2270: avoid FnvHash64to32 with empty inputs +7f6a521 IMPALA-2284: Disallow long (1<<30) strings in group_concat() +* 9aea5d7 Mistake in schema_constraints by the IMPALA-2130 patch (7c7e69b) +f8d32a4 IMPALA-2130: Wrong verification of Parquet file version +9ec5944 IMPALA-2348: The catalog does not close the connection to HMS during table invalidation +a78c303 IMPALA-2364: Wrong DCHECK in PHJ::ProcessProbeBatch +e6dec36 IMPALA-2314: LargestSpilledPartition was not checking if partition is closed +* 56d8bbd Checks for IMPALA-2314 and more accurate calculation of PHJ::LargestSpillPartition +3ad4e71 IMPALA-2256: Handle joins with right side of high cardinality and zero materialized slots +4422462 IMPALA-2165: Avoid cardinality 0 in scan nodes of small tables and low selectivity +1f199ec IMPALA-2292: Change the type of timestamp_col to string in the table no_avro_schema. +f6b4480 IMPALA-1136: Support loading Avro tables without an explicit Avro schema +d93fb5a IMPALA-1899: Cleanup handling of Hive's field schema + +* = no JIRA assigned with it so it definitely is an internal-only modification. +--> + + <concept id="IMPALA-1136"> + <title>Impala is unable to read hive tables created with the "STORED AS AVRO" clause</title> + <conbody> + <p>Impala could not read Avro tables created in Hive with the <codeph>STORED AS AVRO</codeph> clause.</p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-1136"; scope="external" format="html">IMPALA-1136</xref>, + <xref href="https://issues.cloudera.org/browse/IMPALA-2161"; scope="external" format="html">IMPALA-2161</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2213"> + <title>make Parquet scanner fail query if the file size metadata is stale</title> + <conbody> + <p>If a Parquet file in HDFS was overwritten by a smaller file, Impala could encounter a serious error. + Issuing a <codeph>INVALIDATE METADATA</codeph> statement before a subsequent query would avoid the error. + The fix allows Impala to handle such inconsistencies in Parquet file length cleanly regardless of whether the + table metadata is up-to-date.</p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2213"; scope="external" format="html">IMPALA-2213</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2249"> + <title>Avoid allocating StringBuffer > 1GB in ScannerContext::Stream::GetBytesInternal()</title> + <conbody> + <p>Impala could encounter a serious error when reading compressed text files larger than 1 GB. The fix causes Impala + to issue an error message instead in this case.</p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2249"; scope="external" format="html">IMPALA-2249</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2284"> + <title>Disallow long (1<<30) strings in group_concat()</title> + <conbody> + <p>A query using the <codeph>group_concat()</codeph> function could encounter a serious error if the returned string value was larger than 1 GB. + Now the query fails with an error message in this case.</p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2284"; scope="external" format="html">IMPALA-2284</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2270"> + <title>avoid FnvHash64to32 with empty inputs</title> + <conbody> + <p>An edge case in the algorithm used to distribute data among nodes could result in uneven distribution of work for some queries, + with all data sent to the same node.</p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2270"; scope="external" format="html">IMPALA-2270</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2348"> + <title>The catalog does not close the connection to HMS during table invalidation</title> + <conbody> + <p>A communication error could occur between Impala and the Hive metastore database, causing Impala operations that update + table metadata to fail.</p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2348"; scope="external" format="html">IMPALA-2348</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2364-548"> + <title>Wrong DCHECK in PHJ::ProcessProbeBatch</title> + <conbody> + <p>Certain queries could encounter a serious error if the spill-to-disk mechanism was activated.</p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2364"; scope="external" format="html">IMPALA-2364</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2165-548"> + <title>Avoid cardinality 0 in scan nodes of small tables and low selectivity</title> + <conbody> + <p>Impala could generate a suboptimal query plan for some queries involving small tables.</p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2165"; scope="external" format="html">IMPALA-2165</xref></p> + </conbody> + </concept> + + </concept> + + <concept rev="2.2.7" id="fixed_issues_227"> + + <title>Issues Fixed in Impala for CDH 5.4.7</title> + + <conbody> + + <p> + This section lists the most frequently encountered customer issues fixed in Impala for CDH 5.4.7. + </p> + + <note conref="../shared/impala_common.xml#common/only_cdh5_22x"/> + + <p> + For the full list of fixed issues, see + <xref audience="standalone" + href="http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_fixed_in_547.html"; format="html" scope="external">the CDH 5.4.x release notes</xref><xref audience="integrated" href="cdh_rn_fixed_in_54.xml#fixed_issues_547"/>. + </p> + + </conbody> +<!-- +8736dba IMPALA-1983: Warn if table stats are potentially corrupt. +a0911fc IMPALA-2266: Pass correct child node in 2nd phase merge aggregation. +3183878 IMPALA-2216: Set the output smap of an EmptySetNode produced from an empty inline view. +a73d008 IMPALA-2203: Set an InsertStmt's result exprs from the source statement's result exprs. +e641759 IMPALA-2088: Fix planning of empty union operands with analytics. +752b3bd IMPALA-1756: Constant filter expressions are not checked for errors and state cleanup is not done before throwing exception. +d703959 IMPALA-2239: update misc.test to match the new .test file format +8dddd60 IMPALA-2089: Retain eq predicates bound by grouping slots with complex grouping exprs. +ce9fa25 IMPALA-2201: Unconditionally update the partition stats and row count. +ff5bc02 IMPALA-2199: Row count not set for empty partition when spec is used with compute incremental stats +3eb2d80 IMPALA-1898: Explicit aliases + ordinals analysis bug +ba6a8bf IMPALA-1987: Fix TupleIsNullPredicate to return false if no tuples are nullable. +932d4f8 IMPALA-2178: fix Expr::ComputeResultsLayout() logic +4ef0bd8 IMPALA-1737: Substitute an InsertStmt's partition key exprs with the root node's smap. + +Per Juan Yu, almost all of those are critical issues related to query correctness. +The non-critical ones are: + +752b3bd IMPALA-1756: Constant filter expressions are not checked for errors and state cleanup is not done before throwing exception. +d703959 IMPALA-2239: update misc.test to match the new .test file format +ce9fa25 IMPALA-2201: Unconditionally update the partition stats and row count. + +--> + + <concept id="IMPALA-1983"> + <title>Warn if table stats are potentially corrupt.</title> + <conbody> + <p> + Impala warns if it detects a discrepancy in table statistics: a table considered to have zero rows even though there are data files present. + In this case, Impala also skips query optimizations that are normally applied to very small tables. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-1983"; scope="external" format="html">IMPALA-1983:</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2266"> + <title>Pass correct child node in 2nd phase merge aggregation.</title> + <conbody> + <p>A query could encounter a serious error if it included a particular combination of aggregate functions and inline views.</p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2266"; scope="external" format="html">IMPALA-2266</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2216"> + <title>Set the output smap of an EmptySetNode produced from an empty inline view.</title> + <conbody> + <p>A query could encounter a serious error if it included an inline view whose subquery had no <codeph>FROM</codeph> clause.</p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2216"; scope="external" format="html">IMPALA-2216</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2203"> + <title>Set an InsertStmt's result exprs from the source statement's result exprs.</title> + <conbody> + <p> + A <codeph>CREATE TABLE AS SELECT</codeph> or <codeph>INSERT ... SELECT</codeph> statement could produce + different results than a <codeph>SELECT</codeph> statement, for queries including a <codeph>FULL JOIN</codeph> clause + and including literal values in the select list. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2203"; scope="external" format="html">IMPALA-2203</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2088"> + <title>Fix planning of empty union operands with analytics.</title> + <conbody> + <p> + A query could return incorrect results if it contained a <codeph>UNION</codeph> clause, + calls to analytic functions, and a constant expression that evaluated to <codeph>FALSE</codeph>. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2088"; scope="external" format="html">IMPALA-2088</xref></p> + </conbody> + </concept> + +<!-- + <concept id="IMPALA-1756"> + <title>Constant filter expressions are not checked for errors and state cleanup is not done before throwing exception.</title> + <conbody> + <p></p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-1756"; scope="external" format="html">IMPALA-1756</xref></p> + <p><b>Severity:</b> High</p> + </conbody> + </concept> +--> + +<!-- + <concept id="IMPALA-2239"> + <title>update misc.test to match the new .test file format</title> + <conbody> + <p></p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2239"; scope="external" format="html">IMPALA-2239</xref></p> + <p><b>Severity:</b> High</p> + </conbody> + </concept> +--> + + <concept id="IMPALA-2089"> + <title>Retain eq predicates bound by grouping slots with complex grouping exprs.</title> + <conbody> + <p> + A query containing an <codeph>INNER JOIN</codeph> clause could return undesired rows. + Some predicate specified in the <codeph>ON</codeph> clause could be omitted from the filtering operation. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2089"; scope="external" format="html">IMPALA-2089</xref></p> + </conbody> + </concept> + +<!-- + <concept id="IMPALA-2201"> + <title>Unconditionally update the partition stats and row count.</title> + <conbody> + <p></p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2201"; scope="external" format="html">IMPALA-2201</xref></p> + <p><b>Severity:</b> High</p> + </conbody> + </concept> +--> + + <concept id="IMPALA-2199"> + <title>Row count not set for empty partition when spec is used with compute incremental stats</title> + <conbody> + <p> + A <codeph>COMPUTE INCREMENTAL STATS</codeph> statement could leave the row count for an emptyp partition as -1, + rather than initializing the row count to 0. The missing statistic value could result in reduced query performance. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2199"; scope="external" format="html">IMPALA-2199</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-1898"> + <title>Explicit aliases + ordinals analysis bug</title> + <conbody> + <p> + A query could encounter a serious error if it included column aliases with the same names as table columns, and used + ordinal numbers in an <codeph>ORDER BY</codeph> or <codeph>GROUP BY</codeph> clause. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-1898"; scope="external" format="html">IMPALA-1898</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-1987"> + <title>Fix TupleIsNullPredicate to return false if no tuples are nullable.</title> + <conbody> + <p> + A query could return incorrect results if it included an outer join clause, inline views, and calls to functions such as <codeph>coalesce()</codeph> + that can generate <codeph>NULL</codeph> values. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-1987"; scope="external" format="html">IMPALA-1987</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2178"> + <title>fix Expr::ComputeResultsLayout() logic</title> + <conbody> + <p> + A query could return incorrect results if the table contained multiple <codeph>CHAR</codeph> columns with length of 2 or less, + and the query included a <codeph>GROUP BY</codeph> clause that referred to multiple such columns. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2178"; scope="external" format="html">IMPALA-2178</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-1737"> + <title>Substitute an InsertStmt's partition key exprs with the root node's smap.</title> + <conbody> + <p> + An <codeph>INSERT</codeph> statement could encounter a serious error if the <codeph>SELECT</codeph> + portion called an analytic function. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-1737"; scope="external" format="html">IMPALA-1737</xref></p> + </conbody> + </concept> + + </concept> + + <concept rev="2.2.5" id="fixed_issues_225"> + + <title>Issues Fixed in Impala for CDH 5.4.5</title> + + <conbody> + + <p> + This section lists the most frequently encountered customer issues fixed in Impala for CDH 5.4.5. + </p> + + <note conref="../shared/impala_common.xml#common/only_cdh5_22x"/> + + <p> + For the full list of fixed issues, see + <xref audience="standalone" + href="http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_rn_fixed_in_545.html"; format="html" scope="external">the CDH 5.4.x release notes</xref><xref audience="integrated" href="cdh_rn_fixed_in_54.xml#fixed_issues_545"/>. + </p> + + </conbody> + +<!-- +7c01049 IMPALA-1929: Avoiding a DCHECK of NULL hash table in spilled right joins +c98a556 IMPALA-2136: Bug in PrintTColumnValue caused wrong stats for TINYINT partition cols +* 4390a5c Making sure that there are no regressions due to IMPALA-2065 +* 33ee7df IMPALA-2133: Properly unescape string value for HBase filters +* 562b61a IMPALA-1735: ExpandRmReservation only check parent pools with limit +* 5b09eaa IMPALA-2101: fix NULL ptr deref in BufferedBlockMgr::PinBlock +c2b8852 IMPALA-2018: Where clause does not propagate to joins inside nested views +16b79ce IMPALA-2064: Add effective_user() builtin +99a282b IMPALA-2125: Make UTC to local TimestampValue conversion faster. +9198985 IMPALA-2065: Workaround IMPALA-1619 in BufferedBlockMgr::ConsumeMemory() +9ce94eb IMPALA-2048: Set the correct input format when updating partition metadata + +* = non-critical to document according to Juan Yu. +--> + + <concept id="IMPALA-2048"> + <title>Impala DML/DDL operations corrupt table metadata leading to Hive query failures</title> + <!-- Real JIRA title: Class not found: com.cloudera.impala.hive.serde.ParquetInputFormat error --> + <conbody> + <p> + When the Impala <codeph>COMPUTE STATS</codeph> statement was run on a partitioned Parquet table that was created in Hive, the table subsequently became inaccessible in Hive. + The table was still accessible to Impala. Regaining access in Hive required a workaround of creating a new table. The error displayed in Hive was: + </p> +<codeblock>Error: Error while compiling statement: FAILED: SemanticException Class not found: com.cloudera.impala.hive.serde.ParquetInputFormat (state=42000,code=40000)</codeblock> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2048"; scope="external" format="html">IMPALA-2048</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-1929"> + <title>Avoiding a DCHECK of NULL hash table in spilled right joins</title> + <!-- Real JIRA title: Crash because PHJ::NextSpilledProbeRowBatch() tries to use a NULL hash_tbl --> + <conbody> + <p> + A query could encounter a serious error if it contained a <codeph>RIGHT OUTER</codeph>, <codeph>RIGHT ANTI</codeph>, or <codeph>FULL OUTER</codeph> join clause + and approached the memory limit on a host so that the <q>spill to disk</q> mechanism was activated. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-1929"; scope="external" format="html">IMPALA-1929</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2136"> + <title>Bug in PrintTColumnValue caused wrong stats for TINYINT partition cols</title> + <!-- Real JIRA title: Partitions with TINYINT partition columns will always have 0 estimated rows after compute stats --> + <conbody> + <p> + Declaring a partition key column as a <codeph>TINYINT</codeph> caused problems with the <codeph>COMPUTE STATS</codeph> statement. + The associated partitions would always have zero estimated rows, leading to potential inefficient query plans. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2136"; scope="external" format="html">IMPALA-2136</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2018"> + <title>Where clause does not propagate to joins inside nested views</title> + <!-- Real JIRA title: Where clause does not propagate to joins inside nested views --> + <conbody> + <p> + A query that referred to a view whose query referred to another view containing a join, could return incorrect results. + <codeph>WHERE</codeph> clauses for the outermost query were not always applied, causing the result + set to include additional rows that should have been filtered out. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2018"; scope="external" format="html">IMPALA-2018</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2064"> + <title>Add effective_user() builtin</title> + <!-- Real JIRA title: Add effective_user() builtin --> + <conbody> + <p> + The <codeph>user()</codeph> function returned the name of the logged-in user, which might not be the + same as the user name being checked for authorization if, for example, delegation was enabled. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2064"; scope="external" format="html">IMPALA-2064</xref></p> + <p><b>Resolution:</b> Rather than change the behavior of the <codeph>user()</codeph> function, + the fix introduces an additional function <codeph>effective_user()</codeph> that returns the user name that is checked during authorization.</p> + </conbody> + </concept> + + <concept id="IMPALA-2125"> + <title>Make UTC to local TimestampValue conversion faster.</title> + <!-- Real JIRA title: Improve perf when reading timestamps from parquet files written by hive --> + <conbody> + <p> + Query performance was improved substantially for Parquet files containing <codeph>TIMESTAMP</codeph> + data written by Hive, when the <codeph>-convert_legacy_hive_parquet_utc_timestamps=true</codeph> setting + is in effect. + </p> + <p><b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-2125"; scope="external" format="html">IMPALA-2125</xref></p> + </conbody> + </concept> + + <concept id="IMPALA-2065"> + <title>Workaround IMPALA-1619 in BufferedBlockMgr::ConsumeMemory()</title> + <!-- Real JIRA title: Crash: impala::BufferedBlockMgr::ConsumeMemory
<TRUNCATED>
