IMPALA-5031: Zero-length arrays are undefined behavior
This patch prevents the creation of a zero-length array. This is
illegal under paragraph 5 of §6.7.5.2 ("Array declarators") of the C99
standard, which reads:
If the size is an expression that is not an integer constant
expression: if
Repository: impala
Updated Branches:
refs/heads/master e1c9cbd07 -> 93a0ce857
IMPALA-7857: log more information about statestore failure detection
This adds a couple of log messages for state transitions in the
statestore's failure detector.
Testing:
Ran test_statestore.py and checked for
Repository: impala
Updated Branches:
refs/heads/master 93a0ce857 -> 8f151d0e6
IMPALA-5031: method calls on NULL are not UBSAN-clean
According to [expr.post] in the C++14 standard, a call to a member
function like a->b() is interpreted as (a->b)(). In other words, the
dereferencing is done
IMPALA-7871: Don't load Hive builtins
Dataload has a step of "Loading Hive builtins" that
loads a bunch of jars into HDFS/S3/etc. Despite
its name, nothing seems to be using these.
Dataload and core tests succeed without this step.
This removes the Hive builtins step and associated
scripts.
http://git-wip-us.apache.org/repos/asf/impala/blob/2a4835cf/testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering.test
--
diff --git
http://git-wip-us.apache.org/repos/asf/impala/blob/2a4835cf/testdata/workloads/functional-planner/queries/PlannerTest/sort-expr-materialization.test
--
diff --git
Repository: impala
Updated Branches:
refs/heads/master 153663c22 -> 2a4835cfb
IMPALA-5031: signed overflow is undefined behavior
This patch fixes a signed overflow in the backend test
DecimalTest.Overflow. The interesting part of the backtrace is:
runtime/decimal-value.inline.h:254:17:
IMPALA-7367: Pack StringValue and CollectionValue slots
This change packs StringValue and CollectionValue slots to ensure
they now occupy 12 bytes instead of 16 bytes. This reduces the
memory requirements and improves the performance. Since Kudu
tuples are populated using a memcopy, 4 bytes of
http://git-wip-us.apache.org/repos/asf/impala/blob/2a4835cf/testdata/workloads/functional-planner/queries/PlannerTest/tpch-all.test
--
diff --git
a/testdata/workloads/functional-planner/queries/PlannerTest/tpch-all.test
http://git-wip-us.apache.org/repos/asf/impala/blob/2a4835cf/testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test
--
diff --git
IMPALA-7794: Rewrite flaky ownership authorization tests
This patch rewrites the ownership authorization tests to not depend on
delay and timeout, which can help to avoid the flakiness. The patch also
refactors some tests to reuse Sentry and Impala instances without having
to restart them, which
IMPALA-: Fix crash due to arithmetic overflows in Exchange Node
Fixes an arithmetic overflow in ExchangeNode::GetNextMerging. Prior to
this patch, the code read:
int rows_to_keep = num_rows_skipped_ - offset_;
Where num_rows_skipped_ and offset_ were of type int64_t. The result was
cast to
IMPALA-7822: handle overflows in repeat() builtin
We need to carefully check that the intermediate value fits in an
int64_t and the final size fits in an int. If they don't we
raise an error and fail the query.
Testing:
Added a couple of backend tests to exercise the
overflow check code paths.
IMPALA-7809: support Kudu 1.9 in test_concurrent_schema_change
This patch extends the match of allowable errors in
test_concurrent_schema_change to work with Kudu 1.9.
Testing: Local with a Kudu 1.9 minicluster environment.
Change-Id: I7fc24bb6a18aecc0cb726b8d66f0aeccf56bbb9b
Reviewed-on:
IMPALA-7528: Fix division by zero when computing cardinalities
This patch fixes a case where there can be a division by zero when
computing cardinalities of many to many joins on NULL columns.
Testing:
Added a planner test case
Change-Id: Ieedd51d3ad6131a4ea2a5883f05104e6a0f2cd14
Reviewed-on:
IMPALA-6436: exit instead of abort for catalog startup failure
Rename EXIT_WITH_EXC to ABORT_WITH_EXC to make the behaviour more
obvious at callsites.
Handle exceptions from Catalog constructor by logging the backtrace and
exiting cleanly, rather than aborting. This will prevent generation of a
IMPALA-3652: Fix resource transfer in subplans with limits
Impala assumes that when Reset() is called on an ExecNode, all of the
memory returned from that node by GetNext() has been attached to the
output RowBatch. In a query with a LIMIT on the subplan, such that
some nodes don't reach 'eos',
IMPALA-5946,IMPALA-5956: add TPC-DS q31,q59,q89
Q31: the substitution variables didn't match the TPC-DS spec.
After fixing this, the results match up to 4 digits of
rounding (there is some error introduced in intermediate
calculations).
Q59: the results match the reference results
IMPALA-7565: Add startup flag to set thrift connection setup thread
pool size
This patch adds a hidden experimental startup flag called
'accepted_cnxn_setup_thread_pool_size' which enables setting the size
of the thread pool used to process the internal post-accept, pre-setup
connection queue in
IMPALA-7835: Role and user catalog objects with the same name can cause
INVALIDATE METADATA to hang
Before this patch, enabling object ownership and running INVALIDATE
METADATA for the role and user catalog objects with the same principal
name could cause INVALIDATE METADATA to hang because a
IMPALA-7775: fix some lifecycle issues in statestore/session tests
Background threads from the Statestore's thread pool continued
running in the background and could dereference invalid memory.
We make sure these threads are cleaned up before moving onto
the next test. Note that we don't clean up
IMPALA-6249: Expose several build flags via web UI
Exposes a list of build flags via the impalad web UI. The build flags
can be viewed on the root page under the "Version" section. They can
be accessed via other tests through the debug version of the root page
(e.g. adding to the URL). The build
IMPALA-7824: INVALIDATE METADATA should not hang when Sentry is unavailable
Before this patch, running INVALIDATE METADATA when Sentry is
unavailable could cause Impala query to hang. PolicyReader thread in
SentryProxy is used by two use cases, one as a background thread
that periodically
IMPALA-7788: [DOCS] Impala supports ADLS Gen 2 (ABFS)
Change-Id: Ic06d9ac92ed78b9092369e211de8a81db1d7ce90
Reviewed-on: http://gerrit.cloudera.org:8080/11853
Tested-by: Impala Public Jenkins
Reviewed-by: Joe McDonnell
Reviewed-by: Jim Apple
Project:
IMPALA-7791: Compute AggregationNode's estimated rows using # instances
Previously, the AggregationNode calculated the estimated number
of rows based on input cardinality without accounting for the
division of input data across multiple fragment instances. This
bloated up the memory estimates for
IMPALA-6661 Make NaN values equal for grouping purposes.
Similar to the treatment of NULLs, we want to consider NaN values
as equal when grouping.
- When detecting a NaN in a set of row values, the NaN value must
be converted to a canonical value - so that all NaN values have
the same
IMPALA-5031: Make UBSAN-friendly arithmetic generic
ArithmeticUtil::AsUnsigned() makes it possible to do arithmetic on
signed integers in a way that does not invoke undefined behavior, but
it only works on integers. This patch adds ArithmeticUtil::Compute(),
which dispatches (at compile time) to
IMPALA-7244: [DOCS] Remove unsupported format writer support
- Added a "removed" note for the ALLOW_UNSUPPORTED_FORMATS and
SEQ_COMPRESSION_MODE query options.
- Will remove the above options from the docs at the next
compatibility breaking release.
Change-Id:
IMPALA-5031: Jenkins jobs should fail when UBSAN fails
Before this patch, Jenkins builds do not fail or warn if UBSAN
produces an error. This patch introduces two new failure modes,
"error" and "death".
In "error", a junit error is generated, which causes a build to be
labeled "unstable".
In
Repository: impala
Updated Branches:
refs/heads/branch-3.1.0 e3a702707 -> f79f2d4ad
IMPALA-7103: [DOCS] Document the ALLOW_ERASURE_CODED_FILES query option
Change-Id: I63b54031b725e528196d19eac9ddf36a19c43e28
Reviewed-on: http://gerrit.cloudera.org:8080/11855
Tested-by: Impala Public Jenkins
IMPALA-402: test for random partitioning in insert
This adds a basic regression test for the bug reported in IMPALA-402.
Testing:
Exhaustive build.
Looped the modified test overnight.
Change-Id: I4bbca5c64977cadf79dabd72f0c8876a40fdf410
Reviewed-on: http://gerrit.cloudera.org:8080/11799
Bump CDH_BUILD_VERSION to include Hive fix for RELY constraint
This change bumps the CDH_BUILD_VERSION to a version that includes
the fix for a bug that does not correctly propagate "RELY" constraints
from SQL statements in Hive. This is needed for the upcoming patch for
IMPALA-3531.
This new
IMPALA-7840: add missing error to test_concurrent_schema_change
test_concurrent_schema_change runs a series of alters and inserts on
the same Kudu table concurrently to ensure that Impala can handle this
without crashing.
There is a list of expected error messages in the test. One possible
[DOCS] A typo fixed in impala_live_summary
Change-Id: Idb954199955d3af4271b5dc61c284e56536fe649
Reviewed-on: http://gerrit.cloudera.org:8080/11904
Reviewed-by: Alex Rodoni
Tested-by: Impala Public Jenkins
Project: http://git-wip-us.apache.org/repos/asf/impala/repo
Commit:
IMPALA-7477: Batch-oriented query set construction
Rework the row-by-row construction of query result sets in PlanRootSink
so that it materialises an output column at a time. Make some minor
optimisations like preallocating output vectors and initialising
strings more efficiently.
My intent is
IMPALA-5004: Switch to sorting node for large TopN queries
Adds a new query option 'topn_bytes_limit' that places a limit on the
number of estimated bytes that a TopN operator can process. If the
Impala planner estimates that a TopN operator will process more bytes
than this limit, it will
IMPALA-6323 Allow constant analytic window expressions.
The constraint imposed by IMPALA-1354 was artificial.
If there are constant "partition by" expressions, simply drop them,
they are no-ops.
Constant "order by" expressions can be ignored as well, though in effect
they should be accounted for
IMPALA-7789: [DOCS] Admission status in Impala Shell
Change-Id: I17d788eb716c6a2f7a144ee2d81bbe823f74d16a
Reviewed-on: http://gerrit.cloudera.org:8080/11895
Tested-by: Impala Public Jenkins
Reviewed-by: Tim Armstrong
Reviewed-by: Bikramjeet Vig
Project:
IMPALA-7818: Standardize use of Expr predicates
The Expr node provide two kinds of queries about node categories:
predicates and isMumble() functions. This change standardizes two
existing methods to use predicates instead.
First, the existing isLiteral() method is replaced by a new
IS_LITERAL
IMPALA-7808: Refactor Analyzer for easier debugging
Changes two blocks of code to make debugging easier. No functional
changes occur; changes are pure refactoring. A trivial change in
AnalyzerContext removes a nested conditional clause.
A larger change in SelectStmt takes the large analysis
IMPALA-7148: Make test_profile_fragment_instances() more robust
test_profile_fragment_instances() makes assumption about number
of instances of each scan node in the distributed query plan.
The number of instances for each scan nodes can actually vary
depending on how data is loaded and
Repository: impala
Updated Branches:
refs/heads/branch-3.1.0 f79f2d4ad -> 33d943b25
Revert "IMPALA-7148: Make test_profile_fragment_instances() more robust"
This reverts commit 1d412a09ac807c2f7ab39ac4fdc3e21c29024190.
Project: http://git-wip-us.apache.org/repos/asf/impala/repo
Commit:
Repository: impala
Updated Branches:
refs/heads/branch-3.1.0 33d943b25 -> 031d5690f (forced update)
Revert "IMPALA-7477: Batch-oriented query set construction"
This reverts commit 0f63b2c9f9d62b0d22191f454b672a6047206252.
Project: http://git-wip-us.apache.org/repos/asf/impala/repo
Commit:
Repository: impala
Updated Branches:
refs/heads/master 2a4835cfb -> e1c9cbd07
IMPALA-7585: support LDAP in run-workload.py
This patch just threads through the user, password, and ssl settings
all the way back to the ImpalaBeeswaxClient.
Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba
44 matches
Mail list logo
