[jira] [Created] (IMPALA-7518) Release 2.12.1 since 2.12.0 fails to compile
Quanlong Huang created IMPALA-7518: -- Summary: Release 2.12.1 since 2.12.0 fails to compile Key: IMPALA-7518 URL: https://issues.apache.org/jira/browse/IMPALA-7518 Project: IMPALA Issue Type: Task Reporter: Quanlong Huang 2.12.0-release failed to compile since it depends on cdh5.16.0-SNAPSHOT. An update to Sentry 1.5.1-cdh5.16.0-SNAPSHOT broke the build. See the error message in IMPALA-7513. We should release a compilable version of branch-2.12. There're two choices: * cherry-pick the patch in IMPALA-7513 * Downgrade to depend on cdh5.15 stable versions I'm in favor with the last choice since future changes in cdh5.16.0-SNAPSHOT won't break the builds. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7518) Release 2.12.1 since 2.12.0 fails to compile
Quanlong Huang created IMPALA-7518: -- Summary: Release 2.12.1 since 2.12.0 fails to compile Key: IMPALA-7518 URL: https://issues.apache.org/jira/browse/IMPALA-7518 Project: IMPALA Issue Type: Task Reporter: Quanlong Huang 2.12.0-release failed to compile since it depends on cdh5.16.0-SNAPSHOT. An update to Sentry 1.5.1-cdh5.16.0-SNAPSHOT broke the build. See the error message in IMPALA-7513. We should release a compilable version of branch-2.12. There're two choices: * cherry-pick the patch in IMPALA-7513 * Downgrade to depend on cdh5.15 stable versions I'm in favor with the last choice since future changes in cdh5.16.0-SNAPSHOT won't break the builds. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IMPALA-6919) Impala 3.1 Doc: Doc the COMMENT ON statements
[ https://issues.apache.org/jira/browse/IMPALA-6919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-6919: Description: https://gerrit.cloudera.org/#/c/11370/ > Impala 3.1 Doc: Doc the COMMENT ON statements > - > > Key: IMPALA-6919 > URL: https://issues.apache.org/jira/browse/IMPALA-6919 > Project: IMPALA > Issue Type: Sub-task > Components: Docs >Affects Versions: Impala 2.13.0, Impala 3.1.0 >Reporter: Alex Rodoni >Assignee: Alex Rodoni >Priority: Major > Labels: future_release_doc > > https://gerrit.cloudera.org/#/c/11370/ -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7516) Rejected queries dont get removed from the list of running queries
[ https://issues.apache.org/jira/browse/IMPALA-7516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikramjeet Vig updated IMPALA-7516: --- Priority: Critical (was: Major) > Rejected queries dont get removed from the list of running queries > -- > > Key: IMPALA-7516 > URL: https://issues.apache.org/jira/browse/IMPALA-7516 > Project: IMPALA > Issue Type: Bug > Components: Backend >Affects Versions: Impala 3.1.0 >Reporter: Bikramjeet Vig >Assignee: Bikramjeet Vig >Priority: Critical > > Steps to reproduce: > > Eg. > set mem_limit=10b; > select 1; > "Number of running queries with fragments on this host" on the impala queries > debug page will show 1 even though the query was rejected -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7517) Hung scanner when soft memory limit exceeded
[
https://issues.apache.org/jira/browse/IMPALA-7517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated IMPALA-7517:
Description:
As reported on the mailing list, this is a regression due to IMPALA-7096
(7ccf7369085aa49a8fc0daf6f91d97b8a3135682). The scanner thread has the
following code:
{code}
// Stop extra threads if we're over a soft limit in order to free up memory.
if (!first_thread && mem_tracker_->AnyLimitExceeded(MemLimit::SOFT)) {
break;
}
// Done with range and it completed successfully
if (progress_.done()) {
// All ranges are finished. Indicate we are done.
SetDone();
break;
}
if (scan_range == nullptr && num_unqueued_files == 0) {
unique_lock l(lock_);
// All ranges have been queued and DiskIoMgr has no more new ranges for
this scan
// node to process. This means that every range is either done or being
processed by
// another thread.
all_ranges_started_ = true;
break;
}
}
{code}
What if we have the following scenario:
T1) grab scan range 1 and start processing
T2) grab scan range 2 and start processing
T1) finish scan range 1 and see that 'progress_' is not done()
T1) loop around, get no scan range (there are no more), so set
all_ranges_satrted_ and break
T1) thread exits
T2) finish scan range 2
T2) happen to hit a soft memory limit error due to pressure from other exec
nodes, etc. Since we aren't the first thread, we break. (even though the first
thread is no longer running)
T2) thread exits
Note that no one got to the point of calling SetDone() because we break due to
the memory limit error _before_ checking progress_.Done().
Thus, the query will hang forever.
was:
As reported on the mailing list, this is a regression due to IMPALA-7096
(7ccf7369085aa49a8fc0daf6f91d97b8a3135682). The scanner thread has the
following code:
// Stop extra threads if we're over a soft limit in order to free up memory.
if (!first_thread && mem_tracker_->AnyLimitExceeded(MemLimit::SOFT)) {
break;
}
// Done with range and it completed successfully
if (progress_.done()) {
// All ranges are finished. Indicate we are done.
SetDone();
break;
}
if (scan_range == nullptr && num_unqueued_files == 0) {
unique_lock l(lock_);
// All ranges have been queued and DiskIoMgr has no more new ranges for
this scan
// node to process. This means that every range is either done or being
processed by
// another thread.
all_ranges_started_ = true;
break;
}
}
What if we have the following scenario:
T1) grab scan range 1 and start processing
T2) grab scan range 2 and start processing
T1) finish scan range 1 and see that 'progress_' is not done()
T1) loop around, get no scan range (there are no more), so set
all_ranges_satrted_ and break
T1) thread exits
T2) finish scan range 2
T2) happen to hit a soft memory limit error due to pressure from other exec
nodes, etc. Since we aren't the first thread, we break. (even though the first
thread is no longer running)
T2) thread exits
Note that no one got to the point of calling SetDone() because we break due to
the memory limit error _before_ checking progress_.Done().
Thus, the query will hang forever.
> Hung scanner when soft memory limit exceeded
>
>
> Key: IMPALA-7517
> URL: https://issues.apache.org/jira/browse/IMPALA-7517
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: Todd Lipcon
>Assignee: Todd Lipcon
>Priority: Critical
>
> As reported on the mailing list, this is a regression due to IMPALA-7096
> (7ccf7369085aa49a8fc0daf6f91d97b8a3135682). The scanner thread has the
> following code:
> {code}
> // Stop extra threads if we're over a soft limit in order to free up
> memory.
> if (!first_thread && mem_tracker_->AnyLimitExceeded(MemLimit::SOFT)) {
> break;
> }
>
> // Done with range and it completed successfully
> if (progress_.done()) {
> // All ranges are finished. Indicate we are done.
> SetDone();
> break;
> }
>
> if (scan_range == nullptr && num_unqueued_files == 0) {
> unique_lock l(lock_);
> // All ranges have been queued and DiskIoMgr has no more new ranges for
> this scan
> // node to process. This means that every range is either done or being
> processed by
> // another thread.
> all_ranges_started_ = true;
> break;
> }
> }
> {code}
>
> What if we have the following scenario:
>
> T1) grab scan range 1 and start processing
>
> T2) grab scan range 2 and start processing
>
> T1) finish scan range 1 and see that 'progress_' is
[jira] [Created] (IMPALA-7517) Hung scanner when soft memory limit exceeded
Todd Lipcon created IMPALA-7517:
---
Summary: Hung scanner when soft memory limit exceeded
Key: IMPALA-7517
URL: https://issues.apache.org/jira/browse/IMPALA-7517
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.1.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
As reported on the mailing list, this is a regression due to IMPALA-7096
(7ccf7369085aa49a8fc0daf6f91d97b8a3135682). The scanner thread has the
following code:
// Stop extra threads if we're over a soft limit in order to free up memory.
if (!first_thread && mem_tracker_->AnyLimitExceeded(MemLimit::SOFT)) {
break;
}
// Done with range and it completed successfully
if (progress_.done()) {
// All ranges are finished. Indicate we are done.
SetDone();
break;
}
if (scan_range == nullptr && num_unqueued_files == 0) {
unique_lock l(lock_);
// All ranges have been queued and DiskIoMgr has no more new ranges for
this scan
// node to process. This means that every range is either done or being
processed by
// another thread.
all_ranges_started_ = true;
break;
}
}
What if we have the following scenario:
T1) grab scan range 1 and start processing
T2) grab scan range 2 and start processing
T1) finish scan range 1 and see that 'progress_' is not done()
T1) loop around, get no scan range (there are no more), so set
all_ranges_satrted_ and break
T1) thread exits
T2) finish scan range 2
T2) happen to hit a soft memory limit error due to pressure from other exec
nodes, etc. Since we aren't the first thread, we break. (even though the first
thread is no longer running)
T2) thread exits
Note that no one got to the point of calling SetDone() because we break due to
the memory limit error _before_ checking progress_.Done().
Thus, the query will hang forever.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IMPALA-2200) [Cloudera][ODBC] (10000) General error: Unexpected exception has been caught.
[ https://issues.apache.org/jira/browse/IMPALA-2200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598011#comment-16598011 ] John Kew commented on IMPALA-2200: -- Hi. I'm from Tableau. I'm not sure if the driver has changed or if this is a new type of error from Impala but I suspect that suppressing prepared queries may help here. https://onlinehelp.tableau.com/current/pro/desktop/en-us/odbc_capabilities.html CAP_ODBC_SUPPRESS_PREPARED_QUERY_FOR_NON_COMMAND_QUERIES > [Cloudera][ODBC] (1) General error: Unexpected exception has been caught. > - > > Key: IMPALA-2200 > URL: https://issues.apache.org/jira/browse/IMPALA-2200 > Project: IMPALA > Issue Type: Bug > Components: Clients >Affects Versions: Impala 2.2 > Environment: Server: > OS: CentOS 6.6 > CDH 5.4.4 cluster with Cloudera manager - > Version: Cloudera Express 5.4.3 (#258 built by jenkins on 20150625-2046 git: > 1139ffb081360fbbb1be8a19a87ca3c52ee4b1cf) > Impala: impalad version 2.2.0-cdh5.4.4 RELEASE (build > a13d3c6b203e79a284b509df821bffbe229e6dc3) > Client: > OS: Windows 7/8 > client app: Tableau Desktop > Latest, at the moment, Impala-ODBC driver - > http://www.cloudera.com/content/cloudera/en/downloads/connectors/impala/odbc/impala-odbc-v2-5-29.html > Tableau desktop: version 8/9 >Reporter: Boyan Bonev >Assignee: Syed A. Hashmi >Priority: Minor > Labels: impala, odbc > Attachments: SimbaImpalaODBC_driver.log > > > Hi, > We are have a CDH 5.4.4 cluster and use Impala for add-hoc data analysis. We > also use Tableau for data visualization. The Tableau connects to Impala using > your ODBC driver. We experience several issues, but most important one is > that sometimes (1 out of 10 queries) our queries fail for unknown issue. When > we retry the issues is gone till the next occurrence. There is no info in the > server and almost no info in the ODBC client log (see attachment). Actually > the the only message that the client shows is logged in the ODBC log: > Aug 10 14:02:43 ERROR 8284 Statement::SQLPrepareW: [Cloudera][ODBC] (1) > General error: Unexpected exception has been caught. > Judging by this we suspect that this might be an ODBC driver issue in the > client in the prepared statement logic. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7517) Hung scanner when soft memory limit exceeded
Todd Lipcon created IMPALA-7517:
---
Summary: Hung scanner when soft memory limit exceeded
Key: IMPALA-7517
URL: https://issues.apache.org/jira/browse/IMPALA-7517
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.1.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
As reported on the mailing list, this is a regression due to IMPALA-7096
(7ccf7369085aa49a8fc0daf6f91d97b8a3135682). The scanner thread has the
following code:
// Stop extra threads if we're over a soft limit in order to free up memory.
if (!first_thread && mem_tracker_->AnyLimitExceeded(MemLimit::SOFT)) {
break;
}
// Done with range and it completed successfully
if (progress_.done()) {
// All ranges are finished. Indicate we are done.
SetDone();
break;
}
if (scan_range == nullptr && num_unqueued_files == 0) {
unique_lock l(lock_);
// All ranges have been queued and DiskIoMgr has no more new ranges for
this scan
// node to process. This means that every range is either done or being
processed by
// another thread.
all_ranges_started_ = true;
break;
}
}
What if we have the following scenario:
T1) grab scan range 1 and start processing
T2) grab scan range 2 and start processing
T1) finish scan range 1 and see that 'progress_' is not done()
T1) loop around, get no scan range (there are no more), so set
all_ranges_satrted_ and break
T1) thread exits
T2) finish scan range 2
T2) happen to hit a soft memory limit error due to pressure from other exec
nodes, etc. Since we aren't the first thread, we break. (even though the first
thread is no longer running)
T2) thread exits
Note that no one got to the point of calling SetDone() because we break due to
the memory limit error _before_ checking progress_.Done().
Thus, the query will hang forever.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7516) Rejected queries dont get removed from the list of running queries
Bikramjeet Vig created IMPALA-7516: -- Summary: Rejected queries dont get removed from the list of running queries Key: IMPALA-7516 URL: https://issues.apache.org/jira/browse/IMPALA-7516 Project: IMPALA Issue Type: Bug Components: Backend Affects Versions: Impala 3.1.0 Reporter: Bikramjeet Vig Steps to reproduce: Eg. set mem_limit=10b; select 1; "Number of running queries with fragments on this host" on the impala queries debug page will show 1 even though the query was rejected -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-7439) CREATE DATABASE creates a catalog entry with empty location field
[ https://issues.apache.org/jira/browse/IMPALA-7439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved IMPALA-7439. - Resolution: Fixed Fix Version/s: Impala 3.1.0 > CREATE DATABASE creates a catalog entry with empty location field > - > > Key: IMPALA-7439 > URL: https://issues.apache.org/jira/browse/IMPALA-7439 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Major > Fix For: Impala 3.1.0 > > > 'create database foobar' followed by 'describe database foobar' shows no > location field for the newly created DB. If you then invalidate metadata, it > will load the proper location upon a reload. > It seems that the code for CREATE TABLE is currently working around this as > ImpaladCatalog.getTablePath() actually ignores the location stored in the > catalog entry and instead goes directly to the HMS to get the database > location. It seems we should probably populate the catalog entry correctly > upon creation. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-7439) CREATE DATABASE creates a catalog entry with empty location field
[ https://issues.apache.org/jira/browse/IMPALA-7439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved IMPALA-7439. - Resolution: Fixed Fix Version/s: Impala 3.1.0 > CREATE DATABASE creates a catalog entry with empty location field > - > > Key: IMPALA-7439 > URL: https://issues.apache.org/jira/browse/IMPALA-7439 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Major > Fix For: Impala 3.1.0 > > > 'create database foobar' followed by 'describe database foobar' shows no > location field for the newly created DB. If you then invalidate metadata, it > will load the proper location upon a reload. > It seems that the code for CREATE TABLE is currently working around this as > ImpaladCatalog.getTablePath() actually ignores the location stored in the > catalog entry and instead goes directly to the HMS to get the database > location. It seems we should probably populate the catalog entry correctly > upon creation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-7421) Static methods called with wrong JNI function
[ https://issues.apache.org/jira/browse/IMPALA-7421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved IMPALA-7421. - Resolution: Fixed > Static methods called with wrong JNI function > - > > Key: IMPALA-7421 > URL: https://issues.apache.org/jira/browse/IMPALA-7421 > Project: IMPALA > Issue Type: Bug > Components: Backend >Affects Versions: Impala 3.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Major > Fix For: Impala 3.1.0 > > > There are various static Java methods which are loaded with > JniUtil::LoadStaticJniMethod. Many of them are then called using > 'CallObjectMethod' instead of the appropriate 'CallStaticObjectMethod'. This > seems to cause crashes when running with -Xcheck:jni -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (IMPALA-6380) Replace FunctionContext::IsArgConstant(int i) with constant during codegen
[
https://issues.apache.org/jira/browse/IMPALA-6380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Ho reassigned IMPALA-6380:
--
Assignee: (was: Zoram Thanga)
> Replace FunctionContext::IsArgConstant(int i) with constant during codegen
> --
>
> Key: IMPALA-6380
> URL: https://issues.apache.org/jira/browse/IMPALA-6380
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Affects Versions: Impala 2.12.0
>Reporter: Michael Ho
>Priority: Minor
> Labels: codegen, newbie
>
> There are few places in the string related built-in functions which check
> whether the input argument is constant per row:
> {noformat}
> TimestampVal TimestampFunctions::ToTimestamp(FunctionContext* context,
> const StringVal& date, const StringVal& fmt) {
> if (fmt.is_null || fmt.len == 0) {
> TimestampFunctions::ReportBadFormat(context, fmt, false);
> return TimestampVal::null();
> }
> if (date.is_null || date.len == 0) return TimestampVal::null();
> void* state = context->GetFunctionState(FunctionContext::THREAD_LOCAL);
> DateTimeFormatContext* dt_ctx =
> reinterpret_cast(state);
> if (!context->IsArgConstant(1)) {
> dt_ctx->Reset(reinterpret_cast(fmt.ptr), fmt.len);
> if (!TimestampParser::ParseFormatTokens(dt_ctx)) {
>ReportBadFormat(context, fmt, false);
>return TimestampVal::null();
> }
> }
> const TimestampValue& tv = TimestampValue::Parse(
> reinterpret_cast(date.ptr), date.len, *dt_ctx);
> TimestampVal tv_val;
> tv.ToTimestampVal(_val);
> return tv_val;
> }
> {noformat}
> {noformat}
> StringVal StringFunctions::BTrimString(FunctionContext* ctx,
> const StringVal& str, const StringVal& chars_to_trim) {
> if (str.is_null) return StringVal::null();
> bitset<256>* unique_chars = reinterpret_cast*>(
> ctx->GetFunctionState(FunctionContext::THREAD_LOCAL));
> // When 'chars_to_trim' is unique for each element (e.g. when
> 'chars_to_trim'
> // is each element of a table column), we need to prepare a bitset of unique
> // characters here instead of using the bitset from function context.
> if (!ctx->IsArgConstant(1)) {
> unique_chars->reset();
> DCHECK(chars_to_trim.len != 0 || chars_to_trim.is_null);
> for (int32_t i = 0; i < chars_to_trim.len; ++i) {
> unique_chars->set(static_cast(chars_to_trim.ptr[i]), true);
> }
> }
> ...
> ...
> {noformat}
> it seems generally useful to extend {{FunctionContextImpl::GetConstFnAttr()}}
> to also return whether the i-th argument to a UDF is constant or not and then
> replace these calls with constants during codegen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-6661) Group by float results in one group per NaN value
[
https://issues.apache.org/jira/browse/IMPALA-6661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Ho reassigned IMPALA-6661:
--
Assignee: Michal Ostrowski
> Group by float results in one group per NaN value
> -
>
> Key: IMPALA-6661
> URL: https://issues.apache.org/jira/browse/IMPALA-6661
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 2.11.0, Impala 2.12.0
>Reporter: Tim Armstrong
>Assignee: Michal Ostrowski
>Priority: Major
> Labels: correctness, perf, ramp-up
>
> I don't know if this is the desired behaviour but it could be problematic for
> some users since it will blow up the number of distinct groups in an
> aggregation. I suspect that it's more useful to coalesce all the NaNs into a
> single group, similar to how NULL is handled in GROUP BY.
> {noformat}
> [localhost:21000] > select distinct * from (values(cast("nan" as float)),
> (cast("nan" as float)), (sqrt(cast("-1" as float v;
> +--+
> | cast('nan' as float) |
> +--+
> | NaN |
> | NaN |
> | NaN |
> +--+
> Fetched 3 row(s) in 0.11s
> {noformat}
> I suspect IMPALA-6069 slightly changed the behaviour here, although it would
> have been broken beforehand anyway, since not all NaNs have the same bit
> pattern, so Equals() and Hash() were inconsistent.
> We should decided what the preferred behaviour is and tweak the behaviour of
> the hash table to produce it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-6323) Support a constant in a window specification
[
https://issues.apache.org/jira/browse/IMPALA-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Ho reassigned IMPALA-6323:
--
Assignee: Michal Ostrowski
> Support a constant in a window specification
>
>
> Key: IMPALA-6323
> URL: https://issues.apache.org/jira/browse/IMPALA-6323
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
>Affects Versions: Impala 2.9.0
>Reporter: N Campbell
>Assignee: Michal Ostrowski
>Priority: Major
>
> A query may attempt to generate a function/aggregate with a window
> specification.
> Currently, Impala does not support a constant in the specification. Similarly
> it may also fail if an expression is generated which via constant folding
> becomes a constant.
> Potentially an application may have to trigger dynamic expressions which
> participate in a window specification. It may not know that a
> value-expression that is generated might be constant folded and disallowed.
> Hence, while it may attempt to decompose when it knows a literal is being
> used cannot guard for all cases
> Error: [Simba][ImpalaJDBCDriver](500051) ERROR processing query/statement.
> Error Code: 0, SQL state: TStatus(statusCode:ERROR_STATUS, sqlState:HY000,
> errorMessage:AnalysisException: Expressions in the PARTITION BY clause must
> not be constant: upper('a') (in sum(qty) OVER (PARTITION BY upper('a')))
> ), Query: SELECT
> `sno` AS `c1`,
> SUM(`qty`)
> OVER(
> PARTITION BY
> upper('a')
> ) AS `c2`
> FROM `cert`.`tsupply`.
> SQLState: HY000
> ErrorCode: 500051
> -vs-
> SELECT
> `sno` AS `c1`,
> SUM(`qty`)
> OVER(
> PARTITION BY
> upper(concat(`sno`,''))
> ) AS `c2`
> FROM `cert`.`tsupply`
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-6590) Disable expr rewrites and codegen for VALUES() statements
[
https://issues.apache.org/jira/browse/IMPALA-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Ho reassigned IMPALA-6590:
--
Assignee: Michal Ostrowski
> Disable expr rewrites and codegen for VALUES() statements
> -
>
> Key: IMPALA-6590
> URL: https://issues.apache.org/jira/browse/IMPALA-6590
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
>Affects Versions: Impala 2.8.0, Impala 2.9.0, Impala 2.10.0, Impala 2.11.0
>Reporter: Alexander Behm
>Assignee: Michal Ostrowski
>Priority: Critical
> Labels: perf, planner, regression
>
> The analysis of statements with big VALUES clauses like INSERT INTO
> VALUES is slow due to expression rewrites like constant folding. The
> performance of such statements has regressed since the introduction of expr
> rewrites and constant folding in IMPALA-1788.
> We should skip expr rewrites for VALUES altogether since it mostly provides
> no benefit but can have a large overhead due to evaluation of expressions in
> the backend (constant folding). These expressions are ultimately evaluated
> and materialized in the backend anyway, so there's no point in folding them
> during analysis.
> Similarly, there is no point in doing codegen for these exprs in the backend
> union node.
> *Workaround*
> {code}
> SET ENABLE_EXPR_REWRITES=FALSE;
> SET DISABLE_CODEGEN=TRUE;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-2063) impala 'cancelled' query status now has extra characters in it
[ https://issues.apache.org/jira/browse/IMPALA-2063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Ho reassigned IMPALA-2063: -- Assignee: Michal Ostrowski > impala 'cancelled' query status now has extra characters in it > -- > > Key: IMPALA-2063 > URL: https://issues.apache.org/jira/browse/IMPALA-2063 > Project: IMPALA > Issue Type: Bug > Components: Backend >Affects Versions: Impala 2.2 >Reporter: Chris Leroy >Assignee: Michal Ostrowski >Priority: Minor > Labels: query-lifecycle, ramp-up > Attachments: query_cancelled.png, query_completed.png > > > When a query is cancelled (e.g. via Beeswax.cancel()), the query status has a > trailing newline. This is visible in the query profile, see sample images. > The trailing newline should be removed. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-6634) Impala 2.13 Doc: Document leading and trailing whitespace behaviour with string->timestamp conversion
[ https://issues.apache.org/jira/browse/IMPALA-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-6634: Target Version: Impala 3.1.0 (was: Impala 2.13.0) > Impala 2.13 Doc: Document leading and trailing whitespace behaviour with > string->timestamp conversion > - > > Key: IMPALA-6634 > URL: https://issues.apache.org/jira/browse/IMPALA-6634 > Project: IMPALA > Issue Type: Sub-task > Components: Docs >Affects Versions: Impala 3.1.0 >Reporter: Tim Armstrong >Assignee: Alex Rodoni >Priority: Major > Labels: docs, future_release_doc > > Just putting this on your radar. The docs don't mention this behaviour. It > seems fairly minor but may be useful to clarify. > Ref: > https://impala.apache.org/docs/build/html/topics/impala_literals.html#timestamp_literals -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-6634) Impala 2.13 Doc: Document leading and trailing whitespace behaviour with string->timestamp conversion
[ https://issues.apache.org/jira/browse/IMPALA-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-6634: Affects Version/s: (was: Impala 2.13.0) Impala 3.1.0 > Impala 2.13 Doc: Document leading and trailing whitespace behaviour with > string->timestamp conversion > - > > Key: IMPALA-6634 > URL: https://issues.apache.org/jira/browse/IMPALA-6634 > Project: IMPALA > Issue Type: Sub-task > Components: Docs >Affects Versions: Impala 3.1.0 >Reporter: Tim Armstrong >Assignee: Alex Rodoni >Priority: Major > Labels: docs, future_release_doc > > Just putting this on your radar. The docs don't mention this behaviour. It > seems fairly minor but may be useful to clarify. > Ref: > https://impala.apache.org/docs/build/html/topics/impala_literals.html#timestamp_literals -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7515) Remove toolchain Kudu support
Thomas Tauber-Marshall created IMPALA-7515: -- Summary: Remove toolchain Kudu support Key: IMPALA-7515 URL: https://issues.apache.org/jira/browse/IMPALA-7515 Project: IMPALA Issue Type: Improvement Affects Versions: Impala 3.1.0 Reporter: Thomas Tauber-Marshall IMPALA-7499 is adding support for pulling Kudu in from CDH. For now, to limit impact on developers, we've kept the ability to build with the toolchain Kudu since the CDH Kudu is provided for fewer OSes than the toolchain. In particular the CDH Kudu is not available for Ubuntu 14, which is still used by many devs. Eventually, we should remove the toolchain Kudu support as it adds complexity to the build system. A good time to do this would be around EOL for Ubuntu 14, currently scheduled for April 2019 Note that even after removing toolchain Kudu support, it will be possible to build and run Impala on OSes for which CDH tarballs are not provided in two ways: - setting KUDU_IS_SUPPORTED=false, which will build a non-functional Kudu client stub Impala can be linked against - setting KUDU_CLIENT_DIR/KUDU_BUILD_DIR to point to a local build of Kudu -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7515) Remove toolchain Kudu support
Thomas Tauber-Marshall created IMPALA-7515: -- Summary: Remove toolchain Kudu support Key: IMPALA-7515 URL: https://issues.apache.org/jira/browse/IMPALA-7515 Project: IMPALA Issue Type: Improvement Affects Versions: Impala 3.1.0 Reporter: Thomas Tauber-Marshall IMPALA-7499 is adding support for pulling Kudu in from CDH. For now, to limit impact on developers, we've kept the ability to build with the toolchain Kudu since the CDH Kudu is provided for fewer OSes than the toolchain. In particular the CDH Kudu is not available for Ubuntu 14, which is still used by many devs. Eventually, we should remove the toolchain Kudu support as it adds complexity to the build system. A good time to do this would be around EOL for Ubuntu 14, currently scheduled for April 2019 Note that even after removing toolchain Kudu support, it will be possible to build and run Impala on OSes for which CDH tarballs are not provided in two ways: - setting KUDU_IS_SUPPORTED=false, which will build a non-functional Kudu client stub Impala can be linked against - setting KUDU_CLIENT_DIR/KUDU_BUILD_DIR to point to a local build of Kudu -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IMPALA-7513) Fix Sentry compilation error in Impala 2.x branch
[ https://issues.apache.org/jira/browse/IMPALA-7513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-7513: - Labels: broken-build (was: ) > Fix Sentry compilation error in Impala 2.x branch > - > > Key: IMPALA-7513 > URL: https://issues.apache.org/jira/browse/IMPALA-7513 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Labels: broken-build > Fix For: Impala 2.12.0 > > > [ERROR] > /tmp/jenkins/workspace/impala-compile-and-publish/fe/src/main/java/org/apache/impala/catalog/AuthorizationPolicy.java:[58,8] > org.apache.impala.catalog.AuthorizationPolicy is not abstract and does not > override abstract method > listPrivileges(java.util.Set,java.util.Set,org.apache.sentry.core.common.ActiveRoleSet) > in org.apache.sentry.provider.cache.PrivilegeCache -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7487) Failures in stress test when running against minicluster: "int() argument must be a string or a number, not 'NoneType'"
[
https://issues.apache.org/jira/browse/IMPALA-7487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597825#comment-16597825
]
Michael Brown commented on IMPALA-7487:
---
[~tarmstrong] I'm probably not going to get to this before I go on vacation.
> Failures in stress test when running against minicluster: "int() argument
> must be a string or a number, not 'NoneType'"
> ---
>
> Key: IMPALA-7487
> URL: https://issues.apache.org/jira/browse/IMPALA-7487
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.1.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Major
>
> I ran this:
> {code}
> ./tests/stress/concurrent_select.py --tpch-db=tpch_parquet --max-queries
> 1 --timeout-multiplier 100 --mem-overcommit-pct 150
> --fail-upon-successive-errors=1000
> {code}
> After the binary search succeeds lots of queries fail with this error:
> {noformat}
> Process Process-39:
> Traceback (most recent call last):
> File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in
> _bootstrap
> self.run()
> File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
> self._target(*self._args, **self._kwargs)
> File "./tests/stress/concurrent_select.py", line 703, in
> _start_single_runner
> if should_cancel:
> TypeError: int() argument must be a string or a number, not 'NoneType'
> func(*targs, **kargs)
> File "/usr/lib/python2.7/multiprocessing/util.py", line 328, in
> _exit_function
> _run_finalizers()
> File "/usr/lib/python2.7/multiprocessing/util.py", line 274, in
> _run_finalizers
> finalizer()
> File "/usr/lib/python2.7/multiprocessing/util.py", line 207, in __call__
> res = self._callback(*self._args, **self._kwargs)
> File "/usr/lib/python2.7/multiprocessing/queues.py", line 218, in
> _finalize_join
> thread.join()
> File "/usr/lib/python2.7/threading.py", line 940, in join
> self.__block.wait()
> File "/usr/lib/python2.7/threading.py", line 340, in wait
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-3885) Parquet files with multiple blocks cause remote reads
[
https://issues.apache.org/jira/browse/IMPALA-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni updated IMPALA-3885:
Docs Text: (was: Explain this.)
> Parquet files with multiple blocks cause remote reads
> -
>
> Key: IMPALA-3885
> URL: https://issues.apache.org/jira/browse/IMPALA-3885
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 2.7.0
>Reporter: Lars Volker
>Assignee: Sailesh Mukil
>Priority: Critical
>
> For parquet files with multiple blocks we schedule the scan ranges across all
> replicas of the blocks, aiming for local execution on impalads, which are
> running on the datanodes where the blocks are located. However there seems to
> be a high number of remote reads in these scenarios.
> The scheduler makes local assignments:
> {noformat}
> I0613 16:43:01.741288 36424 simple-scheduler.cc:605] Total remote scan volume
> = 0
> I0613 16:43:01.741441 36424 simple-scheduler.cc:607] Total local scan volume
> = 1426.95 GB
> I0613 16:43:01.741576 36424 simple-scheduler.cc:609] Total cached scan volume
> = 0
> {noformat}
> However the profile shows remote scans:
> {noformat}
> - RemoteScanRanges: 283 (283)
> - RowsRead: 139.36M (139355074)
> - RowsReturned: 239 (239)
> - RowsReturnedRate: 127.00 /sec
> - ScanRangesComplete: 304 (304)
> {noformat}
> Somehow Impala seems to read data from the wrong datanodes, possibly reading
> everything from the one where it read the footer from.
> [~sailesh] - We briefly talked about this before in person. Do you have time
> to look at this? If not feel free to re-assign it to me.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-3638) Remove lazy creation of LLVM codegen module
[
https://issues.apache.org/jira/browse/IMPALA-3638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni updated IMPALA-3638:
Docs Text: (was: This change improves performance (2x improvement in
benchmark) by enabling codegen of expressions for queries with plan fragments
that contain only non-codegen enabled operators such as AnalyticEvalNode.)
> Remove lazy creation of LLVM codegen module
> ---
>
> Key: IMPALA-3638
> URL: https://issues.apache.org/jira/browse/IMPALA-3638
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 2.2
>Reporter: Mostafa Mokhtar
>Assignee: Michael Ho
>Priority: Major
> Labels: codegen
> Fix For: Impala 2.8.0
>
> Attachments: primitive_orderby_bigint.test.2.out
>
>
> Query
> {code}
> select *
> FROM (
> SELECT Rank() OVER (
> ORDER BY l_extendedprice
> ,l_quantity
> ,l_discount
> ,l_tax
> ) AS rank
> FROM lineitem
> WHERE l_shipdate < '1992-05-09'
> ) a
> WHERE rank < 10
> {code}
> Plan
> {code}
> 03:SELECT
> | predicates: rank() < 10
> | hosts=20 per-host-mem=unavailable
> | tuple-ids=6,5 row-size=66B cardinality=5897
> |
> 02:ANALYTIC
> | functions: rank()
> | order by: l_extendedprice ASC, l_quantity ASC, l_discount ASC, l_tax ASC
> | window: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
> | hosts=20 per-host-mem=unavailable
> | tuple-ids=6,5 row-size=66B cardinality=58971
> |
> 04:MERGING-EXCHANGE [UNPARTITIONED]
> | order by: l_extendedprice ASC, l_quantity ASC, l_discount ASC, l_tax ASC
> | hosts=20 per-host-mem=unavailable
> | tuple-ids=6 row-size=58B cardinality=58971
> |
> 01:SORT
> | order by: l_extendedprice ASC, l_quantity ASC, l_discount ASC, l_tax ASC
> | hosts=20 per-host-mem=736.00MB
> | tuple-ids=6 row-size=58B cardinality=58971
> |
> 00:SCAN HDFS [tpch_1000_decimal_parquet.lineitem, RANDOM]
>partitions=1/1 files=880 size=216.61GB
>predicates: l_shipdate < '1992-05-09'
>table stats: 589709 rows total
>column stats: all
>hosts=20 per-host-mem=440.00MB
>tuple-ids=0 row-size=58B cardinality=58971
> {code}
> Fragment not getting codegened
> {code}
> Fragment start latencies: Count: 20, 25th %-ile: 1ms, 50th %-ile: 2ms, 75th
> %-ile: 2ms, 90th %-ile: 2ms, 95th %-ile: 3ms, 99.9th %-ile: 3ms
> Per Node Peak Memory Usage: d2406.halxg.cloudera.com:22000(1.38 GB)
> d2413.halxg.cloudera.com:22000(1.52 GB) d2405.halxg.cloudera.com:22000(1.34
> GB) d2414.halxg.cloudera.com:22000(1.51 GB)
> d2416.halxg.cloudera.com:22000(1.48 GB) d2404.halxg.cloudera.com:22000(1.37
> GB) d2420.halxg.cloudera.com:22000(1.40 GB)
> d2410.halxg.cloudera.com:22000(1.61 GB) d2412.halxg.cloudera.com:22000(1.22
> GB) d2419.halxg.cloudera.com:22000(1.38 GB)
> d2409.halxg.cloudera.com:22000(1.41 GB) d2407.halxg.cloudera.com:22000(1.10
> GB) d2411.halxg.cloudera.com:22000(1.34 GB)
> d2418.halxg.cloudera.com:22000(1.27 GB) d2408.halxg.cloudera.com:22000(1.56
> GB) d2421.halxg.cloudera.com:22000(1.35 GB)
> d2403.halxg.cloudera.com:22000(1.28 GB) d2415.halxg.cloudera.com:22000(1.53
> GB) d2417.halxg.cloudera.com:22000(1.31 GB)
> d2402.halxg.cloudera.com:22000(1.25 GB)
> - FiltersReceived: 0 (0)
> - FinalizationTimer: 0.000ns
> Coordinator Fragment F01:(Total: 8m48s, non-child: 3.044ms, % non-child:
> 0.00%)
> MemoryUsage(16s000ms): 16.00 KB, 33.59 MB, 47.13 MB, 47.41 MB, 46.89
> MB, 46.71 MB, 46.93 MB, 46.69 MB, 46.52 MB, 46.67 MB, 46.34 MB, 46.76 MB,
> 46.63 MB, 46.58 MB, 46.44 MB, 46.64 MB, 46.91 MB, 47.02 MB, 46.60 MB, 46.62
> MB, 46.65 MB, 46.60 MB, 46.80 MB, 46.88 MB, 46.69 MB, 46.61 MB, 46.88 MB,
> 46.58 MB, 46.71 MB, 46.76 MB, 46.49 MB, 46.36 MB, 46.07 MB
>- AverageThreadTokens: 0.00
>- BloomFilterBytes: 0
>- PeakMemoryUsage: 59.87 MB (62781648)
>- PerHostPeakMemUsage: 0
>- PrepareTime: 138.114us
>- RowsProduced: 9 (9)
>- TotalCpuTime: 8m49s
>- TotalNetworkReceiveTime: 0.000ns
>- TotalNetworkSendTime: 0.000ns
>- TotalStorageWaitTime: 0.000ns
> BlockMgr:
> - BlockWritesOutstanding: 0 (0)
> - BlocksCreated: 71 (71)
> - BlocksRecycled: 1.33K (1333)
> - BufferedPins: 0 (0)
> - BytesWritten: 0
> - MaxBlockSize: 8.00 MB (8388608)
> - MemoryLimit: 242.23 GB (260091396096)
> - PeakMemoryUsage: 736.00 MB (771751936)
> - TotalBufferWaitTime: 0.000ns
> - TotalEncryptionTime: 0.000ns
> - TotalIntegrityCheckTime: 0.000ns
> - TotalReadBlockTime: 0.000ns
> SELECT_NODE (id=3):(Total: 8m48s, non-child: 8s327ms, % non-child:
> 1.57%)
> - PeakMemoryUsage: 9.01 MB (9449472)
>
[jira] [Updated] (IMPALA-4269) Codegen for Merging-Exchange node
[
https://issues.apache.org/jira/browse/IMPALA-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni updated IMPALA-4269:
Docs Text: (was: This change codegens the sorting expressions in merging
exchange operator. Benchmarking shows there can be up to 50% improvement in
average runtime of merging exchange operator in certain cases. )
> Codegen for Merging-Exchange node
> -
>
> Key: IMPALA-4269
> URL: https://issues.apache.org/jira/browse/IMPALA-4269
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Affects Versions: Impala 2.8.0
>Reporter: Michael Ho
>Assignee: Michael Ho
>Priority: Minor
> Labels: codegen
> Fix For: Impala 2.8.0
>
>
> We should codegen the TupleRowComparator for merging-exchange operator. This
> should help a lot for queries such as:
> {noformat}
> SELECT *
> FROM (SELECT Rank()
> OVER(
>ORDER BY l_orderkey) AS rank
> FROM lineitem
> WHERE l_shipdate < '1992-05-09') a
> WHERE rank < 10;
> {noformat}
> {noformat}
> 03:SELECT
> | predicates: rank() < 10
> |
> 02:ANALYTIC
> | functions: rank()
> | order by: l_orderkey ASC
> | window: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
> |
> 04:MERGING-EXCHANGE [UNPARTITIONED]
> | order by: l_orderkey ASC
> |
> 01:SORT
> | order by: l_orderkey ASC
> |
> 00:SCAN HDFS [tpch_parquet.lineitem]
>partitions=1/1 files=3 size=193.61MB
>predicates: l_shipdate < '1992-05-09'
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-3884) Enable codegen for TIMESTAMP in hash table.
[
https://issues.apache.org/jira/browse/IMPALA-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni updated IMPALA-3884:
Docs Text: (was: Before this fix, codegen was disable for joins on
timestamps, GROUP BY on timestamps, and select distinct of timestamp columns.
With this fix, affected aggregations are significantly faster (e.g. 5x).)
> Enable codegen for TIMESTAMP in hash table.
> ---
>
> Key: IMPALA-3884
> URL: https://issues.apache.org/jira/browse/IMPALA-3884
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Affects Versions: Impala 2.5.0, Impala 2.4.0, Impala 2.6.0
>Reporter: Alexander Behm
>Assignee: Michael Ho
>Priority: Major
> Labels: codegen, compute-stats, performance
> Fix For: Impala 2.8.0
>
>
> Referencing TIMESTAMP columns in joins and aggrregations prevents those exec
> nodes from being codegen, leading to a severe performance hit. This may
> impact queries with group-by or join on timestamp column.
> See this snippet from HashTableCtx::CodegenEvalRow() in hash-table.cc:
> {code}
> // TODO: CodegenAssignNullValue() can't handle TYPE_TIMESTAMP or
> TYPE_DECIMAL yet
> const vector& ctxs = build ? build_expr_ctxs_ :
> probe_expr_ctxs_;
> for (int i = 0; i < ctxs.size(); ++i) {
> PrimitiveType type = ctxs[i]->root()->type().type;
> if (type == TYPE_TIMESTAMP || type == TYPE_CHAR) {
> return Status(Substitute("HashTableCtx::CodegenEvalRow(): type $0 NYI",
> TypeToString(type)));
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-4180) Crash: impala::DiskIoRequestContext::Cancel
[
https://issues.apache.org/jira/browse/IMPALA-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni updated IMPALA-4180:
Docs Text: (was: This change fixes an issue which may crash Impalad for
queries with plan fragments containing multiple HDFS scan node. This is more
likely to happen if the query option num_nodes is set to 1 or if
exec_single_node_rows_threshold is set to a large value which makes it more
likely to generate plan fragments which contain multiple HDFS scan nodes.
Workaround is to set num_nodes to 0 or a number greater than 1 and set
exec_single_node_rows_threshold to 0.)
> Crash: impala::DiskIoRequestContext::Cancel
> ---
>
> Key: IMPALA-4180
> URL: https://issues.apache.org/jira/browse/IMPALA-4180
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 2.7.0
>Reporter: Taras Bobrovytsky
>Assignee: Michael Ho
>Priority: Blocker
> Labels: crash
> Fix For: Impala 2.6.3, Impala 2.8.0
>
>
> A crash occurs if the following query is run in a loop:
> {code}
> for i in {1..1000}; do impala-shell.sh -q "set num_nodes=1; set
> DISABLE_CODEGEN=1; set NUM_SCANNER_THREADS=1; set RUNTIME_FILTER_MODE=0; with
> t as (select int_col x, bigint_col y from functional.alltypestiny order by id
> limit 2) select * from t t1 left outer join t t2 on t1.y = t2.x full outer
> join t t3 on t2.y = t3.x order by t1.x limit 10"; done
> {code}
> The query is run in single-node optimization mode (1 node, 1 scanner thread,
> no codegen, no runtime filters). The crash is non deterministic, so it can
> take a few minutes for it to occur.
> Stack Trace:
> {code}
> #0 0x7f792b3edc37 in __GI_raise (sig=sig@entry=6)
> at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
> #1 0x7f792b3f1028 in __GI_abort () at abort.c:89
> #2 0x7f792d613c55 in os::abort(bool) ()
>from /usr/lib/jvm/java-7-oracle-amd64/jre/lib/amd64/server/libjvm.so
> #3 0x7f792d795cd7 in VMError::report_and_die() ()
>from /usr/lib/jvm/java-7-oracle-amd64/jre/lib/amd64/server/libjvm.so
> #4 0x7f792d618b6f in JVM_handle_linux_signal ()
>from /usr/lib/jvm/java-7-oracle-amd64/jre/lib/amd64/server/libjvm.so
> #5
> #6 __GI___pthread_mutex_lock (mutex=0x0) at ../nptl/pthread_mutex_lock.c:66
> #7 0x010fb23d in pthread_mutex_lock (m=0x71)
> at /tmp/toolchain/boost-1.57.0/include/boost/thread/pthread/mutex.hpp:62
> #8 boost::mutex::lock (this=0x71)
> at /tmp/toolchain/boost-1.57.0/include/boost/thread/pthread/mutex.hpp:116
> #9 0x01104b4c in boost::lock_guard::lock_guard (
> this=0x7f78e7531b50, m_=...)
> at /tmp/toolchain/boost-1.57.0/include/boost/thread/lock_guard.hpp:38
> #10 0x0133a913 in impala::DiskIoRequestContext::Cancel (this=0x1,
> status=...)
> at
> /home/tbobrovytsky/Impala/be/src/runtime/disk-io-mgr-reader-context.cc:30
> #11 0x01328100 in impala::DiskIoMgr::CancelContext (this=0x9046a00,
> context=0x1,
> wait_for_disks_completion=true)
> at /home/tbobrovytsky/Impala/be/src/runtime/disk-io-mgr.cc:457
> #12 0x01327d0e in impala::DiskIoMgr::UnregisterContext
> (this=0x9046a00,
> reader=0x1) at /home/tbobrovytsky/Impala/be/src/runtime/disk-io-mgr.cc:425
> #13 0x0191bcfd in impala::PlanFragmentExecutor::Close (this=0xb231180)
> at /home/tbobrovytsky/Impala/be/src/runtime/plan-fragment-executor.cc:512
> #14 0x0191618d in impala::PlanFragmentExecutor::~PlanFragmentExecutor
> (
> this=0xb231180, __in_chrg=)
> at /home/tbobrovytsky/Impala/be/src/runtime/plan-fragment-executor.cc:73
> #15 0x018f4027 in boost::checked_delete
> (
> x=0xb231180) at
> /tmp/toolchain/boost-1.57.0/include/boost/core/checked_delete.hpp:34
> #16 0x018efd81 in
> boost::scoped_ptr::~scoped_ptr (
> this=0x95eac28, __in_chrg=)
> at /tmp/toolchain/boost-1.57.0/include/boost/smart_ptr/scoped_ptr.hpp:82
> #17 0x018db887 in impala::Coordinator::~Coordinator (this=0x95ea800,
> __in_chrg=)
> at /home/tbobrovytsky/Impala/be/src/runtime/coordinator.cc:365
> #18 0x0147f5ff in boost::checked_delete
> (x=0x95ea800)
> at /tmp/toolchain/boost-1.57.0/include/boost/core/checked_delete.hpp:34
> #19 0x0147b121 in boost::scoped_ptr::~scoped_ptr
> (
> this=0xaf3e3c0, __in_chrg=)
> at /tmp/toolchain/boost-1.57.0/include/boost/smart_ptr/scoped_ptr.hpp:82
> #20 0x01469330 in
> impala::ImpalaServer::QueryExecState::~QueryExecState (
> this=0xaf3e000, __in_chrg=)
> at /home/tbobrovytsky/Impala/be/src/service/query-exec-state.cc:115
> #21 0x01433ed6 in
> std::_Sp_counted_ptr (__gnu_cxx::_Lock_policy)2>::_M_dispose (this=0xb0ac4e0)
> at
[jira] [Updated] (IMPALA-4291) Lazy creation of IRFunction::Type to Function* mapping
[
https://issues.apache.org/jira/browse/IMPALA-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni updated IMPALA-4291:
Docs Text: (was: This change reduces the preparation time during codegen
of a simple query from 23ms to 10ms. Reduces the latency of short-running
query.)
> Lazy creation of IRFunction::Type to Function* mapping
> --
>
> Key: IMPALA-4291
> URL: https://issues.apache.org/jira/browse/IMPALA-4291
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Affects Versions: Impala 2.8.0
>Reporter: Michael Ho
>Assignee: Michael Ho
>Priority: Minor
> Labels: codegen
> Fix For: Impala 2.8.0
>
>
> For simple queries with one or two functions to codegen, we spent about 23ms
> or more out of 40ms of codegen time in preparation. We can save some time by
> not eagerly populating the map of {{IRFunction::Type}} to {{llvm::Function*}}
> map. This should help with the fixes for IMPALA-3638 when we create the LLVM
> module unconditionally.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-4439) Kudu DML should report the number of rows that could not be modified
[ https://issues.apache.org/jira/browse/IMPALA-4439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-4439: Docs Text: (was: Kudu DML executed in shell reuturns # rows modified and # rows attempted to modify but w/ error. No similar feedback over HiveServer2 (jdbc, odbc, etc.) -- only shell. Profile has the same info though.) > Kudu DML should report the number of rows that could not be modified > > > Key: IMPALA-4439 > URL: https://issues.apache.org/jira/browse/IMPALA-4439 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Kudu_Impala >Reporter: Matthew Jacobs >Assignee: Matthew Jacobs >Priority: Critical > Labels: kudu, supportability > Fix For: Impala 2.8.0 > > > Kudu DML operations ignore schema-related conflicts/violations (IMPALA-3710) > because Kudu does not support multi-row transactions. > While the operations will report the number of rows modified through beeswax > and the profile (unfortunately HS2 cannot return this information yet, see > IMPALA-3713), the operations should also report the total number of rows that > could not be modified. There are several reasons why rows might not be > successfully modified, and a more detailed mechanism for reporting these > errors is desirable but out of scope for now (that work is tracked by > IMPALA-4416). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-3771) Expose kudu client timeouts or set to more conservative values
[ https://issues.apache.org/jira/browse/IMPALA-3771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-3771: Docs Text: (was: Kudu client timeouts are exposed via impala gflags.) > Expose kudu client timeouts or set to more conservative values > -- > > Key: IMPALA-3771 > URL: https://issues.apache.org/jira/browse/IMPALA-3771 > Project: IMPALA > Issue Type: Task > Components: Frontend >Affects Versions: Kudu_Impala >Reporter: Matthew Jacobs >Assignee: Matthew Jacobs >Priority: Critical > Labels: kudu > Fix For: Impala 2.8.0 > > > We observed some timeouts creating Kudu tables. > We should either set more conservative timeouts or expose a way to set the > timeouts in both the FE and the BE. (In this case the issue occurred in the > FE but we should evaluate our timeouts in both places.) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-4379) Fix Kudu table type checking
[ https://issues.apache.org/jira/browse/IMPALA-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-4379: Docs Text: (was: Doc type differences) > Fix Kudu table type checking > > > Key: IMPALA-4379 > URL: https://issues.apache.org/jira/browse/IMPALA-4379 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Kudu_Impala >Reporter: Matthew Jacobs >Assignee: Matthew Jacobs >Priority: Critical > Labels: kudu > Fix For: Impala 2.8.0 > > > Creating Kudu tables shouldn't allow types not supported by Kudu (e.g. > VAR/CHAR, DECIMAL, TIMESTAMP, collection types). The behavior is > inconsistent, for some types it throws in the catalog, for VAR/CHAR these > become strings. These should all fail w/ analysis exceptions until Kudu > supports these types. > Similarly, external tables cannot contain Kudu types that Impala doesn't > support (e.g. UNIXTIME_MICROS, BINARY). This does fail, but we need tests to > verify. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-1430) Codegen all aggregate functions, including UDAs
[ https://issues.apache.org/jira/browse/IMPALA-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-1430: Docs Text: (was: This fix enables codegen for aggregate functions that previously didn't support codegen. This can lead to dramatic performance improvements (5x in some cases) The first part enables codegen for all builtin aggregate functions, except those with CHAR arguments. E.g. min(STRING), sum(TIMESTAMP), group_concat()) > Codegen all aggregate functions, including UDAs > --- > > Key: IMPALA-1430 > URL: https://issues.apache.org/jira/browse/IMPALA-1430 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 2.0 >Reporter: Skye Wanderman-Milne >Assignee: Tim Armstrong >Priority: Minor > Labels: codegen, performance, ramp-up > Fix For: Impala 2.9.0 > > > Currently codegen is disabled for the entire aggregation operator if a single > aggregate function can't be codegen'd. We should address this by making it so > all aggregate functions can be codegen'd, including UDAs. For UDAs in .so's, > the codegen'd function will call into the UDA library. This also affects > aggregation operator on timestamp. > This perf hit can be especially bad for COMPUTE STATS which is heavily CPU > bound on the aggregation and because there is no easy way to exclude the > TIMESTAMP columns when computing the column stats (i.e., there is no simple > workaround). > Even if the portions involving TIMESTAMP cannot be codegen'd it would still > be worthwhile to come up with a workaround where codegen for the other types > is still enabled. > *Workaround* > If you are experiencing very slow COMPUTE STATS due to this issue, then you > may be able to temporarily ALTER the TIMESTAMP columns to STRING or INT type > before running COMPUTE STATS. After the command completed, the columns can be > altered back to TIMESTAMP. > Note the workaround is only apply to text data, not parquet data. parquet > require compatibles data type. TIMESTAMP is INT96, it's not compatible with > STRING or BIGINT. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-4691) crash in memcpy
[ https://issues.apache.org/jira/browse/IMPALA-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-4691: Docs Text: (was: # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x003629689a78, pid=114742, tid=140479326738176 # # JRE version: Java(TM) SE Runtime Environment (7.0_80-b15) (build 1.7.0_80-b15) # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.80-b11 mixed mode linux-amd64 compressed oops) # Problematic frame: # C [libc.so.6+0x89a78] memcpy+0x3a8 # # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp # ) > crash in memcpy > --- > > Key: IMPALA-4691 > URL: https://issues.apache.org/jira/browse/IMPALA-4691 > Project: IMPALA > Issue Type: Bug > Components: Backend >Affects Versions: Impala 2.5.0 > Environment: CDH5.7 CentOS release 6.7 x64 >Reporter: Cheng Wang >Assignee: Cheng Wang >Priority: Major > Labels: crash > Attachments: hs_err_pid59248.log > > > when we use impala cluster ，the impalad ofen down。here is log: > # > # http://bugreport.java.com/bugreport/crash.jsp > # If you would like to submit a bug report, please visit: > # > # /var/run/cloudera-scm-agent/process/6930-impala-IMPALAD/hs_err_pid59248.log > # An error report file with more information is saved as: > # > # Failed to write core dump. Core dumps have been disabled. To enable core > dumping, try "ulimit -c unlimited" before starting Java again > # > # C [libc.so.6+0x89a9f] memcpy+0x3ff > # Problematic frame: > # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.80-b11 mixed mode > linux-amd64 compressed oops) > # JRE version: Java(TM) SE Runtime Environment (7.0_80-b15) (build > 1.7.0_80-b15) > # > # SIGSEGV (0xb) at pc=0x7f2569fdaa9f, pid=59248, tid=139788829714176 > # > # A fatal error has been detected by the Java Runtime Environment: > so ,I serch this problem on internet ,somebody said : use the property in > JDK7 - sun.zip.disableMemoryMapping ,we used ,but didn't work. > we used java udf in the query sql ,Is there any relationship with this？ -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-4723) msgF0104 02:03:27.458724 8951 impalad-main.cc:78] ThriftServer 'backend' (on port: 22000) did not start correctly
[ https://issues.apache.org/jira/browse/IMPALA-4723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-4723: Docs Text: (was: ) > msgF0104 02:03:27.458724 8951 impalad-main.cc:78] ThriftServer 'backend' (on > port: 22000) did not start correctly > -- > > Key: IMPALA-4723 > URL: https://issues.apache.org/jira/browse/IMPALA-4723 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: Impala 2.6.0 >Reporter: Vinothkumar >Priority: Critical > Labels: impala > > Error Description: > Log file created at: 2017/01/04 02:03:12 > Running on machine: quickstart.cloudera > Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg > E0104 02:03:12.211601 8951 logging.cc:118] stderr will be logged to this > file. > E0104 02:03:27.162784 8951 tmp-file-mgr.cc:129] Running without spill to > disk: could not use any scratch > directories in list: /tmp. See previous warnings for information on causes. > E0104 02:03:27.458483 9034 thrift-server.cc:176] ThriftServer 'backend' (on > port: 22000) exited due to TE > xception: Could not bind: Transport endpoint is not connected > E0104 02:03:27.458673 8951 thrift-server.cc:165] ThriftServer 'backend' (on > port: 22000) did not start co > rrectly > F0104 02:03:27.458724 8951 impalad-main.cc:78] ThriftServer 'backend' (on > port: 22000) did not start corr > ectly > . Impalad exiting. > *** Check failure stack trace: *** > @ 0x1b5934d (unknown) > @ 0x1b5bc76 (unknown) > @ 0x1b58e6d (unknown) > @ 0x1b5c71e (unknown) > @ 0xae00fc (unknown) > @ 0x7d92d3 (unknown) > @ 0x3da381ed5d (unknown) > @ 0x7d8f8d (unknown) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-5885) Parquet scanner does not free local allocations in filter contexts
[
https://issues.apache.org/jira/browse/IMPALA-5885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni updated IMPALA-5885:
Docs Text: (was: Workarounds:
Either of the following can help:
* Reduce num_scanner_threads to reduce the scanner thread concurrency.
* Set runtime_filter_mode=0 to disable runtime filters)
> Parquet scanner does not free local allocations in filter contexts
> --
>
> Key: IMPALA-5885
> URL: https://issues.apache.org/jira/browse/IMPALA-5885
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 2.6.0, Impala 2.7.0, Impala 2.8.0, Impala 2.9.0,
> Impala 2.10.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Major
> Labels: resource-management
> Fix For: Impala 2.11.0
>
>
> This problem can occur if runtime filter expressions that are evaluated in
> the scan allocate temporary memory - "local allocations". These accumulate
> for each scan range and are only
> freed upon scan range completion.
> A contrived query that exhibits the problem is the following. If I continue
> adding upper() and lower() to the expression the memory consumption of the
> scan node will continue to grow - up to 100MB for each extra function call!
> {code}
> set runtime_filter_wait_time_ms=100;
> select straight_join count(*) from tpch_parquet.lineitem l1 join
> tpch_parquet.lineitem l2 on upper(lower(upper(lower(l1.l_comment =
> concat(l2.l_comment, 'foo');
> summary;
> {code}
> I think other conjuncts in the scanner may be affected by the same problem,
> e.g. the min_max conjuncts.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-4723) msgF0104 02:03:27.458724 8951 impalad-main.cc:78] ThriftServer 'backend' (on port: 22000) did not start correctly
[ https://issues.apache.org/jira/browse/IMPALA-4723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-4723: Docs Text: was:Please show me how to overcome this issue. > msgF0104 02:03:27.458724 8951 impalad-main.cc:78] ThriftServer 'backend' (on > port: 22000) did not start correctly > -- > > Key: IMPALA-4723 > URL: https://issues.apache.org/jira/browse/IMPALA-4723 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: Impala 2.6.0 >Reporter: Vinothkumar >Priority: Critical > Labels: impala > > Error Description: > Log file created at: 2017/01/04 02:03:12 > Running on machine: quickstart.cloudera > Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg > E0104 02:03:12.211601 8951 logging.cc:118] stderr will be logged to this > file. > E0104 02:03:27.162784 8951 tmp-file-mgr.cc:129] Running without spill to > disk: could not use any scratch > directories in list: /tmp. See previous warnings for information on causes. > E0104 02:03:27.458483 9034 thrift-server.cc:176] ThriftServer 'backend' (on > port: 22000) exited due to TE > xception: Could not bind: Transport endpoint is not connected > E0104 02:03:27.458673 8951 thrift-server.cc:165] ThriftServer 'backend' (on > port: 22000) did not start co > rrectly > F0104 02:03:27.458724 8951 impalad-main.cc:78] ThriftServer 'backend' (on > port: 22000) did not start corr > ectly > . Impalad exiting. > *** Check failure stack trace: *** > @ 0x1b5934d (unknown) > @ 0x1b5bc76 (unknown) > @ 0x1b58e6d (unknown) > @ 0x1b5c71e (unknown) > @ 0xae00fc (unknown) > @ 0x7d92d3 (unknown) > @ 0x3da381ed5d (unknown) > @ 0x7d8f8d (unknown) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-6183) Converting Decimal to Double loses precision
[
https://issues.apache.org/jira/browse/IMPALA-6183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni updated IMPALA-6183:
Docs Text: (was: Flag this in release notes as an "incompatible change".)
> Converting Decimal to Double loses precision
>
>
> Key: IMPALA-6183
> URL: https://issues.apache.org/jira/browse/IMPALA-6183
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 2.7.0, Impala 2.8.0, Impala 2.9.0, Impala 2.10.0,
> Impala 2.11.0
>Reporter: Taras Bobrovytsky
>Assignee: Taras Bobrovytsky
>Priority: Major
> Fix For: Impala 2.11.0
>
>
> The following query returns a correct result:
> {code}
> select cast(1.1112223334445559 as double)
> {code}
> Result:
> {code}
> 1.111222333444556
> {code}
> However, if we first convert the literal to a decimal, we lose precision.
> {code}
> select cast(cast(1.1112223334445559 as decimal(38, 37)) as double)
> {code}
> Result:
> {code}
> 1.111222340799091
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-7513) Fix Sentry compilation error in Impala 2.x branch
[ https://issues.apache.org/jira/browse/IMPALA-7513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-7513 started by Fredy Wijaya. > Fix Sentry compilation error in Impala 2.x branch > - > > Key: IMPALA-7513 > URL: https://issues.apache.org/jira/browse/IMPALA-7513 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 2.12.0 > > > [ERROR] > /tmp/jenkins/workspace/impala-compile-and-publish/fe/src/main/java/org/apache/impala/catalog/AuthorizationPolicy.java:[58,8] > org.apache.impala.catalog.AuthorizationPolicy is not abstract and does not > override abstract method > listPrivileges(java.util.Set,java.util.Set,org.apache.sentry.core.common.ActiveRoleSet) > in org.apache.sentry.provider.cache.PrivilegeCache -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7514) Impala Doc: from_unixtime supports literal H
Alex Rodoni created IMPALA-7514: --- Summary: Impala Doc: from_unixtime supports literal H Key: IMPALA-7514 URL: https://issues.apache.org/jira/browse/IMPALA-7514 Project: IMPALA Issue Type: Sub-task Components: Docs Reporter: Alex Rodoni Assignee: Alex Rodoni -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-7514) Impala Doc: from_unixtime supports literal H
Alex Rodoni created IMPALA-7514: --- Summary: Impala Doc: from_unixtime supports literal H Key: IMPALA-7514 URL: https://issues.apache.org/jira/browse/IMPALA-7514 Project: IMPALA Issue Type: Sub-task Components: Docs Reporter: Alex Rodoni Assignee: Alex Rodoni -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-5237) from_unixtime cannot include literal H - incompatible with hive
[
https://issues.apache.org/jira/browse/IMPALA-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni updated IMPALA-5237:
Docs Text: (was: Need docs)
> from_unixtime cannot include literal H - incompatible with hive
> ---
>
> Key: IMPALA-5237
> URL: https://issues.apache.org/jira/browse/IMPALA-5237
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Environment: Impala Shell v2.6.0-cdh5.8.3 (9872875) built on Fri Dec
> 9 14:31:00 PST 2016
>Reporter: Lou Bershad
>Assignee: Jinchul Kim
>Priority: Major
> Labels: newbie
>
> I need to format a date as 2017-04-20H18. No amount of backslashes or ticks
> will get this done.
> In hive, it looks like:
> {code}
> hive> select from_unixtime(1492677561,'-MM-dd\'H\'HH');
> OK
> _c0
> 2017-04-20H08
> {code}
> In impala, the same query returns the wrong thing. Instead of a literal H,
> it gives me '8' (because the hour happens to be 8).
> {code}
> [atl1c1mgr-vip.vldb-bo.secureworks.net:21001] > select
> from_unixtime(1492677561,'-MM-dd\'H\'HH');
> Query: select from_unixtime(1492677561,'-MM-dd\'H\'HH')
> ++
> | from_unixtime(1492677561, '-mm-dd\'h\'hh') |
> ++
> | 2017-04-20'8'08|
> ++
> Fetched 1 row(s) in 0.05s
> {code}
> from_unixtime should support literals. My preference would be that it uses
> the same syntax as hive to achieve this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-6726) Catalog server's kerberos ticket gets deleted after 'ticket_lifetime' on SLES11
[
https://issues.apache.org/jira/browse/IMPALA-6726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni updated IMPALA-6726:
Docs Text: (was: Workaround for this issue:
On 2.11.0, set --use_kudu_kinit=false in Impala startup flag.
On 2.12.0, set --use_kudu_kinit=false and --use_krpc=false in Impala startup
flags.)
> Catalog server's kerberos ticket gets deleted after 'ticket_lifetime' on
> SLES11
> ---
>
> Key: IMPALA-6726
> URL: https://issues.apache.org/jira/browse/IMPALA-6726
> Project: IMPALA
> Issue Type: Sub-task
> Components: Security
>Affects Versions: Impala 2.11.0, Impala 2.12.0
>Reporter: Sailesh Mukil
>Assignee: Michael Ho
>Priority: Blocker
> Labels: kerberos, security
> Fix For: Impala 2.12.0
>
>
> On SLES11, it was noticed that after 'ticket_lifetime', the kerberos ticket
> gets deleted by the Java krb5 library. [~mikesbrown] noticed this from 2.11,
> and we confirmed that it shows up in 2.12 as well.
> I turned on the Java kerberos debug logging and found this in the log
> messages:
> {noformat}
> W0322 07:51:43.617998 12118 UserGroupInformation.java:1403] Not attempting to
> re-login since the last re-login was attempted less than 60 seconds before.
> Last Login=1521730246019
> >>>DEBUG client principal is
> >>>impala/mikeb-sles11-1.vpc.cloudera@vpc.cloudera.com
> >>>DEBUG server principal is
> >>>krbtgt/vpc.cloudera@vpc.cloudera.com
> >>>DEBUG key type: 16
> >>>DEBUG auth time: Thu Mar 22 07:21:58 PDT 2018
> >>>DEBUG start time: Thu Mar 22 07:51:46 PDT 2018
> >>>DEBUG end time: Thu Mar 22 07:51:58 PDT 2018
> >>>DEBUG renew_till time: Thu Mar 22 07:51:58 PDT 2018
> >>> CCacheInputStream: readFlags() FORWARDABLE; RENEWABLE; INITIAL;
> Found ticket for impala/mikeb-sles11-1.vpc.cloudera@vpc.cloudera.com to
> go to krbtgt/vpc.cloudera@vpc.cloudera.com expiring on Thu Mar 22
> 07:51:58 PDT 2018
> Removed and destroyed the expired Ticket
> Destroyed KerberosTicket
> W0322 07:52:04.195199 12201 UserGroupInformation.java:1920]
> PriviledgedActionException
> as:impala/mikeb-sles11-1.vpc.cloudera@vpc.cloudera.com (auth:KERBEROS)
> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> W0322 07:52:04.200016 12201 UserGroupInformation.java:1403] Not attempting to
> re-login since the last re-login was attempted less than 60 seconds before.
> Last Login=1521730306038
> {noformat}
> The backend ticket acquisition thread however keeps running and claiming to
> have re-acquired a ticket every 'ticket_lifetime' period.
> I tried turning off the 'use_kudu_kinit' flag and this bug didn't show up in
> that mode.
> Still investigating the bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-6389) Crash when querying table with "\0" as a row delimiter
[
https://issues.apache.org/jira/browse/IMPALA-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni closed IMPALA-6389.
---
Resolution: Fixed
Reopened temporarily to remove the Doc Text
> Crash when querying table with "\0" as a row delimiter
> --
>
> Key: IMPALA-6389
> URL: https://issues.apache.org/jira/browse/IMPALA-6389
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 2.8.0, Impala 2.9.0, Impala 2.10.0, Impala 2.11.0
>Reporter: Tim Armstrong
>Assignee: Zach Amsden
>Priority: Blocker
> Labels: crash
> Fix For: Impala 2.13.0
>
>
> A user reported this bug here:
> http://community.cloudera.com/t5/Interactive-Short-cycle-SQL/Impala-quot-Cancelled-due-to-unreachable-impalad-s-quot-when/m-p/63577#M4044?eid=1=1
> The following sequence causes an impalad crash:
> {noformat}
> create table tab_separated(id bigint, s string, n int, t timestamp, b boolean)
> row format delimited
> fields terminated by '\t' escaped by '\\' lines terminated by '\000'
> stored as textfile;
> select * from tab_separated; -- Done. 0 results.
> insert into tab_separated (id, s) values (100, ''); -- Success.
> select * from tab_separated; -- 20 second delay before getting "Cancelled due
> to unreachable impalad(s): :22000"
> {noformat}
> {noformat}
> (gdb) bt
> #0 0x7f90414371f7 in raise () from /lib64/libc.so.6
> #1 0x7f90414388e8 in abort () from /lib64/libc.so.6
> #2 0x7f9044105185 in os::abort(bool) () from
> /usr/java/jdk1.8.0_144/jre/lib/amd64/server/libjvm.so
> #3 0x7f90442a7593 in VMError::report_and_die() () from
> /usr/java/jdk1.8.0_144/jre/lib/amd64/server/libjvm.so
> #4 0x7f904410a68f in JVM_handle_linux_signal () from
> /usr/java/jdk1.8.0_144/jre/lib/amd64/server/libjvm.so
> #5 0x7f9044100be3 in signalHandler(int, siginfo*, void*) () from
> /usr/java/jdk1.8.0_144/jre/lib/amd64/server/libjvm.so
> #6
> #7 0x7f904154e574 in __memcpy_ssse3_back () from /lib64/libc.so.6
> #8 0x00dc01a0 in impala::StringBuffer::Append
> (this=this@entry=0x723c5c0, str=0x722a00f "\025\b\034\030\022William S.
> Pollard\030\016Adolfo A. Lieb\026", str_len=-1)
> at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/runtime/string-buffer.h:54
> #9 0x00dbdc05 in impala::HdfsTextScanner::ProcessRange
> (this=this@entry=0x723c400, row_batch=row_batch@entry=0x74157a0,
> num_tuples=num_tuples@entry=0x7f8fe4f3821c)
> at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/exec/hdfs-text-scanner.cc:397
> #10 0x00dbfdeb in impala::HdfsTextScanner::GetNextInternal
> (this=0x723c400, row_batch=0x74157a0)
> at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/exec/hdfs-text-scanner.cc:446
> #11 0x00d9d18d in impala::HdfsScanner::ProcessSplit (this=0x723c400)
> at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/exec/hdfs-scanner.cc:121
> #12 0x00d8e1d9 in impala::HdfsScanNode::ProcessSplit (this=0x5496300,
> filter_ctxs=..., expr_results_pool=0x8000,
> expr_results_pool@entry=0x7f8fe4f38780,
> scan_range=0x64c8000) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/exec/hdfs-scan-node.cc:532
> #13 0x00d8fd65 in impala::HdfsScanNode::ScannerThread
> (this=0x5496300) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/exec/hdfs-scan-node.cc:442
> #14 0x00d90262 in operator() (__closure=0x7f8fe4f38c68) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/exec/hdfs-scan-node.cc:354
> #15
> boost::detail::function::void_function_obj_invoker0,
> void>::invoke(boost::detail::function::function_buffer &)
> (function_obj_ptr=...) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
> #16 0x00d19db3 in operator() (this=0x7f8fe4f38c60)
> at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
> #17 impala::Thread::SuperviseThread(std::string const&, std::string const&,
> boost::function, impala::Promise*) (name=..., category=...,
> functor=...,
> thread_started=) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/util/thread.cc:356
> #18 0x00d1a544 in operator()&,
> const std::basic_string&, boost::function, impala::Promise int>*), boost::_bi::list0> (f=@0x7f705b8: 0xd19b20
> boost::function, impala::Promise*)>, a=,
> this=0x7f705c0) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:457
> #19 operator() (this=0x7f705b8) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
> #20
[jira] [Closed] (IMPALA-6389) Crash when querying table with "\0" as a row delimiter
[
https://issues.apache.org/jira/browse/IMPALA-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni closed IMPALA-6389.
---
Resolution: Fixed
Reopened temporarily to remove the Doc Text
> Crash when querying table with "\0" as a row delimiter
> --
>
> Key: IMPALA-6389
> URL: https://issues.apache.org/jira/browse/IMPALA-6389
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 2.8.0, Impala 2.9.0, Impala 2.10.0, Impala 2.11.0
>Reporter: Tim Armstrong
>Assignee: Zach Amsden
>Priority: Blocker
> Labels: crash
> Fix For: Impala 2.13.0
>
>
> A user reported this bug here:
> http://community.cloudera.com/t5/Interactive-Short-cycle-SQL/Impala-quot-Cancelled-due-to-unreachable-impalad-s-quot-when/m-p/63577#M4044?eid=1=1
> The following sequence causes an impalad crash:
> {noformat}
> create table tab_separated(id bigint, s string, n int, t timestamp, b boolean)
> row format delimited
> fields terminated by '\t' escaped by '\\' lines terminated by '\000'
> stored as textfile;
> select * from tab_separated; -- Done. 0 results.
> insert into tab_separated (id, s) values (100, ''); -- Success.
> select * from tab_separated; -- 20 second delay before getting "Cancelled due
> to unreachable impalad(s): :22000"
> {noformat}
> {noformat}
> (gdb) bt
> #0 0x7f90414371f7 in raise () from /lib64/libc.so.6
> #1 0x7f90414388e8 in abort () from /lib64/libc.so.6
> #2 0x7f9044105185 in os::abort(bool) () from
> /usr/java/jdk1.8.0_144/jre/lib/amd64/server/libjvm.so
> #3 0x7f90442a7593 in VMError::report_and_die() () from
> /usr/java/jdk1.8.0_144/jre/lib/amd64/server/libjvm.so
> #4 0x7f904410a68f in JVM_handle_linux_signal () from
> /usr/java/jdk1.8.0_144/jre/lib/amd64/server/libjvm.so
> #5 0x7f9044100be3 in signalHandler(int, siginfo*, void*) () from
> /usr/java/jdk1.8.0_144/jre/lib/amd64/server/libjvm.so
> #6
> #7 0x7f904154e574 in __memcpy_ssse3_back () from /lib64/libc.so.6
> #8 0x00dc01a0 in impala::StringBuffer::Append
> (this=this@entry=0x723c5c0, str=0x722a00f "\025\b\034\030\022William S.
> Pollard\030\016Adolfo A. Lieb\026", str_len=-1)
> at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/runtime/string-buffer.h:54
> #9 0x00dbdc05 in impala::HdfsTextScanner::ProcessRange
> (this=this@entry=0x723c400, row_batch=row_batch@entry=0x74157a0,
> num_tuples=num_tuples@entry=0x7f8fe4f3821c)
> at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/exec/hdfs-text-scanner.cc:397
> #10 0x00dbfdeb in impala::HdfsTextScanner::GetNextInternal
> (this=0x723c400, row_batch=0x74157a0)
> at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/exec/hdfs-text-scanner.cc:446
> #11 0x00d9d18d in impala::HdfsScanner::ProcessSplit (this=0x723c400)
> at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/exec/hdfs-scanner.cc:121
> #12 0x00d8e1d9 in impala::HdfsScanNode::ProcessSplit (this=0x5496300,
> filter_ctxs=..., expr_results_pool=0x8000,
> expr_results_pool@entry=0x7f8fe4f38780,
> scan_range=0x64c8000) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/exec/hdfs-scan-node.cc:532
> #13 0x00d8fd65 in impala::HdfsScanNode::ScannerThread
> (this=0x5496300) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/exec/hdfs-scan-node.cc:442
> #14 0x00d90262 in operator() (__closure=0x7f8fe4f38c68) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/exec/hdfs-scan-node.cc:354
> #15
> boost::detail::function::void_function_obj_invoker0,
> void>::invoke(boost::detail::function::function_buffer &)
> (function_obj_ptr=...) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
> #16 0x00d19db3 in operator() (this=0x7f8fe4f38c60)
> at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
> #17 impala::Thread::SuperviseThread(std::string const&, std::string const&,
> boost::function, impala::Promise*) (name=..., category=...,
> functor=...,
> thread_started=) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/util/thread.cc:356
> #18 0x00d1a544 in operator()&,
> const std::basic_string&, boost::function, impala::Promise int>*), boost::_bi::list0> (f=@0x7f705b8: 0xd19b20
> boost::function, impala::Promise*)>, a=,
> this=0x7f705c0) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:457
> #19 operator() (this=0x7f705b8) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
> #20
[jira] [Updated] (IMPALA-6389) Crash when querying table with "\0" as a row delimiter
[
https://issues.apache.org/jira/browse/IMPALA-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni updated IMPALA-6389:
Docs Text: (was: Workaround: Use an alternative delimiter (e.g. \001).)
> Crash when querying table with "\0" as a row delimiter
> --
>
> Key: IMPALA-6389
> URL: https://issues.apache.org/jira/browse/IMPALA-6389
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 2.8.0, Impala 2.9.0, Impala 2.10.0, Impala 2.11.0
>Reporter: Tim Armstrong
>Assignee: Zach Amsden
>Priority: Blocker
> Labels: crash
> Fix For: Impala 2.13.0
>
>
> A user reported this bug here:
> http://community.cloudera.com/t5/Interactive-Short-cycle-SQL/Impala-quot-Cancelled-due-to-unreachable-impalad-s-quot-when/m-p/63577#M4044?eid=1=1
> The following sequence causes an impalad crash:
> {noformat}
> create table tab_separated(id bigint, s string, n int, t timestamp, b boolean)
> row format delimited
> fields terminated by '\t' escaped by '\\' lines terminated by '\000'
> stored as textfile;
> select * from tab_separated; -- Done. 0 results.
> insert into tab_separated (id, s) values (100, ''); -- Success.
> select * from tab_separated; -- 20 second delay before getting "Cancelled due
> to unreachable impalad(s): :22000"
> {noformat}
> {noformat}
> (gdb) bt
> #0 0x7f90414371f7 in raise () from /lib64/libc.so.6
> #1 0x7f90414388e8 in abort () from /lib64/libc.so.6
> #2 0x7f9044105185 in os::abort(bool) () from
> /usr/java/jdk1.8.0_144/jre/lib/amd64/server/libjvm.so
> #3 0x7f90442a7593 in VMError::report_and_die() () from
> /usr/java/jdk1.8.0_144/jre/lib/amd64/server/libjvm.so
> #4 0x7f904410a68f in JVM_handle_linux_signal () from
> /usr/java/jdk1.8.0_144/jre/lib/amd64/server/libjvm.so
> #5 0x7f9044100be3 in signalHandler(int, siginfo*, void*) () from
> /usr/java/jdk1.8.0_144/jre/lib/amd64/server/libjvm.so
> #6
> #7 0x7f904154e574 in __memcpy_ssse3_back () from /lib64/libc.so.6
> #8 0x00dc01a0 in impala::StringBuffer::Append
> (this=this@entry=0x723c5c0, str=0x722a00f "\025\b\034\030\022William S.
> Pollard\030\016Adolfo A. Lieb\026", str_len=-1)
> at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/runtime/string-buffer.h:54
> #9 0x00dbdc05 in impala::HdfsTextScanner::ProcessRange
> (this=this@entry=0x723c400, row_batch=row_batch@entry=0x74157a0,
> num_tuples=num_tuples@entry=0x7f8fe4f3821c)
> at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/exec/hdfs-text-scanner.cc:397
> #10 0x00dbfdeb in impala::HdfsTextScanner::GetNextInternal
> (this=0x723c400, row_batch=0x74157a0)
> at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/exec/hdfs-text-scanner.cc:446
> #11 0x00d9d18d in impala::HdfsScanner::ProcessSplit (this=0x723c400)
> at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/exec/hdfs-scanner.cc:121
> #12 0x00d8e1d9 in impala::HdfsScanNode::ProcessSplit (this=0x5496300,
> filter_ctxs=..., expr_results_pool=0x8000,
> expr_results_pool@entry=0x7f8fe4f38780,
> scan_range=0x64c8000) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/exec/hdfs-scan-node.cc:532
> #13 0x00d8fd65 in impala::HdfsScanNode::ScannerThread
> (this=0x5496300) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/exec/hdfs-scan-node.cc:442
> #14 0x00d90262 in operator() (__closure=0x7f8fe4f38c68) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/exec/hdfs-scan-node.cc:354
> #15
> boost::detail::function::void_function_obj_invoker0,
> void>::invoke(boost::detail::function::function_buffer &)
> (function_obj_ptr=...) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
> #16 0x00d19db3 in operator() (this=0x7f8fe4f38c60)
> at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
> #17 impala::Thread::SuperviseThread(std::string const&, std::string const&,
> boost::function, impala::Promise*) (name=..., category=...,
> functor=...,
> thread_started=) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/be/src/util/thread.cc:356
> #18 0x00d1a544 in operator()&,
> const std::basic_string&, boost::function, impala::Promise int>*), boost::_bi::list0> (f=@0x7f705b8: 0xd19b20
> boost::function, impala::Promise*)>, a=,
> this=0x7f705c0) at
> /usr/src/debug/impala-2.11.0-cdh5.15.0-SNAPSHOT/toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:457
> #19 operator() (this=0x7f705b8) at
>
[jira] [Updated] (IMPALA-4172) Switch from using getFileBlockLocations to BlockLocation methods (Potential 50% speedup in metadata loading)
[
https://issues.apache.org/jira/browse/IMPALA-4172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni updated IMPALA-4172:
Docs Text: (was: Improves the performance of block metadata fetching by
the Catalog server from the Namenode by substantially reducing the number of
RPCs.)
> Switch from using getFileBlockLocations to BlockLocation methods (Potential
> 50% speedup in metadata loading)
>
>
> Key: IMPALA-4172
> URL: https://issues.apache.org/jira/browse/IMPALA-4172
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 2.8.0
>Reporter: Mostafa Mokhtar
>Assignee: bharath v
>Priority: Critical
> Labels: performance, ramp-up
> Fix For: Impala 2.8.0
>
> Attachments: query_after_invalidate_store_sales_800Kfiles_test.jfr
>
>
> HDFS-8895 removes the ability to query volume IDs from datanodes. This
> information has instead been added to BlockLocation, which is accessible via
> various FileSystem APIs (namely, anything that returns LocatedFileStatus).
> This new API is more efficient and more accurate. It's also available from
> CDH5.5 onwards, so can be backported as well.
> getFileBlockLocations is a bottle neck during metadata loading for Impala.
> {code}
> Stack Trace Sample CountPercentage(%)
> java.lang.Thread.run()17,837 73.758
>java.util.concurrent.ThreadPoolExecutor$Worker.run() 17,837 73.758
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker)
> 17,837 73.758
> java.util.concurrent.FutureTask.run()17,600 72.778
> com.cloudera.impala.catalog.TableLoadingMgr$2.call() 17,513
> 72.419
>com.cloudera.impala.catalog.TableLoadingMgr$2.call() 17,513
> 72.419
> com.cloudera.impala.catalog.TableLoader.load(Db, String)
> 17,513 72.419
> com.cloudera.impala.catalog.HdfsTable.load(boolean,
> IMetaStoreClient, Table) 17,513 72.419
> com.cloudera.impala.catalog.HdfsTable.load(boolean,
> IMetaStoreClient, Table, boolean, boolean, Set) 17,513 72.419
>
> com.cloudera.impala.catalog.HdfsTable.loadAllPartitions(List, Table)
> 15,721 65.008
>
> com.cloudera.impala.catalog.HdfsTable.createPartition(StorageDescriptor,
> Partition, Map)13,611 56.283
>
> com.cloudera.impala.catalog.HdfsTable.updatePartitionFds(Path, boolean,
> HdfsFileFormat, Map) 7,942 32.841
>
> com.cloudera.impala.catalog.HdfsTable.loadBlockMetadata(FileSystem,
> FileStatus, HdfsPartition$FileDescriptor, HdfsFileFormat, Map)4,319
> 17.86
>
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileBlockLocations(FileStatus,
> long, long) 3,678 15.209
>
> com.cloudera.impala.catalog.HdfsPartition$BlockReplica.parseLocation(String)
> 203 0.839
> {code}
> Pointer to the JAVA docs for the new API
> [https://hadoop.apache.org/docs/r2.6.1/api/org/apache/hadoop/fs/FileSystem.html#listFiles(org.apache.hadoop.fs.Path,
> boolean)]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-4107) APPX_MEDIAN cuts string to 10 chars
[
https://issues.apache.org/jira/browse/IMPALA-4107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni closed IMPALA-4107.
---
Resolution: Fixed
Reopened to remove the Doc Text as it is now documented.
> APPX_MEDIAN cuts string to 10 chars
> ---
>
> Key: IMPALA-4107
> URL: https://issues.apache.org/jira/browse/IMPALA-4107
> Project: IMPALA
> Issue Type: Bug
> Components: Docs
>Affects Versions: Impala 2.6.0
>Reporter: Peter Brejcak
>Assignee: Alex Rodoni
>Priority: Minor
>
> Function APPX_MEDIAN returns only first 10 characters for string input. I
> didn't find such restrictions in documentation.
> Expected result for code below should be that
> length(APPX_MEDIAN(t.value))>=min(length(t.value)) but real result is
> different
> Please let me know if anything is not clear
> {code}
> create table user_agents (id int, value string);
> insert into table user_agents values (1,'Mozilla/5.0 (compatible;
> Googlebot/2.1; +http://www.google.com/bot.html)');
> insert into table user_agents values (2,'Mozilla/5.0 (Windows NT 6.3)
> AppleWebKit/537.36 (KHTML; like Gecko) Chrome/44.0.2441.18 Safari/537.36');
> insert into table user_agents values (3,'Mozilla/5.0 (Windows NT 6.3; Win64;
> x64) AppleWebKit/537.36 (KHTML; like Gecko) Chrome/39.0.2186.5
> Safari/537.36');
> insert into table user_agents values (4,'Mozilla/5.0 (Windows NT 6.3; WOW64)
> AppleWebKit/537.36 (KHTML; like Gecko) Chrome/33.0.1810.81 Safari/537.36');
> insert into table user_agents values (5,'Mozilla/5.0 (Windows NT 6.3; WOW64)
> AppleWebKit/537.36 (KHTML; like Gecko) Chrome/33.0.1810.81 Safari/537.36');
> insert into table user_agents values (6,'Mozilla/5.0 (Windows NT 5.1; WOW64)
> AppleWebKit/537.36 (KHTML; like Gecko) Chrome/33.0.1828.68 Safari/537.36');
> insert into table user_agents values (7,'Mozilla/5.0 (Linux; U; Android
> 4.1.2; en-us; GT-I8190 Build/JZO54K) AppleWebKit/534.30 (KHTML; like Gecko)
> Version/4.0 Mobile Safari/534.30 [FB_IAB/FB4A;FBAV/74.0.0.21.69;]');
> insert into table user_agents values (8,'Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML; like Gecko) Chrome/32.0.1713.94 Safari/537.36');
> insert into table user_agents values (9,'Mozilla/5.0 (Windows NT 6.3; Win64;
> x64) AppleWebKit/537.36 (KHTML; like Gecko) Chrome/44.0.2421.69
> Safari/537.36');
> select count(1), APPX_MEDIAN(value), length(APPX_MEDIAN(t.value)),
> min(length(t.value)) from user_agents t
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (IMPALA-4107) APPX_MEDIAN cuts string to 10 chars
[
https://issues.apache.org/jira/browse/IMPALA-4107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni closed IMPALA-4107.
---
Resolution: Fixed
Reopened to remove the Doc Text as it is now documented.
> APPX_MEDIAN cuts string to 10 chars
> ---
>
> Key: IMPALA-4107
> URL: https://issues.apache.org/jira/browse/IMPALA-4107
> Project: IMPALA
> Issue Type: Bug
> Components: Docs
>Affects Versions: Impala 2.6.0
>Reporter: Peter Brejcak
>Assignee: Alex Rodoni
>Priority: Minor
>
> Function APPX_MEDIAN returns only first 10 characters for string input. I
> didn't find such restrictions in documentation.
> Expected result for code below should be that
> length(APPX_MEDIAN(t.value))>=min(length(t.value)) but real result is
> different
> Please let me know if anything is not clear
> {code}
> create table user_agents (id int, value string);
> insert into table user_agents values (1,'Mozilla/5.0 (compatible;
> Googlebot/2.1; +http://www.google.com/bot.html)');
> insert into table user_agents values (2,'Mozilla/5.0 (Windows NT 6.3)
> AppleWebKit/537.36 (KHTML; like Gecko) Chrome/44.0.2441.18 Safari/537.36');
> insert into table user_agents values (3,'Mozilla/5.0 (Windows NT 6.3; Win64;
> x64) AppleWebKit/537.36 (KHTML; like Gecko) Chrome/39.0.2186.5
> Safari/537.36');
> insert into table user_agents values (4,'Mozilla/5.0 (Windows NT 6.3; WOW64)
> AppleWebKit/537.36 (KHTML; like Gecko) Chrome/33.0.1810.81 Safari/537.36');
> insert into table user_agents values (5,'Mozilla/5.0 (Windows NT 6.3; WOW64)
> AppleWebKit/537.36 (KHTML; like Gecko) Chrome/33.0.1810.81 Safari/537.36');
> insert into table user_agents values (6,'Mozilla/5.0 (Windows NT 5.1; WOW64)
> AppleWebKit/537.36 (KHTML; like Gecko) Chrome/33.0.1828.68 Safari/537.36');
> insert into table user_agents values (7,'Mozilla/5.0 (Linux; U; Android
> 4.1.2; en-us; GT-I8190 Build/JZO54K) AppleWebKit/534.30 (KHTML; like Gecko)
> Version/4.0 Mobile Safari/534.30 [FB_IAB/FB4A;FBAV/74.0.0.21.69;]');
> insert into table user_agents values (8,'Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML; like Gecko) Chrome/32.0.1713.94 Safari/537.36');
> insert into table user_agents values (9,'Mozilla/5.0 (Windows NT 6.3; Win64;
> x64) AppleWebKit/537.36 (KHTML; like Gecko) Chrome/44.0.2421.69
> Safari/537.36');
> select count(1), APPX_MEDIAN(value), length(APPX_MEDIAN(t.value)),
> min(length(t.value)) from user_agents t
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-4107) APPX_MEDIAN cuts string to 10 chars
[
https://issues.apache.org/jira/browse/IMPALA-4107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni updated IMPALA-4107:
Docs Text: (was: Document that APPX_MEDIAN truncates string (varchar,
char) values to 10 characters.)
> APPX_MEDIAN cuts string to 10 chars
> ---
>
> Key: IMPALA-4107
> URL: https://issues.apache.org/jira/browse/IMPALA-4107
> Project: IMPALA
> Issue Type: Bug
> Components: Docs
>Affects Versions: Impala 2.6.0
>Reporter: Peter Brejcak
>Assignee: Alex Rodoni
>Priority: Minor
>
> Function APPX_MEDIAN returns only first 10 characters for string input. I
> didn't find such restrictions in documentation.
> Expected result for code below should be that
> length(APPX_MEDIAN(t.value))>=min(length(t.value)) but real result is
> different
> Please let me know if anything is not clear
> {code}
> create table user_agents (id int, value string);
> insert into table user_agents values (1,'Mozilla/5.0 (compatible;
> Googlebot/2.1; +http://www.google.com/bot.html)');
> insert into table user_agents values (2,'Mozilla/5.0 (Windows NT 6.3)
> AppleWebKit/537.36 (KHTML; like Gecko) Chrome/44.0.2441.18 Safari/537.36');
> insert into table user_agents values (3,'Mozilla/5.0 (Windows NT 6.3; Win64;
> x64) AppleWebKit/537.36 (KHTML; like Gecko) Chrome/39.0.2186.5
> Safari/537.36');
> insert into table user_agents values (4,'Mozilla/5.0 (Windows NT 6.3; WOW64)
> AppleWebKit/537.36 (KHTML; like Gecko) Chrome/33.0.1810.81 Safari/537.36');
> insert into table user_agents values (5,'Mozilla/5.0 (Windows NT 6.3; WOW64)
> AppleWebKit/537.36 (KHTML; like Gecko) Chrome/33.0.1810.81 Safari/537.36');
> insert into table user_agents values (6,'Mozilla/5.0 (Windows NT 5.1; WOW64)
> AppleWebKit/537.36 (KHTML; like Gecko) Chrome/33.0.1828.68 Safari/537.36');
> insert into table user_agents values (7,'Mozilla/5.0 (Linux; U; Android
> 4.1.2; en-us; GT-I8190 Build/JZO54K) AppleWebKit/534.30 (KHTML; like Gecko)
> Version/4.0 Mobile Safari/534.30 [FB_IAB/FB4A;FBAV/74.0.0.21.69;]');
> insert into table user_agents values (8,'Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML; like Gecko) Chrome/32.0.1713.94 Safari/537.36');
> insert into table user_agents values (9,'Mozilla/5.0 (Windows NT 6.3; Win64;
> x64) AppleWebKit/537.36 (KHTML; like Gecko) Chrome/44.0.2421.69
> Safari/537.36');
> select count(1), APPX_MEDIAN(value), length(APPX_MEDIAN(t.value)),
> min(length(t.value)) from user_agents t
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Reopened] (IMPALA-3813) How to handle replication factor while creating KUDU table through impala
[ https://issues.apache.org/jira/browse/IMPALA-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni reopened IMPALA-3813: - > How to handle replication factor while creating KUDU table through impala > - > > Key: IMPALA-3813 > URL: https://issues.apache.org/jira/browse/IMPALA-3813 > Project: IMPALA > Issue Type: Bug > Components: Docs, Frontend >Affects Versions: Kudu_Impala > Environment: CDH 5.4.7 >Reporter: Ravi sharma >Assignee: Alex Rodoni >Priority: Major > Labels: performance > Fix For: Impala 2.13.0, Impala 3.1.0 > > > while to create kudu table from impala shell > facing below error > ERROR: > ImpalaRuntimeException: Error creating Kudu table > CAUSED BY: MasterErrorException: Server[Kudu Master - > cdhnode1.infocepts.com:7051] INVALID_ARGUMENT[code 4]: Not enough live tablet > servers to create a table with the requested replication factor 3. 2 tablet > servers are alive. > [cdhnode1.infocepts.com:21000] > > Please tell me how can i limit the replication factor manually, since i have > only 2 tablet servers. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-3813) How to handle replication factor while creating KUDU table through impala
[ https://issues.apache.org/jira/browse/IMPALA-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni closed IMPALA-3813. --- Resolution: Fixed > How to handle replication factor while creating KUDU table through impala > - > > Key: IMPALA-3813 > URL: https://issues.apache.org/jira/browse/IMPALA-3813 > Project: IMPALA > Issue Type: Bug > Components: Docs, Frontend >Affects Versions: Kudu_Impala > Environment: CDH 5.4.7 >Reporter: Ravi sharma >Assignee: Alex Rodoni >Priority: Major > Labels: performance > Fix For: Impala 2.13.0, Impala 3.1.0 > > > while to create kudu table from impala shell > facing below error > ERROR: > ImpalaRuntimeException: Error creating Kudu table > CAUSED BY: MasterErrorException: Server[Kudu Master - > cdhnode1.infocepts.com:7051] INVALID_ARGUMENT[code 4]: Not enough live tablet > servers to create a table with the requested replication factor 3. 2 tablet > servers are alive. > [cdhnode1.infocepts.com:21000] > > Please tell me how can i limit the replication factor manually, since i have > only 2 tablet servers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (IMPALA-3813) How to handle replication factor while creating KUDU table through impala
[ https://issues.apache.org/jira/browse/IMPALA-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni closed IMPALA-3813. --- Resolution: Fixed > How to handle replication factor while creating KUDU table through impala > - > > Key: IMPALA-3813 > URL: https://issues.apache.org/jira/browse/IMPALA-3813 > Project: IMPALA > Issue Type: Bug > Components: Docs, Frontend >Affects Versions: Kudu_Impala > Environment: CDH 5.4.7 >Reporter: Ravi sharma >Assignee: Alex Rodoni >Priority: Major > Labels: performance > Fix For: Impala 2.13.0, Impala 3.1.0 > > > while to create kudu table from impala shell > facing below error > ERROR: > ImpalaRuntimeException: Error creating Kudu table > CAUSED BY: MasterErrorException: Server[Kudu Master - > cdhnode1.infocepts.com:7051] INVALID_ARGUMENT[code 4]: Not enough live tablet > servers to create a table with the requested replication factor 3. 2 tablet > servers are alive. > [cdhnode1.infocepts.com:21000] > > Please tell me how can i limit the replication factor manually, since i have > only 2 tablet servers. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-3813) How to handle replication factor while creating KUDU table through impala
[
https://issues.apache.org/jira/browse/IMPALA-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni updated IMPALA-3813:
Docs Text: (was: Something like:
By default, Kudu tables created through Impala will use a tablet replication
factor of 3. This can be changed when creating the table by specifying
tblproperties('kudu.num_tablet_replicas' = 'n')
The number of replicas is required to be odd.
Altering this property after table creation has no effect (see IMPALA-6428))
> How to handle replication factor while creating KUDU table through impala
> -
>
> Key: IMPALA-3813
> URL: https://issues.apache.org/jira/browse/IMPALA-3813
> Project: IMPALA
> Issue Type: Bug
> Components: Docs, Frontend
>Affects Versions: Kudu_Impala
> Environment: CDH 5.4.7
>Reporter: Ravi sharma
>Assignee: Alex Rodoni
>Priority: Major
> Labels: performance
> Fix For: Impala 2.13.0, Impala 3.1.0
>
>
> while to create kudu table from impala shell
> facing below error
> ERROR:
> ImpalaRuntimeException: Error creating Kudu table
> CAUSED BY: MasterErrorException: Server[Kudu Master -
> cdhnode1.infocepts.com:7051] INVALID_ARGUMENT[code 4]: Not enough live tablet
> servers to create a table with the requested replication factor 3. 2 tablet
> servers are alive.
> [cdhnode1.infocepts.com:21000] >
> Please tell me how can i limit the replication factor manually, since i have
> only 2 tablet servers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-6435) Codegen crash when UNIONing CHAR(n) literals
[
https://issues.apache.org/jira/browse/IMPALA-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni updated IMPALA-6435:
Docs Text: (was: Workaround: disable expression rewrites by setting the
query option enable_expr_rewrites=0)
> Codegen crash when UNIONing CHAR(n) literals
>
>
> Key: IMPALA-6435
> URL: https://issues.apache.org/jira/browse/IMPALA-6435
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 2.8.0, Impala 2.9.0, Impala 2.10.0, Impala 2.11.0
>Reporter: Balazs Jeszenszky
>Assignee: Anuj Phadke
>Priority: Blocker
> Labels: codegen, crash
> Fix For: Impala 2.12.0
>
>
> A simple repro:
> {code:java}
> create table test (c1 int);
> select cast ('a' as char(30)) as m from test
> union all
> select cast ('a' as char(30)) as m from test
> ;
> {code}
> Stack trace:
> {code:java}
> #0 0x7fb0085275e5 in raise () from /lib64/libc.so.6
> #1 0x7fb008528dc5 in abort () from /lib64/libc.so.6
> #2 0x7fb00a663a55 in os::abort(bool) () from
> /usr/java/jdk1.7.0_67-cloudera/jre/lib/amd64/server/libjvm.so
> #3 0x7fb00a7e3f87 in VMError::report_and_die() ()
>from /usr/java/jdk1.7.0_67-cloudera/jre/lib/amd64/server/libjvm.so
> #4 0x7fb00a66896f in JVM_handle_linux_signal ()
>from /usr/java/jdk1.7.0_67-cloudera/jre/lib/amd64/server/libjvm.so
> #5
> #6 0x01a9042d in llvm::FunctionType::get(llvm::Type*,
> llvm::ArrayRef, bool) ()
> #7 0x00c2b62f in
> impala::LlvmCodeGen::FnPrototype::GeneratePrototype(impala::LlvmBuilder*,
> llvm::Value**, bool) ()
> #8 0x00846617 in
> impala::Expr::CreateIrFunctionPrototype(impala::LlvmCodeGen*,
> std::basic_string, std::allocator >
> const&, llvm::Value* (*) [2]) ()
> #9 0x0085960c in
> impala::Literal::GetCodegendComputeFn(impala::LlvmCodeGen*, llvm::Function**)
> ()
> #10 0x00a78467 in
> impala::Tuple::CodegenMaterializeExprs(impala::LlvmCodeGen*, bool,
> impala::TupleDescriptor const&, std::vector std::allocator > const&, bool, llvm::Function**) ()
> #11 0x00d11df8 in impala::UnionNode::Codegen(impala::RuntimeState*) ()
> #12 0x00a7dd4e in impala::FragmentInstanceState::Prepare() ()
> #13 0x00a7e311 in impala::FragmentInstanceState::Exec() ()
> #14 0x00a6a2f6 in
> impala::QueryState::ExecFInstance(impala::FragmentInstanceState*) ()
> {code}
> A debug build will fail a bit differently, hitting the DCHECK in
> codegen-anyval.cc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-3833) Fix invalid data handling in Sequence and RCFile scanners
[ https://issues.apache.org/jira/browse/IMPALA-3833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-3833: Docs Text: (was: Workaround: remove the corrupt files from the HDFS table.) > Fix invalid data handling in Sequence and RCFile scanners > - > > Key: IMPALA-3833 > URL: https://issues.apache.org/jira/browse/IMPALA-3833 > Project: IMPALA > Issue Type: Bug > Components: Backend >Affects Versions: Impala 2.7.0 >Reporter: Tim Armstrong >Assignee: Pranay Singh >Priority: Critical > Labels: crash, downgraded > Fix For: Impala 2.13.0, Impala 3.1.0 > > > The fuzz testing found multiple crashes in sequence and RCFile scanners. > https://gerrit.cloudera.org/#/c/3448/ > I haven't triaged the crashes, but filing this issue to track them. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-5200) Profile timers not updated during long-running sort
[
https://issues.apache.org/jira/browse/IMPALA-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni updated IMPALA-5200:
Docs Text: (was: Workaround: slow sorts can be identified by looking at
"Peak Mem" in the summary or "PeakMemoryUsage" in the profile. If a sort is
consuming multiple GB of memory per host, it will likely spent a significant
amount of time sorting the data.)
> Profile timers not updated during long-running sort
> ---
>
> Key: IMPALA-5200
> URL: https://issues.apache.org/jira/browse/IMPALA-5200
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Affects Versions: Impala 2.9.0
>Reporter: Tim Armstrong
>Priority: Minor
> Labels: observability, ramp-up, supportability
>
> If you have a query plan with a long-running sort operation (e.g. minutes),
> the profile timers are not updated to reflect the time spent in the sort
> until the sort starts returning rows.
> E.g. this is a summary from a sort query that was running for a few hours
> (!). The summary was misleading and the "heat map" plan in the debug web UI
> is misleading - it showed the join as the "hot" operator. It would be ideal
> if we could somehow at least periodically update the time spent in the
> operator.
> {code}
> Operator #Hosts Avg Time Max Time#Rows Est. #Rows
> Peak Mem Est. Peak Mem Detail
>
> 05:MERGING-EXCHANGE10.000ns0.000ns0 635.58M
>0-1.00 B UNPARTITIONED
> 03:SORT10.000ns0.000ns0 635.58M
> 47.86 GB 800.00 MB
> 02:HASH JOIN 14s859ms4s859ms 771.02M 635.58M
> 162.11 MB 16.03 MB INNER JOIN, BROADCAST
> |--04:EXCHANGE 1 38.988ms 38.988ms 247.20K 247.20K
>0 0 BROADCAST
> | 01:SCAN HDFS18s089ms8s089ms 247.20K 247.20K
> 3.79 MB 32.00 MB product b
> 00:SCAN HDFS 1 209.997ms 209.997ms 15.09M 635.58M
> 185.27 MB 176.00 MB sales a
> {code}
> http://community.cloudera.com/t5/Interactive-Short-cycle-SQL/Impala-join-query-running-slow
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7513) Fix Sentry compilation error in Impala 2.x branch
Fredy Wijaya created IMPALA-7513: Summary: Fix Sentry compilation error in Impala 2.x branch Key: IMPALA-7513 URL: https://issues.apache.org/jira/browse/IMPALA-7513 Project: IMPALA Issue Type: Bug Components: Infrastructure Reporter: Fredy Wijaya Assignee: Fredy Wijaya Fix For: Impala 2.12.0 [ERROR] /tmp/jenkins/workspace/impala-compile-and-publish/fe/src/main/java/org/apache/impala/catalog/AuthorizationPolicy.java:[58,8] org.apache.impala.catalog.AuthorizationPolicy is not abstract and does not override abstract method listPrivileges(java.util.Set,java.util.Set,org.apache.sentry.core.common.ActiveRoleSet) in org.apache.sentry.provider.cache.PrivilegeCache -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IMPALA-3094) Incorrect result due to constant evaluation in query with outer join
[
https://issues.apache.org/jira/browse/IMPALA-3094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni updated IMPALA-3094:
Docs Text: (was: John, let's document this correctness issue. I can
explain in more detail how to hit this if needed.)
> Incorrect result due to constant evaluation in query with outer join
>
>
> Key: IMPALA-3094
> URL: https://issues.apache.org/jira/browse/IMPALA-3094
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 2.5.0
>Reporter: casey
>Priority: Major
> Labels: correctness, downgraded
>
> Please see below
> {noformat}
> [localhost.localdomain:21000] > explain SELECT 1
> FROM alltypestiny a1
> INNER JOIN alltypesagg a2 ON a1.smallint_col = a2.year AND false
> RIGHT JOIN alltypes a3 ON a1.year = a1.bigint_col;
> Query: explain SELECT 1
> FROM alltypestiny a1
> INNER JOIN alltypesagg a2 ON a1.smallint_col = a2.year AND false
> RIGHT JOIN alltypes a3 ON a1.year = a1.bigint_col
> +-+
> | Explain String |
> +-+
> | Estimated Per-Host Requirements: Memory=1.00KB VCores=1 |
> | |
> | 00:EMPTYSET |
> +-+
> Fetched 3 row(s) in 0.00s
> [localhost.localdomain:21000] > SELECT 1
> FROM alltypestiny a1
> INNER JOIN alltypesagg a2 ON a1.smallint_col = a2.year AND false
> RIGHT JOIN alltypes a3 ON a1.year = a1.bigint_col;
> Query: select 1
> FROM alltypestiny a1
> INNER JOIN alltypesagg a2 ON a1.smallint_col = a2.year AND false
> RIGHT JOIN alltypes a3 ON a1.year = a1.bigint_col
> Fetched 0 row(s) in 0.00s
> {noformat}
> Since the query uses a RIGHT JOIN the result have the same number of rows as
> the a3 table.
> Version: 1920aa5ea214ea21917778d5faad3ef1cb3e9f87
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-2689) Log every time codegen is disabled due to NYI
[ https://issues.apache.org/jira/browse/IMPALA-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-2689: Docs Text: (was: If there's a performance section on codegen, it might be useful to mention that codegen cannot be performed in all cases. The runtime profile indicates which operators were codegen'd.) > Log every time codegen is disabled due to NYI > - > > Key: IMPALA-2689 > URL: https://issues.apache.org/jira/browse/IMPALA-2689 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 2.3.0 >Reporter: Skye Wanderman-Milne >Assignee: Anuj Phadke >Priority: Minor > Labels: codegen, ramp-up, supportability > Fix For: Impala 2.10.0 > > > Sometimes a function cannot be codegen'd, even with codegen enabled, because > some functionality is not implemented (e.g. we cannot codegen the Avro > scanner if the file schema does not match the table schema, certain types are > not implemented for aggregates). We should always print a message to the log > explaining why this is the case, otherwise it's very difficult to figure out > why codegen couldn't be used for a certain query. We already log in some > cases but not all. > In addition, we could also put these reasons in the runtime profile. > Currently we only log if codegen is enabled on an operator, but it would be > also be useful to see if it was disabled and why. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-2523) Make HdfsTableSink aware of clustered input
[ https://issues.apache.org/jira/browse/IMPALA-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-2523: Docs Text: (was: This needs to be documented) > Make HdfsTableSink aware of clustered input > --- > > Key: IMPALA-2523 > URL: https://issues.apache.org/jira/browse/IMPALA-2523 > Project: IMPALA > Issue Type: New Feature > Components: Backend >Affects Versions: Impala 2.2, Impala 2.3.0, Impala 2.5.0, Impala 2.4.0, > Impala 2.6.0, Impala 2.7.0 >Reporter: Mostafa Mokhtar >Assignee: Lars Volker >Priority: Critical > Labels: performance, ramp-up > Fix For: Impala 2.8.0 > > > The HdfsParquetTableWriter needs to be aware that incoming data is clustered > if the corresponding insert statement has a "clustered" hint. Only a single > open partition should be maintained and flushed by calling > FinalizePartitionFile() when the partition-key values change. > For now, no changes should be made to other table sinks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7512) test_resolution_by_name failed: did not encounter expected error
Bikramjeet Vig created IMPALA-7512:
--
Summary: test_resolution_by_name failed: did not encounter
expected error
Key: IMPALA-7512
URL: https://issues.apache.org/jira/browse/IMPALA-7512
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.1.0
Reporter: Bikramjeet Vig
Seems like the error reached the coordinator and it transitioned to the Error
state but it was not delivered to the client
{noformat}
I0829 23:38:31.626911 16215 impala-server.cc:1040] Registered query
query_id=54b6c955298cd00:623fd4b5
session_id=f14fb6bdc8ea6686:ad8d4097469f7c8f
I0829 23:38:31.627315 16215 Frontend.java:1029] Analyzing query: select key,
value from switched_map_fields_resolution_test.int_map
I0829 23:38:31.627898 16215 Frontend.java:1041] Analysis finished.
I0829 23:38:31.631665 18839 admission-controller.cc:552] Schedule for
id=54b6c955298cd00:623fd4b5 in pool_name=default-pool
cluster_mem_needed=64.00 MB PoolConfig: max_requests=-1 max_queued=200
max_mem=-1.00 B
I0829 23:38:31.631866 18839 admission-controller.cc:557] Stats:
agg_num_running=5, agg_num_queued=0, agg_mem_reserved=1.51 GB,
local_host(local_mem_admitted=2.17 GB, num_admitted_running=5, num_queued=0,
backend_mem_reserved=514.22 MB)
I0829 23:38:31.632130 18839 admission-controller.cc:589] Admitted query
id=54b6c955298cd00:623fd4b5
I0829 23:38:31.632308 18839 coordinator.cc:91] Exec()
query_id=54b6c955298cd00:623fd4b5 stmt=select key, value from
switched_map_fields_resolution_test.int_map
I0829 23:38:31.633636 18744 query-state.cc:491] Instance completed.
instance_id=5049c19ed24bed19:85ffb5e0001 #in-flight=7 status=OK
I0829 23:38:31.634953 18839 coordinator.cc:330] starting execution on 2
backends for query_id=54b6c955298cd00:623fd4b5
I0829 23:38:31.635996 29061 impala-internal-service.cc:49]
ExecQueryFInstances(): query_id=54b6c955298cd00:623fd4b5
coord=impala-ec2-centos74-m5-4xlarge-ondemand-0bff.vpc.cloudera.com:22000
#instances=1
I0829 23:38:31.638362 18843 query-state.cc:483] Executing instance.
instance_id=54b6c955298cd00:623fd4b5 fragment_idx=0
per_fragment_instance_idx=0 coord_state_idx=0 #in-flight=8
I0829 23:38:31.638617 18839 coordinator.cc:344] started execution on 2 backends
for query_id=54b6c955298cd00:623fd4b5
I0829 23:38:31.656479 4384 coordinator.cc:685] Backend completed:
host=impala-ec2-centos74-m5-4xlarge-ondemand-0bff.vpc.cloudera.com:22001
remaining=2 query_id=54b6c955298cd00:623fd4b5
I0829 23:38:31.656626 4384 coordinator-backend-state.cc:254]
query_id=54b6c955298cd00:623fd4b5: first in-progress backend:
impala-ec2-centos74-m5-4xlarge-ondemand-0bff.vpc.cloudera.com:22000
I0829 23:38:31.656808 4384 coordinator.cc:498] ExecState: query
id=54b6c955298cd00:623fd4b5 finstance=54b6c955298cd00:623fd4b50001
on host=impala-ec2-centos74-m5-4xlarge-ondemand-0bff.vpc.cloudera.com:22001
(EXECUTING -> ERROR) status=File
'hdfs://localhost:20500/test-warehouse/test_resolution_by_name_63ec1576.db/switched_map_fields_resolution_test/switched_map.parq'
has an incompatible Parquet schema for column
'test_resolution_by_name_63ec1576.switched_map_fields_resolution_test.int_map.key'.
Column type: STRING, Parquet schema:
required int32 value [i:0 d:1 r:1]
I0829 23:38:31.657034 4384 coordinator-backend-state.cc:377] sending
CancelQueryFInstances rpc for query_id=54b6c955298cd00:623fd4b5
backend=impala-ec2-centos74-m5-4xlarge-ondemand-0bff.vpc.cloudera.com:22000
I0829 23:38:31.657304 5860 impala-internal-service.cc:71]
CancelQueryFInstances(): query_id=54b6c955298cd00:623fd4b5
I0829 23:38:31.657408 5860 query-exec-mgr.cc:95] QueryState:
query_id=54b6c955298cd00:623fd4b5 refcnt=4
I0829 23:38:31.657490 5860 query-state.cc:504] Cancel:
query_id=54b6c955298cd00:623fd4b5
I0829 23:38:31.657575 5860 krpc-data-stream-mgr.cc:325] cancelling all streams
for fragment_instance_id=54b6c955298cd00:623fd4b5
I0829 23:38:31.657836 4384 coordinator.cc:658] CancelBackends()
query_id=54b6c955298cd00:623fd4b5, tried to cancel 1 backends
I0829 23:38:31.657940 4384 coordinator.cc:792] Release admission control
resources for query_id=54b6c955298cd00:623fd4b5
I0829 23:38:31.662129 18843 query-state.cc:334] Cancelling fragment instances
as directed by the coordinator. Returned status: Cancelled
I0829 23:38:31.662199 18843 query-state.cc:504] Cancel:
query_id=54b6c955298cd00:623fd4b5
I0829 23:38:31.662645 18843 krpc-data-stream-mgr.cc:294] DeregisterRecvr():
fragment_instance_id=54b6c955298cd00:623fd4b5, node=1
I0829 23:38:31.663043 18843 query-state.cc:491] Instance completed.
instance_id=54b6c955298cd00:623fd4b5 #in-flight=7 status=CANCELLED:
Cancelled
I0829 23:38:31.663146 18843 query-state.cc:504]
[jira] [Created] (IMPALA-7512) test_resolution_by_name failed: did not encounter expected error
Bikramjeet Vig created IMPALA-7512:
--
Summary: test_resolution_by_name failed: did not encounter
expected error
Key: IMPALA-7512
URL: https://issues.apache.org/jira/browse/IMPALA-7512
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.1.0
Reporter: Bikramjeet Vig
Seems like the error reached the coordinator and it transitioned to the Error
state but it was not delivered to the client
{noformat}
I0829 23:38:31.626911 16215 impala-server.cc:1040] Registered query
query_id=54b6c955298cd00:623fd4b5
session_id=f14fb6bdc8ea6686:ad8d4097469f7c8f
I0829 23:38:31.627315 16215 Frontend.java:1029] Analyzing query: select key,
value from switched_map_fields_resolution_test.int_map
I0829 23:38:31.627898 16215 Frontend.java:1041] Analysis finished.
I0829 23:38:31.631665 18839 admission-controller.cc:552] Schedule for
id=54b6c955298cd00:623fd4b5 in pool_name=default-pool
cluster_mem_needed=64.00 MB PoolConfig: max_requests=-1 max_queued=200
max_mem=-1.00 B
I0829 23:38:31.631866 18839 admission-controller.cc:557] Stats:
agg_num_running=5, agg_num_queued=0, agg_mem_reserved=1.51 GB,
local_host(local_mem_admitted=2.17 GB, num_admitted_running=5, num_queued=0,
backend_mem_reserved=514.22 MB)
I0829 23:38:31.632130 18839 admission-controller.cc:589] Admitted query
id=54b6c955298cd00:623fd4b5
I0829 23:38:31.632308 18839 coordinator.cc:91] Exec()
query_id=54b6c955298cd00:623fd4b5 stmt=select key, value from
switched_map_fields_resolution_test.int_map
I0829 23:38:31.633636 18744 query-state.cc:491] Instance completed.
instance_id=5049c19ed24bed19:85ffb5e0001 #in-flight=7 status=OK
I0829 23:38:31.634953 18839 coordinator.cc:330] starting execution on 2
backends for query_id=54b6c955298cd00:623fd4b5
I0829 23:38:31.635996 29061 impala-internal-service.cc:49]
ExecQueryFInstances(): query_id=54b6c955298cd00:623fd4b5
coord=impala-ec2-centos74-m5-4xlarge-ondemand-0bff.vpc.cloudera.com:22000
#instances=1
I0829 23:38:31.638362 18843 query-state.cc:483] Executing instance.
instance_id=54b6c955298cd00:623fd4b5 fragment_idx=0
per_fragment_instance_idx=0 coord_state_idx=0 #in-flight=8
I0829 23:38:31.638617 18839 coordinator.cc:344] started execution on 2 backends
for query_id=54b6c955298cd00:623fd4b5
I0829 23:38:31.656479 4384 coordinator.cc:685] Backend completed:
host=impala-ec2-centos74-m5-4xlarge-ondemand-0bff.vpc.cloudera.com:22001
remaining=2 query_id=54b6c955298cd00:623fd4b5
I0829 23:38:31.656626 4384 coordinator-backend-state.cc:254]
query_id=54b6c955298cd00:623fd4b5: first in-progress backend:
impala-ec2-centos74-m5-4xlarge-ondemand-0bff.vpc.cloudera.com:22000
I0829 23:38:31.656808 4384 coordinator.cc:498] ExecState: query
id=54b6c955298cd00:623fd4b5 finstance=54b6c955298cd00:623fd4b50001
on host=impala-ec2-centos74-m5-4xlarge-ondemand-0bff.vpc.cloudera.com:22001
(EXECUTING -> ERROR) status=File
'hdfs://localhost:20500/test-warehouse/test_resolution_by_name_63ec1576.db/switched_map_fields_resolution_test/switched_map.parq'
has an incompatible Parquet schema for column
'test_resolution_by_name_63ec1576.switched_map_fields_resolution_test.int_map.key'.
Column type: STRING, Parquet schema:
required int32 value [i:0 d:1 r:1]
I0829 23:38:31.657034 4384 coordinator-backend-state.cc:377] sending
CancelQueryFInstances rpc for query_id=54b6c955298cd00:623fd4b5
backend=impala-ec2-centos74-m5-4xlarge-ondemand-0bff.vpc.cloudera.com:22000
I0829 23:38:31.657304 5860 impala-internal-service.cc:71]
CancelQueryFInstances(): query_id=54b6c955298cd00:623fd4b5
I0829 23:38:31.657408 5860 query-exec-mgr.cc:95] QueryState:
query_id=54b6c955298cd00:623fd4b5 refcnt=4
I0829 23:38:31.657490 5860 query-state.cc:504] Cancel:
query_id=54b6c955298cd00:623fd4b5
I0829 23:38:31.657575 5860 krpc-data-stream-mgr.cc:325] cancelling all streams
for fragment_instance_id=54b6c955298cd00:623fd4b5
I0829 23:38:31.657836 4384 coordinator.cc:658] CancelBackends()
query_id=54b6c955298cd00:623fd4b5, tried to cancel 1 backends
I0829 23:38:31.657940 4384 coordinator.cc:792] Release admission control
resources for query_id=54b6c955298cd00:623fd4b5
I0829 23:38:31.662129 18843 query-state.cc:334] Cancelling fragment instances
as directed by the coordinator. Returned status: Cancelled
I0829 23:38:31.662199 18843 query-state.cc:504] Cancel:
query_id=54b6c955298cd00:623fd4b5
I0829 23:38:31.662645 18843 krpc-data-stream-mgr.cc:294] DeregisterRecvr():
fragment_instance_id=54b6c955298cd00:623fd4b5, node=1
I0829 23:38:31.663043 18843 query-state.cc:491] Instance completed.
instance_id=54b6c955298cd00:623fd4b5 #in-flight=7 status=CANCELLED:
Cancelled
I0829 23:38:31.663146 18843 query-state.cc:504]
[jira] [Updated] (IMPALA-1286) Pull out common conjuncts from disjunctions
[ https://issues.apache.org/jira/browse/IMPALA-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-1286: Docs Text: (was: Perf improvement.) > Pull out common conjuncts from disjunctions > --- > > Key: IMPALA-1286 > URL: https://issues.apache.org/jira/browse/IMPALA-1286 > Project: IMPALA > Issue Type: New Feature > Components: Frontend >Affects Versions: Impala 2.0 >Reporter: Nong Li >Assignee: Alexander Behm >Priority: Minor > Labels: documentation, performance, planner, tpc-ds > Fix For: Impala 2.8.0 > > > I see this in tpch and tpcds where predicates look like this: > where >( col = 1 AND ... AND ...) or >( col = 1 AND ... AND ...) or >( col = 1 AND ... AND ...) > It's an OR of ANDs and there are predicates that are in each OR (col = 1). > This could be pulled out. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-6644) Add last heartbeat timestamp into Statestore metric
[ https://issues.apache.org/jira/browse/IMPALA-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pooja Nilangekar resolved IMPALA-6644. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 > Add last heartbeat timestamp into Statestore metric > --- > > Key: IMPALA-6644 > URL: https://issues.apache.org/jira/browse/IMPALA-6644 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 2.11.0 >Reporter: Mala Chikka Kempanna >Assignee: Pooja Nilangekar >Priority: Minor > Labels: ramp-up, supportability > Fix For: Impala 3.1.0 > > > In the latest and previous versions, statestore in it's default logging > reports only when it fails to send heartbeat to any host. > There is no way to confirm if Statestore is indeed continuing to heartbeat in > all passing conditions, except for turning on debug logs, which becomes too > noisy. But at the same time its important to know statestore is indeed > heartbeating. > The suggestion here is to add a metric in statestore metric page and also > print the same in log every once in a minute(or any configurable > time-frequency), reporting the last heartbeat timestamp and last heartbeat > host(optional). > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6644) Add last heartbeat timestamp into Statestore metric
[ https://issues.apache.org/jira/browse/IMPALA-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pooja Nilangekar resolved IMPALA-6644. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 > Add last heartbeat timestamp into Statestore metric > --- > > Key: IMPALA-6644 > URL: https://issues.apache.org/jira/browse/IMPALA-6644 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 2.11.0 >Reporter: Mala Chikka Kempanna >Assignee: Pooja Nilangekar >Priority: Minor > Labels: ramp-up, supportability > Fix For: Impala 3.1.0 > > > In the latest and previous versions, statestore in it's default logging > reports only when it fails to send heartbeat to any host. > There is no way to confirm if Statestore is indeed continuing to heartbeat in > all passing conditions, except for turning on debug logs, which becomes too > noisy. But at the same time its important to know statestore is indeed > heartbeating. > The suggestion here is to add a metric in statestore metric page and also > print the same in log every once in a minute(or any configurable > time-frequency), reporting the last heartbeat timestamp and last heartbeat > host(optional). > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7511) translate() string function only works with ascii characters.
Andrew Sherman created IMPALA-7511:
--
Summary: translate() string function only works with ascii
characters.
Key: IMPALA-7511
URL: https://issues.apache.org/jira/browse/IMPALA-7511
Project: IMPALA
Issue Type: Task
Reporter: Andrew Sherman
The query
SELECT translate('Gánémílózú', 'áéíóú', 'aeiou') ;
returns "Gaenaomalaza" instead of the desired "Ganemilozu".
It seems that translate treats strings as 8 bit characters. So it only works
with ascii.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (IMPALA-7511) translate() string function only works with ascii characters.
[
https://issues.apache.org/jira/browse/IMPALA-7511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman reassigned IMPALA-7511:
--
Assignee: Andrew Sherman
> translate() string function only works with ascii characters.
> -
>
> Key: IMPALA-7511
> URL: https://issues.apache.org/jira/browse/IMPALA-7511
> Project: IMPALA
> Issue Type: Task
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
>Priority: Major
>
> The query
> SELECT translate('Gánémílózú', 'áéíóú', 'aeiou') ;
> returns "Gaenaomalaza" instead of the desired "Ganemilozu".
> It seems that translate treats strings as 8 bit characters. So it only works
> with ascii.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7511) translate() string function only works with ascii characters.
Andrew Sherman created IMPALA-7511:
--
Summary: translate() string function only works with ascii
characters.
Key: IMPALA-7511
URL: https://issues.apache.org/jira/browse/IMPALA-7511
Project: IMPALA
Issue Type: Task
Reporter: Andrew Sherman
The query
SELECT translate('Gánémílózú', 'áéíóú', 'aeiou') ;
returns "Gaenaomalaza" instead of the desired "Ganemilozu".
It seems that translate treats strings as 8 bit characters. So it only works
with ascii.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-5654) Disallow managed Kudu table to explicitly set Kudu tbl name in CREATE TABLE
[
https://issues.apache.org/jira/browse/IMPALA-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597326#comment-16597326
]
Sergey Benner commented on IMPALA-5654:
---
Agreed, That's really nasty, I'm using a kundera orm to automatically create a
table which isn't allowing
to set the partitioning options etc. So I have to create a DDL manually and
import the data into this table
then the swapping of the table just doesn't work and an orm is sketchy with
schema names.
Please get that functionality back.
> Disallow managed Kudu table to explicitly set Kudu tbl name in CREATE TABLE
> ---
>
> Key: IMPALA-5654
> URL: https://issues.apache.org/jira/browse/IMPALA-5654
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
>Affects Versions: Impala 2.8.0
>Reporter: Matthew Jacobs
>Assignee: Gabor Kaszab
>Priority: Major
> Labels: kudu
> Fix For: Impala 2.12.0
>
>
> There's no reason to allow this behavior. Managed tables create Kudu tables
> with the name (in Kudu) "impala::db_name.table_name". Renaming (in Impala) a
> managed Kudu table results in renaming the underlying Kudu table, e.g. rename
> table_name to new_table name results in changing the Kudu table to
> "impala::db_name.new_table_name". But allowing a new table to specify the
> kudu table name is inconsistent with the renaming behavior and just
> introduces opportunities for confusion.
> {code}
> private void analyzeManagedKuduTableParams(Analyzer analyzer) throws
> AnalysisException {
> // If no Kudu table name is specified in tblproperties, generate one
> using the
> // current database as a prefix to avoid conflicts in Kudu.
> // TODO: Disallow setting this manually for managed tables
> if (!getTblProperties().containsKey(KuduTable.KEY_TABLE_NAME)) {
> getTblProperties().put(KuduTable.KEY_TABLE_NAME,
> KuduUtil.getDefaultCreateKuduTableName(getDb(), getTbl()));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7502) ALTER TABLE RENAME should require ALL on the source table
[ https://issues.apache.org/jira/browse/IMPALA-7502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597095#comment-16597095 ] ASF subversion and git services commented on IMPALA-7502: - Commit b97e5ba38dcb8128efc18dbc995307386b8d2210 in impala's branch refs/heads/master from [~fredyw] [ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=b97e5ba ] IMPALA-7502: ALTER TABLE RENAME should require ALL on the old table Prior to this patch, ALTER TABLE/VIEW RENAME required ALTER on the old table. This may pose a potential security risk, such as having ALTER on a table and ALL on a particular database allows a user to move the table to a database with ALL, which will automatically grant that user with ALL privilege on that table due to the privilege inherited from the database. This patch fixes the issue by requring ALL on the old table. What this means is moving a table to a database with ALL privilege will not elevate the privilege since ALL is now required for a table to be renamed. Testing: - Ran all FE tests Change-Id: I47a417a77df3f3030cf5f54fd2280b5e5e1fb77a Reviewed-on: http://gerrit.cloudera.org:8080/11344 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > ALTER TABLE RENAME should require ALL on the source table > - > > Key: IMPALA-7502 > URL: https://issues.apache.org/jira/browse/IMPALA-7502 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 3.0 >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Labels: security > Fix For: Impala 3.1.0 > > > For ALTER TABLE RENAME, currently Impala requires ALTER on the source table > and CRATE on the destination database. This may pose a security risk as > described in: https://issues.apache.org/jira/browse/SENTRY-2264. The solution > is to require ALL at the source table and CREATE at the destination database. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6644) Add last heartbeat timestamp into Statestore metric
[ https://issues.apache.org/jira/browse/IMPALA-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597094#comment-16597094 ] ASF subversion and git services commented on IMPALA-6644: - Commit 8692bfbef657fe95da68e9dcaca9b49de331ccc3 in impala's branch refs/heads/master from poojanilangekar [ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=8692bfb ] IMPALA-6644: Add last heartbeat timestamp into Statestore metric After this patch, the statestore keeps track of the time since the last heartbeat for each subscriber. It is exposed as a subscriber metric on the statestore debug page. It also adds a monitoring thread that periodically checks the last heartbeat timestamp for all subscribers and logs the IDs of those that have not been updated since the last periodic check. Testing: Added an end to end test to verify the 'sec_since_heartbeat' metric of a slow subscriber. Change-Id: I754adccc4569e8219d5d01500cccdfc8782953f7 Reviewed-on: http://gerrit.cloudera.org:8080/11052 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Add last heartbeat timestamp into Statestore metric > --- > > Key: IMPALA-6644 > URL: https://issues.apache.org/jira/browse/IMPALA-6644 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 2.11.0 >Reporter: Mala Chikka Kempanna >Assignee: Pooja Nilangekar >Priority: Minor > Labels: ramp-up, supportability > > In the latest and previous versions, statestore in it's default logging > reports only when it fails to send heartbeat to any host. > There is no way to confirm if Statestore is indeed continuing to heartbeat in > all passing conditions, except for turning on debug logs, which becomes too > noisy. But at the same time its important to know statestore is indeed > heartbeating. > The suggestion here is to add a metric in statestore metric page and also > print the same in log every once in a minute(or any configurable > time-frequency), reporting the last heartbeat timestamp and last heartbeat > host(optional). > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7421) Static methods called with wrong JNI function
[ https://issues.apache.org/jira/browse/IMPALA-7421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597092#comment-16597092 ] ASF subversion and git services commented on IMPALA-7421: - Commit 547423007b82c8181be277842f092bfd86f5b720 in impala's branch refs/heads/master from [~tlipcon] [ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=5474230 ] IMPALA-7421. Static methods use wrong JNI call function Various places throughout the code were using the JNI CallObjectMethod function when in fact they should have been using CallStaticObjectMethod. It appears that this doesn't cause crashes in release builds, but if running with -Xcheck:jni, it causes an assertion failure: guarantee(method->size_of_parameters() == size_of_parameters()) failed: wrong no. of arguments pushed This patch cleans up JniUtil a bit to add a "fluent" style utility for calling JNI functions. This calling style should make the above mistake more obvious because it forces the caller to write 'on_class(...)' for static methods and 'on_instance(...)' for instance methods. Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6 Reviewed-on: http://gerrit.cloudera.org:8080/11181 Reviewed-by: Bharath Vissapragada Tested-by: Impala Public Jenkins > Static methods called with wrong JNI function > - > > Key: IMPALA-7421 > URL: https://issues.apache.org/jira/browse/IMPALA-7421 > Project: IMPALA > Issue Type: Bug > Components: Backend >Affects Versions: Impala 3.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Major > Fix For: Impala 3.1.0 > > > There are various static Java methods which are loaded with > JniUtil::LoadStaticJniMethod. Many of them are then called using > 'CallObjectMethod' instead of the appropriate 'CallStaticObjectMethod'. This > seems to cause crashes when running with -Xcheck:jni -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
