[jira] [Commented] (IMPALA-9650) RuntimeFilterTest appears to be flaky
[
https://issues.apache.org/jira/browse/IMPALA-9650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082755#comment-17082755
]
Riza Suminto commented on IMPALA-9650:
--
CR is here: [https://gerrit.cloudera.org/c/15726/]
> RuntimeFilterTest appears to be flaky
> -
>
> Key: IMPALA-9650
> URL: https://issues.apache.org/jira/browse/IMPALA-9650
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.0
>Reporter: Thomas Tauber-Marshall
>Assignee: Riza Suminto
>Priority: Major
>
> Seen in an exhaustive jenkins build:
> {noformat}
> 19:37:30 43/119 Test #43: runtime-filter-test ..***Failed
> 1.32 sec
> 19:37:30 Turning perftools heap leak checking off
> 19:37:30 seed = 1586572649
> 19:37:30 Note: Google Test filter = RuntimeFilterTest.*
> 19:37:30 [==] Running 2 tests from 1 test case.
> 19:37:30 [--] Global test environment set-up.
> 19:37:30 [--] 2 tests from RuntimeFilterTest
> 19:37:30 [ RUN ] RuntimeFilterTest.Canceled
> 19:37:30 20/04/10 19:37:29 INFO util.JvmPauseMonitor: Starting JVM pause
> monitor
> 19:37:30
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/runtime/runtime-filter-test.cc:72:
> Failure
> 19:37:30 Expected: (tc.runtime_filter->arrival_delay_ms()) >=
> (tc.injection_delay), actual: 100 vs 500
> 19:37:30 [ FAILED ] RuntimeFilterTest.Canceled (100 ms)
> 19:37:30 [ RUN ] RuntimeFilterTest.Arrived
> 19:37:30
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/runtime/runtime-filter-test.cc:99:
> Failure
> 19:37:30 Expected: (tc.runtime_filter->arrival_delay_ms()) >=
> (tc.injection_delay), actual: 100 vs 500
> 19:37:30 [ FAILED ] RuntimeFilterTest.Arrived (100 ms)
> 19:37:30 [--] 2 tests from RuntimeFilterTest (200 ms total)
> 19:37:30
> 19:37:30 [--] Global test environment tear-down
> 19:37:30 [==] 2 tests from 1 test case ran. (200 ms total)
> 19:37:30 [ PASSED ] 0 tests.
> 19:37:30 [ FAILED ] 2 tests, listed below:
> 19:37:30 [ FAILED ] RuntimeFilterTest.Canceled
> 19:37:30 [ FAILED ] RuntimeFilterTest.Arrived
> 19:37:30
> 19:37:30 2 FAILED TESTS
> 19:37:30 YOU HAVE 1 DISABLED TEST
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-9650) RuntimeFilterTest appears to be flaky
[
https://issues.apache.org/jira/browse/IMPALA-9650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-9650 started by Riza Suminto.
> RuntimeFilterTest appears to be flaky
> -
>
> Key: IMPALA-9650
> URL: https://issues.apache.org/jira/browse/IMPALA-9650
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.0
>Reporter: Thomas Tauber-Marshall
>Assignee: Riza Suminto
>Priority: Major
>
> Seen in an exhaustive jenkins build:
> {noformat}
> 19:37:30 43/119 Test #43: runtime-filter-test ..***Failed
> 1.32 sec
> 19:37:30 Turning perftools heap leak checking off
> 19:37:30 seed = 1586572649
> 19:37:30 Note: Google Test filter = RuntimeFilterTest.*
> 19:37:30 [==] Running 2 tests from 1 test case.
> 19:37:30 [--] Global test environment set-up.
> 19:37:30 [--] 2 tests from RuntimeFilterTest
> 19:37:30 [ RUN ] RuntimeFilterTest.Canceled
> 19:37:30 20/04/10 19:37:29 INFO util.JvmPauseMonitor: Starting JVM pause
> monitor
> 19:37:30
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/runtime/runtime-filter-test.cc:72:
> Failure
> 19:37:30 Expected: (tc.runtime_filter->arrival_delay_ms()) >=
> (tc.injection_delay), actual: 100 vs 500
> 19:37:30 [ FAILED ] RuntimeFilterTest.Canceled (100 ms)
> 19:37:30 [ RUN ] RuntimeFilterTest.Arrived
> 19:37:30
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/runtime/runtime-filter-test.cc:99:
> Failure
> 19:37:30 Expected: (tc.runtime_filter->arrival_delay_ms()) >=
> (tc.injection_delay), actual: 100 vs 500
> 19:37:30 [ FAILED ] RuntimeFilterTest.Arrived (100 ms)
> 19:37:30 [--] 2 tests from RuntimeFilterTest (200 ms total)
> 19:37:30
> 19:37:30 [--] Global test environment tear-down
> 19:37:30 [==] 2 tests from 1 test case ran. (200 ms total)
> 19:37:30 [ PASSED ] 0 tests.
> 19:37:30 [ FAILED ] 2 tests, listed below:
> 19:37:30 [ FAILED ] RuntimeFilterTest.Canceled
> 19:37:30 [ FAILED ] RuntimeFilterTest.Arrived
> 19:37:30
> 19:37:30 2 FAILED TESTS
> 19:37:30 YOU HAVE 1 DISABLED TEST
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-9651) Update Ranger Impala plugin to replace use of deprecated APIs
Fang-Yu Rao created IMPALA-9651:
---
Summary: Update Ranger Impala plugin to replace use of deprecated
APIs
Key: IMPALA-9651
URL: https://issues.apache.org/jira/browse/IMPALA-9651
Project: IMPALA
Issue Type: Task
Reporter: Fang-Yu Rao
Assignee: Fang-Yu Rao
Ranger Impala plugin currently uses couple of deprecated Ranger APIs. These
need to be replaced as detailed below.
1. RangerImpaladAuthorizationManager has reference to method
{{RangerAuthContext.getResourceACLs()}}, which was deprecated with recent
changes in RANGER-2654. This reference should be replaced with a call to
{{RangerBasePlugin.getResourceACLs()}}, as shown below.
Current call:
{noformat}
RangerResourceACLs acls = authContext_.get().getResourceACLs(request);
{noformat}
Should be updated to:
{noformat}
RangerResourceACLs acls = plugin_.get().getResourceACLs(request);
{noformat}
Also, {{authContext_}} can be removed from RangerImpaladAuthorizationManager,
as this member will not be needed anymore.
2. AuthorizationTestBase has reference to method {{RangerRESTClient(String,
String)}}, which was deprecated with recent changes in RANGER-2646. This
reference should be replaced with a call to {{RangerRESTClient(String, String,
Configuration)}}, as shown below:
Current call:
{noformat}
rangerRestClient_ = new RangerRESTClient(RANGER_ADMIN_URL, null);
{noformat}
Should be updated to:
{noformat}
rangerRestClient_ = new RangerRESTClient(RANGER_ADMIN_URL, null,
rangerImpalaPlugin_.getConfig());
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-9537) Add LDAP auth to the webui
[ https://issues.apache.org/jira/browse/IMPALA-9537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Tauber-Marshall resolved IMPALA-9537. Fix Version/s: Impala 4.0 Resolution: Fixed > Add LDAP auth to the webui > -- > > Key: IMPALA-9537 > URL: https://issues.apache.org/jira/browse/IMPALA-9537 > Project: IMPALA > Issue Type: Improvement > Components: Clients >Affects Versions: Impala 4.0 >Reporter: Thomas Tauber-Marshall >Assignee: Thomas Tauber-Marshall >Priority: Major > Fix For: Impala 4.0 > > > We already support Kerberos auth to the webui over SPNEGO, and Basic HTTP > auth to LDAP over the hs2 http interface. > It would be useful to add the ability to authenticate to the webui using LDAP > over Basic. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-9650) RuntimeFilterTest appears to be flaky
[
https://issues.apache.org/jira/browse/IMPALA-9650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082693#comment-17082693
]
Riza Suminto commented on IMPALA-9650:
--
It seems like the test failed because exhaustive run pass compiler flags
-DNDEBUG and strip out the line that should do the delay injection
[https://github.com/apache/impala/blob/5e69ae1/be/src/runtime/runtime-filter.cc#L85]
I will remove the enclosing macro so that the delay injection is kept.
> RuntimeFilterTest appears to be flaky
> -
>
> Key: IMPALA-9650
> URL: https://issues.apache.org/jira/browse/IMPALA-9650
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.0
>Reporter: Thomas Tauber-Marshall
>Assignee: Riza Suminto
>Priority: Major
>
> Seen in an exhaustive jenkins build:
> {noformat}
> 19:37:30 43/119 Test #43: runtime-filter-test ..***Failed
> 1.32 sec
> 19:37:30 Turning perftools heap leak checking off
> 19:37:30 seed = 1586572649
> 19:37:30 Note: Google Test filter = RuntimeFilterTest.*
> 19:37:30 [==] Running 2 tests from 1 test case.
> 19:37:30 [--] Global test environment set-up.
> 19:37:30 [--] 2 tests from RuntimeFilterTest
> 19:37:30 [ RUN ] RuntimeFilterTest.Canceled
> 19:37:30 20/04/10 19:37:29 INFO util.JvmPauseMonitor: Starting JVM pause
> monitor
> 19:37:30
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/runtime/runtime-filter-test.cc:72:
> Failure
> 19:37:30 Expected: (tc.runtime_filter->arrival_delay_ms()) >=
> (tc.injection_delay), actual: 100 vs 500
> 19:37:30 [ FAILED ] RuntimeFilterTest.Canceled (100 ms)
> 19:37:30 [ RUN ] RuntimeFilterTest.Arrived
> 19:37:30
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/runtime/runtime-filter-test.cc:99:
> Failure
> 19:37:30 Expected: (tc.runtime_filter->arrival_delay_ms()) >=
> (tc.injection_delay), actual: 100 vs 500
> 19:37:30 [ FAILED ] RuntimeFilterTest.Arrived (100 ms)
> 19:37:30 [--] 2 tests from RuntimeFilterTest (200 ms total)
> 19:37:30
> 19:37:30 [--] Global test environment tear-down
> 19:37:30 [==] 2 tests from 1 test case ran. (200 ms total)
> 19:37:30 [ PASSED ] 0 tests.
> 19:37:30 [ FAILED ] 2 tests, listed below:
> 19:37:30 [ FAILED ] RuntimeFilterTest.Canceled
> 19:37:30 [ FAILED ] RuntimeFilterTest.Arrived
> 19:37:30
> 19:37:30 2 FAILED TESTS
> 19:37:30 YOU HAVE 1 DISABLED TEST
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-9649) Exclude shiro-crypto-core and shiro-core jars from maven download
[ https://issues.apache.org/jira/browse/IMPALA-9649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-9649 started by David Knupp. --- > Exclude shiro-crypto-core and shiro-core jars from maven download > - > > Key: IMPALA-9649 > URL: https://issues.apache.org/jira/browse/IMPALA-9649 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 4.0 >Reporter: David Knupp >Assignee: David Knupp >Priority: Major > Fix For: Impala 4.0 > > > These jars have known security vulnerabilities. They are included as part of > Sentry, and are not used by Impala directly. > There's a currently a plan to remove Sentry altogether, but since will > require non-trivial effort, until that time, let's exclude these items from > the maven download. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-9649) Exclude shiro-crypto-core and shiro-core jars from maven download
[ https://issues.apache.org/jira/browse/IMPALA-9649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082681#comment-17082681 ] David Knupp commented on IMPALA-9649: - Patch submitted: https://gerrit.cloudera.org/c/15720/ > Exclude shiro-crypto-core and shiro-core jars from maven download > - > > Key: IMPALA-9649 > URL: https://issues.apache.org/jira/browse/IMPALA-9649 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 4.0 >Reporter: David Knupp >Assignee: David Knupp >Priority: Major > Fix For: Impala 4.0 > > > These jars have known security vulnerabilities. They are included as part of > Sentry, and are not used by Impala directly. > There's a currently a plan to remove Sentry altogether, but since will > require non-trivial effort, until that time, let's exclude these items from > the maven download. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-9650) RuntimeFilterTest appears to be flaky
Thomas Tauber-Marshall created IMPALA-9650:
--
Summary: RuntimeFilterTest appears to be flaky
Key: IMPALA-9650
URL: https://issues.apache.org/jira/browse/IMPALA-9650
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 4.0
Reporter: Thomas Tauber-Marshall
Assignee: Riza Suminto
Seen in an exhaustive jenkins build:
{noformat}
19:37:30 43/119 Test #43: runtime-filter-test ..***Failed1.32
sec
19:37:30 Turning perftools heap leak checking off
19:37:30 seed = 1586572649
19:37:30 Note: Google Test filter = RuntimeFilterTest.*
19:37:30 [==] Running 2 tests from 1 test case.
19:37:30 [--] Global test environment set-up.
19:37:30 [--] 2 tests from RuntimeFilterTest
19:37:30 [ RUN ] RuntimeFilterTest.Canceled
19:37:30 20/04/10 19:37:29 INFO util.JvmPauseMonitor: Starting JVM pause monitor
19:37:30
/data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/runtime/runtime-filter-test.cc:72:
Failure
19:37:30 Expected: (tc.runtime_filter->arrival_delay_ms()) >=
(tc.injection_delay), actual: 100 vs 500
19:37:30 [ FAILED ] RuntimeFilterTest.Canceled (100 ms)
19:37:30 [ RUN ] RuntimeFilterTest.Arrived
19:37:30
/data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/runtime/runtime-filter-test.cc:99:
Failure
19:37:30 Expected: (tc.runtime_filter->arrival_delay_ms()) >=
(tc.injection_delay), actual: 100 vs 500
19:37:30 [ FAILED ] RuntimeFilterTest.Arrived (100 ms)
19:37:30 [--] 2 tests from RuntimeFilterTest (200 ms total)
19:37:30
19:37:30 [--] Global test environment tear-down
19:37:30 [==] 2 tests from 1 test case ran. (200 ms total)
19:37:30 [ PASSED ] 0 tests.
19:37:30 [ FAILED ] 2 tests, listed below:
19:37:30 [ FAILED ] RuntimeFilterTest.Canceled
19:37:30 [ FAILED ] RuntimeFilterTest.Arrived
19:37:30
19:37:30 2 FAILED TESTS
19:37:30 YOU HAVE 1 DISABLED TEST
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-9649) Exclude shiro-crypto-core and shiro-core jars from maven download
David Knupp created IMPALA-9649: --- Summary: Exclude shiro-crypto-core and shiro-core jars from maven download Key: IMPALA-9649 URL: https://issues.apache.org/jira/browse/IMPALA-9649 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 4.0 Reporter: David Knupp Fix For: Impala 4.0 These jars have known security vulnerabilities. They are included as part of Sentry, and are not used by Impala directly. There's a currently a plan to remove Sentry altogether, but since will require non-trivial effort, until that time, let's exclude these items from the maven download. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-9648) Exclude or update netty jar
Abhishek Rawat created IMPALA-9648: -- Summary: Exclude or update netty jar Key: IMPALA-9648 URL: https://issues.apache.org/jira/browse/IMPALA-9648 Project: IMPALA Issue Type: Task Reporter: Abhishek Rawat Add exclusion for netty if not being used. Or update it to version 4.1.44 or later. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-9647) Exclude or update fluent-hc jar
Abhishek Rawat created IMPALA-9647: -- Summary: Exclude or update fluent-hc jar Key: IMPALA-9647 URL: https://issues.apache.org/jira/browse/IMPALA-9647 Project: IMPALA Issue Type: Task Reporter: Abhishek Rawat Add exclusion for fluent-hc-4.3.2.jar or upgrade it to 4.3.6 or later version. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-9646) Clean up README
Tim Armstrong created IMPALA-9646: - Summary: Clean up README Key: IMPALA-9646 URL: https://issues.apache.org/jira/browse/IMPALA-9646 Project: IMPALA Issue Type: Improvement Components: Infrastructure Reporter: Tim Armstrong Assignee: Tim Armstrong A recent mailing list thread revealed that the README was a bit confusing, because it includes some fairly random details about the dev environment. We should clean this up and direct people to the right docs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-9644) Set core file size 0 in docker entrypoint script
[ https://issues.apache.org/jira/browse/IMPALA-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Rawat closed IMPALA-9644. -- Fix Version/s: Impala 3.4.0 Resolution: Fixed > Set core file size 0 in docker entrypoint script > > > Key: IMPALA-9644 > URL: https://issues.apache.org/jira/browse/IMPALA-9644 > Project: IMPALA > Issue Type: Task > Components: Infrastructure >Reporter: Abhishek Rawat >Assignee: Abhishek Rawat >Priority: Major > Labels: docker > Fix For: Impala 3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-9625) Impala's COMPUTE STATS statement generates duplicate ALTER events
[
https://issues.apache.org/jira/browse/IMPALA-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082468#comment-17082468
]
ASF subversion and git services commented on IMPALA-9625:
-
Commit e93405f7600c2cefcf522fdc64162a9ceaac3aa3 in impala's branch
refs/heads/master from Fang-Yu Rao
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=e93405f ]
IMPALA-9625: Convert the name of a TAccessEvent to lowercase
Impala's COMPUTE STATS statement results in two registrations of the
ALTER event for the corresponding table identified by its
fully-qualified table name and a Set is used to maintained the audits.
The first registration is in Analyzer#registerAuthAndAuditEvent() and
the second is in Analyzer#getTable().
In registerAuthAndAuditEvent(), the corresponding full table name
table.getFullName() is produced by a call to Analyzer#resolveTableRef().
The resulting database and table names are both in lowercase.
However, in getTable(), the fully-qualified table name is produced by a
call to Analyzer#getFqTableName(). The resulting database and table
names are in their originally unconverted form provided by the user from
the Impala shell. Hence, there is no guarantee that the database and
table names are both in lowercase.
Therefore, if a user does not provide lowercase database and table
names, the returned full table names from registerAuthAndAuditEvent()
and getTable() would differ, resulting in duplicate ALTER events for the
same table. This patch resolves the inconsistencies by converting the
name of a TAccessEvent to lowercase in addAccessEvent(), which is called
in getTable().
Testing:
- Revised a FE test to verified we do not have duplicate ALTER events for
the COMPUTE STATS statement.
- Verified that the patch passes the exhaustive tests in the DEBUG build
except for a flaky E2E test of test_column_storage_attributes.
Change-Id: If0d9ba58da891921fafbfe7c6db358b51965e178
Reviewed-on: http://gerrit.cloudera.org:8080/15689
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> Impala's COMPUTE STATS statement generates duplicate ALTER events
> -
>
> Key: IMPALA-9625
> URL: https://issues.apache.org/jira/browse/IMPALA-9625
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Reporter: Fang-Yu Rao
>Assignee: Fang-Yu Rao
>Priority: Critical
>
> Impala's COMPUTE STATS statement results in the registration of the ALTER
> event twice. One is in {{Analyzer#registerAuthAndAuditEvent()}} at
> [https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/analysis/Analyzer.java#L3131-L3133]
> and the other is in {{Analyzer#getTable()}} at
> [https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/analysis/Analyzer.java#L2862-L2863].
> In {{registerAuthAndAuditEvent()}}, the corresponding full table name
> {{table.getFullName()}} is produced by a call to
> {{Analyzer#resolveTableRef()}}
> ([https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java#L352]).
> The resulting database and table names are both in lowercase.
> However, in {{getTable()}}, the fully-qualified table name is produce by a
> call to {{Analyzer#getFqTableName()}} at
> [https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/analysis/Analyzer.java#L2836].
> The resulting database and table names are in their originally unconverted
> form provided by the user from the Impala shell. Hence, there is no guarantee
> that the database and table names are both in lowercase.
> Therefore, if a user does not provide lowercase database and table names, the
> returned full table name from {{registerAuthAndAuditEvent()}} and
> {{getTable()}} would differ, resulting in duplicate ALTER events for the same
> table.
> We should at least make the full table name consistent every time when we
> register such an audit event to avoid duplicate entries in the log.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-9428) Add arm64 atomic ops
[ https://issues.apache.org/jira/browse/IMPALA-9428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082466#comment-17082466 ] ASF subversion and git services commented on IMPALA-9428: - Commit c52b1d8a689584f5502822cecd54c363c768f71d in impala's branch refs/heads/master from zhaorenhai [ https://gitbox.apache.org/repos/asf?p=impala.git;h=c52b1d8 ] IMPALA-9428 Add arm64 atomic ops Atomic ops are implemented by asm. Different arc have diffent implementation. Here add arm64 atomic ops implementation. The file atomicops-internals-arm64.h is originally from here: https://github.com/protocolbuffers/protobuf/blob/3.5.x/src/google/protobuf/stubs/atomicops_internals_arm64_gcc.h and the commit hash is ec021f5. And I changed some instructions to newer version base the original file, such as ldxr + memory barrier was changed to ldaxr which has memory barrier natively. And also added some funtions which impala use. Change-Id: I469e0169193ad6ad8acca2a800c8b3f043083ddd > Add arm64 atomic ops > > > Key: IMPALA-9428 > URL: https://issues.apache.org/jira/browse/IMPALA-9428 > Project: IMPALA > Issue Type: Sub-task >Reporter: zhaorenhai >Assignee: zhaorenhai >Priority: Major > Fix For: Impala 4.0 > > > Atomic ops are implemented by asm. > Different arc have diffent implementation. > Here add arm64 atomic ops implementation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-9644) Set core file size 0 in docker entrypoint script
[
https://issues.apache.org/jira/browse/IMPALA-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082467#comment-17082467
]
ASF subversion and git services commented on IMPALA-9644:
-
Commit d6f57eb97d11f9c43c8fe09e4ce80f2e2f539ff1 in impala's branch
refs/heads/master from Abhishek Rawat
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d6f57eb ]
IMPALA-9644: Set core file size 0 in docker
entrypoint script
Sets the core file size 0 in the 'daemon_entrypoint.sh'.
Testing (docker container):
- cat /proc/{pid_impalad}/limits returns core file size 0.
- forced core dump using 'kill -11' and got message
'Failed to write core dump. Core dumps have been disabled.'
Change-Id: Icec7cb64bf1226c5b2ca72d048e0aeb8b7dae86d
Reviewed-on: http://gerrit.cloudera.org:8080/15717
Reviewed-by: Tim Armstrong
Tested-by: Impala Public Jenkins
> Set core file size 0 in docker entrypoint script
>
>
> Key: IMPALA-9644
> URL: https://issues.apache.org/jira/browse/IMPALA-9644
> Project: IMPALA
> Issue Type: Task
> Components: Infrastructure
>Reporter: Abhishek Rawat
>Assignee: Abhishek Rawat
>Priority: Major
> Labels: docker
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-9621) Support iceberg on hdfs
[ https://issues.apache.org/jira/browse/IMPALA-9621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082186#comment-17082186 ] WangSheng edited comment on IMPALA-9621 at 4/13/20, 11:24 AM: -- [~tarmstrong]Thanks for your suggestion, Tim. We will continue to study to see if we can find a suitable implementation solution.If there is any progress, I will update here. was (Author: skyyws): [~tarmstrong]Thanks for your suggestion, Tim. I found that iceberg is not very similar to hive table. Each format has its own input and output like MapredParquetInputFormat/MapredParquetOutputFormat, KuduInputFormat/KuduOutputFormat and so on, but iceberg does not. We cannot just add ICEBERG as a new file format in HdfsFileFormat, whether implement it is as a new table or a special hdfstable. We will continue to study to see if we can find a suitable implementation solution.If there is any progress, I will update here. > Support iceberg on hdfs > --- > > Key: IMPALA-9621 > URL: https://issues.apache.org/jira/browse/IMPALA-9621 > Project: IMPALA > Issue Type: Improvement >Reporter: WangSheng >Assignee: WangSheng >Priority: Major > > We are investigating iceberg recently, and preparing to implement select > iceberg data by impala. Our production use hdfs, so we will try to support > iceberg on hdfs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-9621) Support iceberg on hdfs
[ https://issues.apache.org/jira/browse/IMPALA-9621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082186#comment-17082186 ] WangSheng commented on IMPALA-9621: --- [~tarmstrong]Thanks for your suggestion, Tim. I found that iceberg is not very similar to hive table. Each format has its own input and output like MapredParquetInputFormat/MapredParquetOutputFormat, KuduInputFormat/KuduOutputFormat and so on, but iceberg does not. We cannot just add ICEBERG as a new file format in HdfsFileFormat, whether implement it is as a new table or a special hdfstable. We will continue to study to see if we can find a suitable implementation solution.If there is any progress, I will update here. > Support iceberg on hdfs > --- > > Key: IMPALA-9621 > URL: https://issues.apache.org/jira/browse/IMPALA-9621 > Project: IMPALA > Issue Type: Improvement >Reporter: WangSheng >Assignee: WangSheng >Priority: Major > > We are investigating iceberg recently, and preparing to implement select > iceberg data by impala. Our production use hdfs, so we will try to support > iceberg on hdfs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-8937) Fine grained table metadata loading on Catalog server
[ https://issues.apache.org/jira/browse/IMPALA-8937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quanlong Huang reassigned IMPALA-8937: -- Assignee: Quanlong Huang > Fine grained table metadata loading on Catalog server > - > > Key: IMPALA-8937 > URL: https://issues.apache.org/jira/browse/IMPALA-8937 > Project: IMPALA > Issue Type: Improvement > Components: Catalog, Frontend >Affects Versions: Impala 2.12.0, Impala 3.3.0 >Reporter: Bharath Vissapragada >Assignee: Quanlong Huang >Priority: Major > > *Background*: > Currently the table _on the Catalog server_ is either in a loaded or unloaded > state (IncompleteTable). When Catalog server starts for the first time, we > first fetch a list of table names for each databases and every table in this > list starts as an unloaded table. The table lists are propagated to the > coordinators so that they know whether a table with a given name exists or > not and they can start analyzing the queries. No metadata is loaded in the > incomplete tables (like schema/ownership, comments etc.) > The table metadata is loaded lazily (and the table moves into a loaded state) > when it is referenced in any query. When a load request comes in, all the > table metadata is loaded including file block information. > *Problem:* > Coordinators need some additional information when analyzing unloaded tables. > For example: IMPALA-8228. The ownership information is a part of the HMS > table schema which is not loaded until the table is marked fully loaded. > While this is not a problem for regular queries (like select * from ), > it is an issue with queries like "show tables" which do not trigger a table > load. In this particular case, due to the lack of ownership information, the > output of the table listing could be different depending on whether the table > is loaded. Another example is IMPALA-8606 where the GET_TABLES request does > not return the table comments because they are not available for unloaded > tables. > *Ask:* > We need to consider finer grained loading on the Catalog server in general. > Instead of having a binary state (loaded vs unloaded), the table could be in > a partially loaded state. We could also start with aggressively fetching > certain pieces of information that we think could aid with analysis and > lazily load the remaining pieces of metadata. Finer grained loading also > integrates well with the LocalCatalog implementation on the coordinators > where the the entire table need not be loaded on the Catalog server to serve > partial meta information (e.g: show partitions ). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
